A claim goes in. A truth score 0–100 and a separate, honest confidence come out — with the sources and the reasoning, not just a verdict.
The agentic trust layer for your AI stack: a deterministic verification pipeline your agents call before they act on something. Not another agent — the thing that tells your agents what's true.
Ask any LLM "how sure are you?" and it says ~95–100% on almost everything, including its mistakes. The product is not the answer — anyone can produce an answer. The product is what the answer is worth. Nothing here is answered from model memory: every judgement is made against text fetched live for that specific claim, and the confidence falls when the evidence is thin, stale, or argues with itself.
Two numbers, deliberately separate:
- Truth score 0–100 — above 50 leans true. This is about the claim.
- Confidence — how much to trust our verdict. Falls when lanes disagree, when a lane comes back empty, when the adversarial pass breaks a lane, or when the claim is the kind evidence cannot settle.
No framework, no build step: one Python stdlib server (server.py), one HTML file
(index.html), SSE streaming so you watch the evidence lanes land live.
git clone https://github.com/recommend-dev/recommend-agentic-trust-layer.git
cd recommend-agentic-trust-layer
pip install -r requirements.txt # just `requests` (+ google-auth for the Vertex path)
cp .env.example .env # add GEMINI_API_KEY + EXA_API_KEY — that's the minimum
python3 server.py # → http://localhost:8899Works with two keys: a free Gemini key (aistudio.google.com/apikey) and an Exa key. Better with more — a lane whose key is missing simply sits the check out and the verdict says so (a silent lane costs confidence; it never fakes coverage).
claim
└─ 1. STRUCTURE (Gemini) → normalized falsifiable sentence, claim_type, entities,
sub-claims, 3 query angles (one hunting contradiction)
└─ 2. LANES (parallel, routed by claim_type)
always : Grounded Web · Semantic Web · Live Index
+ deep : Deep Research (checkbox)
+ predictive: Prediction Market (auto)
+ causal /
statistical: Research Literature (auto)
└─ 3. JUDGE per lane — only against what THAT lane fetched, background knowledge forbidden
└─ 3b CHALLENGE per lane — opposing counsel attacks it; if it breaks, strength × 0.55
└─ 4. SUB-CLAIMS — each atomic assertion rated against pooled evidence
└─ 5. AGGREGATE — strength-weighted; disagreement and silence both cost confidence
└─ 6. CALIBRATION — one-way ratchet: may only LOWER confidence, never raise it
└─ 7. READOUT — 2-3 plain sentences on what the evidence showed and why
Firing every source at every claim is slower and noisier. No prediction market exists for settled history; a news write-up doesn't settle cause-and-effect. Knowing which authority settles which kind of question is part of the design.
| Lane | Fires when | Weight | Provider | Key |
|---|---|---|---|---|
| Grounded Web | always | 1.15 | Exa /answer per query |
EXA_API_KEY |
| Semantic Web | always | 0.9 | Exa /search |
EXA_API_KEY |
| Live Index | always | 0.85 | SerpAPI (Google + answer box) | SERPAPI_API_KEY |
| Deep Research | deep=1 |
1.35 | Exa per sub-claim (~7s); DEEP_ENGINE=parallel → Parallel.ai (~70s, better citations) |
EXA_API_KEY / PARALLEL_API_KEY |
| Prediction Market | claim_type predictive |
1.25 | Polymarket Gamma | free, keyless |
| Research Literature | claim_type causal / statistical |
1.4 | OpenAlex + Europe PMC | free, keyless |
- Unfalsifiable claims never get a factual verdict. "X is going to attack Y" used to return REFUTED at 96%. Absence of reporting about the future is not proof. It now returns a capped-confidence lean with a note — "nobody credible is reporting this" is real information, but we don't pretend to close an open question.
- The intake step never corrects the claim. A false claim must be checked as asserted — silently flipping a myth into its debunking makes the system report the opposite of what was asked.
- A prediction market is belief, not fact. Its strength is capped so it can inform a lean but never carry a verdict on its own.
- Every prompt knows today's date. Without it, models guess — and penalise good evidence for being "in the future".
(p_true or 0.5)when every lane said 0.0.0is falsy in Python. The classics.
The whole pipeline is exposed as one MCP tool over streamable HTTP (JSON-RPC over
POST /mcp), gated on a bearer key. A demo key is auto-generated into keys.json on first
run and printed at startup (and shown in the UI on localhost).
claude mcp add --transport http recommend-trust \
http://localhost:8899/mcp \
--header "Authorization: Bearer <key from startup output>"Returns verdict, score, confidence, per-lane breakdown with challenge flags, verified sub-claims with verdicts, and sources. Point your agent at it and stop letting it answer factual questions from memory.
Any verdict can be pinned to your session and is re-checked on a schedule
(TRACK_EVERY_H, default 24h), so you can watch a claim's score move as the evidence does.
Every check costs real API credit, so the server ships with per-IP and global daily caps
(PER_IP_DAY=25, GLOBAL_DAY=400) — tune them in .env before putting an instance on a
public host.
Real-Chrome E2E tests (macOS Chrome path is hardcoded in the scripts — adjust
executablePath for your OS):
cd tests && npm install
node uitest.js # UI invariants (modal, gauge, MCP card, vendor-name leak check)
node tracktest.js # tracking flow
node cmptest.js # compare endpoint
node scaletest.js # layout at widthsThe front-end derives every path from location.pathname, so it works at / locally and
under any path prefix in production — don't reintroduce absolute /api/... paths. If your
proxy buffers responses (Caddy, nginx), disable buffering for this route
(Caddy: flush_interval -1) or SSE will never stream. Set HOST=0.0.0.0 to listen beyond
localhost.
Google's Open Knowledge Format v0.2 records who generated agent-written knowledge and who verified it — but "verified" is a name typed into a field. A signature, not a check, and any agent can type it. This repo ships the machine behind the field, in two complementary directions:
"Is this claim actually true out there?" Extracts each concept's externally checkable claims and runs them through the full pipeline — live evidence lanes, grounded judge, adversarial challenge, calibrated confidence.
python3 okf_verify.py examples/okf-demo # --dry to judge without writingThree honest outcomes, all shown in examples/okf-demo:
| Concept | What happened |
|---|---|
market/croatia-context.md |
Both claims SUPPORTED 100/100 → stamped, with ECB/EU sources |
market/kids-snacks-positioning.md |
"Sugar causes hyperactivity" REFUTED 9/100 → not stamped, failure logged with the refuting studies (DOIs) |
metrics/activation.md |
Internal definition, nothing the web can settle → not stamped |
"Does the source this document CITES actually state this?" No web search at all:
the evidence is the bundle's own declared sources — bundle-relative files read
directly, external URLs fetched full-text. This is the mode for internal knowledge
bases, where the web has no opinion: policies, metric definitions, schema docs.
python3 okf_adapter.py path/to/bundle # --dry · reports in <bundle>-reports/Per-claim verdicts: supports · contradicts · related-only · no-evidence ·
unverifiable (unreachable source). A claim can be perfectly true and still fail
here — true-but-miscited is an attribution failure, caught by an explicit
"does the cited source state this?" check. Reasons outrank labels: a supports
whose own reasoning denies support is downgraded mechanically. Output: a per-concept
report plus a diff table (claims checked · survived · caught · reason), and only
concepts where every cited claim survives get the stamp. Needs only a Gemini key —
in our scale test a 26-concept bundle cost ~$0.02 end to end.
- Stamps are spec-conformant and additive: a
verified: {by: recommend-trust-layer/0.1, …}event (the machine-confirmed trust tier per SPEC §5.3) plus anx_verificationextension block with per-claim results. Existing frontmatter is never deleted. - The gate is the product: a verifier that stamps everything is a rubber stamp. Refuted, inconclusive and unverifiable concepts stay unstamped, with the reasons written down.
- Deprecated concepts and Attested Computations are noted and skipped — deterministic SQL checking is different machinery, and we don't pretend otherwise.
python3 okf_server.py # → http://localhost:8898Drop .md concept files, watch claims extracted and evidence lanes land live,
download the stamped file.
- Pluggable lanes. Add your own evidence lane (Brave Search, internal corpus, BigQuery schema checks, …) without touching the pipeline.
- Continuous verification. Re-run on a schedule and on every bundle change;
pair with
stale_afterso verification ages honestly.