fix(ledger,retention): stop retention manufacturing tamper signals, and make the public counter durable (#9474, #9489) - #9532
Conversation
|
Warning ⏸️ LoopOver review result - manual review recommendedReview updated: 2026-07-28 07:26:51 UTC
Review summary Nits — 6 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
Visual preview
Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy. Scroll preview
A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | 1fb265c | Commit Preview URL Branch Preview URL |
Jul 28 2026, 06:24 AM |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9532 +/- ##
==========================================
+ Coverage 89.56% 95.28% +5.71%
==========================================
Files 843 741 -102
Lines 110201 60572 -49629
Branches 26230 21342 -4888
==========================================
- Hits 98705 57717 -40988
+ Misses 10233 1613 -8620
+ Partials 1263 1242 -21
Flags with carried forward coverage won't be shown. Click here to find out more.
|
loopover-orb
Bot
added
gittensor:bug
The #9474 migration creates orb_outcome_rollups, but schema-drift only accepts a migrated table that either has a Drizzle declaration or is a declared raw-SQL exception -- so both validate-code (db:schema-drift:check) and validate-tests (check-schema-drift-script's real-repo regression guard) failed on it. Raw-SQL-only is the correct classification, not an escape hatch: the table is written by pruneExpiredRecords' fold and read by getOrbGlobalStats, both through env.DB.prepare, with no Drizzle use anywhere. orb_pr_outcomes -- the table it rolls up -- is already in the same list. Refs #9474
Bundle ReportChanges will increase total bundle size by 4.39kB (0.06%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: loopover-uiAssets Changed:
|
…stead of one unbatched delete
|
Addressed the batching concern — it was right, and "safe at today's volume" is exactly the argument that ages badly (one fleet-wide backfill turns that statement into a multi-million-row delete). The prune is now bounded like every other table, without giving up the atomicity the fold requires. Each slice picks a timestamp boundary and both statements share the identical The boundary is taken inclusively ( Three tests added:
152 tests green across the retention, pg-retention, orb-outcomes, decision-record and public-stats suites; patch coverage still 0 uncovered changed lines. |
1 participant
Summary
Retention and verification were written as if the other did not exist. Both consumers that assumed permanence get fixed here, plus #9489's final defect — the
short_tailfalse positive — which belongs with them because it is the same failure shape: a legitimate operation reading as tampering on a public endpoint.Closes #9474
Closes #9489
#9474a — the 180-day record prune would have broken every full-chain verification
decision_recordsprunes at 180 days; ledger rows are kept forever;verifyDecisionLedgerdemands the record for every ledger row and walks from genesis. ~180 days after the rule first bit, every verification would reportmissing_recordat the first pruned row — a false tamper signal manufactured by the published retention policy, on the one endpoint whose entire point is that a skeptic can trust it.The verifier now tolerates a missing record only when the ledger row's
createdAtis older than the publisheddecision_recordswindow, counting it in a newprunedRecordsresult field. Two properties make this legible rather than a loophole:row_hash_mismatchfirst. A recent out-of-band deletion is stillmissing_record.retentionCutoffIsoForTable("decision_records"), readingRETENTION_POLICYitself — the verifier and the prune cannot drift apart again, because they now share one source of truth.What is genuinely given up is exactly what pruning gives up: the content re-check for that row. The chain checks still run over it, and the committed digest stays published, so a challenger holding the original preimage can still prove a historical rewrite by hand. Documented in
what-you-can-verify.mdxand the OpenAPI descriptions.#9474b — the cumulative public counter would have visibly shrunk from ~2026-10-25
getOrbGlobalStatsSUMs the entireorb_pr_outcomestable and public-stats folds it into the homepage's all-time merged/closed/handled totals — but #9415 gave the table a 90-day window, so the "all-time" numbers would have plateaued and then decreased.The prune now folds every row it deletes into a durable
orb_outcome_rollupstable (migration 0198) in the same batch transaction as the delete — a fold and delete that could commit separately would either double-count or under-count — and the stats query adds the rollup back. Three semantics preserved exactly:account_login, soexcludeAccountde-dup keeps working after the raw rows are gone.orb_pr_outcomesmoved to the top ofRETENTION_POLICY, aboveaudit_events: the fold's own-ledger exclusion needs audit rows that the audit prune — same 90-day window — would otherwise have deleted moments earlier in the same pass. An ordering-guard test pins this.Neighbour audit (deliverable 3):
computeFleetAnalyticsis clean — everyorb_signalsread is explicitly windowed (received_at >= cutoff) andorb_signalshas no retention rule at all, so there is no unwindowed aggregate over a pruned table there.#9489d —
short_tailwas transiently false-positive and permanently blind to interior orphansTwo independent defects in the completeness reconciliation:
short_tail— "tampering" — for a state every healthy write passes through. Records younger than a 5-minuteLEDGER_APPEND_GRACE_MSare now simply not yet due; a genuinely failed append still surfaces on the next verify after the grace lapses (and its own error-level alarm fired the moment it happened).created_atagainst the verified tail only, so the moment any newer record chained cleanly, an unchained record behind it became invisible forever. The reconciliation now asks the real question — does any ledger row vouch for this record? — via a NOT EXISTS anti-join (indexed by the newdecision_ledger_record_idindex). The newest orphan decides the break kind: past the tail it is the truncated-tail signatureshort_tailalways meant; behind it, a newunchained_recordbreak carrying the record id — the failed-append signature.Validation
npx tsc --noEmit,db:migrations:check(contiguous 0001..0198),selfhost:env-reference:check,ui:openapiregenerated,git diff --check— all cleanRegressions, each verified to fail against the unfixed code: a pruned record verifies clean and is counted, not reported as tampering; a verify during an in-flight append reports ok; an interior orphan is detected as
unchained_record; the cumulative public total is byte-identical before and after a prune with the aged rows gone.Invariants: a recent record deleted out of band is still
missing_record— the tolerance cannot launder fresh deletions; past the grace window an unchained record IS reported — the grace bounds the blind spot, it does not remove the check; a fully healthy chain reportsprunedRecords: 0and both new mechanisms inert; rows the live query never counted (unregistered installs, own-ledger-published PRs) are deleted without folding; a second prune run folds nothing new;excludeAccountstill de-dups against folded totals; dry-run neither folds nor deletes; and the policy-ordering guard.The pg mock pool gained
connect()(the fold's batch transaction checks out a client) and the fold statements — both still route through the same query fn, so therowid-regression guard sees batch-issued SQL too.