Skip to content

engine: first boxes of the round-trips plan (LTM drift gate, salsa::Update derives)#973

Merged
bpowers merged 2 commits into
mainfrom
steelman-first-boxes
Jul 18, 2026
Merged

engine: first boxes of the round-trips plan (LTM drift gate, salsa::Update derives)#973
bpowers merged 2 commits into
mainfrom
steelman-first-boxes

Conversation

@bpowers

@bpowers bpowers commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Lands the first (leftmost) box of tracks A and B from the "Deleting the Round Trips" architecture note, and settles the note's Fig. 2 open question. Per the note, these are the two cheapest steps that retire the most risk, and neither commits us to the larger rewrites behind them. Related: #965, #966 (neither is resolved by this PR; these are their enabling steps).

Track A box 1: LTM classifier-family differential gate

LTM decides each reference's access shape twice -- once on Expr2 (db/ltm_ir.rs, driving edge emission and per-shape link-score selection) and once on printed-and-reparsed Expr0 (ltm_augment.rs, driving live-ref selection in the ceteris-paribus partial). Drift between the families produces a plausible silent zero, the mechanism behind #759, #913, and the pop[01] canonicalization bug.

The new gate (ltm_classifier_agreement_tests.rs) compares, per (from, to) causal edge, the shape multiset from the production Expr2 walker against classify_expr0_subscript_shape applied to every non-reducer occurrence in equation text printed via the production print path and reparsed -- so printer/parser asymmetry is inside the blast radius. Checked-in per-edge shape pins keep the gate non-vacuous under symmetric harness rot; a harness-level negative test proves the comparator reports mismatches. Reducer-argument occurrences are excluded because both families are reducer-context-free (any drift there manifests identically outside a reducer); the one deliberate asymmetry (the AC1.4 all-StarRange rule) is documented and pinned as excluded. Mutation-verified from both sides: planted one-line drifts in either family fail the gate with a message naming the edge and both classifications. Only production change: pub(crate) visibility on db::ltm_ir and its walker.

Track B box 1: salsa::Update derives on ModelStage0/ModelStage1/Error

The enabling step for #966 (check_model_units rebuilds the whole project's Stage0/Stage1 maps per per-model query -- quadratic in model count). One derive line per type; every field already satisfies salsa's Update-or-'static + PartialEq requirement. A pinning test in model.rs proves the impls exist, and its rustdoc records two non-obvious facts: salsa's fallback means a returns(ref) tracked fn would compile even without the derives (so the test is the sole guard), and the offset-keyed compiler::Expr layer must never receive the same derive because its values are only meaningful relative to one layout.

The Fig. 2 question, settled

The note asked: do edge emission and live-ref selection genuinely need the SAME decision, or would one type conflate two questions? Answer, verified adversarially against the code: the access shape is one per-occurrence decision consumed two ways, so a unified ClassifiedSite is the right home -- but only if it is enumerated and typed more broadly than today's walker output. Re-indexing the current walker would silently lose the module-qualified by-name live channel, conflate the position-mismatched-iterated verdict with a genuine dynamic index, and inherit the element-vs-variable token collision; the type must also carry occurrence identity plus reducer-enclosure, already-lagged, and index-position context (the last two transform-only). Full evidence-anchored analysis in the review record; the architecture note is updated to reflect the settled answer.

Process notes

Each change was implemented and then adversarially reviewed in a loop until no material findings remained; reviewers independently re-ran suites, planted their own mutations to confirm gate sensitivity, and read the vendored salsa 0.26 update machinery to confirm derive soundness. Side discoveries were filed separately: #971 (module-output live-dep selection is HashSet-order nondeterministic) and #972 (misleading Ident<State> salsa::Update doc comment).

bpowers added 2 commits July 17, 2026 20:46
LTM decides each reference's access shape twice: once on Expr2
(db/ltm_ir.rs, driving causal-edge emission and per-shape link-score
selection) and once on printed-and-reparsed Expr0 (ltm_augment.rs,
driving which occurrence stays live in the ceteris-paribus partial).
The two families are held in sync only by docstrings, and drift
produces a plausible silent zero rather than an error -- the mechanism
behind GH #759, GH #913, and the pop[01] canonicalization bug.

This adds a differential gate (ltm_classifier_agreement_tests.rs,
mounted under ltm_augment so it can see the private Expr0 family):
per (from, to) causal edge it compares the multiset of access shapes
the production Expr2 walker (collect_all_reference_sites) produces
against classify_expr0_subscript_shape applied to every non-reducer
occurrence in the equation text printed via the production print path
and reparsed -- so printer/parser asymmetry is inside the blast
radius. Checked-in per-edge shape pins keep the gate non-vacuous
under symmetric harness rot, and a harness-level negative test proves
the comparison reports mismatches. Reducer-argument occurrences are
excluded because both families are reducer-context-free, so any drift
there manifests identically on the same syntactic form outside a
reducer; the one deliberate asymmetry (the AC1.4 all-StarRange rule)
is documented and pinned as excluded.

Mutation-verified: single-line drifts planted in either family (skip
the integer-literal canonicalization; disable the mixed
iterated+pinned arm) each fail the gate with a message naming the
edge and both classifications.

This is the gate only (Track A box 1 of the round-trips plan); it
deliberately does not unify the families. The only production change
is pub(crate) visibility on db::ltm_ir and its walker.
The enabling step for GH #966: check_model_units rebuilds the whole
project's ModelStage0/ModelStage1 maps inside every per-model tracked
query, making whole-project unit diagnostics quadratic in model
count. Caching the two stages as per-project tracked returns(ref)
queries wants the types to participate in salsa's update machinery,
and Error rides along inside both stages' diagnostic fields.

Each derive is one line: every field is already either Update or
'static + PartialEq, so salsa 0.26's per-field UpdateDispatch resolves
them (genuine Update where impls exist, compare-and-replace fallback
elsewhere). A pinning test in model.rs proves all three impls exist;
its rustdoc records two non-obvious facts -- salsa's fallback means a
returns(ref) tracked fn would compile even without the derives, so the
test is the sole guard, and the offset-keyed compiler::Expr layer must
never receive the same derive because its values are only meaningful
relative to one model-global layout.

Track B box 1 of the round-trips plan; the cached-stage queries
themselves are the follow-on.
@codecov

codecov Bot commented Jul 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.53%. Comparing base (dc31462) to head (f9f6d88).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #973   +/-   ##
=======================================
  Coverage   91.52%   91.53%           
=======================================
  Files         234      234           
  Lines      156299   156305    +6     
=======================================
+ Hits       143053   143071   +18     
+ Misses      13246    13234   -12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@claude

claude Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review summary

Reviewed the differential gate + salsa::Update derives. No blocking issues.

salsa::Update on ModelStage0/ModelStage1/Error

Safe: every field satisfies either the Update derive or the 'static + PartialEq blanket fallback. The rustdoc on stage_types_and_error_implement_salsa_update accurately identifies that (1) the derives are not load-bearing for returns(ref) tracked fns (backed by PartialEq via UpdateFallback) and (2) compiler::Expr must NOT receive the same derive because its offsets are keyed to one layout. The compile-time assert_update::<T>() calls are the sole guard, as documented.

LTM classifier agreement gate

The two walkers match on every substantive point I checked:

Overall correctness verdict

Correct. The change is behavior-neutral for production (only test additions + derive expansion + visibility widening) and forward-looking for the #966 tracked-fn migration.

@bpowers
bpowers merged commit 43f3d51 into main Jul 18, 2026
15 checks passed
@bpowers
bpowers deleted the steelman-first-boxes branch July 18, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant