Skip to content

perf(tcl): memoise the argument-list readability scan per walk - #1689

Merged
dekobon merged 3 commits into
mainfrom
fix-1686-tcl-slots-quadratic
Oct 23, 2026
Merged

dekobon merged 3 commits into
mainfrom
fix-1686-tcl-slots-quadratic

Conversation

@dekobon

@dekobon dekobon commented Oct 23, 2026

Copy link
Copy Markdown
Owner

Fixes #1686.

What

tcl_family::argument_slots_are_readable asked, of every braced word under a word_list whose owner has_error(), whether any sibling slot is an ERROR or MISSING token — an O(width) scan per word, and has_error is transitive, so one stray ] admitted it for every word of the line. A wide one-line eval was quadratic in its width on every seam that classifies a braced word: Ast::metrics and Ast::ops through braced_word_op_type, Ast::dump through keeps_children, and find / count --type string.

w bytes bca metrics before after
4 000 24 KB 541 ms 12 ms
8 000 48 KB 1 752 ms 19 ms
16 000 96 KB 4 838 ms 39 ms

Output is byte-identical to main on all four seams, both dialects.

How

The answer depends on the tree alone, so it is stored once per word_list in the FoldCache each walk lends with its ancestor chain. NodeMemo becomes one slot per MemoQuestion (Doxygen's conditional_declares held the single slot). Three things the issue's sketch did not cover decided the shape:

  1. AncestorIter dropped the cache. Each yielded Ancestors was rebuilt with folds: None, so the metric seam — which asks about a brace's parent through ancestors.iter(node).next() — would have stayed quadratic while the predicate read as memoised. The iterator now forwards it; fold_down's step closure still strips it, since FoldCache::fold holds the RefCell borrow there.
  2. Two walks lent no FoldCache: the dump Export and Search::act_on_node. Both do now.
  3. An eval fixture cannot pin the memoised answer (every eval argument is a script regardless), so the correctness test is a mixed-signature after {100} ] {puts hi} / after {100} {puts ]} pair under one cache.

The review round (five parallel reviewers, then a review of the fixes) added the third commit: Ancestors::below re-roots a children_chain with the fold cache — children_chain's one caller had rebuilt with Ancestors::known, leaving Doxygen conditionals under an ERROR O(depth²) per walk; its first draft also forwarded the depth-keyed SiblingCache and tripped the walk-must-record debug_assert! on wide malformed Java, caught in the follow-up review. Plus: a zero-scan test for the has_error fast path, an ops scaling probe, const assertions on FoldLane::COUNT / MemoQuestion::COUNT, and the SLOT_SCAN_WIDTHS rationale rewritten from measurement.

Guards

  • argument_slot_scans counter: one test per walk in the crate that owns it (both dump strategies, act_on_node, Tcl + iRules), one across all four Ast seams in the root crate, a zero-scan test for well-formed input, and an iterator-forwarding test. Each verified by test-via-revert to fail on exactly the line it guards.
  • Four bench-scaling width probes, dump/ halstead/ ops/ count/tcl-wide-erroring-eval, with a structural shape test at every rung. Clean 0.98–1.04; each walk's cache dropped takes its own probe to 2.01–2.22 and no other. Full gate 81/81.
  • make pre-commit → BCA_GATE: pass; make chain-audit passes; make check-test-lang-gates-compare COMPARE_REF=origin/main passes.

Docs

benchmarking.md loses the #1686 caveat and gains the probe rows + clean/regressed table; build_cursor doc updated; CHANGELOG [Unreleased] entry (names the AncestorIter::Chain shape change per the #1584 precedent — the ast crate is outside STABILITY.md's contract, nothing re-exported by the root changed); lesson 101; .bca-baseline.toml refreshed in the fix commit (is_braced_literal_slot grew one parameter).

`argument_slots_are_readable` asked, of every braced word under a
`word_list` whose owner `has_error()`, whether any sibling slot is an
`ERROR` or MISSING token — an O(width) scan per word, and `has_error`
is transitive, so one stray `]` admitted it for every word of the
line. A wide one-line `eval` was quadratic in its width on every seam
that classifies a braced word: `Ast::metrics` and `Ast::ops` through
`braced_word_op_type`, `Ast::dump` through `keeps_children`, and
`find` / `count --type string`. 96 KB took 4.8 s under `bca metrics`;
it now takes 39 ms, and the output is unchanged.

The answer depends on the tree alone, so it is stored once per
`word_list` in the `FoldCache` each walk lends with its ancestor
chain. `NodeMemo` becomes one slot per `MemoQuestion`, since Doxygen's
`conditional_declares` already held the single slot. Two walks lent no
cache and now do: the dump `Export` and `Search::act_on_node`. And
`AncestorIter` now carries the cache onto each yielded ancestor's
chain — the metric walk reaches the question through a braced word's
*parent*, so an iterator that dropped it left `bca metrics` quadratic
while the predicate read as memoised.

Each seam is pinned by a scan counter, `argument_slot_scans`, since
the readings are the same either way: one test per walk in the crate
that owns it, one over all four seams in the root crate, and a
mixed-signature `after` fixture that pins the memoised answer itself
(an `eval` row cannot — every `eval` argument is a script whether or
not its list is readable). Three width probes price the three walks,
each clean at 0.98–1.03 and 2.01–2.07 with its own cache dropped.

Fixes #1686
Review of #1686 found the audit its lesson prescribes unfinished:
`children_chain`'s one caller, `doc_comment::error_holds_target`,
rebuilt the derived chain with `Ancestors::known`, so Doxygen's
`conditional_declares` memo never reached a conditional under a parse
`ERROR` and a doc comment above each level of such a nest re-read the
whole nest on every walk that lent the cache. `Ancestors::below`
re-roots a derived chain with the fold cache carried, and a test pins
the once-per-walk count through `read_past_error`.

It carries the fold cache only. The sibling cache is keyed by walk
depth over the nodes the walk has pushed, and the node a derived chain
descends into is one it has not; forwarding it too let Java's
`is_doc_target`, which seeks a sibling after a `package`, trip
`Ancestors::lookup`'s walk-must-record `debug_assert!` on a wide
`ERROR` — a debug-build panic on malformed Java that review of the
first draft caught. A Java fixture of that shape pins it.

Also from review: a test that a well-formed wide `eval` records zero
scans and stores nothing, since the `has_error` pre-filter is what
keeps clean files from paying for the memo and nothing pinned it; a
fourth scaling probe for the `ops` walk, which lends a cache of its own
and had only a counter test; `const` assertions tying `FoldLane::COUNT`
and `MemoQuestion::COUNT` to their last variant, so a variant added
without the bump is a build error rather than an index panic; the
`SLOT_SCAN_WIDTHS` rationale rewritten from measurement — on
`LINEAR_WIDTHS` the clean cells are 0.2–1.8 ms and the regressed fits
wander 1.62–2.08 run to run; and the benchmarking table requoted as
three-run bands with the `ops` row added.

Refs #1686
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.70238% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.30%. Comparing base (f3527df) to head (02666d8).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
big-code-analysis-ast/src/node/ancestors.rs 98.41% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1689      +/-   ##
==========================================
+ Coverage   98.28%   98.30%   +0.01%     
==========================================
  Files         371      371              
  Lines      108691   109012     +321     
  Branches   108256   108577     +321     
==========================================
+ Hits       106831   107160     +329     
+ Misses       1195     1187       -8     
  Partials      665      665              
Flag Coverage Δ
rust 98.29% <99.70%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
big-code-analysis-ast/src/ast.rs 100.00% <100.00%> (ø)
big-code-analysis-ast/src/getter.rs 96.06% <100.00%> (ø)
...-code-analysis-ast/src/lang_helpers/doc_comment.rs 98.32% <100.00%> (+0.13%) ⬆️
big-code-analysis-ast/src/lang_helpers/doxygen.rs 99.64% <100.00%> (+<0.01%) ⬆️
...g-code-analysis-ast/src/lang_helpers/tcl_family.rs 99.38% <100.00%> (+0.62%) ⬆️
big-code-analysis-ast/src/node.rs 98.39% <100.00%> (+0.04%) ⬆️
big-code-analysis-ast/src/test_support.rs 94.11% <100.00%> (+0.78%) ⬆️
big-code-analysis-ast/src/node/ancestors.rs 98.68% <98.41%> (-0.06%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dekobon
dekobon merged commit 02666d8 into main Oct 23, 2026
49 checks passed
@dekobon
dekobon deleted the fix-1686-tcl-slots-quadratic branch October 23, 2026 04:57

This branch was successfully deployed

1 active deployment
github-pages — 02666d8e Deployed Oct 23, 2026 by dekobon via deploy to GitHub Pages #845
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(tcl): argument_slots_are_readable is O(width) per braced word, making erroring wide script commands quadratic

1 participant