Repository navigation
perf(tcl): memoise the argument-list readability scan per walk - #1689
Merged
Merged
Conversation
`argument_slots_are_readable` asked, of every braced word under a `word_list` whose owner `has_error()`, whether any sibling slot is an `ERROR` or MISSING token — an O(width) scan per word, and `has_error` is transitive, so one stray `]` admitted it for every word of the line. A wide one-line `eval` was quadratic in its width on every seam that classifies a braced word: `Ast::metrics` and `Ast::ops` through `braced_word_op_type`, `Ast::dump` through `keeps_children`, and `find` / `count --type string`. 96 KB took 4.8 s under `bca metrics`; it now takes 39 ms, and the output is unchanged. The answer depends on the tree alone, so it is stored once per `word_list` in the `FoldCache` each walk lends with its ancestor chain. `NodeMemo` becomes one slot per `MemoQuestion`, since Doxygen's `conditional_declares` already held the single slot. Two walks lent no cache and now do: the dump `Export` and `Search::act_on_node`. And `AncestorIter` now carries the cache onto each yielded ancestor's chain — the metric walk reaches the question through a braced word's *parent*, so an iterator that dropped it left `bca metrics` quadratic while the predicate read as memoised. Each seam is pinned by a scan counter, `argument_slot_scans`, since the readings are the same either way: one test per walk in the crate that owns it, one over all four seams in the root crate, and a mixed-signature `after` fixture that pins the memoised answer itself (an `eval` row cannot — every `eval` argument is a script whether or not its list is readable). Three width probes price the three walks, each clean at 0.98–1.03 and 2.01–2.07 with its own cache dropped. Fixes #1686
Review of #1686 found the audit its lesson prescribes unfinished: `children_chain`'s one caller, `doc_comment::error_holds_target`, rebuilt the derived chain with `Ancestors::known`, so Doxygen's `conditional_declares` memo never reached a conditional under a parse `ERROR` and a doc comment above each level of such a nest re-read the whole nest on every walk that lent the cache. `Ancestors::below` re-roots a derived chain with the fold cache carried, and a test pins the once-per-walk count through `read_past_error`. It carries the fold cache only. The sibling cache is keyed by walk depth over the nodes the walk has pushed, and the node a derived chain descends into is one it has not; forwarding it too let Java's `is_doc_target`, which seeks a sibling after a `package`, trip `Ancestors::lookup`'s walk-must-record `debug_assert!` on a wide `ERROR` — a debug-build panic on malformed Java that review of the first draft caught. A Java fixture of that shape pins it. Also from review: a test that a well-formed wide `eval` records zero scans and stores nothing, since the `has_error` pre-filter is what keeps clean files from paying for the memo and nothing pinned it; a fourth scaling probe for the `ops` walk, which lends a cache of its own and had only a counter test; `const` assertions tying `FoldLane::COUNT` and `MemoQuestion::COUNT` to their last variant, so a variant added without the bump is a build error rather than an index panic; the `SLOT_SCAN_WIDTHS` rationale rewritten from measurement — on `LINEAR_WIDTHS` the clean cells are 0.2–1.8 ms and the regressed fits wander 1.62–2.08 run to run; and the benchmarking table requoted as three-run bands with the `ops` row added. Refs #1686
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1689 +/- ##
==========================================
+ Coverage 98.28% 98.30% +0.01%
==========================================
Files 371 371
Lines 108691 109012 +321
Branches 108256 108577 +321
==========================================
+ Hits 106831 107160 +329
+ Misses 1195 1187 -8
Partials 665 665
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1686.
What
tcl_family::argument_slots_are_readableasked, of every braced word under aword_listwhose ownerhas_error(), whether any sibling slot is anERRORor MISSING token — an O(width) scan per word, andhas_erroris transitive, so one stray]admitted it for every word of the line. A wide one-lineevalwas quadratic in its width on every seam that classifies a braced word:Ast::metricsandAst::opsthroughbraced_word_op_type,Ast::dumpthroughkeeps_children, andfind/count --type string.wbca metricsbeforeOutput is byte-identical to
mainon all four seams, both dialects.How
The answer depends on the tree alone, so it is stored once per
word_listin theFoldCacheeach walk lends with its ancestor chain.NodeMemobecomes one slot perMemoQuestion(Doxygen'sconditional_declaresheld the single slot). Three things the issue's sketch did not cover decided the shape:AncestorIterdropped the cache. Each yieldedAncestorswas rebuilt withfolds: None, so the metric seam — which asks about a brace's parent throughancestors.iter(node).next()— would have stayed quadratic while the predicate read as memoised. The iterator now forwards it;fold_down's step closure still strips it, sinceFoldCache::foldholds theRefCellborrow there.FoldCache: the dumpExportandSearch::act_on_node. Both do now.evalfixture cannot pin the memoised answer (everyevalargument is a script regardless), so the correctness test is a mixed-signatureafter {100} ] {puts hi}/after {100} {puts ]}pair under one cache.The review round (five parallel reviewers, then a review of the fixes) added the third commit:
Ancestors::belowre-roots achildren_chainwith the fold cache —children_chain's one caller had rebuilt withAncestors::known, leaving Doxygen conditionals under anERRORO(depth²) per walk; its first draft also forwarded the depth-keyedSiblingCacheand tripped the walk-must-recorddebug_assert!on wide malformed Java, caught in the follow-up review. Plus: a zero-scan test for thehas_errorfast path, anopsscaling probe,constassertions onFoldLane::COUNT/MemoQuestion::COUNT, and theSLOT_SCAN_WIDTHSrationale rewritten from measurement.Guards
argument_slot_scanscounter: one test per walk in the crate that owns it (both dump strategies,act_on_node, Tcl + iRules), one across all fourAstseams in the root crate, a zero-scan test for well-formed input, and an iterator-forwarding test. Each verified by test-via-revert to fail on exactly the line it guards.bench-scalingwidth probes,dump/halstead/ops/count/tcl-wide-erroring-eval, with a structural shape test at every rung. Clean 0.98–1.04; each walk's cache dropped takes its own probe to 2.01–2.22 and no other. Full gate 81/81.make pre-commit→BCA_GATE: pass;make chain-auditpasses;make check-test-lang-gates-compare COMPARE_REF=origin/mainpasses.Docs
benchmarking.mdloses the #1686 caveat and gains the probe rows + clean/regressed table;build_cursordoc updated; CHANGELOG[Unreleased]entry (names theAncestorIter::Chainshape change per the #1584 precedent — the ast crate is outsideSTABILITY.md's contract, nothing re-exported by the root changed); lesson 101;.bca-baseline.tomlrefreshed in the fix commit (is_braced_literal_slotgrew one parameter).