Close twelve of the thirteen recorded e-graph review findings - #1104
Merged
Conversation
…agnosed The review pass on #1101/#1102 recorded thirteen findings it did not fix. Four of them were independent of the open design question about e-node metadata, so they go first. `Extract` declines a candidate its cost model cannot rank. `here >= bestCost` is false whenever either side is NaN, so a NaN cost became the incumbent cheapest and every candidate after it then won unconditionally -- the answer stopped being the cheapest and became whichever member the enumeration reached last. A cost model that throws is already declined by the surrounding catch; one that answers NaN is saying the same thing. `Extract` settles a cost tie on a defined order. HashSet enumeration order is unspecified and string hashing is randomised per process, so one run's tie-break was not the next run's. ENode gains a total order -- ordinal on the operator, then children. `Extract` declines to build past a depth of 256. Its cycle guard bounds the chain only by the number of distinct classes, which unions grow past the input's own syntactic depth, and a StackOverflowException cannot be caught. Same shape as Gruntz.MaxDepth. `WorkBudget.Steps` now bounds something. This is the one the review got wrong: it was recorded as a timing defect -- the growth charge landing after a sweep rather than before the next one. Measuring it found the timing is not the problem. Steps charged the e-graph's node-count growth, and SafeRules is by construction the rules whose Growth does not expand, so on ordinary input the ledger was charged nothing whatever: under Steps = 0, three of five varied expressions ran the entire sweep to saturation and reported that they had completed, never having reached a ceiling. Time was the only bound really holding. A step is now one unit of work attempted, charged before the attempt, as Buchberger, FGLM and MatchPattern all charge; the growth charge stays alongside it and moves after the sweep; and Rebuild's full-graph rescan is charged for the first time. The findings document is updated to match, including the correction to its own account of the budget finding -- the wrong diagnosis was the more flattering one, since a bound that is slightly late reads as a rounding error where a bound that never fires is the feature missing. Full suite: 8885 passed, 14 skipped (pre-existing), 0 failed.
Three more of the recorded findings, all with one cause: the fourteen node types `MatchPattern.Construct` built were what it had accumulated, not a boundary anyone chose. `EGraph` kept a second copy of that list to resolve an e-node's operator back to a type, next to a doc comment on `Construct` warning that a list written twice is a list that drifts. Now one table holds the type, its arity and the constructor call together, and `BuildableNodeTypes`, the name lookup and `CanConstruct` are all derived from its keys -- there is no second list left to drift from. A table rather than a chain of `nodeType == typeof(T)` tests also makes it O(1) rather than linear in the list, which matters more now. `Extract` silently no-opped on any root outside the list, and a rule wrapping a conditional result in `Providedf` -- the registry's own convention -- had that result unioned onto a class nothing could build. Both were the same gap. The table now holds 44 types: every node type whose constructor takes one or two Entity children, less two binders. Comparisons, connectives, the inverse trigonometric functions, floor/ceil/round, mod, gcd, min/max, the set operations and Providedf all rebuild now, where before an expression rooted at any of them came back unchanged reporting `Changed = false` -- indistinguishable from being already at its cheapest. What stays out is now a statement rather than an accident. Binders, because the e-graph has no notion of a bound variable's scope and DirectChildren hands out a capture-avoidingly renamed body, so rebuilding one would produce a term meaning something else. Variable-arity nodes, because the table keys on an arity of one or two. Six rules gain a reverse direction as a consequence -- the four inverse-trigonometric round trips and the two set idempotences, whose left-hand pattern named a type that could not be built. Each reversed rule is Expands, so none joins SafeRules and saturation is unchanged. Two tests that used Modf as their example of an unbuildable node now use Integralf, which is unbuildable for a reason that will not change: it carries an optional range beside its two children. Full suite: 8906 passed, 14 skipped (pre-existing), 0 failed.
…ign question `RewriteRecording` could not see `EqualitySaturation` at all: it is populated inside `RewriteRuleSet.ApplyOnce`, and saturation asks rules directly, so a caller who opened a recording got a real rewrite with an empty derivation and nothing to distinguish "introspection cannot see this" from "there was nothing to see". Now the pass is noted -- one edge, input to output, under the transformation's own name. Deliberately not finer than the pass. A rule set records each firing because a firing there is the rewrite: the node it matched leaves and the replacement takes its place. A firing in saturation adds another member to an e-class whose members are all already believed equal, and the answer is then chosen by Extract from all of them at once. Most firings contribute nothing to what extraction picked, and none is a step on a route from input to output, because there is no route. Reporting them as RewriteSteps would name rewrites that are not in the answer. `MatchPattern.RequiredRootType` was already there and never consulted, so every rule ran a full pattern match against every class of every pass. A pattern requiring a root type cannot match a class holding no node of it, and being a necessary condition is what makes it a filter: it licenses skipping a rule, never firing one. Gathering each class's types once per sweep and consulting them cuts match attempts about thirteenfold -- 1076 steps to 78 on the largest of four expressions, 87 to 3 on `x + 0` -- with the same answer in every case. `NeutralClass` hand-rolled an identity table that InnerSimplify already implements and tests, with nothing keeping the two in step; the divergence would have been silent in the worst direction, the e-graph merging two classes the rest of the library no longer believes equal. It is derived now, by asking InnerSimplified whether `op(x, leaf)` really is `x`. That settles the asymmetries without anyone having to remember them -- `0 - x` is a negation, `1 / x` a reciprocal, `1 ^ x` the constant 1 -- and handles what a written table cannot: an arm answering with a condition attached does not answer with the bare operand, so no fold is claimed. `Entity.SimplifiedRate` answered one cost model's question with another's cached number: the cache is one slot per instance and the criteria is ambient, so the two do not agree about what the cached number is a rate of. Not merely stale -- `Simplificator.PickSimplest` compares candidates by this property, so it weighed one model's cached rate against another's fresh one and chose on the strength of it. Cached now only while nobody has scoped the setting, which is the `IsOverriden` test `BudgetLedger.For` already applies to the budget, and one ambient read rather than a read plus a delegate comparison on a hot path. Fourteen of the fifteen findings are now closed. What remains is a design question rather than a defect -- how much should ride along on an e-node beyond its bare shape -- and the notable thing is how little of the cluster actually depended on it: one finding, Codomain. The Providedf case this document had offered as evidence for it was the buildable-type table and nothing deeper. Full suite: 8919 passed, 14 skipped (pre-existing), 0 failed.
…s fixed MathS.Settings.ComplexityCriteria's own example read its rate through FromString(expr, useCache: false), to keep the parser from handing back an instance whose SimplifiedRate had already been computed under the default criteria. That was a workaround for the bug the previous commit fixed, and a doc teaching a workaround for a bug that is gone is worse than no doc. Measured both ways before changing it: with the cache left on, the example prints the same 24 / 24 / 2 / 1 it prints with it defeated.
Two corrections to the root-type filter added a commit ago. `held.Any(type => required.IsAssignableFrom(type))` captures `required`, so it allocated a closure every time the exact-type test missed -- which is most rules of most classes, since the filter's whole job is to miss. Paying an allocation to avoid a pattern match is not a pre-filter. Written as a loop instead. And the set is gathered before the class's sweep, so a union later in that same sweep can add a type it does not have, and a rule can be skipped in the pass where it had just become applicable. That costs nothing and is worth saying rather than leaving a reader to work out: a union is exactly what sets `merged`, so there is another pass, and the set is gathered again there. Full suite: 8919 passed, 14 skipped (pre-existing), 0 failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The code review on #1101/#1102 found fifteen issues and fixed two, recording the other thirteen in
EqualitySaturationReviewFindings.mdrather than fixing them. This closes twelve of the thirteen. What is left is a design question rather than a defect, and the interesting result is how little of the cluster depended on it.Three commits, each independently reviewable.
Correctness in
ExtractCostModelansweringNaNcorrupted every later comparison.here >= bestCostis false whenever either side isNaN, so aNaNbecame the incumbent cheapest and every candidate after it then won unconditionally — the answer stopped being the cheapest and became whichever member the enumeration reached last. A cost model that throws was already declined by the surroundingcatch; one answeringNaNis saying the same thing.HashSet<ENode>enumeration order — unspecified, and string hashing is randomised per process, so one run's tie-break was not the next run's.ENodegains a total order.StackOverflowExceptioncannot be caught. Capped at 256, declining to build — the answer the cycle case already gives, and the shapeGruntz.MaxDepthalready uses.And one the review diagnosed wrongly.
WorkBudget.Stepswas recorded as a timing defect: the growth charge landing after a sweep rather than before the next one. Measuring it found otherwise —Stepscharged the e-graph's node-count growth, andSafeRulesis by construction the rules whoseGrowthdoes not expand, so on ordinary input the ledger was charged nothing whatever. UnderSteps = 0, three of five varied expressions ran the entire sweep to saturation and reported that they had completed, never having reached a ceiling.Timewas the only bound really holding.A step is now one unit of work attempted, charged before the attempt, as Buchberger, FGLM and
MatchPatternall charge; the growth charge stays alongside it and moves after the sweep; andRebuild's full-graph rescan is charged for the first time.One table of buildable node types
EGraphkept a second copy of the fourteen-type listMatchPattern.Constructhardcodes, next to a doc comment onConstructwarning that a list written twice is a list that drifts. One table now holds the type, its arity and the constructor call together, with the list, the name lookup andCanConstructall derived from its keys. Being a table rather than a chain ofnodeType == typeof(T)tests also makes it O(1).Fourteen types was what
Constructhad accumulated, not a boundary anyone chose — soExtractsilently no-opped on any root outside it, and a rule wrapping a conditional result inProvidedf(the registry's own convention) had that result unioned onto a class nothing could build. The table now holds 44: every node type whose constructor takes one or twoEntitychildren, less two binders.What stays out is now a statement rather than an accident. Binders, because the e-graph has no notion of a bound variable's scope and
DirectChildrenhands out a capture-avoidingly renamed body. Variable-arity nodes, because the table keys on an arity of one or two.Six rules gain a reverse direction as a consequence — the four inverse-trigonometric round trips and the two set idempotences, whose left-hand pattern named a type that could not be built. Each reversed rule is
Expands, so none joinsSafeRules.Introspection, a filter, a derived table, a cache
RewriteRecordingcould not seeEqualitySaturationat all, so a caller who opened a recording got a real rewrite with an empty derivation and no way to tell that introspection had missed it. The pass is noted now — and deliberately not finer, because a firing in saturation is not a rewrite: it adds a member to an e-class whose members are already believed equal, and most firings contribute nothing to what extraction chose. Reporting them asRewriteSteps would name rewrites that are not in the answer.MatchPattern.RequiredRootTypewas never consulted, so every rule ran a full pattern match against every class of every pass. Consulting it cuts match attempts about thirteenfold — 1076 steps to 78 on the largest of four expressions — with the same answer in every case.NeutralClasshand-rolled an identity tableInnerSimplifyalready implements, with nothing keeping them in step; the divergence would have been silent in the worst direction. It is derived now, by askingInnerSimplifiedwhetherop(x, leaf)really isx— which settles0 - x,1 / xand1 ^ xwithout anyone having to remember them, and declines a fold whose answer carries a condition.Entity.SimplifiedRateanswered one cost model's question with another's cached number. Not merely stale:Simplificator.PickSimplestcompares candidates by this property, so it weighed a cached rate against a fresh one and chose on the strength of it.What is left
One entry, and it is a decision rather than a bug: how much should ride along on an e-node beyond its bare shape. Only
Codomainturned out to depend on it. TheProvidedfcase the findings document had offered as evidence that the e-graph could not represent a conditional equivalence was the buildable-type table and nothing deeper — a cluster of findings around one subsystem invites a single grand explanation, and thirteen of these fourteen did not have one.Verification
Full suite: 8919 passed, 14 skipped (pre-existing), 0 failed.
Every fix has a regression test, named beside its entry in the findings document. Two tests that used
Modfas their example of an unbuildable node now useIntegralf, which is unbuildable for a reason that will not change: it carries an optional range beside its two children.Part of #746 tier 2.