Skip to content

util: Quantity provenance is on by default — 5.5x slower, 6.9x memory, stack overflow in Debug #538

Description

@Yaraslaut

Part of the sweep tracked in #518. Finding F20.

Summary

MORPH_QUANTITY_PROVENANCE defaults to 1 (quantity.hpp:38-40). Every leaf construction and every binary operation allocates an ASTNode (sizeof == 200) holding shared_ptrs to its operands (quantity.hpp:833-839), so the ordinary running-total pattern builds a linear chain that is never collapsed:

Quantity<eur> total{Rational{0, DecimalPlaces{2}}};
for (int i = 0; i < n; ++i) total = total + one;    // n retained ASTNodes

Verification status

Reproduced. Revision: master @ 4017228d. n = 200,000:

build max RSS wall
MORPH_QUANTITY_PROVENANCE=1 (default), -O2 54,156 KB 0.033 s
MORPH_QUANTITY_PROVENANCE=0, -O2 7,884 KB 0.006 s

Destruction is recursive (~shared_ptr~ASTNode~shared_ptr → …), and at -O0 it segfaults:

$ ./prov0 200000
total = 200000EUR (chain of 200000 provenance nodes retained)
destroying...
$ echo $?
139                 # SIGSEGV, stack overflow in the destructor chain

At -O2 clang tail-call-optimises the chain away, so this crashes in Debug and survives in Release.

Not verified: I did not determine the exact depth at which the Debug crash begins, nor whether equation() (quantity_equation.hpp:91-110, :163-210) overflows at the same depth — it recurses over the same DAG, so it should, but I did not measure it.

Why this matters

5.5x slower and 6.9x the memory for a loop that adds integers, by default, in a framework whose Quantity sits in model state and on the wire path. If the loop bound comes from wire input — a ledger replay, a batch of rows — this is a remote memory-exhaustion and crash vector, and the crash signature (Debug-only) is the worst possible one for finding it.

Suggested fix

In order of value:

  1. Flip the default to 0. Provenance is a debugging/explanation feature.
  2. Make node destruction iterative — a ~ASTNode that walks the left spine into a local worklist and releases it flat — so depth cannot overflow the stack regardless of the toggle. This is the standard fix for a shared_ptr list and should land even if (1) is rejected.
  3. Make equation()'s traversal iterative for the same reason.
  4. Consider a depth cap: past N nodes, collapse to a named leaf holding the value. An explanation 200,000 steps deep is not an explanation.

What would change the verdict

  • Close (1) if provenance-by-default is a deliberate product decision — but (2) and (3) should land regardless, since a stack overflow is not an acceptable failure mode for either setting.
  • Regression test: build a chain of 100,000 nodes and destroy it, at -O0. A test at -O2 passes with or without the fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: utilSubsystem: utilbugSomething isn't workingtriage: rescopeReal problem, wrong framing; rewrite before building

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions