core/forms: the compile-time budget — core drags the whole JSON/schema stack, and schema generation is superlinear in DAG paths

[Yaraslaut]

Consolidates #521 (F3) and #545 (F27) from the #518 sweep. Both are closed in favour of this one; both were measured, and both measurements are reproduced here in full.

They are merged because they are two halves of one number — seconds of compile time per translation unit — and because #521's own suggested fix already points at #545: "Split forms/forms.hpp — see the split proposed in F27's companion measurements." A fix for either that ignores the other cannot report an honest before/after.

Part A — the async core cannot be used without the JSON/schema stack (was #521)

core/backend.hpp -> registry.hpp -> forms/forms.hpp + glaze + util/rational.hpp, and core/model.hpp -> journal/action_log.hpp -> <glaze/glaze.hpp>. So LocalBackend, which never serialises anything, transitively instantiates the entire JSON/schema stack, and every GUI translation unit that includes bridge.hpp pays for it.

Measured. Revision: master @ 4017228d. One TU per header, g++ 15 -std=c++23 -O2 -c, plus -E | wc -l for preprocessed size:

header                        compile    preprocessed lines
core/executor.hpp               0.72s       125,892
core/completion.hpp             0.73s       128,448
core/strand.hpp                 0.74s       128,167
---- the cliff ----
core/model.hpp                  2.20s       278,483
core/registry.hpp               2.69s       284,395
core/backend.hpp                2.96s       285,562
core/bridge.hpp                 3.16s       287,858
core/remote.hpp                 3.51s       296,980
net/socket_backend.hpp          3.30s       298,998
journal/journal.hpp             2.69s       284,647
offline/offline_queue.hpp       0.36s        93,591

The async primitives are cheap. The cliff is model.hpp, which more than doubles preprocessed size via journal/action_log.hpp:5 (#include <glaze/glaze.hpp>).

Not verified: only per-header TU cost was measured, not a real example app's full build. The "160 CPU-seconds of framework headers per clean build for a 50-TU GUI" figure in the original issue is arithmetic from the per-TU number, not a measured build. Measure a real ladder-rung build as step 1 of this ticket — if the framework headers do not dominate it, Part A shrinks or closes.

Part B — nested-aggregate schema generation is exponential in DAG paths (was #545)

The nested-aggregate cycle guard carries the ancestor chain as template arguments (forms.hpp:2195-2222, :2252-2270):

recurseIntoNestedAggregateIfAny<Member, Ancestors..., Sub>(dom, property);

so annotateNestedAggregate<Leaf, Ancestors...> is a distinct instantiation per distinct root-to-node path. Any realistic domain model is a DAG — an Address under both Customer and Supplier, a Money everywhere — so the instantiation count is the path count, not the type count.

Measured. Revision: master @ 4017228d. Ln { L(n-1) a; L(n-2) b; }, so path count = Fibonacci(n):

depth	paths	compile
6	8	4.1 s
10	55	5.2 s
14	377	13.7 s
18	2584	86.6 s
22	10946	did not finish in 120 s

Growth tracks path count (86.6/13.7 = 6.3 vs 2584/377 = 6.9), confirming the mechanism. A plain chain of the same depth (Ln { L(n-1) a; L(n-1) b; }, one chain shape) stays flat at ~3.9 s — so it is specifically ancestor-set diversity that blows up, not depth.

Not verified: no in-tree action was checked for a problematic path count; the ladder examples are shallow. This is a scaling property reachable by a real domain model with shared sub-aggregates, not a defect biting today.

Part C — the DOM round-trip, the five passes, and the copy (was #545's companion measurements)

Single TU, gcc, -std=c++23, no optimisation:

TU	wall	marginal
int main(){}	10 ms	—
+ <glaze/glaze.hpp>	1834 ms	the dependency
+ <morph/forms/forms.hpp>	2074 ms	240 ms forms.hpp parse
+ glz::read_json/write_json on glz::generic_u64	2640 ms	566 ms DOM machinery
+ glz::write_json_schema<A1>()	2776 ms	702 ms glaze schema writer
+ morph::forms::schemaJson<A1>()	4008 ms	~660 ms morph's walkers
... over 10 distinct action types	4376 ms	41 ms per extra type

The generic DOM is instantiated solely to add ~8 extension keys to a document glaze already built structurally. And the pipeline is longer than it looks: registry.hpp:368-412 parses the result again to add x-payloadFingerprint/x-payloadShape, so a served descriptor is text->DOM->text->DOM->text — five full passes.

Also: schemaJson<A>() caches correctly in a function-local static and then returns it by value (forms.hpp:3020-3025) — measured at 200k calls on a 1153-byte schema = 5 ms, i.e. one allocation + memcpy per call. payload_schema.hpp:249,266 returns const std::string& for the identical pattern. Same at views.hpp:393, compounded at :419,430. This one is a two-line fix and is worth taking first.

Suggested direction

Ordered by value per unit of risk:

1. schemaJson<A>() returns const std::string& (and the views.hpp sites). Trivial, measurable, no design decision.
2. Contribute the extension keys through glz::json_schema<T> (schema::extra / merge_schema_attrs, schema.hpp:1015-1021) instead of the generic DOM. Removes the generic_u64 instantiation entirely (~570 ms/TU) and two of the five passes, with the same escaping guarantee. Failing that, registry.hpp's fingerprint keys can be contributed by mergeSchemaExtras itself — it already has the DOM open — collapsing five passes to three for free.
3. Replace the ancestor type-list with a depth NTTP plus a runtime visited-set:

   template <typename Sub, std::size_t Depth = 0>
   void annotateNestedAggregate(glz::generic_u64& dom, glz::generic_u64& node,
                                std::unordered_set<std::string_view>& seen);

   Bound Depth with static_assert(Depth < kMaxNestDepth, "...cyclic or excessively nested..."), preserving the existing diagnostic — it is good (forms.hpp:2199-2204, :2211-2217 names the problem, why it cannot be supported, and two concrete remedies). Caps instantiations at kMaxNestDepth per type instead of one per path, and the runtime seen set also removes the redundant re-annotation of the same $defs node once per path.
4. Break the core/glaze coupling: put the journal's JSON codec behind a non-template interface so model.hpp stops including glaze; split forms/forms.hpp (registry.hpp needs 5 symbols from it, bridge.hpp needs 3); ship a morph/core/async.hpp facade so a consumer can adopt the concurrency model at 0.7 s/TU without the serialisation model.

Steps 1-3 are forms; step 4 is core and is the larger design change. They can be separate PRs on one branch — but report one before/after table covering all of them, because that is the number this issue exists to move.

Coordination

Step 4 touches core/model.hpp, registry.hpp, backend.hpp and bridge.hpp. #567-#571 is reshaping IBackend and #572 is reworking executeVia. Sequence behind them or agree the split — do not land a competing change to those headers.

Architecture note

Invariant 4 of the triage skill applies and should be checked rather than assumed: morph is header-only and its cost centre is instantiation of Completion<T>/BridgeHandler<Model>/ActionTraits<A> against the application's types, which a library cannot pre-instantiate. Steps 1-3 move framework-side cost, which is real; step 4 moves what a consumer is forced to include, which is also real. Neither claims to move the application's own instantiations. Say which you moved.

What would change the verdict

• Close Part A if the coupling is deliberate and the cost is accepted — but then say so in docs/spec/core/locality.md or the README, because "header-only C++23" currently reads as cheap.
• Close Part B if a depth limit is introduced and judged low enough that path count never grows. Nothing enforces a limit today, and the failure mode is a 90-second compile rather than an error.
• Acceptance: a recorded before/after compile-time table on a real ladder-rung build, plus a regression guard for Part B (a DAG fixture whose compile time is asserted bounded). Per AGENTS.md: a guard that would pass with the fix reverted is not evidence — revert it and watch it fail.

[Yaraslaut]

Triage verdict: valid

Re-measured against 8858450d (current master), same method as the issue — one TU per header, g++ 16.2.1 -std=c++23 -O2 -c, plus -E | wc -l:

header                               compile  preproc lines     (filed @4017228d)
core/executor.hpp                      0.81s         125891      0.72s / 125,892
core/completion.hpp                    0.73s         128451      0.73s / 128,448
core/strand.hpp                        0.73s         128166      0.74s / 128,167
---- the cliff ----
core/model.hpp                         2.22s         278482      2.20s / 278,483
core/registry.hpp                      2.68s         284694      2.69s / 284,395
core/backend.hpp                       3.12s         286272      2.96s / 285,562
core/bridge.hpp                        3.41s         288581      3.16s / 287,858
core/remote.hpp                        3.56s         297793      3.51s / 296,980
journal/journal.hpp                    2.77s         284946      2.69s / 284,647
offline/offline_queue.hpp              0.38s          93591      0.36s /  93,591

The premise holds, and Part A has got slightly worse since it was filed. The cliff is still model.hpp — 0.73s to 2.22s, preprocessed size more than doubling via journal/action_log.hpp's #include <glaze/glaze.hpp>. backend.hpp moved 2.96s → 3.12s and bridge.hpp 3.16s → 3.41s, consistent with #577 adding 489 lines to backend.hpp last night. The async primitives are still cheap: executor/completion/strand are all under 0.82s.

I did not re-verify Part B's Fibonacci path-count table or Part C's DOM round-trip table; those were measured at filing and nothing in the intervening merges touches forms.hpp's nested-aggregate walker. Treat them as carried forward, not re-confirmed.

Invariant 4 applies and the issue already says so

docs/spec/ and the triage skill both note that morph is header-only and its cost centre is instantiation of Completion<T>/BridgeHandler<Model>/ActionTraits<A> against the application's types, which a library cannot pre-instantiate. The issue names this itself and asks the implementer to say which cost they moved. That framing is correct and should survive into the PR: steps 1-3 move framework-side cost, step 4 moves what a consumer is forced to include. Neither moves the application's own instantiations, and the PR should not imply it does.

Sequencing

Step 4 touches core/model.hpp, registry.hpp, backend.hpp and bridge.hpp. #568-#571 are actively migrating IBackend and #572 is queued against executeVia. Steps 1-3 are forms only and can proceed independently — prefer starting there, since step 1 (schemaJson<A>() returning const std::string&) is a two-line change with a measurable result and no design decision.

What would change the verdict

Unchanged: close Part A if the coupling is deliberate and the cost accepted — but then say so in docs/spec/core/locality.md or the README, because "header-only C++23" currently reads as cheap. The acceptance criterion stands: a before/after table on a real ladder-rung build, not arithmetic from per-TU numbers. The issue is explicit that the 160 CPU-second figure is extrapolated, and that measuring it is step 1.

[Yaraslaut]

Not dispatched this sweep — file-tree contention, not a verdict on the ticket.

A concurrent lane holds include/morph/core/backend.hpp, include/morph/core/bridge.hpp and include/morph/core/executor.hpp while it resolves #593 (the registration-surface discriminator that is currently holding PR #585 red). Part A's whole subject is the core/backend.hpp -> registry.hpp -> forms/forms.hpp include chain, so the two would collide on the file that lane is actively rewriting.

I considered dispatching Part B and Part C only (the forms.hpp nested-aggregate blow-up), which is disjoint from that lane. I did not, because this issue explicitly forbids it:

A fix for either that ignores the other cannot report an honest before/after.

That is the stated reason #521 and #545 were merged here in the first place, and splitting them back apart to fill a lane slot would discard it.

Re-entry condition: dispatch once PR #585 lands (or once the core-header lane reports), as a single lane covering Parts A–C together. Nothing about the ticket needs to change first.

One thing worth flagging to whoever picks it up, unmeasured by me: #580 records 18 clang-tidy findings latent in include/morph/forms/forms.hpp. Part B edits that file, so the clang-tidy-diff changed-lines gate may surface findings this change did not introduce. That is #580's trap, not a defect in this ticket — but it is worth knowing before the first red build rather than after.