perf(shadow): build TypeScript mismatch warnings in event-loop steps

design/2026-10-24-shadow-mismatch-paths.md

@@ -2,7 +2,7 @@
 
 - **Status:** Implemented
 - **Issue:** #243, which tracked the work after the TypeScript reference; #242 completes it
-- **Pull requests:** #242
+- **Pull requests:** #242, #250
 - **Baseline:** `main` at `b4e4fba`, the commit the record was written against
 - **Supersedes:** the design in pull request #138, which is unmerged; the
   decisions it carries are listed under [Carried decisions](#carried-decisions)
@@ -998,3 +998,29 @@ Reviewed and approved by the maintainer on #138; carried unchanged:
   `kernel/shadow.qnt` with logging off. `kernel/shadow.qnt` keeps the
   captured `Logging` leaves, `LOGGING_OFF`, `logs` and the diff levels, and
   does not import the new module.
+- **2026-10-28, #250, TypeScript:** at the maintainer's request in session
+  that day ("how we can leverage setImmediete and some other optimizations to
+  make sure we are not taking as long and also we are not blocking event
+  loop and also that its still correct"), the warning returns to the event
+  loop after each native rendering or parse of a side whose rendering is at
+  least 65,536 UTF-16 code units long; each such step, the walk and the hook
+  still run without a break. The replay histories' values are shorter than
+  that, so they replay the warning as one step, and native tests cover the
+  breaks. The warning also holds the job's shadow capacity slot until it
+  reaches the logger, as Go and Rust do, so Failure handling's "TypeScript
+  and Python render after the slot is released" no longer holds for
+  TypeScript; the slot stays a port detail. With the default
+  `JsonSerializer`, the walk reads the cached value that `JSON.parse` of the
+  payload produced instead of parsing that side's rendering again, comparing
+  a number beyond double range as the `null` the rendering writes. The two
+  hold the same JSON, so the output D3 and step 1 of the computation describe
+  is unchanged; only a comparator that mutates the cached value, which its
+  contract forbids, can make the diff name members that `cachedValueJson`
+  does not show, or fail the warning with `render_error`. The walk also skips
+  members and items whose values are identical before sorting, and a
+  built-in `diffJson` side is rendered only as far as its 1 KiB clamp reads.
+  The vectors, the existing tests and a seeded fuzz against `main` at
+  `bae1e5e` agree on every output. On one Apple-silicon Mac, with the
+  default `JsonSerializer`, a 7.3 MB object of 150,000 string members took
+  about 0.11 s per warning at `diff: "paths"`, against 0.14 s on `main`, in
+  steps of at most about 0.03 s, and 0.05 s for the value fields alone.

docs/shadow-validation.md

@@ -305,7 +305,8 @@ holding the shadow slot after timeout.
 Scheduling and shadow deadline timers are unreferenced and do not keep an
 otherwise idle Node process alive. Detachment uses the event loop, not a worker;
 synchronous source, codec, comparator and logger work still consumes it, as do
-mismatch-warning rendering and `shadowMismatchDiff`.
+mismatch-warning rendering and `shadowMismatchDiff`. A mismatch warning keeps
+the job's capacity until it reaches the logger.
 
 </LanguageContent>
 
@@ -692,10 +693,20 @@ const getUser = dialcache.cached(fetchUser, {
 });
 ```
 
-Rendering, the walk, and the hook run on the event loop after the job releases
-its capacity slot. On one Apple-silicon Mac, a 6.3 MB object of 150,000 members
-took about 0.12 seconds per warning at `diff: "paths"`, against 0.06 seconds for
-the value fields alone.
+Rendering, the walk, and the hook run on the event loop after the verdict and
+hold the job's shadow capacity until the warning reaches the logger, so while
+one is built, further jobs for the same key and jobs beyond capacity report
+`dropped`. After each `JSON.stringify` or `JSON.parse` of a side whose rendering
+is at least 65,536 characters long, the warning returns to the event loop, so
+other work runs between those steps; each of them, the walk and the hook still
+run without a break. With the default `JsonSerializer`, the walk reads the
+cached value as it was decoded instead of parsing its rendering again, so a
+comparator that mutates the cached value, which its contract forbids, can make
+the diff name members that `cachedValueJson` does not show, or fail the
+warning with `render_error`. On one Apple-silicon Mac, with the default
+`JsonSerializer`, a 7.3 MB object of 150,000 string members took about 0.11
+seconds per warning at `diff: "paths"`, in steps of at most about 0.03 seconds,
+against 0.05 seconds for the value fields alone.
 
 </LanguageContent>
 

docs/upgrading.md

@@ -243,9 +243,18 @@ characters.
 
 `DialCacheKeyConfig.disabled()` now turns logging off with
 `shadow.mismatchLogging: { value: false, diff: "off" }` in place of
-`shadow.logMismatches: false`. On one Apple-silicon Mac, a 6.3 MB object of
-150,000 members took about 0.12 seconds per warning at `diff: "paths"`, against
-0.06 seconds for the value fields alone.
+`shadow.logMismatches: false`. On one Apple-silicon Mac, with the default
+`JsonSerializer`, a 7.3 MB object of 150,000 string members took about 0.11
+seconds per warning at `diff: "paths"`, against 0.05 seconds for the value
+fields alone.
+
+A warning now holds the job's shadow capacity until it reaches the logger, as
+in Go and Rust; earlier releases released it before building the warning. So
+while a warning is built, further jobs for the same key and jobs beyond
+`shadowMaxInFlight` report `dropped`. For a side whose rendering is at least
+65,536 characters long, the warning returns to the event loop after each
+`JSON.stringify` and `JSON.parse` of it, in steps of at most about 0.03 seconds
+on that object.
 
 </LanguageContent>
 

formal/catalogs/feature-coverage.json

@@ -1547,19 +1547,19 @@
           "test": "byte-clamps keys and JSON without splitting UTF-8 sequences",
           "path": "typescript/test/shadow-log-json.test.ts",
           "scope": "Asserts that the TypeScript binding byte-clamps keys and JSON without splitting UTF-8 sequences.",
-          "sha256": "2dab2b5558563125f731410b170a41c8c216ff28d54a012b3744ce387c99213f"
+          "sha256": "e14254c394cef87dc3ab5cd71a40550738dc8dba4eda5a35903b6dcb7e64664b"
         },
         {
           "test": "catches user JSON hook failures",
           "path": "typescript/test/shadow-log-json.test.ts",
           "scope": "Asserts that the TypeScript binding catches user JSON hook failures.",
-          "sha256": "2dab2b5558563125f731410b170a41c8c216ff28d54a012b3744ce387c99213f"
+          "sha256": "e14254c394cef87dc3ab5cd71a40550738dc8dba4eda5a35903b6dcb7e64664b"
         },
         {
           "test": "returns null independently for values that native JSON cannot serialize",
           "path": "typescript/test/shadow-log-json.test.ts",
           "scope": "Asserts that the TypeScript binding returns null independently for values that native JSON cannot serialize.",
-          "sha256": "2dab2b5558563125f731410b170a41c8c216ff28d54a012b3744ce387c99213f"
+          "sha256": "e14254c394cef87dc3ab5cd71a40550738dc8dba4eda5a35903b6dcb7e64664b"
         }
       ],
       "go": [

formal/catalogs/go-parity.json

@@ -10,8 +10,8 @@
   "inputs": {
     "semanticCasesSha256": "380273598e1d956a3f9fce270cc84424b342e456139b279bd56576bd3b837f65",
     "executionSha256": "eaf117c3e8630013cc8df0524f84af440c470f42e8711ab4dd07b931161c784f",
-    "sourceAuditDispositionsSha256": "c95ce03c30df574246e5943e2d69e084ed809b5d3e2a3d0f0abf5a2123bc8aaa",
-    "featureCoverageSha256": "3f33092274a0046c2d793790b1b4999021021e018f47ae65c15d3a0313776fd2",
+    "sourceAuditDispositionsSha256": "308073255328096e21c27cfa90e73e6d9e39bf9e2aad15e73abd10a390480230",
+    "featureCoverageSha256": "e494cdc5c0d26f495de2fe780b5c43d2e028d252cfaa6e8a10c544b6d58d1584",
     "coverageWitnessesSha256": "745cebececddbfa97912621ada94072d648e8e60e8b3eb013b91eefd16f352a1",
     "profilesSha256": "888a522f8ad97862ebcb0b5fe363ec7bec60367fa42c60e1623141814b1991b3"
   },
@@ -21,7 +21,7 @@
     "withoutRequiredGeneratedWitnesses": [],
     "wireCases": 23,
     "sourceFiles": 28,
-    "sourceDeclarations": 878,
+    "sourceDeclarations": 887,
     "casesWithScheduledChecks": 272,
     "requiredWitnesses": 535,
     "casesWithQuintRegressionReplay": 198,
@@ -3499,7 +3499,7 @@
     },
     {
       "path": "typescript/src/dialcache.ts",
-      "sha256": "88e1e9bdf70bba085f4c8f2861cfac89ec021a5af318e21dc94e9f54c75b583c",
+      "sha256": "04d7eda97dd9da30e3d7fe97117de4f69e4ad7d43ecd8ff1ba9f2b2a61118e6c",
       "candidateGoFiles": [
         "go/cache.go",
         "go/bindings.go",
@@ -3511,7 +3511,7 @@
       ],
       "mappingReview": {
         "status": "reviewed-mapping-not-execution",
-        "rationale": "The public orchestration is divided into Cache lifecycle, Cached registration, GetOrLoad execution, source/read deadlines and detached shadow work. GetOrLoad snapshots inputs before awaiting policy; singleFlight owns leaders/followers. Go contexts, generics, returned errors and recovered callback panics replace TypeScript overloads, promises and exceptions; default diagnostic logging lives in an internal package, while public logger callbacks and inspection remain root-package bindings. The warning's leaves come from the invocation's ResolvedShadow, Operation.MismatchDiff replaces the built-in walk, and warnShadowMismatch emits one shadowMismatchLogging event; a Warn panic is the Go logger_error.",
+        "rationale": "The public orchestration is divided into Cache lifecycle, Cached registration, GetOrLoad execution, source/read deadlines and detached shadow work. GetOrLoad snapshots inputs before awaiting policy; singleFlight owns leaders/followers. Go contexts, generics, returned errors and recovered callback panics replace TypeScript overloads, promises and exceptions; default diagnostic logging lives in an internal package, while public logger callbacks and inspection remain root-package bindings. The warning's leaves come from the invocation's ResolvedShadow, Operation.MismatchDiff replaces the built-in walk, and warnShadowMismatch emits one shadowMismatchLogging event; a Warn panic is the Go logger_error. Both ports hold the job's shadow capacity until the warning is logged; TypeScript returns to the event loop between the native JSON steps of a long side, where Go builds the warning on the job's goroutine.",
         "nativeAdaptations": [
           "B01",
           "B02",
@@ -3918,7 +3918,7 @@
     },
     {
       "path": "typescript/src/internal/redis-cache.ts",
-      "sha256": "6fd0f9d16289efd66d60cfdf538d44fb520301a45634f72c5aaaaf4f79deae79",
+      "sha256": "3c6e14bc665a7807870752d8a346144efa4e7b892a4d4087465a33cce78b4143",
       "candidateGoFiles": [
         "go/engine.go",
         "go/shadow.go",
@@ -3927,7 +3927,7 @@
       ],
       "mappingReview": {
         "status": "reviewed-mapping-not-execution",
-        "rationale": "Serving, retained stale recovery, compression, timestamp validation and local publication are in execution methods; detached shadow validation is separate. A raw semantic read is normalized before serving, elapsed deadlines retain acquired snapshots, and both tracked refill timestamp checks precede a complete frame write.",
+        "rationale": "Serving, retained stale recovery, compression, timestamp validation and local publication are in execution methods; detached shadow validation is separate. A raw semantic read is normalized before serving, elapsed deadlines retain acquired snapshots, and both tracked refill timestamp checks precede a complete frame write. loadsParsedJson, which lets the TypeScript warning walk the JSON serializer's decoded value, is a TypeScript speed-up with no Go counterpart.",
         "nativeAdaptations": [
           "B02",
           "B03"
@@ -4116,15 +4116,15 @@
     },
     {
       "path": "typescript/src/internal/shadow-log-json.ts",
-      "sha256": "ddab3366b00b83f19c796f3e457b9a12dc422e61f3f2b1b1d1c41731785687d5",
+      "sha256": "d87c767194f855013cf2626e77abe82e7106e9d1145b9d51337a098f669e2bdb",
       "candidateGoFiles": [
         "go/metrics.go",
         "go/codec.go",
         "go/shadow_log.go"
       ],
       "mappingReview": {
         "status": "reviewed-mapping-not-execution",
-        "rationale": "Warning content renders each compared value once with the Go JSON compatibility writer; the built-in walk runs over the parsed renderings with the same member order, array rule, root cases, depth cap (deep equality past it) and caps, materializing only the kept pointers up to their clamp, and the MismatchDiff hook's entries take the same clamps. Byte-bounded UTF-8 output uses the same truncation marker; a failure nulls only the fields it feeds and cannot suppress the verdict or caller result. Native type representation is a codec binding (B03): Go's decoder fails a rendering nested past 10,000 levels as hook_error. Hook entry sides honor json.Marshaler at every depth, as TypeScript sides honor toJSON, while the codec stored encoding reaches MarshalJSON only through a struct.",
+        "rationale": "Warning content renders each compared value once with the Go JSON compatibility writer; the built-in walk runs over the parsed renderings with the same member order, array rule, root cases, depth cap (deep equality past it) and caps, materializing only the kept pointers up to their clamp, and the MismatchDiff hook's entries take the same clamps. TypeScript's walk reads the JSON serializer's decoded cached value in place of a second parse, skips identical members and items, and renders a built-in side only as far as its clamp reads; the output is the same. Byte-bounded UTF-8 output uses the same truncation marker; a failure nulls only the fields it feeds and cannot suppress the verdict or caller result. Native type representation is a codec binding (B03): Go's decoder fails a rendering nested past 10,000 levels as hook_error. Hook entry sides honor json.Marshaler at every depth, as TypeScript sides honor toJSON, while the codec stored encoding reaches MarshalJSON only through a struct.",
         "nativeAdaptations": [
           "B03",
           "X01",