Repository navigation
feat(shadow): log the paths of mismatching fields in confirmed-mismatch warnings - #242
Merged
Merged
Conversation
…rnings Add the design record for a mismatchLogging content group with paths, value and diff leaves, RFC 6901 pointers for the differing fields, a built-in JSON differ with hand-authored protocol vectors, and a per-use-case paths hook. Every decision is Open; the decisions reviewed on #138 are carried.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #242 +/- ##
==========================================
+ Coverage 93.73% 94.67% +0.94%
==========================================
Files 78 84 +6
Lines 11357 13026 +1669
Branches 656 805 +149
==========================================
+ Hits 10645 12332 +1687
+ Misses 642 628 -14
+ Partials 70 66 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…ts one at a time Add D10: logMismatches stays permanently and expands to every mismatchLogging leaf before the leaf-wise merge, with explicit leaves winning. The shared corpus therefore stays valid, so TypeScript lands first in #242, each port follows in its own pull request, and the vector group and contract row land with the last port.
…paths proposal shadowMismatchPaths is the only hook. paths is the value-free tier and diff narrows values to the differing leaves, so the projector moves to Out of scope with the evidence that would reopen it.
…e list
shadowMismatchDiff(cached, source) returns { path, cached?, source? }
entries; mismatchPaths and diffJson are two projections of that list, so no
pointer is ever resolved against a rendering. The built-in walk attaches both
values as it emits each entry. Dropping diff and a textual diff are recorded
as rejected and out of scope.
mismatchLogging is now { value: boolean, diff: "off" | "paths" | "values" }.
The two diff levels are projections of one entry list and read as the
exposure ladder they are; logMismatches: true expands to the maximal setting.
The three-boolean shape moves to Rejected with the reason.
…failure handling D11: a dedicated metric with outcome logged, hook_error, render_error or logger_error, one event per warning attempt and nothing when logging is off; the warning carries loggingOutcome when it is not logged. The Proposal now states that warning work is detached and never reaches the caller, that the verdict is decided before it starts, and that a failure logs less, never more.
The shorthand sets a baseline for the group and leaves beside it are the exceptions, in both directions; a table gives the same-object and cross-layer cases, and a dominant false is recorded as rejected under D10.
D3 and D5 speak of entries, D6 of mismatchPaths, D7 is titled by diff level, the projector and values-level alternatives use the current leaf names, and the TypeScript step of the Plan lists the shadowMismatchLogging metric.
The maintainer confirmed D0 through D11 in session on 2026-10-24; every decision records that, and the status moves from Proposed to Accepted.
8 tasks
…smatch warnings Add the shadow.mismatchLogging group with the value and diff leaves, keep logMismatches as the shorthand that expands to them before the leaf-wise merge, and validate each leaf independently. Report unknown key-config fields as config_unknown_field. Add the built-in JSON differ (RFC 6901 pointers, UTF-16 member order, the array-length rule, root and depth handling, and the entry, pointer, side and diffJson caps), the shadowMismatchDiff hook with fail-closed handling, the new warning fields, and the shadowMismatchLogging outcome metric with its Datadog and Prometheus exporters. The replay driver records only the mismatchWarning fields every port emits (#243). Re-anchor mutants M08 and M09 on the new resolution code and add M65 for a true shorthand that leaves logging off.
…come metric Reword C60 and the SPEC mismatch-logging rows for the value and diff leaves, and describe the leaves, the shorthand expansion, the warning fields, the shadowMismatchDiff hook, the shadowMismatchLogging metric and config_unknown_field in the shadow validation, observability, configuration, API, upgrading and TypeScript pages. Reference #243 from the design record.
Re-pin the reviewed docs, guides and tests in the source audit with contract dispositions for the new tests, the feature-coverage test hashes, and the Go parity source rows, whose mapping reviews now name the TypeScript-only parts tracked by #243. Update the audit counts lesson 13 quotes.
The diff derives from the same renderings as the value dumps, so it logs
nothing outside them, but the walk reads each whole rendering while the
dumps stop at 8 KiB, so for a larger value it shows differing fields past
that cut. Reword the Proposal, the hook paragraph and D10's rejected
{ value: true } expansion accordingly; the maintainer kept D10.
…quest The maintainer chose to implement Go, Rust and Python in #242 once the TypeScript reference is settled, with the vector group, the contract row, the corpus re-encoding and the differ and hook mutants in every port, and to ship without a breaking-change marker. Record both in the Plan and the Release impact.
…them Walk through a GetUser record that differs at four fields, with the logger call at diff "paths" and the diffJson at diff "values", ahead of the computation rules, and add a table of smaller payload pairs with the paths each one produces. Pin every example: the pairs as path vectors, the Date row with its own values, and both GetUser warnings end to end, field order included.
…fig fields A disabled call is pass-through and observes no config, so only an enabled getOrLoad call reports config_unknown_field for its default config. Make the fetchKeyConfig unknown-field callback optional so mutant M29's edit, which calls it with two arguments, compiles again.
…ow profile Add D12, Open: the shadow profile predicts each confirmed-mismatch warning's fields, null fields, scalar paths and entries, age and shadowMismatchLogging outcome, which every port's replay checks, with three failure inputs and a group of shared regression runs beside the path vectors. Record the companion changes it implies in the contract, Quint, observation and native bullets, Plan steps 1 and 4 and Impact, state the outcome precedence as a fixed order, and return the record to Proposed; only Plan step 4 waits for D12.
Static defaults and runtime resolution validated the diff level with two unlinked lists, neither checked against ShadowMismatchDiffLevel. Both now use one predicate over a record that fails to compile when the union grows, and membership uses Object.hasOwn so Object.prototype member names stay invalid.
Every DialCacheKeyConfig construction scanned its input for unknown fields, including the default snapshot built on each getOrLoad() and the merged config built on each provider call, and a clean constructed config was scanned again at every read. Constructed configs now keep a definite answer, configs DialCache assembles from known fields skip the scan, and objects construction never saw are still scanned. A provider result that is the key's own default snapshot is treated like null: the snapshot was observed at registration or when the call began, and merging it with itself resolves the same policy. Reporting no longer depends on whether the unknown field sits in a layer map or elsewhere.
Mismatch details were set only when logging was on, yet the verdict step re-derived that from a separately threaded plan with a default constant and a helper. Admission now resolves the plan to null when both leaves are off, the details carry it, and the warning runs exactly when details exist.
The mismatch warning reached the raw logger through a second wrapper only to learn whether warn threw. The shared logger wrapper now reports that for every method, and the observer helper returns it directly.
…tured A null or non-function hook took the hook path and turned every confirmed mismatch warning into hook_error. cached() and getOrLoad() now throw a TypeError for it, as they do for shouldAttemptStaleRecovery, before the use case registers.
…n DialCache
fulfills is try { await value } catch, and its doc comment now says only
that it never rejects. The removed comment and tests covered how await
adopts a native promise whose own then was replaced and a global Promise
swapped mid-microtask, which no supported environment does: the
replaced-then logger row and hook, the two foreign-global-Promise logger
cases with their ForeignPromise class, and the bare-thenable iteration
of the hook rejection test, which could never report an unhandled
rejection. The rejecting-promise and rejecting-thenable logger rows and
the rejecting-promise hook keep the consumption covered.
The fifteen cases exercised five branches of runShadowMismatchHook. One case each now covers the catch, the promise-like, the non-array result, the non-object member and the non-string path, beside the invalid member past the 64-entry cut that the contract still rules on and the throwing side getter that shows the try covers member reads.
Re-key the three changed test files in the source audit, filling the retitled pointer-bound and construction-net tests and the new failed-fields test, and dropping the removed walk-throw and foreign-Promise entries. Re-pin the shadow-log-json test in feature coverage, the five changed typescript/src files in Go parity with one more declaration, and the source-audit test count the course quotes.
Joining every reference token for each kept entry made a deep value cost up to 32 escaped 512-code-unit names per pointer, 64 times per warning: about 13 ms on a 31-level chain of 512-character "~" names, against 0.13 ms before the token stack. keptPointer now stops appending once the pointer passes SHADOW_MISMATCH_PATH_MAX_BYTES code units, as childPointer did, so each kept pointer again materializes at most three times the cap and that chain takes 0.5 ms. The clamped pointers are unchanged, and the bound test returns to 3 * 512 + 1, which the unbounded join fails at 1543.
Re-key the shadow-log-json test in the source audit with the restored pointer-bound title and its W10 and X01 dispositions, re-pin that test in feature coverage, and re-pin shadow-log-json.ts in Go parity; the declaration count is unchanged.
…tate the warning-construction fallback per port
lan17
marked this pull request as ready for review
October 26, 2026 07:54
At the maintainer's decision, DialCache no longer validates the entries a shadowMismatchDiff hook returns. A hook that throws, returns a promise-like or returns a non-array still fails with hook_error and never falls back to the built-in walk. Only the first 64 entries are read, inside the hook's catch, and each path is logged in string form, so a hook that returns wrong entries logs wrong paths. Go's and Rust's types already require a string path. The record carries it as a Deviation of D6.
The static shadow group is written from typed fields: its shorthand is a bool and its mismatchLogging is always an object, so expanding it cannot fail and never marks the group invalid. Drop the unreachable error return and the inherited invalid flag; the runtime group keeps both.
Only the library calls the default logger, and the one map it logs, the mismatch warning, holds strings, finite numbers, string lists and nulls, which encoding/json always encodes. Drop the fmt.Sprint fallback for a map that fails to encode and the test that fed it NaN.
The library logs mismatchPaths only as a []string or a nil, and the driver already records nil as null, so encoding the list cannot fail.
Since Go 1.21 panic(nil) is recovered as *runtime.PanicNilError, so the row tested the runtime rather than DialCache; the panics row covers the same branch.
…given The only Mismatch verdict carries both, so the warning no longer keeps a metadata-only fallback for one without them.
The walk is our own code and does not panic: a rendering too deep for the parser is an error, which still fails the walk with hook_error. The CPU job's own net stays.
Bounded entries hold a string pointer and JSON value sides, which always serialize, so the warning no longer tracks a render_error for it.
…ration's type The comparator accepted both values as T before any Mismatch verdict, so the erased hook returns its entries directly and the JSON renderer keeps no fallback for another type. A failed downcast would still land in each one's panic net. Drops the test that pinned only the impossible case.
…the merge The merge now checks the runtime group with the same policy_object call and message static parsing uses, so expanding the leaves cannot fail and resolution keeps no fallback for a shape the merge already rejected.
The walk only parses renderings that already succeeded and compares them iteratively under the 32-token depth cap, so it cannot raise for a value that rendered. The warning's outer net still catches anything unforeseen. Drop the try, the two tests that pinned it by patching _walk, and the docstring's mention of the walk as a hook_error source.
Each bounded entry holds a str path and sides that are either parsed from a rendering of at most 1024 bytes or clamped text, so the list always renders. Render it directly instead of checking for a None rendering.
Every shadow group reaching expand_mismatch_logging belongs to a constructed Policy, whose construction already raises ConfigError for a group that is not a mapping. Drop the second check, the merge's guard that only that branch needed, and the test rows that fed raw non-mapping groups.
…ey outside the net A mismatch outcome within budget always follows capturing the compared pair when a plan exists, so the membership check never fails. Clamping our own logical key cannot raise, so it moves into the warning literal and the net covers only rendering application values; its comment now names the case it exists for. The fallback test loses its clamp_utf8 row.
Six rows reached the hook's catch or its list check the same way another row already does. Keep raises, raises-cancelled, coroutine, tuple (the only row the list check decides), member-without-path and failing-side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Shadow validation is DialCache's background correctness check. For a sampled share of keys, it
compares the value cached in Redis with a fresh result from the source, the function the cache
wraps, off the caller's path. When the two disagree it reads Redis again: if the cached payload is
still there, the mismatch is confirmed. Each outcome, such as
match,mismatchorsuperseded,is counted on the
shadowValidationmetric.When the mismatch rate rises, the only way to learn what disagreed has been
shadow.logMismatches: true. It logs the cache key and the complete JSON of both values, each cutat 8 KiB. That is all or nothing: to learn which field keeps disagreeing, a team has to put whole
records into its logs, and for a large record the cut often lands before the field that differs.
The questions a mismatch rate raises are structural: is it always the same field, is a field missing
on one side, did a list change length, or is the difference something JSON cannot show at all? None
of them need the values.
Pull request #138 built an earlier, TypeScript-only version in August 2026. It predates the Go,
Rust and Python ports, can no longer be rebased, and is superseded here; its reviewed decisions are
carried over. Closes #243, which tracks the ports and the cross-port evidence.
Change
A confirmed-mismatch warning can now say where the cached and source values differ without logging
what they hold. A new
shadow.mismatchLogginggroup chooses what the warning carries:diff: "paths"logs the JSON Pointers of the differing fields.diff: "values"also logs what each side holds at those pointers.value: truelogs both whole values, aslogMismatches: truedid.logMismatchesstays as shorthand for everything on. A use case can replace the built-in pathcomputation with its own hook, and a new metric reports whether each warning was actually logged.
All of it is off by default and runs in the background shadow check after the verdict, so it never
changes the
shadowValidationoutcome, and no failure reaches the caller.The design record
design/2026-10-24-shadow-mismatch-paths.mdholds the decisions (D0 to D15), thealternatives rejected, and the shapes each port uses. D12 makes the Quint shadow profile predict
each confirmed-mismatch warning's fields,
nullfields, scalar paths and entries, age andshadowMismatchLoggingoutcome, checked in every port's replay. Names below are TypeScript's. Change type:Portable behavior (C60's wording and a new contract for the path computation) and Public API (every
port mirrors the config, the warning fields, the hook and the metric).
What a warning looks like
A
GetUseruse case on theusers-apinamespace has this user cached, and the source has changedit since:
With
shadow: { ramp: 5, mismatchLogging: { diff: "paths" } }, once the mismatch is confirmed,DialCache makes this call to the configured logger:
/id,/emailand/profile/namematch, so they are absent. Paths follow member names in UTF-16order, so
/rolescomes after/profile. Only the source has/profile/timezone, and/roleschanged length, so that array is reported once, whole, rather than element by element.
cachedValueAgeSeconds, new on every warning, is the cached value's age at the verdict. Withdiff: "values", the warning also carriesdiffJson:[{"path":"/plan","cached":"free","source":"pro"},{"path":"/profile/locale","cached":"en-US","source":"en-GB"},{"path":"/profile/timezone","source":"Europe/London"},{"path":"/roles","cached":["viewer"],"source":["viewer","editor"]}]docs/shadow-validation.mdwalks through this example and a table of smaller cases. Native testspin both of the example's warnings and every row of the table.
Choosing what to log
shadow.mismatchLoggingholds two independent settings, both off by default. Like other configleaves, each can be overridden on its own by a runtime config:
diff"off","paths","values""paths": where the values differ."values": also what each side holds therevaluetrue,false"paths"logs no field values, so it is the level meant to stay on in production for user data. Itstill logs the cache key, and member names appear in the pointers, so an object keyed by
identifiers logs those identifiers.
"values"andvalue: truelog application data and suit aninvestigation.
shadow.logMismatchesstays permanently, because it is live in consumers' runtime config stores andremoving it would silently stop their logging.
truestands for{ value: true, diff: "values" }and
falsefor both off. A use case has a static default config, and a runtime config provider canreturn an overlay. Within each of the two, the shorthand expands first and a leaf written beside it
overrides the expansion; then the overlay's leaves win over the default's. So a runtime
logMismatches: falsewith no leaves beside it turns everything off, whatever the default sets.In a runtime config, an invalid leaf acts as off, and an invalid
logMismatchesexpands to bothleaves off before the leaves written beside it apply. Either records one
config_resolutionerror,once for each shadow check that actually runs. In a static default config, an invalid leaf makes
cached()orgetOrLoad()throw. AmismatchLoggingthat is not an object is rejected the way amalformed
ttlSecmap is: the constructor throws, and a runtime config with one makes that call rununcached.
DialCacheKeyConfig.disabled()turns both leaves off.How the paths are computed
The built-in computation never logs anything outside the JSON that
valuewould log. Each value isrendered to JSON once, and the value dumps and the built-in diff both come from that rendering, so a
member that a
toJSONmethod leaves out of the dump stays out of the diff too. The two renderingsare parsed and walked together:
the same order. A member present on one side only is reported once, with only that side.
array, so one inserted element does not report every later index.
the root pointer
"".container 32 levels down.
undefined, which the JSON serializer stores for a negative-cache entry, is anabsent value, not a rendering failure (D13). Against a present value it gives one entry at
""carrying only the present side; the absent side's value field is
nulland the outcome stayslogged. A cycle, abigintor a throwingtoJSONstill has no rendering and logsrender_error.The output is capped at 64 entries (
mismatchPathCountkeeps the full count), 512 bytes perpointer, 1 KiB per value inside
diffJsonand 8 KiB fordiffJson. The caps bound what is logged,not the work: the full count needs a full parse and walk of both renderings, so the CPU and memory
cost grows with the values' size (about 86 ms per warning for a 5.6 MB value, against 13 ms for the value dumps
alone). The record lists this under Deviations, and whether to bound it is an open question below. An empty
mismatchPathsmeansboth values have the same JSON, so the comparator, Node's
util.isDeepStrictEqualunless the usecase sets
shadowComparator, saw a difference JSON cannot show, such as aDateagainst its ISOstring.
Computing the differences yourself
For values the JSON walk explains poorly, such as lists that should be matched by ID, a use case can
compute the differences itself.
shadowMismatchDiff(cached, source), an option besideshadowComparatoroncached()andgetOrLoad(), returns the entries{ path, cached?, source? };a hook that knows only where values differ returns paths alone. It follows the comparator's rules:
synchronous, deterministic, side-effect-free, non-mutating and bounded. Its entries are logged as it
returns them, so the
toJSONguarantee above covers only the built-in computation.If the hook throws, returns a promise-like or a non-array, or returns a member that is not an object
with a string
path, thenmismatchPaths,mismatchPathCountanddiffJsonare logged asnull.DialCache never falls back to the built-in computation, so a broken hook cannot log more than its
author intended. A
shadowMismatchDiffthat is not a function makescached()orgetOrLoad()throw a
TypeError, asshouldAttemptStaleRecoverydoes.Knowing whether a warning was logged
Logging can fail quietly, so a new optional metrics-adapter method,
shadowMismatchLogging, reportsone outcome per warning attempt:
loggedhook_errorshadowMismatchDiffor the built-in walk threw or returned an invalid listrender_errorlogger_errorThe first failing stage wins. When a
hook_errororrender_errorleaves fieldsnull, thewarning carries that outcome as
loggingOutcome. The metric fires only while logging is on, socount mismatches with
shadowValidation, not with this metric. Datadog exports it asdialcache.shadow_mismatch_logging.countand Prometheus asdialcache_shadow_mismatch_logging_counter.Compatibility
logMismatches: truekeeps logging every confirmed mismatch,and its warnings gain
cachedValueAgeSeconds,mismatchPaths,mismatchPathCountanddiffJson. The diff reads each whole rendering while the value dumps stop at 8 KiB, so for alarger value it also logs differing fields past that cut. That is deliberate (D10): truncation was
never redaction, and a team that set
logMismatches: truehas already accepted these values inits logs. Each confirmed mismatch also costs more CPU and memory in proportion to the values'
size, and a warning can carry up to about 40 KiB more. To keep the previous fields plus
cachedValueAgeSeconds, writemismatchLogging: { diff: "off" }beside the shorthand. To logwhere values differ without logging them, replace
logMismatcheswithmismatchLogging: { diff: "paths" }: a leaf besidelogMismatches: trueleaves the otherexpanded leaf on, and a service still on an earlier DialCache version reads only the shorthand.
shadowMismatchDiffoption, theshadowMismatchLoggingadapter method, and the exported typesShadowMismatchDiffLevel,ShadowMismatchLoggingConfig,ShadowMismatchDiff,ShadowMismatchEntry,ShadowMismatchLoggingMetricLabelsandShadowMismatchLoggingOutcome. Apart from the newleaves, config is read as on
main: unknown fields stay ignored, and an inheritedshadowgroupstill applies (D14).
DialCacheKeyConfig.disabled().shadownow carriesmismatchLogging: { value: false, diff: "off" }in place of
logMismatches: false.shadow_mismatch_logging.countis now the longest metric suffix, so the adapterconstructor throws a
RangeErrorfor a custom namespace longer than 170 characters (175 before),keeping every name within Datadog's 200-character limit.
types and delete what becomes unused, so Go and Rust callers can see compile errors. Go adds
ShadowPolicy.MismatchLogging, replacesResolvedShadow.LogMismatcheswithLogValueandLogDiff, and removesShadowMismatchLogDetails,ShadowMismatchDetailsandPreviewShadowLogJSON. Rust'sShadowPolicygainsmismatch_logging,ResolvedShadowandShadowMismatchDetailschange shape,Operationholds a private renderer, and exhaustive matchesover
EventandMetricKindgain a variant.docs/upgrading.mdnames each change. At themaintainer's request the title carries no
!.previewreplaces the JSON for the whole warning, so a redactingpreview also bounds the difference fields: they come only from a
mismatch_diffhook, and arerender_errorwithout one. An operation whosepreviewis explicitlyNonekeepsmain'smeaning and logs no values, at the maintainer's choice. Operations built with
with_codechavedifferences only through the hook (Rust: let with_codec operations supply a JSON renderer for mismatch paths #245 tracks a JSON renderer for them).
cachedValueJsonandsourceValueJsonnow escape lone surrogates asJsonSerializerdoes, and a static default with an invalid new leaf raises
ConfigError, as invalid staticlog_mismatchesalready did. A logger whosewarning()returns an awaitable is awaited for theoutcome (D15).
Formal evidence
DialCache keeps its ports in step with a shared formal layer: numbered contracts in
formal/guides/CONTRACTS.md, Quint models that generate call histories every port must replayidentically, and a catalog of mutants (deliberate bugs) that each port's tests must catch.
valueis true ordiffis above"off"oncelogMismatchesexpands, and states the outcome rule: oneshadowMismatchLoggingevent per warningattempt, none with logging off, ranked
hook_error>render_error>logger_error. A newcontract, W10, states the path computation as a portable function, and the hand-written
mismatchPathVectorsgroup (57 vectors: the documented rows, D13's absent sides, own members namedlike
Object.prototypemembers, the depth cap, the array rule, the root cases, every cap, theuncut count and the number rule) is replayed by every port.
SPEC.mdsays the same.valueanddiffleaves atadmission and predicts each warning's fields,
nullfields, scalar paths and entries, age andoutcome. It has starting and runtime choices for every level, the shorthand rows and written
offleaves, three failure inputs (a failing hook, a failing logger, a source value with no JSON
rendering), six new invariants and 21 new exported regressions. Admission, dark-layers,
shadow-layers and shadow-read-deadlines keep their versions and agree with
mainon every history.the array rule, a root kind change, an absent side as
render_error, a hook failure falling back tothe walk, the hook at
diff: "off", an ignored writtenoffleaf, both precedence swaps, a warningwith logging off, entries logged at
"paths"and an invaliddiffleaf read asvalues, inTypeScript and Go, with Rust twins M14 to M25. Eleven C60 model-fault challenges map to them. The
Go and Rust mutation matrices grow to 15 and 4 shards so the larger catalogs fit their timeouts.
Python has no mutation catalog, so its evidence for these rules is the shared replay, the vectors
and its native tests. The walk's own-member rule has no mutant, because Go maps have no inherited
members; the vectors pin it in every port.
Where the code lives
typescript/src/internal/shadow-log-json.ts: rendering, the built-in differ, its caps and the hookrunner.
typescript/src/internal/runtime-config.ts: the shorthand expansion and the leaf-wise merge.typescript/src/config.ts: the config types anddisabled().typescript/src/dialcache.ts: theshadowMismatchDiffoption and the warning.typescript/src/metrics.ts,datadog.ts,prometheus.ts: the outcome metric.typescript/src/internal/thenable.ts: one helper that consumes a promise-like returned by alogger, metrics adapter, mismatch hook, comparator or stale recovery predicate, so a rejection is
never left unhandled. It replaces
main'ssettleUnexpectedThenableat the comparator andpredicate sites; the comparator now waits one more microtask before its deadline check when it
returns a value that is neither a boolean nor a promise, which no caller can observe.
go/policy.go(leaves and merge),go/shadow_log.go(rendering, walk, hook),go/shadow.go(the warning),
go/metrics*.go(the outcome metric).rust/src/policy.rs,rust/src/protocol/mismatch.rs(walk),rust/src/shadow_log.rs(warning content on the bounded CPU executor),
rust/src/operation.rs(hook and renderer).python/dialcache/config.py,python/dialcache/_shadow_log.py,python/dialcache/cache.py.formal/models/kernel/shadow.qnt,formal/models/dialcache-shadow-conformance.qnt,formal/replay/features.mjs,formal/catalogs/protocol-vectors.json, and every port's replaydriver.
Ports
The temporary bridges the TypeScript-first stage needed are gone: every driver records the whole
warning and the
shadowMismatchLoggingevent, the corpus exercises the new leaves, the vectors andW10 are in, and the Go parity ledger maps the feature to its Go bindings. Each port documents its
native differences (B03, X01) in
docs/shadow-validation.mdand the record's Deviations: Go's andRust's JSON parsers fail very deep nesting as
hook_error,diffJsonmember order follows eachport's maps, and the walk's cost grows with value size and member count in every port.
Validation
Head:
934d5b8.TypeScript review: multi-lane review rounds (correctness, tests, simplicity,
architecture, contracts, security, reliability, performance, then two holistic passes each), with
a plan critic before each set of fixes.
84dcfbd) led to6cd9fe3toc289ed3: one clamp rule for built-in and hookentries, a registration
TypeErrorfor a non-function hook, three simplifications, and the docsand record notes above.
c289ed3) led to D13 (0e6fb37,32d59d9,3239e49).6684ac7) led toa919d50: the per-call shadow merge carries the raw groups andexpands the shorthand once at admission, checked against expanding each layer first across
8,064 combinations.
main's config reads (D14): unknown-field reporting dropped (ed80bbe,646d5d9) andshadowread as ordinary properties again (377526c,964ade2). With it gone,a local-hit
getOrLoadand acached()local hit with a provider and a shadow group measurewithin noise of
main, and a disabledgetOrLoadwith an inline shadow config about 8% over.45cc3d2led to658d6d2: the merge expands each layer's shorthand firstagain, as the record states the rule, at
main's per-call cost; its second round led to70b5e58, an independent oracle for the merge test. The maintainer then made an asynchronouslogger failure a
logger_error(D15,efa6b33,aad2ae7).e508c68,d79e4a7) pinned the walk to own members (f6a8aba,e1537c1) and D15's timing: a hook or render failure is recorded at once even while the logger'spromise is pending, and a promise that never settles records nothing (
48b3dc0). A returnedpromise is now observed through
await(2cbca04), so a globalPromisereplaced after loadcannot miscount a delivered log or leak a rejection.
a9c47d9) led tod8a6412: one internal helper,isPromiseLikeandfulfills, now consumes a promise returned by a logger, metrics adapter, hook, comparator orstale recovery predicate, replacing four per-site copies; and
7cdef0f, which states the walk'smemory and sorting cost.
842e5f9) was clean in every lane and in both holistic passes, with no fixapplied: the TypeScript reference's clean confirmation round.
the
thenable.tscomment and a test name were narrowed (a native promise whose ownconstructoris also replaced is adopted through its ownthen, as onmain), the go-parityrationale names
fulfills, a test pins that the hook receives the rawundefinedfor an absentside, and the upgrade note's 40 KiB figure says field bytes.
Ports and Plan step 4: each piece was built on its own branch, reviewed by fresh reviewers until a
round came back clean, and merged here.
through each port against TypeScript) and a quality reviewer per port. Round 1 found the same
defect in all three: the walk copied the whole pointer prefix for every member, quadratic where
TypeScript's rope strings are linear; each now builds pointers only for kept entries, pinned by an
allocation test. Rust's review also found that a redacting
previewno longer bounded thedifference fields; it now fails closed. Each port planted 20 to 29 bugs against its new tests.
and vector runners, the formal additions (written
offleaves) and the shared docs (three rounds,each fixing one accuracy slip), then the mutants and challenges (two rounds: the larger catalog
needed more Go and Rust mutation shards, and the Rust driver had dropped
diffJsonat"paths", which hid a mutant).Whole-PR review round 1 (
0532f83): every port with all eight lanes, the formal layer with six, the docsand record with two, and the TypeScript delta with three, then a holistic pass. It accepted 21 findings, fixed
in
a424f87to80d41ddand each verified clean:check-modelsalready ran 143–148 of its 150 minutes onmain, and the 11 new challenges push it over (Fullrun 40245022083 confirmed it: every other lane passed and
check-modelswas cancelled at its limit). Itstimeout is now 210 minutes, and
formal-validation.test.tspins its capacity, as it does the mutation lanes.Kept pointers are clamped while they are built in Go and TypeScript, which ends a 64× memory amplification
from one huge member name. Go builds warning content while the shadow job still holds its capacity.
the documented rule, which five new vectors now pin in every port. Rust keeps one value renderer, and the
stale vector copies are gone.
a redundant invariant was deleted, and the docs say when an outcome waits for an async logger.
comparison, which would need lossless JSON parsing in TypeScript (see Follow-ups).
Whole-PR review round 2 (
80d41dd), the same lanes again: 24 of 35 clean; 11 findings, none high, fixed inf16f75fto85febccand each verified clean, plus round 2b from its holistic challenge pass:[""]) no longer prints like nodifference (
[]), and hook sides honor a redactingMarshalJSONat every depth (the codec's stored encodingis unchanged).
storage, and compares integers beyond the double range as doubles too (the maintainer confirmed the double
rule as W10's in session, 2026-10-25: "Confirm doubles").
when every reported difference builds a pointer;
check-rustlints the published Rust build withouttest-util.its five challenges no longer run against four profiles that cannot reach it.
parse-failure rule and the Rust executor coupling. The walk's cost stays as documented, by the maintainer's
decision on 2026-10-25, with a faster implementation tracked in Shadow mismatch paths: make the built-in walk faster, TypeScript first #248.
A final review of the delta since
80d41dd(85febcc), with lanes sized to each area and only medium-or-higherfindings blocking: Rust, Python, formal and docs clean. Three Go lanes and the holistic pass found one medium
issue: a hook side marshalled a struct from a copy, so a field's pointer-receiver
MarshalJSONdid not run anda redacted value reached
diffJson.a8fd864marshals an addressable struct through its pointer, asencoding/jsondoes. An independent check matchedencoding/jsonon 38 value shapes, found the storedencoding byte-identical, and confirmed the new test fails on the old code.
make check,make formal-go,make smokeandmake integration-gopassed atef9618f.Local validation at
85febcc, each a separate command, all passed:make formal-generate(fresh corpus),make fixtures-check,make check(TypeScript 3,944 tests; Go, Rust and Python native suites; docs;make audit),make smoke,make model-check,make formal-ts(7,920 tests),make formal-go,make formal-rustandmake formal-python(7,837 required cases each), andmake integration(every portagainst Redis, Valkey and Cluster, and the wire vectors).
TypeScript review on this pull request (posted 2026-10-26 at
80d41dd, folded in at the maintainer's request): thedefensive code is cut to the real boundaries, config parsing and the diff generator.
a74b2a4merges it:expandMismatchLoggingand the catch around the built-in walk are gone;nullwithrender_errorinstead of dropping them (a Deviation records it);
typescript/srcshrank by 10 lines andtypescript/testby 106. Behavior a caller, logger or metrics adapter cansee is unchanged: all 57 vectors and every warning-shape test pass. A verifier tried to make each deleted guard
matter and could not. It did catch one regression in the first cut (a deep pointer rebuilt from the whole
stack, 14–100× slower), fixed in
8f7a264.Validation for that TypeScript-only delta, by the maintainer's decision, is the TypeScript lanes: at
d756793,make check-ts,make formal-ts,make docsandmake audit; atdcb7784(the same TypeScript source),make mutations-ts MUTATION_ONLY=M66,M67,M68,M69,M70,M72,M75, with every required detection held. No Go, Rust,Python, model or catalog source changed after
ef9618f.Custom differ entries are no longer validated (
99880fe), at the maintainer's decision: "if a custom differreturns non correct results its fine". A hook that throws, returns a promise-like or returns a non-array still
gives
hook_errorand never falls back to the walk; the first 64 entries are logged as returned, with paths instring form. The record carries it as a Deviation of D6. Validated with
make check-ts,make formal-ts,make check-python,make formal-python,make docsandmake auditat4515217.The same defensive-code trim in Go, Rust and Python (
d2f5960), under the maintainer's rule to remove defensivecode that no real boundary needs: Go drops re-validation of the already-validated static policy and an encoder
fallback that could not fire; Rust reads a mismatch verdict's values as given and drops dead branches; Python
drops the catch around the built-in walk and duplicate checks. 96 lines added, 199 removed, no observable
change for any supported input. Each port's native checks (
make check-go,make check-rust,make check-python, ruff) passed and every mutant anchor still matches; PR CI covers the replay and the rest.Full run 40274292379 at
ef9618f: passed, every lane.Full run 40259905276 at
80d41dd: passed, every lane,check-modelswithin its new 210-minute limit.Full run 40245022083 at
0532f83: every parity and mutation lane passed;check-modelstimed out at its old150-minute limit.
Full run 40272657294 at
85febccwas cancelled for the Go fix; nothing is running.Follow-ups
A faster built-in walk (Shadow mismatch paths: make the built-in walk faster, TypeScript first #248): the measured no-dependency speed-up, TypeScript first.
Faster validation for portable-behavior changes (Formal: cut wall time for portable-behavior changes without weakening the evidence #247): shard the model-fault campaign, scope mutants to the
cohorts that can catch them, cache Full lanes by input closure.
Go hook sides and
encoding.TextMarshaler. Outside a struct, a hook side honorsjson.Marshalerbut notencoding.TextMarshaler, whichencoding/jsonalso calls; the docs promise onlyjson.Marshaler.A comparator that returns a promise that never settles (pre-existing on
main, unchangedhere). The shadow operation awaits a comparator's accidental promise-like so the flight is retained
until it settles, by design. If it never settles, the flight is never released. With the default
shadowMaxInFlightof 1, every later shadow job on that instance is thendropped, for every key.The hook and the stale recovery predicate consume such a value without awaiting it. Whether the
comparator should do the same, or release the flight at the deadline, is a separate decision.
A JSON renderer for Rust
with_codecoperations (Rust: let with_codec operations supply a JSON renderer for mismatch paths #245): today they get difference fields onlythrough a
mismatch_diffhook.Go's member sort allocates on every comparison (
utf16Less), which dominates rendering a verywide map; the cost is linear-logarithmic and documented, so it is a performance follow-up.
Exact comparison of 64-bit integers in the walk. Numbers compare as doubles in every port, so two ids
that differ beyond a double's precision log no path. Exact comparison would need lossless JSON parsing in
TypeScript (for example
JSON.parsewith source access).A Python mutation catalog, so the Python port's tests are measured like the others.
Strict runtime config validation (D14): a runtime config that fails validation is rejected
as a whole, records an error metric and runs the call uncached, as one invariant in every port.
It gets its own design record and pull request.