Skip to content

fix(coordination): preserve shadow lineage across root aliases - #4896

Merged
huangruiteng merged 4 commits into
loopx-project:mainfrom
Duang777:codex/fix-shadow-root-digest-parity
Sep 22, 2026
Merged

huangruiteng merged 4 commits into
loopx-project:mainfrom
Duang777:codex/fix-shadow-root-digest-parity

Conversation

@Duang777

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

Scope And Continuation

  • Completed scope and remaining work: Python now accepts the lexical or canonical spelling for the same runtime root. Python Todo capture, TypeScript lease capture, and Python receipt validation use the already-validated active lineage digest. The journal and receipt schemas are unchanged.
  • Slice boundary / successor: Complete within this scope. The long-running sustained-parity soak remains the existing RFC gate and is not changed by this fix.

Validation

  • Tested revision: c0cb1b3878e2f30baff7665ae3eb072e83d349a0
  • Run state: finished
  • Input classes: synthetic, public_fixture
Check kind Result Public-safe evidence / limitation
regression_parity passed Before the fix, the explicit symlink-root reader, Python outbox, and TypeScript outbox tests failed on unequal root digests. After the fix, tests/control_plane/test_shadow_management.py plus tests/control_plane/test_local_authority_shadow_outbox.py passed 24/24, and tests/control_plane_ts/local_authority_shadow_outbox.test.ts passed 21/21.
real_entrypoint passed python -m loopx.control_plane.testing.authority_e2e_ladder --row s2c2.outbox_prepared_then_committed_entries passed under the default macOS temporary directory and delivered both writer partitions exactly once.
integration passed The complete Stage 2C2 selection passed all 11 executable rows. The pre-existing >=10-day sustained parity soak remains pending by design.
static passed ruff check passed for all changed Python files. npm run typecheck:control-plane passed.
integration passed Standard loopx canary premerge --from-git-diff passed 13 selected checks with no failures, warnings, or manual holds.
  • Coverage and gaps: Explicit directory symlinks make the regression deterministic on platforms whose default temporary path has no alias. Tests cover journal readback, both writer runtimes, cross-root rejection, exact drain delivery, and the wider Stage 2C2 paths. Only the existing long-duration soak was not run.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • States and viewports shown: N/A
  • Source data: none
  • Attention review: N/A because this change has no user-visible UI.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: Shared Goal Authority local runtime-shadow correctness.

Shared-authority RFC fixture impact

  • Production-scale fixture schema: unchanged.
  • Semantic dimensions changed, or reviewed no-impact rationale: Runtime-root aliases now retain one immutable lineage identity across management readback, Python Todo capture, TypeScript lease capture, and receipt validation.
  • Provider conformance arms run: file_v0 runtime-shadow Stage 2C2, all 11 executable rows passed.
  • Read-only legacy/file/PostgreSQL three-arm rehearsal (required for promotion, runtime-routing, or compatibility-projection changes): Not applicable. This change does not alter promotion, provider routing, or compatibility projections.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths (including .loopx/, .codex/goals/, and live ACTIVE_GOAL_STATE.md).
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer (git commit -s).

Accept both lexical and canonical runtime-root digests when Python reads the TypeScript-owned management journal. Bind Python and TypeScript outbox entries, plus drain receipt validation, to the active lineage digest instead of recomputing a conflicting path identity. Add explicit symlink-root regression coverage so macOS temporary roots exercise the cross-runtime path.

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
…-digest-parity

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777

Copy link
Copy Markdown
Contributor Author

Synced current main (27e291a44) into exact head c1a91a55c76fe649ca375944736f2dd14f1cf91e without conflicts. The newly merged quota recovery change does not overlap this PR's coordination files.

Fresh validation on the merged head:

  • Python shadow-management/outbox regression suites: 24 passed
  • TypeScript outbox suite: 21 passed
  • control-plane TypeScript typecheck: passed
  • s2c2.outbox_prepared_then_committed_entries: passed on the real CLI path
  • standard diff-selected canary: 13/13 passed, no warnings or manual holds
  • DCO trailers present

CI is rerunning for the new exact head. This control-plane fix remains maintainer-merge only.

@Duang777

Copy link
Copy Markdown
Contributor Author

Coordination note: #4898 was opened shortly after this PR with the same production scope. I compared the implementations and left one concrete invariant finding there: its Python reader currently accepts a mismatch between the journal-level and binding-level root digests, while the TypeScript decoder and this PR require exact equality after validating the journal digest against the lexical/canonical root set. I reproduced that mismatch acceptance on #4898's exact head.

I am keeping this PR open pending maintainer choice rather than duplicating the other PR's additional test coverage. The current head remains the smaller strict-invariant implementation and is rebased on main@27e291a44.

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777

Copy link
Copy Markdown
Contributor Author

Added a focused fail-closed regression on exact head fa26a5cfe75322c1786b6a4e91d4dbb54ca24762: a lexical journal digest paired with a canonical binding digest for the same physical root must still be rejected. This preserves the existing TypeScript state/binding equality contract while allowing either spelling as the journal identity. The updated shadow-management suite passes 11/11 and Ruff/diff checks pass.

…-digest-parity

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777

Copy link
Copy Markdown
Contributor Author

Synced latest main@952c1c0ae into exact head 98c400a5009e08d972add5126a6d77dab2dce3a9 without conflicts. Fresh validation: Python shadow management/outbox 24/24, TypeScript outbox 21/21, and Ruff passed. The superseded head also passed both Stage 2C e2e jobs. #4898 remains unchanged and has not addressed the mixed journal/binding digest invariant. New exact-head CI is starting.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

我按 exact head 98c400a5009e08d972add5126a6d77dab2dce3a9 审阅了 #4892、完整 diff、相邻实现 #4898,以及 Python/TypeScript 的 shadow-management reader、两个 outbox writer 与 drain。这个 PR 解决的是一个真实的跨运行时持久状态缺口:macOS 默认临时目录含路径别名,TypeScript owner 写 canonical root digest,而 Python 旧 reader/writer/drain 从 lexical spelling 重算 digest,导致 Stage 2C2 在 guard 或 receipt history 处 fail closed。

相邻 #4898 的正常路径也能恢复,但它会接受 journal 顶层与 binding 分别使用 lexical/canonical 两个 digest 的 mixed state;本 PR 选择更窄的修复,保留既有 state→binding exact equality,并已补上这个负向回归。因此它不是重复增加第二套语义,而是当前两个候选中与既有跨语言 contract 对齐的实现。

改动思路

正确边界分两步:首先,runtime root 的 lexical/canonical spelling 只用于判断 persisted journal 的顶层 source_root_digest 是否属于同一个物理 root;其次,一旦顶层 digest 被接受,它就是该 journal 的精确 lineage identity,binding、Python Todo entry、TypeScript lease entry 与 drain receipt 都必须复用这个单值。

本 PR 复用了这个既有 owner:Python _source_root_digests() 对齐 TypeScript sourceRootDigests(),而 _binding(..., root_digest) 仍要求精确相等。随后两个 writer 与 drainer 从已验证的 active binding 取 digest,不再各自从调用路径推导。这同时保持了 foreign-root、stale-lineage、sequence 与 receipt mismatch 的原有 fail-closed 行为。

具体改动

  • shadow_management.py 增加 lexical/canonical digest set,只放宽顶层 root spelling admission,不放宽 state/binding identity。
  • Python Todo capture 与 TypeScript lease capture 改用 binding.source_root_digest 生成 entry identity 和持久字段。
  • _PartitionDrainer 缓存同一 binding digest,并用它校验 receipt history。
  • Python/TypeScript tests 增加显式 symlink root、双 writer alias、foreign root,以及 mixed journal/binding digest 必须拒绝的覆盖。

关键代码讲解

  • _source_root_digests / read_shadow_management_state:允许 abspathrealpath 的顶层 digest;通过后把 exact state.source_root_digest 传给 _binding,所以两个合法 spelling 可以分别作为完整 journal identity,但不能在同一 journal 内混用。
  • TodoPartitionCapture.prepare:entry id 与 record 的 root digest 都来自 active binding;primary source write 仍只有在 outbox prepare 成功后继续,default-off 时保持 shadow_disabled
  • beginLeaseOutboxEntry:TypeScript lease writer删除 resolve(runtime_root) 的独立重算,复用 binding digest;lease partition 的其它投影、sequence 和 commit marker 语义未变。
  • _PartitionDrainer._partition_history:receipt 与当前 binding digest exact 比较;root alias 不再制造假 mismatch,真实 foreign/mixed receipt 仍被拒绝。

对主干的风险

最高风险是把“同一物理 root 的两种 spelling”误实现为“同一 journal 内两个 identity 字段可以不同”。该 exact head 已在 Python 回归中构造 lexical 顶层 + canonical binding,并验证 shadow_management_state_invalid;TypeScript decoder 的 exact equality 与之相同。正常 alias 路径则覆盖了 Python reader、Python Todo writer、TypeScript lease writer 和一次真实 drain。

验证结果:Python shadow-management/outbox suites 24 passed;TypeScript outbox suite 21/21 passed;control-plane typecheck 与 changed-file Ruff 均通过;真实 s2c2.outbox_prepared_then_committed_entries 行执行成功,Python/TypeScript 两个 partition 各交付一次;标准 premerge canary 13/13 通过,无 warning/manual hold。当前 head 相对最新主干无 changed-file 交集,merge-tree 无冲突。未重跑既有 ≥10-day sustained parity soak,这是原 promotion gate,不属于本 bug fix 的合入证据。

语义与 CI 对齐

这里没有创建新 vocabulary:它复用 source_root_digest 的既有含义,并让 Python reader/writers/drain 与 TypeScript owner 对齐。packet 明确 wait_for_ci=false,结论来自 exact-head 本地 filesystem、Node/Python 与真实 CLI 路径,不用远端 CI 替代证据。该变更触及 control-plane persisted-state 行为,因此即使验证全绿也不授予自合并权限。

我的整体评价

这是 #4892 的完整、最小且可回滚修复:入口、权威 identity、两个 effect writer、receipt consumer 和负向错误 owner 都闭合;相邻重复实现中的关键 invariant 风险也已通过 strict equality 消除。未发现 correctness、default-off、authority、typed-state 或 domain-neutrality blocker,我赞成在保持当前 exact head 的前提下合入,并由 maintainer 保留 merge authority。

English verdict: APPROVE - head 98c400a; lexical/canonical root aliases now share one strict state-to-binding lineage identity, including a mixed-digest fail-closed regression; 24 Python tests, 21 TypeScript tests, typecheck, the real Stage 2C2 row, and 13/13 premerge canaries passed.

@huangruiteng
huangruiteng merged commit 2e94e62 into loopx-project:main Sep 22, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants