Skip to content

fix: dedupe byte-identical durable event appends without consuming seq - #527

Merged
LeXwDeX merged 1 commit into
devfrom
fix/523-event-idempotency-gate
Sep 3, 2026
Merged

fix: dedupe byte-identical durable event appends without consuming seq#527
LeXwDeX merged 1 commit into
devfrom
fix/523-event-idempotency-gate

Conversation

@LeXwDeX

@LeXwDeX LeXwDeX commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Closes #523

Why

481e31e2b7 闸住 summary.diffs 之后,事件持久化浪费的主因是会话内逐字节重复写入packages/core/src/event.ts.insert(EventTable)(全仓唯一事件写入点)零重复检测。修复后数据实测(issue #523):message.updated.1 28,059 条 / 208.9 MiB 中 64.3%(134.3 MiB)是逐字节重复ses_fe5fecc1e45c 平均每条消息被重发 3.9 次全量快照,信息增量为零。

What changed

  • EventTable 新增 data_hash(nullable text,sha256 of 序列化 payload,append 时算一次落列,比对不重算 MiB 级 payload)。Legacy 行 hash 为 NULL 永不匹配 → 零回填ALTER TABLE event ADD data_hash text 是 O(1) 元数据操作,对生产 8.2 GiB event 表无启动负担
  • 幂等闸commitDurableEventInner fresh-append 路径):经 event_aggregate_type_seq_idx 定位同 aggregate_id+type 的上一条事件,data_hash 相同 → return undefined = 完全静默(语义 B,已确认):不占 seq、不跑 projectors、不调 commit hook、不 wake durable 流。publisher 收到 unstamped payload —— 与单 publish 既有 unstamped 分支一致;session/input.ts admit 已有 catchDefect → find(stored) 恢复路径,行为闭环
  • seq 保持连续致密 = 已持久化事件数。这正是必须选 B 的硬证据:replayAll(event.ts:574-585)断言重放序列连续(start+index),A 语义(占号留空洞)会让跨实例 sync 重放直接 die
  • replay(显式 input.seq)路径不受闸影响:跨实例同步的精确 seq 契约、divergence 检测原样保留
  • publishMany 对齐修复:results 改为与 entries 按位对齐(原 if (result) push + 按 index 配对在去重产生 undefined 时会给后续 payload 错标 seq);被去重 entry 仍 notify(unstamped),与单 publish 路径镜像
  • sync.ts history 显式列选择data_hash 不出机器,/sync/history 响应形状不变 → 无 SDK 再生成、无 httpapi-exercise 变更
  • Migration 20260903053128_add_event_data_hash + schema.gen.ts / migration.gen.ts / schema.json 再生成(--check 干净)

Evidence

新增 9 个单测(packages/core/test/event.test.ts)钉住验收与 B 语义:

  • 同 aggregate 连续 publish 相同事件 → 只新增一行,第二次返回 unstamped
  • 跳过不占 seq:dup 后下一事件 seq = prev+1(致密)
  • payload 不同 / 被不同事件隔开 → 正常追加;跨 type、跨 aggregate 不去重
  • publishMany 批内重复 → payload 对齐钉住([a→0, a→unstamped, b→1]
  • projectors / commit hook 只对落库 append 执行
  • NULL-hash legacy 行永不匹配;显式 seq replay 永不去重
  • durable 流读者只见持久化事件(gt(seq, after) 路径)

回归与门禁:

  • core event + event-batch 67 pass;core 回归 159 pass(control-plane/workspace、session-create/projector/prompt/runner/hotpath、database-migration、legacy-event-schema)
  • opencode 492 pass(httpapi-sync、workspace-proxy/routing、test/session/ 全目录)
  • bun run test:dag-core 60 pass + 覆盖率地板全过(DAG 持久化骑在事件层上)
  • bun typecheck(core + opencode)exit 0;root lint 4839 = dev 基线零新增
  • 三处读取路径核验:core readAftersync.ts replay、workspace.ts 均为显式字段映射,新列不外泄
  • 实测口径(issue 验收第 5 条)由单测等价覆盖:新增会话中 message.updated.1 distinct/total → 1(post-fix 基线 0.78);生产库实测需已装 binary 更新后另行验证,不在本 PR 范围

Checklist

  • Why, What changed, and Evidence are filled in.
  • specgit finish exits 0.

@LeXwDeX LeXwDeX changed the title Delivery event-idempotency-gate fix: dedupe byte-identical durable event appends without consuming seq Sep 3, 2026
@LeXwDeX
LeXwDeX changed the base branch from main to dev September 3, 2026 05:41
@LeXwDeX
LeXwDeX force-pushed the fix/523-event-idempotency-gate branch from c03ee70 to 772ea03 Compare September 3, 2026 06:24
@LeXwDeX
LeXwDeX marked this pull request as ready for review September 3, 2026 06:24
@LeXwDeX
LeXwDeX merged commit b111b2a into dev Sep 3, 2026
12 checks passed
@LeXwDeX
LeXwDeX deleted the fix/523-event-idempotency-gate branch September 3, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: event append has no idempotency guard so identical payloads are re-persisted

1 participant