Skip to content

perf(status): eliminate redundant registry reads in status hotpath - #4695

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
Duang777:codex/optimize-core-hotpath-10
Sep 18, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
Duang777:codex/optimize-core-hotpath-10

Conversation

@Duang777

Copy link
Copy Markdown
Contributor

Problem

The registry JSON file is loaded and parsed 5 times per single loopx status invocation:

  1. collect_status — loads registry (line 95)
  2. collect_history → loads registry again (history.py:317)
  3. check_contract → loads registry again (contract.py:939)
  4. build_promotion_gate → loads registry again (promotion_gate.py:147)
  5. collect_runtime_projection_route_diagnostics → loads registry again (runtime_projection_route.py:610)

For a registry with many goals, each load_registry reads and parses the full JSON file. These are pure overhead — the registry is already loaded by collect_status before any of these callers run.

Solution

Add an optional registry keyword parameter to each downstream function. When the parent (collect_status) already holds the loaded dict, pass it through instead of re-reading the file. The fallback pattern:

if registry is None:
    registry = load_registry(registry_path)

preserves full backward compatibility for all existing standalone callers.

Changed files

  • loopx/control_plane/status/collection.py (+4) — pass registry to 4 downstream callers
  • loopx/history.py (+5/-1) — collect_history and collect_status_history accept optional registry
  • loopx/contract.py (+4/-1) — check_contract accepts optional registry
  • loopx/promotion_gate.py (+4/-1) — build_promotion_gate accepts optional registry
  • loopx/control_plane/runtime/runtime_projection_route.py (+9/-2) — _source_routes_for_registry and collect_runtime_projection_route_diagnostics accept optional registry

Total: +22/−5 lines across 5 product files.

Validation

  • 80 focused tests: passed
  • 9/10 catalog canaries: passed (1 pre-existing TS parser failure)
  • 8/8 risk-profile smokes: passed
  • py_compile + maintainability ratchet: passed

Performance Impact

Removes up to 4 redundant file reads + JSON parses per status request. The improvement scales linearly with registry size: negligible for 1–2 goal projects, material for projects with 20+ registered goals.

Backward Compatibility

All new parameters are keyword-only with None defaults. Zero risk: every existing caller continues to work unchanged.

The registry JSON file is loaded and parsed 5 times per single status
request: once in collect_status, then independently by collect_history,
check_contract, build_promotion_gate, and
collect_runtime_projection_route_diagnostics — each of which calls
load_registry(registry_path) again.

Add an optional `registry` keyword parameter to each function. When the
caller (collect_status) already holds the loaded dict, pass it through
instead of re-reading and re-parsing the same file. The fallback
`if registry is None: registry = load_registry(registry_path)` preserves
full backward compatibility for all existing callers.

This removes up to 4 redundant file reads + JSON parses per `loopx
status` invocation. The change is pure machinery: no effect on status
output, schema, or observable behavior.

Signed-off-by: Duang <duang777@gmail.com>
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>

@Duang777 Duang777 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

动机

这个 PR 试图让一次 loopx status 请求复用入口处已经加载的 registry,避免下游重复读文件和解析 JSON。方向成立,但 exact head c0869ef504ee85af55bc453a1e939b79324d18e2 的真实入口证据与当前描述不一致:同一夹具下,base 657902286b01e7c81a5f4b13ff1b16a3a5dbb9c4 是 9 次读取,head 是 4 次读取,并非文案隐含的 5 次降到 1 次。

改动思路

由 status assembly 持有一次请求内的 registry 快照,再通过 keyword-only 参数传给 history、contract、promotion gate 和 runtime route,是合适的现有边界复用方式,不需要增加全局缓存。standalone caller 保留 registry is None 的文件读取回退也合理。

目前实现只完成了部分链路。historypromotion_gate 已复用快照,但 contract inspection 与当前 source route 仍读取同一个 registry 文件。

具体改动

关键代码讲解

  • control_plane.status.collection.collect_status 将入口加载的 registry 传给四个下游读取者。
  • history.collect_historypromotion_gate.build_promotion_gate 在传入 payload 时不再读取 registry。
  • contract.check_contract 虽然接收 payload,但在使用前仍调用 inspect_registry(registry_path)inspect_registry_boundary(registry_path),产生两次读取。
  • runtime_projection_route._source_routes_for_registry 对非 global registry 将 source 解析为当前 registry_path,随后仍通过 deadline worker 再次 load_registry,产生一次读取。

对主干的风险

  1. [P1] check_contract 未完整消费新增 snapshot 参数。 每次 status 仍对同一文件执行两次 inspection 读取。最低修复是让两个 inspection 支持已加载 payload,同时保留独立调用时的路径读取行为。
  2. [P1] current-source route 未复用新增 snapshot 参数。 当 source 与当前 registry 是同一路径时,应直接使用传入 payload;deadline worker 只保留给真正不同的 source registry。

现有测试验证了输出兼容性,但没有断言真实入口的 registry 打开次数,因此无法阻止这个不完整实现。建议在 tests/control_plane/test_status_history_reuse.py 增加 real-path 回归:普通非 global registry 的一次 collect_status 只打开当前 registry 一次,并另测 distinct source registry 仍走 deadline 读取。

验证结果:

  • 同一真实夹具:base registry_reads=9,head registry_reads=4,两者均 ok=truegoal_count=2
  • head 的四条读取路径已逐条捕获:入口 load、两个 contract inspection、一个 current-source deadline read。
  • GitHub 当前 18 项成功、8 项失败;根 TypeScript 与 mutation 失败也在 exact base 复现,因此不是这组 Python 改动引入,但 exact head 仍未满足 required-check gate。

我的整体评价

这是有实际价值的部分优化,读取次数降低了 55.6%,代码范围也合适;但 PR 标题、问题模型和完成性结论超过了 exact-head 证据,且缺少能约束目标的回归测试。请用追加 commit 补齐上述三次同文件读取或将交付明确收窄为经验证的 9 次降到 4 次,并同步修正文案与测试后再复审。

English verdict: REQUEST_CHANGES - exact head c0869ef reduces registry reads from 9 to 4, but three redundant same-file reads and the missing real-entry regression leave the stated optimization incomplete.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

评审 head:c0869ef504ee85af55bc453a1e939b79324d18e2(作者 Duang777;本 PR 上已有一条作者本人账号发出的 COMMENTED 评审,它不是维护者结论——按能力规则作者自有 COMMENTED 只有在 reviewer 与 PR 作者同一账号时才生效,这里正确地被判定为 invalid)。

动机

一次 loopx status 会在入口加载 registry,然后被下游各自重复加载:collect_statuscollect_historycheck_contractbuild_promotion_gatecollect_runtime_projection_route_diagnostics 各自调用 load_registry,而它是无缓存的"读文件 + 解析 JSON"。Goal 越多,重复成本越高。

方向与实现都成立,判定 goal_achieved。作者本人已在这条 PR 上指出描述与实测不一致("base 9 次、head 4 次,而不是 5 → 1"),我的独立测量与他的数字完全一致。

改动思路

不做缓存、不引入 context,只把"已经加载好的 registry"沿既有调用链传下去:每个消费函数新增一个仅关键字、默认 Noneregistry 参数,内部保持 if registry is None: registry = load_registry(registry_path)。这样一次请求只加载一次,而所有既有独立调用方(CLI 其它入口、测试、第三方调用)行为逐字不变。

这个取舍是对的:全局缓存 load_registry 也能省下读取,但会让跨请求的 registry 重写不可见;请求内传值是能同时保住新鲜度与正确性的最小机制。

具体改动

5 个文件、+22/-5,全部是既有函数上的可选参数与传参,没有新模块、新字段、新测试文件。

  • loopx/control_plane/status/collection.py(+4):把 registry 传给 history / contract / promotion gate / runtime route 四个消费者。
  • loopx/history.py(+6/-1):collect_historycollect_status_history 接受可选 registry。
  • loopx/contract.py(+4/-1)、loopx/promotion_gate.py(+4/-1):同名可选参数。
  • loopx/control_plane/runtime/runtime_projection_route.py(+9/-2):_source_routes_for_registrycollect_runtime_projection_route_diagnostics 接受可选 registry。

关键代码讲解

  1. collection.collect_status(第 119 行起)的四处传参:请求顶部已经拿到 registry,之后四个消费者收到同一实例;这是本 PR 的唯一"行为"变化——读取来源。
  2. contract.check_contract(第 940 行)的 None 回退:保持独立调用方语义;我用探针单独调用它(不传 registry)验证它仍自行加载并返回完整 payload。
  3. runtime_projection_route.collect_runtime_projection_route_diagnostics(第 604 行)与 _source_routes_for_registry:只复用"被调用的 registry";工作线程里对 source registry 的读取是另一个文件,保留其超时与失败诊断,这一点没有被误优化。

对主干的风险

我先独立测量再下判断:用包装过 load_registry 的探针在同一个 2-Goal 夹具上跑一次真实的 status.collect_status,base 657902286 是 9 次总加载(其中同一 fixture 路径 7 次),head 是 4 次(fixture 路径 2 次),payload 顶层键一致;跑完后 registry 文件字节不变,传入的 dict 与快照深度相等。剩余的 4 次里,2 次是另一路径的全局/source registry(设计如此)、1 次是入口加载,另有 2 次仍经 resolve_runtime_projection_route(该函数没有把 registry 传下来)——所以优化是部分完成而非全量,这属于可选的后续,不是缺陷。

真正的风险是"共享可变 dict":同一实例现在被四个消费者共用,如果将来有消费者就地修改它,就会把改动泄漏给下一个消费者,而当前测试并不会拦住(没有针对该 dict 不可变的断言)。我这一轮用运行前后深度比较做了替代验证,并把缺少该断言记为覆盖缺口。

兼容性:探针直接调用 check_contractcollect_history 而不传新参数,两者均正常返回并各自完成加载;本 head 上我跑了 68 个聚焦测试(status history reuse、contract health、status rollout snapshot、runtime summaries、agent lane projection、status server fast path、summary all)全部通过。

CI:本 head 的 8 个失败检查(kernel-static-checks、node-forward-compatibility、windows-powershell、stage2c (mutants 0) 及级联的 stage2c-correctness-e2e / pytest / checks / merge-gate)都不是本 PR 造成的——前三个来自 main 侧既有的 local_authority_provider.test.ts provider 故障断言(我在 origin/main 657902286 复现 27 pass / 8 fail),mutants 那条是 shared-goal-authority-e2e/mutants.py 的 locator drift;修这两个测试文件的提交目前在 #4688 上,本分支基于更早的 main。合并前需要在 main 侧修复落地后重跑,并让 check-merge-readiness 在同一 head 转 ready。

两条 P3(非阻塞):描述里的"加载 5 次 / 最多省 4 次"与实测(同路径 7 → 2、总 9 → 4)不一致,建议改成实测值与测量方法;resolve_runtime_projection_route 这条路径可以顺手把 registry 传下去,或明确写成后续。

我的整体评价

approve。这是小而有据的性能修复:机制落在既有的 registry 加载 owner 上,一次请求一次加载,独立调用方语义原样保留,source registry 的独立读取被正确保留;我用真实入口的插桩测量确认了收益(同路径 7 → 2),并用深度比较排除了本轮可见的共享可变 dict 泄漏。测试覆盖了受影响的 status/contract/history 面。

残余风险:共享 dict 的不可变约束缺少断言,建议补一条(对同一实例在请求前后做快照比较,或断言消费者不写它);真实 20+ Goal registry 的收益是外推;合并前需先解决 main 侧既有失败并重跑。

English verdict: APPROVE - 4695@c0869ef504ee85af55bc453a1e939b79324d18e2; the registry is now loaded once per status request instead of once per consumer (measured with an instrumented load_registry: fixture-path loads 7 -> 2, total 9 -> 4, identical payload keys, unchanged file and dict), standalone callers keep their fallback (verified), 68 focused tests pass, and the shared dict is not mutated during a request. Two non-blocking P3s: the description's 5-load / 4-removed numbers do not match the measurement, and resolve_runtime_projection_route could thread the registry to remove the last two in-request loads. The head's red checks are pre-existing main-side test drift (local_authority_provider assertions plus shared-goal-authority mutant locator drift), so merge waits for a re-run after that repair lands.

@huangruiteng
huangruiteng merged commit 0d90d6f into loopx-project:main Sep 18, 2026
18 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants