Add Graph Execution to host_build_graph by TaoZQY · Pull Request #1444 · hw-native-sys/simpler

TaoZQY · 2026-07-22T08:57:26Z

Summary

add Graph Execution to the a2a3 host_build_graph runtime as a composite incore task alongside AIC, AIV, MIX, and SPMD;
use the existing L0TaskArgs boundary directly; there are no public GraphArgs, GraphBindings, Patch, or ScalarRef types;
record and execute the first structurally compatible invocation, then submit one outer GRAPH task for each cache hit;
build and upload the complete host task image before launching the resident Scheduler; this PR does not add intra-run Host-Orchestrator/Scheduler concurrency;
allocate Graph execution storage from host-owned, Worker-accounted GM and reuse it across runs by pipeline slot, Graph key, and occurrence index;
keep host bookkeeping in C++ STL containers and compact it into exact-size, contiguous, pointer-free POD Definitions at the host-device boundary;
add AIC/AIV/MIX/SPMD coverage, DFX conversion tests, execution-storage/empty-boundary regressions, and a three-layer Qwen3-14B hardware integration case.

Public API

void qwen_decoder_layer(const L0TaskArgs &args) {
    const Tensor &hidden = args.tensor(0).ref();
    const Tensor &weight = args.tensor(1).ref();
    const Tensor &output = args.tensor(2).ref();

    L0TaskArgs task_args;
    task_args.add_input(hidden, weight);
    task_args.add_output(output);
    task_args.add_scalar(uint32_t{16});  // fixed Definition data
    rt_submit_aic_task(FUNC_MATMUL, task_args);
}

void submit_qwen_decoder_layer(const L0TaskArgs &args) {
    rt_submit_graph(&qwen_decoder_layer, args);
}

The function pointer is the default Graph identity. Optional trailing integral, floating-point, or boolean construction parameters are forwarded to the function and hashed by value into the Definition key.

Callers may use an explicit GRAPH_KEY, but that key must be unique for every distinct Graph function in an orchestration callable. The explicit-key overload deliberately excludes the Graph function pointer so the key remains stable.

Boundary Tensor addresses may change between invocations, while Tensor metadata and aliasing must match the recorded Definition. Scalars used by internal tasks are fixed Definition data. Dynamic scalars and empty boundaries are not cached and execute through the ordinary submit path.

See GRAPH_EXECUTION.md for the complete contract and Qwen wrapper.

Core execution flow

Host orchestration runs to completion against a host shared-memory/runtime-arena image.
A cache miss executes ordinary task submissions and records the fixed DAG topology. A cache hit reserves one outer task plus one heap extent and stages one exact-size Graph POD submission.
After orchestration, the host computes the exact execution-block size from the recorded node count and Definition bytes, acquires aligned persistent GM from the Worker's tracked MemoryAllocator, writes its fixed-width device address into the submission POD, and uploads the complete task image.
The resident Scheduler launches only after the complete image is visible. All AICPU threads classify disjoint slices of the completed task window behind one startup barrier.
Each outer Graph enters graph_prepare_queue during initial classification, while its external fanin follows the ordinary ready/wake path into graph_ready_queue.
Bounded preparation and external readiness meet at one atomic activation gate, which starts saved root nodes exactly once.
The Scheduler placement-constructs descriptors, payloads, and slot state in the host-owned GM block. The final internal node completes the one outer ring task and wakes external consumers.

Prepared-successor pipelining may overlap preparation of run N+1 with device execution of run N. This PR intentionally does not overlap orchestration and scheduling within one run.

The per-run Definition cache contains at most 16 actual-size STL-backed entries. Execution storage is retained per (pipeline slot, Graph key, occurrence index), grows only to observed demand, contributes to committed_device_memory(), and is released during Worker finalization. Distinct pipeline slots never share an active block, and repeated uses of one key within a run receive distinct blocks. There is no device-side malloc/posix_memalign, persistent cache schema, or PTO2_GRAPH_CACHE_SCHEMA_VERSION.

Scope

Graph Execution behavior is confined to a2a3 host_build_graph;
the common HostApi/runner gains only the tracked retained-buffer capability; other runtimes do not use it and keep their existing behavior;
Graph DFX adds the Graph Execution envelope and bounded graph_prepare scheduler phases without a cross-clock Host-Orchestrator lane;
the branch is one commit on top of current upstream/main.

Validation

editable full-runtime build after rebasing current main;
all changed-file pre-commit hooks, including clang-format, clang-tidy, cpplint, and markdownlint;
C++ no-hardware tests: 80/80 passed, including empty-boundary and execution-storage layout regressions;
full a2a3 host_build_graph simulation: 27 passed, 4 skipped, 185 deselected;
final Graph Execution simulation after the latest rebase: 3 passed, 1 manual hardware case deselected;
local a2a3 hardware run is unavailable because npu-smi cannot return Chip/NPU Name; PR onboard CI is the authoritative hardware validation.

gemini-code-assist · 2026-07-22T08:57:29Z

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

coderabbitai · 2026-07-22T08:57:45Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 67d056a8-62cb-4d7c-8b88-c51c6ff722cd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds host-side graph capture and replay for host_build_graph, including cache-key generation, graph execution storage, device upload, graph-aware scheduling, scalar provenance, documentation, and an end-to-end record/replay test.

Changes

Graph Execution Runtime

Layer / File(s)	Summary
Graph contracts and submission API `src/a2a3/runtime/host_build_graph/runtime/pto_graph_cache.h`, `src/a2a3/runtime/host_build_graph/runtime/pto_types.h`, `src/a2a3/runtime/host_build_graph/orchestration/pto_orchestration_api.h`, `src/a2a3/runtime/host_build_graph/runtime/pto_runtime2*.h`, `src/a2a3/runtime/host_build_graph/runtime/pto_orchestrator.h`	Adds cache keys, cacheability checks, scalar provenance, graph task metadata, runtime graph callbacks, and `rt_submit_graph` wrappers.
Graph capture and replay construction `src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp`	Records task topology and tensor/scalar sources, builds cached definitions, submits replay executions, and materializes graph nodes.
Execution storage and device upload `src/a2a3/runtime/host_build_graph/runtime/pto_graph_execution.*`, `src/a2a3/runtime/host_build_graph/host/runtime_maker.cpp`	Adds pooled execution images, lifecycle state transitions, pointer relocation, pending graph uploads, and device buffer ownership.
Graph scheduling and completion `src/a2a3/runtime/host_build_graph/runtime/scheduler/*`, `src/a2a3/runtime/host_build_graph/runtime/shared/pto_runtime2_init.cpp`	Adds graph queues, sliced preparation and activation, graph-node fanout handling, and stream-aware completion accounting.
Graph execution test and documentation `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, `tests/st/a2a3/host_build_graph/graph_execution/*`	Documents graph capture/replay semantics and adds an end-to-end orchestration test with dynamic and fixed scalars.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

hw-native-sys/simpler#1379: Shares host graph runtime operations and scheduler publication paths.
hw-native-sys/simpler#1415: Overlaps the scheduler task-state and synchronization paths.
hw-native-sys/simpler#1435: Overlaps scheduler completion and dispatch accounting.

Poem

A bunny hops through cached graphs,
With tensor trails and scalar tags.
Roots wake up, then nodes race,
Uploading dreams to device space.
The DAG remembers every way—
And replays hops another day.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.47% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding Graph Execution functionality to the host_build_graph runtime.
Description check	✅ Passed	The description is comprehensive and directly related to the changeset, covering the feature implementation, public API, core execution flow, and validation results.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which hw-native-sys#1494 added by accident: it is documentation from the open hw-native-sys#1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state hw-native-sys#1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. - It asserts the parent died from its own SIGKILL. A parent that fails before forking otherwise surfaces as an unexplained "expected 3 pids, got []", which reads like a defect in the code under test rather than an environment problem in the harness. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which hw-native-sys#1494 added by accident: it is documentation from the open hw-native-sys#1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state hw-native-sys#1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A forked sub/child/chip worker never checked whether its parent was still alive. When the parent died without `Worker.close()` — SIGKILL from a `timeout`, an OOM kill, a cancelled CI job, an editor killing a wedged pytest — every child was reparented and kept polling a mailbox nobody would ever write to. Because that poll is an unbounded busy-wait, each survivor held a full core indefinitely. This is not theoretical. On a shared dev box six such processes belonging to two different users had been alive 5-7 days each at ~99.6% CPU, six cores permanently consumed. Four of the six carried `tests/ut/py/test_worker/test_startup_readiness.py` on their command line: runs whose parent was killed rather than allowed to finish. `_run_mailbox_loop` now samples `getppid()` on its idle path and leaves by the same SHUTDOWN route once it changes, so a child tears down exactly as it would on a clean shutdown. Compare against the pid captured at loop entry rather than testing for pid 1: a subreaper (container init, a systemd user session) adopts orphans instead of init, so the pid changes but never becomes 1. A live parent's pid cannot change, so the check cannot fire spuriously. Sampling every `_PARENT_LIVENESS_POLL_INTERVAL` idle polls puts a `getppid()` roughly every 100 us against a ~0.1 us poll — fast enough that an orphan goes away before it is noticeable, cheap enough to be lost in the noise of the poll itself. Residual gap: a parent that dies between `os.fork()` and the child reaching the loop is already gone when the pid is captured, so that child is not detected. Closing it needs the pid handed in from before the fork, or `PR_SET_PDEATHSIG`, which is Linux-only and therefore an optimisation on top of this rather than the mechanism. `test_orphan_child_reaping.py` SIGKILLs a subprocess parent and asserts every child is gone within 20 s. Three details are load-bearing: - It reads the pids from the Worker's own `_sub_pids` rather than `pgrep -P`. Enumerating children externally also catches the transient shell that runs pgrep, and inside a container's shallow pid namespace it picks up unrelated low pids: on CI that produced "1 of 6 children outlived ... [3]", waiting on and then SIGKILLing a container process that was never ours. - It writes to a file and never reads a pipe. Orphans inherit the parent's stdout, so on regression they hold a pipe's write end open and reading one would hang the test for as long as the bug survives instead of failing it. - Its `finally` kills whatever survived, so a regression does not hand spinning processes to the next test. - It asserts the parent died from its own SIGKILL. A parent that fails before forking otherwise surfaces as an unexplained "expected 3 pids, got []", which reads like a defect in the code under test rather than an environment problem in the harness. It runs on macOS as well as Linux: reparenting on parent death is POSIX behaviour, and nothing here is platform-specific once the pid list comes from the Worker. Also drop `src/a2a3/runtime/host_build_graph/docs/GRAPH_EXECUTION.md`, which #1494 added by accident: it is documentation from the open #1444 (`feat/graph-execution`) that happened to be sitting untracked in the working tree, and a `git add -A` swept it into that commit. Removing it restores main to the state #1444 expects to merge into. It is unrelated to everything else here and can be reviewed independently of the fix

ChaoZheng109 · 2026-08-04T04:37:53Z

总体: Graph Execution 的核心实现质量很高 —— POD/wire 纪律严谨、wire 格式校验完整、poll/wake-list 的图节点调度器经并发追踪确认正确、scene 测试用 golden 真正验证了重放正确性(不是"不崩就过")。我的顾虑在架构层面:这个 PR 捆绑了一个可分离的 intra-run 流式优化,它带来实打实的复杂度和一类并发 bug,而它的收益系统里已有其他机制在提供。建议 Request changes。

1. 取消 intra-run 的 O/S 流式(核心请求)。

现在的做法是:设备对空 SM 镜像先启动,host 编排(O)在一个并发线程上跑,每提交一次(每个 rt_submit_graph 一个 commit)就通过 release 写 current_task_index 增量发布一个任务前缀;设备 Scheduler(S)在 host 继续编排的同时消费这些前缀,orchestrator_done 作为流结束(EOS)标志。目的是把 host 编排延迟藏到设备执行背后(描述里那 ~260 µs 的重叠)。

问题是这个重叠已经由两个现成机制提供了,流式是重复的第三套:

run 间流水。 native-run 路径本就把 run N+1 的准备/编排与 run N 的设备执行并发(allow_prepared_successor / overlaps_active_run / "concurrent HBG preparation")。由于设备每个 context 一次只服务一个 run,轮到 run N+1 上设备前,它的编排早已藏在 run N 的执行背后。稳态 decode 下 O 本就不在关键路径上。
graph 缓存本身。 cache hit 把一整层的编排塌缩成一个 GRAPH task,即 O ≪ S。压缩 O 正是这个特性的真正价值;剩下那一点再由 run 流水盖掉。

而代价是具体的:

它引入了一个消息传递的内存序缺陷(scheduler_cold_path.cpp:77-80)。host 是先发数据、后发标志(先推进 current_task_index,之后才置 orchestrator_done=1);正确的消费者必须先读标志、再读数据。但完成判定里把 current_task_index(数据)读在了 orchestrator_done(标志)之前,于是一个线程可能同时观察到 orchestrator_done==1 和一个偏小的陈旧 published 值,满足完成条件、在后续任务还没跑完时就关停 run —— 静默截断/挂死。触发概率低(host 在两次设备可见写之间还有大量工作),但这是真实的序违规,而串行模型里根本不存在——那时总数在启动前就由 total_tasks_ 已知。
空镜像启动、增量上传路径、swimlane_converter.py 里的跨时钟锚点,都是大片新增面。
流式发布只在 rt_submit_graph 的 commit 边界触发,所以纯非-graph 的 hbg run 根本吃不到流式——即便在适用处,收益也很窄。

建议:GRAPH 作为与 AIC/AIV/MIX 同级的加法型 task kind,O→S 在调度层面串行——把完整任务窗口(含 GRAPH task)建好、上传、启动、再调度。隐藏 O 交给 run 流水 + graph 缓存。同步更新本 PR 的 GRAPH_EXECUTION.md。poll/wake-list 的图节点调度器独立于流式,保持不变。取消流式还会直接删掉那个内存序缺陷。

2. device launch 与 O 并行的优化,请另起独立 PR。

取消流式后,device launch 回到常规的串行位置(O 之后、S 之前)。如果仍想隐藏 per-run 的 device launch 延迟,干净的做法与 graph 无关:让 launch 与 host 编排并发(两者本就在不同线程),再让 Scheduler 等一个单一的"编排完成"握手后再调度完整镜像。这能把 launch 延迟藏起来(对冷启动/第一个 run——run 流水唯一盖不到的场景),代价是设备短暂空等,而一个启动等待即可干净处理(启动握手豁免于 no-sleep-on-dispatch 规则)。这是一个聚焦、可分离的优化,不应搭在一个已 3800+ 行的 graph PR 里,单独评审才能审清它自己的并发。

3. 把 PTO2TaskKind 改名为 TaskKind(pto_runtime2_types.h:22)。

这是一个全新的 enum class PTO2TaskKind(merge base 处不存在)。codestyle §9 无条件禁止新增 PTO2 前缀标识符,本子系统自身的命名规则也要求不带 PTO2 前缀。PR 里其余所有 PTO2* 都是复用既有类型(允许),这是唯一的新违规项。改名为 TaskKind(需消歧就放进 namespace),并按 codestyle §10 在同一 commit 内改完定义与全部约 17 处引用,使两种拼写不共存。

4. 三处小修。

(a) WAKE_LIST_SENTINEL 注释(graph_execution.cpp:288-295)。 注释称"观察到 sentinel 即可见生产者的 release task_state"。并非如此——wake_list_head 的载入都是 relaxed。正确性实际来自重试循环重扫 fanin CSR、以及对 task_state 的 acquire 载入按一致性收敛。现措辞可能误导后人删掉重试循环(信了所谓的 release/acquire),那样会真的破坏它。改述为"retry + coherence"。
(b) 显式 GRAPH_KEY 的唯一性。 显式 key 重载把 name + config 折进 key,但有意排除函数指针(好让稳定名跨 build 存活)。这本身是设计使然,但意味着:两个不同的 graph 函数,若用同一 GRAPH_KEY 且边界签名相同,会缓存命中并重放第一个函数的拓扑——静默出错。请在文档显著位置写明契约:显式 GRAPH_KEY 必须每个 graph 函数唯一。
(c) 零边界张量的 graph。 tensor_count()==0 的 graph 能通过 rt_graph_args_cacheable,但 graph_build_definition 把空的边界签名段(offset 0)当作缺失的必需段而失败,永久回退并触发误导性的 debug_assert(false)。建议在 rt_graph_args_cacheable 更早拒绝,或允许空边界段。

5. 合入前 rebase —— 分支落后 upstream/main 4 个 commit。

备注:content_hash 内容校验、alias 不匹配回退普通路径、无持久化缓存 schema —— 这些实现是对的/合理的,无需改代码。

TaoZQY · 2026-08-04T07:28:17Z

@ChaoZheng109 已按意见更新，当前 head 为 73da979f：

已移除 run 内 Host Orchestrator / Scheduler 流式并发和 device-launch-vs-O 并发；恢复为 host 构建完整 task image、统一上传 Graph POD/SM/runtime arena 后再启动 Scheduler。graph_commit 不再产生设备侧发布动作。
保留 Graph polling/wake、graph_prepare_queue 与 graph_ready_queue；完整 task window 在 AICPU 启动屏障内并行初始分类，Graph 与普通任务同层调度。
新类型已从 PTO2TaskKind 全量重命名为 TaskKind。
已修正 WAKE_LIST_SENTINEL 的内存序说明：正确性依赖重试和 acquire 读取 task state，不再声称 relaxed sentinel load 提供 release 可见性。
文档明确：显式 GRAPH_KEY 不包含 Graph function pointer，因此每个不同 Graph function 必须使用唯一 key。
空 boundary 在 cache 入口提前拒绝，并新增 GraphCache.RejectsEmptyBoundary 回归测试。
已移除跨时钟 Host-Orchestrator DFX 与 HostApi::publish_i32 等非 Graph 核心改动；无 persistent cache schema/version。
已 rebase 最新 upstream/main，仍保持单 commit。

本地验证：PR 变更文件 clang-format/clang-tidy 通过；77 个 C++ 无硬件测试通过；完整 a2a3 host_build_graph sim 为 20 passed / 4 skipped / 6 deselected；最终 rebase 后 Graph + converter 为 25 passed / 1 deselected。当前机器 DCMI 无法识别芯片，Qwen 真机项交由 PR onboard CI 验证。

ChaoZheng109 · 2026-08-04T11:43:56Z

补充:Graph 执行池在设备侧分配,游离于 host 的设备内存记账之外

graph 执行池(graph_execution.cpp)在设备侧用 posix_memalign 分配每个 GraphExecution 块(约 L330),仅由设备侧常量(GRAPH_EXECUTION_POOL_MAX_BYTES = 16 MiB / GRAPH_EXECUTION_POOL_MAX_BLOCKS = 64)封顶。这块内存来自 aicpu-sd 进程堆,而它是 HBM ——和 host 分配 GM 用的是同一块物理设备内存——只是经由 libc、而非 host 的分配器拿到。(这里 posix_memalign 相对 malloc 纯粹是为了 alignas(64) 的 GraphNodeStorage 做对齐,并不改变内存来源。)

为什么这是个问题:

未被记账的设备内存。 最多 16 MiB 的 HBM 被占用,却不出现在 host 的设备内存记账里,同时又在争用 host 自以为在管理的同一块物理 HBM。对共享机器 / OOM 预测来说这是一个盲区——用量只由一个内部常量约束,而不受 host 内存预算约束。
一个隐式的 AICore 可见性依赖。 在 ready 派发路径(scheduler_dispatch.cpp 约 L147),调度器把指向这些块的裸指针交给 AICore(args[n] = &payload.tensors[i]),因此 AICore 必须能读到它们。a2a3 硬件测试通过说明这块堆目前对 AICore 可见,但该设计是隐式依赖这一点,而非加以保证。

建议方向: 编排(orch)在 host 上运行,且已经记录了每个 graph 的节点数——所以 host 天生知道所需块的精确大小。因此这个池应当在 orch 时由 host 侧分配、并纳入 host 的设备内存预算,而不是放在一个 host 看不到的设备侧堆上。为了保持"按需"而非"按最坏情况固定预留",可采用一个按 graph 身份(key)复用的持久池——跨 run 复用、在 worker 销毁时释放——它只增长到工作负载的实际需求,同时全程被记账。由于它是 host 拥有的 GM,也天生对 AICore 可见,从而消除上面那个隐式依赖。

TaoZQY · 2026-08-04T13:29:05Z

@ChaoZheng109 已按补充意见修改：

删除了 AICPU 侧 posix_memalign、16 MiB/64-block 上限和设备侧全局 execution pool。
Host 根据 Definition 的节点数和实际字节数计算精确执行块大小，通过新增的 HostApi::acquire_graph_execution_buffer 从 Worker 的 MemoryAllocator 分配 GM。
持久块按 (pipeline slot, Graph key, occurrence index) 复用：同 key 在一个 run 内多次出现会使用不同块，run N/N+1 的两个 slot 也不会共享活跃块；容量不足时只扩容对应项。
分配现在计入 committed_device_memory()，并在 sim/onboard Worker finalize、设备上下文仍有效时统一释放。
GraphSubmission POD 显式携带对齐后的 device address 和 usable capacity；Scheduler 校验地址、对齐、容量和上一轮完成/退休状态后原地构造或复用，因此 AICore 可见性由 host rtMalloc/GM 分配路径保证。
文档与 PR 描述已同步；新增 execution-storage 布局/溢出回归。验证通过：80/80 C++ 无硬件测试、完整 a2a3 host_build_graph sim（27 passed / 4 skipped），以及最终 Graph sim（3 passed / 1 manual deselected）。

ChaoZheng109 · 2026-08-05T01:42:56Z

补充:文档描述的 graph-affine 重放快路径未实现 —— 每次重放都在 AICPU 上全量 re-materialize

GRAPH_EXECUTION.md 描述了重放的 graph-affine 快路径:当复用到之前被同一 graph key 用过的块时,"everything static is skipped,只刷新 task id、packed-buffer 指针、refcount、task state、边界来源的 tensor 地址、payload 原子量"(另一处也写作 "keeps the local definition and static node fields and refreshes only task IDs, tensor addresses, packed-buffer bases, and scheduling state")。

实现并没有这么做。在 graph_execution.cpp 里,definition_affine_reuse 只管两件事:跳过 graph_execution_localize 中的 Definition memcpy(约 L403),以及跳过 materialize_slice 中 GraphNodeStorage 的 placement new(约 L490)。materialize_slice 里的逐节点字段填充每次重放都无条件执行。逐字段对照如下:

#	字段 / 步骤	当前(每次重放)	Level 1 目标(affine 命中)	归类
1	`task.task_id`	重建	刷新(新合成 id)	动态
2	`task.kernel_id[]`	重建(逐 slot 拷)	跳过	静态
3	`task.packed_buffer_base/end`	重建	刷新(新 outer_base + offset)	动态
4	`slot.reset_for_reuse()`	执行	执行(调度态重置)	动态
5	`slot.bind_buffers(&payload,&task)`	执行	跳过(同块,指针不变)	静态
6	`slot.task_state`	重置	刷新(→ PENDING)	动态
7	`slot.active_mask`	重建	跳过	静态
8	`slot.task_attrs`	重建	跳过	静态
9	`slot.total_required_subtasks`	重建	跳过	静态
10	`slot.logical_block_num`	重建	跳过	静态
11	`slot.graph_node_index`	重建	跳过	静态
12	`slot.task_kind`	重建	跳过	静态
13	`slot.graph_context`	重建	跳过(同实例)	静态
14	`payload.tensor_count`	重建	跳过	静态
15	`payload.scalar_count`	重建	跳过	静态
16	per-node count/offset 边界校验	重跑	跳过(首次已验)	静态
17	tensor 循环:读来源配方 + BOUNDARY/INTERNAL 分类	重跑	跳过(改用补丁表)	静态
18	tensor 循环:`graph_tensor_wire_valid`(×2)	重跑	跳过	静态
19	tensor 地址写入 —— 边界来源	经全量分类得出	刷新:从 `boundary_tensors` 按补丁表覆盖 `buffer.addr`(VIEW 加 offset)	动态
20	tensor 地址写入 —— 内部来源	经全量分类得出	刷新:`new_outer_base + offset`(补丁表,一次加法)	动态
21	scalar `memcpy`	重拷	跳过	静态
22	`reset_graph_payload`(派发原子量)	执行	执行	动态
23	`register_initial_graph_waiter`(wake-list)	执行	执行(重注册)	动态

差异集中在第 2、5、7–18、21 行(共 15 项静态):当前每次重放全量重建 / 重跑,而按文档应全部跳过。其中第 17/18 的逐 tensor 来源分类 + 双重 graph_tensor_wire_valid 校验是主要开销。

影响:每次重放都在 AICPU 调度器上做一次全量节点 re-materialize,而不是文档设想的"复用除动态地址外的一切"。由于 materialize 跑在 AICPU(常常是调度瓶颈)上,这等于每次重放都加重瓶颈处理器,削弱了 GRAPH 这个任务类型本应带来的重放复用价值。同时这也是一处文档与代码不一致。

方向:实现文档描述的快路径 —— affine 命中时,跳过静态的每节点字段和来源分类/校验(在首次 materialize 时预计算一张边界/内部地址补丁表),只刷新动态部分(task id、packed-buffer 基址、边界/内部 tensor 地址、调度态)。这还要求 affine 复用对重复的层确定命中,而目前它取决于 recycle 池的时序。

TaoZQY · 2026-08-05T07:21:13Z

@ChaoZheng109 已按补充意见实现 graph-affine 重放快路径：首次 materialize 在 host-owned 连续执行块中生成 POD tensor 地址补丁表；同 graph key/content hash/node+patch count 的块再次命中时，保留 Definition、拓扑绑定和所有静态 node/payload/slot 字段，跳过 count/offset 校验、tensor source 分类、两次 wire 校验及 scalar memcpy，只刷新 task id、packed-buffer base/end、边界/内部 tensor 地址、调度状态、dispatch 原子量和 wake 注册。内部 tensor 地址现在直接按 new_outer_base + 预计算偏移更新。host 复用仍由 (pipeline slot, graph key, occurrence) 直接索引，occurrence 每轮按提交顺序确定，不依赖 recycler 时序。新增回归测试会用写探针确认静态字段未被覆盖，并覆盖 boundary/internal 地址刷新。验证：pre-commit 全通过；C++ no-hardware 82/82；a2a3sim host_build_graph 27 passed/4 skipped；a2a3 onboard Graph 3/3；Qwen3-14B 三层 onboard 1/1。

- Accept CoreTaskArgs and cache fixed-shape AIC/AIV/MIX/SPMD DAGs - Build and upload the complete host graph before Scheduler launch - Retain Graph blocks in host-owned, worker-accounted GM pools - Reuse POD address patches and refresh only dynamic replay state - Schedule Graph preparation beside external fanin readiness - Validate fixed boundary contracts and reject empty boundaries - Cover Graph variants, DFX, and a three-layer Qwen3-14B workload Co-authored-by: Crane-Liu <c.wliu@outlook.com>

HBG's per-dispatch wall is 96-99.6% host bind, and bind was dominated by rebuilding and H2D-uploading full, ring-sized host structures every run -- the shared-memory mirror and the ~20 MB prebuilt runtime arena -- even though a run touches a tiny fraction. The device boots scheduler-only and reads no SM slot past total_tasks, so the SM mirror is made init-on-write and shipped bounded to the task count, and the arena's host-only orchestrator block is dropped from the upload. Shared memory: - descriptors and payloads are written per task at submit; slot_states and completion_flags are reset per slot in orch::prepare_task as it is claimed, dropping the window-wide reset loop in init_header_per_ring; only the header is zeroed on the host; each segment is H2D-uploaded bounded to [0, total_tasks). Runtime arena: - skip uploading the orchestrator block (fanin_seen_epoch / scope / tensormap, ~8.5 MB): host-only dep-computation scratch the AICPU scheduler never reads. The scheduler block onward (ready queues, runtime header, mailbox) still ships whole. - drop the redundant per-entry stores in the tensormap reset (the preceding memset already zeroes the link pointers and producer_task_id). The ready queues are shipped in full, not bounded to total_tasks: graph execution (hw-native-sys#1444) replays a cached GRAPH task that the device Scheduler expands into on-device nodes, and those nodes push into the ready queues past the host task count, so every queue slot must carry a valid Vyukov sequence on the device. Every scheduler-read field is initialized at submit, so nothing depends on the removed blanket zeros; reads are bounded by current_task_index. total_tasks is range-checked before it sizes the SM copies. Single-ring (PTO2_MAX_RING_DEPTH == 1). Guardrails and tests: - push_ready_routed latches PTO2_ERROR_READY_QUEUE_OVERFLOW when a push finds no free slot (a genuinely full queue, or a task window / graph expansion past ready-queue capacity), turning a would-be silent drop and forward-progress stall into a named error. - bind_callable_to_runtime_impl always_asserts orch_start <= orch_end before slicing the orchestrator block out of the upload. - test_hbg_submit_poison fills the SM window with 0xAA, submits a representative mix (real mixed task with tensors + scalar, multi-fanin consumer, hidden-alloc, dummy) and asserts every device-read slot field is written, not left poison -- pinning the init-on-write contract so a future unwritten device-read field fails in-tree rather than non-deterministically on device. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MqeALZTPEnDXTnbYfcnPfq

…1659) HBG's per-dispatch wall is 96-99.6% host bind, and bind was dominated by rebuilding and H2D-uploading full, ring-sized host structures every run -- the shared-memory mirror and the ~20 MB prebuilt runtime arena -- even though a run touches a tiny fraction. The device boots scheduler-only and reads no SM slot past total_tasks, so the SM mirror is made init-on-write and shipped bounded to the task count, and the arena's host-only orchestrator block is dropped from the upload. Shared memory: - descriptors and payloads are written per task at submit; slot_states and completion_flags are reset per slot in orch::prepare_task as it is claimed, dropping the window-wide reset loop in init_header_per_ring; only the header is zeroed on the host; each segment is H2D-uploaded bounded to [0, total_tasks). Runtime arena: - skip uploading the orchestrator block (fanin_seen_epoch / scope / tensormap, ~8.5 MB): host-only dep-computation scratch the AICPU scheduler never reads. The scheduler block onward (ready queues, runtime header, mailbox) still ships whole. - drop the redundant per-entry stores in the tensormap reset (the preceding memset already zeroes the link pointers and producer_task_id). The ready queues are shipped in full, not bounded to total_tasks: graph execution (#1444) replays a cached GRAPH task that the device Scheduler expands into on-device nodes, and those nodes push into the ready queues past the host task count, so every queue slot must carry a valid Vyukov sequence on the device. Every scheduler-read field is initialized at submit, so nothing depends on the removed blanket zeros; reads are bounded by current_task_index. total_tasks is range-checked before it sizes the SM copies. Single-ring (PTO2_MAX_RING_DEPTH == 1). Guardrails and tests: - push_ready_routed latches PTO2_ERROR_READY_QUEUE_OVERFLOW when a push finds no free slot (a genuinely full queue, or a task window / graph expansion past ready-queue capacity), turning a would-be silent drop and forward-progress stall into a named error. - bind_callable_to_runtime_impl always_asserts orch_start <= orch_end before slicing the orchestrator block out of the upload. - test_hbg_submit_poison fills the SM window with 0xAA, submits a representative mix (real mixed task with tensors + scalar, multi-fanin consumer, hidden-alloc, dummy) and asserts every device-read slot field is written, not left poison -- pinning the init-on-write contract so a future unwritten device-read field fails in-tree rather than non-deterministically on device.

TaoZQY force-pushed the feat/graph-execution branch 2 times, most recently from fd8fc05 to 66f183e Compare July 22, 2026 12:36

ChaoWao mentioned this pull request Jul 26, 2026

Fix: exit a forked worker child once its parent is gone #1495

Merged

coderabbitai Bot mentioned this pull request Jul 28, 2026

host_build_graph: dedicated resolution thread (3S+1P) #1544

Merged

TaoZQY force-pushed the feat/graph-execution branch 2 times, most recently from 3a1347f to f1fbc59 Compare August 3, 2026 09:22

TaoZQY changed the title ~~Add Graph Execution to host_build_graph~~ Add streamed Graph Execution to host_build_graph Aug 3, 2026

coderabbitai Bot mentioned this pull request Aug 3, 2026

Fix: align hbg docs/comments with the polling runtime model #1655

Merged

3 tasks

SergioMartin86 mentioned this pull request Aug 3, 2026

[Performance] HBG: up to 99% host-side overhead reduction #1659

Merged

ChaoZheng109 mentioned this pull request Aug 3, 2026

Refactor: host build graph, eager completion watermark, 4 AICPU schedulers #1618

Closed

TaoZQY force-pushed the feat/graph-execution branch from f1fbc59 to 70cf581 Compare August 4, 2026 01:48

coderabbitai Bot mentioned this pull request Aug 4, 2026

Fix host-build-graph validation and teardown defects #1674

Merged

TaoZQY force-pushed the feat/graph-execution branch from 70cf581 to 73da979 Compare August 4, 2026 07:23

TaoZQY changed the title ~~Add streamed Graph Execution to host_build_graph~~ Add Graph Execution to host_build_graph Aug 4, 2026

coderabbitai Bot mentioned this pull request Aug 4, 2026

Fix: guard tensor producer resolution on the returned slot pointer #1686

Merged

TaoZQY force-pushed the feat/graph-execution branch from 73da979 to 1fcca25 Compare August 4, 2026 13:27

TaoZQY force-pushed the feat/graph-execution branch from 1fcca25 to afa26fb Compare August 4, 2026 16:42

TaoZQY force-pushed the feat/graph-execution branch from afa26fb to 742774c Compare August 5, 2026 07:21

TaoZQY force-pushed the feat/graph-execution branch from 742774c to 015a88c Compare August 5, 2026 08:07

ChaoZheng109 approved these changes Aug 5, 2026

View reviewed changes

ChaoZheng109 merged commit 0659745 into hw-native-sys:main Aug 5, 2026
18 checks passed

ChaoWao mentioned this pull request Aug 5, 2026

Refactor: give each run its own host tensor accessor #1695

Merged

Navigation Menu

Uh oh!

Add Graph Execution to host_build_graph - #1444

Add Graph Execution to host_build_graph#1444
ChaoZheng109 merged 1 commit into
hw-native-sys:mainfrom
Crane-Liu:feat/graph-execution

TaoZQY commented Jul 22, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Jul 22, 2026

Uh oh!

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading

Review skipped

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

ChaoZheng109 commented Aug 4, 2026 •

edited

Loading

Uh oh!

TaoZQY commented Aug 4, 2026

Uh oh!

ChaoZheng109 commented Aug 4, 2026

Uh oh!

TaoZQY commented Aug 4, 2026

Uh oh!

ChaoZheng109 commented Aug 5, 2026

Uh oh!

TaoZQY commented Aug 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

TaoZQY commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Public API

Core execution flow

Scope

Validation

Uh oh!

gemini-code-assist Bot commented Jul 22, 2026

Uh oh!

coderabbitai Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

ChaoZheng109 commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TaoZQY commented Aug 4, 2026

Uh oh!

ChaoZheng109 commented Aug 4, 2026

Uh oh!

TaoZQY commented Aug 4, 2026

Uh oh!

ChaoZheng109 commented Aug 5, 2026

Uh oh!

TaoZQY commented Aug 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

TaoZQY commented Jul 22, 2026 •

edited

Loading

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading

ChaoZheng109 commented Aug 4, 2026 •

edited

Loading