issue/1565 fix(runtime): support Qwen MTP and recurrent graph replay

[big-hip]

Problem and result

Closes #1565. Target: InfiniLM-v0.2.9c.

Qwen greedy MTP needs E4M3/BOOL conversions, correct TP communicator teardown, head-256 paged Prefill, and reliable graph storage/replay. This PR supplies the complete shared-runtime prerequisites for InfiniTensor/InfiniLM#584. Its graph fixes are also used by Mamba-2 in InfiniTensor/InfiniLM#575.

Implementation

• Reuse ATen scalar mappings for F8/BOOL; no new conversion kernel.
• Preserve the caller's CUDA device across ncclCommDestroy.
• Instantiate the existing NVIDIA paged Prefill warp kernel for head size 256; other vendor/MLA dispatch defaults are unchanged.
• Keep reused captured allocator blocks pinned and protect reactivated graph storage from trimming.
• Record Sum/scalar-power and MetaX last-axis concatenation for replay; include reduction axes in descriptor-cache identity and use capture-safe output zeroing.
• Retain focused cast, Prefill, allocator, changed-input reduction/power and strided-concatenation regressions.

The implementation is organized into two reviewable commits: operator/communication support, then graph prerequisites. The latter consolidates #1560 without changing its code or tests. No Mamba scan, Prefill compiler, model weights or experiment artifacts are included. Native W8A8 is not introduced.

Validation

Current-head local verification (2026-09-23): rebuilt and installed a3ac4df4 for NVIDIA SM86 with graph support on the A6000 server. 13 graph-replay/FP8/BOOL checks and 88 paged Prefill cases passed. Paired with InfiniLM #584 31ef4290, the small hybrid fixture passes TP1 (66 passed, one TP2-only skip) and TP2 (67 passed); LM totals include CPU checks. A stale September 19 installed library initially caused a graph allocator abort; the same reproducer passes after installing this PR's current library, without source/test changes. Current logs, hashes, conditions and commands. These shared-GPU correctness checks do not rerun the real 27B model or throughput benchmarks.

Earlier platform evidence:

Consolidation audit and new focused logs; A6000 evidence; RTX 5090 evidence. Evidence lives on independent fork documentation branches.

Environment	Checks	Result / scope
A6000, original isolated operator/runtime build	E4M3/BOOL, paged Prefill, communicator teardown	2 cast checks, 88 Prefill cases and caller-device checks passed
A6000, matching integrated runtime	Migrated changed-input graph tests	11 passed; files byte-identical to #1560; no fresh isolated combined-head build claim
C500, archived matching graph implementation	Sum/scalar-power and MetaX concatenation	5 + 6 passed; not rerun on C500 during consolidation
Two RTX 5090, recorded graph/SM120 overlay	Core casts/Prefill/Decode; LM tiny TP1/TP2; real single-request MTP	2 / 88 / Decode suite passed; 65+1 skip / 66 tiny checks; real K1/K2/K4 exact

The 5090 runtime's graph production changes are now inside this PR; SM120 build-option enumeration remains a documented validation overlay. Builds/tests above retain their exact recorded revisions. The earlier consolidation checks reused the matching prebuilt runtime; the September 23 checks above rebuilt the current head. Project formatting (clang-format 21.1.8/Ruff 0.15.20) and whitespace checks pass.

Performance and limitations

On TP2 RTX 5090, real 27B FP8 weights/BF16 compute, batch=1 greedy, 80×64-token pages, 63/127/1023-input and 64-output tokens, three warm repeats: ordinary Decode graph gives 63.21/62.43/52.16 tok/s; K2 eager MTP gives 107.25/116.50/86.79. These are whole-stack MTP gains, not isolated Core-patch speedups. Sampled K2 device peaks are 24394/24398 MiB (100 ms sampling).

Resolved integration issue: the previously reported 5090 cancel/re-admit mismatch at token 21 was fixed in InfiniTensor/InfiniLM#584, production commit 78d19f74. Aligning the per-token GDN gate projection arithmetic between batched Decode and checkpointed verification resolved the reproduced divergence. The original controlled lifecycle test now passes for K1/K2/K4, including cancellation, re-admission and complete KV/state reclamation; the same-state TP2 probe matches all 384 Conv/GDN tensors and both hidden vectors exactly. No comparison was relaxed, no target replay was added, and this Core implementation is unchanged. Fix, reproducer and verification.

vLLM K2 failed its own ordinary-output comparison in the archived experiment; no validated cross-framework MTP speed claim is made.

Review and CI

Head a3ac4df4: fork Ruff passed; Linux/Windows build matrix passed all four Ubuntu/Windows build and CPU-test jobs. The existing workflow labels jobs debug/release but does not pass matrix.type to the build command; these results do not certify distinct build modes or GPU execution. Previous original-head build matrices passed. Upstream external-PR jobs need maintainer approval; no upstream success is implied.

Please review allocator/graph ownership separately from numerical dispatch. @PanZezhong1725 and @spike-zhu were identified for review; formal assignment previously failed for lack of upstream permissions. Keep approval and required CI as merge gates. The MetaX code is supported by the linked C500 evidence, not a new vendor-wide validation claim.