[AaronStGeorge]

Overview

Qwen3.8-27B can fail while materializing duplicate HRX weight representations. This change shares eligible Q4_K/Q6_K row64 weights across consumers and restores the short-prefill kernels from #171. In the tested cold runs, the candidate completes both pp66+tg16 and pp512+tg16 with zero duplicate-weight warnings; the parent fails pp66+tg16. Completed pp512 live-weight counters fall from 22,300,747,776 to 17,522,558,976 bytes (-21.43%). These counters measure live weights, not total VRAM.

Draft only: source acceptance remains held for the unresolved 128-token Q/K activation-format mismatch described below. This replaces the earlier combined experimental PR #171 as the first source entry in the serial model rollout. Further implementation changes belong in separate stacked PRs.

The implementation:

• Sets immutable shared-layout policy at llama context initialization for non-Gemma architectures; preserves backend-default-off, Gemma opt-out, and direct-caller behavior through dispatch, scheduling, and graph caches.
• Shares eligible layouts across matrix, gated, attention, and SSM consumers, including Q6 SSM and native Q4 F16 readers. Preserves Qwen3-30B native attention, the Qwen3-1.7B tied endpoint, and native packed GLU.
• Restores Q6 prefill at 64-127 tokens and Q4 row64 metadata reuse in single and paired tiled matmul. Factors/scales are loaded once per 256-element block and reused across eight groups; payload decoding remains per group, with separate gate/up state.

Metadata reuse also works with standard layout, but this source adds it only for row64 format 44. The later native-layout ablation implementation is not included here. Llama and Mistral coverage uses this same source rather than additional model implementations.

Additional information

Exact validation pins: candidate 5d4d8981b837bb5270265b3e94b24ec69f001bac, parent 373507afc8764340d1265d0445ee5a9e54104a92, HRX 82214dfb029975205d1c3a59164b678e2e21fc41, Radeon RX 7900 XTX / gfx1100. The target hrx-graph-develop-v2 has since advanced three commits to 62eb8dc18998e1c5a044cf3cf35f8f2967f2caa3; the combined result has not been validated. Publication preserves the tested source without rebasing.

Completed validation and remaining regressions

• Twelve-model survey: 48 ABBA benchmark calls / 1,440 samples; 96 finite PPL estimates passing the unchanged HRX/Vulkan ratio <= 1.10 criterion; 156/156 candidate serving requests pass. Both builds together pass 300/312, retaining the parent's Qwen27/Mistral failures. These PPL checks do not directly exercise the 64-127-token route.
• Qwen27 pp66 throughput improves 26.56%; decode changes +0.19%, with longer prefill near flat. All six candidate HRX serving requests pass where all six parent requests fail. Failed lifetimes provide no timing or memory denominator. Cold instrumented timings are excluded from throughput.
• Gemma preserves parent routes and residency, including its intentional 182 duplicate warnings. Decode changes +0.11%; this is not an equivalence proof.
• Regressions remain: Phi 4 Mini pp66 -3.45%, decode -1.34%, short serving prefill -7.95%; LFM2 pp66 -5.30%, pp512 -1.53%, pp1024 -0.78%, serving prefill -10.07%/-7.55%, despite 41.10% lower live-weight counters; MiniCPM5 pp66 -4.12% and longer-output serving prefill -5.79%; MiniCPM-V text pp66 -3.11%.
• Direct HRX MTP with --spec-type draft-mtp passes 12 measured requests plus four warmups, with positive acceptance counters and matching parent/candidate outputs. Lemonade's requested-on HRX launch omitted that flag, so automatic integration enablement is not established. Short responses reach their token cap inside reasoning; this is execution evidence, not a complete-answer quality or speed claim.

Correctness hold

A subsequent Llama3.1-8B diagnostic found NaNs in both cold and warm 128-token serial traces. Independent inspection of the frozen source confirms that the legacy Q/K core selects a dense F32 reader while receiving packed Q8 activations: a 589,824-byte binding versus a 2,097,152-byte F32 span at 128 x 4096. The first Q/K projections produce NaNs; Q6 V remains finite. Tracing forces serial execution, so ordinary graph output was not directly checked. Resolve this contract mismatch before acceptance; prior 128-token timing gains are withheld as correctness-qualified performance claims.

Separately, two original paired-Q4 fixtures retain four failing observations across replays. Constant-fixture relative L2 is 0.016664 (limit 0.01; maximum absolute error about 7.47e-8); the varied fixture passes scalar L2 at 0.006352 but fails CPU pointwise comparisons. The path uses F16 WMMA accumulators and the CPU reference quantizes activations. Outputs match the earlier control byte-for-byte, establishing no restoration delta, not absolute correctness. Tolerances, references, and accumulators are unchanged.

Separating layout from metadata reuse

A separate controlled Llama3.1-8B Q4_K_M experiment at 66 tokens measured standard-layout throughput of 1,185 -> 1,382 tok/s with reuse and row64 throughput of 1,120 -> 1,379 tok/s. Paired-block gains were +16.68% and +23.13%, respectively. This supports an interaction between layout and reuse, but the larger row64 gain closes its initial deficit: with reuse enabled, row64's measured model throughput is 0.28% lower. Row64 helps the paired FFN kernel (+18.29% versus standard with reuse) while six single-matrix kernels are 7.5-12.2% slower. These results do not establish overall equivalence or partition the original combined gain, because residency and Q6 routes were controlled. All samples were retained; 28 finite captured-output checks matched bitwise. The experimental source remains separate from this PR.

Requirements

• The agent has read the contributing guidelines; human review and acceptance remain pending for this draft.
• AI usage disclosure: YES. Codex assisted implementation, validation, independent review, and preparation of this description. The user requested draft publication; the unresolved findings above remain explicit.