sycl: fuse the PQ2_0/PTQ1_0 XMX mat-muls with their neighbours (depends on #327)

[kiljoy001]

Overview

Depends on #327 (its two commits are in this branch: the first and the third, a cherry-pick of #327's follow-up; review #327 first). This PR adds the second commit: the graph loop now fuses the PQ2_0/PTQ1_0 XMX mat-muls with the ops around them, so far fewer small kernels run per token.

• Epilogues in the mat-mul store (TernSYCL's postops 1 and 2): {mul_mat(gate), mul_mat(up), SWIGLU} runs as two mat-muls from one quantized activation, the gate writing silu(gate) * up; mul_mat -> (reshape) -> ADD (the residual adds after ffn_down, ssm_out and the attention output) writes the sum.
• Shared activations. A quantized activation is kept for the length of a graph compute and reused by every XMX mat-mul that reads it (q/k/v; qkv and z of the gated delta net layers; gate and up), then freed after its last user. They live in a separate pool (xmx_act_pool()), because the VMM pool only frees in reverse order.
• Hadamard folding (Bonsai 2). Every rotated mat-mul input is x * signs -> FWHT_1024 (hinted mul_mat) -> mul_mat. The sign flip, the FWHT and the int8 quantization now run in one kernel (the butterflies follow ggml_sycl_op_fwht). When every user of the FWHT output is an XMX mat-mul (checked against the graph's use counts), that output is never written; otherwise one kernel does sign flip + FWHT for the other users.
• Deferred gate. A mat-mul whose only use is the gate of a later SWIGLU (z of the gated delta net layers, three nodes before its GLU) keeps its quantized activation and runs at the GLU, with the SWIGLU in its store.

GGML_SYCL_ENABLE_FUSION=0 turns all of this off. Other models and types take exactly the paths of #327.

The fusion rules are independent of the weight type; only ggml_sycl_pq2_xmx_eligible() (the side-effect-free half of #294's gate, split out here) decides which mat-muls take part.

Results

Same setup as #327: Intel Arc Pro B50 (16 GB), oneAPI 2026.1, Level Zero 1.14.37020, Linux 7.0 xe driver, -ngl 99. "#327" is the first commit alone; all columns measured in the same session.

llama-bench, t/s:

model	test	prism	#327	this PR
1.7B PQ2_0	pp512	8198	9536	10505
	tg64	185.6	213.9	228.7
4B PQ2_0	pp512	3508	4387	4773
	tg64	106.0	121.1	130.5
27B PQ2_0	pp128	457	643	713
	pp512	491	627	722
	tg64	19.27	20.66	21.78
27B PTQ1_0	pp128	456	548	589
	pp512	490	572	633
	tg64	19.02	22.09	23.16
27B PQ2_0	tg64 @ 16K depth (-fa 1)	13.89	14.73	15.49
27B PTQ1_0	tg64 @ 16K depth (-fa 1)	13.88	15.35	16.19

Parallel decode (llama-batched-bench -npp 128 -ntg 64), aggregate decode t/s, #327 -> this PR:

sequences	1.7B PQ2_0	4B PQ2_0	27B PQ2_0	27B PTQ1_0
1	210.7 -> 218.5	122.5 -> 128.5	20.6 -> 21.7	21.7 -> 22.9
2	404.4 -> 430.6	232.9 -> 246.9	37.6 -> 39.3	38.1 -> 40.0
4	737.5 -> 783.5	430.3 -> 455.1	59.6 -> 61.5	59.9 -> 61.7
8	1172.5 -> 1223.8	724.3 -> 753.1	81.7 -> 83.1	82.2 -> 83.8
16	1574.6 -> 1643.5	1026.5 -> 1075.4	101.8 -> 105.5	105.7 -> 107.8
32	2264.8 -> 2367.2	1577.5 -> 1661.1	117.7 -> 122.0	119.7 -> 122.5

Speculative decoding, Bonsai 2 27B PTQ1_0 with the grafted MTP head (sudoingx/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF), llama-server, 3 chat prompts (prose, code, bash) x 256 tokens, greedy, thinking off, decode t/s:

	prism	#327	this PR
target only	18.7	21.8	23.0
--spec-type draft-mtp --spec-draft-n-max 1	29.8	33.5	35.3 (75% accepted)
--spec-type draft-mtp --spec-draft-n-max 2	34.2	38.5	39.4 (62% accepted)

With this stack the B50 decodes Bonsai 2 27B at ~39 t/s (code and bash ~42, prose ~33) on one 16 GB, 70 W card, short context.

Correctness

• test-backend-ops -b SYCL0, run per op over all 130 ops: MUL_MAT 1401/1401, MUL_MAT_ID 1035/1035, MUL_MAT_VEC_FUSION 506/506 (gate/up + GLU and mat-mul + bias ADD on PQ2_0 and PTQ1_0, 1-8 tokens), ADD 99/99, GET_ROWS 119/119. The ops that do not pass fully (CONV_2D, FLASH_ATTN_EXT, LIGHTNING_INDEXER, ROLL; CPY and SET_ROWS abort) do the same on prism.
• The Hadamard folding has no test-backend-ops case. It is covered by the 27B perplexity below and by checking, with GGML_SYCL_DEBUG=1, that every Hadamard site of the 27B decode graph takes the fused path (257 sites per decode graph).
• Perplexity (-c 512, 5 chunks of the repository's docs/*.md): identical to sycl: PQ2_0/PTQ1_0 XMX kernels from TernSYCL, PTQ1_0 kept at 1.75 bpw #327 for 1.7B b1/b512, 4B b1/b512, 1.7B -ngl 0, 4B -ngl 20; 27B PQ2_0/PTQ1_0 b512 6.2356 (sycl: PQ2_0/PTQ1_0 XMX kernels from TernSYCL, PTQ1_0 kept at 1.75 bpw #327) -> 6.2367 (this PR), prism 6.2370; 27B PTQ1_0 with -ub 16 6.2389.
• KL divergence against a CPU-only reference (1.7B PQ2_0): identical to sycl: PQ2_0/PTQ1_0 XMX kernels from TernSYCL, PTQ1_0 kept at 1.75 bpw #327 (b1 0.00130, b512 0.00120).

Additional information

• The activation cache is per graph compute and keyed by the activation tensor (a reshape or view of all of it resolves to the same key) and the weight type, since PQ2_0 and PTQ1_0 read it in different K orders. Consumers of a plain activation are counted within a window of 64 nodes; later ones quantize again. For Hadamard inputs the count is exact, from the graph's use counts, and the fused path is only taken when all users are in this graph.
• An epilogue in the GEMM is applied per element on the way out instead of on the accumulators before the 2D store; the backend compiler rejected the inline-asm store operand otherwise.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - the code and this description were written by Claude (Anthropic) under my direction; I tested it on my own hardware (Intel Arc Pro B50) and reviewed the changes.