HRX b129 on an RX 7900 XTX: numbers and a few questions about where the advantage comes from

[briansp2020]

Thank you for publishing HRX and the nightly integration builds. The fused Qwen3-MoE decode path is impressive: on our RX 7900 XTX, b129 runs Qwen3-30B-A3B decode 56 % faster than the llama.cpp tree it pins, built with HIP on the same ROCm. We spent a couple of days trying to understand where that comes from, and we're sharing the numbers in case they're useful. We'd also like to ask how you see HRX's advantage, since we were curious which parts come from the runtime and which from the fused kernels.

Setup

• RX 7900 XTX (gfx1100), 1 GPU, PCIe Gen4 x16, Ryzen 9 5950X, amdgpu DKMS 7.1.9; rocm/dev-ubuntu-24.04:10.1.0-full userspace (HIP 7.16.26385) for the HIP builds, HRX's bundled runtime for hrx; -ngl 99 -fa 1.
• hrx: the prebuilt llama-hrx-b129 tarball (sha256 309e124c2c532980c5541cc2857f187a2a3f5cc82b781d01c05ebb498b524058, matches the release digest), unmodified, device HRX0.
• hip: the llama.cpp tree b129 pins (44ee5710d19f), built with HIP (-DGGML_HIP=ON -DGGML_HIP_GRAPHS=ON, gfx1100, Release).
• A third column shows stew675/llama-cpp-rdna-boosts release r177, a community RDNA-tuned HIP fork built the same way on the same image, to give a sense of how far tuned HIP gets on this card. It isn't meant as a project comparison; the two have different goals.
• Models:
  • Llama-3.2-1B-Instruct UD-Q4_K_XL (unsloth);
  • Qwen3-30B-A3B Q4_K_M (unsloth, sha256 9f1a24700a339b09c06009b729b5c809e0b64c213b8af5b711b3dbdfd0c5ba48);
  • Qwen3.6-35B-A3B Q4_K_M (lmstudio-community, sha256 4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d).
• llama-bench -p 512 -n 128 -d 0,4096 -r 5, two alternating rounds. HRX JIT-compiles on the first rep at a new depth, so each cell is the mean of reps 2-5, averaged over both rounds.

End-to-end (t/s):

model	test	hip	hrx	rdna-boosts r177
Llama-3.2-1B	pp512	19963	19855	20855
	tg128	446.0	512.2	533.1
	tg128 @ d4096	392.0	254.0	453.7
Qwen3-30B-A3B	pp512	2897	4755	4883
	tg128	138.8	216.8	199.9
	pp512 @ d4096	2444	3180	3633
	tg128 @ d4096	126.4	75.4	175.6
Qwen3.6-35B-A3B	pp512	2508	4322	4894
	tg128	110.8	90.4	156.1
	tg128 @ d4096	113.9	72.0	152.6

Bold: the faster of hip and hrx (same llama.cpp tree, same card).

On b127 we also checked wikitext-2 KLD (24 x 512) of hrx against the hip build: 0.0029 (1B), 0.0058 (30B), 0.0092 (35B), so the outputs stay close to the hip build.

Where the Qwen3-30B decode gain comes from. We profiled one decode token on b127 with HRX_PROFILE_MODE=dispatch (read with iree-profile dispatch), and the RDNA-tuned HIP build with rocprofv3 (rdna-boosts r170 plus our own patches, a development build on the older ROCm 10.0 image, 166 t/s tg128 on the same card). We profiled b127 because it was the current build at the time; we haven't repeated this on b129.

per decode token	hrx b127	RDNA-tuned HIP build
kernels	304 (one submission)	850
GPU busy	3960 us	4535 us
GPU idle between kernels	~606 us	~1485 us
idle per kernel	~2.0 us	~1.75 us

So on this card the gain comes from fewer, fused kernels (2.8x fewer launches) plus about 13 % less GPU time; the idle time per dispatch was similar for the two (about 2.0 us vs 1.75 us). The fused kernels appear to be where the work went: for example, qwen3_moe_attention_qkv_postprocess_fused_decode does the Q/K/V projections, per-head norm, rope and KV write in one launch (10.9 us, vs 19.0 us in 5 launches on the HIP build).

A side note, possibly expected at this stage: we also tried HRX's HIP-compatible libamdhip64.so (b127) under that same HIP build (LD_PRELOAD). It runs cleanly; decode was about 26-29 % below the stock HIP runtime on all three models.

Questions, if you have a moment:

1. Is per-dispatch latency a goal for the HRX runtime on discrete RDNA3? Or is the expected advantage elsewhere: whole-graph scheduling, APUs like gfx1151, or something we didn't measure?
2. Decode at 4096 tokens of context drops to 60-65 % of the hip build on the 1B and 30B (and to about a third of its own depth-0 speed on the 30B). We'd be curious whether long-context attention is something you're planning to look at.
3. Qwen3.6 (qwen35moe) decode is slower than hip at every depth, though its prefill is 72 % faster. Is fused decode for qwen35moe, like the Qwen3-MoE path, something you're considering?
4. On an R9700 (gfx1201), b127 lists the device, but every graph fails with no HRX dispatch registry for target gfx1201. Is RDNA4 planned?

We plan to rerun the same comparison on b130 once it is out with Loom v0.1.0, and we're happy to run any configuration, trace or build you'd like on a 7900 XTX or an R9700. Nothing urgent on our side; we're mainly curious where you see HRX heading.

(Claude-assisted.)