fix(minimax-h3): support ComfyUI GGUF tensor layouts

[aegioscy]

Summary

• Restore ComfyUI comfy.gguf.orig_shape metadata so MiniMax-H3 tensors use their logical dimensions.
• Transcode incompatible quantized ComfyUI rows to Q4_0 without expanding the full tensor to a multi-GB F32 buffer.
• Allow MiniMax-H3 AdaLN modulation weights to remain quantized and improve hidden-size detection for ComfyUI projections.

Model and quantization comparison

Model family	Weight format / layout	Expected quality	Compatibility	Notes
Official Comfy-Org MiniMax-H3	BF16, INT8 ConvRot, pruned INT8, NVFP4/AWQ; native ComfyUI layout	Best practical quality among local variants; BF16 is the reference	Native ComfyUI workflows	Comfy-Org reports that AdaLN pruning is functionally equivalent and reduces memory; ConvRot/INT8 retains more precision than Q4 GGUF
realrebelai MiniMax-H3 GGUFs	ComfyUI-exported GGUF, typically Q2/Q3/Q4; packed physical shapes plus comfy.gguf.orig_shape	Q4 is a practical balance; Q3/Q2 trade quality for memory	ComfyUI GGUF loaders; supported by this PR in stable-diffusion.cpp	This is the ComfyUI Q4/Q3 model family validated in this work
Unsloth MiniMax-H3 GGUF	Standard GGUF quantization of the pruned H3 denoiser, Q2 through Q8, plus Qwen GGUF encoders	Q4 is a practical baseline; Q5/Q8 should preserve more detail	Standard stable-diffusion.cpp and other GGUF runtimes	Unsloth documents that its denoiser uses the Comfy-Org pruned architecture; quantization still changes numerics
leejet MiniMax-H3 GGUF	Standard stable-diffusion.cpp-converted GGUF, including Q2/Q4 variants	Higher quant levels should be closer to source; Q4 is the usual balance	Native stable-diffusion.cpp path	The upstream H3 documentation points directly to this repository: leejet H3 docs

At the same quantization level, there is no evidence that the ComfyUI GGUF is inherently higher quality than the leejet or Unsloth GGUF. The main difference addressed here is tensor layout and memory handling, not an improvement to the underlying model fidelity.

What this PR enables

Realrebelai/ComfyUI MiniMax-H3 GGUF denoisers can now load in stable-diffusion.cpp. ComfyUI stores some tensors in quantization-friendly physical shapes and records the original PyTorch dimensions in comfy.gguf.orig_shape.*; the loader now restores those logical shapes while preserving an efficient quantized representation.

The tested configuration used the ComfyUI/realrebelai FL2VA Q4 denoiser together with the compatible Qwen3-VL encoder and MiniMax-H3 video/audio VAEs. It is specifically targeted at the ComfyUI-exported H3 denoiser layout, not a claim that every ComfyUI model is automatically supported.

Why memory usage improves

Previously, incompatible quantized rows could be expanded to F32 during conversion. For a large H3 denoiser this required tens of gigabytes of temporary/runtime memory. Conversion now runs in block-aligned row groups, avoiding a full-tensor F32 scratch buffer, and AdaLN modulation weights are no longer unconditionally forced to F32.

The tested ComfyUI Q4 parameter footprint decreased from approximately 86 GB to 43 GB. The Q3/Q2 configuration decreased from approximately 79 GB to 36 GB. Both allowed full-resolution 960×544 H3 coffee-commercial generation on an RTX 5090.

Validation

• Built the Vulkan engine successfully on RTX 5090.
• Generated full-resolution H3 ComfyUI Q4 and Q3/Q2 coffee-commercial AVIs with native stereo audio.
• Regression-tested standard Unsloth Q4 H3 loading and generation.
• Confirmed the PR changes only src/model/diffusion/minimax_h3.hpp, src/model_io/gguf_io.cpp, and src/model_loader.cpp; no ggml implementation files were modified.
• No controlled frame-by-frame quality or audio A/B comparison between realrebelai, Unsloth, and leejet Q4 has been completed yet.

Related addon controls

The separate QVAC addon PR #3923 exposes backend, params_backend, max_vram, and stream_layers for constrained execution. Those runtime controls are complementary to this PR's ComfyUI tensor-layout and loader changes.

References

• Comfy-Org MiniMax-H3 announcement
• MiniMax-AI H3 integration list
• Community quantization overview

Speed comparison

Measured on the same RTX 5090 using the 960×544, 124-frame, 8-step coffee-commercial recipe, with native stereo audio:

Model	Wall-clock time	Generation time	Diffusion parameter memory	Comparison status
Unsloth Q4	124.3 s	118.8 s	11.0 GB	Controlled baseline
realrebelai/ComfyUI Q4	182.8 s	178.3 s	19.0 GB	Controlled comparison; approximately 47% slower overall / 50% slower during generation
realrebelai/ComfyUI Q3 + Q2 encoder	186.5 s	181.8 s	59.2 GB before the loader memory reduction	Different quantization and placement; directional only
Unsloth Q2 + Q2 encoder	123.6 s	119.5 s	6.5 GB	Different quantization; directional only

The ComfyUI Q4 path is approximately 1.47× slower than Unsloth Q4 in this test. The loader PR primarily solves compatibility and memory expansion; it is not a speed optimization. The ComfyUI Q4 export still has more F32 diffusion tensors than the Unsloth Q4 export, which helps explain the higher runtime.

The standard leejet MiniMax-H3 GGUF has not yet been benchmarked with identical settings. A leejet Q4 run should be added before making a three-way speed claim.

Streaming smoke tests were not included in the controlled table: 5 frames / 1 step completed in approximately 20–24 seconds depending on CPU versus disk-backed parameters, but that workload is too small to predict full-render throughput.

[amangupta-tether]

Automated findings: will review manually a bit later

---

1. AdaLN change is not ComfyUI-specific

src/model/diffusion/minimax_h3.hpp lines 298-302

          blocks["linear"] = std::make_shared<Linear>(time_dim,
                                                      hidden_size * expand * modalities,
                                                      true,
                                                      false,
                                                      force_f32);

Previously force_f32 was the 4th Linear argument, so AdaLN-curve models (adaln_t_table present) stored modulation weights as F32. Now that
flag is force_prec_f32: quantized weights stay quantized, and only ggml_mul_mat_set_prec(..., GGML_PREC_F32) is set.

That is accumulator precision, not F32 weights. It changes Unsloth/leejet H3 too, not just realrebelai. For this architecture that is a
large quality surface: 50 blocks x Linear(2688, 537663) is on the order of 13B AdaLN params. The 86 GB -> 43 GB drop is mostly this, which
is why it is tempting, but the old F32 force was probably load-bearing.

The PR itself says there is no frame-by-frame A/B. I would not ship this mixed into the layout fix. Split it, or keep F32 storage on the
non-ComfyUI path until there is a real quality comparison.

2. Q4_K/Q2_K/Q3_K -> Q4_0 is a second quant, and Q4_0 is hardcoded

src/model_io/gguf_io.cpp lines 137-141

      if (restored_comfy_shape &&
          ggml_is_quantized(dummy->type) &&
          logical_ne[0] % ggml_blck_size(dummy->type) != 0 &&
          logical_ne[0] % ggml_blck_size(GGML_TYPE_Q4_0) == 0) {
          tensor_storage.expected_type = GGML_TYPE_Q4_0;

This is necessary for K-quants (blck=256) with logical ne[0] like 96 (video_patch_proj is Linear(96, 5376)). You cannot view Q4_K
superblocks as 96-wide rows.

Q4_0-on-Q4_0 packed files are fine without transcode: flatten order is unchanged and 96 is a multiple of 32, so memcpy of the same bytes is
correct. The bug is the dest type:

• Q8_0 / Q6_K / Q5_K sources also get crushed to Q4_0
• If logical_ne[0] % 32 != 0, nothing is set and Linear still falls back to F32, which is the original memory problem for those tensors

Prefer dest by source (Q8_0 if the source is Q5+ and 32-aligned; Q4_0 only for Q4/Q3/Q2), and log how many tensors were remapped.

3. GGUFReader fallback never applies orig_shape

If gguf_init_from_file fails, the reader path still builds TensorStorage from the physical shape. Packed H3 then detects
video_latent_channels = 256/4 = 64 and a bogus hidden_size. That should use the same helper, or fail loudly.

────────────────────────────────────────

[github-actions]

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

[aegioscy]

Implemented the requested PR #30 fixes in commit d83cd77. The inline notes below map each substantive review concern to the corrected code and validation coverage.