[perNyfelt]

Summary

Implements Phase 7.3b WP4: Core RoPE and decoder integration (req/plans/phase7-3b-plan.md §6), per the design frozen in req/plans/phase7-3b-reference-findings.md §6.9 (committed in this branch, Unit 1).

Lands the core seam the later Qwen3-VL work (WP5–WP7) builds on — no model, no loader, no dispatch yet:

1. MultiAxisRope / MultiAxisRotation (jmlx-core, se.alipsa.jmlx.nn) — the dedicated explicit FLOAT32 frequency/cos/sin and half-rotation path for Qwen3-VL. Pinned interleaved MRoPE frequency selection (T-owning round-robin overwrite per findings §6.5), NeoX half-split rotation, dtype-cast order per the frozen compute/cast sequence, validated [3, B·T] positions and section/rotary-dim/head-dim bounds, all checked before any array is written. A dedicated component — RopeSpec gains no new permitted variant (sealed hierarchy and MLXFast are byte-identical to main).
2. Additive decoder seam — DecoderAttention/DecoderBlock (core) and DecoderModel (models) gain overloads threading a request-supplied MultiAxisRotation and a package-private PostBlockHook (invoked after each block's residual output, before final normalization — the DeepStack injection point; vision extraction indices vs text injection indices documented and never mixed). A DecodeStep + 9-arg runStepLoop lets a request supply its decode path (decode coordinates = logical cache position + delta). Every existing signature is unchanged and delegates null, so absent multimodal inputs route through the identical old graph; the scheduler's EmbeddingHook stays at its original post-lookup/pre-scale point; no new public API.
3. VisualFeatureInjection (jmlx-models, package-private) — visual row-gather replacement and masked DeepStack residual add, pure compositions of existing core facades (concatenate/takeAxis/zeros/add); no new FFI primitive.
4. Strict tests + pinned-vendor cross-check probe — MultiAxisRopeTest, SingleRowDecodeEquivalenceTest, DecoderRotationSeamTest, DecoderSeamTest, VisualFeatureInjectionTest (float32-comparison cases tagged full-float32), and the opt-in tools/hf-reference/qwen3vl_rope_interleaved_check.{py,jsh} probe pair cross-checking the Java component element-wise against the pinned transformers 4.57.6 apply_interleaved_mrope + rotate-half in the locked venv (per-case max abs error ≤ 2.4e-7 against the 1e-5 threshold; never Gradle-invoked, documented in the README).

Per-path kernel matrix (frozen in findings §6.9)

Path	Kernel
Qwen3-VL multimodal prefill, cached decode, text-only prefill, fresh full forward	dedicated explicit FLOAT32 freq/cos/sin + half rotation (MultiAxisRope/MultiAxisRotation)
Ordinary decoders (llama, llama31, mistral, mixtral, phi3, qwen2, qwen3, gemma) and T5/encoder	unchanged mlx_fast_rope/mlx_fast_rope_dynamic via RopeSpec.apply

No fast-kernel optimization of the Qwen3-VL path in this WP (any later one requires a dated amendment per plan §6).

Exit gate — exact before/after decoder bits (both TF32 modes)

• Before (recorded at base commit 128cd1e, clean tree, via the reused 7.3a :jmlx-models:exactBitsRecord): 14 variants (8 Tier-A families + 6 quantized) × 9 captures (direct-prefill bits, per-step decode bits, batched prefill/decode with and without the EmbeddingHook, exact greedy IDs, full scheduler run) in each of MLX_ENABLE_TF32=1 and =0; spec hash 846396b1dbab5a770a4f2c36841fe5ed1570a586b908956ed6e224fe5248e687; recordings are device-specific PR evidence in jmlx-models/build/exact-bits/tf32-{1,0}/ (host: M5 Max, macOS 27.0.1, mlx-metal 0.31.2, mlx-c fba4470b).
• After (this branch): :jmlx-models:exactBitsVerifyTf32On and :jmlx-models:exactBitsVerifyTf32Off both PASS — 126 captures per mode bit-exact, greedy IDs exact, only the permitted git-commit metadata field drifts. The old 1-D decoder path is bit-identical in both precision modes.

Validation (all local, green)

• :jmlx-core:check (414 test / 30 float32, 0 failures), :jmlx-models:check (376 test / 43 float32 / 2 thread-probe, 0 failures), root ./gradlew check, ./gradlew verifyMlxApiCallSites.
• Python probe re-run in the locked venv: exit 0, all 7 index-coded cases.
• git diff main -- RopeSpec.java MLXFast.java empty; no jmlx-ffi/jmlx-vision/build/CI changes; no new Gradle tasks; no bindings regeneration.

Docs

• Findings §6.9: design freeze (kernel matrix, compute/cast order, signatures, test matrix) + WP4 acceptance record.
• Plan: header status → WP0–WP4 complete; §6 WP4 status paragraph; §11 — checked Old one-dimensional decoder path remains bit-identical in both precision modes and Qwen kernel matrix, delta-shifted decode tests and unchanged RopeSpec variants verified.
• AGENTS.md: MultiAxisRope/MultiAxisRotation in the core nn list; WP4 paragraph in "Phase 7.3b additions".

Explicitly deferred (documented in findings §6.9)

• CI required-suite-execution list additions for the new native suites → WP8 (plan §9).
• DeepStack consumption of the PostBlockHook, Qwen3VlModel composition and qwen3_vl dispatch → WP5/WP6; generation/lifecycle/cache policy → WP7.

Non-goals held: no native-pin change, no generated-binding edit, no loader, no model class, no public API.