Ideogram4's 9.3B transformer and modified Qwen3-VL text encoder, running on Apple Silicon through the standalone mlx-nf4 Metal extension and stock MLX.
Model weights are under the Ideogram 4 Non-Commercial License. This repo is a technical demo, not a product.
Historical 1024×1024 result from the predecessor integrated-kernel route on an M4 Max. The standalone-package proof and its exact receipt appear below.
Native NF4 (NormalFloat4) quantized matrix multiplication for MLX, loading official bitsandbytes 4-bit weights directly on Apple Silicon. Ideogram4 is the proof at model scale: both 9.3B DiT passes and the language path of its modified Qwen3-VL text encoder run from the original NF4 checkpoint files and generate coherent images with legible rendered text. The text-only pipeline loads all 252 language NF4 projections and explicitly excludes the checkpoint's 116 unused vision projections.
The kernels live in mlx-nf4, a standalone custom extension that installs alongside stock MLX. The package keeps weights compressed through its native Metal matmul, supplies NF4Linear, understands bitsandbytes nibble order and nested scale state, and avoids the old permanent-MLX-fork dependency trap. This repository owns the Ideogram-specific architecture, checkpoint mapping, prompting, sampling, VAE decode, UI, and receipts.
Representative receipts on Apple Silicon. Timing moves with machine contention and route setup, so the committed standalone-package smoke is listed separately from predecessor integrated-kernel measurements. Memory is MLX-reported peak active memory during sampling (excludes model loading overhead):
How memory is measured. Peak figures come from MLX's own allocator counter, mlx.core.get_peak_memory() (bytes of GPU/unified memory at the high-water mark), read at the point noted per number — sampling-only for the benchmark tables (counter reset after model load), full-run including load for the 16 GB verification below. We report MLX's allocator peak rather than process RSS because RSS undercounts MLX's unified-memory buffers. As an independent cross-check, the 16 GB run was also traced at the system level (vm_stat / vm.swapusage sampled every 2s) to observe swap/compression behavior; that trace is what backs the "leaned on compression, never swapped" statement, not the memory numbers themselves.
| Route | Format | Steps | s/step | Sampling | Peak Memory |
|---|---|---|---|---|---|
Standalone mlx-nf4 smoke |
NF4 4-bit | 20 | 6.36 | 127.1s | 11.55 GB |
| Predecessor integrated-kernel fast receipts | NF4 4-bit | 20 | 3.3-3.4 | 67-69s | 11.52 GB |
| MFLUX | FP8 8-bit | 20 | 8.9 | 178s | 28.1 GB |
Standalone-package smoke: evidence/live_runs/20260825_standalone-mlx-nf4_512x512_seed2025.json. It records clean Ideogram source commit 3f626c9, installed mlx-nf4 commit 6fa1281, wheel-installed stock MLX 0.32.2, the pinned model revision, Greenroom route, output hash, and a visually inspected 512×512 image. The fast matrix receipts are older uncontended runs from the predecessor integrated-kernel route under evidence/matrix/; they establish the latency floor, while the new receipt establishes package extraction and model-scale composition. The defensible headline is memory: NF4 uses roughly 2.4× less memory than the recorded MFLUX FP8 comparison at 512×512.
| Route | Format | Steps | s/step | Sampling | Peak Memory |
|---|---|---|---|---|---|
| Predecessor integrated-kernel route | NF4 4-bit | 20 | 30.4 | 608s | 13.7 GB |
| MFLUX | FP8 8-bit | 20 | 25.3 | 505s | 30.6 GB |
On that predecessor 1024×1024 comparison, MFLUX was 17% faster per-step, while using 2.2× more memory. That receipt remains useful historical evidence for the kernel family, but it is not presented as a standalone-package run.
The following machine-fit table and M2 Pro receipt predate the standalone package. They establish that this NF4 kernel family can fit the workload under those memory ceilings; they are retained as predecessor evidence, not silently promoted into standalone-package runs.
| Mac | NF4/MLX | MFLUX FP8 | GGUF Q4 |
|---|---|---|---|
| 16 GB (base MacBook Pro) | 512 ✓ | ✗ | ✗ |
| 24 GB | 1024 ✓ | ✗ | 512 maybe |
| 32 GB | 1024 ✓ | 512 ✓, 1024 barely | 512 ✓ |
| 48 GB+ | All ✓ | All ✓ | All ✓ |
Predecessor integrated-kernel evidence on a 16 GB M2 Pro: 512×512 / 20 steps ran to completion
in 9m48s (21.8s/step — slower than the M4 Max purely from the smaller GPU)
at 11.51 GB full-run peak (MLX get_peak_memory(), including model load —
essentially identical to the sampling-only 11.5 GB measured on the M4 Max, so
load does not blow the budget). A 2-second system-level trace (vm_stat) shows
the machine leaned hard on memory compression — peak ~7.8 GB compressed, free
memory routinely under ~100 MB — but never paged to swap (swap stayed at 0
throughout). It stayed responsive; the slowdown tracks the GPU gap, not memory
pressure. The M2 Pro's GPU is a step up from the base M-series chip in the
cheapest 16 GB MacBook Pro, but the 16 GB memory ceiling — the thing that gates
whether the model fits at all — is identical.
Requires macOS on Apple Silicon (M1+) and Python 3.10+.
# 1. Clone this repo
git clone https://github.com/lyonsno/mlx-ideogram4.git
cd mlx-ideogram4
# 2. Install the project. Its pinned dependency builds mlx-nf4 alongside
# stock MLX; no fork replacement or install-order trick is required.
pip install -e .
# 3. Accept the Ideogram4 license and log in to HuggingFace
# Visit: https://huggingface.co/ideogram-ai/ideogram-4-nf4
# Then:
hf auth login
# 4. Generate! (first run downloads ~16 GB of model weights)
python generate.py \
--prompt "a red cat sitting on a blue couch" \
--output cat.pngOr with uv:
git clone https://github.com/lyonsno/mlx-ideogram4.git
cd mlx-ideogram4
uv sync
uv run python generate.py \
--prompt "a red cat sitting on a blue couch" \
--output cat.pngpip install gradio
python app.py # local UI at http://127.0.0.1:7860
# stable ngrok public tunnel, using a free ngrok dev domain you control.
NGROK_DOMAIN=your-assigned-name.ngrok-free.dev ./serve.sh --public --tunnel ngrok
# Temporary Gradio share URL; useful for a quick private smoke, not a stable public link.
python app.py --sharepython generate.py --prompt "your prompt" --preset V4_TURBO_12 # fast preview
python generate.py --prompt "your prompt" --preset V4_DEFAULT_20 # good balance (default)
python generate.py --prompt "your prompt" --preset V4_QUALITY_48 # highest qualityThree NF4-quantized model components:
| Component | Params | NF4 Layers | Memory |
|---|---|---|---|
| Text encoder language path (modified Qwen3-VL-8B checkpoint) | checkpoint model 8.8B | 252 loaded; 116 unused vision projections excluded | 5.5 GB |
| Conditional transformer (34-layer DiT) | 9.3B | 211 | 5.2 GB |
| Unconditional transformer | 9.3B | 211 | 5.2 GB |
| VAE (Flux2 KL-VAE) | 45M | 0 | 168 MB |
Pipeline: tokenize → Qwen3-VL hidden state extraction (13 layers) → Euler flow-matching with asymmetric CFG → Flux2 VAE decode.
The core contribution now ships independently as mlx-nf4:
- 16-element LUT derived from normal distribution quantiles (QLoRA)
- Scaled LUT optimization: precompute
lut[i] × absmaxper group, eliminating one multiply from the inner loop - Native packed-weight matmul: weights stay compressed through the Metal fast path
- Public package API:
quantize,dequantize,quantized_matmul,NF4Linear, and an explicit reference path - Model-shaped activations: arbitrary leading dimensions are flattened only at the native boundary and restored on output
- bitsandbytes interoperability: high-first nibble intake plus plain and nested/double-quantized scale reconstruction
- Group sizes and dtypes: 32/64/128 with float32, float16, and bfloat16 activations
The implementation began inside MLX core. The standalone package preserves essentially the same measured kernel latency without asking users to replace MLX itself; its repository carries the build, clean-install verification, native matrix, API documentation, provenance, and GPT-2 consumer smoke.
- Ideogram for the model and weights (non-commercial license)
- MLX by Apple
- MFLUX for prior FP8 Ideogram4 on MLX
- stable-diffusion.cpp and GGUF quantization work
- bitsandbytes for the NF4 format
- mlx-vlm for Qwen3-VL architecture
Code: MIT. Model weights: Ideogram 4 Non-Commercial License.