Repository navigation
Add fixed diffusion tensor execution for SDXL and PixArt - #314
schopde-nvidia wants to merge 1 commit into
Conversation
Flare's token-serving contract does not describe complete diffusion tensor requests. Comparing diffusion against native Flare requires loading exported computation independently of Python. Keep model composition and solver math in Ignition's prepared graphs, and let Helios validate tensor storage and replay the declared graph sequence. Before this change, signed cores and the CLI required token-serving metadata and offered no offline tensor-request command. Add physical graph-only core publication/loading, a generic TensorRunner, and flare run tensor input, output, repetition, and timing options. Validate tensor bounds and input overlap, retain GPU resource owners through queued work, and preserve the paired metadata requirements for token-serving cores. Add fixed SDXL and PixArt preparation recipes, geometry and solver helpers, intermediate-output comparison, and reproducible setup instructions. A separate iterative-serving contract is documented as proposed and is not implemented. The resulting executor runs both diffusion architectures without model-specific native computation. The measured result is byte-identical native replay of captured compiled outputs for the tested profiles. Canonical eager PyTorch numerical equivalence remains unresolved: exact comparisons fail, and the comparator withholds performance ratios. This does not establish concurrent diffusion serving, broader model support, or a speedup. On an RTX PRO 6000 Blackwell Server Edition (SM120), tested SDXL at 1024x1024 with 4 and 30 updates and PixArt at 512x512 with 20 updates, each with two prompts and seeds 42/43. Composed eager recipes match canonical Diffusers; independently loaded native outputs and repetitions match captured outputs. The native process loads neither Python nor Torch. Focused validation passed 114 Rust tests, 28 SDXL Python checks, 41 PixArt Python checks, and 38 geometry and schedule checks. After tightening native acceptance to compare bytes including signed zero, 15 comparator tests pass and all six retained GPU comparisons retain their native-pass/canonical-fail result. These are focused checks, not the full repository suite. GPU execution used source based on a3e8778; integration with subsequent upstream changes has not been built or GPU-tested. Reproduction details and open qualification are in docs/diffusion-design.md and docs/TODO.txt. Design-Signed-Off-By: Codex (GPT-6); design-review=ed22
|
Keeping this in design review, not treating the passing native replay comparisons as canonical model qualification. The description explicitly reports unresolved canonical numerical differences and an untested combined tree. In addition, the fixed offline tensor-request runner and graph-only package contract need an explicit owner decision under AGENTS.md's rule for new captured-operation contracts; reusing physical graphs or reserving iterative serving for a later proposal does not by itself establish that approval. Please link the applicable design authorization. Until that decision and the numerical acceptance requirements are satisfied, I recommend against merging this as supported diffusion execution. No diffusion hardware validation was performed for this triage. |
Flare's token-serving contract does not describe complete diffusion tensor requests. A meaningful comparison with PyTorch needs a separate native process that loads the exported text encoder, denoiser, solver updates, and image decoder. This change keeps those computations in Ignition's prepared graphs and adds shared native tensor transport and graph replay.
Before this change, signed cores required token-serving metadata and the CLI had no offline tensor-request execution path. This PR adds:
TensorRunnerandflare run --tensor-request --tensor-output. They validate tensor extents, input overlap, and graph order, retain resource owners until queued work drains, and write intermediate tensors, repeated outputs, and timing samples.The demonstrated result is complete native execution of two diffusion architectures and byte-identical export/replay for the tested profiles. Canonical eager PyTorch numerical equivalence remains unresolved. Compiled outputs differ from the canonical references, so this is a draft PR with no speedup claim. The command handles fixed offline tensor requests; it does not provide an HTTP image endpoint, concurrent diffusion scheduling, native prompt processing, or general diffusion-model support.
Validation used one NVIDIA RTX PRO 6000 Blackwell Server Edition (SM120), PyTorch 2.14.1+cu132, Triton 3.8.0, and Diffusers 0.43.0. Every profile used two prompts with seeds 42 and 43, three warmups, and five measured native repetitions.
SDXL uses FP16 text encoders/UNet, the canonical FP32 VAE upcast, guidance 5, and negative prompt
blurry, low quality. PixArt uses canonical mixed FP16/FP32 T5 parameters, an FP16 transformer/VAE, guidance 4.5, 120 text positions, and a second-order midpoint solver with an empty negative prompt. Checkpoint revisions, prompts, and executable reproduction commands are in the diffusion design document linked below.Independent native processes loaded neither Python nor Torch. Isolated compiled update and decoder comparisons also differ when supplied canonical intermediates; PixArt's second update differs with canonical solver history. Native replay fidelity therefore does not establish canonical numerical correctness. Raw decoded-image RMSE for the two SDXL 30-step cases is 0.009159 and 0.018393; for PixArt it is 0.008709 and 0.085828. No performance ratio is claimed.
Focused validation passed 114 Rust tests covering schema, protection, export, conversion, tensor execution, and CLI behavior; 28 SDXL Python checks; 41 PixArt Python checks; and 38 geometry/schedule checks. The PixArt count includes the original 13 comparator tests. The byte-sensitive comparator now passes 15 tests, and rechecking all six retained GPU comparisons preserves the native-pass/canonical-fail result. The lightweight local Python run also reported four dependency-related skips. These are focused checks, not a full repository suite.
GPU validation used the implementation based on
a3e8778. The combined tree with subsequent upstream changes has not been built or GPU-tested. Broader checkpoints, shapes, solvers, tensor parallelism, concurrent serving, and numerical acceptance bounds beyond exact comparison remain unqualified. Open qualification work is recorded indocs/TODO.txt.Reproduction: pinned checkpoints, environment setup, preparation, and native comparison commands. Qualification requirements: diffusion TODOs.