chore(deps): update dependency flashinfer-python to v0.6.17

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
flashinfer-python	==0.6.14 → ==0.6.17

---

Release Notes

flashinfer-ai/flashinfer (flashinfer-python)

v0.6.17

Compare Source

These highlights are also published at flashinfer.ai/releases.

v0.6.17 Highlights

This release brings MoE expert parallelism into serving engines, refreshes the Blackwell SM12x fused-MoE kernels with an FP4 accuracy fix, extends the unified MoE API to MXFP4 W4A8/W4A16 and shared experts, and adds decode coverage for Kimi K3 MLA and MiniMax-M3 sparse attention on vLLM.

MoE expert parallelism production-ready in vLLM

The MegaMoE path in flashinfer.moe_ep is ready for serving engines: full CUDA-graph capture and replay, a fused single-launch quantize-and-stage hot path, prequantized weight packs, symmetric-buffer workspaces pooled across layers, and a persistent knob cache that resolves tuned knobs by lookup, so production sessions run with no in-engine autotuning. An opt-in fault-tolerance rank mask, available over both NCCL-EP and NIXL-EP, masks and skips a peer that times out during dispatch or combine, keeping the job alive when a rank dies. Deployment gets simpler too: the CuTe-DSL runtime floor returns to 4.5.2, the version vLLM 0.25.1 pins, and a new BootstrapConfig.device lets the host framework pin each worker's CUDA device.

• #​4079
• #​4183
• #​4101
• #​4348

Blackwell SM12x fused MoE refreshed, with an FP4 accuracy fix

W4A4 serving on DGX Spark and RTX PRO parts (GB10, SM120/SM121) now delivers the output quality its benchmark scores imply, with two NVFP4 quantization bugs fixed and a new input_global_scale that lets integrators pass a checkpoint's weight scale directly. The SM12x fused-MoE families are synced to current b12x: the NVFP4 W4A4 backend reaches kernel parity across decode and prefill, and the W4A16 family adds cooperative persistent launches, a tensor-core decode path for small batches, and shape-stable route packing that keeps decode batch-size changes recompile-free.

• #​3932
• #​4285
• #​4255
• #​4253
• #​4130

Unified MoE API adds MXFP4 W4A8 and W4A16, shared experts, and SiTU

TRTLLM-gen MXFP4 weights now run through the unified MoELayer API against both MXFP8 activations (W4A8) and BF16 activations (W4A16), completing the FP8 series begun in 0.6.16, alongside per-tensor routed FP8. Routing adds an unpacked pre-routed FP4 mode that accepts topk_ids and topk_weights as separate contiguous tensors. TRTLLM-gen FP4 MoE also gains shared-expert fusion, extending to FP4 what 0.6.15 added for FP8, plus SiTU activation for MXFP4 x MXFP8 and NVFP4 x NVFP4.

• #​4159
• #​4088
• #​4104
• #​4239
• #​4180

Kimi K3 MLA decode, and MiniMax-M3 sparse attention under vLLM

Blackwell decode now covers Kimi K3's MLA geometry — 96 global query heads against one KV head, TP-local head counts down to 6, speculative query lengths up to 8, and context parallelism for long contexts — by packing query-token and query-head rows into shared CuTe-DSL tiles, adding compact variable-length Q, and extending TRTLLM-gen dense and sparse MLA to non-power-of-two head counts. Sparse MLA decode also serves the no-rotary-tail shape (kv_lora_rank=512, qk_rope_head_dim=0) natively. MiniMax Sparse Attention accepts vLLM's packed paged KV layout for MiniMax-M3, unblocking the vLLM integration on SM120/SM121, with lower per-call overhead on paged decode.

• #​4178
• #​4108
• #​4039
• #​4324

Ulysses sequence parallelism for long-context and video diffusion

Ulysses sequence parallelism gets its head-scatter / sequence-gather all-to-all as a public API, for video diffusion transformers and other long-sequence attention workloads. A fused NVLink-P2P kernel folds the layout permutation directly into the cross-GPU writes over CUDA IPC for a single coalesced push, with automatic NCCL fallback when P2P is unavailable.

• #​3820
• #​4240

What's Changed

• feat: close feature gap by wiring up per-tensor routed FP8 fused-moe by @​jdebache in #​4088
• Revert PR 4122 by @​jimmyzho in #​4171
• [GDN] improve sm100 GDN performance by @​Observer007 in #​4133
• fix(gdn): support WY decode on SM121 by @​kahyunnam in #​4117
• fix(norm): convert float2 to e4m3 directly in packed cast by @​elwhyjay in #​4167
• perf(test): bulk-precompile XQA decode kernels to cut test wall time ~4x by @​bkryu in #​4119
• [perf] Optimize TRT-LLM routing for high-expert, high-top-k workloads by @​jiahanc in #​4152
• feat(xqa): ragged Q and per-row sliding-window masking for speculative decode by @​yichengj0 in #​4137
• test(jit): assert BMM export symlink under GEN_SRC_DIR by @​kahyunnam in #​4187
• Feat/ulysses p2p a2a by @​forrestl111 in #​3820
• feat(moe_ep): MegaMoE framework integration ready: CUDA graph support, fused quant+stage launch, persistent knob cache, and prequantized weight packs by @​mhoqueanik in #​4079
• docs: document CuTe prefill scheduling override by @​kangbintNV in #​4162
• docs: add missing trtllm_fp8_per_tensor_scale_routed_moe API entry by @​kangbintNV in #​4175
• docs(mamba): document checkpointing varlen arguments by @​hebo1221 in #​4129
• Yanqinz/fix-gemm-and-grouped-mm-test-issue by @​yanqinz2 in #​4185
• feat(comm): extend trtllm_allreduce to SM12x and fix lamport buffer pointer packing by @​yichengj0 in #​3903
• feat(moe): add unified unpacked pre-routed FP4 mode by @​feih-nv in #​4104
• feat(mla): support packed low-head and variable-Q decode by @​PerkzZheng in #​4178
• bump version to 0.6.16 by @​jimmyzho in #​4142
• [fix]fix xqa flaky test on spark by @​qsang-nv in #​4161
• fix: make mxfp8 gemm test pass by having it quantize along the correct dimension by @​jdebache in #​3882
• fix(moe): serialize CuTe DSL autotune replay by @​zianglih in #​4192
• feat(moe_ep): fault-tolerance rank mask (NCCL-EP + NIXL-EP) by @​Anerudhan in #​4183
• fix(xqa): fix PDL load ordering and SM90 fp8 draft-mask dispatch by @​yichengj0 in #​4199
• feat(msa): accept K/V views split from a packed paged KV cache by @​yichengj0 in #​4039
• feat(moe): add TRTLLM MXFP4 W4A8 and W4A16 unified API support by @​feih-nv in #​4159
• [cli] add CLI helper for flashinfer-jit-cache and flashinfer-cubin wheel installs by @​dierksen in #​3142
• [feat] Add SITU trtllmgen MOE by @​jiahanc in #​4180
• feat(sm120): fused MoE (SwiGLU) via moe_gemm is_gated for cute SM120 groupwise GEMM by @​CarstyYou in #​4130
• fix(moe): pad trtllm-gen route map by one element to avoid OOB read by @​syuoni in #​4237
• perf(moe_ep): CuTe-DSL 4.5.2 mainloop WAR — drop the 4.6.1 runtime floor by @​mhoqueanik in #​4101
• Fix the routing inconsistency for num_groups > 1 by @​b8zhong in #​3946
• fix: support host global scale in CuTe-DSL NVFP4 quantization by @​akurathiswaraj in #​4138
• fix/test(moe_ep): self-bootstrap 1-rank process group in dg mega oracle test by @​mhoqueanik in #​4221
• feat: support native qk_rope_head_dim=0 sparse MLA decode in trtllm-gen by @​JustinTong0323 in #​4108
• feat(topk): support separate page table row starts by @​zianglih in #​4169
• feat(comm): make mixed-comm VMM workspaces checkpointable by @​galletas1712 in #​3910
• SM 107 Reland + Merge Back from v0.6.16 Release Branch by @​Vinnie6167 in #​4280
• test(sm103): fix FP4 autotuner cache inspection by @​tiffany940107 in #​4145
• bump version to 0.6.17 by @​aleozlx in #​4283
• feat(gemm): sync mm_fp4 SM120 NVFP4 dense GEMM kernel to b12x HEAD by @​yichengj0 in #​4253
• fix(b12x): correct fp4 quantization numerics and add input_global_scale to decouple weight and activation scales by @​yichengj0 in #​3932
• feat(moe): support shared expert fusion for trtllm-gen fp4 moe by @​Aneureka in #​4239
• fix(test): repair MoEFinalizeConfig call site in b12x unified MoE tests (#​4395) by @​aleozlx in #​4408
• revert(moe): revert #​3738 SM90 CUTLASS MoE backend (+ dependents #​4025, #​4080) on release-v0.6.17 by @​aleozlx in #​4411

New Contributors

• @​hebo1221 made their first contribution in #​4129
• @​akurathiswaraj made their first contribution in #​4138
• @​JustinTong0323 made their first contribution in #​4108

Full Changelog: flashinfer-ai/flashinfer@v0.6.16rc5...v0.6.17

v0.6.16.post4

Compare Source

v0.6.16.post4

Restores import flashinfer.comm on Python 3.10 and 3.11. A type annotation in
flashinfer/comm/fd_exchange.py evaluated only on Python 3.12+, and flashinfer.comm
imports that module at import time, so the package failed to import on interpreters
inside the supported range. Downstream packages that touch flashinfer.comm during
their own initialization were affected as well.

Upgrade to this release if you run FlashInfer on Python 3.10 or 3.11.

• #​4354

Full Changelog: flashinfer-ai/flashinfer@v0.6.16.post3...v0.6.16.post4

v0.6.16.post3

Compare Source

What's Changed

• revert(moe): revert #​3738 SM90 CUTLASS MoE backend (+ dependents #​4025, #​4080) on release-v0.6.16 by @​bkryu in #​4412

Full Changelog: flashinfer-ai/flashinfer@v0.6.16.post2...v0.6.16.post3

v0.6.16.post2

Compare Source

FlashInfer 0.6.16.post2 includes the tvm-ffi v0.1.13-post2 hotfix regarding its ABI compatibility. We recommend upgrading to this latest version.

Full Changelog: flashinfer-ai/flashinfer@v0.6.16.post1...v0.6.16.post2

v0.6.16.post1

Compare Source

⚠️ FlashInfer 0.6.16.post2 has picked up the latest tvm-ffi compatibility fix. We recommend upgrading to the latest version.

Full Changelog: flashinfer-ai/flashinfer@v0.6.16...v0.6.16.post1

v0.6.16

Compare Source

These highlights are also published at flashinfer.ai/releases.

v0.6.16 Highlights

This release delivers the MegaMoE kernels promised for expert parallelism in 0.6.15 and extends Blackwell model coverage — MiniMax-M3 sparse attention on Blackwell RTX and DGX Spark, and a unified MoE API that now executes per-tensor FP8, block-scale FP8, and B12x NVFP4/W4A16 — plus distributed serving under Confidential Computing with multicast-free all-reduce fusion. Under the hood, an on-disk JIT cache for CuTe-DSL kernels cuts cold-start compilation and shrinks the install.

⚠️ FlashInfer 0.6.16.post2 has picked up the latest tvm-ffi compatibility fix.
⚠️ FlashInfer 0.6.16.post4 adds a Python 3.10 compatibility hotfix.
We recommend upgrading to the latest 0.6.16 post-release.

MegaMoE kernels land in expert parallelism (moe_ep)

The MoEEpLayer now unifies the split and mega execution paths under one entry point and adds three mega backends — deep_gemm_mega (FP8/FP4), nvfp4_cutedsl, and mxfp8_cutedsl — each fusing expert-parallel communication with the local MoE in a single symmetric-memory kernel (Blackwell SM100+, NVSHMEM). In a microbenchmark, on a GB200 node (EP=4, DeepSeek-V3-like geometry) the tuned CuTeDSL NVFP4 backend with in-register FP4 combine reaches up to 1.89× the throughput of deep_gemm_mega at 8192 tokens/rank. NIXL-EP transport gaps and combine deadlocks blocking the vLLM Fleet/Handle adapters are also closed.

• #​3852
• #​3980
• #​4075
• #​4139

MiniMax Sparse Attention (MSA) on Blackwell RTX and DGX Spark

MiniMax-M3's MSA — a proxy/top-k indexer plus sparse prefill, decode, and combine — now runs on Blackwell RTX and DGX Spark (SM120/121) GPUs. The tensor-core kernels are rebuilt on SM12x warp-level mma.sync, top-k and combine are rewritten in CuTe-DSL, and an optional NVFP4 indexer is available, with prefill/decode accepting FP8 or NVFP4 KV (paged or flat). New APIs live under flashinfer.msa_ops.

• #​3655

XQA decode adds sliding-window, attention sinks, and ragged Q for speculative decode

The XQA decode kernel — used on Blackwell RTX and DGX Spark (SM120/121) for models with attention sinks — expands coverage for sliding-window attention and ragged Q, across causal and non-causal draft-block mask modes and combinations of them. Ragged Q lets each request in a batch verify a different number of draft tokens, and sliding-window masking is now computed per draft-token row, making the kernel useful for speculative-decoding workloads across models on SM120/121.

• #​4137

Unified MoE API reaches parity with the legacy FP8 and NVFP4 paths

The unified MoELayer API continues to expand quantization formats to reach parity with legacy flat APIs: TRTLLM per-tensor FP8 (SM100/SM103, with Llama4 routing-scale-on-input), DeepSeek FP8 and MXFP8 block-scale (SM100/SM103), and SM120/SM121 B12x NVFP4 and W4A16 backends. In-kernel routing (FromLogits) is now wired through the unified API and fuzzer, so precomputed and in-kernel routing share one path with CUDA-graph and autotuning coverage.

• #​4091
• #​4026
• #​3983
• #​3892

Confidential Computing: multicast-free all-reduce fusion and FP8 AllReduce

The TRT-LLM AllReduce-fusion workspace now allocates a multicast-free IPC workspace when NVIDIA Confidential Computing is detected (is_confidential_compute(), overridable via FLASHINFER_CONFIDENTIAL_COMPUTE), so one-shot Lamport and two-shot sync fusion run under CC where cuMulticast setup otherwise fails. Separately, a new flashinfer.comm.quantized_all_reduce() halves AllReduce transfer volume by quantizing activations to FP8 before P2P transfer over symmetric memory (SM90+, NVSwitch).

• #​3993
• #​3759

Faster JIT cold-start and smaller install

CuTe-DSL kernels now persist to an on-disk cache (JitSpec gains a JitSpecCuteDsl backend) and reload via JITLink in about 3–30 ms instead of recompiling in every new process; mm_fp4 autotuning additionally compiles tactics in parallel and reuses the shared disk cache, cutting autotune wall time. Pruning architecture gencode that dispatch can never load removes roughly 1.6 GB of installed size from the CUDA-13 aarch64 JIT-cache wheel.

• #​3874
• #​4029
• #​3947
• #​4073

What's Changed

• feat(comm): preserve all-reduce graph VAs across checkpoint restore by @​galletas1712 in #​3745
• fix(attention): clamp CTA_TILE_Q for fp8 h512 ragged prefill on 99KB-smem GPUs by @​qsang-nv in #​3863
• bump version to 0.6.15 by @​kahyunnam in #​3900
• fix(moe): fail loudly instead of IMA when no routing tier covers (numExperts, topK) by @​jinzh-nvidia in #​3840
• Add set_autotune_process_group to synchronize tactic choice across ranks by @​thanhhao98 in #​3187
• test(moe): align EP-offset tests and docs with global expert ids (gh #​3547) by @​waynehacking8 in #​3591
• docs: install flashinfer-cubin from flashinfer.ai index; drop cubin PyPI upload by @​aleozlx in #​3853
• Tune SM90 FP8 (e4m3) warp-specialized prefill by @​HongminTan in #​3760
• feat: E2E MoE LoRA integration with bgmv backend by @​JyChang012 in #​3746
• perf(moe): support caller-provided combine output by @​samnordmann in #​3776
• fix(attention): correct SM120 NVFP4 qk_correction layout, row-sum reduction, and lse by @​waynehacking8 in #​3838
• feat(gdn): add output-only BF16-state WY MTP decode kernel by @​ameynaik-hub in #​3720
• [Bugfix] Skip autotuning for small token counts on SM100 to prevent TMA crash by @​InfoSage05 in #​3837
• [CuTe DSL][MoE] Optimize PDL startup and balance GEMM2 partial-tile stores by @​dishengbin in #​3756
• fix (test): release CUDA cache before prefill compile subprocess by @​cindyzxq in #​3914
• [fix] fix autotuner non tensor guard by @​jiahanc in #​3918
• Add BF16 unified MoE conformance coverage by @​feih-nv in #​3867
• Fix: graph-safe uniform multi-token decode on FA2 tensor-core path by @​lucifer1004 in #​3871
• Add mm_fp8 into benchmark by @​sychen52 in #​3924
• feat(attention): enable SM121 (GB10/DGX Spark) for NVFP4 attention by @​bkryu in #​3897
• feat: MegaMoE Kernel integration in moe_ep by @​mhoqueanik in #​3852
• [feat] New CuTe DSL JIT kernels for FMHA: Dense & MXFP8 & NVFP4 formats by @​xrq-phys in #​3857
• fix(mla): allow native-width block tables for TRTLLM-GEN by @​yihwang-nv in #​3916
• Add FP8 groupwise MoE GEMM entry (cute SM120 backend) by @​CarstyYou in #​3891
• fix: autotuner memory leak follow up by @​jdebache in #​3912
• Add per-token NVFP4 quantization to CuTe DSL MoE by @​zianglih in #​3645
• test: unified GEMM/BMM fuzzer + convention auditor by @​YangXu1990uiuc in #​3539
• perf(attention): reuse TRT-LLM-gen counter buffers by @​saltyminty in #​3949
• perf(gemm): update mm_fp4 cute-dsl tactic heuristic and autotune over a top-N ranked list by @​bkryu in #​3948
• fix (test): mock MoE EP arch validation in split unit tests by @​cindyzxq in #​3964
• fix(cute_dsl): migrate off APIs removed in nvidia-cutlass-dsl 4.6 (keeps 4.5.x compat) by @​YangXu1990uiuc in #​3922
• Fix cuDNN batch prefill test by @​jhjpark in #​3784
• Update moe cubins to fix 2CTA hanging issue by @​IwakuraRein in #​3973
• perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 by @​StudyingShao in #​3738
• docs: add fused_moe LoRA delta APIs to rst by @​kangbintNV in #​3941
• docs: cover remaining @​flashinfer_api symbols missing from rst by @​kangbintNV in #​3942
• feat(attn): Enable MiniMax Sparse Attention (MSA) for Consumer Blackwell GPUs (SM120/121) by @​yichengj0 in #​3655
• feat(jit): JitSpec ABC + disk cache for JIT-compiled CuTe-DSL kernels by @​bkryu in #​3874
• Add BF16 weight preparation and unified fuzzer coverage by @​feih-nv in #​3940
• Yanqinz/autotuner tactic by @​yanqinz2 in #​3707
• chore: fix cute-dsl moe test by @​IwakuraRein in #​3979
• feat(moe_ep): restructure MegaMoE kernel sources into kernel_src + latest CuTeDSL MegaMoE kernels with improved performance by @​mhoqueanik in #​3980
• feat(attention): accept token-unit batch offsets on the cudnn prefill path by @​egilliam-nv in #​3921
• Disable TMA kernel for nvfp4 quantize by @​kahyunnam in #​3977
• perf: vectorize AppendPagedKVMlaCache to 128-bit access by @​aws-jiadingg in #​3524
• fix(autotuner): time tactics with %globaltimer under Confidential Computing by @​elvischenv in #​3870
• feat: expose trtllm-gen block-sparse attention for decode by @​PerkzZheng in #​3955
• feat(moe): wire in-kernel routing (FromLogits) into unified MoE API + routing axes in fuzzer by @​feih-nv in #​3892
• docs: fix 3 documentation issues from nightly doc-check by @​kangbintNV in #​3911
• fix(comm): restore MNNVL flags under inference mode by @​galletas1712 in #​3950
• fix(docs): mock optional quack dependency by @​kangbintNV in #​4021
• feat: enable fmha_v2 HMMA attention for SM120 standard shapes by @​blake-snc in #​3016
• Autotuner mem leak follow up 2 by @​jdebache in #​3970
• test(comm): add additional comm tests to multi-node CI by @​nvamyt in #​3968
• Ameyn/gdn wy perf followup by @​ameynaik-hub in #​3908
• fix(benchmark): don't fail non-trtllm mm_fp4 backends on unaligned n by @​sychen52 in #​4033
• perf(moe): optimize one-sided EP4 NVFP4 dispatch by @​samnordmann in #​3846
• perf(jit): drop dead SM12x gencode from SM10x-serving modules by @​yichengj0 in #​3947
• fix(cudnn): include attn scale in the prefill/decode graph-cache keys by @​YangXu1990uiuc in #​4014
• feat: SM100 CUTLASS NVFP4 SVDQuant fused GEMM (mm_nvfp4_svdquant) by @​jingyu-ml in #​3858
• fix(moe): restore legacy fused_moe.core imports for interleave helpers by @​StudyingShao in #​4025
• feat(moe): add deterministic CuTe DSL NVFP4 finalize mode by @​zianglih in #​3976
• fix(moe): allocate trtllm-gen expert_weights as bf16 in all launchers by @​aleozlx in #​3925
• feat(quantization): support configurable scale-factor layouts for MXFP4 by @​huangzhilin-hzl in #​3927
• test(comm): make checkpoint worker importable by @​galletas1712 in #​4034
• feat(moe): add B12x backends to unified MoE API and add corresponding tests by @​Aneureka in #​3983
• feat: trtllm FP4 routed-MoE: accept fp32 topk_weights (copy-free) by @​jdebache in #​3763
• feat: CuTe-DSL Fused SwiGLU NVFP4 Quantization by @​philipphack in #​4040
• docs: fix MSA and GDN doc-check findings by @​kangbintNV in #​4058
• Yanqinz/fix autotuner using autotuning plan from other runners by @​yanqinz2 in #​4004
• perf(jit): prune arch gencode that dispatch can never load by @​yichengj0 in #​4073
• mamba checkpointing SSU: two-kernel split + ring-buffer cache for checkpointing SSU by @​ishovkun in #​3975
• perf(kda): use a one-warp/grouped recurrent decode hybrid by @​djmmoss in #​4001
• perf(gemm): Improve mm_fp4 cute-dsl autotune time via disk-cache and parallel compilation by @​bkryu in #​4029
• feat: add optional normed-output (y_out) to add_rmsnorm_fp4quant by @​nv-guomingz in #​4072
• Code review guidance proposal by @​aleozlx in #​3790
• Port the TensorRT-LLM one-sided A2A optimizations to Flashinfer by @​b8zhong in #​3697
• feat(moe): add unified block-scale FP8 support by @​feih-nv in #​4026
• feat: allow using NVL a2a kernels without MNNVL as long as all peers are within the same node / intra-node NVL domain by @​jdebache in #​3701
• [feat] Add TRTLLM/SageAttention quantization routine by @​xrq-phys in #​3982
• fix: Fix the MOE tests to properly scale NVFP4 bias by @​djns99 in #​3755
• feat(moe): support caller-owned CUTLASS MoE workspace by @​feih-nv in #​4057
• feat: add collect-env environment report tool + issue templates (bug, feature request) by @​YangXu1990uiuc in #​4007
• feat(gdn): index GDN prefill state pool via state_indices (SM100/SM103) by @​nv-guomingz in #​4084
• fix(fused_moe): avoid repeated runner setup in host dispatch by @​YAMY1234 in #​4045
• feat(moe_ep): close the nixl_ep transport gaps blocking the vLLM Fleet/Handle adapters by @​Anerudhan in #​4075
• fix: keep HEAD_DIM 256 prefill within 64 KiB smem on SM75/SM70 by @​HuChundong in #​3526
• fix(gemm): prevent out-of-bounds read in MaskedScheduler by @​dashanji in #​4046
• fix: bug-report issue template silently dropped by chooser (description over length limit) by @​YangXu1990uiuc in #​4100
• perf(sm103): optimize NVFP4 epilogue stores and scaling by @​tiffany940107 in #​4063
• test: compute mm_fp4 cosine similarity in fp32 to avoid bf16 rounding cliff by @​Adnios in #​4105
• fix(comm): sync all sender threads before publishing the final FIFO step to fix the intermittent race issue by @​Aneureka in #​4092
• fix: Add missing nvfp4 sizing branch in getProfilerWorkspaces by @​jimmyzho in #​4080
• docs: fix flashinfer doc check findings by @​kangbintNV in #​4097
• Revert "[Bugfix] Skip autotuning for small token counts on SM100 to prevent TMA crash" by @​b8zhong in #​4041
• fix(trtllm-gen): select dynamic-page decode kernels for MHA (numHeadsQPerKv == 1) by @​syuoni in #​4095
• fix: move JIT symlinks to writable gen dir by @​namgyu-youn in #​3468
• feat(comm): Add FP8 quantized two-shot AllReduce via symmetric memory by @​kailashbuki in #​3759
• fix: let FLASHINFER_CUBIN_DIR env var override flashinfer-cubin package path by @​ianliuy in #​3062
• feat(moe): add unified per-tensor FP8 support by @​feih-nv in #​4091
• Update MoE CODEOWNERS by @​feih-nv in #​4127
• comm: multicast-free (IPC) workspace for TRT-LLM AR fusion under Confidential Computing by @​elvischenv in #​3993
• feat: fused FP8 quantization output for LayerNorm by @​elwhyjay in #​3962
• chore: Add Anerudhan to comm codeowner by @​bkryu in #​4135
• CuTe-DSL modular prefill: sliding-window support + band-mask performance by @​pgera in #​3952
• fix(gdn): compile SM12x CuteDSL kernels as sm_121a on DGX Spark by @​nv-tusharma in #​3960
• fix(moe_ep): fix two NIXL-EP combine deadlocks under real serving load by @​Anerudhan in #​4139
• Adds SM107 support by @​Vinnie6167 in #​4122
• Split trtllm-gen cubin pins per arch by @​Vinnie6167 in #​4191
• perf(sampling): gate launch_bounds to SM107 only (NVBug 6517769) by @​kahyunnam in #​4189
• release-v0.6.16: cherry-pick #​4167, #​4137, #​4187, #​4178 and bump to 0.6.16rc2 by @​aleozlx in #​4197
• release-v0.6.16: fix cubin checksum collision and unguarded Sm107a build break by @​aleozlx in #​4200
• fix(moe): reject incompatible output-scale cubins by @​aleozlx in #​4213
• fix(jit): pass map_sm107_to_100f in gen_moe_utils_module by @​aleozlx in #​4215
• test: fix #​4191 regressions in autotuner_core and symlink race test by @​kahyunnam in #​4225
• cherry-pick: #​4145 test(sm103): fix FP4 autotuner cache inspection by @​kahyunnam in #​4231
• cherry-pick: #​4199 fix(xqa): PDL load ordering and SM90 fp8 draft-mask dispatch by @​kahyunnam in #​4232
• cherry-pick: #​3882 fix mxfp8 gemm quantization / scale-layout warnings by @​kahyunnam in #​4233
• fix(trtllm): restrict routed-MoE backends to supported architectures (release port for #​4107) by @​kahyunnam in #​4230
• fix(moe): name-filter output-scale-incompatible cubins until pinned packages carry mDtypeSfC by @​Vinnie6167 in #​4235
• Make CuTe-DSL arch guard env-aware and gate norm's DSL dispatch on it by @​Vinnie6167 in #​4226
• chore: bump version to 0.6.16rc4 by @​kahyunnam in #​4236
• cherry-pick: #​4129 and #​4162 by @​jimmyzho in #​4245
• revert: #​3975 and #​4129 (mamba checkpointing ssu 2-split + doc) by @​jimmyzho in #​4246
• cherry-pick: #​4185 (for issue 4090) by @​jimmyzho in #​4247
• cherry-pick: #​4152 ([perf] Optimize TRT-LLM routing for high-expert, high-top-k workloads) by @​jimmyzho in #​4251
• test: skip NVFP4-KV decode tests on SM107 by @​Vinnie6167 in #​4257
• artifacts: add DEEPGEMM_RUBIN artifact path and checksum by @​jimmyzho in #​4258
• deep_gemm: arch-based split for DEEPGEMM_RUBIN cubins by @​jimmyzho in #​4261
• cherry-pick: #​4180 ([feat] Add SITU trtllmgen MOE) + conflict resolution by @​jimmyzho in #​4252
• chore: bump version to 0.6.16rc5 by @​jimmyzho in #​4264
• cherry-pick: #​4237 (fix(moe): pad trtllm-gen route map by one element to avoid OOB read) by @​jimmyzho in #​4288
• bump version to 0.6.16 by @​jimmyzho in #​4289

New Contributors

• @​jinzh-nvidia made their first contribution in #​3840
• @​thanhhao98 made their first contribution in #​3187
• @​HongminTan made their first contribution in #​3760
• @​InfoSage05 made their first contribution in #​3837
• @​dishengbin made their first contribution in #​3756
• @​cindyzxq made their first contribution in #​3914
• @​YangXu1990uiuc made their first contribution in #​3539
• @​egilliam-nv made their first contribution in #​3921
• @​nvamyt made their first contribution in #​3968
• @​jingyu-ml made their first contribution in #​3858
• @​nv-guomingz made their first contribution in #​4072
• @​HuChundong made their first contribution in #​3526
• @​dashanji made their first contribution in #​4046
• @​Adnios made their first contribution in #​4105
• @​namgyu-youn made their first contribution in #​3468
• @​kailashbuki made their first contribution in #​3759
• @​nv-tusharma made their first contribution in #​3960

Full Changelog: flashinfer-ai/flashinfer@v0.6.15rc4...v0.6.16

v0.6.15.post1

Compare Source

Full Changelog: <h

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.