Benchmark comparing Flash-Decoding-style chunked KV-parallel attention against standard SDPA at decode time (q_len=1) on RTX 2070, measuring the crossover point where KV-parallelism outperforms fused attention kernels.
benchmarking cuda pytorch transformer decode attention kv-cache long-context llm-inference flash-attention flash-decoding mlsystems
-
Updated
Jul 19, 2026 - Python