Sync with Microsoft ONNX Runtime - 28082026 - #1272
Merged
Merged
Conversation
### Description <!-- Describe your changes. --> Fix ort-ci/SKILL.md yaml frontmatter syntax. `.github\skills\ort-ci\SKILL.md: failed to parse YAML frontmatter: mapping values are not allowed in this context at line 2 column 385` There's a literal colon in the description that it doesn't like. Made the string a block scalar literal instead. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Fix yaml syntax.
…2263) Extend the paged-KV XQA decode path to consume a native FP16 KV cache for the `head_size=256`, group-size-6 geometry used by Qwen3.8 full-attention layers. - Add an FP16 query/output + FP16 paged-KV H256 XQA instantiation. - Select it only for metadata-backed one-token-per-sequence decode on SM80+. - Reuse the existing 128-token XQA page mapping for PagedAttention block sizes divisible by 128. - Preserve paged Flash Attention for prefill, ragged/multi-token decode, metadata-free callers, unsupported devices, disabled XQA, BF16, and other geometries. - Keep the runtime dynamic shared-memory fit check authoritative. - Add CPU-reference numerical coverage and verify XQA dispatch for the Qwen group-size-6 shape. On an A100-SXM4-80GB with Qwen3.8 27B, the native FP16 XQA kernel reduces B1 x 64K attention time from 0.764 ms to 0.176 ms per full-attention layer per token. End-to-end eager decode improves from 32.9 to 47.5 tok/s (+44%); stable CUDA graphs reach 52.6 tok/s. The 128-token 64K output matches the existing FP16 Flash path token-for-token. | Batch / context | FP16 Flash | FP16 XQA | Gain | |---|---:|---:|---:| | B1 x 64K, eager | 32.9 tok/s | 47.5 tok/s | +44% | | B8 x 8K, eager | 143.5 | 171.3 | +19% | | B8 x 32K, eager | 86.3 | 144.3 | +67% | | B8 x 64K, eager | 43.2 | 77.7 | +80% |
ai-fw-intg
requested review from
Jaswanth51,
ankitm3k,
jatinwadhwa921 and
vthaniel
August 27, 2026 20:36
hdharpure9922
self-requested a review
August 28, 2026 07:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated daily backmerge from ORT main to ovep-develop. No conflicts detected. Do NOT squash or rebase - use merge commit only.