Repository navigation
Conversation
Resolve wrapper language-model paths and nested text configuration through existing presets. Preserve sorted top-k scores in the gate projection dtype under autocast and return raw router logits. Native Transformers model oracle: text backbone and wrapper, with and without BF16 autocast. Standalone suite: 196 passed, 6 CUDA-only skips. Changed-file pre-commit passed. Signed-off-by: hazelduan <duanxueqi@outlook.com>
Remove redundant local imports after moving the native model oracle into the AutoEP suite. All four parity cases and changed-file pre-commit pass. Signed-off-by: hazelduan <duanxueqi@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AutoEP rejects the Qwen3.5 MoE wrapper and diverges from its native router semantics. This change accepts text input through both the text backbone and the wrapper's language-model path, reads nested text configuration, and preserves native sorted top-k routing, gate-projection score dtype (including BF16 autocast with FP32 inputs), and raw router logits.
It uses the existing preset/spec/router extension points, with defaults preserving other presets. The regression compares Transformers' native models for both classes, with and without CPU BF16 autocast. Vision input remains untested.
Split from #8784; no HiFloat8 or NPU permutation dependency. Related RFC: #8777. Base: 3b80932; initial validated source: c0eb9e1. An open-PR search for Qwen3.5 returned no match on 2026-10-14.
Validation:
PYTHONPATH=<isolated-qwen-repo> python -m pytest tests/unit/v1/moe/test_autoep_unit.py -q -rs.The private combined-stack validation also completed Qwen3.5-35B-A3B text-only EP4/ZeRO2 for 30 matched steps. All 40 layers execute; only routed/shared experts in layers 0/39 train (12 tensors/rank), batch 1/GAS1, sequence length 256, eager attention/experts, no activation checkpointing or checkpoint save, seed 42, AdamW 1e-5/cosine, same model weights and 400-row data. Every rank has exact input hashes, loss, gradient norms, native operator counts and all final trainable tensors. Maximum-rank steady-step median (excluding 5 warm-up steps) is 4.1886s baseline vs 4.1906s candidate (+0.046%); training NPU peak is 33,487,014,400 bytes and initialization peak is 86,776,612,352 bytes in both. This combined-stack result does not attribute an independent speedup or numerical improvement to this split.
Native commands and logs are retained at
/data/dxq/hif8_a5_20260914/experiments/quantize_convergence_20261014_01:bash native_gates.sh,bash paired_ep4.sh,qwen_native_stack.log,permutation_native.log,comparison_ep4_baseline_r1_ep4_candidate_r1.json. The runner bind-mounts the original Conda environment (dxq_hif8_torchao_py311) and CANN/driver; it changes no global packages or device permissions. Standalone runtime file hashes match each immutable source revision below. The model run uses a private composition of main PR and split runtime hunks, with the overlapping dispatch hunks reconciled; this composition is not the main PR contents.Standalone model parity covers BF16 text inference at EP1 with eager attention and sequential experts. Vision, FP16, CUDA-native parity, ZeRO/offload combinations, activation checkpointing, AutoTP, LoRA and multi-node EP remain untested by this split.
AI assistance was used to implement and review this change; the human submitter owns review and validation.
Final published source: d8045b1. Post-push cleanup only removes three duplicate local imports; all four parity cases pass again on macOS and in the A5 software environment. Changed-file pre-commit passes. Runtime source is identical to the initial validated revision.