Repository navigation
Conversation
run_examples.sh always enables reference-cpu, which pulls in candle.
candle's gemm-f16 uses FP16 arithmetic instructions (fmla .8h) that the
generic aarch64 target does not enable, so on aarch64 every example fails
to build ("instruction requires: fullfp16"), not only the two that need
the CPU reference.
When the CPU reports asimdhp, add -C target-feature=+fp16 to RUSTFLAGS:
only that extension, so the rest of the build stays generic. Without
asimdhp, drop reference-cpu and skip async_gemm and flash_attention.
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/run_examples.shalways enables--features reference-cpu,experimental-tune.reference-cpupulls in candle, and candle'sgemm-f16uses FP16 arithmetic instructions (fmla v0.8h, v1.8h, v2.h[0]). The generic aarch64 target does not enable those. On aarch64, the wholecutile-examplespackage therefore fails to build, and every example fails, not only the two that need the CPU reference:cutile-examples/Cargo.tomlalready documents the problem (DGX Spark) and suggestsRUSTFLAGS="-C target-cpu=native"for manual runs, but the runner script does not apply it.Changes
run_examples.sh, on aarch64 when the features includereference-cpu:/proc/cpuinfolistsasimdhp, append-C target-feature=+fp16toRUSTFLAGS, keeping any value the caller set;reference-cpuand skip the two examples that require it (async_gemm,flash_attention).Nothing changes on x86_64.
Why
+fp16rather than-C target-cpu=native:Testing
GH200 (aarch64, Neoverse V2), CUDA 13.4 tile toolchain:
The 3 skips in both runs are
mxfp8,nvfp4andnvfp4_moe, which the examples themselves skip below sm_100. Before this change, the same machine reported 28 examples failed and 0 passed.bash -n scripts/run_examples.shChecklist
git commit -s)