Skip to content

fix(cutile): build the reference-cpu examples on aarch64 - #1636

Open
0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:fix/cutile-examples-aarch64-fp16
Open

0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:fix/cutile-examples-aarch64-fp16

Conversation

@0z5a

@0z5a 0z5a commented Oct 24, 2026

Copy link
Copy Markdown

Summary

scripts/run_examples.sh always enables --features reference-cpu,experimental-tune. reference-cpu pulls in candle, and candle's gemm-f16 uses FP16 arithmetic instructions (fmla v0.8h, v1.8h, v2.h[0]). The generic aarch64 target does not enable those. On aarch64, the whole cutile-examples package therefore fails to build, and every example fails, not only the two that need the CPU reference:

error: instruction requires: fullfp16
    --> …/gemm-common-0.19.0/src/simd.rs:1994:18

cutile-examples/Cargo.toml already documents the problem (DGX Spark) and suggests RUSTFLAGS="-C target-cpu=native" for manual runs, but the runner script does not apply it.

Changes

run_examples.sh, on aarch64 when the features include reference-cpu:

  • if /proc/cpuinfo lists asimdhp, append -C target-feature=+fp16 to RUSTFLAGS, keeping any value the caller set;
  • otherwise, drop reference-cpu and skip the two examples that require it (async_gemm, flash_attention).

Nothing changes on x86_64.

Why +fp16 rather than -C target-cpu=native:

  • It enables only the extension candle needs, so the rest of the build stays generic, and the binaries run on any aarch64 CPU with FP16.
  • The fallback keeps the other examples running on CPUs without FP16.

Testing

GH200 (aarch64, Neoverse V2), CUDA 13.4 tile toolchain:

cargo build --examples --features reference-cpu,experimental-tune                         → error: instruction requires: fullfp16
RUSTFLAGS="-C target-feature=+fp16" cargo build --examples --features reference-cpu,…      → ok
cargo build --examples --features experimental-tune                                       → ok

scripts/run_examples.sh (asimdhp present):              ✓ 25 examples passed, 3 skipped
scripts/run_examples.sh with the asimdhp check forced off (fallback path):
  Skipping example: async_gemm
  Skipping example: flash_attention
  ✓ 23 examples passed, 5 skipped

The 3 skips in both runs are mxfp8, nvfp4 and nvfp4_moe, which the examples themselves skip below sm_100. Before this change, the same machine reported 28 examples failed and 0 passed.

  • bash -n scripts/run_examples.sh
  • examples run on GH200

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files -- no new files

run_examples.sh always enables reference-cpu, which pulls in candle.
candle's gemm-f16 uses FP16 arithmetic instructions (fmla .8h) that the
generic aarch64 target does not enable, so on aarch64 every example fails
to build ("instruction requires: fullfp16"), not only the two that need
the CPU reference.

When the CPU reports asimdhp, add -C target-feature=+fp16 to RUSTFLAGS:
only that extension, so the rest of the build stays generic. Without
asimdhp, drop reference-cpu and skip async_gemm and flash_attention.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@0z5a
0z5a requested a review from elibol as a code owner October 24, 2026 20:35
@copy-pr-bot

copy-pr-bot Bot commented Oct 24, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant