Skip to content

ci: share Rust caches across runner images - #3520

Merged
kevinjqliu merged 2 commits into
apache:mainfrom
kevinjqliu:kevinjqliu-public-api-check-without-cargo-install
Oct 26, 2026
Merged

kevinjqliu merged 2 commits into
apache:mainfrom
kevinjqliu:kevinjqliu-public-api-check-without-cargo-install

Conversation

@kevinjqliu

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • N/A (CI maintenance).

What changes are included in this PR?

rust-cache hashes every installed toolchain into its cache key, including the runner's preinstalled stable. We build with the nightly in rust-toolchain.toml and never use stable. But runner images roll out with different stable versions (1.99.0 vs 1.99.1 this week, on Linux, macOS and Windows), so each job ends up with 2 cache keys that can't restore from each other.

What that looks like over ~45 recent runs per workflow:

  • The less common image misses a lot more. macOS build missed 4/4, Windows bindings 7/10, check-public-api 64% vs 37%.
  • A miss rebuilds every dependency. For check-public-api that's ~6 min vs ~1 min.
  • The duplicate entries take ~2.8GB. We're at 10.7GB of the 10GB cache limit, so they push out caches we still need.

Remove stable in setup-builder when it uses rust-toolchain.toml. Jobs that pass rust-version (audit, MSRV, publish, Python release) are unchanged. rust-toolchain.toml is still part of the key, so a nightly bump still invalidates the cache.

Apache Iggy does the same in its setup action.

Are these changes tested?

CI only. actionlint and zizmor --offline show no new findings. Checked locally that rustup toolchain uninstall stable exits 0 when stable is the default toolchain and when it's not installed. The hit rate only shows after this lands on main and the caches are re-saved.

AI Disclosure

Prepared with Copilot, including pulling the cache hit/miss numbers from run logs. Worth a look: that nothing in these jobs calls cargo +stable or relies on stable being installed.

rust-cache hashes every installed toolchain into its key. Runner images
roll out with different preinstalled stable versions (1.99.0 vs 1.99.1),
which splits each job's cache into two keys that miss each other and
duplicate ~2.8GB of cache storage. Jobs using rust-toolchain.toml never
use stable, so setup-builder removes it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@kevinjqliu
kevinjqliu marked this pull request as ready for review October 24, 2026 04:25
@kevinjqliu
kevinjqliu requested review from CTTY and blackmwk and a balanced review from Copilot and removed request for Copilot October 24, 2026 04:25
@kevinjqliu

Copy link
Copy Markdown
Contributor Author

this should improve cache hits and make ci faster

@blackmwk
blackmwk added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 26, 2026
Copilot AI balanced review requested due to automatic review settings October 26, 2026 17:40
@kevinjqliu

Copy link
Copy Markdown
Contributor Author

rejected by merge queue due to teh asf-allowlist-check, gh-action-pypi-publish needed a new version (fixed by #3531)

@kevinjqliu
kevinjqliu enabled auto-merge October 26, 2026 17:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

1 open issue
What changed in this PR

Updates the composite CI action to improve rust-cache hit rates by avoiding cache key divergence caused by runner-preinstalled stable toolchain versions.

Changes:

  • Uninstalls the stable toolchain when using rust-toolchain.toml so caches are keyed only by the toolchain the repo actually uses.
  • Adds inline rationale explaining why stable is removed (runner image variance and cache key splitting).
File Description
.github/​actions/​setup-builder/​action.yml Removes stable toolchain to keep rust-cache keys consistent across runner images.

Review effort: Lite

Comment on lines 44 to +46
rustup component add rustfmt clippy
# Unused, and its version varies by runner image, which splits rust-cache keys.
rustup toolchain uninstall stable
@kevinjqliu
kevinjqliu added this pull request to the merge queue Oct 26, 2026
Merged via the queue into apache:main with commit cfa8b9c Oct 26, 2026
23 of 25 checks passed
@kevinjqliu
kevinjqliu deleted the kevinjqliu-public-api-check-without-cargo-install branch October 26, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants