Skip to content

ci: compare Nx 24 full cache bust without task packing flags - #8809

Closed
beaussan wants to merge 4 commits into
mainfrom
beaussan/ci-cache-bust-env-comparison
Closed

beaussan wants to merge 4 commits into
mainfrom
beaussan/ci-cache-bust-env-comparison

Conversation

@beaussan

@beaussan beaussan commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Changes

Run a full-cache-bust CI timing experiment without the six task-packing/assignment environment flags added in #8807. Remove their obsolete comment and set a unique nx.json bust value; nx.json is in sharedGlobals for cacheable targets.

Compare this run with #8807 at 29be168, its current non-Ultracache head. Nx 24.0.0, task-based affected selection, application code, worker allocation, NX_CLOUD_ACCESS_TOKEN, and SERVER_PRESET are unchanged from that commit. Ultracache remains disabled. This branch starts from that commit; the experimental delta is limited to .github/workflows/pr.yml and nx.json.

The first #8807 head (9cd1fa67ed, run 40129619367) is a separate, confounded historical baseline: it used Ultracache and legacy affected selection. That run was green: Test 14m04s (Run Checks 11m59s, Nx startup 36s, tool setup 1m07s), Preview 5m10s. Do not use it for the controlled comparison.

AI disclosure: implemented and validated with Codex assistance. This is a draft experiment; do not merge either PR as part of this comparison.

Validation: Node 26.12.1, pnpm 11.21.0; frozen-lockfile install, pnpm format, targeted Prettier checks, parsed workflow/JSON and exact-delta assertions, shared-input coverage, and git diff --check. Application tests do not apply to this configuration-only experiment.

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested code changes locally with the relevant test commands, or tests do not apply to this pull request.
  • I fully understand the code in this pull request, including any code generated with AI assistance.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Timing comparison

Both runs use Nx 24, task-based affected selection, a fresh workspace cache bust, and the same code/worker configuration.

  • With the six flags (PR ci: upgrade Nx and migrate worker configuration #8807, 29be168): Test job 12m 09s, Run Checks 10m 37s, Nx execution 10m 26s; passed.
  • Without the flags (this PR, 122292b): Test job 11m 52s, Run Checks 10m 44s, Nx execution 10m 35s; failed.
  • Without flags, the Test job was 17s shorter, while Run Checks was 7s longer. Setup Tools was 18s shorter. This single comparison does not establish a task-packing speedup.

Failure: tanstack-react-start-e2e-hmr:test:e2e--vite-ssr, “picks up latest child loader when navigating after multiple HMR saves”: page.waitForFunction exceeded 20 seconds (tests/app.spec.ts:919, helper at line 376). The run completed 863 tasks successfully, failed one, and skipped its dependent aggregate task. No retry or test change was made, and the failure is not attributed to the environment flags.

With-flags run · Without-flags run

Cache caveat: the with-flags Test run reported 41 cache hits, while the no-flags Test run reported 14. All 41 cached task names match package builds executed as cache misses by the parallel Preview job in the with-flags workflow. This is consistent with same-workflow cache warming, even though the initial hashes were busted. Therefore these are not equivalent cold-cache Test runs, and the timing comparison cannot isolate the flags. The reported task totals also differ (875 versus 865); no speedup conclusion is established.

Controlled follow-up

Fresh cache bust on both PRs, with NX_NO_CLOUD=true on Preview so it cannot warm the distributed Test cache. Test/Preview check out the exact PR head, and Test uses the same fixed base (6b9fe6d29552270836c511bc9b343d73541f4424) to prevent main-branch changes from altering the workload. The six task-packing flags remain the only workflow difference between the two PRs.

Nx Cloud Public API investigation of the previous runs: complete lists for 2 CIPEs, 2 run groups, 4 runs, 32 workers, and 1,740 task records, scoped to 2026-10-24 20:45–20:55 UTC. Both distributed runs used 16 linux-medium-js workers. Recorded task intervals show ~345 aggregate worker-seconds above three simultaneous noncached tasks with flags, versus ~7 without; this is evidence of increased concurrency, not proof of scheduler configuration or a speedup. E2E tasks account for about 55%/50% of aggregate recorded task time. Local resolved e2e targets set parallelism: false, which may limit the benefit; this hypothesis needs the controlled follow-up.

Clean comparison results

Both runs succeeded with zero cache hits and the same 856 unique task IDs. Checkout and Run Checks logs confirm exact branch-head checkouts and the common NX_BASE; the CIPE VCS metadata still reports each PR’s natural merge base.

  • With flags (9d5d9cc): Test job 12m 57s; Run Checks 11m 34s; Nx command 11m 26s. CIPE.
  • Without flags (3335538): Test job 12m 54s; Run Checks 11m 37s; Nx command 11m 18s. CIPE.

The Public API returned 880/868 task execution records, with the same 856 unique task IDs. The differing totals are multiple batch records for hydration benchmark builds, not different sets of selected tasks. For concurrency analysis, exclude both hydration build IDs that have repeated records in either run, cache hits, and intervals <=0.2s. With flags, 15 of 16 workers reached 4–7 overlapping tasks and accumulated 304.2 worker-seconds above concurrency three. Without flags, every worker peaked at three, with zero worker-seconds above three. This supports packing changing execution concurrency; it does not establish a performance benefit.

E2E accounts for about half the remaining recorded task time; locally resolved E2E targets and their application builds use parallelism: false. This is a plausible constraint on the benefit, not a proven explanation of the entire wall time. One clean run per configuration shows no meaningful improvement and is insufficient to generalize. No retries, test fixes, or further scheduler changes were applied.

API coverage: CIPEs created 2026-10-24 21:00–21:25 UTC, filtered to these two PRs; returned links followed through run groups and completed distributed runs; complete task pagination (2 pages each, final cursor null), 1,748 task records total. The no-flags worker-list endpoint temporarily returned 409 not_terminal during workflow shutdown; the task-record comparison above uses the completed runs and does not depend on that unavailable worker resource data.

@changeset-bot

changeset-bot Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 3335538

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit 3335538

Command Status Duration Result
nx affected --targets=test:eslint,test:unit,tes... ✅ Succeeded 11m 18s View ↗

☁️ Nx Cloud last updated this comment at 2026-10-24 21:28:32 UTC

@socket-security

socket-security Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednx@​23.2.0 ⏵ 24.0.067 -1100 +292100100
Updated@​nx/​devkit@​23.2.0 ⏵ 24.0.099 +11009299100

View full report

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Changeset Version Preview

No changeset entries found. Merging this PR will not cause a version bump for any packages.

@github-actions

Copy link
Copy Markdown
Contributor

Bundle Size Benchmarks

This pull request does not affect bundle size in any measured scenario.

@pkg-pr-new

pkg-pr-new Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
More templates

@tanstack/arktype-adapter

npm i https://pkg.pr.new/@tanstack/arktype-adapter@8809

@tanstack/eslint-plugin-router

npm i https://pkg.pr.new/@tanstack/eslint-plugin-router@8809

@tanstack/eslint-plugin-start

npm i https://pkg.pr.new/@tanstack/eslint-plugin-start@8809

@tanstack/history

npm i https://pkg.pr.new/@tanstack/history@8809

@tanstack/react-router

npm i https://pkg.pr.new/@tanstack/react-router@8809

@tanstack/react-router-devtools

npm i https://pkg.pr.new/@tanstack/react-router-devtools@8809

@tanstack/react-router-ssr-query

npm i https://pkg.pr.new/@tanstack/react-router-ssr-query@8809

@tanstack/react-start

npm i https://pkg.pr.new/@tanstack/react-start@8809

@tanstack/react-start-client

npm i https://pkg.pr.new/@tanstack/react-start-client@8809

@tanstack/react-start-rsc

npm i https://pkg.pr.new/@tanstack/react-start-rsc@8809

@tanstack/react-start-server

npm i https://pkg.pr.new/@tanstack/react-start-server@8809

@tanstack/router-cli

npm i https://pkg.pr.new/@tanstack/router-cli@8809

@tanstack/router-core

npm i https://pkg.pr.new/@tanstack/router-core@8809

@tanstack/router-devtools

npm i https://pkg.pr.new/@tanstack/router-devtools@8809

@tanstack/router-devtools-core

npm i https://pkg.pr.new/@tanstack/router-devtools-core@8809

@tanstack/router-generator

npm i https://pkg.pr.new/@tanstack/router-generator@8809

@tanstack/router-plugin

npm i https://pkg.pr.new/@tanstack/router-plugin@8809

@tanstack/router-ssr-query-core

npm i https://pkg.pr.new/@tanstack/router-ssr-query-core@8809

@tanstack/router-utils

npm i https://pkg.pr.new/@tanstack/router-utils@8809

@tanstack/router-vite-plugin

npm i https://pkg.pr.new/@tanstack/router-vite-plugin@8809

@tanstack/solid-router

npm i https://pkg.pr.new/@tanstack/solid-router@8809

@tanstack/solid-router-devtools

npm i https://pkg.pr.new/@tanstack/solid-router-devtools@8809

@tanstack/solid-router-ssr-query

npm i https://pkg.pr.new/@tanstack/solid-router-ssr-query@8809

@tanstack/solid-start

npm i https://pkg.pr.new/@tanstack/solid-start@8809

@tanstack/solid-start-client

npm i https://pkg.pr.new/@tanstack/solid-start-client@8809

@tanstack/solid-start-server

npm i https://pkg.pr.new/@tanstack/solid-start-server@8809

@tanstack/start-client-core

npm i https://pkg.pr.new/@tanstack/start-client-core@8809

@tanstack/start-fn-stubs

npm i https://pkg.pr.new/@tanstack/start-fn-stubs@8809

@tanstack/start-plugin-core

npm i https://pkg.pr.new/@tanstack/start-plugin-core@8809

@tanstack/start-server-core

npm i https://pkg.pr.new/@tanstack/start-server-core@8809

@tanstack/start-static-server-functions

npm i https://pkg.pr.new/@tanstack/start-static-server-functions@8809

@tanstack/start-storage-context

npm i https://pkg.pr.new/@tanstack/start-storage-context@8809

@tanstack/valibot-adapter

npm i https://pkg.pr.new/@tanstack/valibot-adapter@8809

@tanstack/virtual-file-routes

npm i https://pkg.pr.new/@tanstack/virtual-file-routes@8809

@tanstack/vue-router

npm i https://pkg.pr.new/@tanstack/vue-router@8809

@tanstack/vue-router-devtools

npm i https://pkg.pr.new/@tanstack/vue-router-devtools@8809

@tanstack/vue-router-ssr-query

npm i https://pkg.pr.new/@tanstack/vue-router-ssr-query@8809

@tanstack/vue-start

npm i https://pkg.pr.new/@tanstack/vue-start@8809

@tanstack/vue-start-client

npm i https://pkg.pr.new/@tanstack/vue-start-client@8809

@tanstack/vue-start-server

npm i https://pkg.pr.new/@tanstack/vue-start-server@8809

@tanstack/zod-adapter

npm i https://pkg.pr.new/@tanstack/zod-adapter@8809

commit: 3335538

@beaussan beaussan closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant