Skip to content

perf(ci): overlap independent steps and reduce cache overhead - #6072

Merged
fengmk2 merged 2 commits into
mainfrom
perf/parallel-ci-setup
Oct 15, 2026
Merged

fengmk2 merged 2 commits into
mainfrom
perf/parallel-ci-setup

Conversation

@fengmk2

@fengmk2 fengmk2 commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

CI waits for independent work and repeats cache work across jobs. These waits delay tests.

Run service startup, tool setup, and cache restoration in parallel. Compute the dependency hash once before installation. Use one writer per shared cache key, and skip saves after exact cache hits.

Start ecosystem checkout and coverage downloads early. Wait where their results are first required. Upload main-suite results while example tests run. Build packages and the site in parallel after static checks.

Tests still wait for service startup. Windows Chocolatey installations remain sequential because they share installation state.

This change applies the approaches from Oxc #27476 and Oxc #27482.

The baseline CI run, setup/cache CI run, and current CI run use the same PR matrix.

Measurement Baseline Setup/cache changes Current
CI workflow elapsed 13m36s 12m07s 12m57s
CI runner time, all jobs 92m52s 83m25s 89m12s
Main-test setup, sum of 9 jobs 8m52s 6m51s 6m45s
Tool jobs, sum of 10 jobs 15m36s 11m51s 13m32s
Main-test execution, sum of 9 jobs 58m26s 58m32s 63m08s
Package and site build span 27s 30s 17s
E2E workflow elapsed 5m23s 6m43s 5m13s

Runner time sums job durations without billing weights. Setup starts when checkout finishes and ends when tests start. Workflow elapsed time includes scheduling waits.

Against the baseline, setup time fell 23.9%, and CI elapsed time fell 4.8%. The current CI run took 50 seconds longer than the setup/cache run. Main-test execution increased by 4m36s across nine jobs between those runs.

The package/site build span fell from 30s to 17s after the builds ran in parallel. The package build still took 13s. The site build still took 17s.

The cnpmcore E2E test step fell from 5m29s to 4m14s between the last two runs. Its setup fell from 27s to 24s. The faster tests account for most of the E2E reduction.

These measurements use one run per version, with different commits and runner allocation. They do not establish a controlled benchmark or a guaranteed speed increase.

Summary by CodeRabbit

  • Chores
    • CI workflows now restore and save dependency and compile caches across relevant jobs.
    • Independent setup, build, service startup, artifact handling, and coverage-report tasks can run concurrently, with waits added where later steps depend on their completion.
    • End-to-end test workflows now clone the ecosystem project while tool setup runs, then wait for the clone before continuing with dependency overrides and tests.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: eddf3b34-89ca-4ff5-a723-49584bf970b7
📥 Commits

Reviewing files that changed from the base of the PR and between e9cc326 and 27e35a7.

📒 Files selected for processing: 2
  • .github/workflows/ci.yml
  • .github/workflows/e2e-test.yml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

The CI workflow updates dependency cache handling and runs selected setup, build, service, and artifact steps in parallel. The end-to-end workflow runs its ecosystem-project clone alongside tool setup and waits for the clone before running ecosystem tests.

Changes: CI workflow execution

Layer / File(s) Summary
Dependency cache handling
.github/workflows/ci.yml
CI jobs compute dependency hashes for cache keys and restore caches alongside tool setup. Cache saves run only on cache misses under specified job and shard conditions.
Parallel steps and synchronization
.github/workflows/ci.yml
Selected build, service startup, and artifact operations run in the background. The workflow waits before database initialization, subsequent tests, or coverage processing.
End-to-end clone and setup
.github/workflows/e2e-test.yml
The ecosystem-project clone runs alongside utoo and Node.js setup. The workflow waits for the clone after package packing.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Suggested reviewers: killagu

Merge Risk: ⚪ Minimal · up to 27e35

The change reorganizes CI steps to run in parallel and adjusts cache handling. No concrete merge-blocking risk was found, and the PR reports successful CI runs with this setup.

Pre-merge checks: Passed 5
Passed Passed checks: 5 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely summarizes the main changes: parallelizing independent CI steps and reducing cache overhead.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Deploying egg-v3 with  Cloudflare Pages  Cloudflare Pages

Latest commit: 27e35a7
Status: ✅  Deploy successful!
Preview URL: https://2dddb3c6.egg-v3.pages.dev
Branch Preview URL: https://perf-parallel-ci-setup.egg-v3.pages.dev

View logs

@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.02%. Comparing base (e9cc326) to head (27e35a7).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6072      +/-   ##
==========================================
+ Coverage   83.68%   84.02%   +0.34%     
==========================================
  Files         734      734              
  Lines       22639    22639              
  Branches     4560     4560              
==========================================
+ Hits        18945    19023      +78     
+ Misses       3193     3125      -68     
+ Partials      501      491      -10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Deploying egg with  Cloudflare Pages  Cloudflare Pages

Latest commit: 27e35a7
Status: ✅  Deploy successful!
Preview URL: https://4d9484c2.egg-cci.pages.dev
Branch Preview URL: https://perf-parallel-ci-setup.egg-cci.pages.dev

View logs

@fengmk2 fengmk2 changed the title perf(ci): overlap setup and reduce cache overhead perf(ci): overlap independent steps and reduce cache overhead Oct 14, 2026
@fengmk2
fengmk2 marked this pull request as ready for review October 15, 2026 03:22
Copilot AI balanced review requested due to automatic review settings October 15, 2026 03:22
@fengmk2

fengmk2 commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 27e35a7225

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T03:26:56.640207Z 27e35a7 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@fengmk2
fengmk2 merged commit 95bd919 into main Oct 15, 2026
39 checks passed
@fengmk2
fengmk2 deleted the perf/parallel-ci-setup branch October 15, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants