ci: shard Ruby tests across parallel jobs - #684
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Castiron custom code✅ No new custom-code files detected. 50 mixed files remain; 0 existing customizations changed. Compared 50 existing customizations unchanged
10 more in the full report. A changed generated baseline means this report cannot reliably identify which handwritten lines changed. Inspect the custom-code diffDownload the exact patch produced by this run (requires repository access): gh run download 34248543568 --repo openai/openai-ruby \
--name castiron-custom-code-34248543568-1 --dir /tmp/castiron-custom-code-34248543568-1
git apply --stat /tmp/castiron-custom-code-34248543568-1/custom-code.patch
cat /tmp/castiron-custom-code-34248543568-1/custom-code.patchOr reproduce it from an SDK checkout containing the vendored reporter: git fetch --no-tags origin c368a06db374892273812f64c823f8d7b21871a2 50de9558463d2a8d34452725fd8b7cd520c18e49
python3 scripts/castiron/custom_code_report.py report \
--base c368a06db374892273812f64c823f8d7b21871a2 \
--head 50de9558463d2a8d34452725fd8b7cd520c18e49 --fetch --require-head-hash --public \
--out /tmp/castiron-custom-code-50de9558463d
cat /tmp/castiron-custom-code-50de9558463d/custom-code.patchThis is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR. |
|
Hosted CI is green: https://github.com/openai/openai-ruby/actions/runs/34248506645
Shard durations:
These are single-run comparisons with substantial runner variation. The remaining opportunities are better shard balance and reducing repeated whole-tree typechecking/merge work while preserving compatibility checks. |
markstuart-oai
left a comment
markstuart-oai
left a comment
There was a problem hiding this comment.
Reviewed all five changed files and the existing test launcher, Rake selection, serial scheduling, and required CI gate at 50de955. No actionable correctness or structural findings. The small pure selector keeps partitioning in test infrastructure without complicating scheduling; the nine matrix jobs remain covered by ci-required.
Independent Ruby 4.0.6 checks confirmed the actual Rake inventory partitions 251 primary files into 84/84/83 with unchanged default order, no duplication, and no Bedrock leakage. The four new tests passed through ./scripts/test; additional entrypoint probes passed for unset/1-of-1 sharding, invalid and empty selections, and explicit Bedrock rejection. The related launcher suite had one local readiness-timeout assertion failure; I did not complete a full local suite or run other Ruby versions.
Read-only Actions metadata confirms 4m13s elapsed and 25m37s summed job time versus 6m53s/19m15s on preceding main and 9m15s/25m43s on #683. These single-run results support faster feedback through concurrency, not reduced aggregate compute; setup duplication and existing whole-tree typecheck/RBI work remain.
2 participants
The Ruby test matrix determines CI completion: PR #683 spent 6m47s–8m47s in each
Run testsstep while all other CI jobs finished within 53 seconds. Split the primary test suite into three disjoint groups of whole files on each of Ruby 3.3, 3.4, and 4.0. Every existing test still runs on every version, and existing serial scheduling—including large-payload cases—is preserved.TEST_SHARD=1/3 ./scripts/testreproduces a group locally; omittingTEST_SHARDstill runs the complete primary suite. Invalid configurations and empty groups fail visibly. The existingci-requiredgate requires all nine matrix jobs to succeed. Permissions, action pins, Bedrock testing, and the 15-minute timeout from #683 are unchanged.Validation
Tradeoff: six additional runner jobs add setup overhead and consume more concurrency. This reduces feedback latency while preserving coverage; it does not eliminate the repeated whole-tree typechecking and RBI-merge work identified during profiling.