Skip to content

ci: allow 15 minutes for Ruby test jobs - #683

Merged
apcha-oai merged 1 commit into
mainfrom
apcha/ruby-ci-test-timeout-15m
Sep 8, 2026
Merged

ci: allow 15 minutes for Ruby test jobs#683
apcha-oai merged 1 commit into
mainfrom
apcha/ruby-ci-test-timeout-15m

Conversation

@apcha-oai

Copy link
Copy Markdown
Contributor

Ruby test jobs can run close to the current 10-minute limit, leaving too little headroom for setup and runner variance. Increase the test-ruby matrix job timeout to 15 minutes as an interim measure while test-suite performance is investigated separately.

This applies to Ruby 3.3, 3.4, and 4.0. Other job timeouts and test coverage are unchanged.

Validation: reviewed the one-line workflow diff and ran git diff --check.

@apcha-oai
apcha-oai requested a review from a team September 8, 2026 15:30
@apcha-oai
apcha-oai requested a review from a team as a code owner September 8, 2026 15:30
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-08T15:31:47.085251Z 5e033fb PR opened
🔒 Security Review Completed 2026-09-08T15:33:35.987215Z 5e033fb PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@apcha-oai
apcha-oai enabled auto-merge September 8, 2026 15:31
@openai-sdks

openai-sdks Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

236/236 SDK tests passed in 10.157s for Ruby SDK PR #683.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 242ms
tests/chat-completions-create.test.ts ✅ Passed 324ms
tests/chat-completions-stream.test.ts ✅ Passed 150ms
tests/files-content-binary.test.ts ✅ Passed 177ms
tests/files-create-multipart.test.ts ✅ Passed 174ms
tests/files-list-pagination.test.ts ✅ Passed 164ms
tests/initialize-config.test.ts ✅ Passed 177ms
tests/instance-isolation.test.ts ✅ Passed 183ms
tests/models-list.test.ts ✅ Passed 159ms
tests/responses-background-lifecycle.test.ts ✅ Passed 269ms
tests/responses-body-method-errors.test.ts ✅ Passed 575ms
tests/responses-cancel-timeout.test.ts ✅ Passed 254ms
tests/responses-cancel.test.ts ✅ Passed 270ms
tests/responses-compact-retries.test.ts ✅ Passed 385ms
tests/responses-compact.test.ts ✅ Passed 317ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 209ms
tests/responses-create-advanced.test.ts ✅ Passed 292ms
tests/responses-create-disconnect.test.ts ✅ Passed 151ms
tests/responses-create-errors.test.ts ✅ Passed 340ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 172ms
tests/responses-create-retries.test.ts ✅ Passed 214ms
tests/responses-create-stream-failures.test.ts ✅ Passed 154ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 242ms
tests/responses-create-stream-wire.test.ts ✅ Passed 2.061s
tests/responses-create-stream.test.ts ✅ Passed 119ms
tests/responses-create-terminal-states.test.ts ✅ Passed 420ms
tests/responses-create-timeout.test.ts ✅ Passed 255ms
tests/responses-create.test.ts ✅ Passed 321ms
tests/responses-delete.test.ts ✅ Passed 194ms
tests/responses-input-items-errors.test.ts ✅ Passed 190ms
tests/responses-input-items-list.test.ts ✅ Passed 201ms
tests/responses-input-items-options.test.ts ✅ Passed 175ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 283ms
tests/responses-input-tokens-count.test.ts ✅ Passed 346ms
tests/responses-malformed-inputs.test.ts ✅ Passed 2.176s
tests/responses-not-found-errors.test.ts ✅ Passed 442ms
tests/responses-parse.test.ts ✅ Passed 255ms
tests/responses-retrieve-retries.test.ts ✅ Passed 316ms
tests/responses-retrieve.test.ts ✅ Passed 330ms
tests/responses-stored-method-errors.test.ts ✅ Passed 836ms
tests/retry-behavior.test.ts ✅ Passed 3.333s
tests/sdk-error-shape.test.ts ✅ Passed 429ms

View OkTest run #34245213507

SDK merge (227a483b5db6) · head (5e033fb491a2) · base (21b8a70d48e2) · OkTest (08ec4299e769)

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

50 mixed files remain; 0 existing customizations changed.

Compared 21b8a70d48e25e033fb491a2. Generated baselines verified.

50 existing customizations unchanged
  • lib/openai.rb
  • lib/openai/client.rb
  • lib/openai/models/audio/transcription_create_response.rb
  • lib/openai/models/chat/chat_completion_message.rb
  • lib/openai/models/chat/chat_completion_message_function_tool_call.rb
  • lib/openai/models/chat/completion_create_params.rb
  • lib/openai/models/response_format_json_schema.rb
  • lib/openai/models/responses/function_tool.rb
  • lib/openai/models/responses/response.rb
  • lib/openai/models/responses/response_create_params.rb
  • lib/openai/models/responses/response_format_text_config.rb
  • lib/openai/models/responses/response_format_text_json_schema_config.rb
  • lib/openai/models/responses/response_function_tool_call.rb
  • lib/openai/models/responses/response_function_web_search.rb
  • lib/openai/models/responses/response_output_text.rb
  • lib/openai/models/responses/tool.rb
  • lib/openai/resources/beta/threads.rb
  • lib/openai/resources/chat/completions.rb
  • lib/openai/resources/containers/files.rb
  • lib/openai/resources/files.rb
  • lib/openai/resources/responses.rb
  • lib/openai/resources/vector_stores/file_batches.rb
  • lib/openai/resources/vector_stores/files.rb
  • lib/openai/resources/webhooks.rb
  • rbi/openai/client.rbi
  • rbi/openai/models/chat/chat_completion_message.rbi
  • rbi/openai/models/chat/chat_completion_message_function_tool_call.rbi
  • rbi/openai/models/chat/completion_create_params.rbi
  • rbi/openai/models/response_format_json_schema.rbi
  • rbi/openai/models/responses/response.rbi
  • rbi/openai/models/responses/response_create_params.rbi
  • rbi/openai/models/responses/response_function_tool_call.rbi
  • rbi/openai/models/responses/response_function_web_search.rbi
  • rbi/openai/models/responses/response_output_text.rbi
  • rbi/openai/resources/chat/completions.rbi
  • rbi/openai/resources/files.rbi
  • rbi/openai/resources/responses.rbi
  • rbi/openai/resources/vector_stores/file_batches.rbi
  • rbi/openai/resources/vector_stores/files.rbi
  • scripts/castiron/README.md

10 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 34245253284 --repo openai/openai-ruby \
  --name castiron-custom-code-34245253284-1 --dir /tmp/castiron-custom-code-34245253284-1
git apply --stat /tmp/castiron-custom-code-34245253284-1/custom-code.patch
cat /tmp/castiron-custom-code-34245253284-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 21b8a70d48e2623e32aa01f95503adcf3fa2ba75 5e033fb491a29b93252aed1994835e900c428b5b
python3 scripts/castiron/custom_code_report.py report \
  --base 21b8a70d48e2623e32aa01f95503adcf3fa2ba75 \
  --head 5e033fb491a29b93252aed1994835e900c428b5b --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-5e033fb491a2
cat /tmp/castiron-custom-code-5e033fb491a2/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete exact-head one-line workflow diff. Raising only the test-ruby matrix timeout from 10 to 15 minutes gives Ruby 3.3/3.4/4.0 jobs additional runner/setup variance without changing test commands, coverage, fail-fast behavior, required-job aggregation, permissions, action pins, credentials, or any other job timeout. The workflow remains least-privilege, git diff --check passes, all completed exact-head lint/type/package/Bedrock/CodeQL/Castiron checks are green, and only the three Ruby matrix jobs are still running. No blocking correctness, CI-architecture, security, performance, or scope findings.

@markstuart-oai markstuart-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete diff and workflow, CI caller, scripts/test, and Rakefile. The job-level timeout correctly changes from 10 to 15 minutes for all Ruby 3.3/3.4/4.0 matrix jobs, covering setup and tests. Test selection/commands, Bedrock coverage, fail-fast behavior, required-job gating, permissions, action pins, and other timeouts remain unchanged. The workflow stays at 178 lines; this direct configuration change introduces no structural complexity or useful missed simplification. No findings.

Validation: git diff --check passes; no local suite or CI reruns performed. Inspected job/step timing metadata: recent run 34188709792 completed the Ruby jobs in 7m08s–7m56s; exact-head run 34245224446 was still running when inspected. This small sample does not establish the slowdown cause or prove 15 minutes sufficient. The bump provides bounded interim headroom; performance investigation remains separate.

@apcha-oai
apcha-oai added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit c368a06 Sep 8, 2026
20 checks passed
@apcha-oai
apcha-oai deleted the apcha/ruby-ci-test-timeout-15m branch September 8, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants