ci: publish Castiron statuses despite stale PR base metadata - #687
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Castiron custom code✅ No new custom-code files detected. 50 mixed files remain; 2 existing customizations changed. Compared
48 existing customizations unchanged
8 more in the full report. A changed generated baseline means this report cannot reliably identify which handwritten lines changed. Inspect the custom-code diffDownload the exact patch produced by this run (requires repository access): gh run download 34265631834 --repo openai/openai-ruby \
--name castiron-custom-code-34265631834-1 --dir /tmp/castiron-custom-code-34265631834-1
git apply --stat /tmp/castiron-custom-code-34265631834-1/custom-code.patch
cat /tmp/castiron-custom-code-34265631834-1/custom-code.patchOr reproduce it from an SDK checkout containing the vendored reporter: git fetch --no-tags origin 107973fe5db38f5ef5aaee9f590c70d4cf77a1c5 fe571bae6877032015c578161073dabf5e772e0c
python3 scripts/castiron/custom_code_report.py report \
--base 107973fe5db38f5ef5aaee9f590c70d4cf77a1c5 \
--head fe571bae6877032015c578161073dabf5e772e0c --fetch --require-head-hash --public \
--out /tmp/castiron-custom-code-fe571bae6877
cat /tmp/castiron-custom-code-fe571bae6877/custom-code.patchThis is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR. |
markstuart-oai
left a comment
markstuart-oai
left a comment
There was a problem hiding this comment.
Reviewed all seven changed files at fe571ba, including the surrounding trusted evaluator, Git-object verification, publisher workflows, and tests. No substantive findings.
The change cleanly separates PR identity from evaluation freshness: repository/ref/head checks remain exact, while the budget evaluator and publication checks use current main independently of PR base metadata. Missing, stale, and failed evaluations produce failing statuses and a visible publisher failure; report comments recheck main after pagination. The existing resolver owns the change, with no new modes or helper layers. Production scripts plus workflows grow by only 3 lines / 240 bytes, and no file crosses 1,000 lines.
Validation: the full Castiron suite ran 55 tests successfully with one optional compiler-contract skip (CASTIRON_TEST_BIN unset); six additional isolated probes passed, covering the stale-metadata status matrix, identity mismatches, a real newer-main budget decrease, comment retry idempotency, replacement-ref isolation, and mocked status transport failures. Reporter checksum and git diff --check passed; observed PR checks were green. Local Ruff was blocked by the package-registry proxy, and actionlint was unavailable. No live publisher or workflow was invoked.
2 participants
Summary
GitHub can retain an older
base.shaon an open PR after main advances. The Castiron handler treated that metadata as PR identity, so rerunning a valid PR could leave both required budget statuses missing while the publisher appeared green.Resolve the current PR by its verified repository, branch, and head identity. Continue evaluating current-main policy against trusted Git objects and checking the actual evaluation revisions before publication. Publish failing statuses and fail the publisher job when evaluation is unavailable, stale, or fails. Recheck main before writing the report comment.
Validation
CASTIRON_TEST_BINis unset), including stale PR base metadata, stale heads, evaluation base changes, failure visibility, and comment publication freshness.git diff --checkpassed.The trusted workflow runs from main, so the updated publisher takes effect after this PR merges. Budget policy, accounting, and permissions are unchanged.