Repository navigation
Stop reuse self-tests posting fake CI notices - #1634
Open
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Open
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
Conversation
The fallback tests for ci-reuse-merge-group.py and ci-reuse-pr-head.py let the scripts print their real "::notice::Could not verify merge-queue CI (...)" lines to stdout. lint-ecosystems runs those tests in a GitHub job, so every run on PRs, the merge queue and main gained ten fake annotations claiming the reuse check had failed. They fill the step's ten-notice cap and send anyone debugging a reuse decision down the wrong path. Capture stdout in those cases and assert that the notice is printed. The fallback path is now checked, not just silenced. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeHhBxEvePf8ERTaTTenMK
Collaborator
Author
|
bugbot run Generated by Claude Code |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 564c31a. Configure here.
Collaborator
Author
|
Labeled Ready for review (burn-down agent).
Slack announcement not sent this run (the Slack send tool was unavailable). The next run will retry. Generated by Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every
lint-ecosystemsrun (PR, merge queue and main push) posts 10 fake notice annotations, for example on main's latest push run (check run 117423824414 at 7d3086f):Nothing failed. The reuse scripts never run in that job. These lines come from the job's
python3 -B -m unittest discover -s scripts/testsstep: the fallback self-tests letci-reuse-merge-group.pyandci-reuse-pr-head.pyprint their real::notice::workflow commands to stdout, and GitHub turns those lines into annotations. A local run of the suite prints 15 of them, and GitHub keeps 10 (its per-step cap).The harm:
CI is otherwise healthy this run: the merge queue is empty, nothing is queued or running, main is green, and the only failed run in 3 days (38806080761) was fixed by that PR's next push. So this falls under priority 4 (make CI more robust).
Root cause
test_api_failure_timeout_and_bad_payload_*inscripts/tests/test_ci_reuse_merge_group.pyandscripts/tests/test_ci_reuse_pr_head.pycallreusable_runon its error path without capturing stdout.Fix
A small
assert_falls_backhelper in each test file. It runsreusable_runundercontextlib.redirect_stdout, asserts that the result isNone(unchanged), and also asserts that the::notice::fallback line was printed. The fallback is now checked, not just silenced. No production script or workflow changes.Proof
python3 -B -m unittest discover -s scripts/tests: 347 tests, OK (6 skipped). Lines starting with::in stdout went from 15 to 0.::notice::fromci-reuse-merge-group.pymakes the new assertion fail 7 subtests.Removed or moved tests
None. Every test still runs in the same place (
lint-ecosystems).https://claude.ai/code/session_01SeHhBxEvePf8ERTaTTenMK
Generated by Claude Code