Skip to content

fix(ci): keep draft-mode success from standing as a ready PR's proof - #2398

Merged
schlessera merged 6 commits into
mainfrom
fix/2363-draft-proof-live-readiness
Oct 18, 2026
Merged

schlessera merged 6 commits into
mainfrom
fix/2363-draft-proof-live-readiness

Conversation

@schlessera

@schlessera schlessera commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Closes #2363

What changed

A ready PR could have a passing proof check that came from a draft run with every product job skipped. Draft runs can no longer produce that check.

  • Distinct check name. The aggregate job now reports as draft gates for a draft payload, proof for other pull-request payloads, and integrated or exhaustive proof otherwise (.github/workflows/ci.yml). A proof row on a PR therefore only comes from ready feedback, which fails by design, or from confirmation. The workflow guard pins the new expression and rejects dropping the draft branch (scripts/check-ci-workflows.ts).
  • Live readiness check. The draft cadence now reads the live PR through the existing read-only client. It fails with the confirmation-required action when the PR is ready, closed, at another head, or in a malformed state. If the read fails or no token is available, it fails closed (requireLiveDraft, scripts/candidate-proof.ts).
  • Producer audit. ci.yml is a pinned generated-CSS producer. Under test(ci): freeze audited generated stylesheet fixtures #2388's ruling, legitimate drift requires fresh proof and does not redefine the audit, so its digest in scripts/generated-build-inputs.ts stays at the audited value. Generated-CSS binding falls back to fresh proof until a re-audit; #2408 asks how that should happen.
  • Docs. docs/decisions/ci-utility.md (Scheduling and selection) and the work-issue merge reference describe both mechanisms, including that the live read only fails when it observes readiness.

Why

In #2359, a draft push and ready_for_review happened 2 s apart. The draft run was queued second, cancelled the confirmation run through the per-PR concurrency group, and passed proof on a ready PR. The full reconstruction is in the issue. Under the 2026-10-14 cadence ruling, a ready PR's proof must not pass without confirmation.

Either mechanism alone leaves a gap:

  • Live check alone: it reads a snapshot. A PR marked ready after that read could briefly show a passing proof.
  • Rename alone: an inverted queue leaves no proof row at all, with no visible failure.

Together:

  • Admission: a draft success never counts as proof.
  • Signal: a draft run whose read finds the PR ready fails visibly.

A cancelled confirmation still shows as a missing proof, which the merge procedure already treats as not green.

Validation

  • bun run check:pr --base origin/main at dbec504b2 on c4d31c6bd: 959 fast tests and 78 affected tests, all passing. bun run lint is clean; tsc --noEmit exits 0.
  • Focused tests: tests/candidate-proof.test.ts has 5 passing tests; the producer-closure, workflow, hosted-proof and citation suites pass 196 with 0 failures; tests/inclusion-inputs.test.ts passes 11.

Mutation receipts. Each mutation was applied alone, the named test failed, and the source was restored byte-identically (cmp):

Mutation Fails at
Remove the draft dispatch in validateCandidate tests/candidate-proof.test.ts:43, the live-ready rejection (receipt taken at 2e0e1d140)
Restrict the guard to synchronize tests/candidate-proof.test.ts:44
Drop the missing-head identity check tests/candidate-proof.test.ts:53
requireCandidate no longer calls validateCandidate tests/candidate-proof.test.ts:55
requireCandidate builds no API client tests/candidate-proof.test.ts:87, the present-token read
State check weakened to === "closed" tests/candidate-proof.test.ts:53, malformed state
Drop the draft gates branch from the job name the guard test in tests/ci-workflows.test.ts, "must not impersonate candidate confirmation"

Independent review: two read-only gpt-6-astra passes.

  • First pass: it found the snapshot race and gaps in the tests, which led to the rename and the extra tests.
  • Second pass: it confirmed the expression's semantics against proofCadence, found no other consumer of the proof display name, and verified the digest. Its remaining mutation and doc findings are fixed in 97a544351.

Hosted proof: this PR's own ready run is the confirmation. A draft run of this branch would show draft gates.

Gaps

No changeset: no shipped package files changed.

🤖 Generated with Claude Code

@schlessera schlessera added this to the 0.44.0 milestone Oct 18, 2026
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: ci CI workflows, the gates they run, release scripts, packaging. labels Oct 18, 2026
@schlessera
schlessera marked this pull request as draft October 18, 2026 01:00
@schlessera
schlessera marked this pull request as ready for review October 18, 2026 01:02
@schlessera
schlessera marked this pull request as draft October 18, 2026 03:18
@schlessera
schlessera force-pushed the fix/2363-draft-proof-live-readiness branch 2 times, most recently from 0df8160 to caca24b Compare October 18, 2026 08:03
@schlessera
schlessera marked this pull request as ready for review October 18, 2026 08:05
@schlessera
schlessera marked this pull request as draft October 18, 2026 09:19
schlessera and others added 6 commits October 18, 2026 11:19
A draft push and a ready_for_review transition seconds apart can be queued
out of order: the draft run cancels the confirmation run, then passes
`proof` with every product job unselected, leaving a green draft check as
the ready PR's latest proof. The draft cadence now reads the live PR and
refuses when it is ready, closed or at another head, with the same
confirmation-required action as ready feedback. Unreadable live state
fails closed.

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oint

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The live readiness read is a snapshot: a PR marked ready between that
read and the draft run's completion could still show a passing `proof`
until its confirmation reported. Draft runs now report their aggregate
as `draft gates`, so a `proof` row on a pull request only ever comes
from feedback (which fails) or confirmation. The workflow guard pins the
new name and rejects dropping the draft branch. The audited producer
digest for ci.yml is refreshed for the one-line job-name change.

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cover a present token through the real API client, malformed live state,
and the emitted aggregate refusing a draft run it cannot verify. State
in the docs that the live read only fails when it observes readiness;
the distinct check name covers a later transition.

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Planning the whole repository twice (fixture outputs, then the child's
selection check) took 20 s locally and timed out at 30 s on a hosted
unit shard. A cheap version needs the aggregate to validate the
candidate before planning, which changes a pinned producer; #2400
tracks it. The present-token entry-point control remains.

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#2388 binds generated-CSS positive fixtures to producer bytes frozen at
the audited revision, and its ruling says legitimate drift requires
fresh proof rather than redefining the audit. Refreshing the ci.yml
digest here made six frozen fixtures fail. The job-name change now
leaves the binding to fresh proof; #2408 asks how re-audits happen.

Refs #2363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, the gates they run, release scripts, packaging. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: ready PR can retain green draft-mode proof with product jobs skipped

1 participant