Repository navigation
Redeploy main after a merge when prod has not converged on it #3848
Description
🤖 Generated by the Agentic Engineer
Part of #2838. It follows #3846, whose resolver merged in #3847.
Evidence
- Since feat(ci): tell whether prod has converged on main from signed provenance #3847,
scripts/resolve-prod-convergenceanswers from signed evidence whether prod's published artifact matchesmain. The answer isCONVERGED,BEHIND,DIVERGEDorUNKNOWN. - Nothing runs it. After the interleaving A merge-group heal can leave prod behind a merge that lands just after it #2838 describes, prod still stays behind
mainuntil an unrelated deploy happens to land. - The resolver needs registry read access to verify the attestation. That exists only on a runner with
packages: read, so its last acceptance point (a liveCONVERGEDread) is proven here, not in Tell whether prod has converged on main from the published artifact's signed provenance #3846.
Audience and impact
This serves the platform operator. It closes the last path by which prod can silently disagree with main after a merge.
Smallest useful change
After a merge lands on main, run the resolver against the digest :latest names:
CONVERGED: do nothing.BEHINDorDIVERGED: run the normal production deploy ofmain, through the same composite and the sharedprod-deploylock.UNKNOWN: do not deploy; fail visibly, so a broken check is never mistaken for a converged prod.
Because this job reaches production, it inherits every gate the existing deploy paths carry: the publication gate, the authorization and signing contracts, and the per-workflow coverage guards. It lands behind an opt-in switch, defaulting off, and is activated separately.
Acceptance criteria
- On a merge where prod already matches
main, the job readsCONVERGEDand does not deploy (proven on a real merge). - A
BEHINDorDIVERGEDverdict triggers exactly one deploy ofmain, and a second evaluation afterwards readsCONVERGED. -
UNKNOWNnever deploys and fails the job with the resolver's reason. - The job holds the shared
prod-deploylock and cannot run concurrently with a merge-group deploy or heal. - Tell whether prod has converged on main from the published artifact's signed provenance #3846's live
CONVERGEDacceptance point is recorded from this job's first real run.
Verification — 2026-11-03
The real merge Validate Main run 41663342775, at b76dd8340c2d51922e8beb65d7612c039af80c09, completed successfully. Its production-convergence job emitted CONVERGED and skipped the redeploy step. This confirms the no-deploy path and the live resolver acceptance point; it does not prove automatic recovery.
The complete seven-run CD history on main since 2026-10-26 contains no successful github-actions[bot] dispatch. The remaining acceptance is a real BEHIND observation, exactly one successful automatic deploy, then a CONVERGED recheck. As recorded in #3869 and the earlier acceptance table, DIVERGED remains a warning while a queued group may be deploying; this check does not authorize a rollback or a production trial.
Size
M: one workflow job plus contract-test updates, behind an opt-in switch.
Blocker: the next real BEHIND observation followed by one successful automatic main deployment and a CONVERGED recheck | outcome | last-verified 2026-11-03: the latest real merge observer returned CONVERGED and skipped redeployment; the complete seven-run main CD history since 2026-10-26 contains no successful automated dispatch, so the remaining recovery sequence is unverified. Collect that sequence when it occurs; no production mutation is needed for this verification
Activity
chore: agent-claim #3848 nonce=63414cca9ef443b07a5ef2250891eb5f
fix(ci): fetch attested commits and keep docker stderr out of the digest
ci: test the convergence report on pull requests
devantler commentedon Sep 16, 2026
First slice merged: #3864 (
5e3049c9). After every push tomain, a read-only job reports the resolver's verdict.CONVERGEDbecomes a notice.BEHINDandDIVERGEDbecome warnings, because a queued merge group can briefly showDIVERGED.UNKNOWNfails the job. The job never deploys. Before resolving, it fetches every commit the provenance names, so an artifact left by a merge-group heal does not read asUNKNOWN. Its first live run readCONVERGED(run 35150608562).main→ readsCONVERGED, no deploy (real merge)BEHIND/DIVERGEDtriggers exactly one deploy ofmain, thenCONVERGEDUNKNOWNnever deploys and fails with the resolver's reasonprod-deploylockCONVERGEDpoint recordedNext slice: a deploy of
mainonBEHIND/DIVERGEDunderprod-deploy, behind a switch that defaults to off. It reaches production, so it carries the deploy lane's usual gates.devantler commentedon Sep 17, 2026
Status 2026-09-17: the deploy slice merged as #3870 (for #3869)
main→ readsCONVERGED, no deploy (real merge)CONVERGEDfor1dff6239, skipped the redeploy step, and dispatched no CD run.BEHINDtriggers exactly one deploy ofmain, thenCONVERGEDCDonmainonly onBEHIND, skips when an unfinished CD run already targetsmain's tip, and waits for its dispatched run to be listed before releasing the lock. Proven by the script's tests. A realBEHINDneeds the #2838 interleaving, so this stays open until one happens.DIVERGEDDIVERGED, so redeploying on it would undo a deploy that is about to land (decision recorded on #3869). It stays a warning.UNKNOWNnever deploys and fails with the resolver's reasonprod-deploylockprod-deploywithqueue: max.CONVERGEDpoint recordedRemaining: record the first live
BEHIND→ one CD run →CONVERGEDsequence. Nothing else is left to build here.devantler commentedon Sep 19, 2026
Status 2026-09-19: still waiting on a live
BEHIND. Nothing is left to build.Across the last 100
validate-main.yamlruns, 32 reached the🧭 Observe Production Convergencejob, and every one of them recordedCONVERGEDin its annotations. None recordedBEHIND,DIVERGEDorUNKNOWN. The only failed run in that window (35100863274, 2026-09-16) predates the observer.So the one open criterion (a real
BEHIND, then one CD run, thenCONVERGED) is waiting for the #2838 interleaving to actually happen. There is no code change to make until it does. Any later run can re-check with the check-run annotations on that job.chore: agent-claim #3848 nonce=4618a0dbb51c90187804db151af28a7d
chore: agent-claim #3848 nonce=7297c77df8b688becee57152bcfab65e