Skip to content

fix(deploy): bypass deployment checks for automatic rollbacks - #571

Merged
Brooooooklyn merged 4 commits into
voidzero-dev:mainfrom
davidgustys:fix/rollback-bypass-deployment-checks
Oct 26, 2026
Merged

Brooooooklyn merged 4 commits into
voidzero-dev:mainfrom
davidgustys:fix/rollback-bypass-deployment-checks

Conversation

@davidgustys

@davidgustys davidgustys commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Cloudflare refuses a deployment that moves traffic back to an older Worker version when a secret has changed since that version was active (error 10220), unless the request is forced. Void's automatic recovery runs into this. After a failed readiness check, activation or trigger sync, it deploys the previous version at 100%. Cloudflare rejects that, the Worker stays at the previous version at 100% plus the failed one at 0%, and the next deploy stops with deployment-changed until someone fixes the deployment by hand.

before
  deploy v2 (changes a secret) → readiness fails → restore v1@100
    → Cloudflare 10220 ✗ → stuck at v1@100 + v2@0, next deploy refuses

after
  deploy v2 (changes a secret) → readiness fails → restore v1@100
    → Cloudflare 10220 → Void still owns traffic? ── no ──→ stop ('unsafe')
                                   │ yes
                                   └→ restore v1@100 with force ✓
                                      → error says: old secret values are live,
                                        deploy again to apply the new ones

What changed

  • versions.deploy gets an optional bypassDeploymentChecks. Both cf runners pass it as --bypass-deployment-checks, which sends force=true to the Deployments API.
  • restorePreviousVersion (readiness, uploader-check, activation and trigger-sync failures) first tries the restore without force. Only when Cloudflare answers 10220 does it read the owner again and retry with force. Wrangler's own rollback does the same: it tries without force and forces only after a 10220.
  • When the restore had to be forced, the activation result carries restoredOlderSecrets: true and the deploy error adds:

    Cloudflare refused to restore the previous version because this deploy changed a secret, so Void forced the restore. The previous version runs with its old secret values; deploy again to apply the new ones.

Deploy Forced?
normal deploy, staging, activation never
void rollback (user asked for it) never
automatic app rollback (restorePreviousVersion) only on retry after 10220, if Void still owns traffic
trigger-sync rollback in activateCloudflareExistingVersion never
platform installer activation and restoration never

Why the last two are not forced

  • Trigger-sync rollback: it goes back to the version that served traffic seconds before, and Cloudflare had just accepted the user's rollback without force. So a 10220 there means someone else changed a secret in between, and Void must not overwrite that.
  • Platform installer: fix(platform): repair failures found in Cloudflare acceptance #569 decided that a credential update is never rolled back to versions holding the old secrets. Traffic stays closed and the next run rolls forward (see the comment in platform-installer.ts and the test "rolls forward a partially restored credential update"). Forcing restorePlatformWorkerDeployment would undo that for the one Worker whose trigger or domain publish failed, and put it back on the old, possibly revoked, API token. Without a credential update, Void's own platform deploys don't cause a 10220.

Where this applies

The app path is the one that hits this: a deploy whose new version changes a secret and then fails readiness, activation or trigger sync. I found it in a fork that publishes prebuilt artifacts with explicit secrets. With this change, the same live run on Cloudflare restored the previous version and the next deploy went through.

Tests

  • operations.test.ts runs every recovery path (readiness, activation, triggers; with and without preserveTriggers) in 4 cases:
    • Cloudflare accepts the restore: nothing is forced.
    • Cloudflare refuses with 10220: one unforced try, an owner read, then one forced retry, and restoredOlderSecrets: true.
    • 10220, and someone else deployed in between: result unsafe, no forced retry.
    • Any other error: result failed, no retry.
  • cloudflare-deploy.test.ts checks that the readiness, trigger and uploader errors add the old-secrets line only when restoredOlderSecrets is set.
  • cloudflare-operations.test.ts checks that the trigger-sync rollback and user rollbacks are not forced. platform-install-cutover.test.ts checks that no platform activation or restoration is forced.
  • One test runs the real pinned cf (1.0.0-beta.18) against a local server and checks that ?force=true is sent next to the deployment body.
  • Mutation checks, each caught by at least one test: always force, never retry, retry on any error, skip the owner re-read, force the trigger-sync rollback, force the platform restoration, force the platform activation, drop or always set restoredOlderSecrets, drop it on the trigger path, skip the message line.
  • vp fmt, vp lint, the deploy-cloudflare and void typechecks, and vp run test --base origin/main pass.

Known limits

  • Cloudflare describes force only as skipping checks "such as" the secret one. This PR forces only after Cloudflare answers 10220, so force is used only in the case Cloudflare names.
  • Between Void's owner read and the forced retry there is still a short gap, about one cf process start. The Deployments API has no compare-and-set, so it can't be closed fully. That gap already exists for every deploy Void makes.

🤖 Generated with Claude Code

@davidgustys
davidgustys force-pushed the fix/rollback-bypass-deployment-checks branch from 058c950 to c16802d Compare October 23, 2026 13:05
@cpojer
cpojer requested a review from Brooooooklyn October 26, 2026 07:23
@Brooooooklyn
Brooooooklyn force-pushed the fix/rollback-bypass-deployment-checks branch from c16802d to 96e5881 Compare October 26, 2026 08:19
@Brooooooklyn

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-26T08:23:02.624969Z
We couldn’t complete this review. Comment @codex review to try again.
96e5881 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Brooooooklyn

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/deploy-cloudflare/src/operations.ts
Brooooooklyn added a commit that referenced this pull request Oct 26, 2026
#594)

The Astro playground's dev smoke test (`astro.test.ts > playground/astro
dev`) sometimes misses its 20 s start limit during a full parallel run.
The server log shows why:

```
[WARN] Failed to resolve dependency: drizzle-orm/sqlite-core, present in ssr 'optimizeDeps.include'
astro ready
first GET / ──→ Vite finds drizzle-orm/sqlite-core now
             ──→ "optimized dependencies changed. reloading" / "program reload"
             ──→ 200 after 6.5 s  (limit already used up)
```

`astro.config.mjs` lists `drizzle-orm/sqlite-core` so it is optimized up
front, but `drizzle-orm` is Void's dependency, not the playground's, so
Vite can't resolve that name and skips it. This names it through Void
(`void > drizzle-orm/sqlite-core`), the same way Void's own plugin names
`void > email-validator`.

| Fresh `astro dev` (`.void` and `.wrangler` removed) | Warning | Reload
in first request | First `GET /` |
|---|---|---|---|
| before | yes | yes | 1.05 s on an idle machine |
| after | none | none | 0.42 s |

Only the playground config changes.

Seen while working on #571. `vinext-pages dev` failed the same way once
in another run. I didn't capture its log, and its config has no
`optimizeDeps`, so this PR doesn't claim to fix that one.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
davidgustys and others added 4 commits October 26, 2026 17:05
Cloudflare rejects a deployment that rolls back to an older version after
a secret changed (error 10220) unless it is forced. Void's automatic
recovery after failed readiness, activation or trigger synchronization
therefore failed whenever the failed version changed a secret, leaving
the Worker at previous@100 plus failed@0 and blocking later deploys with
deployment-changed.

Add an optional bypassDeploymentChecks flag to versions.deploy, mapped to
cf's --bypass-deployment-checks (force=true), and set it only on automatic
recovery paths that have already verified ownership of current traffic:
restorePreviousVersion, the trigger-synchronization rollback, and platform
deployment restoration. Normal deploys and user-requested rollbacks are
unchanged.
…an old secret version

Try the automatic app rollback without force first, as Wrangler's
rollback does. Only when Cloudflare answers 10220 (a secret changed since
the previous version was active), read the owner again and retry with
force, so a secret changed outside Void is not overwritten and any other
Cloudflare check still applies.

Keep the trigger-sync rollback and the platform installer unforced. The
first returns to the version that served traffic seconds before, so a
10220 there comes from someone else. The second would undo voidzero-dev#569: a
credential update rolls forward instead of restoring old secrets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When Cloudflare refused the automatic restore over a secret change and
Void forced it, the activation result now carries
`restoredOlderSecrets: true`, and the deploy error tells the user that
the previous version runs with its old secret values and that deploying
again applies the new ones.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Brooooooklyn
Brooooooklyn force-pushed the fix/rollback-bypass-deployment-checks branch from 7ac03d2 to 1df0efe Compare October 26, 2026 09:05
@Brooooooklyn
Brooooooklyn merged commit a71da90 into voidzero-dev:main Oct 26, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants