Skip to content

Skip redundant Cloudflare D1 schema preparation#1455

Open
jadch wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
jadch:fix-cloudflare-d1-session-init
Open

Skip redundant Cloudflare D1 schema preparation#1455
jadch wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
jadch:fix-cloudflare-d1-session-init

Conversation

@jadch

@jadch jadch commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Problem

createD1ExecutorDb() is used by both the edge Worker app and each MCP session Durable Object. It currently replays the full idempotent runtime-schema ensure on every open, even after the database is current.

With the current schema, a second open emits 46 CREATE TABLE / CREATE INDEX / ALTER TABLE statements before normal data-migration bookkeeping. D1 executes those as remote database operations, so a warm MCP session start can pay the schema bring-up cost again even though nothing changed.

Moving schema preparation only to the edge Worker would make the common path faster, but it would couple Durable Object correctness to edge startup. A session DO can wake independently, so this keeps the safety check inside the shared D1 opener.

Change

  • Generate a SHA-256 fingerprint from the same runtime-schema SQL used to build the Drizzle schema.
  • Store the last successfully prepared fingerprint in a private Cloudflare host table.
  • Read the marker through a first-primary D1 session so a matching value is current even if read replication is enabled later.
  • When the fingerprint matches, skip only the 46 schema DDL statements.
  • When it is missing or stale, run the normal schema ensure, run the existing data migrations, repair the schema again only if legacy compatibility code actually rebuilt/altered a table, then stamp the fingerprint.
  • Keep runCloudflareDataMigrations()'s existing return contract; the new internal preparation result only exposes whether compatibility changed the schema.

Data migrations still consult their own ledger on every open. A migration-only release therefore does not depend on the schema fingerprint changing.

Safety properties

  • A generated schema change automatically changes the fingerprint; no manual version bump is required.
  • Clean databases and existing unmarked databases take the full preparation path once.
  • The marker is written only after schema preparation and data migrations succeed, so a failed boot retries rather than recording a false success.
  • Concurrent stale readers converge through idempotent schema DDL and an upsert of the same fingerprint.
  • Legacy connection.item_id compatibility rebuilds get a final schema ensure before stamping, preserving newer nullable columns and indexes.
  • Any D1 error other than the expected first-boot “marker table does not exist” error still fails startup loudly.

Verification

The new regression suite covers:

  1. a second open emits no runtime-schema DDL while still checking the data-migration ledger;
  2. a stale fingerprint reruns schema preparation;
  3. concurrent opens seeing the same stale marker converge safely;
  4. a legacy connection rebuild ends with item_ids, last_health, and connection_uidx before stamping; and
  5. a failed schema ensure does not create the marker and succeeds on retry.

Before the change, the first test fails with all 46 schema statements present on the second open.

Merge-ready gates run locally:

  • bun run format:check
  • bun run lint
  • bun run typecheck — 43/43 packages
  • TURBO_TEST_CONCURRENCY=2 bun run test — 37/37 package tasks
  • bun run --cwd apps/host-cloudflare test — 22/22 tests, including the real workerd/Miniflare MCP flow

No changeset: @executor-js/host-cloudflare is private.

Prior work checked

No existing issue or PR title covers repeated Cloudflare D1 schema preparation. #1313 addresses session-start serialization, while #1409 reuses an already-open local/self-host database; neither changes this D1 path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant