Skip to content

record-adapter-do-sqlite: a Workers story for SQLite records - #215

Open
cuibonobo wants to merge 5 commits into
mainfrom
claude/issue-161-plan-s7lxp0
Open

record-adapter-do-sqlite: a Workers story for SQLite records#215
cuibonobo wants to merge 5 commits into
mainfrom
claude/issue-161-plan-s7lxp0

Conversation

@cuibonobo

Copy link
Copy Markdown
Member

Summary

  • Adds @haverstack/record-adapter-do-sqlite, a StackRecordAdapter over Cloudflare Durable Objects' SQLite storage — a Workers-native record adapter with no Node runtime dependency, closing record-adapter-do-sqlite — a Workers story that fits the sync SqlExecutor and the ownership rule #161.
  • Reuses SharedSqlRecordLogic, the FTS5 schema/strategy, query builder, cursor codec, and row mappers from @haverstack/sqlite-shared, via a new ./record subpath that excludes the Node-only token-store and file-lock code (a bare import of either survives esbuild's tree-shaking as a dead-but-still-imported node:crypto/node:fs module, which throws at load time in a Worker without nodejs_compat).
  • No lock file: a Durable Object id maps to exactly one running instance, so the platform itself is the single-writer guarantee. No persist/flush step: every write through ctx.storage.sql is durable by the time the call returns.
  • SqlExecutor (in @haverstack/sqlite-shared) gains a transaction<T>(fn: () => T): T primitive, replacing the raw BEGIN/COMMIT/ROLLBACK statements record-logic.ts issued directly. A pre-implementation spike against the real Workers runtime found DO SQLite rejects those statements outright, and does not roll back a write on a later exception the way an open SQL transaction would — its real primitive is ctx.storage.transactionSync(fn), a callback boundary three independent string-based exec() calls can't reach. record-adapter-sqlite's executor implements transaction() as literal BEGIN/COMMIT/ROLLBACK around fn(), behavior-identical to the code it replaces.

Spec

Yes — docs/spec/adapters.md updated:

  • New row in the adapter backends table, and a capabilities note (shares record-adapter-sqlite's capabilities via SharedSqlRecordLogic).
  • @haverstack/sqlite-shared's two entry points (. vs ./record) and why the split exists.
  • SqlExecutor.transaction() as the interface's transaction primitive, and why it replaced raw BEGIN/COMMIT/ROLLBACK.
  • § Concurrency & storage ownership: a new bullet for record-adapter-do-sqlite — "the DO is the lock."

README.md's package table and directory tree also updated.

Verification

  • pnpm run format:check && pnpm run lint && pnpm test && pnpm run build && pnpm run typecheck — all green across the entire workspace (11 packages), not just the touched ones.
  • record-adapter-sqlite's full existing suite (116 tests) passes unchanged after the transaction() refactor — confirms it's behavior-preserving.
  • record-adapter-do-sqlite's new suite (16 tests) runs against the real Workers runtime (@cloudflare/vitest-pool-workers), not environment: 'node' — a targeted subset (CRUD, FK/unique constraint mapping, FTS5, cursor pagination, associations, commitMigration), plus the load-bearing case for the transaction refactor: a rejected patchContent must leave the FTS index consistent with stored content, which only holds if the rollback actually rolls back.
  • The Workers-runtime findings that shaped this design (raw transactions rejected, no auto-rollback-on-throw, PRAGMA journal_mode = WAL rejected, FK/unique error strings match, FTS5 available, cursor/binding shape) came from a spike against the real runtime, not documentation assumptions.

Notes for reviewers

  • record-adapter-do-sqlite is deliberately absent from scripts/verify-pack.mjs's EXPECTATIONS map (see the comment added there) — its entry point uses ambient Durable Object globals that don't exist under plain Node, so a bare import() there would fail for reasons unrelated to packaging correctness. It's still packed and installed into the throwaway consumer, so a missing files entry or an accidentally-unbundled @haverstack/sqlite-shared would still be caught.
  • Adds .npmrc to keep @types/chai (pulled in only by this package's vitest 4 / @cloudflare/vitest-pool-workers) out of pnpm's shared @types hoist folder, where it was colliding with the chai types vendored in @vitest/expect@2 that the rest of the repo uses on vitest 2 — a real cross-package typecheck regression during development, fixed before it could land.
  • .gitignore gained a narrow exception for hand-authored ambient .d.ts files under tests/support/ (this package's cloudflare:test type reference) — the blanket *.d.ts rule is meant for build output, and was silently dropping this file.
  • Not urgent per the issue itself, but there's a real demand signal in the issue comments (a Cloudflare Pages + D1 + R2 site's migration decision hinges on whether this exists).

Generated by Claude Code

claude added 2 commits August 31, 2026 00:25
…ehavior for #161

Throwaway probes against the real Workers runtime (@cloudflare/vitest-pool-workers),
not the shipped adapter. Findings, to inform the real implementation:

- Raw BEGIN/COMMIT/ROLLBACK via sql.exec() is rejected outright.
- DO does NOT auto-rollback a write when a later exception is thrown --
  each sql.exec() auto-commits immediately, so no-op'ing BEGIN/COMMIT/ROLLBACK
  would silently break atomicity.
- ctx.storage.transactionSync(fn) does roll back on throw, commits on success,
  and passes return values through synchronously.
- PRAGMA foreign_keys = ON works and enforces; PRAGMA journal_mode = WAL is
  rejected (not authorized) -- expected, DO owns its own durability.
- FK/unique violation error strings match what isForeignKeyViolation/
  isUniqueConstraintViolation already check for.
- Cursor exposes toArray()/one()/raw()/rowsRead/rowsWritten; parameter
  binding is spread-args, same convention as node:sqlite.

Net conclusion: SharedSqlRecordLogic's exec('BEGIN')/'COMMIT'/'ROLLBACK' calls
can't translate to DO as-is -- transactionSync's callback boundary doesn't
match three independent string-based exec() calls. Needs a
SqlExecutor.transaction<T>(fn) primitive instead, touching record-logic.ts
and record-adapter-sqlite's executor, not just this new package.

pnpm-workspace.yaml: allow workerd's postinstall build script (needed by
wrangler/miniflare for the Workers test pool).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJoUkCo4dLjXsDsNDkYjbx
Implements StackRecordAdapter over Cloudflare Durable Objects' SQLite
storage (ctx.storage.sql) for Workers deployments with no Node runtime.
Reuses SharedSqlRecordLogic, the FTS5 schema/strategy, query builder,
cursor codec, and row mappers from @haverstack/sqlite-shared, via its new
./record subpath (the token-store and file-lock pieces stay Node-only and
unreachable from this adapter's bundle — a bare import of either survives
esbuild's tree-shaking as a dead-but-still-imported `node:crypto`/`node:fs`
module, which throws at load time in a Worker without nodejs_compat).

No lock file: a Durable Object id maps to exactly one running instance, so
the platform itself is the single-writer guarantee. No persist/flush step:
every write through ctx.storage.sql is durable by the time the call
returns.

sqlite-shared's SqlExecutor gains a transaction<T>(fn: () => T): T
primitive, replacing the raw BEGIN/COMMIT/ROLLBACK statements
record-logic.ts issued directly. A pre-implementation spike against the
real Workers runtime (@cloudflare/vitest-pool-workers) found DO SQLite
rejects those statements outright, and does not roll back a write on a
later exception the way an open SQL transaction would — its real
primitive is ctx.storage.transactionSync(fn), a callback boundary three
independent string-based exec() calls can't reach. record-adapter-sqlite's
executor implements transaction() as literal BEGIN/COMMIT/ROLLBACK around
fn(), behavior-identical to the code it replaces — its full 116-test suite
passes unchanged.

Tests run against the real Workers runtime, not `environment: 'node'` —
a targeted subset (CRUD, FK/unique constraint mapping, FTS5, cursor
pagination, associations, commitMigration), plus the load-bearing case for
the transaction refactor: a rejected patchContent must leave the FTS index
consistent with stored content, which only holds if the rollback actually
rolls back.

Also:
- docs/spec/adapters.md: new adapter in the backends table, capabilities
  notes, and Concurrency & storage ownership section ("the DO is the
  lock"); sqlite-shared's two entry points and transaction() rationale.
- README.md: package table and directory tree entries.
- scripts/verify-pack.mjs: notes why the new package is deliberately
  absent from the Node-import EXPECTATIONS map (its entry point uses
  ambient Durable Object globals that don't exist under plain Node).
- .npmrc: keep @types/chai (pulled in only by this package's vitest 4)
  out of pnpm's shared @types hoist, so it can't collide with the chai
  types vendored in @vitest/expect@2 that the rest of the repo uses.
- .gitignore: carve out an exception for hand-authored ambient .d.ts
  files under tests/support/ (the blanket *.d.ts rule is for build
  output; this package's cloudflare:test type reference isn't generated).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJoUkCo4dLjXsDsNDkYjbx
@changeset-bot

changeset-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9154fe3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@haverstack/record-adapter-do-sqlite Patch
@haverstack/record-adapter-sqlite Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

claude added 3 commits August 31, 2026 15:29
CI failed on every job (lint/build/test/format) with:
  [ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION] 1 lockfile entries failed
  verification: @cloudflare/workers-types@5.20260830.1 was published at
  2026-08-30T01:19:42.000Z, within the minimumReleaseAge cutoff.

That version is a transitive dependency of wrangler / @cloudflare/vitest-
pool-workers, not something this PR asked for directly, but @cloudflare/
workers-types publishes a new calendar-versioned release roughly every
24h — floating on "latest" for it is structurally incompatible with a
minimum-release-age policy, since the newest version is almost never
older than the policy's window. Pinned it to 5.20260817.1 (~2 weeks
aged) via a root pnpm.overrides entry, applying repo-wide regardless of
which dependent requests it.

That override reintroduces a real conflict this branch had already
worked around once: adding @cloudflare/workers-types back as a direct
devDependency (needed for tsup's isolated dts build of src/executor.ts,
which uses the ambient DurableObjectStorage/SqlStorage/SqlStorageValue
globals with no import and doesn't see wrangler's generated
worker-configuration.d.ts) reintroduced the ambient Env/Cloudflare.Env
duplicate-declaration conflict between the two type sources for the
*test* tsconfig, which only needs wrangler's generated types and never
needed workers-types at all. Fixed by scoping workers-types to tsup's
dts step alone (dts.compilerOptions.types), rather than the shared
tsconfig.json every tsc --noEmit run reads — the build gets the globals
it needs, the test typecheck never sees the conflicting second copy.

Verified with the same commands CI runs (format:check, build, test,
lint, typecheck, and a frozen-lockfile install) across the whole
workspace, plus confirmed no other lockfile entry is within the
release-age window.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJoUkCo4dLjXsDsNDkYjbx
… for pnpm 11

Second round of the same CI failure — this repo's workflow pins
pnpm/action-setup to major version 11, but my local pnpm was 10.33, which
silently accepted config locations that v11 no longer reads:

- package.json's "pnpm.overrides" field: v11 logs a warning and ignores
  it entirely, so the @cloudflare/workers-types pin from the previous fix
  never took effect in CI — [ERR_PNPM_LOCKFILE_CONFIG_MISMATCH], since
  the lockfile itself did carry the pinned resolution but the running
  config didn't request it.
- .npmrc's hoist-pattern: v11 doesn't read it either (no warning, just
  silently not applied) — confirmed via `pnpm config get hoist-pattern`
  returning undefined despite the file being present. @types/chai was
  back in the shared hoist folder, reproducing the duplicate-identifier
  typecheck failure across every package that the previous branch state
  had (apparently only coincidentally) stopped showing.

Both now live in pnpm-workspace.yaml, which v11 does read for both keys —
verified with `pnpm config get hoist-pattern`/`get overrides` actually
resolving, and a full clean install + frozen-lockfile install + build +
lint + test + typecheck pass, all run with pnpm@11.24.0 specifically
(matching what pnpm/action-setup resolves version: 11 to right now)
rather than relying on local pnpm 10, which had masked both of these.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJoUkCo4dLjXsDsNDkYjbx
Resolves conflicts with #160 (blob-adapter-s3), which merged into main
after this branch was created — both PRs added a row to the same
adapter-backends tables in README.md and docs/spec/adapters.md. Kept
both additions in each table; scripts/verify-pack.mjs and the README
package-tree section merged cleanly on their own.

Lockfile regenerated via `pnpm install --no-frozen-lockfile` (pnpm
11.24.0, matching CI) rather than hand-resolved, per the repo's own
tooling. Verified with format:check, build, lint, typecheck, and test
across the full merged workspace (blob-adapter-s3 and
record-adapter-do-sqlite both green), plus a from-scratch
--frozen-lockfile install with pnpm 11.24.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants