Skip to content

Add blob-adapter-s3: StackBlobAdapter over the S3 API - #213

Merged
cuibonobo merged 1 commit into
mainfrom
claude/issue-160-plan-w8kulm
Aug 31, 2026
Merged

Add blob-adapter-s3: StackBlobAdapter over the S3 API#213
cuibonobo merged 1 commit into
mainfrom
claude/issue-160-plan-w8kulm

Conversation

@cuibonobo

Copy link
Copy Markdown
Member

Summary

Adds @haverstack/blob-adapter-s3, a StackBlobAdapter over the S3 API for S3 and S3-compatible stores (Cloudflare R2 via its S3-compatible endpoint). Closes #160.

blob-adapter-disk was previously the only shipped StackBlobAdapter, even though combineAdapters() names SQLite + S3 as its own motivating example. This package closes that gap:

  • S3BlobAdapter mirrors blob-adapter-disk's content-addressed design — SHA-256 hex keys, same C3 error contract (StackNotFoundError for a missing key, StackQueryError for a malformed fileId).
  • endpoint + forcePathStyle options route through the same client to R2 or any other S3-compatible store, per the issue's ask.
  • Implements listFiles() (paginated ListObjectsV2) so Stack.collectAttachmentGarbage() can find bare-bytes orphans in object storage, same as the disk adapter.
  • maxAttachmentBytes isn't declared here — that field lives on AdapterCapabilities, which combineAdapters() always sources from the paired StackRecordAdapter, not the blob half. Documented this explicitly in docs/spec/adapters.md since the issue's own checklist item ("decide whether maxAttachmentBytes stays null here") assumed the blob adapter owns it.
  • Unlike disk's temp-file-plus-rename dance (needed because a local write can be torn), S3's PutObject is atomic per key, so putAttachment just does a HeadObject existence check (to skip a redundant upload) then PutObject — no rename trick needed, and a concurrent write of identical content to the same key is harmless by construction.

Uses @aws-sdk/client-s3 rather than a hand-rolled fetch + SigV4 implementation — this repo's adapters otherwise have zero non-@haverstack/* runtime dependencies, but getting AWS request signing and S3's XML error/listing parsing right by hand isn't worth the risk versus the maintained SDK.

Also wired the package into the repo's package-list surfaces: README (package table, capability matrix, packages tree), docs/spec/adapters.md (adapter backends table, and turned its S3BlobAdapter code example from "hypothetical" into a real one), CONTRIBUTING.md's package tree, scripts/verify-pack.mjs, and the .github/workflows/changeset.yml package dropdown (now the 9th manually-listed package — updated the "ninth package" note in CONTRIBUTING.md to "tenth" for the next one).

Out of scope per the issue's own cross-refs:

Not included, and needs a manual step outside this repo: the actual first npm publish + npm trusted-publisher configuration for @haverstack/blob-adapter-s3, per CONTRIBUTING.md § Releasing ("a ninth [now tenth] package needs one manual publish... then its own trusted publisher"). No changeset was added for this PR since changesets doesn't manage a package's first-ever publish, only version bumps of already-published packages.

Spec

Yes — updated docs/spec/adapters.md § Adapter backends: added blob-adapter-s3 to the adapter table, replaced the S3BlobAdapter code example (previously marked // hypothetical) with the real package, and added a paragraph clarifying that maxAttachmentBytes is declared by the paired StackRecordAdapter/combineAdapters(), not by a blob-only adapter.

Verification

pnpm run format:check && pnpm run lint && pnpm run build && pnpm run typecheck && pnpm run test
node scripts/verify-pack.mjs

All green, including verify-pack.mjs, which packs @haverstack/blob-adapter-s3 into a tarball, installs it into a project outside the workspace, and confirms S3BlobAdapter imports from it — catching the case where @aws-sdk/client-s3 resolves via the workspace link but isn't actually a declared dependency.

blob-adapter-s3's own test suite (18 tests) mocks @aws-sdk/client-s3 with aws-sdk-client-mock, backed by a small in-memory fake bucket (the mock library itself has no server-side state), and tracks blob-adapter-disk's test list: dedup, not-found, invalid-fileId, delete-is-idempotent, and listFiles() (empty/populated/ignores-non-hash-keys/reflects-deletes/carries-modifiedAt).

No live AWS or R2 calls in CI or locally — everything runs against the mock.

Notes for reviewers

  • SDK choice was a real trade-off: I chose @aws-sdk/client-s3 over a zero-dependency hand-rolled SigV4 client for correctness (signing and S3 XML parsing are easy to get subtly wrong), at the cost of being the first package in this repo with a non-@haverstack/*, non-tooling runtime dependency (@aws-sdk/client-s3 + its @smithy/* tree).
  • The client constructor option is an escape hatch for callers who need control over retry policy, credential providers, or middleware beyond what region/endpoint/forcePathStyle/credentials expose — and it's what the test suite uses to inject the mocked client.
  • Happy to add a follow-up issue for the manual first-publish/trusted-publisher step if useful, since I can't perform that from here.

Generated by Claude Code

Closes the gap combineAdapters() names as its own motivating example:
S3/R2-backed blob storage previously had no shipped adapter, so any
server deployment wanting object storage had to implement
StackBlobAdapter itself. blob-adapter-s3 mirrors blob-adapter-disk's
content-addressed design (SHA-256 keys, same C3 error contract) and
supports S3-compatible endpoints like Cloudflare R2 via `endpoint` +
`forcePathStyle`.

Uses @aws-sdk/client-s3 for correct SigV4 signing rather than a
hand-rolled implementation. Tests mock the S3 client with a small
stateful in-memory fake (aws-sdk-client-mock has no server-side state
of its own), mirroring blob-adapter-disk's test list.

Wires the new package into README, docs/spec/adapters.md,
CONTRIBUTING.md, scripts/verify-pack.mjs, and the changeset workflow's
package dropdown.
@changeset-bot

changeset-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 12f17be

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cuibonobo
cuibonobo merged commit 39bef78 into main Aug 31, 2026
5 checks passed
@cuibonobo
cuibonobo deleted the claude/issue-160-plan-w8kulm branch August 31, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

blob-adapter-s3 — blob storage is Node-disk-only, so object storage means writing your own

2 participants