From c24a92bb824cf4922040bff9516a2c061147c55e Mon Sep 17 00:00:00 2001 From: Julien Goux Date: Tue, 20 Oct 2026 10:14:36 +0000 Subject: [PATCH 01/24] feat(cli): lock and override service versions in stack mode (#7195) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Stack mode can now: - run any service version that slim-services has published; - match the hosted project's versions after `supabase link`; - reproduce the whole local stack from one committed file, `supabase/supabase.lock`. Before this change, every service version was compiled into the CLI. You couldn't try a hotfix, match production, or keep a team on the same builds without everyone running the same CLI release. Legacy (docker-compose) mode does not change, except for one fix: a `SUPABASE_PROJECT_ID` naming another project than the linked one now gets its pooler URL from the Management API, as `--project-ref` already did, instead of reusing the linked project's saved `supabase/.temp/pooler-url`. Everything below applies only to stack mode (`SUPABASE_EXPERIMENTAL_STACK=1` or `[experimental] stack = true`). > Version numbers and project refs in the examples are illustrative. --- ## The mental model: it works like a package manager If you know `package.json` + `pnpm-lock.yaml`, or `Cargo.toml` + `Cargo.lock`, you already know this. | Package manager | Supabase stack mode | Committed? | Purpose | | --- | --- | --- | --- | | `package.json` | `[local.services]` in `supabase/config.toml` | ✅ | What you **ask for**: `linked`, `bundled`, `v2.197.0`, `v2.197.0-r1` | | `pnpm-lock.yaml` | `supabase/supabase.lock` | ✅ | What you **got**: the exact build of every service, pinned by digest and checksum | | — | `supabase/.link.json` | ❌ (gitignored) | Which hosted project or branch *this checkout* targets | | `pnpm install` | `supabase start` / `supabase stack install` | | Resolves what changed, then runs or downloads the locked builds | | `pnpm update` | `supabase stack update [svc…]` | | Re-resolves on purpose and rewrites the lock | | `pnpm outdated` | `supabase stack outdated` | | Read-only report of what an update would change | | `pnpm install --frozen-lockfile` | `supabase start --frozen` / `supabase stack install --frozen` (implied when `CI` is set and a lock exists) | | Fails instead of changing the lock | ```mermaid flowchart LR subgraph git["Committed to git"] CFG["supabase/config.toml
[local.services]
what you ask for"] LOCK["supabase/supabase.lock
what you got"] end subgraph checkout["This checkout only"] LINK["supabase/.link.json
target hosted project"] end HOSTED[("Hosted project")] RELEASES[("slim-services releases")] CFG -->|specifiers| START["supabase start
stack install / stack update"] RELEASES -->|published builds + checksums| START START -->|writes when stale| LOCK HOSTED -->|versions it runs| LINKCMD["supabase link"] LINKCMD --> LINK LINKCMD -->|records observations,
resolves 'linked'| LOCK LOCK -->|exact pins| STACK[("Local stack")] ``` --- ## How a version is asked for ```toml [local.services] default = { version = "linked" } # the default: follow the linked hosted project [local.services.auth] version = "v2.197.0" # an upstream version: its highest published revision [local.services.realtime] version = "v2.144.2-r1" # exactly revision 1 of that upstream version ``` `bundled` means "the version this CLI ships with". `[db] orioledb_version` is shorthand for the database specifier, and a database version must match `db.major_version`. ```mermaid flowchart TD SPEC["specifier of a service"] --> KIND{"which kind?"} KIND -->|"linked (default)"| LINKED{"is the project linked,
and does it report a version?"} LINKED -->|never linked| B1["bundled pin
unlinked"] LINKED -->|service not reported| B2["bundled pin
unobserved"] LINKED -->|reported| PUB{"published by
slim-services?"} PUB -->|no| B3["bundled pin
unpublished"] PUB -->|yes| R1["the hosted release
resolved"] KIND -->|bundled| R2["this CLI's bundled pin"] KIND -->|"v2.197.0"| R3["highest published revision
of v2.197.0"] KIND -->|"v2.197.0-r1"| R4["exactly v2.197.0-r1"] ``` The fallback reasons (`unlinked`, `unobserved`, `unpublished`) are recorded in the lock, so nobody has to guess why a service isn't running the hosted version. ## What the lock contains One entry per service (database, rest, auth, realtime, storage, imgproxy, functions, studio, pgmeta, mail, analytics, pooler). Keys are sorted and the document is written atomically, so diffs and merge conflicts stay readable. ```jsonc { "$schema": "https://supabase.com/docs/cli/supabase-lock/v1.schema.json", "lockfileVersion": 1, "linked": { // the remote the observations came from "project": "abcdefghijklmnopqrst", "branch": { "ref": "abcdefghijklmnopqrst", "isDefault": true }, "observedAt": "2026-10-16T09:12:44.000Z" }, "services": { "auth": { "specifier": "linked", // what config.toml asked for "linked": { "raw": "v2.196.1", "version": "v2.196.1" }, // what the hosted project reported "resolution": "resolved", // or unlinked / unobserved / unpublished "resolved": { // the exact build that runs "upstreamVersion": "v2.196.1", "revision": 0, "upstreamImage": "supabase/gotrue:v2.196.1", "image": "ghcr.io/supabase/cli/gotrue:v2.196.1-r0@sha256:…", "natives": { "darwin-arm64": { "archive": "…", "manifest": "…" }, "…": {} } } }, "database": { "majorVersion": 17, "line": "stock", "…": "…" } // …one entry per service } } ``` --- ## What `supabase start` does with the lock ```mermaid flowchart TD S["supabase start"] --> FRESH{"lock exists, and every entry
still matches config.toml?"} FRESH -->|yes| RUN["run the locked builds
(no network, no Management API)"] FRESH -->|"no (missing or stale)"| FROZEN{"frozen?
--frozen, or CI with a lock"} FROZEN -->|yes| FAIL["fail: lock missing or stale
→ run supabase start locally and commit it"] FROZEN -->|no| LEASE["take the project lease
(serializes concurrent starts)"] LEASE --> RESOLVE["re-resolve only the services
whose specifier changed"] RESOLVE --> WRITE["write supabase.lock atomically"] WRITE --> RUN ``` A start never calls the Management API and needs no login. It only warns when: - `.link.json` targets another project or branch than the lock observed (run `supabase stack update`); - this CLI bundles a newer revision of a locked version. --- ## Typical workflows ### 1. New project, never linked ```console $ supabase start Resolved service versions: database 17.11.0.004-r0 (specifier linked; unlinked) auth v2.197.0-r0 (specifier linked; unlinked) realtime v2.144.2-r0 (specifier linked; unlinked) … Created supabase/supabase.lock. ``` Nothing is linked yet, so `linked` falls back to the bundled builds and says so. Commit `supabase/supabase.lock`, and every teammate and CI job now runs exactly these builds, whatever CLI version they have. ### 2. Match production: `supabase link` ```console $ supabase link --project-ref abcdefghijklmnopqrst Hosted versions: service hosted locked resolution database 17.6.1.011 17.6.1.011-r0 resolved auth v2.196.1 v2.196.1-r0 resolved realtime v2.143.0 v2.143.0-r2 resolved studio - 2026.10.12-… unobserved … Updated supabase/supabase.lock: now observing abcdefghijklmnopqrst (5 version changes). Review and commit it. Finished supabase link. ``` `link` probes the hosted project, or the branch, and records what it runs. Every `linked` service moves to that release. `link` writes `.link.json` and the lock together, under the same project lease as `start`. The next `supabase start` runs the hosted versions. Linking a preview branch works the same way. The lock records which project and branch it observed, so `start` warns if the checkout's link later points elsewhere. ### 3. Try one specific version, or a hotfix revision ```toml [local.services.auth] version = "v2.198.0" ``` ```console $ supabase start Resolved service versions: auth v2.196.1-r0 → v2.198.0-r0 (specifier v2.198.0 ≠ locked linked) Updated supabase/supabase.lock. ``` Only the changed entry is re-resolved, and every other pin stays where it was. Remove the override to go back to `linked`. Pinning `v2.198.0-r1` selects exactly that revision. ### 4. A teammate clones the repo, and CI ```console $ git clone … && cd my-app $ supabase start # the lock is fresh: runs the committed builds, no resolution ``` In CI (`CI` set and a lock present), `start`, `stack install` and `link` are frozen: a stale or missing lock fails instead of being rewritten. ```console $ CI=1 supabase start supabase/supabase.lock is stale for: auth (specifier v2.198.0 ≠ locked linked); run supabase start locally and commit the updated lock ``` `supabase stack install --frozen --runtime native` downloads everything the lock pins without creating a stack, which makes it handy as a cached CI warm-up step. ### 5. Upgrading the CLI Upgrading never changes locked versions. If the new CLI bundles a newer revision of a version you locked, `start` mentions it: ```console $ supabase start This CLI bundles newer revisions of locked versions: realtime v2.144.2-r0 → v2.144.2-r1. Run supabase stack outdated to review, supabase stack update to adopt them. ``` ```console $ supabase stack outdated SERVICE | SPECIFIER | HOSTED | LOCKED | UPDATE | BUNDLED | STATUS ----------|-----------|----------|-------------|-------------|-------------|------------------ auth | linked | v2.196.1 | v2.196.1-r0 | - | v2.197.0-r0 | up to date realtime | bundled | - | v2.144.2-r0 | v2.144.2-r1 | v2.144.2-r1 | update available … Run supabase stack update to adopt the updates. $ supabase stack update realtime # or no argument for every service ``` `stack outdated` is read-only. `stack update` re-resolves the named services even when the lock is fresh: - an upstream version moves to its newest revision; - `bundled` moves to the version this CLI bundles; - `linked` probes the hosted project again; - an exact `-rN` pin stays. ### 6. Unlinking `supabase unlink` removes `.link.json` and the stored credential under the project lease. The lock is not touched: it stays a valid, reviewed lock, and its `linked` entry still shows where the observations came from. --- ## Your local data is protected when versions change Changing a version must never silently corrupt a local database. | Change | What happens | | --- | --- | | A new revision of the same database build | Accepted | | Any other database build (another upstream version or Postgres major, or switching between stock and OrioleDB) with initialized data | Refused. The advice offers pinning the recorded build under `[local.services.database]` to keep the data (setting `[db] major_version` back or removing `[db] orioledb_version` where they would conflict with that pin), or `supabase stack destroy --stack-id …` (this deletes the local data) | | The same change on a database that never initialized | Adopted | | Moving auth, storage or realtime **back** to a version older than one that already migrated this database | Refused: a downgrade doesn't restore the old schema | `link` and `stack update` still move `linked` pins, but they warn when a saved stack's data would refuse the new pins. The database records the build that initialized it. Each schema-owning service records the newest version that migrated it. Snapshots, restores and `db reset` carry and respect both records. --- ## Stack mode no longer uses `supabase/.temp` | Before (`supabase/.temp`) | Now in stack mode | | --- | --- | | Project ref, pooler URL, branch | `supabase/.link.json` (gitignored) | | Postgres and edge-runtime versions | `supabase/supabase.lock` | | Storage uploads | `$SUPABASE_HOME/stacks//uploads`, removed by `stack destroy` | | Scratch files (pgdelta debug bundles, Functions bundling) | `$SUPABASE_HOME/tmp` | | Release-check cache | `$SUPABASE_HOME/cli-latest` | Delegated Go commands skip `.temp` too. Remote commands resolve the hosted target once per command, in this order: 1. the `--project-ref` flag; 2. `SUPABASE_PROJECT_ID`; 3. the link; 4. the interactive picker. Three commands change with this beyond the `.temp` move: - `gen keys` resolves its project this way, picker included, and passes it to the delegated command; - the upgrade notice picks its release-check cache by the project's mode, read from the working directory it was handed; - `db pull` builds its shadow database from the locked pins, falling back to the bundled database pin of the major a `[remotes]` block selects. `supabase stack install` replaces the experimental `supabase stack prepare`. --- ## Reading this PR There is one commit per chunk, in dependency order; each builds on the previous one. | Commit | What it adds | | --- | --- | | `docs(stack)` | ADRs 0026 and 0017: the lock-based version model | | `feat(config)` | `[local.services]` specifiers in the config schema | | `feat(stack)` resolve pins | Resolving slim-services releases with checksum, manifest and provenance verification | | `refactor(cli)` `.temp` layout | One owner for every legacy `.temp` path | | `feat(stack)` explicit pin | Every service creation carries the exact pin it runs | | `feat(cli)` `.link.json` | The hosted link file in stack mode | | `feat(stack)` lock format | The `supabase.lock` schema, deterministic write, typed read errors | | `feat(stack)` test stacks | `@supabase/stack` test stacks run the project lock | | `feat(stack)` database build guard | Initialized data is tied to its build | | `feat(cli)` hosted target | One hosted-target resolution per command | | `feat(stack)` schema writers | Refuses older schema writers on migrated databases | | `feat(cli)` lock at start | Resolve and lock at `start`; `--frozen` and CI behavior | | `feat(cli)` install | `stack install`, `stack prepare` removed | | `feat(cli)` update/outdated | `stack update` and `stack outdated` | | `feat(cli)` linked versions | `link` observes hosted versions and resolves `linked` | | `feat(cli)` no `.temp` | Stack mode leaves `supabase/.temp` alone | | `feat(cli)` Go off `.temp` | Delegated Go commands do the same | | `test(cli)` | Explicit timeouts on owner-spawning destroy tests | Review fixes and merges from `develop` follow these commits. User-facing reference: [`apps/cli/docs/stack-commands.md`](https://github.com/supabase/cli/blob/juliengoux/cli-2688-stack-mode-lockable-service-versions-with-hosted-parity-via/apps/cli/docs/stack-commands.md#service-versions) and the [`start` side effects](https://github.com/supabase/cli/blob/juliengoux/cli-2688-stack-mode-lockable-service-versions-with-hosted-parity-via/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md#service-versions-and-the-lock). --------- Co-authored-by: Colum Ferry --- .gitattributes | 1 + .github/scripts/slim-mirror-payload.ts | 17 +- .../scripts/sync-artifacts-catalog.test.ts | 36 +- .github/scripts/sync-artifacts-catalog.ts | 68 +- .oxlintrc.json | 4 + apps/cli-go/cmd/db.go | 2 + apps/cli-go/cmd/root.go | 5 +- apps/cli-go/cmd/root_test.go | 7 + apps/cli-go/internal/telemetry/project.go | 16 + .../cli-go/internal/telemetry/project_test.go | 25 + .../internal/utils/flags/project_ref.go | 4 + .../internal/utils/flags/project_ref_test.go | 14 + apps/cli-go/pkg/config/config.go | 26 +- apps/cli-go/pkg/config/config_test.go | 62 + apps/cli-go/pkg/config/utils.go | 34 + apps/cli/docs/go-cli-divergences.md | 15 +- apps/cli/docs/go-cli-porting-status.md | 7 + apps/cli/docs/stack-commands.md | 143 +- apps/cli/docs/supabase-home.md | 15 +- apps/cli/scripts/generate-docs.ts | 23 + apps/cli/src/cli/complete.ts | 43 +- apps/cli/src/cli/main.ts | 29 +- apps/cli/src/cli/root.ts | 13 +- ...undled-postgres-client.integration.test.ts | 5 +- .../bundled-postgres-client.ts | 9 +- .../command-permissions.unit.test.ts | 10 +- .../db-bootstrap/bootstrap-config.ts | 15 +- .../db-bootstrap/reset-local-database.ts | 18 +- .../db-bootstrap/rollback.unit.test.ts | 8 +- .../db-bootstrap/shadow-database.ts | 9 +- .../db-config.integration.test.ts | 152 +- .../src/command-internal/db-config.layer.ts | 76 +- .../src/command-internal/db-config.service.ts | 6 +- .../command-internal/db-config.toml-read.ts | 29 +- .../db-config.toml-read.unit.test.ts | 40 +- .../src/command-internal/db-config.types.ts | 10 +- apps/cli/src/command-internal/db-image.ts | 20 +- apps/cli/src/command-internal/db-pull-run.ts | 36 +- apps/cli/src/command-internal/db-push-core.ts | 2 +- .../command-internal/edge-runtime-image.ts | 74 +- ...e-runtime-script.layer.integration.test.ts | 100 ++ .../edge-runtime-script.layer.ts | 13 +- .../go-proxy-target-env.integration.test.ts | 223 +++ .../command-internal/go-proxy-target-env.ts | 39 + .../src/command-internal/go-proxy.layer.ts | 64 +- .../go-proxy.layer.unit.test.ts | 135 +- .../src/command-internal/hosted-versions.ts | 94 ++ .../legacy-project-state.layer.ts | 88 + .../legacy-project-state.paths.ts | 58 + .../legacy-project-state.service.ts | 55 + ...t.ts => legacy-project-state.unit.test.ts} | 59 +- .../link-services-core.integration.test.ts | 3 +- .../command-internal/link-services-core.ts | 80 +- .../cli/src/command-internal/link-snapshot.ts | 155 ++ apps/cli/src/command-internal/linked-state.ts | 42 +- .../management-api-runtime.layer.ts | 26 +- .../command-internal/migration-fetch-run.ts | 17 +- .../command-internal/parent-project-ref.ts | 153 +- .../pg-dump.native.integration.test.ts | 28 +- .../pg-dump.run.integration.test.ts | 5 +- apps/cli/src/command-internal/pg-dump.run.ts | 11 +- .../cli/src/command-internal/pgdelta.paths.ts | 9 +- .../command-internal/project-create-core.ts | 25 +- apps/cli/src/command-internal/project-link.ts | 230 +++ .../command-internal/project-state-mode.ts | 77 + .../src/command-internal/project-target.ts | 18 +- .../scratch-paths.integration.test.ts | 42 + .../cli/src/command-internal/scratch-paths.ts | 30 + .../service-version-overrides.ts | 32 +- .../service-version-overrides.unit.test.ts | 28 +- .../stack-artifact-selection.ts | 60 + .../stack-artifact-selection.unit.test.ts | 58 + .../cli/src/command-internal/stack-backend.ts | 89 +- .../stack-catalog-setup.integration.test.ts | 101 +- .../command-internal/stack-catalog-setup.ts | 20 +- apps/cli/src/command-internal/stack-config.ts | 187 ++- .../command-internal/stack-data-conflicts.ts | 43 + .../stack-database-version.ts | 31 - .../stack-database-version.unit.test.ts | 48 - .../command-internal/stack-local-database.ts | 50 +- .../stack-lock.integration.test.ts | 243 +++ apps/cli/src/command-internal/stack-lock.ts | 1431 +++++++++++++++++ .../command-internal/stack-lock.unit.test.ts | 16 + .../stack-project-selection.ts | 14 + .../stack-shadow-cache.integration.test.ts | 55 +- .../command-internal/stack-shadow-cache.ts | 46 +- .../stack-shadow.integration.test.ts | 258 ++- apps/cli/src/command-internal/stack-shadow.ts | 41 +- .../stack-storage.native.integration.test.ts | 1 + .../command-internal/start-secrets-cleanup.ts | 4 +- .../command-internal/storage-runtime.layer.ts | 8 +- apps/cli/src/command-internal/temp-paths.ts | 81 - .../tenant-versions.integration.test.ts | 90 ++ .../src/command-internal/tenant-versions.ts | 140 +- .../src/command-internal/test-db.handler.ts | 28 +- .../test-db.native.integration.test.ts | 3 +- .../upgrade-notice.integration.test.ts | 112 +- .../src/command-internal/upgrade-notice.ts | 32 +- .../upgrade-notice.unit.test.ts | 1 + .../src/commands/backups/list/list.command.ts | 2 +- .../src/commands/backups/list/list.handler.ts | 6 +- .../backups/restore/restore.command.ts | 2 +- .../backups/restore/restore.handler.ts | 6 +- .../src/commands/bootstrap/SIDE_EFFECTS.md | 17 +- .../commands/bootstrap/bootstrap.handler.ts | 387 +++-- .../bootstrap/bootstrap.integration.test.ts | 113 +- .../commands/bootstrap/bootstrap.layers.ts | 4 +- .../bootstrap/bootstrap.layers.unit.test.ts | 10 +- ...ootstrap.workdir-cache.integration.test.ts | 2 + .../branches/create/create.command.ts | 2 +- .../branches/create/create.handler.ts | 5 +- .../branches/delete/delete.command.ts | 2 +- .../branches/delete/delete.handler.ts | 5 +- .../branches/disable/disable.command.ts | 2 +- .../branches/disable/disable.handler.ts | 5 +- .../src/commands/branches/get/get.command.ts | 2 +- .../src/commands/branches/get/get.handler.ts | 5 +- .../commands/branches/list/SIDE_EFFECTS.md | 6 +- .../commands/branches/list/list.command.ts | 2 +- .../commands/branches/list/list.handler.ts | 16 +- .../commands/branches/pause/pause.command.ts | 2 +- .../commands/branches/pause/pause.handler.ts | 5 +- .../branches/unpause/unpause.command.ts | 2 +- .../branches/unpause/unpause.handler.ts | 5 +- .../branches/update/update.command.ts | 2 +- .../branches/update/update.handler.ts | 5 +- .../src/commands/config/diff/diff.command.ts | 2 +- .../src/commands/config/pull/pull.command.ts | 2 +- .../config/push/push.branch-target.ts | 23 +- .../src/commands/config/push/push.command.ts | 2 +- .../src/commands/config/push/push.handler.ts | 5 +- .../config/push/push.integration.test.ts | 2 +- .../commands/db/advisors/advisors.handler.ts | 8 +- .../db/advisors/advisors.integration.test.ts | 16 +- .../commands/db/advisors/advisors.layers.ts | 6 +- apps/cli/src/commands/db/diff/SIDE_EFFECTS.md | 6 +- apps/cli/src/commands/db/diff/diff.handler.ts | 15 +- .../commands/db/diff/diff.integration.test.ts | 14 +- .../db/diff/diff.stack-cache.e2e.test.ts | 3 +- apps/cli/src/commands/db/dump/SIDE_EFFECTS.md | 7 +- apps/cli/src/commands/db/dump/dump.handler.ts | 51 +- .../commands/db/dump/dump.integration.test.ts | 21 +- apps/cli/src/commands/db/dump/dump.layers.ts | 8 +- apps/cli/src/commands/db/lint/lint.handler.ts | 13 +- .../commands/db/lint/lint.integration.test.ts | 18 +- apps/cli/src/commands/db/lint/lint.layers.ts | 4 +- .../commands/db/lint/lint.layers.unit.test.ts | 8 +- apps/cli/src/commands/db/pull/SIDE_EFFECTS.md | 6 +- .../commands/db/pull/pull.integration.test.ts | 83 +- apps/cli/src/commands/db/push/SIDE_EFFECTS.md | 2 +- apps/cli/src/commands/db/push/push.command.ts | 2 +- apps/cli/src/commands/db/push/push.handler.ts | 9 +- .../commands/db/push/push.integration.test.ts | 88 +- apps/cli/src/commands/db/push/push.layers.ts | 4 +- .../src/commands/db/query/query.handler.ts | 10 +- .../db/query/query.integration.test.ts | 16 +- .../db/remote/changes/SIDE_EFFECTS.md | 1 + .../db/remote/changes/changes.handler.ts | 7 +- .../cli/src/commands/db/reset/SIDE_EFFECTS.md | 2 +- .../src/commands/db/reset/reset.handler.ts | 10 +- .../db/reset/reset.integration.test.ts | 146 +- .../cli/src/commands/db/reset/reset.layers.ts | 4 +- .../commands/db/reset/reset.stack.e2e.test.ts | 5 +- ...eclarative.orchestrate.integration.test.ts | 1 - .../declarative/declarative.smart-target.ts | 30 +- .../declarative/generate/generate.handler.ts | 45 +- .../generate/generate.integration.test.ts | 33 +- .../schema/declarative/sync/SIDE_EFFECTS.md | 4 +- .../schema/declarative/sync/sync.handler.ts | 34 +- .../declarative/sync/sync.integration.test.ts | 62 +- ...elta-engine.next.layer.integration.test.ts | 3 +- .../db/shared/pgdelta-engine.next.layer.ts | 11 +- .../db/shared/pgdelta-next-artifacts.ts | 9 +- .../pgdelta-next-artifacts.unit.test.ts | 9 +- ...elta-next-shadow.stack.integration.test.ts | 1 - .../cli/src/commands/db/start/SIDE_EFFECTS.md | 11 + .../db/start/start.integration.test.ts | 133 +- .../domains/activate/activate.command.ts | 2 +- .../domains/activate/activate.handler.ts | 6 +- .../commands/domains/create/create.command.ts | 2 +- .../commands/domains/create/create.handler.ts | 6 +- .../commands/domains/delete/delete.command.ts | 2 +- .../commands/domains/delete/delete.handler.ts | 6 +- .../src/commands/domains/get/get.command.ts | 2 +- .../src/commands/domains/get/get.handler.ts | 6 +- .../domains/reverify/reverify.command.ts | 2 +- .../domains/reverify/reverify.handler.ts | 6 +- .../get-root-key/get-root-key.command.ts | 2 +- .../get-root-key/get-root-key.handler.ts | 6 +- .../update-root-key.command.ts | 2 +- .../update-root-key.handler.ts | 6 +- .../experimental/compute/compute.output.ts | 2 +- .../compute/delete/delete.command.ts | 2 +- .../compute/delete/delete.handler.ts | 6 +- .../experimental/compute/list/list.command.ts | 2 +- .../experimental/compute/list/list.handler.ts | 6 +- .../experimental/compute/logs/logs.command.ts | 2 +- .../experimental/compute/logs/logs.handler.ts | 6 +- .../experimental/compute/push/push.command.ts | 2 +- .../experimental/compute/push/push.handler.ts | 6 +- .../compute/status/status.command.ts | 2 +- .../compute/status/status.handler.ts | 6 +- .../stack/destroy/SIDE_EFFECTS.md | 16 +- .../stack/destroy/destroy.command.ts | 2 +- .../stack/destroy/destroy.errors.ts | 11 +- .../stack/destroy/destroy.handler.ts | 54 +- .../stack/destroy/destroy.integration.test.ts | 628 +++++--- .../stack/install/SIDE_EFFECTS.md | 100 ++ .../stack/install/install.command.ts | 41 + .../stack/install/install.errors.ts | 52 + .../stack/install/install.handler.ts | 154 ++ .../stack/install/install.integration.test.ts | 350 ++++ .../stack/logs/logs.integration.test.ts | 12 +- .../stack/outdated/SIDE_EFFECTS.md | 67 + .../stack/outdated/outdated.command.ts | 23 + .../stack/outdated/outdated.errors.ts | 30 + .../stack/outdated/outdated.handler.ts | 133 ++ .../outdated/outdated.integration.test.ts | 218 +++ .../stack/prepare/SIDE_EFFECTS.md | 58 - .../stack/prepare/prepare.command.ts | 48 - .../stack/prepare/prepare.errors.ts | 39 - .../stack/prepare/prepare.handler.ts | 185 --- .../stack/prepare/prepare.integration.test.ts | 483 ------ .../stack/restart/restart.integration.test.ts | 3 + .../stack/stack-backend.integration.test.ts | 65 +- ...tack-command-telemetry.integration.test.ts | 20 - ...ack-config-environment.integration.test.ts | 10 +- .../stack/stack-config.integration.test.ts | 41 +- .../stack-forwarding.integration.test.ts | 7 +- .../experimental/stack/stack.command.ts | 19 +- .../experimental/stack/stack.flags.ts | 9 + .../experimental/stack/start/SIDE_EFFECTS.md | 148 +- .../start-export-pointer.integration.test.ts | 3 +- .../experimental/stack/start/start.command.ts | 4 +- .../experimental/stack/start/start.errors.ts | 9 +- .../experimental/stack/start/start.handler.ts | 137 +- .../stack/start/start.integration.test.ts | 842 ++++++++-- .../start/start.native.integration.test.ts | 82 +- .../experimental/stack/start/start.options.ts | 5 - .../experimental/stack/status/SIDE_EFFECTS.md | 8 +- .../stack/status/status.handler.ts | 17 +- .../stack/status/status.integration.test.ts | 18 +- .../stack/stop/stop.integration.test.ts | 8 +- .../experimental/stack/update/SIDE_EFFECTS.md | 114 ++ .../stack/update/update.command.ts | 61 + .../stack/update/update.errors.ts | 37 + .../stack/update/update.handler.ts | 143 ++ .../stack/update/update.integration.test.ts | 441 +++++ .../src/commands/feedback/add/SIDE_EFFECTS.md | 2 +- .../src/commands/feedback/add/add.handler.ts | 2 +- .../feedback/add/add.integration.test.ts | 6 +- .../feedback/delete/delete.handler.ts | 2 +- .../delete/delete.integration.test.ts | 4 +- .../commands/feedback/feedback-project-ref.ts | 28 +- .../functions/delete/delete.command.ts | 2 +- .../functions/delete/delete.handler.ts | 11 +- .../functions/deploy/deploy.command.ts | 2 +- .../functions/deploy/deploy.handler.ts | 15 +- .../deploy/deploy.integration.test.ts | 124 ++ .../functions/download/SIDE_EFFECTS.md | 4 +- .../functions/download/download.command.ts | 2 +- .../functions/download/download.handler.ts | 18 +- .../download/download.integration.test.ts | 54 + .../commands/functions/list/list.command.ts | 2 +- .../commands/functions/list/list.handler.ts | 11 +- .../functions/list/list.integration.test.ts | 8 +- .../functions/serve/serve.stack.e2e.test.ts | 9 +- .../functions/serve/serve.stack.handler.ts | 37 +- .../serve/serve.stack.integration.test.ts | 76 +- .../commands/gen/bearer-jwt/SIDE_EFFECTS.md | 2 +- .../cli/src/commands/gen/keys/SIDE_EFFECTS.md | 1 + .../cli/src/commands/gen/keys/keys.handler.ts | 8 +- .../src/commands/gen/types/SIDE_EFFECTS.md | 2 +- .../src/commands/gen/types/types.command.ts | 2 +- .../src/commands/gen/types/types.handler.ts | 44 +- .../gen/types/types.integration.test.ts | 38 + .../src/commands/gen/types/types.layers.ts | 8 +- .../gen/types/types.layers.unit.test.ts | 8 +- .../src/commands/inspect/db/SIDE_EFFECTS.md | 6 +- .../commands/inspect/report/SIDE_EFFECTS.md | 2 +- apps/cli/src/commands/link/SIDE_EFFECTS.md | 141 +- apps/cli/src/commands/link/link.command.ts | 17 +- apps/cli/src/commands/link/link.handler.ts | 322 +++- .../commands/link/link.integration.test.ts | 1 + .../link/link.lock.integration.test.ts | 636 ++++++++ .../link/link.stack-mode.integration.test.ts | 371 +++++ .../commands/migration/down/down.handler.ts | 12 +- .../migration/down/down.integration.test.ts | 8 +- .../migration/fetch/fetch.integration.test.ts | 8 +- .../commands/migration/list/list.handler.ts | 15 +- .../migration/list/list.integration.test.ts | 40 +- .../migration/repair/repair.handler.ts | 12 +- .../repair/repair.integration.test.ts | 8 +- .../commands/migration/squash/SIDE_EFFECTS.md | 2 +- .../migration/squash/squash.handler.ts | 11 +- .../squash/squash.integration.test.ts | 8 +- .../squash/squash.native.integration.test.ts | 8 +- .../src/commands/migration/up/up.handler.ts | 15 +- .../migration/up/up.integration.test.ts | 8 +- .../commands/network-bans/get/get.command.ts | 2 +- .../commands/network-bans/get/get.handler.ts | 6 +- .../network-bans/remove/remove.command.ts | 2 +- .../network-bans/remove/remove.handler.ts | 6 +- .../network-restrictions/get/get.command.ts | 2 +- .../network-restrictions/get/get.handler.ts | 6 +- .../update/update.command.ts | 2 +- .../update/update.handler.ts | 6 +- .../commands/notebooks/pull/pull.command.ts | 2 +- .../commands/notebooks/pull/pull.handler.ts | 6 +- .../commands/notebooks/push/push.command.ts | 2 +- .../commands/notebooks/push/push.handler.ts | 6 +- .../postgres-config/delete/delete.command.ts | 2 +- .../postgres-config/delete/delete.handler.ts | 6 +- .../postgres-config/get/get.command.ts | 2 +- .../postgres-config/get/get.handler.ts | 6 +- .../postgres-config/update/update.command.ts | 2 +- .../postgres-config/update/update.handler.ts | 6 +- .../projects/api-keys/api-keys.command.ts | 2 +- .../projects/api-keys/api-keys.handler.ts | 6 +- .../commands/projects/delete/SIDE_EFFECTS.md | 42 +- .../projects/delete/delete.handler.ts | 58 +- .../delete/delete.integration.test.ts | 83 + .../commands/projects/list/list.handler.ts | 14 +- apps/cli/src/commands/pull/SIDE_EFFECTS.md | 2 +- apps/cli/src/commands/pull/pull.command.ts | 2 +- apps/cli/src/commands/pull/pull.layers.ts | 4 +- apps/cli/src/commands/pull/pull.steps.ts | 7 +- .../src/commands/secrets/list/list.command.ts | 2 +- .../src/commands/secrets/list/list.handler.ts | 6 +- .../src/commands/secrets/set/set.command.ts | 2 +- .../src/commands/secrets/set/set.handler.ts | 6 +- .../commands/secrets/unset/unset.command.ts | 2 +- .../commands/secrets/unset/unset.handler.ts | 6 +- .../commands/seed/buckets/buckets.handler.ts | 6 +- .../seed/buckets/buckets.integration.test.ts | 43 +- .../cli/src/commands/services/SIDE_EFFECTS.md | 6 +- .../commands/services/services-local-stack.ts | 70 +- .../src/commands/services/services.handler.ts | 28 +- .../services/services.integration.test.ts | 30 +- .../snippets/download/download.command.ts | 2 +- .../snippets/download/download.handler.ts | 6 +- .../commands/snippets/list/list.command.ts | 2 +- .../commands/snippets/list/list.handler.ts | 6 +- .../ssl-enforcement/get/get.command.ts | 2 +- .../ssl-enforcement/get/get.handler.ts | 6 +- .../ssl-enforcement/update/update.command.ts | 2 +- .../ssl-enforcement/update/update.handler.ts | 6 +- apps/cli/src/commands/sso/add/add.command.ts | 2 +- apps/cli/src/commands/sso/add/add.handler.ts | 6 +- .../cli/src/commands/sso/info/info.command.ts | 2 +- .../cli/src/commands/sso/info/info.handler.ts | 6 +- .../cli/src/commands/sso/list/list.command.ts | 2 +- .../cli/src/commands/sso/list/list.handler.ts | 6 +- .../src/commands/sso/remove/remove.command.ts | 2 +- .../src/commands/sso/remove/remove.handler.ts | 6 +- .../cli/src/commands/sso/show/show.command.ts | 2 +- .../cli/src/commands/sso/show/show.handler.ts | 6 +- .../src/commands/sso/update/update.command.ts | 2 +- .../src/commands/sso/update/update.handler.ts | 6 +- .../status/status.integration.test.ts | 72 +- .../cli/src/commands/storage/cp/cp.handler.ts | 6 +- .../cli/src/commands/storage/ls/ls.handler.ts | 6 +- .../cli/src/commands/storage/mv/mv.handler.ts | 6 +- .../cli/src/commands/storage/rm/rm.handler.ts | 6 +- apps/cli/src/commands/test/db/SIDE_EFFECTS.md | 7 +- apps/cli/src/commands/unlink/SIDE_EFFECTS.md | 19 +- apps/cli/src/commands/unlink/unlink.errors.ts | 8 +- .../cli/src/commands/unlink/unlink.handler.ts | 136 +- .../unlink/unlink.integration.test.ts | 236 ++- .../activate/activate.command.ts | 2 +- .../activate/activate.handler.ts | 6 +- .../check-availability.command.ts | 2 +- .../check-availability.handler.ts | 6 +- .../delete/delete.command.ts | 2 +- .../delete/delete.handler.ts | 6 +- .../vanity-subdomains/get/get.command.ts | 2 +- .../vanity-subdomains/get/get.handler.ts | 6 +- .../project-ref.layer.integration.test.ts | 123 -- apps/cli/src/config/project-ref.layer.ts | 155 -- apps/cli/src/config/project-ref.service.ts | 77 - .../remote-target.layer.integration.test.ts | 255 +++ apps/cli/src/config/remote-target.layer.ts | 277 ++++ ...st.ts => remote-target.layer.unit.test.ts} | 92 +- apps/cli/src/config/remote-target.service.ts | 113 ++ .../src/shared/cli/hidden-flag.unit.test.ts | 2 + apps/cli/src/shared/cli/run.ts | 4 + apps/cli/src/shared/compute/compute-paths.ts | 3 +- apps/cli/src/shared/functions/deploy.ts | 3 +- apps/cli/src/shared/functions/download.ts | 44 +- .../src/shared/functions/functions.shared.ts | 55 +- .../functions/functions.shared.unit.test.ts | 78 +- apps/cli/src/shared/functions/serve.ts | 8 +- .../telemetry/__fixtures__/error-tags.txt | 8 +- .../telemetry/identify-project-groups.ts | 29 + .../analytics.layer.integration.test.ts | 78 +- apps/cli/src/telemetry/analytics.layer.ts | 210 +-- .../linked-project-cache.integration.test.ts | 60 + .../telemetry/linked-project-cache.layer.ts | 81 +- .../telemetry/linked-project-cache.service.ts | 5 +- apps/cli/tests/helpers/command-mocks.ts | 18 +- apps/cli/tests/helpers/compute.ts | 14 +- apps/cli/tests/helpers/legacy-state-guard.ts | 78 + apps/cli/tests/helpers/mocks.ts | 2 + apps/cli/tests/helpers/notebooks.ts | 14 +- .../tests/helpers/permission-real-layers.ts | 7 +- apps/cli/tests/helpers/pins.ts | 21 + apps/cli/tests/helpers/release-archive.ts | 25 + apps/cli/tests/helpers/release-fixture.ts | 120 ++ apps/cli/tests/helpers/remote-target.ts | 73 + .../helpers/stack-cleanup.integration.test.ts | 8 +- apps/cli/tests/helpers/stack-lock.ts | 65 + apps/cli/tests/helpers/storage.ts | 14 +- apps/docs/public/cli/config.schema.json | 170 ++ .../public/cli/supabase-lock/v1.schema.json | 267 +++ ...7-simplified-managed-stack-architecture.md | 6 + docs/adr/0026-slim-artifact-mirrors.md | 51 +- packages/config/docs/cli-config-loading.md | 2 +- packages/config/src/base.ts | 2 + .../config/src/entrypoint-purity.unit.test.ts | 1 + packages/config/src/local.ts | 75 + packages/config/src/local.unit.test.ts | 66 + .../src/project-config/hosted-sections.ts | 3 +- .../project-config.unit.test.ts | 1 + packages/stack/ARCHITECTURE.md | 26 +- packages/stack/README.md | 29 +- .../stack/scripts/prepare-test-artifacts.ts | 8 +- packages/stack/src/Artifacts.ts | 142 +- .../stack/src/Commands.integration.test.ts | 30 +- packages/stack/src/Commands.ts | 33 +- packages/stack/src/Defaults.ts | 3 + packages/stack/src/FailureKind.ts | 3 + .../stack/src/HostProcess.integration.test.ts | 5 +- packages/stack/src/Lock.integration.test.ts | 261 +++ packages/stack/src/Lock.ts | 227 +++ .../src/Owner.analytics.integration.test.ts | 11 +- packages/stack/src/Owner.integration.test.ts | 77 +- .../stack/src/Owner.logs.integration.test.ts | 15 +- ...er.native-socket-sweep.integration.test.ts | 3 +- packages/stack/src/Owner.ts | 121 +- packages/stack/src/Ports.integration.test.ts | 19 +- packages/stack/src/PromiseClient.ts | 28 +- packages/stack/src/Rpc.ts | 5 +- ...ost.container-shutdown.integration.test.ts | 13 +- .../stack/src/StackHost.integration.test.ts | 22 +- packages/stack/src/StackHost.ts | 10 +- packages/stack/src/StackNamespace.ts | 1 + .../stack/src/composition/EndpointReplan.ts | 3 +- .../Supabase.native.integration.test.ts | 100 +- packages/stack/src/composition/Supabase.ts | 72 +- .../src/composition/Supabase.unit.test.ts | 44 +- .../stack/src/credentials.integration.test.ts | 6 +- .../src/data-conflicts.integration.test.ts | 216 +++ .../stack/src/defaults.integration.test.ts | 9 +- packages/stack/src/effect.integration.test.ts | 241 ++- packages/stack/src/effect.ts | 95 +- .../CommandAttachments.integration.test.ts | 3 +- ...dRunner.initialization.integration.test.ts | 132 +- ...dRunner.native-cleanup.integration.test.ts | 4 +- packages/stack/src/host/CommandRunner.ts | 47 +- .../src/host/LogForwarder.integration.test.ts | 7 +- .../host/LogflareStorage.integration.test.ts | 7 +- packages/stack/src/index.ts | 1 + packages/stack/src/internal/artifacts.ts | 12 +- packages/stack/src/internal/data-conflicts.ts | 119 ++ packages/stack/src/internal/database-reuse.ts | 237 ++- .../src/internal/database-reuse.unit.test.ts | 181 ++- .../schema-writers.integration.test.ts | 45 + packages/stack/src/internal/schema-writers.ts | 307 ++++ .../src/internal/schema-writers.unit.test.ts | 61 + .../stack/src/lifetime.integration.test.ts | 3 + packages/stack/src/namespace/Lock.ts | 49 + .../stack/src/namespace/Namespace.e2e.test.ts | 7 +- .../namespace/Namespace.integration.test.ts | 6 +- packages/stack/src/namespace/Publication.ts | 4 +- packages/stack/src/namespace/Registry.ts | 41 +- .../stack/src/port-reservation.e2e.test.ts | 8 +- .../stack/src/preparation/PrepareSelection.ts | 109 ++ .../ReleaseResolver.integration.test.ts | 462 ++++++ .../stack/src/preparation/ReleaseResolver.ts | 462 ++++++ .../stack/src/preparation/SlimEndpoints.ts | 22 + packages/stack/src/preparation/SlimRelease.ts | 66 + .../slim-services.integration.test.ts | 47 +- .../stack/src/promise-api.integration.test.ts | 28 +- packages/stack/src/promise-api.unit.test.ts | 18 +- packages/stack/src/public.e2e.test.ts | 41 +- .../src/runtime/Container.integration.test.ts | 8 +- packages/stack/src/runtime/Container.ts | 24 +- .../src/schema-writers.integration.test.ts | 386 +++++ .../stack/src/services/Analytics.unit.test.ts | 2 + .../AnalyticsImgproxy.integration.test.ts | 6 +- .../src/services/AuthSettings.unit.test.ts | 3 + .../services/AuthStorage.integration.test.ts | 15 +- .../src/services/Catalog.integration.test.ts | 11 +- packages/stack/src/services/Catalog.ts | 29 +- .../src/services/Database.integration.test.ts | 281 +++- packages/stack/src/services/Database.ts | 222 +-- .../DatabaseSnapshot.integration.test.ts | 133 +- .../stack/src/services/DatabaseSnapshot.ts | 116 +- .../services/Functions.integration.test.ts | 9 + packages/stack/src/services/Initialization.ts | 10 +- .../src/services/Mail.integration.test.ts | 22 +- .../src/services/Pooler.integration.test.ts | 4 +- .../ProcessRecipe.integration.test.ts | 15 +- packages/stack/src/services/ProcessRecipe.ts | 8 +- .../src/services/Realtime.integration.test.ts | 12 +- .../stack/src/services/Realtime.unit.test.ts | 2 + packages/stack/src/services/Recipe.ts | 6 +- .../src/services/Rest.integration.test.ts | 7 +- .../src/services/ServiceIdentity.unit.test.ts | 7 + .../StorageGateway.integration.test.ts | 4 +- .../src/services/Studio.integration.test.ts | 7 +- .../stack/src/services/Studio.unit.test.ts | 7 +- .../DockerDatabaseStorage.integration.test.ts | 429 ++--- .../src/storage/DockerDatabaseStorage.ts | 172 +- .../src/storage/DockerSnapshotBackend.ts | 3 +- .../stack/src/testing.integration.test.ts | 71 +- packages/stack/src/testing.ts | 502 ++++-- .../tests/docker-fixture.integration.test.ts | 5 +- .../docker-storage-publish-loop-fixture.ts | 3 +- packages/stack/tests/helper-owner-fixture.ts | 5 +- packages/stack/tests/locks.ts | 24 + .../stack/tests/namespace-lock-fixture.ts | 6 +- packages/stack/tests/owner-exit-client.ts | 2 + packages/stack/tests/pins.ts | 11 + packages/stack/tests/public-client-fixture.ts | 2 + .../stack/tests/session-client-fixture.ts | 2 + .../tests/snapshot-restore-barrier-fixture.ts | 64 + packages/stack/tests/test-stack-client.ts | 3 +- packages/stack/tests/whole-stack/fixture.ts | 22 +- turbo.json | 5 +- 530 files changed, 21067 insertions(+), 5241 deletions(-) create mode 100644 apps/cli/src/command-internal/go-proxy-target-env.integration.test.ts create mode 100644 apps/cli/src/command-internal/go-proxy-target-env.ts create mode 100644 apps/cli/src/command-internal/hosted-versions.ts create mode 100644 apps/cli/src/command-internal/legacy-project-state.layer.ts create mode 100644 apps/cli/src/command-internal/legacy-project-state.paths.ts create mode 100644 apps/cli/src/command-internal/legacy-project-state.service.ts rename apps/cli/src/command-internal/{temp-paths.unit.test.ts => legacy-project-state.unit.test.ts} (63%) create mode 100644 apps/cli/src/command-internal/link-snapshot.ts create mode 100644 apps/cli/src/command-internal/project-link.ts create mode 100644 apps/cli/src/command-internal/project-state-mode.ts create mode 100644 apps/cli/src/command-internal/scratch-paths.integration.test.ts create mode 100644 apps/cli/src/command-internal/scratch-paths.ts create mode 100644 apps/cli/src/command-internal/stack-artifact-selection.ts create mode 100644 apps/cli/src/command-internal/stack-artifact-selection.unit.test.ts create mode 100644 apps/cli/src/command-internal/stack-data-conflicts.ts delete mode 100644 apps/cli/src/command-internal/stack-database-version.ts delete mode 100644 apps/cli/src/command-internal/stack-database-version.unit.test.ts create mode 100644 apps/cli/src/command-internal/stack-lock.integration.test.ts create mode 100644 apps/cli/src/command-internal/stack-lock.ts create mode 100644 apps/cli/src/command-internal/stack-lock.unit.test.ts create mode 100644 apps/cli/src/command-internal/stack-project-selection.ts delete mode 100644 apps/cli/src/command-internal/temp-paths.ts create mode 100644 apps/cli/src/command-internal/tenant-versions.integration.test.ts create mode 100644 apps/cli/src/commands/experimental/stack/install/SIDE_EFFECTS.md create mode 100644 apps/cli/src/commands/experimental/stack/install/install.command.ts create mode 100644 apps/cli/src/commands/experimental/stack/install/install.errors.ts create mode 100644 apps/cli/src/commands/experimental/stack/install/install.handler.ts create mode 100644 apps/cli/src/commands/experimental/stack/install/install.integration.test.ts create mode 100644 apps/cli/src/commands/experimental/stack/outdated/SIDE_EFFECTS.md create mode 100644 apps/cli/src/commands/experimental/stack/outdated/outdated.command.ts create mode 100644 apps/cli/src/commands/experimental/stack/outdated/outdated.errors.ts create mode 100644 apps/cli/src/commands/experimental/stack/outdated/outdated.handler.ts create mode 100644 apps/cli/src/commands/experimental/stack/outdated/outdated.integration.test.ts delete mode 100644 apps/cli/src/commands/experimental/stack/prepare/SIDE_EFFECTS.md delete mode 100644 apps/cli/src/commands/experimental/stack/prepare/prepare.command.ts delete mode 100644 apps/cli/src/commands/experimental/stack/prepare/prepare.errors.ts delete mode 100644 apps/cli/src/commands/experimental/stack/prepare/prepare.handler.ts delete mode 100644 apps/cli/src/commands/experimental/stack/prepare/prepare.integration.test.ts create mode 100644 apps/cli/src/commands/experimental/stack/stack.flags.ts create mode 100644 apps/cli/src/commands/experimental/stack/update/SIDE_EFFECTS.md create mode 100644 apps/cli/src/commands/experimental/stack/update/update.command.ts create mode 100644 apps/cli/src/commands/experimental/stack/update/update.errors.ts create mode 100644 apps/cli/src/commands/experimental/stack/update/update.handler.ts create mode 100644 apps/cli/src/commands/experimental/stack/update/update.integration.test.ts create mode 100644 apps/cli/src/commands/link/link.lock.integration.test.ts create mode 100644 apps/cli/src/commands/link/link.stack-mode.integration.test.ts delete mode 100644 apps/cli/src/config/project-ref.layer.integration.test.ts delete mode 100644 apps/cli/src/config/project-ref.layer.ts delete mode 100644 apps/cli/src/config/project-ref.service.ts create mode 100644 apps/cli/src/config/remote-target.layer.integration.test.ts create mode 100644 apps/cli/src/config/remote-target.layer.ts rename apps/cli/src/config/{project-ref.layer.unit.test.ts => remote-target.layer.unit.test.ts} (84%) create mode 100644 apps/cli/src/config/remote-target.service.ts create mode 100644 apps/cli/src/shared/telemetry/identify-project-groups.ts create mode 100644 apps/cli/tests/helpers/legacy-state-guard.ts create mode 100644 apps/cli/tests/helpers/pins.ts create mode 100644 apps/cli/tests/helpers/release-archive.ts create mode 100644 apps/cli/tests/helpers/release-fixture.ts create mode 100644 apps/cli/tests/helpers/remote-target.ts create mode 100644 apps/cli/tests/helpers/stack-lock.ts create mode 100644 apps/docs/public/cli/supabase-lock/v1.schema.json create mode 100644 packages/config/src/local.ts create mode 100644 packages/config/src/local.unit.test.ts create mode 100644 packages/stack/src/Lock.integration.test.ts create mode 100644 packages/stack/src/Lock.ts create mode 100644 packages/stack/src/data-conflicts.integration.test.ts create mode 100644 packages/stack/src/internal/data-conflicts.ts create mode 100644 packages/stack/src/internal/schema-writers.integration.test.ts create mode 100644 packages/stack/src/internal/schema-writers.ts create mode 100644 packages/stack/src/internal/schema-writers.unit.test.ts create mode 100644 packages/stack/src/namespace/Lock.ts create mode 100644 packages/stack/src/preparation/PrepareSelection.ts create mode 100644 packages/stack/src/preparation/ReleaseResolver.integration.test.ts create mode 100644 packages/stack/src/preparation/ReleaseResolver.ts create mode 100644 packages/stack/src/preparation/SlimEndpoints.ts create mode 100644 packages/stack/src/preparation/SlimRelease.ts create mode 100644 packages/stack/src/schema-writers.integration.test.ts create mode 100644 packages/stack/tests/locks.ts create mode 100644 packages/stack/tests/pins.ts create mode 100644 packages/stack/tests/snapshot-restore-barrier-fixture.ts diff --git a/.gitattributes b/.gitattributes index 7f30866cca..5aa13a6aaa 100644 --- a/.gitattributes +++ b/.gitattributes @@ -30,6 +30,7 @@ packages/stack/src/functions/generated/* text eol=lf linguist-generated=true apps/cli/src/shared/feedback/database.types.ts linguist-generated=true apps/docs/public/cli/config.schema.json linguist-generated=true apps/docs/public/cli/project-config.schema.json linguist-generated=true +apps/docs/public/cli/supabase-lock/v1.schema.json linguist-generated=true # Lockfiles. Collapsed rather than hidden -- a dependency change is still one # click away in review, which is the same treatment GitHub gives package-lock.json. diff --git a/.github/scripts/slim-mirror-payload.ts b/.github/scripts/slim-mirror-payload.ts index 5b61fff58e..67996f9bd8 100644 --- a/.github/scripts/slim-mirror-payload.ts +++ b/.github/scripts/slim-mirror-payload.ts @@ -2,6 +2,10 @@ * Untrusted `mirror-slim-image` dispatch fields. Source and destination URLs * are derived here; payload strings are only accepted when they match. */ +import { + NATIVE_TARGETS, + parseSha256Sums, +} from "../../packages/stack/src/preparation/SlimRelease.ts"; const SERVICE_PATTERN = /^[a-z][a-z0-9-]*$/; const VERSION_PATTERN = /^[A-Za-z0-9._-]+$/; @@ -9,7 +13,6 @@ export const DIGEST_PATTERN = /^sha256:[0-9a-f]{64}$/; export const SOURCE_REGISTRY = "ghcr.io/supabase/cli"; export const DEST_REGISTRY = "public.ecr.aws/supabase/cli"; -const NATIVE_TARGETS = ["linux-arm64", "linux-amd64", "darwin-arm64"] as const; export class InvalidPayloadError extends Error {} @@ -209,12 +212,12 @@ export const nativeTripletDigests = (rawManifest: string): NativeFiles | undefin }; }; -export const checksumFor = (contents: string, archiveName: string): string | undefined => - contents - .split(/\r?\n/u) - .map((line) => line.trim().match(/^([a-f0-9]{64})\s+[* ]?(.+)$/iu)) - .find((match) => match?.[2] === archiveName || match?.[2]?.endsWith(`/${archiveName}`))?.[1] - ?.toLowerCase(); +/** The lowercase digest `archiveName` has in a `SHA256SUMS` body; throws on a malformed body. */ +export const checksumFor = (contents: string, archiveName: string): string | undefined => { + const parsed = parseSha256Sums(contents); + if (parsed.kind === "malformed") throw new InvalidPayloadError(parsed.message); + return parsed.sums.get(archiveName); +}; export const manifestMatches = ( manifestJson: string, diff --git a/.github/scripts/sync-artifacts-catalog.test.ts b/.github/scripts/sync-artifacts-catalog.test.ts index 648b4d65a4..c2272e6c48 100644 --- a/.github/scripts/sync-artifacts-catalog.test.ts +++ b/.github/scripts/sync-artifacts-catalog.test.ts @@ -57,12 +57,6 @@ const manifestWithUpstreamImage = async () => JSON.stringify({ upstream_image: image }); -/** A `fetchProvenance` returning a mirrored service's `source`. */ -const provenanceWithSource = - (source: string): RevisionIo["fetchProvenance"] => - async () => - JSON.stringify({ source }); - const unusedFetchManifest: RevisionIo["fetchManifest"] = async () => { throw new Error("fetchManifest should not be called for this service"); }; @@ -1510,22 +1504,44 @@ describe("refreshCatalogPin backfills upstreamImage", () => { ); }); - test("resolves a mirrored service's upstreamImage from its oci-provenance source", async () => { + const mirroredIo = (provenanceLine: (provenance: string) => string | undefined): RevisionIo => { const digests = nativeDigests("ag"); - const io: RevisionIo = { + const provenance = JSON.stringify({ source: "docker.io/timberio/vector:0.53.0-alpine" }); + const line = provenanceLine(provenance); + return { listReleaseTags: async () => ["vector-0.53.0-r0"], - fetchChecksums: async () => checksumsFor("vector", "0.53.0-r0", digests), + fetchChecksums: async () => + [checksumsFor("vector", "0.53.0-r0", digests), line] + .filter((entry) => entry !== undefined) + .join("\n"), imageDigest: async () => digest("ah"), s3Sha256: matchingS3("vector", "0.53.0-r0", digests), fetchManifest: unusedFetchManifest, - fetchProvenance: provenanceWithSource("docker.io/timberio/vector:0.53.0-alpine"), + fetchProvenance: async () => provenance, }; + }; + + test("resolves a mirrored service's upstreamImage from its oci-provenance source", async () => { + const io = mirroredIo((provenance) => `${hex(provenance)} vector-0.53.0.oci-provenance.json`); const result = await refreshCatalogPin({ catalog: vectorFixture, service: "vector", io }); expect(result.source).toContain('upstreamImage: "timberio/vector:0.53.0-alpine"'); }); + test.each([ + ["has no SHA256SUMS line", () => undefined, /has no line for vector-0\.53\.0\.oci-provenance/], + [ + "does not match SHA256SUMS", + () => `${hex("other")} vector-0.53.0.oci-provenance.json`, + /does not match SHA256SUMS/, + ], + ])("rejects an oci-provenance file that %s", async (_name, line, message) => { + await expect( + refreshCatalogPin({ catalog: vectorFixture, service: "vector", io: mirroredIo(line) }), + ).rejects.toThrow(message); + }); + test("rejects a manifest upstream_image carrying a quote or template expression, writing nothing", async () => { const digests = nativeDigests("ak"); const io: RevisionIo = { diff --git a/.github/scripts/sync-artifacts-catalog.ts b/.github/scripts/sync-artifacts-catalog.ts index bce8748fb9..52f2fce9c5 100644 --- a/.github/scripts/sync-artifacts-catalog.ts +++ b/.github/scripts/sync-artifacts-catalog.ts @@ -3,11 +3,19 @@ * `docs/adr/0026-slim-artifact-mirrors.md` for the catalog model and this script's three modes. */ +import { createHash } from "node:crypto"; +import { + MIRROR_PRODUCERS, + NATIVE_TARGETS, + UPSTREAM_IMAGE_PATTERN, + normalizeUpstreamImage, + parseSha256Sums, + type SlimNativeTarget, +} from "../../packages/stack/src/preparation/SlimRelease.ts"; import { DIGEST_PATTERN, InvalidPayloadError, SOURCE_REGISTRY, - checksumFor, escapeRegExp, nativeFileNames, nativeObjectUrl, @@ -17,9 +25,7 @@ export const CATALOG_PATH = "packages/stack/src/Artifacts.ts"; const SLIM_IMAGE_PREFIX = `${SOURCE_REGISTRY}/`; -/** Native targets the catalog pins per revision, in `Artifacts.ts`'s serialization order. */ -const NATIVE_TARGETS = ["darwin-arm64", "linux-amd64", "linux-arm64"] as const; -type NativeTargetName = (typeof NATIVE_TARGETS)[number]; +type NativeTargetName = SlimNativeTarget; /** Overridable so an integration test can point at a local fixture server instead of GitHub. */ const RELEASES_API = @@ -308,11 +314,15 @@ export async function resolveRevisionPin( }; } + const parsedSums = parseSha256Sums(checksums); + if (parsedSums.kind === "malformed") { + throw new InvalidPayloadError(`${service}-${releaseVersion}: ${parsedSums.message}.`); + } const natives: Record = {}; for (const target of NATIVE_TARGETS) { const files = nativeFileNames(service, releaseVersion, target); - const archive = checksumFor(checksums, files.archive); - const manifest = checksumFor(checksums, files.manifest); + const archive = parsedSums.sums.get(files.archive); + const manifest = parsedSums.sums.get(files.manifest); if (archive === undefined || manifest === undefined) { return { status: "incomplete", @@ -384,14 +394,6 @@ export async function resolveRevisionPin( }; } -/** - * Services slim-services mirrors from an unmodified upstream image rather than building from - * source: their `upstreamImage` comes from the release's `oci-provenance.json` `source` field, - * not a native target's manifest. Keep this in sync with `SlimServicesSource`'s mirror-mode - * services. - */ -const MIRROR_MODE_SOURCE_SERVICES: ReadonlySet = new Set(["vector", "mailpit", "imgproxy"]); - function parseJsonRecord(raw: string): Record | undefined { let parsed: unknown; try { @@ -409,26 +411,9 @@ function stringField(record: Record | undefined, key: string): return typeof value === "string" && value.length > 0 ? value : undefined; } -/** Strips a `docker.io/` prefix and any `@sha256:…` digest, to match the Dockerfile's `FROM` form. */ -function normalizeUpstreamImage(image: string): string { - const withoutDigest = image.split("@")[0] ?? image; - return withoutDigest.startsWith("docker.io/") - ? withoutDigest.slice("docker.io/".length) - : withoutDigest; -} - -/** - * A normalized `upstreamImage`: one or more lowercase `registry`/`repository` path segments - * (each `[a-z0-9]`, optionally separated internally by `.`/`_`/`-`), a `:`, and a tag matching - * Docker's own tag grammar. Anchored, so no whitespace, quote, or template/expression syntax can - * slip through — this value gets embedded as a TypeScript string literal in `Artifacts.ts`. - */ -const IMAGE_REFERENCE_PATTERN = - /^[a-z0-9]+(?:[._-][a-z0-9]+)*(?:\/[a-z0-9]+(?:[._-][a-z0-9]+)*)*:[A-Za-z0-9_][A-Za-z0-9._-]{0,127}$/; - /** Validates a normalized `upstreamImage` before it's ever written to `Artifacts.ts`. */ function validateUpstreamImage(image: string, context: string): string { - if (!IMAGE_REFERENCE_PATTERN.test(image)) { + if (!UPSTREAM_IMAGE_PATTERN.test(image)) { throw new InvalidPayloadError( `${context} has an invalid upstreamImage ${JSON.stringify(image)}.`, ); @@ -450,11 +435,28 @@ async function resolveUpstreamImage( upstreamVersion: string, io: RevisionIo, ): Promise { - if (MIRROR_MODE_SOURCE_SERVICES.has(service)) { + if (MIRROR_PRODUCERS.has(service)) { const provenance = await io.fetchProvenance(service, releaseVersion, upstreamVersion); if (provenance === undefined) { throw new InvalidPayloadError(`${service}-${releaseVersion} has no oci-provenance asset.`); } + const name = `${service}-${upstreamVersion}.oci-provenance.json`; + const checksums = await io.fetchChecksums(service, releaseVersion); + const parsed = parseSha256Sums(checksums ?? ""); + if (parsed.kind === "malformed") { + throw new InvalidPayloadError(`${service}-${releaseVersion}: ${parsed.message}.`); + } + const expected = parsed.sums.get(name); + if (expected === undefined) { + throw new InvalidPayloadError( + `${service}-${releaseVersion} SHA256SUMS has no line for ${name}.`, + ); + } + if (createHash("sha256").update(provenance).digest("hex") !== expected) { + throw new InvalidPayloadError( + `${service}-${releaseVersion} ${name} does not match SHA256SUMS.`, + ); + } const source = stringField(parseJsonRecord(provenance), "source"); if (source === undefined) { throw new InvalidPayloadError( diff --git a/.oxlintrc.json b/.oxlintrc.json index cb9deecdd7..9df557b0bb 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -96,6 +96,7 @@ "packages/stack/src/effect.integration.test.ts", "packages/stack/src/port-reservation.e2e.test.ts", "packages/stack/src/host/CommandRunner.ts", + "packages/stack/src/internal/schema-writers.ts", "packages/stack/src/preparation/ArtifactStore.ts", "packages/stack/src/preparation/artifacts.integration.test.ts", "packages/stack/src/runtime/Container.ts", @@ -267,6 +268,8 @@ // - NativeProcess.ts, native-process.integration.test.ts: need Environment. // - postgres-user.ts: needs the no-follow lstatPath. // - DockerDatabaseStorage.ts: needs Publication. + // - schema-writers.ts: needs namespace Lock and Publication; the facade reaches it through + // Registry, Catalog and Database. "files": [ "packages/stack/src/Network.ts", "packages/stack/src/Network.integration.test.ts", @@ -276,6 +279,7 @@ "packages/stack/src/Owner.integration.test.ts", "packages/stack/src/effect.integration.test.ts", "packages/stack/src/port-reservation.e2e.test.ts", + "packages/stack/src/internal/schema-writers.ts", "packages/stack/src/preparation/ArtifactStore.ts", "packages/stack/src/preparation/artifacts.integration.test.ts", "packages/stack/src/runtime/Container.ts", diff --git a/apps/cli-go/cmd/db.go b/apps/cli-go/cmd/db.go index 044a639783..bf913d2aa3 100644 --- a/apps/cli-go/cmd/db.go +++ b/apps/cli-go/cmd/db.go @@ -128,6 +128,8 @@ func init() { remoteFlags.String("db-url", "", "Connect using the specified Postgres URL (must be percent-encoded).") remoteFlags.Bool("linked", true, "Connect to the linked project.") dbRemoteCmd.MarkFlagsMutuallyExclusive("db-url", "linked") + remoteFlags.StringVar(&flags.ProjectRef, "project-ref", "", "Project ref of the Supabase project.") + markFlagTelemetrySafe(remoteFlags.Lookup("project-ref")) remoteFlags.StringVarP(&dbPassword, "password", "p", "", "Password to your remote Postgres database.") cobra.CheckErr(viper.BindPFlag("DB_PASSWORD", remoteFlags.Lookup("password"))) dbRemoteCmd.AddCommand(dbRemoteChangesCmd) diff --git a/apps/cli-go/cmd/root.go b/apps/cli-go/cmd/root.go index 97aafb07fd..1c48ab558c 100644 --- a/apps/cli-go/cmd/root.go +++ b/apps/cli-go/cmd/root.go @@ -22,6 +22,7 @@ import ( "github.com/supabase/cli/internal/telemetry" "github.com/supabase/cli/internal/utils" "github.com/supabase/cli/internal/utils/flags" + "github.com/supabase/cli/pkg/config" "golang.org/x/mod/semver" ) @@ -207,7 +208,7 @@ func Execute() { // Checks auth before calling the API to avoid the log.Fatalln in GetSupabase(). func ensureProjectGroupsCached(ctx context.Context, service *telemetry.Service) { ref := flags.ProjectRef - if ref == "" { + if ref == "" || config.UsesStackState() { return } fsys := afero.NewOsFs() @@ -239,7 +240,7 @@ func updateNotifierEnabled() bool { // Read the env directly instead of viper: the upgrade check also runs for // --help and --version, which skip the cobra initializer that binds env vars. disabled, _ := strconv.ParseBool(os.Getenv("SUPABASE_NO_UPDATE_NOTIFIER")) - return !disabled + return !disabled && !config.UsesStackState() } func checkUpgrade(ctx context.Context, fsys afero.Fs) (string, error) { diff --git a/apps/cli-go/cmd/root_test.go b/apps/cli-go/cmd/root_test.go index ea82156213..8dc63afd48 100644 --- a/apps/cli-go/cmd/root_test.go +++ b/apps/cli-go/cmd/root_test.go @@ -9,6 +9,7 @@ import ( "github.com/stretchr/testify/require" "github.com/supabase/cli/internal/telemetry" "github.com/supabase/cli/internal/utils" + "github.com/supabase/cli/pkg/config" ) func clearTelemetryEnv(t *testing.T) { @@ -135,3 +136,9 @@ func TestUpdateNotifierEnabled(t *testing.T) { assert.Equal(t, wantEnabled, updateNotifierEnabled(), "value %q", value) } } + +func TestUpdateNotifierDisabledInStackMode(t *testing.T) { + t.Setenv("SUPABASE_NO_UPDATE_NOTIFIER", "") + t.Setenv(config.StackStateEnv, "1") + assert.False(t, updateNotifierEnabled()) +} diff --git a/apps/cli-go/internal/telemetry/project.go b/apps/cli-go/internal/telemetry/project.go index a8ceafd1a7..8fe77a2096 100644 --- a/apps/cli-go/internal/telemetry/project.go +++ b/apps/cli-go/internal/telemetry/project.go @@ -10,6 +10,7 @@ import ( "github.com/spf13/afero" "github.com/supabase/cli/internal/utils" "github.com/supabase/cli/pkg/api" + "github.com/supabase/cli/pkg/config" ) type LinkedProject struct { @@ -19,6 +20,10 @@ type LinkedProject struct { OrganizationSlug string `json:"organization_slug"` } +// linkedProjectEnv carries the linked project as the JSON of LinkedProject when the TypeScript +// CLI keeps the link outside supabase/.temp. +const linkedProjectEnv = "SUPABASE_INTERNAL_LINKED_PROJECT" + func linkedProjectPath() string { return filepath.Join(utils.TempDir, "linked-project.json") } @@ -38,10 +43,21 @@ func SaveLinkedProject(project api.V1ProjectWithDatabaseResponseOutput, fsys afe } func LoadLinkedProject(fsys afero.Fs) (LinkedProject, error) { + if config.UsesStackState() { + contents := os.Getenv(linkedProjectEnv) + if contents == "" { + return LinkedProject{}, os.ErrNotExist + } + return parseLinkedProject([]byte(contents)) + } contents, err := afero.ReadFile(fsys, linkedProjectPath()) if err != nil { return LinkedProject{}, err } + return parseLinkedProject(contents) +} + +func parseLinkedProject(contents []byte) (LinkedProject, error) { var linked LinkedProject if err := json.Unmarshal(contents, &linked); err != nil { return LinkedProject{}, errors.Errorf("failed to parse linked project: %w", err) diff --git a/apps/cli-go/internal/telemetry/project_test.go b/apps/cli-go/internal/telemetry/project_test.go index 206855518a..2cf1fa89ce 100644 --- a/apps/cli-go/internal/telemetry/project_test.go +++ b/apps/cli-go/internal/telemetry/project_test.go @@ -1,6 +1,7 @@ package telemetry import ( + "os" "testing" "time" @@ -8,6 +9,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "github.com/supabase/cli/pkg/api" + "github.com/supabase/cli/pkg/config" ) var testProject = api.V1ProjectWithDatabaseResponseOutput{ @@ -123,4 +125,27 @@ func TestLinkedProjectGroups(t *testing.T) { GroupProject: "proj_abc", }, groups) }) + + t.Run("reads the linked project from the env in stack mode", func(t *testing.T) { + t.Setenv(config.StackStateEnv, "1") + t.Setenv(linkedProjectEnv, `{"ref":"proj_abc","name":"n","organization_id":"org_123","organization_slug":"acme"}`) + fsys := afero.NewMemMapFs() + // The legacy cache must stay unread. + require.NoError(t, SaveLinkedProject(api.V1ProjectWithDatabaseResponseOutput{Ref: "stale", OrganizationId: "stale"}, fsys)) + assert.Equal(t, map[string]string{ + GroupOrganization: "org_123", + GroupProject: "proj_abc", + }, linkedProjectGroups(fsys)) + assert.True(t, HasLinkedProject(fsys)) + }) + + t.Run("reads as unlinked in stack mode without the env", func(t *testing.T) { + t.Setenv(config.StackStateEnv, "1") + t.Setenv(linkedProjectEnv, "") + fsys := afero.NewMemMapFs() + require.NoError(t, SaveLinkedProject(testProject, fsys)) + _, err := LoadLinkedProject(fsys) + assert.ErrorIs(t, err, os.ErrNotExist) + assert.Nil(t, linkedProjectGroups(fsys)) + }) } diff --git a/apps/cli-go/internal/utils/flags/project_ref.go b/apps/cli-go/internal/utils/flags/project_ref.go index fab2920014..74d7c9a08e 100644 --- a/apps/cli-go/internal/utils/flags/project_ref.go +++ b/apps/cli-go/internal/utils/flags/project_ref.go @@ -11,6 +11,7 @@ import ( "github.com/spf13/afero" "github.com/spf13/viper" "github.com/supabase/cli/internal/utils" + "github.com/supabase/cli/pkg/config" "golang.org/x/term" ) @@ -63,6 +64,9 @@ func LoadProjectRef(fsys afero.Fs) error { fmt.Fprintf(debuglogger, "Loading project ref from env var: %s\n", ProjectRef) return utils.AssertProjectRefIsValid(ProjectRef) } + if config.UsesStackState() { + return errors.New(utils.ErrNotLinked) + } // Load from local file last projectRefBytes, err := afero.ReadFile(fsys, utils.ProjectRefPath) fmt.Fprintf(debuglogger, "Loading project ref from file: %s\n", utils.ProjectRefPath) diff --git a/apps/cli-go/internal/utils/flags/project_ref_test.go b/apps/cli-go/internal/utils/flags/project_ref_test.go index 27627a1b08..b497a7b211 100644 --- a/apps/cli-go/internal/utils/flags/project_ref_test.go +++ b/apps/cli-go/internal/utils/flags/project_ref_test.go @@ -17,6 +17,7 @@ import ( "github.com/supabase/cli/internal/testing/fstest" "github.com/supabase/cli/internal/utils" "github.com/supabase/cli/pkg/api" + "github.com/supabase/cli/pkg/config" ) func TestProjectRef(t *testing.T) { @@ -44,6 +45,19 @@ func TestProjectRef(t *testing.T) { assert.NoError(t, err) }) + t.Run("ignores the linked file in stack mode", func(t *testing.T) { + t.Setenv(config.StackStateEnv, "1") + ProjectRef = "" + fsys := afero.NewMemMapFs() + decoy := apitest.RandomProjectRef() + require.NoError(t, afero.WriteFile(fsys, utils.ProjectRefPath, []byte(decoy), 0644)) + // Run test + err := LoadProjectRef(fsys) + // Check error + assert.ErrorIs(t, err, utils.ErrNotLinked) + assert.Empty(t, ProjectRef) + }) + t.Run("throws error on read failure", func(t *testing.T) { ProjectRef = "" // Setup in-memory fs diff --git a/apps/cli-go/pkg/config/config.go b/apps/cli-go/pkg/config/config.go index 5c2eca797e..d9a36221a4 100644 --- a/apps/cli-go/pkg/config/config.go +++ b/apps/cli-go/pkg/config/config.go @@ -245,6 +245,8 @@ type ( // The json tag is the one that matters: the decoder runs with // dc.TagName = "json". Omitted from toml so Go never emits the section. Compute map[string]any `toml:"-" json:"compute"` + // Local is parsed but never read here; [local] is owned by the TS CLI (see Compute). + Local map[string]any `toml:"-" json:"local"` } config struct { @@ -858,7 +860,7 @@ func (c *config) Load(path string, fsys fs.FS, overrides ...ConfigEditor) error return err } // TODO: move linked pooler connection string elsewhere - if connString, err := fs.ReadFile(fsys, builder.PoolerUrlPath); err == nil && len(connString) > 0 { + if connString, err := readLinkedPoolerUrl(fsys, builder.PoolerUrlPath); err == nil && len(connString) > 0 { c.Db.Pooler.ConnectionString = string(connString) } if len(c.Api.ExternalUrl) == 0 { @@ -890,41 +892,41 @@ func (c *config) Load(path string, fsys fs.FS, overrides ...ConfigEditor) error c.Db.Image = Images.Pg15 } if c.Db.MajorVersion > 14 { - if version, err := fs.ReadFile(fsys, builder.PostgresVersionPath); err == nil { + if version, err := readTempFile(fsys, builder.PostgresVersionPath); err == nil { // Only replace image if postgres version is above 15.1.0.55 if i := strings.IndexByte(c.Db.Image, ':'); VersionCompare(c.Db.Image[i+1:], "15.1.0.55") >= 0 { c.Db.Image = replaceImageTag(Images.Pg, string(version)) } } - if version, err := fs.ReadFile(fsys, builder.RestVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.RestVersionPath); err == nil && len(version) > 0 { c.Api.Image = replaceImageTag(Images.Postgrest, string(version)) } - if version, err := fs.ReadFile(fsys, builder.GotrueVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.GotrueVersionPath); err == nil && len(version) > 0 { c.Auth.Image = replaceImageTag(Images.Gotrue, string(version)) } } - if version, err := fs.ReadFile(fsys, builder.StorageVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.StorageVersionPath); err == nil && len(version) > 0 { c.Storage.Image = replaceImageTag(Images.Storage, string(version)) } - if version, err := fs.ReadFile(fsys, builder.StorageMigrationPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.StorageMigrationPath); err == nil && len(version) > 0 { c.Storage.TargetMigration = strings.TrimSpace(string(version)) } - if version, err := fs.ReadFile(fsys, builder.EdgeRuntimeVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.EdgeRuntimeVersionPath); err == nil && len(version) > 0 { c.EdgeRuntime.Image = replaceImageTag(Images.EdgeRuntime, string(version)) } - if version, err := fs.ReadFile(fsys, builder.PoolerVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.PoolerVersionPath); err == nil && len(version) > 0 { c.Db.Pooler.Image = replaceImageTag(Images.Supavisor, string(version)) } - if version, err := fs.ReadFile(fsys, builder.RealtimeVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.RealtimeVersionPath); err == nil && len(version) > 0 { c.Realtime.Image = replaceImageTag(Images.Realtime, string(version)) } - if version, err := fs.ReadFile(fsys, builder.StudioVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.StudioVersionPath); err == nil && len(version) > 0 { c.Studio.Image = replaceImageTag(Images.Studio, string(version)) } - if version, err := fs.ReadFile(fsys, builder.PgmetaVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.PgmetaVersionPath); err == nil && len(version) > 0 { c.Studio.PgmetaImage = replaceImageTag(Images.Pgmeta, string(version)) } - if version, err := fs.ReadFile(fsys, builder.LogflareVersionPath); err == nil && len(version) > 0 { + if version, err := readTempFile(fsys, builder.LogflareVersionPath); err == nil && len(version) > 0 { c.Analytics.Image = replaceImageTag(Images.Logflare, string(version)) } // TODO: replace derived config resolution with viper decode hooks diff --git a/apps/cli-go/pkg/config/config_test.go b/apps/cli-go/pkg/config/config_test.go index 0f51bf5153..6ddd051a53 100644 --- a/apps/cli-go/pkg/config/config_test.go +++ b/apps/cli-go/pkg/config/config_test.go @@ -311,6 +311,21 @@ instances = 3 assert.NoError(t, config.Load("", fsys)) }) + t.Run("accepts the TS-owned local section", func(t *testing.T) { + config := NewConfig() + fsys := fs.MapFS{ + "supabase/config.toml": &fs.MapFile{Data: []byte(` +project_id = "test" + +[local.services] +default = { version = "bundled" } +auth = { version = "v2.197.0" } +`)}, + } + + assert.NoError(t, config.Load("", fsys)) + }) + for _, tt := range []struct { name string configData string @@ -1263,3 +1278,50 @@ orioledb_version = "15.1.0.150" assert.Contains(t, stderr, "WARN: remotes.staging.experimental.orioledb_version is deprecated. Please use remotes.staging.db.orioledb_version instead.") }) } + +func TestLoadLinkedStateFiles(t *testing.T) { + linkedFiles := fs.MapFS{ + "supabase/.temp/pooler-url": &fs.MapFile{Data: []byte("postgresql://postgres.decoy:[YOUR-PASSWORD]@pooler.example.com:6543/postgres")}, + "supabase/.temp/rest-version": &fs.MapFile{Data: []byte("v0.0.1-decoy")}, + "supabase/.temp/gotrue-version": &fs.MapFile{Data: []byte("v0.0.2-decoy")}, + "supabase/.temp/storage-version": &fs.MapFile{Data: []byte("v0.0.3-decoy")}, + "supabase/.temp/storage-migration": &fs.MapFile{Data: []byte("decoy-migration")}, + "supabase/.temp/edge-runtime-version": &fs.MapFile{Data: []byte("v0.0.4-decoy")}, + } + + t.Run("legacy mode applies the pooler url and image pins", func(t *testing.T) { + t.Setenv(StackStateEnv, "") + config := NewConfig() + require.NoError(t, config.Load("", linkedFiles)) + assert.Contains(t, config.Db.Pooler.ConnectionString, "pooler.example.com") + assert.True(t, strings.HasSuffix(config.Api.Image, ":v0.0.1-decoy"), config.Api.Image) + assert.True(t, strings.HasSuffix(config.Auth.Image, ":v0.0.2-decoy"), config.Auth.Image) + assert.True(t, strings.HasSuffix(config.Storage.Image, ":v0.0.3-decoy"), config.Storage.Image) + assert.Equal(t, "decoy-migration", config.Storage.TargetMigration) + assert.True(t, strings.HasSuffix(config.EdgeRuntime.Image, ":v0.0.4-decoy"), config.EdgeRuntime.Image) + }) + + t.Run("stack mode ignores the pooler url file and image pins", func(t *testing.T) { + t.Setenv(StackStateEnv, "1") + t.Setenv(StackPoolerUrlEnv, "") + want := NewConfig() + require.NoError(t, want.Load("", fs.MapFS{})) + config := NewConfig() + require.NoError(t, config.Load("", linkedFiles)) + assert.Equal(t, want.Db.Pooler.ConnectionString, config.Db.Pooler.ConnectionString) + assert.Equal(t, want.Api.Image, config.Api.Image) + assert.Equal(t, want.Auth.Image, config.Auth.Image) + assert.Equal(t, want.Storage.Image, config.Storage.Image) + assert.Equal(t, want.Storage.TargetMigration, config.Storage.TargetMigration) + assert.Equal(t, want.EdgeRuntime.Image, config.EdgeRuntime.Image) + }) + + t.Run("stack mode applies the pooler url from the environment", func(t *testing.T) { + const poolerUrl = "postgresql://postgres.abcdefghijklmnopqrst:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.com:6543/postgres" + t.Setenv(StackStateEnv, "1") + t.Setenv(StackPoolerUrlEnv, poolerUrl) + config := NewConfig() + require.NoError(t, config.Load("", linkedFiles)) + assert.Equal(t, poolerUrl, config.Db.Pooler.ConnectionString) + }) +} diff --git a/apps/cli-go/pkg/config/utils.go b/apps/cli-go/pkg/config/utils.go index 2bb6db6f4b..bd2b9512b6 100644 --- a/apps/cli-go/pkg/config/utils.go +++ b/apps/cli-go/pkg/config/utils.go @@ -5,10 +5,44 @@ import ( "crypto/sha256" "encoding/hex" "fmt" + "io/fs" + "os" "path/filepath" + "strconv" "strings" ) +// StackStateEnv is set by the TypeScript CLI when the project keeps its linked state outside +// supabase/.temp, so the files under it are neither read nor written. +const StackStateEnv = "SUPABASE_INTERNAL_STACK_STATE" + +// StackPoolerUrlEnv carries the linked pooler connection string in stack mode, where +// supabase/.temp/pooler-url is not read. +const StackPoolerUrlEnv = "SUPABASE_INTERNAL_POOLER_URL" + +// UsesStackState reports whether supabase/.temp is off limits for this invocation. +func UsesStackState() bool { + enabled, _ := strconv.ParseBool(os.Getenv(StackStateEnv)) + return enabled +} + +// readTempFile reads a file under supabase/.temp, which reads as absent in stack mode. +func readTempFile(fsys fs.FS, path string) ([]byte, error) { + if UsesStackState() { + return nil, fs.ErrNotExist + } + return fs.ReadFile(fsys, path) +} + +// readLinkedPoolerUrl returns the linked pooler connection string from supabase/.temp, or from the +// environment in stack mode. +func readLinkedPoolerUrl(fsys fs.FS, path string) ([]byte, error) { + if UsesStackState() { + return []byte(os.Getenv(StackPoolerUrlEnv)), nil + } + return fs.ReadFile(fsys, path) +} + type pathBuilder struct { SupabaseDirPath string ConfigPath string diff --git a/apps/cli/docs/go-cli-divergences.md b/apps/cli/docs/go-cli-divergences.md index 1abb22e6bb..54ef768b90 100644 --- a/apps/cli/docs/go-cli-divergences.md +++ b/apps/cli/docs/go-cli-divergences.md @@ -34,14 +34,16 @@ These commands exist in the TS CLI today but have no direct top-level equivalent - Every command that resolves a linked project ref for its own database connection has a TS-only `--project-ref` flag (no Go equivalent on any user-facing command — only the `SUPABASE_PROJECT_ID` env var could override - the linked ref; the sole Go registration is a hidden, non-user-facing seam, - `db declarative __catalog --project-ref`, `cmd/pgdelta_catalog.go:44`). This + the linked ref; the Go registrations are a hidden, non-user-facing seam, + `db declarative __catalog --project-ref`, `cmd/pgdelta_catalog.go:44`, and the + deprecated Go-delegated `db remote changes`, which receives the linked ref this + way in stack mode). This covers: `db push`, `db pull`, `db diff`, `db dump`, `db reset`, `db lint`, `db advisors`, `db query`; `migration list`/`up`/`down`/`repair`/`fetch`/`squash`; `seed buckets`; `storage ls`/`cp`/`mv`/`rm`; every `inspect db` subcommand and `inspect report`; and `test db` (and its hidden `db test` alias, which shares `test db`'s flag config verbatim). It feeds - `ProjectRefResolver.loadProjectRef`, keeping Go's precedence (flag > + `RemoteTarget.load`, keeping Go's precedence (flag > `SUPABASE_PROJECT_ID` > `supabase/.temp/project-ref`) and taking effect only on the linked path. It shares ONLY that ref-resolution precedence with `SUPABASE_PROJECT_ID` — unlike the env var, it does not affect the local @@ -51,6 +53,11 @@ These commands exist in the TS CLI today but have no direct top-level equivalent flag, since Go's env var going unused on a non-linked target has no TS-only flag counterpart to accidentally discard. Default behavior (omitted flag) matches Go exactly. +- A hosted ref from `SUPABASE_PROJECT_ID` that differs from the linked ref gets + its pooler URL from the Management API, like a `--project-ref` flag does. Go + reuses the saved `supabase/.temp/pooler-url` whichever ref the env var names. + The saved pooler URL, parent project, and branch name apply only when the + resolved ref is the linked one. - `projects api-keys` has a TS-only `--reveal` flag (no Go equivalent). It sends `reveal=true` so the Management API returns the full secret keys (`sb_secret_...`) in full instead of redacting them, addressing issue #4775. Default behavior (omitted flag) @@ -84,6 +91,8 @@ These commands exist in the TS CLI today but have no direct top-level equivalent a timeout, or any transport/status/decode failure (fail-safe: an unverifiable divergent cache is untrustworthy). Both are best-effort and never affect `link`'s own outcome — see `link/SIDE_EFFECTS.md`. +- `link` has a TS-only `--frozen` flag (no Go equivalent) that leaves `supabase/supabase.lock` + untouched in stack mode; it is a no-op in legacy mode, which has no lock. ## Behavioral divergences from the Go reference diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index 60d7ab2215..5e37434a43 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -45,6 +45,13 @@ All delegation goes through the shared `GoProxy` service - **Child telemetry suppression**: a proxy handler that itself owns the parent `cli_command_executed` event (wrapped in TS command instrumentation) suppresses the child's own telemetry; a bare pass-through proxy leaves it enabled so the Go child stays the sole emitter. +- **Stack-mode state**: when the project keeps its state outside `supabase/.temp`, `GoProxy` sets + `SUPABASE_INTERNAL_STACK_STATE=1` so the child skips `supabase/.temp`, and passes the linked + project from `supabase/.link.json` as the JSON of `linked-project.json` in + `SUPABASE_INTERNAL_LINKED_PROJECT` so the child keeps its telemetry groups. A command that + passes `--project-ref` also passes the resolved target's `pooler_url` in + `SUPABASE_INTERNAL_POOLER_URL` when that ref is the linked one; the child reads it in place of + `supabase/.temp/pooler-url` for its IPv4 pooler fallback. The spawn surface is guarded by [`apps/cli-e2e/src/tests/go-binary-surface.e2e.test.ts`](../../cli-e2e/src/tests/go-binary-surface.e2e.test.ts) — diff --git a/apps/cli/docs/stack-commands.md b/apps/cli/docs/stack-commands.md index ba2c5cc14f..46fa0e8a17 100644 --- a/apps/cli/docs/stack-commands.md +++ b/apps/cli/docs/stack-commands.md @@ -14,16 +14,18 @@ Native pg_cron runs jobs in background workers, so a job body cannot contain its default `max_worker_processes` to 17 to leave room for those workers; `postgresql.conf`, `ALTER SYSTEM`, and `[db.settings]` values take precedence. -| Command | Purpose | -| ------------------------ | --------------------------------------------------------------------------------- | -| `supabase stack destroy` | Permanently delete stacks and their data. | -| `supabase stack list` | List persisted managed local stacks. | -| `supabase stack prepare` | Download artifacts without starting services. | -| `supabase stack start` | Create or resume the project's stack. | -| `supabase stack status` | Show identity, readiness, and drift, or export connection variables with `--env`. | -| `supabase stack logs` | Print retained stack logs; `-f` streams new lines. | -| `supabase stack restart` | Restart an existing stack using its saved effective configuration. | -| `supabase stack stop` | Stop a stack while retaining its data. | +| Command | Purpose | +| ------------------------- | --------------------------------------------------------------------------------- | +| `supabase stack destroy` | Permanently delete stacks and their data. | +| `supabase stack list` | List persisted managed local stacks. | +| `supabase stack install` | Resolve and lock service versions, then download artifacts without a stack. | +| `supabase stack update` | Re-resolve locked service versions and rewrite `supabase/supabase.lock`. | +| `supabase stack outdated` | Report which locked service versions `stack update` would change. | +| `supabase stack start` | Create or resume the project's stack. | +| `supabase stack status` | Show identity, readiness, and drift, or export connection variables with `--env`. | +| `supabase stack logs` | Print retained stack logs; `-f` streams new lines. | +| `supabase stack restart` | Restart an existing stack using its saved effective configuration. | +| `supabase stack stop` | Stop a stack while retaining its data. | Use each command's `--help` for its available targeting and runtime options. @@ -37,15 +39,30 @@ stack, cannot reach its engine, the failure asks you to install or start it, and on platforms that support native. Project stacks created by database commands, and shadow stacks created without a project stack, use the same selection. -`supabase stack prepare` downloads or pulls artifacts for the selected stack without starting -services. If the target does not exist, prepare creates and registers it; the stack then appears in -`supabase stack list` and can be removed with `supabase stack destroy`. Omit `--capability` to -prepare every enabled capability, or repeat `--capability` up to ten times to select specific -capabilities. Each occurrence names one capability; use separate flags rather than CSV. +`supabase stack install` resolves the project's service versions into `supabase/supabase.lock`, the +same way `stack start` does, then downloads everything the lock pins for the chosen runtime without +creating or starting a stack. `--runtime native` downloads archives into the artifact cache; `docker` +and `podman` pull images; `auto` selects as `stack start` does. With `--frozen`, or with `CI` set +and an existing lock, a missing or out-of-date lock fails instead of being written. Artifacts that +are already present are skipped, so a second run reports everything as cached. ```sh -supabase stack prepare -supabase stack prepare --capability rest --capability auth --output-format json +supabase stack install +supabase stack install --frozen --runtime native --output-format json +``` + +`supabase stack update [service...]` re-resolves the named services, or all of them, even when the +lock is fresh: an upstream version moves to its newest revision, `bundled` moves to the version this +CLI bundles, `linked` probes the linked project or branch again (this needs a login, like `supabase +link`), and an exact `-r` pin stays. It prints the diff and rewrites the lock without +downloading anything; `CI` does not freeze it. `supabase stack outdated` is the read-only +counterpart: it reports each service's locked and would-be version without a lease, a write, or a +download, and reports a failed probe per row without failing. It reads only the lock and the +release host, so a hosted version that moved since `supabase link` appears after `stack update`. + +```sh +supabase stack outdated +supabase stack update auth realtime ``` ## Connection details after start @@ -99,10 +116,10 @@ for example `api: 54321 → 26199`, and JSON/stream-json output add an `endpoint entry naming the endpoint, the `endpoints` keys it moved in `keys`, and its `from`/`to` port) to the success payload, present only when a change applied. A requested fixed port that is already taken fails the start with the usual port conflict; the new port stays saved and the previous automatic -port is released, so starting again succeeds once the port is free. A changed artifact version or -PostgreSQL major version still fails, naming the changed setting and suggesting -`supabase stack destroy`. A changed port met by a stack whose owner is still running (for example -after `stack prepare`) fails with advice to run `supabase stack stop`, then `supabase stack start`. +port is released, so starting again succeeds once the port is free. A PostgreSQL build, line or major that differs +from the one that initialized the data still fails, suggesting `supabase stack destroy`; only a new +revision of the same build reuses it. A changed port met by a stack whose owner is still +running fails with advice to run `supabase stack stop`, then `supabase stack start`. ## Exporting environment variables @@ -173,18 +190,22 @@ unavailable. An invalid `SUPABASE_EXPERIMENTAL_STACK` value is still an error. For temporary selection, set `SUPABASE_EXPERIMENTAL_STACK=1` to select the new backend or `SUPABASE_EXPERIMENTAL_STACK=0` to select the legacy backend. This environment variable takes precedence over `experimental.stack`; an unset or empty value falls back to the file setting. -Other values are rejected. The override is applied before reading the project configuration. +Other values are rejected, including by `supabase link`, `supabase unlink` and `supabase projects +delete`, which read or write the project link. The override is applied before reading the project +configuration. `supabase services` follows the same backend selection. In stack mode it lists image versions and -canonical `ghcr.io/supabase/cli/...` names from the installed CLI's artifact catalog, including -Mailpit. PostgreSQL uses the selected major version (15 or 17), or the catalog's OrioleDB build -when `db.orioledb_version` is set; invalid configuration, an unsupported PostgreSQL major, or an -OrioleDB version the catalog does not pin warns with the cause and uses catalog defaults. This inventory describes the -CLI catalog, not running containers, downloaded images, or service health. The Docker and native -stack runtimes use the same catalog versions, though a running stack launched by another CLI -version or using mirrored images may differ. Legacy version pins and slim-image overrides do not -apply in stack mode. Linked versions remain best-effort; catalog mismatches cannot be changed with -`supabase link`. +canonical `ghcr.io/supabase/cli/...` names, including Mailpit. The versions come from +`supabase/supabase.lock` when the project has a fresh one, else from the installed CLI's artifact +catalog. PostgreSQL uses the selected major version (15 or 17), or the catalog's OrioleDB build when +`db.orioledb_version` is set; invalid configuration, an unsupported PostgreSQL major, an OrioleDB +version the catalog does not pin, or a stale or unreadable lock warns with the cause and uses +catalog defaults. This inventory describes the lock or catalog, not running containers, downloaded +images, or service health. The Docker and native stack runtimes use the same versions, though a +running stack launched by another CLI version or using mirrored images may differ. Legacy version +pins and slim-image overrides do not apply in stack mode. Linked versions remain best-effort; +a mismatch warning recommends `supabase link`, which updates services set to `linked` (the +default) in the lock, while services pinned in `[local.services]` keep their version. `SUPABASE_EXPERIMENTAL_STACK=1 supabase init` writes `[experimental] stack = true` into the new project config and omits the Docker-era default ports so the stack is not pinned to them. @@ -287,6 +308,59 @@ The flag is local CLI configuration in `supabase/config.toml` or `supabase/confi excluded from hosted project configuration. Routing applies the CLI's working-directory rules, including `--workdir` and `SUPABASE_WORKDIR`, and prefers JSON when both files exist. +Stack mode keeps its state under `SUPABASE_HOME` and never reads or writes `supabase/.temp`: Storage +uploads live in `stacks//uploads`, command scratch output (pgdelta debug bundles, Functions +bundling) in `tmp`, the release-check cache in `cli-latest`, and service versions come from +`supabase/supabase.lock`. See [Supabase home](supabase-home.md). + +## Service versions + +A stack runs the service versions pinned in `supabase/supabase.lock`, which `supabase start` writes +and the project commits so a team and CI run the same builds. Request versions under +`[local.services]`, per service or for all of them with `default`: + +```toml +[local.services] +default = { version = "bundled" } # `linked` (the default) or `bundled` + +[local.services.auth] +version = "v2.197.0" # an upstream version: the highest published revision +[local.services.realtime] +version = "v1.30.2-r1" # exactly revision 1 of that upstream version +``` + +`linked` follows the hosted project: `supabase link` (in stack mode) probes the linked project or +branch and records the version each service runs in the lock, then resolves `linked` entries to +those releases. A project that was never linked pins the bundled version and records it as +`unlinked`; a service the platform does not report, or a version slim-services does not publish, +pins the bundled version as `unobserved` or `unpublished`. `supabase link --frozen`, or `link` under +`CI` with an existing lock, records the link without changing the lock and warns when the hosted +versions differ. `bundled` pins the bundled version explicitly. An upstream version, optionally +with a `-r` revision suffix, is resolved against the published slim-services releases and pinned +by content. `[db] orioledb_version` is shorthand for +the `database` specifier of that OrioleDB build, and a database version must match +`db.major_version`. + +`supabase start` resolves only the services whose specifier changed, under a project lease that +serializes concurrent starts (a start waits up to 60 seconds for the lease, then fails), and leaves +the other pins where they are. Resolving a release specifier needs the network, so a first start +with an explicit service version or `[db] orioledb_version` cannot run offline, even when the CLI +bundles that version; a first start that keeps `linked` or `bundled` does not need it. `supabase +start --frozen` fails instead of resolving when the lock is missing or stale; it is implied when +`CI` is set and a lock exists. A start never calls the Management API and needs no login: it warns when `supabase/.link.json` +targets another project or branch than the lock observed (run `supabase stack update`), and when +this CLI bundles a newer revision of a locked version. A lock that is malformed, does not match the +format, or comes from a newer CLI always fails the start. The lock's JSON Schema is published at +`https://supabase.com/docs/cli/supabase-lock/v1.schema.json`. A database with initialized data keeps +its build: any change other than a new `-rN` revision of the same build (a different upstream build, +release line or major) is refused. The refusal offers to keep the data by pinning the recorded build with +`[local.services.database] version`, setting `[db] major_version` back when the major differs and removing any +`[db] orioledb_version` when an OrioleDB build is requested, or to run `supabase stack destroy` and recreate it. `supabase link` +and `supabase stack update` warn when the versions they lock would be refused by an existing stack's +data. A database that never initialized adopts the change. See the [`start` side +effects](../src/commands/experimental/stack/start/SIDE_EFFECTS.md#service-versions-and-the-lock) for +details. + ## Port intents Host listener assignment for `supabase stack` is documented in [Port intents](./supabase-home.md#port-intents). @@ -328,8 +402,8 @@ directory itself fails discovery, before any stop is attempted. Individual stop failures make the command fail and identify the affected stack IDs with their error details; no success or not-running summary is emitted when a stop fails. -`supabase stack destroy --stack feature-a` permanently removes exactly that stack and its data after -confirmation. Use `--yes` for unattended execution. Repeat `--stack-id` to destroy several stacks +`supabase stack destroy --stack feature-a` permanently removes exactly that stack, its data and its Storage +uploads (`stacks//uploads`) after confirmation. Use `--yes` for unattended execution. Repeat `--stack-id` to destroy several stacks after one confirmation; every id must resolve before anything is destroyed, and a stack that fails to be destroyed does not stop the rest, though the command still fails. @@ -383,8 +457,7 @@ way. A manually stopped capability keeps wake-up disabled until the stack starts states are observed directly by Storage operations, including `stack start`. **`stack start` seeds on first configured start.** When a `stack start` (or the top-level `start` -under the flag) invocation runs the stack's first configured start, including after `stack prepare` -— never when it resumes an existing one — and +under the flag) invocation runs the stack's first configured start (never when it resumes an existing one) and Storage is not `disabled`, it seeds `[storage.buckets]` against the stack before printing status, reusing the `seed buckets` core. A project with no `[storage.buckets]` or `[storage.vector.buckets]` configured resolves no credentials and prints nothing. Auto-confirm is diff --git a/apps/cli/docs/supabase-home.md b/apps/cli/docs/supabase-home.md index 35bcd0272d..6d4a6b9e46 100644 --- a/apps/cli/docs/supabase-home.md +++ b/apps/cli/docs/supabase-home.md @@ -60,6 +60,8 @@ Managed stack documents and runtime artifacts are shared through the global CLI state.json data/ / + uploads/ + tmp/ artifacts/ ``` @@ -68,7 +70,9 @@ context, and stack name that make up stack identity, together with sticky port i assignments, the runtime selection, the service definitions, and the composition needed to resume the stack. Runtime-only observations and active owner state are kept in memory. Service data lives under the stack's `data//` directories, while downloaded native artifacts are shared -under `artifacts/`. +under `artifacts/`. Storage objects live in the stack's `uploads/` directory, which `stack destroy` removes, and scratch output of +database and Functions commands (pgdelta debug bundles, Functions bundling) lives in `tmp/`. +Stack mode reads and writes nothing under `supabase/.temp`. There is no project-local `stacks/` directory or second state format. @@ -121,12 +125,17 @@ Auth, telemetry, traces, and downloaded binaries remain machine-global under `SU Legacy installer artifacts may coexist under that root, but they are not part of managed stack coordination. +In stack mode the post-run linked-project fill writes `/project-cache/.json` +atomically after the first successful `GET /v1/projects/{ref}`. It is only a marker that the +project's analytics groups were identified once per ref on this machine; nothing else reads it. +Legacy mode keeps `supabase/.temp/linked-project.json`, which `RemoteTarget.parent` also reads. + ## Release check cache After a successful command, the upgrade notice compares the running version with the latest GitHub release. The release tag is cached for ten hours in `supabase/.temp/cli-latest` when the -project directory has a real `supabase/` directory, and in `/cli-latest` -otherwise. `--version` and `--help` check only the current directory, so from a project +project directory has a real `supabase/` directory and uses legacy project state, and in +`/cli-latest` otherwise, including every stack-mode project. `--version` and `--help` check only the current directory, so from a project subdirectory they use `/cli-latest`. A failed fetch writes an empty file, which suppresses the notice and further fetches until it expires. The cache is skipped when the file, `supabase/.temp`, or `SUPABASE_HOME` is a symlink. `SUPABASE_NO_UPDATE_NOTIFIER=1` disables the check. diff --git a/apps/cli/scripts/generate-docs.ts b/apps/cli/scripts/generate-docs.ts index 74e70b3cb1..e359d5ca00 100644 --- a/apps/cli/scripts/generate-docs.ts +++ b/apps/cli/scripts/generate-docs.ts @@ -2,6 +2,8 @@ import { copyFileSync, mkdirSync, writeFileSync } from "node:fs"; import path from "node:path"; import process from "node:process"; import { CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL } from "@supabase/config"; +import { Lock, LOCK_SCHEMA_URL } from "@supabase/stack/effect"; +import { Schema } from "effect"; import { rootCommand } from "../src/cli/root.ts"; import { collectCommands, getHelpDoc } from "../src/shared/cli/command-docs.ts"; import { formatHelpDocAsMarkdown } from "../src/shared/cli/markdown-formatter.ts"; @@ -90,6 +92,27 @@ function copyConfigSchemaAsset(schemaUrl: string, distFileName: string) { console.log(`Generated: ${path.relative(path.resolve(import.meta.dir, "../../.."), destPath)}`); } +/** Writes the `supabase.lock` v1 JSON Schema, generated from the stack's lock schema, to its docs-site public path. */ +function writeLockSchemaAsset() { + const { schema, definitions } = Schema.toJsonSchemaDocument(Lock); + const document = { + $schema: "https://json-schema.org/draft/2020-12/schema", + $id: LOCK_SCHEMA_URL, + title: "supabase.lock", + description: "The service versions that supabase start pins in supabase/supabase.lock.", + ...schema, + $defs: definitions, + }; + const schemaPathname = new URL(LOCK_SCHEMA_URL).pathname.replace(/^\/docs/, ""); + const destPath = path.join(defaultDocsPublicDir, schemaPathname); + + mkdirSync(path.dirname(destPath), { recursive: true }); + writeFileSync(destPath, `${JSON.stringify(document, null, 2)}\n`); + + console.log(`Generated: ${path.relative(path.resolve(import.meta.dir, "../../.."), destPath)}`); +} + generateCommandDocs(); +writeLockSchemaAsset(); copyConfigSchemaAsset(CLI_CONFIG_SCHEMA_URL, "schema.json"); copyConfigSchemaAsset(PROJECT_CONFIG_SCHEMA_URL, "project-schema.json"); diff --git a/apps/cli/src/cli/complete.ts b/apps/cli/src/cli/complete.ts index c00c89fbe9..b171822fb3 100644 --- a/apps/cli/src/cli/complete.ts +++ b/apps/cli/src/cli/complete.ts @@ -11,6 +11,7 @@ import { } from "../command-internal/go-output-flag.ts"; import { unwrapParam } from "../command-internal/param-introspection.ts"; import { isValidBase0Int64, parseUintBase0 } from "../command-internal/parse-uint.ts"; +import type { ProjectStateMode } from "../command-internal/project-state-mode.ts"; import { parseStringSliceFlag } from "../command-internal/string-slice-flag.ts"; import { withAnalyticsContext } from "../shared/telemetry/analytics-context.ts"; import { Analytics } from "../shared/telemetry/analytics.service.ts"; @@ -25,7 +26,7 @@ import { cliSettingsLayer } from "../shared/config/cli-settings.layer.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; import { ttyLayer } from "../shared/runtime/tty.layer.ts"; import { cliConfigProviderLayer } from "../shared/config/cli-config-provider.layer.ts"; -import { analyticsLayer } from "../telemetry/analytics.layer.ts"; +import { analyticsLayerFor } from "../telemetry/analytics.layer.ts"; import { formatCliError, normalizeCliError } from "../shared/output/normalize-error.ts"; /** @@ -1118,19 +1119,23 @@ const COMPLETE_TELEMETRY_TIMEOUT = "2 seconds"; // Completion telemetry fires before `runCli` builds its layer tree, so the // `CliSettings`/`RuntimeInfo`/`Tty` set that `analyticsLayer` needs is resolved here. -const completeAnalyticsLayer = analyticsLayer.pipe( - Layer.provide( - Layer.mergeAll( - cliSettingsLayer.pipe(Layer.provide(cliProjectContextLayer), Layer.provide(runtimeInfoLayer)), - runtimeInfoLayer, - ttyLayer, +const completeAnalyticsLayer = (mode: ProjectStateMode) => + analyticsLayerFor(mode).pipe( + Layer.provide( + Layer.mergeAll( + cliSettingsLayer.pipe( + Layer.provide(cliProjectContextLayer), + Layer.provide(runtimeInfoLayer), + ), + runtimeInfoLayer, + ttyLayer, + ), ), - ), - Layer.provide(FetchHttpClient.layer), - Layer.provide(cliConfigProviderLayer), - Layer.provide(BunServices.layer), - Layer.merge(BunCrypto.layer), -); + Layer.provide(FetchHttpClient.layer), + Layer.provide(cliConfigProviderLayer), + Layer.provide(BunServices.layer), + Layer.merge(BunCrypto.layer), + ); /** * Production default for `CompleteDeps.captureTelemetry`. Best-effort and bounded: a missing @@ -1138,9 +1143,13 @@ const completeAnalyticsLayer = analyticsLayer.pipe( * must run this to completion before exiting, since `process.exit` kills the process without * waiting for pending async work. */ -function captureCompleteTelemetry(exitCode: number, durationMs: number): Effect.Effect { +function captureCompleteTelemetry( + mode: ProjectStateMode, + exitCode: number, + durationMs: number, +): Effect.Effect { return captureCompleteTelemetryEffect(exitCode, durationMs).pipe( - Effect.provide(completeAnalyticsLayer), + Effect.provide(completeAnalyticsLayer(mode)), Effect.timeout(COMPLETE_TELEMETRY_TIMEOUT), Effect.ignore, ); @@ -1180,6 +1189,7 @@ export const tryComplete = Effect.fn("cli.complete")(function* (deps: CompleteDe export function defaultCompleteDeps( root?: Command.Command.Any, routingFailure?: Cause.Cause, + projectState: ProjectStateMode = "legacy", ): CompleteDeps { return { root, @@ -1195,6 +1205,7 @@ export function defaultCompleteDeps( exit: (code) => { process.exit(code); }, - captureTelemetry: captureCompleteTelemetry, + captureTelemetry: (exitCode, durationMs) => + captureCompleteTelemetry(projectState, exitCode, durationMs), }; } diff --git a/apps/cli/src/cli/main.ts b/apps/cli/src/cli/main.ts index c22a5386db..4975ac6bee 100644 --- a/apps/cli/src/cli/main.ts +++ b/apps/cli/src/cli/main.ts @@ -4,9 +4,9 @@ import { Effect, Exit, Layer, Stdio } from "effect"; import { FetchHttpClient } from "effect/http"; import { runCli } from "../shared/cli/run.ts"; import { upgradeNoticeHook } from "../command-internal/upgrade-notice.ts"; -import { analyticsLayer } from "../telemetry/analytics.layer.ts"; +import { analyticsLayerFor } from "../telemetry/analytics.layer.ts"; import { defaultCompleteDeps, tryComplete } from "./complete.ts"; -import { resolveStackBackend } from "../command-internal/stack-backend.ts"; +import { resolveProjectStateSelection } from "../command-internal/stack-backend.ts"; import { resolveComputeEnabled } from "../commands/experimental/compute/compute-backend.ts"; import { cliEntrypointForFeatures } from "./root.ts"; @@ -19,13 +19,17 @@ const args = await Effect.runPromise( const selectionExit = await Effect.runPromiseExit( Effect.gen(function* () { - const stackBackend = yield* resolveStackBackend({ args, cwd: process.cwd(), env: process.env }); + const { projectState, stackBackend } = yield* resolveProjectStateSelection({ + args, + cwd: process.cwd(), + env: process.env, + }); const computeEnabled = yield* resolveComputeEnabled({ args, cwd: process.cwd(), env: process.env, }); - return { stackBackend, computeEnabled }; + return { projectState, stackBackend, computeEnabled }; }).pipe(Effect.provide(BunServices.layer)), ); const { rootCommand: selectedRoot, agentDefaultOutputFormat } = cliEntrypointForFeatures( @@ -35,17 +39,30 @@ const { rootCommand: selectedRoot, agentDefaultOutputFormat } = cliEntrypointFor args, ); const selectionCause = Exit.isFailure(selectionExit) ? selectionExit.cause : undefined; +const selectedProjectState = Exit.isSuccess(selectionExit) + ? selectionExit.value.projectState + : undefined; if ( !(await Effect.runPromise( tryComplete( - defaultCompleteDeps(Exit.isSuccess(selectionExit) ? selectedRoot : undefined, selectionCause), + defaultCompleteDeps( + Exit.isSuccess(selectionExit) ? selectedRoot : undefined, + selectionCause, + selectedProjectState !== undefined && Exit.isSuccess(selectedProjectState) + ? selectedProjectState.value + : "legacy", + ), ), )) ) { await Effect.runPromise( runCli(selectedRoot, { - analyticsLayer: analyticsLayer.pipe(Layer.provide(FetchHttpClient.layer)), + analyticsLayer: analyticsLayerFor( + selectedProjectState !== undefined && Exit.isSuccess(selectedProjectState) + ? selectedProjectState.value + : "legacy", + ).pipe(Layer.provide(FetchHttpClient.layer)), agentDefaultOutputFormat, afterSuccess: upgradeNoticeHook, ...(selectionCause ? { beforeParse: Effect.failCause(selectionCause) } : {}), diff --git a/apps/cli/src/cli/root.ts b/apps/cli/src/cli/root.ts index 50f60aa281..03a6fa9a48 100644 --- a/apps/cli/src/cli/root.ts +++ b/apps/cli/src/cli/root.ts @@ -1,4 +1,4 @@ -import { Effect, Layer, Option } from "effect"; +import { Effect, type Exit, Layer, Option } from "effect"; import { Command } from "effect/cli"; import { backupsCommand } from "../commands/backups/backups.command.ts"; import { bootstrapCommand } from "../commands/bootstrap/bootstrap.command.ts"; @@ -18,6 +18,11 @@ import { stackStopCommand } from "../commands/experimental/stack/stop/stop.comma import { stackStatusCommand } from "../commands/experimental/stack/status/status.command.ts"; import type { StackBackend } from "../command-internal/stack-backend.ts"; import { isFunctionsServePath, stackBackendLayer } from "../command-internal/stack-backend.ts"; +import type { ExperimentalFeatureFlagError } from "../command-internal/experimental-feature.ts"; +import { + type ProjectStateMode, + projectStateModeLayer, +} from "../command-internal/project-state-mode.ts"; import { computeCommand } from "../commands/experimental/compute/compute.command.ts"; import { feedbackCommand } from "../commands/feedback/feedback.command.ts"; import { functionsCommand } from "../commands/functions/functions.command.ts"; @@ -144,6 +149,7 @@ export function rootDescription(version: string): string { } interface RootCommandFeatures { + readonly projectState?: Exit.Exit; readonly stackBackend?: StackBackend; readonly computeEnabled?: boolean; } @@ -258,11 +264,14 @@ export const rootCommandForFeatures = (options: RootCommandFeatures = {}) => : outputLayerFor(outputFormat); return Layer.mergeAll( + options.projectState === undefined + ? Layer.empty + : projectStateModeLayer(options.projectState), options.stackBackend === undefined ? Layer.empty : stackBackendLayer(options.stackBackend), outputLayer, - makeGoProxyLayer({ globalArgs, parentOwnsCapturedSuccessTail: true }), + makeGoProxyLayer({ globalArgs, workdir, parentOwnsCapturedSuccessTail: true }), ); }), ), diff --git a/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts b/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts index e784f51404..5cf56eaa7d 100644 --- a/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts +++ b/apps/cli/src/command-internal/bundled-postgres-client.integration.test.ts @@ -11,6 +11,7 @@ import { dockerRunLayer } from "./docker-run.layer.ts"; import { ProcessControl } from "../shared/runtime/process-control.service.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { mockOutput } from "../../tests/helpers/mocks.ts"; +import { selection } from "../../tests/helpers/pins.ts"; import { mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; const processControl = Layer.succeed(ProcessControl, { @@ -55,7 +56,7 @@ describe("BundledPostgresClient", { timeout: 180_000 }, () => { const result = yield* Effect.gen(function* () { const client = yield* BundledPostgresClient; return yield* client.run({ - version: "17", + pin: selection.database, runtime, argv: runtime.kind === "native" @@ -82,7 +83,7 @@ describe("BundledPostgresClient", { timeout: 180_000 }, () => { const chunks = yield* Ref.make>([]); const client = yield* BundledPostgresClient; const result = yield* client.run({ - version: "17", + pin: selection.database, runtime: { kind: "container", engine: "docker" }, argv: ["pg_dump", "--version"], env: {}, diff --git a/apps/cli/src/command-internal/bundled-postgres-client.ts b/apps/cli/src/command-internal/bundled-postgres-client.ts index d46edd0017..e30c5ae3f7 100644 --- a/apps/cli/src/command-internal/bundled-postgres-client.ts +++ b/apps/cli/src/command-internal/bundled-postgres-client.ts @@ -14,7 +14,7 @@ import { FetchHttpClient } from "effect/http"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import { ArtifactError, - postgresVersion, + type ArtifactPin, resolveArtifact, useNativeArtifact, } from "@supabase/stack/internal/artifacts"; @@ -48,7 +48,7 @@ interface PostgresClientResult { } export interface RunPostgresClientOptions { - readonly version?: string; + readonly pin: ArtifactPin; readonly runtime?: StackRuntimePreference; readonly argv: ReadonlyArray; readonly env: Readonly>; @@ -242,9 +242,8 @@ export const bundledPostgresClientLayer: Layer.Layer< Effect.gen(function* () { yield* processControl.holdSignals(["SIGINT", "SIGTERM", "SIGHUP"]); const runtime = options.runtime ?? { kind: "native" }; - const version = postgresVersion(options.version ?? "17"); if (runtime.kind === "container") { - const resolved = yield* resolveArtifact({ service: "database", version }).pipe( + const resolved = yield* resolveArtifact({ service: "database", pin: options.pin }).pipe( Effect.mapError(mapArtifactError), ); return yield* dockerRun(options, resolved.image, docker); @@ -252,7 +251,7 @@ export const bundledPostgresClientLayer: Layer.Layer< // Pins the generation for the life of this scope, before the spawn inside `nativeRun`: // the client child always exits before this scope closes, so the pin covers it fully. const prepared = yield* useNativeArtifact( - { service: "database", version }, + { service: "database", pin: options.pin }, path.join(settings.supabaseHome, "cache", "stack"), { os: runtimeInfo.platform, arch: runtimeInfo.arch }, ).pipe( diff --git a/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts b/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts index 1f65b32b40..d620c292b2 100644 --- a/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts +++ b/apps/cli/src/command-internal/command-permissions/command-permissions.unit.test.ts @@ -1,4 +1,4 @@ -import { Option } from "effect"; +import { Exit, Option } from "effect"; import { describe, expect, it } from "vitest"; import { rootCommandForFeatures } from "../../cli/root.ts"; @@ -43,9 +43,11 @@ const VARIANTS: ReadonlyArray<{ readonly name: string; readonly variant: Permiss { name: "stack backend", // Sets every option, so a new option added to `rootCommandForFeatures` is a type error here. - variant: { stackBackend: "stack", computeEnabled: true } satisfies Required< - NonNullable - >, + variant: { + projectState: Exit.succeed("stack"), + stackBackend: "stack", + computeEnabled: true, + } satisfies Required>, }, ]; diff --git a/apps/cli/src/command-internal/db-bootstrap/bootstrap-config.ts b/apps/cli/src/command-internal/db-bootstrap/bootstrap-config.ts index fec6e16175..067ce31ae0 100644 --- a/apps/cli/src/command-internal/db-bootstrap/bootstrap-config.ts +++ b/apps/cli/src/command-internal/db-bootstrap/bootstrap-config.ts @@ -29,7 +29,8 @@ import { readServiceVersionOverrides, } from "../service-version-overrides.ts"; import { ramInBytes } from "../size-units.ts"; -import { tempPaths } from "../temp-paths.ts"; +import { legacyProjectStatePaths } from "../legacy-project-state.paths.ts"; +import { usesStackState } from "../stack-backend.ts"; export interface DbBootstrapConfigInput { readonly config: CliConfig; @@ -260,12 +261,12 @@ export const resolveDbBootstrapConfig = ( // Feeds `DB_MIGRATIONS_FREEZE_AT` for the one-shot Storage migrate job. Any read error // (including not-exist) or blank content resolves to "". - const storageTargetMigration = yield* fs - .readFileString(tempPaths(path, workdir).storageMigration) - .pipe( - Effect.map((content) => content.trim()), - Effect.orElseSucceed(() => ""), - ); + const storageTargetMigration = (yield* usesStackState) + ? "" + : yield* fs.readFileString(legacyProjectStatePaths(path, workdir).storageMigration).pipe( + Effect.map((content) => content.trim()), + Effect.orElseSucceed(() => ""), + ); return { majorVersion, diff --git a/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts b/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts index 695b632193..8089335953 100644 --- a/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/reset-local-database.ts @@ -47,6 +47,7 @@ import { stackStorageEndpointFor, } from "../stack-storage.ts"; import { loadStackConfig } from "../stack-config.ts"; +import { readSelectionOrBundled } from "../stack-lock.ts"; import { catalogDatabaseServices, StackCatalogSetup } from "../stack-catalog-setup.ts"; import { initializeStackDatabase, @@ -130,9 +131,10 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu const catalog = yield* Effect.serviceOption(StackCatalogSetup); if (Option.isNone(catalog)) return yield* resetFailed("stack catalog setup is unavailable"); const context = yield* loadLocalProjectContext(workdir, (message) => resetFailed(message)); - const config = yield* loadStackConfig(workdir, { context }).pipe( - Effect.mapError((cause) => resetFailed(cause.message)), - ); + const config = yield* loadStackConfig(workdir, { + context, + selection: readSelectionOrBundled, + }).pipe(Effect.mapError((cause) => resetFailed(cause.message, cause))); const toml = yield* readDbToml(fs, path, workdir); const databaseStatus = yield* opened.value.database.status.pipe( Effect.mapError((cause) => resetFailed(`failed to inspect stack: ${cause.message}`)), @@ -160,6 +162,15 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu (service) => config.source[service].enabled, ) : catalogDatabaseServices(members); + // The one-shots run the artifacts the saved services were created with. + const savedPins = yield* Effect.forEach(members, (member) => + member.status.pipe( + Effect.map(({ config: saved }) => [saved.service, saved.pin] as const), + Effect.mapError((cause) => + resetFailed(`failed to inspect stack service: ${cause.message}`), + ), + ), + ); yield* output.raw(`Resetting local database${toLogMessage(input.version)}\n`, "stderr"); yield* Effect.gen(function* () { yield* opened.value.stack.composition.stop.pipe( @@ -188,6 +199,7 @@ export const resetLocalDatabase = Effect.fn("DbBootstrap.resetLocalDatabase")(fu stack: opened.value.stack, database: opened.value.database, databaseServices, + selection: { ...config.selection, ...Object.fromEntries(savedPins) }, }, overlay: projectCatalogOverlay(toml, workdir), migrations: { diff --git a/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts index 77faa99f50..7aecc2c49f 100644 --- a/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/rollback.unit.test.ts @@ -71,7 +71,7 @@ describe("rollbackStart", () => { false, "/tmp/rollback-unit-test-workdir", false, - ); + ).pipe(Effect.provide(BunServices.layer)); expect(stderr).toHaveBeenCalledTimes(1); expect(stderr).toHaveBeenCalledWith("Stopping containers...\n"); expect(mock.spawned.map((args) => args[0])).toEqual(["ps", "container", "network"]); @@ -87,7 +87,7 @@ describe("rollbackStart", () => { true, "/tmp/rollback-unit-test-workdir", false, - ); + ).pipe(Effect.provide(BunServices.layer)); expect(mock.spawned.map((args) => args[0])).toEqual([ "ps", "container", @@ -108,7 +108,7 @@ describe("rollbackStart", () => { false, "/tmp/rollback-unit-test-workdir", false, - ); + ).pipe(Effect.provide(BunServices.layer)); expect(stderr).toHaveBeenCalledTimes(2); expect(stderr).toHaveBeenNthCalledWith(1, "Stopping containers...\n"); expect(stderr).toHaveBeenNthCalledWith(2, "failed to list containers: permission denied\n"); @@ -125,7 +125,7 @@ describe("rollbackStart", () => { false, "/tmp/rollback-unit-test-workdir", false, - ); + ).pipe(Effect.provide(BunServices.layer)); expect(stderr).toHaveBeenCalledTimes(2); expect(stderr).toHaveBeenNthCalledWith(1, "Stopping containers...\n"); expect(stderr).toHaveBeenNthCalledWith(2, "failed to list containers\n"); diff --git a/apps/cli/src/command-internal/db-bootstrap/shadow-database.ts b/apps/cli/src/command-internal/db-bootstrap/shadow-database.ts index cf0c6fb4ba..e833233bf9 100644 --- a/apps/cli/src/command-internal/db-bootstrap/shadow-database.ts +++ b/apps/cli/src/command-internal/db-bootstrap/shadow-database.ts @@ -22,6 +22,7 @@ import { Output } from "../../shared/output/output.service.ts"; import type { RuntimeInfo } from "../../shared/runtime/runtime-info.service.ts"; import { actionability, + causeDeclaration, type CliErrorActionabilityDeclaration, ErrorActionabilityId, } from "../../shared/telemetry/error-actionability.ts"; @@ -82,6 +83,7 @@ export class ShadowDbError extends Data.TaggedError("ShadowDbError")<{ | "port_conflict" | "filesystem" | "database"; + readonly cause?: unknown; }> { get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { switch (this.reason) { @@ -98,7 +100,12 @@ export class ShadowDbError extends Data.TaggedError("ShadowDbError")<{ case "database": return { ...actionability.dbFinding, fingerprint_suffix: "database" }; default: - return { ...actionability.invalidConfig, fingerprint_suffix: "container_configuration" }; + return ( + causeDeclaration(this.cause) ?? { + ...actionability.invalidConfig, + fingerprint_suffix: "container_configuration", + } + ); } } } diff --git a/apps/cli/src/command-internal/db-config.integration.test.ts b/apps/cli/src/command-internal/db-config.integration.test.ts index a787b37606..95b7ca5b86 100644 --- a/apps/cli/src/command-internal/db-config.integration.test.ts +++ b/apps/cli/src/command-internal/db-config.integration.test.ts @@ -62,6 +62,9 @@ import { DbConnectError } from "./db-connection.errors.ts"; import { DbConnection, type DbSession, type PgConnInput } from "./db-connection.service.ts"; import { StackApi } from "./stack-api.ts"; import { stackBackendLayer } from "./stack-backend.ts"; +import { writeProjectLink } from "./project-link.ts"; +import { projectStateModeLayer, type ProjectStateMode } from "./project-state-mode.ts"; +import { selection } from "../../tests/helpers/pins.ts"; // `--local` / `--db-url` never touch the Management API stack, so the resolver // builds with simple ambient stubs. The `--linked` sub-flow (login-role, @@ -85,6 +88,7 @@ function buildResolver( readonly debugLogger?: Layer.Layer; readonly configEnv?: Record; readonly stackApi?: Layer.Layer; + readonly projectStateMode?: ProjectStateMode; } = {}, ) { const deps = Layer.mergeAll( @@ -145,7 +149,12 @@ const resolve = ( const resolver = yield* DbConfigResolver; return yield* resolver.resolve(flags); }).pipe( - Effect.provide(buildResolver(workdir, opts)), + Effect.provide( + Layer.merge( + buildResolver(workdir, opts), + projectStateModeLayer(opts?.projectStateMode ?? "legacy"), + ), + ), Effect.provideService( ConfigProvider.ConfigProvider, ConfigProvider.fromEnvRecord(opts?.configEnv ?? {}, { preserveEmptyStrings: true }), @@ -161,7 +170,12 @@ const resolvePoolerFallback = ( const resolver = yield* DbConfigResolver; return yield* resolver.resolvePoolerFallback(flags); }).pipe( - Effect.provide(buildResolver(workdir, opts)), + Effect.provide( + Layer.merge( + buildResolver(workdir, opts), + projectStateModeLayer(opts?.projectStateMode ?? "legacy"), + ), + ), Effect.provideService( ConfigProvider.ConfigProvider, ConfigProvider.fromEnvRecord(opts?.configEnv ?? {}, { preserveEmptyStrings: true }), @@ -460,8 +474,8 @@ describe("dbConfigResolver (db-url under the stack backend)", () => { : [], config: { service: "database", + pin: selection.database, config: { - version: "17.6.1.173", databasePassword: Redacted.make("stack-password"), jwtSecret: Redacted.make("secret"), jwtExpiry: 3600, @@ -1380,6 +1394,107 @@ describe("dbConfigResolver (--project-ref pooler fetch decoupled from adHocProje }, ); + it.effect( + "a SUPABASE_PROJECT_ID that differs from the link fetches that ref's pooler instead of using the saved pooler URL", + () => { + const linkedRef = "workdirlinkedrefabcd"; + const envRef = "envprojectrefabcdefg"; + + const previousFetch = globalThis.fetch; + const requests: Array<{ readonly method: string; readonly path: string }> = []; + const dbConnection = Layer.succeed(DbConnection, { + connect: () => + Effect.die("unexpected connect() — the ambient password path never verify-connects"), + }); + const fetchMock = Object.assign( + (input: string | URL | Request, init?: RequestInit): Promise => { + const url = new URL( + typeof input === "string" || input instanceof URL ? input : input.url, + ); + const method = init?.method ?? (input instanceof Request ? input.method : "GET"); + requests.push({ method, path: url.pathname }); + const connectionString = `postgres://postgres.${envRef}:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.com:6543/postgres`; + if ( + method === "GET" && + url.pathname === `/v1/projects/${envRef}/config/database/pooler` + ) { + return Promise.resolve( + new Response( + JSON.stringify([ + { + identifier: "primary", + database_type: "PRIMARY", + is_using_scram_auth: true, + db_user: "postgres", + db_host: "db.example", + db_port: 5432, + db_name: "postgres", + connection_string: connectionString, + connectionString, + default_pool_size: null, + max_client_conn: null, + pool_mode: "transaction", + }, + ]), + { status: 200, headers: { "content-type": "application/json" } }, + ), + ); + } + return Promise.resolve( + new Response(JSON.stringify({ message: "unexpected request" }), { + status: 404, + headers: { "content-type": "application/json" }, + }), + ); + }, + { preconnect: previousFetch.preconnect }, + ); + + globalThis.fetch = fetchMock; + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const dir = yield* withWorkdir(); + yield* fs.makeDirectory(path.join(dir, "supabase", ".temp"), { recursive: true }); + yield* fs.writeFileString(path.join(dir, "supabase", ".temp", "project-ref"), linkedRef); + yield* fs.writeFileString( + path.join(dir, "supabase", ".temp", "pooler-url"), + `postgres://postgres.${linkedRef}:saved-workdir-password@stale.pooler.supabase.com:6543/postgres`, + ); + return yield* resolve(dir, linkedFlags, { + projectHost: "invalid", + dbConnection, + configEnv: { + SUPABASE_ACCESS_TOKEN: VALID_TOKEN, + SUPABASE_DB_PASSWORD: "ambient-password", + SUPABASE_PROJECT_ID: envRef, + }, + }); + }).pipe( + Effect.tap((r) => + Effect.sync(() => { + expect(r.conn).toMatchObject({ + host: "aws-0-us-east-1.pooler.supabase.com", + user: `postgres.${envRef}`, + password: "ambient-password", + }); + expect(requests).toEqual([ + { method: "GET", path: `/v1/projects/${envRef}/config/database/pooler` }, + ]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + globalThis.fetch = previousFetch; + }), + ), + Effect.scoped, + Effect.provide(BunServices.layer), + ); + }, + ); + it.effect( "an explicit --project-ref with no password mints a login role and clears network bans after three failed temp-role connects", () => { @@ -1914,3 +2029,34 @@ describe("dbConfigResolver (container pooler fallback)", () => { }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); }); }); + +describe("dbConfigResolver (stack-mode link document)", () => { + it.effect( + "the pooler fallback reuses the pooler URL saved in .link.json without any API call", + () => { + const ref = "stackmodelinkedrefab"; + return Effect.gen(function* () { + const dir = yield* withWorkdir(); + yield* writeProjectLink(dir, { + project: { ref }, + active_branch: { + ref, + pooler_url: `postgres://postgres.${ref}:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.com:6543/postgres`, + }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const pooler = yield* resolvePoolerFallback(dir, linkedFlags, { + projectStateMode: "stack", + configEnv: { SUPABASE_DB_PASSWORD: "linked-password" }, + }); + + expect(Option.getOrUndefined(pooler)).toMatchObject({ + host: "aws-0-us-east-1.pooler.supabase.com", + user: `postgres.${ref}`, + password: "linked-password", + }); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + }, + ); +}); diff --git a/apps/cli/src/command-internal/db-config.layer.ts b/apps/cli/src/command-internal/db-config.layer.ts index 0f9a50a969..d11fe6db49 100644 --- a/apps/cli/src/command-internal/db-config.layer.ts +++ b/apps/cli/src/command-internal/db-config.layer.ts @@ -5,7 +5,7 @@ import { Config, Crypto, Duration, Effect, FileSystem, Layer, Option, Path } fro import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; -import { ProjectRefResolver, PROJECT_REF_PATTERN } from "../config/project-ref.service.ts"; +import { RemoteTarget, PROJECT_REF_PATTERN } from "../config/remote-target.service.ts"; import { DebugFlag, DnsResolverFlag, @@ -56,14 +56,14 @@ const linkedEntries: ReadonlyArray> = [ kind: "required", when: [{ flag: "project-ref" }], context: - "the direct database host is unreachable (IPv4-only network) and there is no usable saved supabase/.temp/pooler-url", + "the direct database host is unreachable (IPv4-only network) and there is no usable saved pooler URL (supabase/.temp/pooler-url, or supabase/.link.json in stack mode), or a SUPABASE_PROJECT_ID ref differs from the link", }, { operationId: "v1-get-pooler-config", kind: "required", when: [{ flag: "project-ref", oneOf: [""] }], context: - "an empty --project-ref falls back to the linked project but still allows the pooler fetch, when the direct database host is unreachable (IPv4-only network) and there is no usable saved supabase/.temp/pooler-url", + "an empty --project-ref falls back to the linked project but still allows the pooler fetch, when the direct database host is unreachable (IPv4-only network) and there is no usable saved pooler URL (supabase/.temp/pooler-url, or supabase/.link.json in stack mode), or a SUPABASE_PROJECT_ID ref differs from the link", }, { operationId: "v1-list-all-network-bans", @@ -107,7 +107,7 @@ const poolerFallbackEntries: ReadonlyArray> = [ operationId: "v1-get-pooler-config", kind: "required", context: - "the command's database work fails with an IPv6 connectivity error against the direct database host, and there is no usable saved supabase/.temp/pooler-url", + "the command's database work fails with an IPv6 connectivity error against the direct database host, and there is no usable saved pooler URL (supabase/.temp/pooler-url, or supabase/.link.json in stack mode), or a SUPABASE_PROJECT_ID ref differs from the link", }, { operationId: "v1-create-login-role", @@ -345,11 +345,13 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( poolerHost: string, dnsResolver: "native" | "https", password: string, + // The pooler URL the link recorded for `ref`. + savedPoolerUrl: Option.Option, // The container-fallback path falls back to the Management API's primary pooler config when - // no `.temp/pooler-url` is saved; the resolve-time IPv6 path uses the saved URL only and + // no pooler URL is saved; the resolve-time IPv6 path uses the saved URL only and // errors otherwise, so this defaults off. fetchFromApi = false, - // For an ad-hoc `--project-id` ref the saved `.temp/pooler-url` belongs to the + // For an ad-hoc `--project-id` ref the saved pooler URL belongs to the // (possibly different) linked workdir, so ignore it and resolve the pooler for // `ref` from the Management API instead. ignoreSavedUrl = false, @@ -362,12 +364,10 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( // on the main linked branch rather than validating base config. For an ad-hoc `--project-id` // ref, skip the saved workdir pooler URL because it belongs to the linked project, not // necessarily the explicit ref. - const tomlValues = yield* readDbToml(fs, path, workdir, ref, { + yield* readDbToml(fs, path, workdir, ref, { resolveVaultSecrets, }); - let connectionString = ignoreSavedUrl - ? undefined - : Option.getOrUndefined(tomlValues.poolerConnectionString); + let connectionString = ignoreSavedUrl ? undefined : Option.getOrUndefined(savedPoolerUrl); if (connectionString === undefined) { if (!fetchFromApi) return Option.none(); // No saved pooler URL → fetch the primary pooler config from the Management API. Any API @@ -418,7 +418,7 @@ const resolvePoolerConn = Effect.fn("DbConfig.resolvePoolerConn")(function* ( * from `CommandSettings.workdir`'s cwd-walked value — e.g. `bootstrap`, whose own * `process.chdir` happens after that layer is built. `bootstrap` calls this directly with its * own local `workdir`/`projectRef`/`created.dbPassword`, without going through - * `DbConfigResolver`/`ProjectRefResolver` (both keyed off the ambient, potentially-stale + * `DbConfigResolver`/`RemoteTarget` (both keyed off the ambient, potentially-stale * `CommandSettings.workdir`). */ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(function* ( @@ -431,12 +431,14 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio options: { readonly adHocProjectRef?: boolean; readonly resolveVaultSecrets?: boolean; + /** The pooler URL the link recorded for `ref`, used when the direct host is unreachable. */ + readonly poolerUrl?: Option.Option; /** * Requests the Management API pooler-config fetch on an IPv4-only network * independent of `adHocProjectRef`'s credential/saved-URL semantics — see * `DbConfigFlags.linkedProjectRef`'s doc comment. Set when the caller - * supplied an explicit ref (`--project-ref`/`--project-id`) rather than - * falling back to `.temp/project-ref`, so an unlinked or mismatched-tenant + * supplied an explicit ref (`--project-ref`/`--project-id` or + * `SUPABASE_PROJECT_ID`) rather than falling back to the link, so an unlinked or mismatched-tenant * workdir still reaches the primary pooler config instead of dead-ending in * the "run supabase link" IPv6 error. */ @@ -447,6 +449,7 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio adHocProjectRef = false, resolveVaultSecrets = true, fetchPoolerFromApi = false, + poolerUrl = Option.none(), } = options; const debug = yield* DebugLogger; // Read lazily (per invocation) rather than at layer build, so tests and @@ -490,6 +493,7 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio poolerHost, dnsResolver, base.password, + poolerUrl, adHocProjectRef || fetchPoolerFromApi, adHocProjectRef, resolveVaultSecrets, @@ -661,20 +665,22 @@ export const dbConfigResolverLayer = Layer.effect( }; } - // --linked. The lazy Management API runtime (project-ref resolver + lazy platform API + // --linked. The lazy Management API runtime (remote target + lazy platform API // factory) is provided here at runtime so it is only built on this branch — // `--local`/`--db-url` never touch it. The factory resolves the access token only on // first use (minting a temp role), so a `--linked --password` invocation stays // auth-free. if (flags.connType === "linked") { const linked = yield* Effect.gen(function* () { - const projectRef = yield* ProjectRefResolver; - // Load-or-fail with no prompt: use `loadProjectRef` (not the soft - // `resolveOptional`, which swallows a read error to `None`) — an unlinked workdir - // fails with a not-linked error, a bad ref with the invalid-ref error, and an - // unreadable ref file surfaces the filesystem problem, for every caller of this - // resolver (`test db --linked`, dump, declarative). - const ref = yield* projectRef.loadProjectRef(flags.linkedProjectRef ?? Option.none()); + const remoteTarget = yield* RemoteTarget; + // Load-or-fail with no prompt: use `load` (not the soft `resolveOptional`, which + // swallows a read error to `None`) — an unlinked workdir fails with a not-linked + // error, a bad ref with the invalid-ref error, and an unreadable link surfaces the + // filesystem problem, for every caller of this resolver (`test db --linked`, dump, + // declarative). A caller that already resolved passes `flags.target`. + const target = + flags.target ?? (yield* remoteTarget.load(flags.linkedProjectRef ?? Option.none())); + const { ref } = target; // The `[remotes.]`-merged config (e.g. an unsupported remote // `db.major_version`/`edge_runtime.deno_version`) is validated as a pure config // error before any network work. The base read in `resolve` above only validates @@ -694,11 +700,15 @@ export const dbConfigResolverLayer = Layer.effect( { adHocProjectRef: flags.adHocProjectRef ?? false, resolveVaultSecrets, - // An explicit ref (the eight `db` commands' `--project-ref`, or - // `gen types --project-id`) independently unlocks the Management API - // pooler fetch, regardless of `adHocProjectRef`'s credential semantics - // — see `DbConfigFlags.linkedProjectRef`'s doc comment. - fetchPoolerFromApi: Option.isSome(flags.linkedProjectRef ?? Option.none()), + poolerUrl: Option.fromNullishOr(target.poolerUrl), + // A ref from a flag or `SUPABASE_PROJECT_ID` independently unlocks the + // Management API pooler fetch, regardless of `adHocProjectRef`'s credential + // semantics. The target carries the saved pooler URL only when its ref is the + // linked one. + fetchPoolerFromApi: + target.source !== "link" || + // Covers an empty `--project-ref` value and `gen types`, which always passes the resolved ref. + Option.isSome(flags.linkedProjectRef ?? Option.none()), }, ); // The linked-project telemetry cache (GET /v1/projects/{ref}) is not issued here: @@ -749,23 +759,27 @@ export const dbConfigResolverLayer = Layer.effect( Effect.gen(function* () { if (flags.connType !== "linked") return Option.none(); return yield* Effect.gen(function* () { - const projectRef = yield* ProjectRefResolver; - const refOpt = yield* projectRef.resolveOptional(flags.linkedProjectRef ?? Option.none()); - if (Option.isNone(refOpt)) return Option.none(); - const ref = refOpt.value; + const remoteTarget = yield* RemoteTarget; + const targetOpt = + flags.target === undefined + ? yield* remoteTarget.resolveOptional(flags.linkedProjectRef ?? Option.none()) + : Option.some(flags.target); + if (Option.isNone(targetOpt)) return Option.none(); + const { ref } = targetOpt.value; if (!PROJECT_REF_PATTERN.test(ref)) return Option.none(); const adHocProjectRef = flags.adHocProjectRef ?? false; const password = adHocProjectRef ? (Option.getOrUndefined(flags.password ?? Option.none()) ?? "") : yield* resolveDbPassword(flags.password ?? Option.none(), cliSettings.workdir); // Container-fallback: fetch the primary pooler config from the Management API when - // no `.temp/pooler-url` is saved. + // the link recorded no pooler URL. return yield* resolvePoolerConn( ref, cliSettings.workdir, cliSettings.poolerHost, flags.dnsResolver, password, + Option.fromNullishOr(targetOpt.value.poolerUrl), true, adHocProjectRef, flags.resolveVaultSecrets ?? true, diff --git a/apps/cli/src/command-internal/db-config.service.ts b/apps/cli/src/command-internal/db-config.service.ts index f4e5bdfadf..f8da056666 100644 --- a/apps/cli/src/command-internal/db-config.service.ts +++ b/apps/cli/src/command-internal/db-config.service.ts @@ -8,7 +8,7 @@ import type { ProjectRefNotLinkedError, } from "../config/project-ref.errors.ts"; import type { ProfileLoadError } from "./profile-load.ts"; -import type { ProjectRefReadError } from "./temp-paths.ts"; +import type { LinkedProjectRefReadError } from "./project-link.ts"; import type { DbConnectError } from "./db-connection.errors.ts"; import type { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; import type { @@ -34,9 +34,9 @@ export type DbConfigError = | LocalDbRunningError | ProjectRefNotLinkedError | InvalidProjectRefError - // A hard linked-ref load surfaces a real `.temp/project-ref` read error instead of masking it + // A hard linked-ref load surfaces a real link-state read error instead of masking it // as not-linked. - | ProjectRefReadError + | LinkedProjectRefReadError | DbConfigLoginRoleNetworkError | DbConfigLoginRoleStatusError | DbConfigListBansNetworkError diff --git a/apps/cli/src/command-internal/db-config.toml-read.ts b/apps/cli/src/command-internal/db-config.toml-read.ts index ac0c9112fb..cb9cbc4558 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.ts +++ b/apps/cli/src/command-internal/db-config.toml-read.ts @@ -38,9 +38,9 @@ import { Output } from "../shared/output/output.service.ts"; type EnvLookup = (name: string) => string | undefined; /** - * Subset of `supabase/config.toml` (plus the linked pooler URL) the db-config - * resolver needs. A missing config file yields defaults; a malformed one aborts - * the command instead of running against the default local database. + * Subset of `supabase/config.toml` the db-config resolver needs. A missing config file yields + * defaults; a malformed one aborts the command instead of running against the default local + * database. */ export interface DbTomlValues { readonly projectEnv: Readonly>; @@ -57,12 +57,6 @@ export interface DbTomlValues { readonly shadowPort: number; /** `[db] password`, runtime default `"postgres"` (not in the config schema). */ readonly password: string; - /** - * Linked pooler connection string, used by the `--linked` pooler fallback. - * Read from `supabase/.temp/pooler-url` (written by `supabase link`); not - * part of the config schema. - */ - readonly poolerConnectionString: Option.Option; /** top-level `project_id`, used to name the local docker network. */ readonly projectId: Option.Option; /** `[db] major_version`, default 17. */ @@ -1103,12 +1097,11 @@ export const warnDeprecatedOrioleDBVersionEnv = Effect.fnUntraced(function* ( }); /** - * Reads `/supabase/config.toml` (db subtree + project id) and the linked - * `/supabase/.temp/pooler-url`. `fs`/`path` are passed in so the resolver - * can capture them once and keep its own `R` at `never`. + * Reads `/supabase/config.toml` (db subtree + project id). `fs`/`path` are passed in so + * the resolver can capture them once and keep its own `R` at `never`. * - * Fails with `DbConfigLoadError` only when the config file is present but - * unparseable; an absent file (and an absent/empty pooler-url file) is not an error. + * Fails with `DbConfigLoadError` only when the config file is present but unparseable; an absent + * file is not an error. */ const readDbTomlCore = Effect.fnUntraced(function* ( fs: FileSystem.FileSystem, @@ -1258,13 +1251,6 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // comment for why this is typed narrower than the `ReadonlySet` it wraps. const remoteWins = makeRemoteWins(remoteOverrideKeys); - // Read the linked pooler URL from `.temp/pooler-url` and treat it as configured only - // when the file exists and is non-empty. - const poolerUrlPath = path.join(supabaseDir, ".temp", "pooler-url"); - const poolerConnectionString = yield* fs - .readFileString(poolerUrlPath) - .pipe(Effect.map(nonEmptyString), Effect.orElseSucceed(Option.none)); - // `SUPABASE_PROJECT_ID` overrides the top-level `project_id` used to name the local // stack's Docker resources, unless a matched `[remotes.]` block already set // `project_id` at override tier — that value must win over a stale/differently-scoped @@ -2197,7 +2183,6 @@ const readDbTomlCore = Effect.fnUntraced(function* ( port, shadowPort, password: passwordRaw !== undefined ? expandEnv(passwordRaw, lookup) : DEFAULT_PASSWORD, - poolerConnectionString, projectId, majorVersion, orioledbVersion, diff --git a/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts b/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts index 2f88cc8ef1..7c15a71d5c 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts +++ b/apps/cli/src/command-internal/db-config.toml-read.unit.test.ts @@ -16,7 +16,7 @@ import { type CommandTelemetryAttributeValues, } from "../telemetry/command-telemetry-attributes.ts"; -function withConfig(content: string | undefined, poolerUrl?: string) { +function withConfig(content: string | undefined) { return Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -25,10 +25,6 @@ function withConfig(content: string | undefined, poolerUrl?: string) { yield* fs.makeDirectory(path.join(dir, "supabase"), { recursive: true }); yield* fs.writeFileString(path.join(dir, "supabase", "config.toml"), content); } - if (poolerUrl !== undefined) { - yield* fs.makeDirectory(path.join(dir, "supabase", ".temp"), { recursive: true }); - yield* fs.writeFileString(path.join(dir, "supabase", ".temp", "pooler-url"), poolerUrl); - } return dir; }).pipe(Effect.provide(BunServices.layer)); } @@ -165,7 +161,6 @@ describe("readDbToml", () => { expect(v.port).toBe(54322); expect(v.shadowPort).toBe(54320); expect(v.password).toBe("postgres"); - expect(Option.isNone(v.poolerConnectionString)).toBe(true); expect(Option.isNone(v.projectId)).toBe(true); expect(v.denoVersion).toBe(2); }), @@ -1494,13 +1489,12 @@ describe("readDbToml", () => { Effect.sync(() => { expect(v.port).toBe(5000); expect(v.password).toBe("postgres"); - expect(Option.isNone(v.poolerConnectionString)).toBe(true); }), ), ); }); - it.effect("reads db + project_id from config.toml and pooler url from .temp", () => { + it.effect("reads db + project_id from config.toml", () => { return withConfig( [ 'project_id = "my-project"', @@ -1510,7 +1504,6 @@ describe("readDbToml", () => { 'password = "hunter2"', "", ].join("\n"), - "postgres://postgres.ref:[YOUR-PASSWORD]@pool:6543/postgres", ).pipe( Effect.flatMap(read), Effect.tap((v) => @@ -1519,7 +1512,6 @@ describe("readDbToml", () => { expect(v.shadowPort).toBe(55556); expect(v.password).toBe("hunter2"); expect(Option.getOrNull(v.projectId)).toBe("my-project"); - expect(Option.getOrNull(v.poolerConnectionString)).toContain("postgres.ref"); }), ), ); @@ -2273,34 +2265,6 @@ describe("readDbToml", () => { ); }, ); - - it.effect("ignores a [db.pooler] connection_string in config.toml (Go reads .temp only)", () => { - return withConfig( - [ - "[db.pooler]", - 'connection_string = "postgres://postgres.ref:[YOUR-PASSWORD]@pool:6543/postgres"', - "", - ].join("\n"), - ).pipe( - Effect.flatMap(read), - Effect.tap((v) => - Effect.sync(() => { - expect(Option.isNone(v.poolerConnectionString)).toBe(true); - }), - ), - ); - }); - - it.effect("treats an empty .temp/pooler-url as no pooler configured", () => { - return withConfig(undefined, "").pipe( - Effect.flatMap(read), - Effect.tap((v) => - Effect.sync(() => { - expect(Option.isNone(v.poolerConnectionString)).toBe(true); - }), - ), - ); - }); }); describe("readDbToml [experimental.pgdelta]", () => { diff --git a/apps/cli/src/command-internal/db-config.types.ts b/apps/cli/src/command-internal/db-config.types.ts index 6307d18a0e..508507dbc8 100644 --- a/apps/cli/src/command-internal/db-config.types.ts +++ b/apps/cli/src/command-internal/db-config.types.ts @@ -1,4 +1,5 @@ import type { Option } from "effect"; +import type { ResolvedRemoteTarget } from "../config/remote-target.service.ts"; import type { PgConnInput } from "./db-connection.service.ts"; import type { DbConnType } from "./db-target-flags.ts"; @@ -35,15 +36,20 @@ export interface DbConfigFlags { * Optional explicit linked project ref override, letting e.g. `gen types --project-id ` * reuse the linked resolver's temp-role and pooler fallback without the workdir being linked. * Shares only `SUPABASE_PROJECT_ID`'s ref-resolution precedence (flag > env > - * `.temp/project-ref`) — it never drives the local container id or pg-delta project id the way + * the link) — it never drives the local container id or pg-delta project id the way * that env var does. `None` preserves the normal `--linked` fallback. */ readonly linkedProjectRef?: Option.Option; + /** + * The target the caller already resolved through `RemoteTarget`. The `--linked` path uses it + * as is instead of resolving again, and takes the saved pooler URL from it. + */ + readonly target?: ResolvedRemoteTarget; /** * Marks `linkedProjectRef` as an ad-hoc remote target (e.g. `gen types --project-id `) * rather than the current linked workdir, so the resolver must not reuse workdir-scoped * credentials or cached state for it: it ignores the ambient `SUPABASE_DB_PASSWORD` and skips - * the saved `.temp/pooler-url`, fetching pooler config from the Management API instead. Absent + * the saved pooler URL, fetching pooler config from the Management API instead. Absent * for the normal `--linked` path, which may reuse both. */ readonly adHocProjectRef?: boolean; diff --git a/apps/cli/src/command-internal/db-image.ts b/apps/cli/src/command-internal/db-image.ts index 7436a030a8..29ef955f5e 100644 --- a/apps/cli/src/command-internal/db-image.ts +++ b/apps/cli/src/command-internal/db-image.ts @@ -1,3 +1,5 @@ +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { usesStackState } from "./stack-backend.ts"; import { Effect, type FileSystem, type Path } from "effect"; import { dockerfileServiceImageRaw } from "../shared/services/dockerfile-images.ts"; import { postgresImageForDbMajorVersion } from "../shared/services/services.shared.ts"; @@ -72,9 +74,10 @@ export const recordOrioleDbTelemetry = ( /** * Resolves the Postgres image for `majorVersion`, honoring the pinned version written by * `supabase start` to `supabase/.temp/postgres-version`. The tag is only replaced when the - * configured image is at/above 15.1.0.55. + * configured image is at/above 15.1.0.55. Stack state mode has no `.temp` pin: the bundled + * image applies, and callers that run it take the stack's selection instead. */ -export const resolveDbImage = Effect.fnUntraced(function* ( +export const resolveDbImage = Effect.fn("DbImage.resolve")(function* ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, @@ -93,12 +96,13 @@ export const resolveDbImage = Effect.fnUntraced(function* ( } const currentRaw = postgresImageForDbMajorVersion(majorVersion) ?? pgImageRaw(); let appliedPin: string | undefined; - if (majorVersion > 14) { - const versionPath = path.join(workdir, "supabase", ".temp", "postgres-version"); - const pinned = yield* fs.readFileString(versionPath).pipe( - Effect.map((s) => s.trim()), - Effect.orElseSucceed(() => ""), - ); + if (majorVersion > 14 && !(yield* usesStackState)) { + const pinned = yield* fs + .readFileString(legacyProjectStatePaths(path, workdir).postgresVersion) + .pipe( + Effect.map((s) => s.trim()), + Effect.orElseSucceed(() => ""), + ); if (pinned.length > 0) { const colon = currentRaw.indexOf(":"); const currentTag = colon >= 0 ? currentRaw.slice(colon + 1) : currentRaw; diff --git a/apps/cli/src/command-internal/db-pull-run.ts b/apps/cli/src/command-internal/db-pull-run.ts index 88448adb7b..c10db0437a 100644 --- a/apps/cli/src/command-internal/db-pull-run.ts +++ b/apps/cli/src/command-internal/db-pull-run.ts @@ -12,7 +12,7 @@ import { CliArgs } from "../shared/cli/cli-args.service.ts"; import { Output } from "../shared/output/output.service.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../config/remote-target.service.ts"; import { bold } from "./colors.ts"; import { promptYesNo } from "./prompt-yes-no.ts"; import { ipv6Suggestion, isIPv6ConnectivityError } from "./connect-errors.ts"; @@ -70,7 +70,8 @@ import { } from "../commands/db/shared/pgdelta-engine.service.ts"; import { type PgDeltaContext, isPgDeltaDebugEnabled, resolvePgDeltaProjectId } from "./pgdelta.ts"; import { prepareShadowSource } from "../commands/db/shared/shadow-source.ts"; -import { currentStackBackend } from "./stack-backend.ts"; +import { currentStackBackend, usesStackState } from "./stack-backend.ts"; +import { selectionForRequest } from "./stack-project-selection.ts"; import { stackRejectNativeDockerDiffEngine } from "./stack-local-database.ts"; import { stackPrepareShadowSource, stackWithShadowDatabase } from "./stack-shadow.ts"; import type { DbPullFlags } from "../commands/db/pull/pull.command.ts"; @@ -257,9 +258,11 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( // tell the user which `[remotes.*]` block matched. `--local`/`--db-url` never merge a // remote block, so only the linked path pre-resolves a ref. let linkedRef: string | undefined; + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const projectRefResolver = yield* ProjectRefResolver; - linkedRef = yield* projectRefResolver.loadProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + linkedRef = linkedTarget.ref; // Cache the ref the moment it's known, not after `toml`/`localInputs` below resolve // (both fallible): the project cache should still be written even when a later step // (config validation, connection, the pull itself) fails. @@ -300,6 +303,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( dnsResolver, password: flags.password ?? Option.none(), linkedProjectRef: flags.projectRef, + target: linkedTarget, }); if (linkedRef === undefined) { linkedRef = Option.getOrUndefined(resolved.ref ?? Option.none()); @@ -349,6 +353,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( dnsResolver, password: flags.password ?? Option.none(), linkedProjectRef: flags.projectRef, + target: linkedTarget, }) .pipe(Effect.orElseSucceed(() => Option.none())); if (Option.isSome(pooler)) { @@ -484,13 +489,21 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( yield* makeDir(fs, path.dirname(migrationPath)).pipe( Effect.mapError((cause) => new DbPullWriteError({ message: cause.message })), ); - const image = (yield* resolveDbImage( - fs, - path, - cliSettings.workdir, - toml.majorVersion, - Option.getOrUndefined(toml.orioledbVersion), - )).image; + const image = (yield* usesStackState) + ? (yield* selectionForRequest({ + projectRoot: cliSettings.workdir, + majorVersion: toml.majorVersion, + orioledbVersion: Option.getOrUndefined(toml.orioledbVersion), + services: pullLocalInputs.context.config.local?.services, + appliedRemote: toml.appliedRemote, + })).database.image + : (yield* resolveDbImage( + fs, + path, + cliSettings.workdir, + toml.majorVersion, + Option.getOrUndefined(toml.orioledbVersion), + )).image; // Default dump options: no schema filter (so the internal-schema exclude // list applies) and comments stripped. const dumpEnvOpt: DumpOptions = { @@ -584,6 +597,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( dnsResolver, password: flags.password ?? Option.none(), linkedProjectRef: flags.projectRef, + target: linkedTarget, }) .pipe(Effect.orElseSucceed(() => Option.none())), runWithConn: runSchemaDump, diff --git a/apps/cli/src/command-internal/db-push-core.ts b/apps/cli/src/command-internal/db-push-core.ts index 594a8eaa3a..d80305ffb5 100644 --- a/apps/cli/src/command-internal/db-push-core.ts +++ b/apps/cli/src/command-internal/db-push-core.ts @@ -43,7 +43,7 @@ const applyError = (message: string) => new DbPushApplyError({ message }); /** * Everything `db push` does once its target connection and config are already resolved. Callers * (`db push`, `bootstrap`) resolve the project ref, connection, and `config.toml` themselves and - * pass the results in; this core never touches `ProjectRefResolver`/`DbConfigResolver` itself. + * pass the results in; this core never touches `RemoteTarget`/`DbConfigResolver` itself. * * The "DRY RUN: …" heads-up line prints first here, after the caller's own connection-resolution * output (e.g. "Initialising login role..."), so it lives here rather than at either call site. diff --git a/apps/cli/src/command-internal/edge-runtime-image.ts b/apps/cli/src/command-internal/edge-runtime-image.ts index 2f1c29d222..b47205bd89 100644 --- a/apps/cli/src/command-internal/edge-runtime-image.ts +++ b/apps/cli/src/command-internal/edge-runtime-image.ts @@ -1,10 +1,17 @@ -import { Effect, type FileSystem, type Path } from "effect"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { usesStackState } from "./stack-backend.ts"; +import { Effect, type FileSystem, Option, type Path } from "effect"; import { DENO1_EDGE_RUNTIME_VERSION } from "../shared/functions/functions.shared.ts"; import { dockerfileServiceImage, dockerfileServiceImageRaw, } from "../shared/services/dockerfile-images.ts"; -import { slimImageForCurrentPin, slimImagesEnabled } from "../shared/services/slim-images.ts"; +import { + imageTag, + slimImageForCurrentPin, + slimImagesEnabled, +} from "../shared/services/slim-images.ts"; +import { readLockedPins } from "./stack-lock.ts"; /** * Resolves the edge-runtime Docker image: the default tag comes from the Dockerfile image, a @@ -21,13 +28,27 @@ export const edgeRuntimeDockerfileImage = Effect.map(slimImagesEnabled, (slim) = // applies for the functions Docker paths reading the same pin file. const EDGE_RUNTIME_DENO1_IMAGE = `supabase/edge-runtime:${DENO1_EDGE_RUNTIME_VERSION}`; +const imageForPin = Effect.fnUntraced(function* (pinned: string) { + if (pinned === DENO1_EDGE_RUNTIME_VERSION) { + return EDGE_RUNTIME_DENO1_IMAGE; + } + return slimImageForCurrentPin( + "edgeruntime", + dockerfileServiceImageRaw("edgeruntime"), + pinned.length > 0 ? pinned : undefined, + yield* slimImagesEnabled, + ); +}); + /** * Resolves the edge-runtime image, honoring the pinned tag in - * `supabase/.temp/edge-runtime-version` and the `deno_version` selector (default 2 → Dockerfile - * image; 1 → `deno1`). The version pin is applied first, then `deno_version = 1` overrides to - * `deno1`. Historical pins stay on docker.io — those slim tags are not published. + * `supabase/.temp/edge-runtime-version` (legacy state only; stack state uses + * {@link resolveLockedEdgeRuntimeImage}) and the `deno_version` selector + * (default 2 → Dockerfile image; 1 → `deno1`). The version pin is applied first, then + * `deno_version = 1` overrides to `deno1`. Historical pins stay on docker.io — those slim tags + * are not published. */ -export const resolveEdgeRuntimeImage = Effect.fnUntraced(function* ( +export const resolveEdgeRuntimeImage = Effect.fn("EdgeRuntimeImage.resolve")(function* ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, @@ -36,19 +57,36 @@ export const resolveEdgeRuntimeImage = Effect.fnUntraced(function* ( if (denoVersion === 1) { return EDGE_RUNTIME_DENO1_IMAGE; } - const raw = dockerfileServiceImageRaw("edgeruntime"); - const versionPath = path.join(workdir, "supabase", ".temp", "edge-runtime-version"); - const pinned = yield* fs.readFileString(versionPath).pipe( - Effect.map((s) => s.trim()), - Effect.orElseSucceed(() => ""), - ); - if (pinned === DENO1_EDGE_RUNTIME_VERSION) { + const pinned = (yield* usesStackState) + ? "" + : yield* fs.readFileString(legacyProjectStatePaths(path, workdir).edgeRuntimeVersion).pipe( + Effect.map((s) => s.trim()), + Effect.orElseSucceed(() => ""), + ); + return yield* imageForPin(pinned); +}); + +/** + * The edge-runtime image for tool containers in stack state mode: the release locked for + * Functions in `supabase/supabase.lock` (the pinned slim image when slim images are enabled, else + * its upstream image), else the bundled one. Reads only the lock, so a stale lock or an invalid + * config does not block it; an unreadable lock fails. + */ +export const resolveLockedEdgeRuntimeImage = Effect.fn("EdgeRuntimeImage.resolveLocked")(function* ( + workdir: string, + denoVersion: number, +) { + if (denoVersion === 1) { return EDGE_RUNTIME_DENO1_IMAGE; } - return slimImageForCurrentPin( - "edgeruntime", - raw, - pinned.length > 0 ? pinned : undefined, - yield* slimImagesEnabled, + const locked = yield* readLockedPins(workdir); + if (Option.isSome(locked) && (yield* slimImagesEnabled)) { + return locked.value.functions.image; + } + return yield* imageForPin( + Option.match(locked, { + onNone: () => "", + onSome: (selection) => imageTag(selection.functions.upstreamImage) ?? "", + }), ); }); diff --git a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts index 7b39451be5..572df0bea5 100644 --- a/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts +++ b/apps/cli/src/command-internal/edge-runtime-script.layer.integration.test.ts @@ -5,10 +5,15 @@ import { vi } from "vitest"; import { DebugFlag, NetworkIdFlag } from "./global-flags.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; +import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; import { DockerRun, type DockerRunOpts } from "./docker-run.service.ts"; import { edgeRuntimeScriptLayer } from "./edge-runtime-script.layer.ts"; import { EdgeRuntimeScript } from "./edge-runtime-script.service.ts"; +import { projectStateModeLayer } from "./project-state-mode.ts"; +import { mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; +import { seedLegacyStateDecoys } from "../../tests/helpers/legacy-state-guard.ts"; +import { seedLock } from "../../tests/helpers/stack-lock.ts"; // Fakes a `docker run --rm` capture. A real diff always exits non-zero with "main worker has // been destroyed" in stderr; the crash path only differs by the added sentinel line. @@ -175,6 +180,101 @@ describe("edgeRuntimeScriptLayer sentinel handling", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.effect( + "runs on the Functions release locked in supabase.lock in stack state mode, ignoring the legacy pin", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const workdir = yield* fs.makeTempDirectoryScoped({ prefix: "edge-runtime-locked-" }); + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(workdir, "supabase", "config.toml"), + 'project_id = "test-project"\n', + ); + // A stale lock fails the local stack commands; the tool container only reads the pin. + yield* seedLock(workdir, (services) => ({ + ...services, + rest: { ...services.rest, specifier: "v99.0.0" }, + functions: { + ...services.functions, + resolved: { + ...services.functions.resolved, + upstreamVersion: "v1.99.0", + revision: 1, + image: `ghcr.io/supabase/cli/edge-runtime:v1.99.0-r1@sha256:${"a".repeat(64)}`, + upstreamImage: "supabase/edge-runtime:v1.99.0", + }, + }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir, supabaseHome: workdir }), + ), + ), + ); + seedLegacyStateDecoys(workdir); + + const { layer, docker } = setup( + { exitCode: 1, stdout: "", stderr: "main worker has been destroyed\n" }, + { cliSettingsWorkdir: workdir }, + ); + yield* runScript().pipe(Effect.provide(Layer.merge(layer, projectStateModeLayer("stack")))); + + expect(docker.lastOpts?.image).toContain("edge-runtime:v1.99.0"); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + ); + + it.effect( + "runs on the locked slim image revision in stack state mode with the slim-images flag on", + () => + Effect.gen(function* () { + vi.stubEnv("SUPABASE_USE_SLIM_IMAGES", "1"); + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const workdir = yield* fs.makeTempDirectoryScoped({ prefix: "edge-runtime-locked-slim-" }); + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(workdir, "supabase", "config.toml"), + 'project_id = "test-project"\n', + ); + const lockedImage = `ghcr.io/supabase/cli/edge-runtime:v1.99.0-r1@sha256:${"a".repeat(64)}`; + yield* seedLock(workdir, (services) => ({ + ...services, + functions: { + ...services.functions, + resolved: { + ...services.functions.resolved, + upstreamVersion: "v1.99.0", + revision: 1, + image: lockedImage, + upstreamImage: "supabase/edge-runtime:v1.99.0", + }, + }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir, supabaseHome: workdir }), + ), + ), + ); + + const { layer, docker } = setup( + { exitCode: 1, stdout: "", stderr: "main worker has been destroyed\n" }, + { cliSettingsWorkdir: workdir }, + ); + yield* runScript().pipe(Effect.provide(Layer.merge(layer, projectStateModeLayer("stack")))); + + expect(docker.lastOpts?.image).toBe(lockedImage); + }).pipe( + Effect.scoped, + Effect.provide(BunServices.layer), + Effect.ensuring(Effect.sync(() => vi.unstubAllEnvs())), + ), + ); + it.effect("rewrites the runner onto the slim image with the slim-images flag on", () => { vi.stubEnv("SUPABASE_USE_SLIM_IMAGES", "1"); const { layer, docker } = setup({ diff --git a/apps/cli/src/command-internal/edge-runtime-script.layer.ts b/apps/cli/src/command-internal/edge-runtime-script.layer.ts index 7b3c39687a..94fb44e84d 100644 --- a/apps/cli/src/command-internal/edge-runtime-script.layer.ts +++ b/apps/cli/src/command-internal/edge-runtime-script.layer.ts @@ -7,7 +7,8 @@ import { CommandSettings } from "../config/command-settings.service.ts"; import { readDbToml } from "./db-config.toml-read.ts"; import { getRegistryImageUrl } from "./docker-registry.ts"; import { DockerRun } from "./docker-run.service.ts"; -import { resolveEdgeRuntimeImage } from "./edge-runtime-image.ts"; +import { resolveEdgeRuntimeImage, resolveLockedEdgeRuntimeImage } from "./edge-runtime-image.ts"; +import { usesStackState } from "./stack-backend.ts"; import { EdgeRuntimeScriptError } from "./edge-runtime-script.errors.ts"; import { EDGE_RUNTIME_SCRIPT_ERROR_SENTINEL, @@ -82,7 +83,15 @@ export const edgeRuntimeScriptLayer = Layer.effect( Effect.mapError((error) => new EdgeRuntimeScriptError({ message: error.message })), )).denoVersion; const registryImage = yield* getRegistryImageUrl( - yield* resolveEdgeRuntimeImage(fs, path, workdir, denoVersion), + (yield* usesStackState) + ? yield* resolveLockedEdgeRuntimeImage(workdir, denoVersion).pipe( + Effect.provideService(FileSystem.FileSystem, fs), + Effect.provideService(Path.Path, path), + Effect.mapError( + (error) => new EdgeRuntimeScriptError({ message: error.message }), + ), + ) + : yield* resolveEdgeRuntimeImage(fs, path, workdir, denoVersion), opts.projectEnvValues, ).pipe( Effect.mapError( diff --git a/apps/cli/src/command-internal/go-proxy-target-env.integration.test.ts b/apps/cli/src/command-internal/go-proxy-target-env.integration.test.ts new file mode 100644 index 0000000000..91125ffe75 --- /dev/null +++ b/apps/cli/src/command-internal/go-proxy-target-env.integration.test.ts @@ -0,0 +1,223 @@ +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; + +import { useTempWorkdir, VALID_TOKEN } from "../../tests/helpers/command-mocks.ts"; +import { isolatedHomeLayer } from "../../tests/helpers/command-mocks.ts"; +import { + mockAnalytics, + mockOutput, + mockTelemetryRuntime, + mockTty, +} from "../../tests/helpers/mocks.ts"; +import { + projectResponse, + provideEnv, + stubManagementApi, +} from "../../tests/helpers/permission-real-layers.ts"; +import { dbRemoteChanges } from "../commands/db/remote/changes/changes.handler.ts"; +import { genKeys } from "../commands/gen/keys/keys.handler.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../config/remote-target.service.ts"; +import { CliArgs } from "../shared/cli/cli-args.service.ts"; +import { + DebugFlag, + DnsResolverFlag, + OutputFlag, + ProfileFlag, + WorkdirFlag, +} from "./global-flags.ts"; +import { GoProxy } from "./go-proxy.service.ts"; +import { projectStateModeLayer, type ProjectStateMode } from "./project-state-mode.ts"; + +const LINKED_REF = "abcdefghijklmnopqrst"; + +const setup = ( + workdir: string, + mode: ProjectStateMode, + profile = "supabase", + interactive?: { readonly chosenRef: string }, +) => { + const calls: Array<{ readonly args: ReadonlyArray }> = []; + const poolerUrls: Array = []; + const layer = Layer.mergeAll( + BunServices.layer, + isolatedHomeLayer(workdir, { + SUPABASE_NO_KEYRING: "1", + ...(interactive === undefined ? {} : { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }), + }), + projectStateModeLayer(mode), + mockTty({ stdinIsTty: interactive !== undefined }), + mockOutput({ + promptSelectResponses: interactive === undefined ? [] : [interactive.chosenRef], + }).layer, + mockAnalytics().layer, + mockTelemetryRuntime(), + Layer.succeed(CliArgs, { args: [] }), + Layer.succeed(DebugFlag, false), + Layer.succeed(ProfileFlag, profile), + Layer.succeed(WorkdirFlag, Option.some(workdir)), + Layer.succeed(OutputFlag, Option.none()), + Layer.succeed(DnsResolverFlag, "native"), + Layer.succeed(GoProxy, { + exec: (args, execOpts) => + Effect.sync(() => { + calls.push({ args }); + poolerUrls.push(execOpts?.env?.SUPABASE_INTERNAL_POOLER_URL); + }), + execCapture: () => Effect.die("the delegated commands do not capture output"), + }), + ); + return { calls, poolerUrls, layer }; +}; + +const POOLER_URL = + "postgresql://postgres.abcdefghijklmnopqrst:[YOUR-PASSWORD]@pooler.example.com:6543/postgres"; + +const writeStackLink = Effect.fnUntraced(function* (workdir: string, poolerUrl?: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(workdir, "supabase", ".link.json"), + JSON.stringify({ + project: { ref: LINKED_REF }, + active_branch: { + ref: LINKED_REF, + ...(poolerUrl === undefined ? {} : { pooler_url: poolerUrl }), + }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }), + ); +}); + +const changesFlags = (overrides: { dbUrl?: Option.Option } = {}) => ({ + schema: [], + dbUrl: overrides.dbUrl ?? Option.none(), + linked: false, + password: Option.none(), +}); + +describe("Go-delegated linked target", () => { + const tmp = useTempWorkdir("supabase-go-target-env-"); + const env = { SUPABASE_NO_KEYRING: "1", SUPABASE_HOME: undefined }; + + it.effect("passes the linked ref to `db remote changes` as --project-ref in stack mode", () => { + const { calls, layer } = setup(tmp.current, "stack"); + return Effect.gen(function* () { + yield* writeStackLink(tmp.current); + yield* dbRemoteChanges(changesFlags()); + expect(calls).toEqual([{ args: ["db", "remote", "changes", "--project-ref", LINKED_REF] }]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("passes no ref when `db remote changes` has its own --db-url", () => { + const { calls, layer } = setup(tmp.current, "stack"); + return Effect.gen(function* () { + yield* writeStackLink(tmp.current); + yield* dbRemoteChanges(changesFlags({ dbUrl: Option.some("postgres://x@localhost/db") })); + expect(calls).toEqual([ + { + args: ["db", "remote", "changes", "--db-url", "postgres://x@localhost/db"], + }, + ]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("adds no ref and builds no target runtime in legacy mode", () => { + // An unknown profile fails the settings layer, so a legacy run that succeeds never built it. + const { calls, layer } = setup(tmp.current, "legacy", "no-such-profile"); + return Effect.gen(function* () { + yield* dbRemoteChanges(changesFlags()); + yield* genKeys({ projectRef: Option.none(), overrideName: [] }); + expect(calls).toEqual([{ args: ["db", "remote", "changes"] }, { args: ["gen", "keys"] }]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("passes the linked ref to `gen keys` as --project-ref in stack mode", () => { + const { calls, layer } = setup(tmp.current, "stack"); + return Effect.gen(function* () { + yield* writeStackLink(tmp.current); + yield* genKeys({ projectRef: Option.none(), overrideName: [] }); + expect(calls).toEqual([{ args: ["gen", "keys", "--project-ref", LINKED_REF] }]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("prompts for a project in `gen keys` when stack mode has nothing linked", () => { + const { calls, layer } = setup(tmp.current, "stack", "supabase", { chosenRef: LINKED_REF }); + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, (method, path) => + method === "GET" && path === "/v1/projects" + ? Response.json([projectResponse(LINKED_REF)]) + : undefined, + ); + yield* genKeys({ projectRef: Option.none(), overrideName: [] }); + expect(requests).toEqual([{ method: "GET", path: "/v1/projects" }]); + expect(calls).toEqual([{ args: ["gen", "keys", "--project-ref", LINKED_REF] }]); + }).pipe(Effect.scoped, Effect.provide(layer), provideEnv(env)); + }); + + it.effect("keeps the link ahead of the picker in `gen keys`", () => { + const { calls, layer } = setup(tmp.current, "stack", "supabase", { + chosenRef: "zyxwvutsrqponmlkjihg", + }); + return Effect.gen(function* () { + yield* writeStackLink(tmp.current); + yield* genKeys({ projectRef: Option.none(), overrideName: [] }); + expect(calls).toEqual([{ args: ["gen", "keys", "--project-ref", LINKED_REF] }]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect( + "fails `db remote changes` without prompting when stack mode has nothing linked", + () => { + const { calls, layer } = setup(tmp.current, "stack", "supabase", { chosenRef: LINKED_REF }); + const requests: Array<{ readonly method: string; readonly path: string }> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, () => undefined); + const error = yield* dbRemoteChanges(changesFlags()).pipe(Effect.flip); + expect(error).toMatchObject({ + _tag: "ProjectRefNotLinkedError", + message: PROJECT_NOT_LINKED_MESSAGE, + }); + expect(requests).toEqual([]); + expect(calls).toEqual([]); + }).pipe(Effect.scoped, Effect.provide(layer), provideEnv(env)); + }, + ); + + it.effect("passes an explicit --project-ref to `gen keys` in stack mode", () => { + const { calls, layer } = setup(tmp.current, "stack"); + const explicitRef = "zyxwvutsrqponmlkjihg"; + return Effect.gen(function* () { + yield* writeStackLink(tmp.current); + yield* genKeys({ projectRef: Option.some(explicitRef), overrideName: [] }); + expect(calls).toEqual([{ args: ["gen", "keys", "--project-ref", explicitRef] }]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("forwards the linked pooler URL with the linked ref", () => { + const { calls, poolerUrls, layer } = setup(tmp.current, "stack"); + return Effect.gen(function* () { + yield* writeStackLink(tmp.current, POOLER_URL); + yield* dbRemoteChanges(changesFlags()); + yield* genKeys({ projectRef: Option.some(LINKED_REF), overrideName: [] }); + expect(calls.map((call) => call.args.slice(-2))).toEqual([ + ["--project-ref", LINKED_REF], + ["--project-ref", LINKED_REF], + ]); + expect(poolerUrls).toEqual([POOLER_URL, POOLER_URL]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); + + it.effect("forwards no pooler URL when an explicit --project-ref is not the linked ref", () => { + const { calls, poolerUrls, layer } = setup(tmp.current, "stack"); + const explicitRef = "zyxwvutsrqponmlkjihg"; + return Effect.gen(function* () { + yield* writeStackLink(tmp.current, POOLER_URL); + yield* genKeys({ projectRef: Option.some(explicitRef), overrideName: [] }); + expect(calls).toEqual([{ args: ["gen", "keys", "--project-ref", explicitRef] }]); + expect(poolerUrls).toEqual([undefined]); + }).pipe(Effect.provide(layer), provideEnv(env)); + }); +}); diff --git a/apps/cli/src/command-internal/go-proxy-target-env.ts b/apps/cli/src/command-internal/go-proxy-target-env.ts new file mode 100644 index 0000000000..a7ee04a76f --- /dev/null +++ b/apps/cli/src/command-internal/go-proxy-target-env.ts @@ -0,0 +1,39 @@ +import { Effect, Option } from "effect"; + +import { RemoteTarget } from "../config/remote-target.service.ts"; +import { remoteTargetRuntimeLayer } from "./management-api-runtime.layer.ts"; +import { softProjectStateMode } from "./project-state-mode.ts"; + +const loadTarget = (projectRef: Option.Option, picker: boolean) => + Effect.gen(function* () { + const remoteTarget = yield* RemoteTarget; + const target = yield* picker ? remoteTarget.resolve(projectRef) : remoteTarget.load(projectRef); + yield* Effect.annotateCurrentSpan({ + "remote_target.source": target.source, + "project.ref": target.ref, + }); + return target; + }); + +/** + * The `--project-ref` argument and env for a Go-delegated command that accepts one: stack mode + * resolves the target in TypeScript and passes its ref explicitly, with the linked pooler URL + * only when the ref is the linked one; legacy mode forwards only the flag the user typed and + * never builds the target runtime. With `picker`, a TTY run with nothing linked prompts for a + * project instead of failing. + */ +export const goProjectTarget = Effect.fn("GoProxy.projectTarget")(function* ( + projectRef: Option.Option, + options: { readonly picker?: boolean } = {}, +) { + if ((yield* softProjectStateMode) !== "stack") { + const env: Record = {}; + return { args: Option.isSome(projectRef) ? ["--project-ref", projectRef.value] : [], env }; + } + const target = yield* loadTarget(projectRef, options.picker === true).pipe( + Effect.provide(remoteTargetRuntimeLayer), + ); + const env: Record = {}; + if (target.poolerUrl !== undefined) env["SUPABASE_INTERNAL_POOLER_URL"] = target.poolerUrl; + return { args: ["--project-ref", target.ref], env }; +}); diff --git a/apps/cli/src/command-internal/go-proxy.layer.ts b/apps/cli/src/command-internal/go-proxy.layer.ts index 8b679e649e..2bcd265e60 100644 --- a/apps/cli/src/command-internal/go-proxy.layer.ts +++ b/apps/cli/src/command-internal/go-proxy.layer.ts @@ -3,14 +3,18 @@ import { createRequire } from "node:module"; import os from "node:os"; import path from "node:path"; import process from "node:process"; -import { Effect, Layer, Option, Stream } from "effect"; +import { Config, Effect, FileSystem, Layer, Option, Path, Stream } from "effect"; import * as ChildProcess from "effect/process/ChildProcess"; import { ChildProcessSpawner } from "effect/process/ChildProcessSpawner"; +import { resolveWorkdir } from "../config/command-settings.layer.ts"; import { CLI_VERSION } from "../shared/cli/version.ts"; import { ProcessControl } from "../shared/runtime/process-control.service.ts"; +import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { GoChildExitError } from "./go-child-exit.error.ts"; import { GoProxyInvocation } from "./go-proxy-invocation.ts"; import { GoProxy } from "./go-proxy.service.ts"; +import { readProjectLinkFile } from "./project-link.ts"; +import { usesStackState } from "./stack-backend.ts"; import { withProcessSpan } from "../shared/telemetry/spans.ts"; const markDelegated = Effect.serviceOption(GoProxyInvocation).pipe( @@ -19,6 +23,35 @@ const markDelegated = Effect.serviceOption(GoProxyInvocation).pipe( ), ); +/** + * Tells the Go child to leave `supabase/.temp` alone when the project keeps its state in + * `$SUPABASE_HOME`, and hands it the linked project's identity as the JSON of its + * `linked-project.json` record so it can still attribute telemetry. Read from the calling fiber, + * like `markDelegated`. + */ +const makeStackStateEnv = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: Effect.Effect, +): Effect.Effect> => + Effect.gen(function* (): Effect.fn.Return> { + if (!(yield* usesStackState)) return {}; + const link = yield* readProjectLinkFile(fs, path, yield* workdir).pipe( + Effect.orElseSucceed(() => Option.none()), + ); + if (Option.isNone(link)) return { SUPABASE_INTERNAL_STACK_STATE: "1" }; + const { project } = link.value; + return { + SUPABASE_INTERNAL_STACK_STATE: "1", + SUPABASE_INTERNAL_LINKED_PROJECT: JSON.stringify({ + ref: project.ref, + name: project.name ?? "", + organization_id: project.organization_id ?? "", + organization_slug: project.organization_slug ?? "", + }), + }; + }); + const PLATFORM_CANDIDATES: Partial>>>> = { darwin: { arm64: ["darwin-arm64"], x64: ["darwin-x64"] }, @@ -134,6 +167,10 @@ export function makeGoProxyLayer(opts?: { */ env?: Record; globalArgs?: ReadonlyArray; + /** + * The `--workdir` the user passed, which locates the project link handed to stack-mode children. + */ + workdir?: Option.Option; /** * Let the parent emit its success tail after re-emitting captured stdout. */ @@ -149,17 +186,38 @@ export function makeGoProxyLayer(opts?: { * artifact for the host platform. */ binary?: string | BinaryResolution; -}): Layer.Layer { +}): Layer.Layer< + GoProxy, + never, + ProcessControl | ChildProcessSpawner | FileSystem.FileSystem | Path.Path | RuntimeInfo +> { return Layer.effect( GoProxy, Effect.gen(function* () { const processControl = yield* ProcessControl; const spawner = yield* ChildProcessSpawner; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const runtimeInfo = yield* RuntimeInfo; const resolved: BinaryResolution = typeof opts?.binary === "string" ? { found: opts.binary } : (opts?.binary ?? resolveBinary()); const globalArgs = opts?.globalArgs ?? []; + const projectWorkdir = Effect.gen(function* () { + const envValue = yield* Config.option(Config.String("SUPABASE_WORKDIR")).pipe( + Effect.orElseSucceed(() => Option.none()), + ); + const { workdir } = yield* resolveWorkdir( + opts?.workdir ?? Option.none(), + envValue, + opts?.cwd ?? runtimeInfo.cwd, + (candidate) => fs.exists(candidate).pipe(Effect.orElseSucceed(() => false)), + path, + ); + return workdir; + }); + const stackStateEnv = makeStackStateEnv(fs, path, projectWorkdir); return GoProxy.of({ exec: (args, execOpts) => @@ -193,6 +251,7 @@ export function makeGoProxyLayer(opts?: { ...(execOpts?.suppressChildTelemetry === true ? { SUPABASE_TELEMETRY_DISABLED: "1" } : {}), + ...(yield* stackStateEnv), }; const argv = [...globalArgs, ...args]; const exitCode = yield* withProcessSpan( @@ -251,6 +310,7 @@ export function makeGoProxyLayer(opts?: { ...(opts?.parentOwnsCapturedSuccessTail === true ? { SUPABASE_NO_UPDATE_NOTIFIER: "1" } : {}), + ...(yield* stackStateEnv), }; // Capture stdout while keeping stderr inherited, so progress still reaches the user // while stdout is collected for wrapping. Callers pass stdin: "ignore" to give the diff --git a/apps/cli/src/command-internal/go-proxy.layer.unit.test.ts b/apps/cli/src/command-internal/go-proxy.layer.unit.test.ts index 552e8a8cf2..61bfd1cdad 100644 --- a/apps/cli/src/command-internal/go-proxy.layer.unit.test.ts +++ b/apps/cli/src/command-internal/go-proxy.layer.unit.test.ts @@ -1,13 +1,29 @@ import { describe, expect, it, vi } from "@effect/vitest"; -import { Cause, Deferred, Effect, Exit, Fiber, Layer, Sink, Stream } from "effect"; +import { BunServices } from "@effect/platform-bun"; +import { + Cause, + Deferred, + Effect, + Exit, + Fiber, + FileSystem, + Layer, + Path, + Sink, + Stream, +} from "effect"; import { ChildProcessSpawner } from "effect/process"; import { type CliProcessSignal, ProcessControl, } from "../shared/runtime/process-control.service.ts"; +import { mockRuntimeInfo } from "../../tests/helpers/mocks.ts"; +import { useTempWorkdir } from "../../tests/helpers/command-mocks.ts"; import { GoChildExitError } from "./go-child-exit.error.ts"; import { GoProxyInvocation, goProxyInvocationLayer } from "./go-proxy-invocation.ts"; import { GoProxy } from "./go-proxy.service.ts"; +import { projectStateModeLayer } from "./project-state-mode.ts"; +import { stackBackendLayer } from "./stack-backend.ts"; import { formatGoBinaryNotFoundError, makeGoProxyLayer } from "./go-proxy.layer.ts"; /** @@ -229,13 +245,20 @@ describe("formatGoBinaryNotFoundError - pinned snippet", () => { }); }); +const proxyDeps = ( + spawner: Layer.Layer, + processControl: Layer.Layer, +) => Layer.mergeAll(BunServices.layer, mockRuntimeInfo(), spawner, processControl); + describe("makeGoProxyLayer", () => { + const tmp = useTempWorkdir("supabase-go-proxy-"); + it.effect("records delegated execution for the parent success trailer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = Layer.mergeAll( makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ), goProxyInvocationLayer, ); @@ -257,7 +280,7 @@ describe("makeGoProxyLayer", () => { binary: TEST_BINARY, env: { SUPABASE_NO_UPDATE_NOTIFIER: "0" }, parentOwnsCapturedSuccessTail: true, - }).pipe(Layer.provide(Layer.mergeAll(spawner.layer, pc.layer))), + }).pipe(Layer.provide(proxyDeps(spawner.layer, pc.layer))), goProxyInvocationLayer, ); return Effect.gen(function* () { @@ -277,7 +300,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY, globalArgs: ["--debug"] }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -299,7 +322,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -316,7 +339,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -333,13 +356,91 @@ describe("makeGoProxyLayer", () => { }).pipe(Effect.provide(layer)); }); + it.effect( + "tells children to leave supabase/.temp alone and who is linked in stack mode only", + () => { + const spawner = mockSpawner({ kind: "success", code: 0 }); + const pc = mockProcessControl(); + const layer = makeGoProxyLayer({ binary: TEST_BINARY, cwd: tmp.current }).pipe( + Layer.provide(proxyDeps(spawner.layer, pc.layer)), + ); + const run = (modeLayer: Layer.Layer | undefined) => + Effect.gen(function* () { + const proxy = yield* GoProxy; + yield* proxy.exec(["db", "remote", "changes"]); + yield* proxy.execCapture(["db", "diff"]); + }).pipe(Effect.provide(modeLayer === undefined ? layer : Layer.merge(layer, modeLayer))); + const childEnv = () => + spawner.spawned.splice(0).map((captured) => ({ + stackState: captured.options.env?.SUPABASE_INTERNAL_STACK_STATE, + linked: captured.options.env?.SUPABASE_INTERNAL_LINKED_PROJECT, + pooler: captured.options.env?.SUPABASE_INTERNAL_POOLER_URL, + })); + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const unlinked = { stackState: "1", linked: undefined, pooler: undefined }; + const absent = { stackState: undefined, linked: undefined, pooler: undefined }; + + yield* run(projectStateModeLayer("stack")); + expect(childEnv()).toEqual([unlinked, unlinked]); + + yield* fs.makeDirectory(path.join(tmp.current, "supabase"), { recursive: true }); + const writeLink = (poolerUrl?: string) => + fs.writeFileString( + path.join(tmp.current, "supabase", ".link.json"), + JSON.stringify({ + project: { + ref: "abcdefghijklmnopqrst", + name: "Acme", + organization_id: "org_123", + organization_slug: "acme", + }, + active_branch: { + ref: "tsrqponmlkjihgfedcba", + name: "feature-x", + is_default: false, + ...(poolerUrl === undefined ? {} : { pooler_url: poolerUrl }), + }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }), + ); + const linked = { + stackState: "1", + linked: JSON.stringify({ + ref: "abcdefghijklmnopqrst", + name: "Acme", + organization_id: "org_123", + organization_slug: "acme", + }), + pooler: undefined, + }; + yield* writeLink(); + yield* run(projectStateModeLayer("stack")); + yield* run(stackBackendLayer("stack")); + expect(childEnv()).toEqual([linked, linked, linked, linked]); + + const poolerUrl = + "postgresql://postgres.tsrqponmlkjihgfedcba:[YOUR-PASSWORD]@pooler.example.com:6543/postgres"; + yield* writeLink(poolerUrl); + yield* run(projectStateModeLayer("stack")); + expect(childEnv()).toEqual([linked, linked]); + + yield* run(projectStateModeLayer("legacy")); + yield* run(undefined); + expect(childEnv()).toEqual([absent, absent, absent, absent]); + }).pipe(Effect.provide(BunServices.layer)); + }, + ); + it.effect("passes the layer's env to children, letting per-call env win", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY, env: { SUPABASE_NO_UPDATE_NOTIFIER: "1" }, - }).pipe(Layer.provide(Layer.mergeAll(spawner.layer, pc.layer))); + }).pipe(Layer.provide(proxyDeps(spawner.layer, pc.layer))); return Effect.gen(function* () { const proxy = yield* GoProxy; yield* proxy.exec(["projects", "list"]); @@ -356,7 +457,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 7 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -375,7 +476,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 5 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); let finalizerRan = false; return Effect.gen(function* () { @@ -396,7 +497,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -409,7 +510,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -428,7 +529,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -445,7 +546,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "fail", error: "spawn failed" }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -461,7 +562,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "never" }, spawned); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -492,7 +593,7 @@ describe("makeGoProxyLayer", () => { "/usr/local/bin/supabase-go (not found alongside the shim)", ]; const layer = makeGoProxyLayer({ binary: { notFound: tried } }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -525,7 +626,7 @@ describe("makeGoProxyLayer", () => { const stderr = vi.spyOn(process.stderr, "write").mockImplementation(() => true); const tried = ["$SUPABASE_GO_BINARY (unset)"]; const layer = makeGoProxyLayer({ binary: { notFound: tried } }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; @@ -549,7 +650,7 @@ describe("makeGoProxyLayer", () => { const spawner = mockSpawner({ kind: "success", code: 0 }); const pc = mockProcessControl(); const layer = makeGoProxyLayer({ binary: TEST_BINARY }).pipe( - Layer.provide(Layer.mergeAll(spawner.layer, pc.layer)), + Layer.provide(proxyDeps(spawner.layer, pc.layer)), ); return Effect.gen(function* () { const proxy = yield* GoProxy; diff --git a/apps/cli/src/command-internal/hosted-versions.ts b/apps/cli/src/command-internal/hosted-versions.ts new file mode 100644 index 0000000000..f53a813f91 --- /dev/null +++ b/apps/cli/src/command-internal/hosted-versions.ts @@ -0,0 +1,94 @@ +import type { ApiClient } from "@supabase/api/effect"; +import { Effect, Option } from "effect"; +import * as HttpClientError from "effect/http/HttpClientError"; + +import { CommandSettings } from "../config/command-settings.service.ts"; +import type { ProjectLink } from "./project-link.ts"; +import type { HostedProbe } from "./stack-lock.ts"; +import { extractServiceKeys } from "./tenant-keys.ts"; +import { probeFailure, probeTenantVersions, type HostedObservation } from "./tenant-versions.ts"; + +/** The version the platform reports for the database; an empty string means it reports none. */ +const databaseObservation = (version: string): HostedObservation => { + const raw = version.trim(); + return raw.length === 0 ? { _tag: "Unreported" } : { _tag: "Observed", raw, version: raw }; +}; + +/** + * The remote a link document names, as the lock records it: the project and the branch ref, so the + * same branch records the same remote however it was linked. A document that does not say whether + * its branch is the default one is the default branch when its ref is the project's. + */ +export const linkRemote = (link: ProjectLink): HostedProbe["remote"] => { + const { active_branch: branch, project } = link; + return { + project: project.ref, + branch: { ref: branch.ref, isDefault: branch.is_default ?? branch.ref === project.ref }, + }; +}; + +/** + * Observes the hosted versions of `ref`: the database from the project, or from the branch's own + * configuration when `ref` is a branch, and Auth, REST and Storage from the tenant endpoints. A + * request that fails is reported as such, never as an absent service. + */ +export const observeHostedVersions = Effect.fn("link.probe")(function* (input: { + readonly api: ApiClient; + readonly ref: string; + readonly serviceKey: string; + /** The version `getProject` reported; absent when `ref` is a branch. */ + readonly databaseVersion: Option.Option; +}) { + const settings = yield* CommandSettings; + const [database, tenant] = yield* Effect.all( + [ + Option.isSome(input.databaseVersion) + ? Effect.succeed(databaseObservation(input.databaseVersion.value)) + : input.api.v1.getABranchConfig({ branch_id_or_ref: input.ref }).pipe( + Effect.map((config) => databaseObservation(config.postgres_version)), + Effect.catch((error) => Effect.succeed(probeFailure(error))), + ), + probeTenantVersions({ + ref: input.ref, + projectHost: settings.projectHost, + serviceKey: input.serviceKey, + userAgent: settings.userAgent, + }), + ], + { concurrency: 2 }, + ); + const observations: HostedProbe["observations"] = { database, ...tenant }; + const found = Object.values(observations); + yield* Effect.annotateCurrentSpan({ + "link.observed_count": found.filter(({ _tag }) => _tag === "Observed").length, + "link.unreported_count": found.filter(({ _tag }) => _tag === "Unreported").length, + "link.failed_count": found.filter(({ _tag }) => _tag === "Failed").length, + }); + return observations; +}); + +const isNotFound = (error: unknown) => + HttpClientError.isHttpClientError(error) && error.response?.status === 404; + +/** + * Probes the hosted target of a link document for `stack update`. Fails when the project or its + * keys cannot be read; a version that cannot be read becomes a failed observation. + */ +export const probeLinkedTarget = Effect.fn("update.probe")(function* ( + api: ApiClient, + link: ProjectLink, +) { + const ref = link.active_branch.ref; + const project = yield* api.v1.getProject({ ref }).pipe( + Effect.asSome, + Effect.catchIf(isNotFound, () => Effect.succeedNone), + ); + const keys = yield* api.v1.getProjectApiKeys({ ref, reveal: true }); + const observations = yield* observeHostedVersions({ + api, + ref, + serviceKey: extractServiceKeys(keys).serviceRole, + databaseVersion: Option.map(project, (found) => found.database.version), + }); + return { remote: linkRemote(link), observations } satisfies HostedProbe; +}); diff --git a/apps/cli/src/command-internal/legacy-project-state.layer.ts b/apps/cli/src/command-internal/legacy-project-state.layer.ts new file mode 100644 index 0000000000..d0063565bb --- /dev/null +++ b/apps/cli/src/command-internal/legacy-project-state.layer.ts @@ -0,0 +1,88 @@ +import { Effect, FileSystem, Layer, Option, Path } from "effect"; + +import { type LegacyStateFile, legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { LegacyProjectState, ProjectRefReadError } from "./legacy-project-state.service.ts"; + +/** + * Reads the linked project ref, `None` when the file is absent or blank. Any other read error + * (a directory, permission denied) fails instead of reading as unlinked. + */ +export const readLegacyProjectRef = Effect.fn("LegacyProjectState.readProjectRef")(function* ( + fs: FileSystem.FileSystem, + path: Pick, + workdir: string, +): Effect.fn.Return, ProjectRefReadError> { + const content = yield* fs.readFileString(legacyProjectStatePaths(path, workdir).projectRef).pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.succeed("") + : Effect.fail( + new ProjectRefReadError({ + message: `failed to load project ref: ${error.message}`, + }), + ), + ), + ); + const trimmed = content.trim(); + return trimmed.length === 0 ? Option.none() : Option.some(trimmed); +}); + +const makeLegacyProjectState = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const paths = (workdir: string) => legacyProjectStatePaths(path, workdir); + + const exists = Effect.fn("LegacyProjectState.exists")(function* ( + workdir: string, + file: LegacyStateFile, + ) { + yield* Effect.annotateCurrentSpan("legacy_state.file", file); + return yield* fs.exists(paths(workdir)[file]); + }); + + const read = Effect.fn("LegacyProjectState.read")(function* ( + workdir: string, + file: LegacyStateFile, + ) { + yield* Effect.annotateCurrentSpan("legacy_state.file", file); + return yield* fs.readFileString(paths(workdir)[file]); + }); + + const write = Effect.fn("LegacyProjectState.write")(function* ( + workdir: string, + file: LegacyStateFile, + content: string, + ) { + yield* Effect.annotateCurrentSpan("legacy_state.file", file); + const filePath = paths(workdir)[file]; + yield* fs.makeDirectory(path.dirname(filePath), { recursive: true }); + yield* fs.writeFileString(filePath, content); + }); + + const remove = Effect.fn("LegacyProjectState.remove")(function* ( + workdir: string, + file: LegacyStateFile, + ) { + yield* Effect.annotateCurrentSpan("legacy_state.file", file); + yield* fs.remove(paths(workdir)[file], { recursive: true, force: true }); + }); + + const removeAll = Effect.fn("LegacyProjectState.removeAll")(function* (workdir: string) { + yield* fs.remove(paths(workdir).tempDir, { recursive: true, force: true }); + }); + + const readProjectRef = (workdir: string) => readLegacyProjectRef(fs, path, workdir); + + return LegacyProjectState.of({ + paths, + exists, + read, + write, + remove, + removeAll, + readProjectRef, + }); +}); + +/** Needs only `FileSystem` and `Path`, so the root provides it to every command. */ +export const legacyProjectStateLayer = Layer.effect(LegacyProjectState, makeLegacyProjectState); diff --git a/apps/cli/src/command-internal/legacy-project-state.paths.ts b/apps/cli/src/command-internal/legacy-project-state.paths.ts new file mode 100644 index 0000000000..5d31152878 --- /dev/null +++ b/apps/cli/src/command-internal/legacy-project-state.paths.ts @@ -0,0 +1,58 @@ +import type { Path } from "effect"; + +/** Directory under `/supabase` that holds legacy link and cache state. */ +export const LEGACY_PROJECT_STATE_DIRNAME = ".temp"; + +/** A single-value file in the legacy project state directory. */ +export type LegacyStateFile = + | "projectRef" + | "poolerUrl" + | "postgresVersion" + | "edgeRuntimeVersion" + | "gotrueVersion" + | "restVersion" + | "storageVersion" + | "realtimeVersion" + | "studioVersion" + | "pgmetaVersion" + | "logflareVersion" + | "poolerVersion" + | "storageMigration" + | "linkedProjectCache" + | "cliLatest"; + +/** Absolute locations of everything the CLI keeps under `/supabase/.temp`. */ +export interface LegacyProjectStatePaths extends Readonly> { + readonly tempDir: string; + readonly pgdeltaDir: string; + readonly startSecretsDir: string; +} + +/** The only definition of the legacy `supabase/.temp` layout. Pure, so it runs before any layer. */ +export function legacyProjectStatePaths( + path: Pick, + workdir: string, +): LegacyProjectStatePaths { + const tempDir = path.join(workdir, "supabase", LEGACY_PROJECT_STATE_DIRNAME); + const inTemp = (name: string) => path.join(tempDir, name); + return { + tempDir, + projectRef: inTemp("project-ref"), + poolerUrl: inTemp("pooler-url"), + postgresVersion: inTemp("postgres-version"), + edgeRuntimeVersion: inTemp("edge-runtime-version"), + gotrueVersion: inTemp("gotrue-version"), + restVersion: inTemp("rest-version"), + storageVersion: inTemp("storage-version"), + realtimeVersion: inTemp("realtime-version"), + studioVersion: inTemp("studio-version"), + pgmetaVersion: inTemp("pgmeta-version"), + logflareVersion: inTemp("logflare-version"), + poolerVersion: inTemp("pooler-version"), + storageMigration: inTemp("storage-migration"), + linkedProjectCache: inTemp("linked-project.json"), + cliLatest: inTemp("cli-latest"), + pgdeltaDir: inTemp("pgdelta"), + startSecretsDir: inTemp("start-secrets"), + }; +} diff --git a/apps/cli/src/command-internal/legacy-project-state.service.ts b/apps/cli/src/command-internal/legacy-project-state.service.ts new file mode 100644 index 0000000000..232992d2f2 --- /dev/null +++ b/apps/cli/src/command-internal/legacy-project-state.service.ts @@ -0,0 +1,55 @@ +import { Context, Data, type Effect, type Option } from "effect"; +import type { PlatformError } from "effect/PlatformError"; + +import { + actionability, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../shared/telemetry/error-actionability.ts"; +import type { LegacyProjectStatePaths, LegacyStateFile } from "./legacy-project-state.paths.ts"; + +/** + * A real failure reading `/supabase/.temp/project-ref` (e.g. the + * path is a directory, or permissions deny access) — distinct from the file + * simply not existing, which means "not linked" rather than an error. + */ +export class ProjectRefReadError extends Data.TaggedError("ProjectRefReadError")<{ + readonly message: string; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} + +interface LegacyProjectStateShape { + /** Locations to hand to child processes or to I/O that needs its own error mapping. */ + readonly paths: (workdir: string) => LegacyProjectStatePaths; + readonly exists: ( + workdir: string, + file: LegacyStateFile, + ) => Effect.Effect; + readonly read: (workdir: string, file: LegacyStateFile) => Effect.Effect; + /** Writes `content`, creating the state directory first. */ + readonly write: ( + workdir: string, + file: LegacyStateFile, + content: string, + ) => Effect.Effect; + /** Removes `file`; an absent file is not an error. */ + readonly remove: (workdir: string, file: LegacyStateFile) => Effect.Effect; + /** Removes the whole state directory; an absent directory is not an error. */ + readonly removeAll: (workdir: string) => Effect.Effect; + /** + * The linked project ref, `None` when the file is absent or blank. Any other read error + * fails instead of being swallowed into an unlinked result. + */ + readonly readProjectRef: ( + workdir: string, + ) => Effect.Effect, ProjectRefReadError>; +} + +/** Owns reads and writes of the legacy `supabase/.temp` link and cache state. */ +export class LegacyProjectState extends Context.Service< + LegacyProjectState, + LegacyProjectStateShape +>()("supabase/cli/LegacyProjectState") {} diff --git a/apps/cli/src/command-internal/temp-paths.unit.test.ts b/apps/cli/src/command-internal/legacy-project-state.unit.test.ts similarity index 63% rename from apps/cli/src/command-internal/temp-paths.unit.test.ts rename to apps/cli/src/command-internal/legacy-project-state.unit.test.ts index 7b2bc740d6..de9281a443 100644 --- a/apps/cli/src/command-internal/temp-paths.unit.test.ts +++ b/apps/cli/src/command-internal/legacy-project-state.unit.test.ts @@ -1,26 +1,29 @@ import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; import { classifyCliErrorActionability } from "../shared/telemetry/error-actionability.ts"; -import { ProjectRefReadError, readProjectRefFile, tempPaths } from "./temp-paths.ts"; +import { legacyProjectStateLayer } from "./legacy-project-state.layer.ts"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { LegacyProjectState, ProjectRefReadError } from "./legacy-project-state.service.ts"; + +const services = legacyProjectStateLayer.pipe(Layer.provideMerge(BunServices.layer)); const readRef = (workdir: string) => Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - return yield* readProjectRefFile(fs, path, workdir); + const state = yield* LegacyProjectState; + return yield* state.readProjectRef(workdir); }); const REF = "abcdefghijklmnopqrst"; -describe("tempPaths", () => { +describe("legacyProjectStatePaths", () => { it.effect("maps a workdir to the supabase/.temp/* layout", () => Effect.gen(function* () { const path = yield* Path.Path; const workdir = path.join(path.sep, "home", "user", "project"); const tempDir = path.join(workdir, "supabase", ".temp"); - const paths = tempPaths(path, workdir); + const paths = legacyProjectStatePaths(path, workdir); expect(paths.tempDir).toBe(tempDir); expect(paths.projectRef).toBe(path.join(tempDir, "project-ref")); @@ -31,13 +34,16 @@ describe("tempPaths", () => { expect(paths.storageVersion).toBe(path.join(tempDir, "storage-version")); expect(paths.storageMigration).toBe(path.join(tempDir, "storage-migration")); expect(paths.linkedProjectCache).toBe(path.join(tempDir, "linked-project.json")); + expect(paths.cliLatest).toBe(path.join(tempDir, "cli-latest")); + expect(paths.pgdeltaDir).toBe(path.join(tempDir, "pgdelta")); + expect(paths.startSecretsDir).toBe(path.join(tempDir, "start-secrets")); }).pipe(Effect.provide(BunServices.layer)), ); it.effect("every temp path is nested under tempDir", () => Effect.gen(function* () { const path = yield* Path.Path; - const paths = tempPaths(path, "/tmp/wd"); + const paths = legacyProjectStatePaths(path, "/tmp/wd"); const { tempDir, ...rest } = paths; for (const value of Object.values(rest)) { expect(path.dirname(value)).toBe(tempDir); @@ -46,7 +52,28 @@ describe("tempPaths", () => { ); }); -describe("readProjectRefFile", () => { +describe("LegacyProjectState file access", () => { + it.effect("writes a state file, reads it back, and removes the directory", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const state = yield* LegacyProjectState; + const workdir = yield* fs.makeTempDirectoryScoped({ prefix: "legacy-state-" }); + + yield* state.write(workdir, "postgresVersion", "17.0.0"); + expect(yield* state.read(workdir, "postgresVersion")).toBe("17.0.0"); + expect(yield* state.exists(workdir, "postgresVersion")).toBe(true); + + yield* state.remove(workdir, "postgresVersion"); + expect(yield* state.exists(workdir, "postgresVersion")).toBe(false); + + yield* state.write(workdir, "restVersion", "v1"); + yield* state.removeAll(workdir); + expect(yield* fs.exists(state.paths(workdir).tempDir)).toBe(false); + }).pipe(Effect.scoped, Effect.provide(services)), + ); +}); + +describe("LegacyProjectState.readProjectRef", () => { const workdir = Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -59,7 +86,7 @@ describe("readProjectRefFile", () => { Effect.gen(function* () { const { dir } = yield* workdir; expect(Option.isNone(yield* readRef(dir))).toBe(true); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide(services)), ); it.effect("returns the trimmed ref when the file holds a value", () => @@ -68,7 +95,7 @@ describe("readProjectRefFile", () => { yield* fs.makeDirectory(tempDir, { recursive: true }); yield* fs.writeFileString(refPath, ` ${REF}\n`); expect(Option.getOrNull(yield* readRef(dir))).toBe(REF); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide(services)), ); it.effect("treats a blank project-ref file as None", () => @@ -77,21 +104,19 @@ describe("readProjectRefFile", () => { yield* fs.makeDirectory(tempDir, { recursive: true }); yield* fs.writeFileString(refPath, " \n"); expect(Option.isNone(yield* readRef(dir))).toBe(true); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide(services)), ); it.effect("fails with ProjectRefReadError when the ref path is unreadable", () => Effect.gen(function* () { - // Returns `failed to load project ref` for a non-not-exist - // read error. Seeding project-ref as a DIRECTORY makes the - // read fail with EISDIR (a non-NotFound PlatformError), so it must surface, not - // collapse to "unlinked". + // Seeding project-ref as a directory makes the read fail with EISDIR, which must surface + // rather than collapse to "unlinked". const { fs, dir, refPath } = yield* workdir; yield* fs.makeDirectory(refPath, { recursive: true }); const error = yield* Effect.flip(readRef(dir)); expect(error).toBeInstanceOf(ProjectRefReadError); expect(error.message).toMatch(/^failed to load project ref: /); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe(Effect.provide(services)), ); it("classifies an unreadable ref file as permission without an unrelated command", () => { diff --git a/apps/cli/src/command-internal/link-services-core.integration.test.ts b/apps/cli/src/command-internal/link-services-core.integration.test.ts index b59827264f..34bce477c1 100644 --- a/apps/cli/src/command-internal/link-services-core.integration.test.ts +++ b/apps/cli/src/command-internal/link-services-core.integration.test.ts @@ -1,3 +1,4 @@ +import { legacyProjectStateLayer } from "./legacy-project-state.layer.ts"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer } from "effect"; @@ -76,7 +77,7 @@ describe("linkServicesCore", () => { api.layer, api.httpClientLayer, mockCommandSettings({ workdir: workdir.current }), - ).pipe(Layer.provideMerge(BunServices.layer)), + ).pipe(Layer.provideMerge(legacyProjectStateLayer), Layer.provideMerge(BunServices.layer)), ), Effect.as(api.requests), ); diff --git a/apps/cli/src/command-internal/link-services-core.ts b/apps/cli/src/command-internal/link-services-core.ts index 7f72794b89..c10aa43350 100644 --- a/apps/cli/src/command-internal/link-services-core.ts +++ b/apps/cli/src/command-internal/link-services-core.ts @@ -1,10 +1,9 @@ import type { ApiClient } from "@supabase/api/effect"; -import { Effect, FileSystem, Option, Path } from "effect"; -import type { PlatformError } from "effect/PlatformError"; +import { Effect, Option } from "effect"; import { CommandSettings } from "../config/command-settings.service.ts"; import { CommandPlatformApi } from "../auth/command-platform-api.service.ts"; -import { tempPaths } from "./temp-paths.ts"; +import { LegacyProjectState } from "./legacy-project-state.service.ts"; import { fetchGotrueVersion, fetchPostgrestVersion, @@ -39,8 +38,6 @@ export interface LinkServicesInput { readonly workdir: string; } -type WriteTempFile = (filePath: string, content: string) => Effect.Effect; - /** * Writes the best-effort portion of linking: * `supabase/.temp/{storage-migration,pooler-url,rest-version,gotrue-version, @@ -58,24 +55,11 @@ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( }); const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const paths = tempPaths(path, input.workdir); - - const writeTempFile: WriteTempFile = (filePath, content) => - fs - .makeDirectory(path.dirname(filePath), { recursive: true }) - .pipe(Effect.andThen(() => fs.writeFileString(filePath, content))); + const state = yield* LegacyProjectState; + const { workdir } = input; - yield* linkStorageMigration(api, input.ref, paths.storageMigration, writeTempFile); - yield* linkPooler({ - api, - ref: input.ref, - skipPooler: input.skipPooler, - fs, - poolerUrlPath: paths.poolerUrl, - writeTempFile, - }); + yield* linkStorageMigration(api, input.ref, state, workdir); + yield* linkPooler({ api, ref: input.ref, skipPooler: input.skipPooler, state, workdir }); const tenantOpts = { ref: input.ref, @@ -85,19 +69,19 @@ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( }; yield* fetchPostgrestVersion(tenantOpts).pipe( Effect.flatMap((v) => - Option.isSome(v) ? writeTempFile(paths.restVersion, v.value) : Effect.void, + Option.isSome(v) ? state.write(workdir, "restVersion", v.value) : Effect.void, ), Effect.ignore, ); yield* fetchGotrueVersion(tenantOpts).pipe( Effect.flatMap((v) => - Option.isSome(v) ? writeTempFile(paths.gotrueVersion, v.value) : Effect.void, + Option.isSome(v) ? state.write(workdir, "gotrueVersion", v.value) : Effect.void, ), Effect.ignore, ); yield* fetchStorageVersion(tenantOpts).pipe( Effect.flatMap((v) => - Option.isSome(v) ? writeTempFile(paths.storageVersion, v.value) : Effect.void, + Option.isSome(v) ? state.write(workdir, "storageVersion", v.value) : Effect.void, ), Effect.ignore, ); @@ -106,40 +90,52 @@ export const linkServicesCore = Effect.fn("LinkServices.write")(function* ( const linkStorageMigration = ( api: ApiClient, ref: string, - storageMigrationPath: string, - writeTempFile: WriteTempFile, + state: LegacyProjectState["Service"], + workdir: string, ) => api.v1.getStorageConfig({ ref }).pipe( Effect.flatMap((config) => config.migrationVersion === null ? Effect.void - : writeTempFile(storageMigrationPath, config.migrationVersion), + : state.write(workdir, "storageMigration", config.migrationVersion), ), Effect.ignore, ); +/** + * The session-mode connection string for a pooler config: drops the `[YOUR-PASSWORD]` + * placeholder and forces port 5432 unless the pooler already reports session mode. + */ +const normalizePoolerUrl = (connectionString: string, poolMode: string): string => { + const withoutPassword = connectionString.replaceAll(":[YOUR-PASSWORD]", ""); + return poolMode === "session" ? withoutPassword : withoutPassword.replaceAll(":6543/", ":5432/"); +}; + +/** The normalized primary pooler URL for `ref`, `None` when the project reports no primary. */ +export const fetchPoolerUrl = Effect.fn("link.fetchPoolerUrl")(function* ( + api: ApiClient, + ref: string, +) { + const configs = yield* api.v1.getPoolerConfig({ ref }); + const primary = configs.find((c) => c.database_type === "PRIMARY"); + return primary === undefined + ? Option.none() + : Option.some(normalizePoolerUrl(primary.connection_string, primary.pool_mode)); +}); + const linkPooler = (opts: { api: ApiClient; ref: string; skipPooler: boolean; - fs: FileSystem.FileSystem; - poolerUrlPath: string; - writeTempFile: WriteTempFile; + state: LegacyProjectState["Service"]; + workdir: string; }) => Effect.gen(function* () { if (opts.skipPooler) { // Use direct connection: drop any cached pooler URL. - yield* opts.fs.remove(opts.poolerUrlPath, { recursive: true }).pipe(Effect.ignore); + yield* opts.state.remove(opts.workdir, "poolerUrl").pipe(Effect.ignore); return; } - const configs = yield* opts.api.v1.getPoolerConfig({ ref: opts.ref }); - const primary = configs.find((c) => c.database_type === "PRIMARY"); - if (primary === undefined) return; - // Strip the [YOUR-PASSWORD] placeholder; force session mode 5432 unless the - // pooler already reports session mode. - let connectionString = primary.connection_string.replaceAll(":[YOUR-PASSWORD]", ""); - if (primary.pool_mode !== "session") { - connectionString = connectionString.replaceAll(":6543/", ":5432/"); - } - yield* opts.writeTempFile(opts.poolerUrlPath, connectionString); + const url = yield* fetchPoolerUrl(opts.api, opts.ref); + if (Option.isSome(url)) yield* opts.state.write(opts.workdir, "poolerUrl", url.value); }).pipe(Effect.ignore); diff --git a/apps/cli/src/command-internal/link-snapshot.ts b/apps/cli/src/command-internal/link-snapshot.ts new file mode 100644 index 0000000000..c1ed57d287 --- /dev/null +++ b/apps/cli/src/command-internal/link-snapshot.ts @@ -0,0 +1,155 @@ +import { Effect, type FileSystem, Option, type Path } from "effect"; + +import { readLegacyProjectRef } from "./legacy-project-state.layer.ts"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { type LinkedProjectRefReadError, readProjectLinkFile } from "./project-link.ts"; +import { currentProjectStateMode } from "./project-state-mode.ts"; + +/** + * The parent project's record: `ref`, plus `name`, `organizationSlug` and `organizationId` when + * the source holds a non-empty string for them. + */ +export interface CachedLinkedProject { + readonly ref: string; + readonly name?: string; + readonly organizationSlug?: string; + readonly organizationId?: string; +} + +/** + * Everything the checkout records about its hosted link, read from `supabase/.temp` in legacy + * mode or `supabase/.link.json` in stack mode. `ref` is the ref commands act on, unvalidated. + */ +export interface LinkSnapshot { + readonly ref: Option.Option; + readonly parent: Option.Option; + readonly branchName: Option.Option; + readonly poolerUrl: Option.Option; +} + +/** The snapshot of a checkout with no link. */ +const emptyLinkSnapshot: LinkSnapshot = { + ref: Option.none(), + parent: Option.none(), + branchName: Option.none(), + poolerUrl: Option.none(), +}; + +function readOptionalString(record: Record, key: string): string | undefined { + const value = record[key]; + return typeof value === "string" && value.length > 0 ? value : undefined; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null; +} + +/** Parses `.temp/linked-project.json`; a malformed document or an empty `ref` reads as `None`. */ +export function parseCachedLinkedProject(content: string): Option.Option { + try { + const parsed: unknown = JSON.parse(content); + if (isRecord(parsed)) { + const ref = parsed.ref; + if (typeof ref === "string" && ref.length > 0) { + const name = readOptionalString(parsed, "name"); + const organizationSlug = readOptionalString(parsed, "organization_slug"); + const organizationId = readOptionalString(parsed, "organization_id"); + return Option.some({ + ref, + ...(name === undefined ? {} : { name }), + ...(organizationSlug === undefined ? {} : { organizationSlug }), + ...(organizationId === undefined ? {} : { organizationId }), + }); + } + } + } catch { + // A malformed document reads as no parent, like a missing file. + } + return Option.none(); +} + +const readLegacySnapshot = Effect.fn("LinkSnapshot.readLegacy")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + strict: boolean, +): Effect.fn.Return { + const paths = legacyProjectStatePaths(path, workdir); + const ref = strict + ? yield* readLegacyProjectRef(fs, path, workdir) + : yield* readLegacyProjectRef(fs, path, workdir).pipe( + Effect.orElseSucceed(() => Option.none()), + ); + const parent = yield* fs.readFileString(paths.linkedProjectCache).pipe( + Effect.map(parseCachedLinkedProject), + Effect.orElseSucceed(() => Option.none()), + ); + const poolerUrl = yield* fs.readFileString(paths.poolerUrl).pipe(Effect.option); + return { ref, parent, branchName: Option.none(), poolerUrl }; +}); + +const readStackSnapshot = Effect.fn("LinkSnapshot.readStack")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +): Effect.fn.Return { + const link = yield* readProjectLinkFile(fs, path, workdir); + if (Option.isNone(link)) return emptyLinkSnapshot; + const { project, active_branch: active } = link.value; + const isBranch = active.ref !== project.ref; + return { + ref: Option.some(active.ref), + parent: Option.some({ + ref: project.ref, + ...(project.name === undefined ? {} : { name: project.name }), + ...(project.organization_slug === undefined + ? {} + : { organizationSlug: project.organization_slug }), + ...(project.organization_id === undefined ? {} : { organizationId: project.organization_id }), + }), + branchName: isBranch ? Option.fromNullishOr(active.name) : Option.none(), + poolerUrl: Option.fromNullishOr(active.pooler_url), + }; +}); + +const readSnapshot = Effect.fn("LinkSnapshot.read")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + strict: boolean, +): Effect.fn.Return { + const mode = yield* currentProjectStateMode; + const snapshot = + mode === "legacy" + ? yield* readLegacySnapshot(fs, path, workdir, strict) + : yield* readStackSnapshot(fs, path, workdir); + yield* Effect.annotateCurrentSpan({ + "project_state.mode": mode, + "link.linked": Option.isSome(snapshot.ref), + }); + return snapshot; +}); + +/** + * Reads the hosted link recorded in the checkout: the legacy `supabase/.temp` files, or the + * stack-mode `supabase/.link.json`. Only a failure to read the linked ref fails; the parent + * record and pooler URL degrade to `None`. Commands resolve their target through `RemoteTarget`; + * this is for readers that run without it. + */ +export const readLinkSnapshot = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +): Effect.Effect => + readSnapshot(fs, path, workdir, true); + +/** + * {@link readLinkSnapshot} for best-effort readers. An unreadable legacy ref file reads as no ref + * while the parent record and pooler URL still load; an unreadable `.link.json` reads as unlinked. + */ +export const readLinkSnapshotSoft = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +): Effect.Effect => + readSnapshot(fs, path, workdir, false).pipe(Effect.orElseSucceed(() => emptyLinkSnapshot)); diff --git a/apps/cli/src/command-internal/linked-state.ts b/apps/cli/src/command-internal/linked-state.ts index c7ad6cc1a3..83dc1ce133 100644 --- a/apps/cli/src/command-internal/linked-state.ts +++ b/apps/cli/src/command-internal/linked-state.ts @@ -1,11 +1,10 @@ import { Effect, FileSystem, Option, Path } from "effect"; import { CommandSettings } from "../config/command-settings.service.ts"; -import { PROJECT_REF_PATTERN } from "../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../config/remote-target.service.ts"; import { findBranchName } from "./branch-target.ts"; -import { type CachedLinkedProject, parseCachedLinkedProject } from "./parent-project-ref.ts"; import { formatNamedRef, sanitizeInlineName } from "./http-errors.ts"; -import { readProjectRefFile, tempPaths } from "./temp-paths.ts"; +import { type LinkSnapshot, readLinkSnapshotSoft } from "./link-snapshot.ts"; /** * Discriminated linked-state result. @@ -32,33 +31,23 @@ export type LinkedState = }; /** - * Soft lookup of the currently linked ref: env `SUPABASE_PROJECT_ID`, else the - * `/supabase/.temp/project-ref` file. Never prompts or fails, and does not depend on - * `ProjectRefResolver`, so runtimes like `status` can use it without wiring that resolver up. + * Soft lookup of the currently linked ref: env `SUPABASE_PROJECT_ID`, else the checkout's link. + * Never prompts or fails, and does not depend on `RemoteTarget`, so runtimes like `status` can + * use it without wiring that resolver up. * * Candidates must match `PROJECT_REF_PATTERN`: the value is echoed verbatim by `status -o json` * and the file read follows symlinks, so an unvalidated value could leak a secret file's contents. * Reports which source won, since the env override and the cache file carry different trust. */ -const resolveSoftLinkedRef = Effect.fnUntraced(function* () { - const cliSettings = yield* CommandSettings; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - - if ( - Option.isSome(cliSettings.projectId) && - PROJECT_REF_PATTERN.test(cliSettings.projectId.value) - ) { - return { ref: cliSettings.projectId, source: "env" as const }; +function resolveSoftLinkedRef(projectId: Option.Option, snapshot: LinkSnapshot) { + if (Option.isSome(projectId) && PROJECT_REF_PATTERN.test(projectId.value)) { + return { ref: projectId, source: "env" as const }; } - const fileRef = yield* readProjectRefFile(fs, path, cliSettings.workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - ); return { - ref: Option.filter(fileRef, (ref) => PROJECT_REF_PATTERN.test(ref)), + ref: Option.filter(snapshot.ref, (ref) => PROJECT_REF_PATTERN.test(ref)), source: "file" as const, }; -}); +} /** * Resolves the linked-state display (project or branch) for `status` without requiring a link. @@ -76,7 +65,8 @@ export const resolveLinkedState = Effect.fn("LinkedState.resolve")(function* () const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const soft = yield* resolveSoftLinkedRef(); + const snapshot = yield* readLinkSnapshotSoft(fs, path, cliSettings.workdir); + const soft = resolveSoftLinkedRef(cliSettings.projectId, snapshot); const linkedRef = soft.ref; if (Option.isNone(linkedRef)) { yield* Effect.annotateCurrentSpan("linked_state.kind", "unlinked"); @@ -88,11 +78,7 @@ export const resolveLinkedState = Effect.fn("LinkedState.resolve")(function* () "project.ref": linkedRef.value, }); - const paths = tempPaths(path, cliSettings.workdir); - const cached = yield* fs.readFileString(paths.linkedProjectCache).pipe( - Effect.map(parseCachedLinkedProject), - Effect.orElseSucceed(() => Option.none()), - ); + const cached = snapshot.parent; if (Option.isSome(cached)) { const cacheFields = { @@ -131,7 +117,7 @@ export const resolveLinkedState = Effect.fn("LinkedState.resolve")(function* () } // No cache at all: render the plain shape with zero API calls. Querying - // `resolveLinkedParentRef` here would only match the ref's own default + // `RemoteTarget.parent` here would only match the ref's own default // branch (misrendering it as "a branch of itself") or 403 when it // genuinely is a branch — there's no positive-confirmation case to attempt. return { linked: true, projectRef: linkedRef.value } as const; diff --git a/apps/cli/src/command-internal/management-api-runtime.layer.ts b/apps/cli/src/command-internal/management-api-runtime.layer.ts index 1340d805c3..0be3a1706d 100644 --- a/apps/cli/src/command-internal/management-api-runtime.layer.ts +++ b/apps/cli/src/command-internal/management-api-runtime.layer.ts @@ -13,8 +13,8 @@ import { CommandPlatformApi } from "../auth/command-platform-api.service.ts"; import { commandPlatformApiLayer } from "../auth/command-platform-api.layer.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; import { commandSettingsLayer } from "../config/command-settings.layer.ts"; -import { ProjectRefResolver } from "../config/project-ref.service.ts"; -import { projectRefLayer } from "../config/project-ref.layer.ts"; +import { RemoteTarget } from "../config/remote-target.service.ts"; +import { remoteTargetLayer } from "../config/remote-target.layer.ts"; import { DebugLogger } from "./debug-logger.service.ts"; import { debugLoggerLayer } from "./debug-logger.layer.ts"; import { dohFetchLayer } from "./http-dns.ts"; @@ -68,7 +68,7 @@ export function managementApiRuntimeLayer(subcommand: ReadonlyArray) { httpClient, credentials, cliSettings, - projectRefLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), + remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), linkedProjectCacheLayer.pipe( Layer.provide(credentials), Layer.provide(cliSettings), @@ -113,7 +113,7 @@ type ManagementApiServices = | HttpClient.HttpClient | CommandCredentials | CommandSettings - | ProjectRefResolver + | RemoteTarget | LinkedProjectCache | TelemetryState | CommandRuntime @@ -143,7 +143,7 @@ export function linkedDbResolverRuntimeLayer(subcommand: ReadonlyArray) httpClient, credentials, cliSettings, - projectRefLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), + remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), linkedProjectCacheLayer.pipe( Layer.provide(credentials), Layer.provide(cliSettings), @@ -155,6 +155,22 @@ export function linkedDbResolverRuntimeLayer(subcommand: ReadonlyArray) return built; } +/** Resolves only the hosted target, for commands that delegate to the Go binary. */ +export const remoteTargetRuntimeLayer = (() => { + const cliSettings = commandSettingsLayer.pipe(Layer.provide(debugLoggerLayer)); + const credentials = commandCredentialsLayer.pipe( + Layer.provide(cliSettings), + Layer.provide(debugLoggerLayer), + ); + const platformApiFactory = commandPlatformApiFactoryLayer.pipe( + Layer.provide(credentials), + Layer.provide(cliSettings), + Layer.provide(debugLoggerLayer), + Layer.provide(identityStitchLayer), + ); + return remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)); +})(); + type LinkedDbResolverRuntime = ReturnType; export type LinkedDbResolverRuntimeRequirements = LinkedDbResolverRuntime extends Layer.Layer ? R : never; diff --git a/apps/cli/src/command-internal/migration-fetch-run.ts b/apps/cli/src/command-internal/migration-fetch-run.ts index c18cf7ddc9..2e6c480f8b 100644 --- a/apps/cli/src/command-internal/migration-fetch-run.ts +++ b/apps/cli/src/command-internal/migration-fetch-run.ts @@ -4,7 +4,7 @@ import { DnsResolverFlag, resolveYesWithProjectEnv } from "./global-flags.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../shared/output/errors.ts"; import { Output } from "../shared/output/output.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../config/project-ref.service.ts"; +import { RemoteTarget } from "../config/remote-target.service.ts"; import { bold } from "./colors.ts"; import { DbConfigResolver } from "./db-config.service.ts"; import { DbConnection } from "./db-connection.service.ts"; @@ -72,11 +72,15 @@ export const runMigrationFetch = Effect.fn("MigrationFetch.run")(function* ( // `--db-url`/`config.toml` then fails immediately, instead of first creating // `supabase/migrations` or letting a declined overwrite prompt mask the real error // with `context canceled`. Same fix as `migration repair`. + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + connType === "linked" ? yield* remoteTarget.load(flags.projectRef) : undefined; const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType, dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); // The project .env loads after the parse-time flag-group validation above — so a @@ -86,15 +90,12 @@ export const runMigrationFetch = Effect.fn("MigrationFetch.run")(function* ( const projectEnv = yield* loadProjectEnv(fs, path, cliSettings.workdir); const yes = yield* resolveYesWithProjectEnv(projectEnv); - // Linked fetch caches the project ref on success. The ref is - // loaded now (pre-run), but the cache write is attached to the body via `Effect.ensuring`, - // so a declined prompt returns before it runs. + // Linked fetch caches the project ref on success. The cache write is attached to the body + // via `Effect.ensuring`, so a declined prompt returns before it runs. let cacheLinkedRef: Effect.Effect | undefined; - if (connType === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const ref = yield* projectRef.loadProjectRef(flags.projectRef); - cacheLinkedRef = linkedProjectCache.cache(ref); + cacheLinkedRef = linkedProjectCache.cache(linkedTarget.ref); } const fetchBody = Effect.gen(function* () { diff --git a/apps/cli/src/command-internal/parent-project-ref.ts b/apps/cli/src/command-internal/parent-project-ref.ts index 751f259c44..9bac16fb00 100644 --- a/apps/cli/src/command-internal/parent-project-ref.ts +++ b/apps/cli/src/command-internal/parent-project-ref.ts @@ -1,152 +1,3 @@ -import { Effect, FileSystem, Option, Path } from "effect"; - -import { CommandSettings } from "../config/command-settings.service.ts"; -import { ProjectRefResolver, PROJECT_REF_PATTERN } from "../config/project-ref.service.ts"; -import { readProjectRefFile, tempPaths } from "./temp-paths.ts"; - -export type ParentRefResolution = - | { readonly kind: "resolved"; readonly ref: string } - | { readonly kind: "invalid" } - | { readonly kind: "absent" }; - -/** - * The subset of `/supabase/.temp/linked-project.json` callers care about: the - * parent project's `ref`, `name`, `organization_slug`, and `organization_id`. The latter - * three are optional, present only when the cache holds a non-empty string value. - */ -export interface CachedLinkedProject { - readonly ref: string; - readonly name?: string; - readonly organizationSlug?: string; - readonly organizationId?: string; -} - -function readOptionalCacheString(record: Record, key: string): string | undefined { - const value = record[key]; - return typeof value === "string" && value.length > 0 ? value : undefined; -} - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null; -} - -/** - * Best-effort parse of `/supabase/.temp/linked-project.json`'s `ref` (and, when - * present, `name`/`organization_slug`/`organization_id`) fields. A missing/unreadable file, - * malformed JSON, or non-string/empty `ref` all degrade to `None` rather than failing. - */ -export function parseCachedLinkedProject(content: string): Option.Option { - try { - const parsed: unknown = JSON.parse(content); - if (isRecord(parsed)) { - const ref = parsed.ref; - if (typeof ref === "string" && ref.length > 0) { - const name = readOptionalCacheString(parsed, "name"); - const organizationSlug = readOptionalCacheString(parsed, "organization_slug"); - const organizationId = readOptionalCacheString(parsed, "organization_id"); - return Option.some({ - ref, - ...(name === undefined ? {} : { name }), - ...(organizationSlug === undefined ? {} : { organizationSlug }), - ...(organizationId === undefined ? {} : { organizationId }), - }); - } - } - } catch { - // Malformed JSON degrades to "no candidate", same as a missing file. - } - return Option.none(); -} - -function parseCachedParentRef(content: string): Option.Option { - return Option.map(parseCachedLinkedProject(content), (project) => project.ref); -} - -function classifyParentCandidates( - candidates: ReadonlyArray>, -): ParentRefResolution { - // The first present candidate decides: a malformed higher-priority candidate (e.g. a - // typo'd SUPABASE_PROJECT_ID) hard-classifies as invalid rather than falling through to a - // lower-priority project, which could otherwise run a mutation against the wrong project. - for (const candidate of candidates) { - if (Option.isSome(candidate)) { - return PROJECT_REF_PATTERN.test(candidate.value) - ? { kind: "resolved", ref: candidate.value } - : { kind: "invalid" }; - } - } - return { kind: "absent" }; -} - -/** - * Resolves the currently-linked PARENT project ref, not the branch's own ref that - * `ProjectRefResolver.resolveOptional`/`resolve` would return right after `link `. - * The first present candidate wins, in order: `SUPABASE_PROJECT_ID` env, the - * `linked-project.json` cache, then the temp `project-ref` file. A malformed candidate - * hard-classifies as `"invalid"` rather than falling through to a lower-priority one; no - * candidate at all means the workdir was never linked (`"absent"`). - */ -export const resolveLinkedParentRef = Effect.fnUntraced(function* () { - const cliSettings = yield* CommandSettings; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const paths = tempPaths(path, cliSettings.workdir); - - const fileRef = yield* readProjectRefFile(fs, path, cliSettings.workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - ); - // The cache alone isn't proof a link completed — `link` writes it on both success and - // failure — so only read it as parent recovery when the temp file confirms a link happened. - const cachedRef = Option.isSome(fileRef) - ? yield* fs.readFileString(paths.linkedProjectCache).pipe( - Effect.map(parseCachedParentRef), - Effect.orElseSucceed(() => Option.none()), - ) - : Option.none(); - - // An env candidate that merely restates the file candidate's own (valid) value carries no - // parent information; drop it so the cache gets a chance to win instead. Restricted to - // pattern-valid values so a garbage env equal to a garbage file still hard-classifies as - // invalid below instead of sneaking past to the cache. - const envRef = - Option.isSome(cliSettings.projectId) && - Option.isSome(fileRef) && - cliSettings.projectId.value === fileRef.value && - PROJECT_REF_PATTERN.test(cliSettings.projectId.value) - ? Option.none() - : cliSettings.projectId; - - return classifyParentCandidates([envRef, cachedRef, fileRef]); -}); - -/** - * Project-ref resolver for the PARENT-SCOPED `branches` command family: after `link `, - * the temp `project-ref` file holds the branch's own ref, but branch-management endpoints - * return 403 for branch refs, so this resolves the parent instead. An explicit `flagValue` - * always wins; otherwise it prefers `resolveLinkedParentRef`'s cached parent, falling back to - * `resolver.resolve` unchanged when no parent is resolvable. - */ -export const resolveParentScopedProjectRef = Effect.fn("ParentProjectRef.resolve")(function* ( - flagValue: Option.Option, -) { - const resolver = yield* ProjectRefResolver; - - if (Option.isSome(flagValue) && flagValue.value.length > 0) { - yield* Effect.annotateCurrentSpan("parent_project_ref.source", "flag"); - return yield* resolver.resolve(flagValue); - } - - const parent = yield* resolveLinkedParentRef(); - yield* Effect.annotateCurrentSpan("parent_project_ref.linked_parent", parent.kind); - if (parent.kind === "resolved") { - yield* Effect.annotateCurrentSpan("parent_project_ref.source", "linked_parent"); - return parent.ref; - } - - yield* Effect.annotateCurrentSpan("parent_project_ref.source", "resolver"); - return yield* resolver.resolve(Option.none()); -}); - /** * A value made entirely of lowercase letters (but not 20 of them, or it would already have * been treated as a ref) is a plausible ref typo. Appended to both {@link parentNotLinkedMessage} @@ -159,7 +10,7 @@ export function parentRefTypoHint(value: string): string { /** * Shared "no project is linked to search for branches" message, produced when - * {@link resolveLinkedParentRef} reports `"absent"` for a non-ref-shaped + * `RemoteTarget.parent` reports `"absent"` for a non-ref-shaped * `--project-ref`/positional value. Used by both `link` and `config diff`. */ export function parentNotLinkedMessage(value: string): string { @@ -174,7 +25,7 @@ export function parentNotLinkedMessage(value: string): string { /** * Shared "the linked project ref is invalid" message, produced when - * {@link resolveLinkedParentRef} reports `"invalid"` for a non-ref-shaped + * `RemoteTarget.parent` reports `"invalid"` for a non-ref-shaped * `--project-ref`/positional value. Used by both `link` and `config diff`. */ export function parentRefInvalidMessage(value: string): string { diff --git a/apps/cli/src/command-internal/pg-dump.native.integration.test.ts b/apps/cli/src/command-internal/pg-dump.native.integration.test.ts index 33d82fdc5c..09d9b28825 100644 --- a/apps/cli/src/command-internal/pg-dump.native.integration.test.ts +++ b/apps/cli/src/command-internal/pg-dump.native.integration.test.ts @@ -13,6 +13,7 @@ import { BundledPostgresClient } from "./bundled-postgres-client.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { mockOutput } from "../../tests/helpers/mocks.ts"; import { destroyTestStack } from "../../tests/helpers/stack-cleanup.ts"; +import { selection } from "../../tests/helpers/pins.ts"; const runtimes = ["native", "docker"] as const; const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); @@ -34,8 +35,8 @@ describe("managed pg_dump against a live stack", { timeout: 180_000 }, () => { yield* Effect.addFinalizer(() => destroyTestStack(stack)); const creation: Extract = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("cli-dump-password"), jwtSecret: Redacted.make("cli-dump-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -59,17 +60,20 @@ describe("managed pg_dump against a live stack", { timeout: 180_000 }, () => { PGPASSWORD: decodeURIComponent(connection.password), PGDATABASE: connection.pathname.slice(1), }; - const setupResult = yield* stack.commands.run(postgres.psql({ major: 17 }), { - args: ["-X", "-v", "ON_ERROR_STOP=1"], - env, - stdin: Stream.make( - new TextEncoder().encode( - "CREATE TABLE public.cli_dump_story(value text); INSERT INTO public.cli_dump_story VALUES ('runtime-row');", + const setupResult = yield* stack.commands.run( + postgres.psql({ pin: selection.database }), + { + args: ["-X", "-v", "ON_ERROR_STOP=1"], + env, + stdin: Stream.make( + new TextEncoder().encode( + "CREATE TABLE public.cli_dump_story(value text); INSERT INTO public.cli_dump_story VALUES ('runtime-row');", + ), ), - ), - stdout: (bytes) => Ref.update(output, (chunks) => [...chunks, bytes]), - stderr: (bytes) => Ref.update(stderr, (chunks) => [...chunks, bytes]), - }); + stdout: (bytes) => Ref.update(output, (chunks) => [...chunks, bytes]), + stderr: (bytes) => Ref.update(stderr, (chunks) => [...chunks, bytes]), + }, + ); expect(setupResult.exitCode).toBe(0); const dumpEnv = { ...env, @@ -81,7 +85,7 @@ describe("managed pg_dump against a live stack", { timeout: 180_000 }, () => { script: "pg_dump --data-only", env: dumpEnv, onStdout: (bytes) => Ref.update(chunks, (current) => [...current, bytes]), - client: { kind: "stack", stack, command: "pg_dump", major: 17 }, + client: { kind: "stack", stack, command: "pg_dump", pin: selection.database }, }); expect(result.exitCode).toBe(0); const dump = new TextDecoder().decode( diff --git a/apps/cli/src/command-internal/pg-dump.run.integration.test.ts b/apps/cli/src/command-internal/pg-dump.run.integration.test.ts index a2a35a8118..37b85cfeb6 100644 --- a/apps/cli/src/command-internal/pg-dump.run.integration.test.ts +++ b/apps/cli/src/command-internal/pg-dump.run.integration.test.ts @@ -7,6 +7,7 @@ import { NetworkIdFlag } from "./global-flags.ts"; import { DockerRun } from "./docker-run.service.ts"; import { BundledPostgresClient } from "./bundled-postgres-client.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; +import { selection } from "../../tests/helpers/pins.ts"; import { mockOutput } from "../../tests/helpers/mocks.ts"; const stackFixture = (output: string) => { @@ -115,7 +116,7 @@ describe("managed pg_dump runner", () => { EXTRA_FLAGS: '--exclude-table "public"."Users"', }, onStdout: () => Effect.void, - client: { kind: "stack", stack: fixture.stack, command: "pg_dump", major: 17 }, + client: { kind: "stack", stack: fixture.stack, command: "pg_dump", pin: selection.database }, }).pipe( Effect.provide( Layer.mergeAll( @@ -173,7 +174,7 @@ describe("managed pg_dump runner", () => { Effect.sync(() => { chunks.push(bytes); }), - client: { kind: "stack", stack: fixture.stack, command: "pg_dump", major: 17 }, + client: { kind: "stack", stack: fixture.stack, command: "pg_dump", pin: selection.database }, }).pipe( Effect.provide( Layer.mergeAll( diff --git a/apps/cli/src/command-internal/pg-dump.run.ts b/apps/cli/src/command-internal/pg-dump.run.ts index 651012deea..15ba8c4f2a 100644 --- a/apps/cli/src/command-internal/pg-dump.run.ts +++ b/apps/cli/src/command-internal/pg-dump.run.ts @@ -1,5 +1,6 @@ import { Effect, Option } from "effect"; import type { Stack } from "@supabase/stack/effect"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; import { postgres } from "@supabase/stack/commands"; type StackRuntimePreference = | { readonly kind: "native" } @@ -91,12 +92,12 @@ export type PgDumpClient = readonly kind: "stack"; readonly stack: Stack; readonly command: "pg_dump" | "pg_dumpall"; - readonly major: 15 | 17; + readonly pin: ArtifactPin; } | { readonly kind: "bundled"; readonly command: "pg_dump" | "pg_dumpall"; - readonly version: string; + readonly pin: ArtifactPin; readonly runtime?: StackRuntimePreference; }; @@ -301,8 +302,8 @@ export const streamPgDumpWithClient = Effect.fn("streamPgDumpWithClient")(functi yield* params.onStdout(new TextEncoder().encode("SET session_replication_role = replica;\n")); const result = yield* params.client.stack.commands.run( params.client.command === "pg_dump" - ? postgres.pgDump({ major: params.client.major }) - : postgres.pgDumpAll({ major: params.client.major }), + ? postgres.pgDump({ pin: params.client.pin }) + : postgres.pgDumpAll({ pin: params.client.pin }), { args, env: params.env, @@ -332,7 +333,7 @@ export const streamPgDumpWithClient = Effect.fn("streamPgDumpWithClient")(functi ? ["host.docker.internal:host-gateway"] : []; return yield* bundled.run({ - version: params.client.version, + pin: params.client.pin, runtime, argv: ["bash", "-c", params.script, "--"], env: params.env, diff --git a/apps/cli/src/command-internal/pgdelta.paths.ts b/apps/cli/src/command-internal/pgdelta.paths.ts index f4f10f139c..63603fe700 100644 --- a/apps/cli/src/command-internal/pgdelta.paths.ts +++ b/apps/cli/src/command-internal/pgdelta.paths.ts @@ -2,8 +2,8 @@ * On-disk locations for pg-delta-adjacent cache/snapshot artefacts. * * Two roots: - * - {@link pgDeltaTempPath}: project-local (`supabase/.temp/pgdelta`) — catalog - * snapshots and debug bundles (Go-shared, workspace-mounted). + * - `pgdeltaScratchDir`: `supabase/.temp/pgdelta` in legacy state (Go-shared, workspace-mounted), + * `$SUPABASE_HOME/tmp/pgdelta/` in stack state — catalog snapshots and debug bundles. * - {@link shadowBaselineCacheDir}: global under `SUPABASE_HOME` — the shadow * baseline PGDATA tars, shared across worktrees with the same settings. */ @@ -14,11 +14,6 @@ import type { Path } from "effect"; import { resolveSupabaseHome } from "../shared/config/supabase-home.ts"; -/** `supabase/.temp/pgdelta` — catalog snapshots and debug bundles. */ -export function pgDeltaTempPath(path: Path.Path, workdir: string): string { - return path.join(workdir, "supabase", ".temp", "pgdelta"); -} - /** * Global shadow-baseline cache directory: * `${SUPABASE_HOME}/cache/shadow-baseline` (default `~/.supabase/cache/shadow-baseline`). diff --git a/apps/cli/src/command-internal/project-create-core.ts b/apps/cli/src/command-internal/project-create-core.ts index 1fb37eaabc..4735453730 100644 --- a/apps/cli/src/command-internal/project-create-core.ts +++ b/apps/cli/src/command-internal/project-create-core.ts @@ -64,9 +64,9 @@ function printKeyValue(key: string, value: string): string { * Prompts for any missing project fields, creates the project via `POST /v1/projects`, and * echoes the result (`Created a new project at …` plus the `-o`/pretty render). * - * Returns the created ref and the resolved db password. Does not validate required flags - * (the standalone command's own pre-run does that) and does not write the linked-project - * cache (the caller owns that via `Effect.ensuring`). + * Returns the created project's ref, name, organization, and the resolved db password. Does not + * validate required flags (the standalone command's own pre-run does that) and does not write the + * linked-project cache (the caller owns that via `Effect.ensuring`). */ export const projectCreateCore = Effect.fn("ProjectCreate.run")(function* ( input: ProjectCreateInput, @@ -145,6 +145,13 @@ export const projectCreateCore = Effect.fn("ProjectCreate.run")(function* ( yield* creating?.clear ?? Effect.void; const id = readProjectField(created, "id"); + const result = { + ref: id, + name: readProjectField(created, "name"), + organizationId: readProjectField(created, "organization_id"), + organizationSlug: readProjectField(created, "organization_slug"), + dbPassword, + }; yield* Effect.annotateCurrentSpan("project.ref", id); // Printed to stderr for every output format. @@ -154,19 +161,19 @@ export const projectCreateCore = Effect.fn("ProjectCreate.run")(function* ( const goFmt = Option.getOrUndefined(goOutputFlag); if (goFmt === "json") { yield* output.raw(encodeGoJson(created)); - return { ref: id, dbPassword }; + return result; } if (goFmt === "yaml") { yield* output.raw(encodeGoYaml(created, GO_PROJECT_RESPONSE)); - return { ref: id, dbPassword }; + return result; } if (goFmt === "toml") { yield* output.raw(encodeGoToml(created, GO_PROJECT_RESPONSE)); - return { ref: id, dbPassword }; + return result; } if (goFmt === "env") { yield* output.raw(encodeEnv(created) + "\n"); - return { ref: id, dbPassword }; + return result; } if (output.format === "json" || output.format === "stream-json") { @@ -174,9 +181,9 @@ export const projectCreateCore = Effect.fn("ProjectCreate.run")(function* ( const data = typeof created === "object" && created !== null ? created : {}; yield* output.success("Created project", { ...data }); } - return { ref: id, dbPassword }; + return result; } yield* output.raw(renderProjectCreateTable(created)); - return { ref: id, dbPassword }; + return result; }); diff --git a/apps/cli/src/command-internal/project-link.ts b/apps/cli/src/command-internal/project-link.ts new file mode 100644 index 0000000000..77fb833b83 --- /dev/null +++ b/apps/cli/src/command-internal/project-link.ts @@ -0,0 +1,230 @@ +import { NamespaceError, publish } from "@supabase/stack/effect"; +import { Effect, FileSystem, Option, Path, Schema } from "effect"; + +import type { ExperimentalFeatureFlagError } from "./experimental-feature.ts"; +import type { ProjectRefReadError } from "./legacy-project-state.service.ts"; +import { + actionability, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../shared/telemetry/error-actionability.ts"; + +/** Document name under `/supabase`, local to one checkout. */ +const PROJECT_LINK_FILENAME = ".link.json"; + +/** Lines `supabase/.gitignore` needs so the link, the lock, their staged writes, and the stack lease stay out of version control. */ +const GITIGNORE_LINES = [ + PROJECT_LINK_FILENAME, + `.${PROJECT_LINK_FILENAME}.*.tmp`, + ".supabase.lock.*.tmp", + ".supabase.lock.lease", +] as const; + +const ProjectLinkProject = Schema.Struct({ + ref: Schema.String, + name: Schema.optionalKey(Schema.String), + organization_id: Schema.optionalKey(Schema.String), + organization_slug: Schema.optionalKey(Schema.String), +}); + +const ProjectLinkActiveBranch = Schema.Struct({ + ref: Schema.String, + name: Schema.optionalKey(Schema.String), + is_default: Schema.optionalKey(Schema.Boolean), + pooler_url: Schema.optionalKey(Schema.String), +}); + +/** + * The hosted link target of a stack-mode checkout. `project` is the parent project and + * `active_branch` the ref commands act on, which is the project's own ref unless a branch was + * linked. + */ +const ProjectLinkDocument = Schema.Struct({ + project: ProjectLinkProject, + active_branch: ProjectLinkActiveBranch, + fetchedAt: Schema.String, +}); + +export type ProjectLink = typeof ProjectLinkDocument.Type; + +const decodeProjectLink = Schema.decodeUnknownEffect(Schema.fromJsonString(ProjectLinkDocument)); + +/** `supabase/.link.json` exists but does not decode. */ +export class ProjectLinkMalformedError extends Schema.TaggedError()( + "ProjectLinkMalformedError", + { path: Schema.String, message: Schema.String }, +) { + get suggestion(): string { + return "Run `supabase link` to write the project link again."; + } + + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.relinkProject; + } +} + +/** Reading, writing, or removing `supabase/.link.json` failed for a reason other than absence. */ +export class ProjectLinkIoError extends Schema.TaggedError()( + "ProjectLinkIoError", + { + path: Schema.String, + message: Schema.String, + }, +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} + +const projectLinkPath = (path: Path.Path, workdir: string): string => + path.join(workdir, "supabase", PROJECT_LINK_FILENAME); + +export const readProjectLinkFile = Effect.fn("ProjectLink.readFile")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +) { + const file = projectLinkPath(path, workdir); + const content = yield* fs.readFileString(file).pipe( + Effect.map(Option.some), + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.succeed(Option.none()) + : Effect.fail( + new ProjectLinkIoError({ + path: file, + message: `failed to read ${file}: ${error.message}`, + }), + ), + ), + ); + if (Option.isNone(content)) return Option.none(); + const link = yield* decodeProjectLink(content.value).pipe( + Effect.mapError( + (cause) => + new ProjectLinkMalformedError({ + path: file, + message: `${file} is not a valid project link: ${cause.message}`, + }), + ), + ); + return Option.some(link); +}); + +/** Reads the link document, `None` when absent. A document that does not decode fails. */ +export const readProjectLink = Effect.fn("ProjectLink.read")(function* (workdir: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* readProjectLinkFile(fs, path, workdir); +}); + +/** + * Appends the entries of the local-only files under `supabase/` to `supabase/.gitignore`. A + * symlinked `.gitignore` is user-managed and stays untouched, so a checkout cannot aim the write + * at a file outside the project. + */ +export const ensureGitignore = Effect.fn("ProjectLink.ensureGitignore")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +) { + const gitignorePath = path.join(workdir, "supabase", ".gitignore"); + if (Option.isSome(yield* fs.readLink(gitignorePath).pipe(Effect.option))) return; + const existing = yield* fs + .readFileString(gitignorePath) + .pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" ? Effect.succeed("") : Effect.fail(error), + ), + ); + const present = new Set(existing.split("\n").map((line) => line.trim())); + const missing = GITIGNORE_LINES.filter((line) => !present.has(line)); + if (missing.length === 0) return; + const separator = existing.length === 0 || existing.endsWith("\n") ? "" : "\n"; + yield* publish(fs, path, { + target: gitignorePath, + content: `${existing}${separator}${missing.join("\n")}\n`, + mode: 0o644, + }); +}); + +/** + * Publishes the link document atomically after making sure `supabase/.gitignore` ignores it, so + * a reader sees the prior link or the complete new one. + */ +export const writeProjectLink = Effect.fn("ProjectLink.write")(function* ( + workdir: string, + link: ProjectLink, +) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const file = projectLinkPath(path, workdir); + yield* Effect.annotateCurrentSpan({ + "project.ref": link.project.ref, + "link.active_ref": link.active_branch.ref, + "link.has_pooler_url": link.active_branch.pooler_url !== undefined, + }); + yield* Effect.gen(function* () { + yield* fs.makeDirectory(path.dirname(file), { recursive: true }); + yield* ensureGitignore(fs, path, workdir); + yield* publish(fs, path, { target: file, content: `${JSON.stringify(link, null, 2)}\n` }); + }).pipe( + Effect.catchTags({ + PlatformError: (error) => + Effect.fail( + new ProjectLinkIoError({ + path: file, + message: `failed to write ${file}: ${error.message}`, + }), + ), + "Namespace.NamespaceError": (error: NamespaceError) => + Effect.fail( + new ProjectLinkIoError({ + path: file, + message: `failed to write ${file}: ${error.message}`, + }), + ), + }), + ); +}); + +/** Creates `supabase/`, where the link document and the project lease live. */ +export const ensureProjectDirectory = Effect.fn("ProjectLink.ensureDirectory")(function* ( + workdir: string, +) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const directory = path.dirname(projectLinkPath(path, workdir)); + yield* fs.makeDirectory(directory, { recursive: true }).pipe( + Effect.mapError( + (error) => + new ProjectLinkIoError({ + path: directory, + message: `failed to create ${directory}: ${error.message}`, + }), + ), + ); +}); + +/** Removes the link document; an absent file is not an error. */ +export const removeProjectLink = Effect.fn("ProjectLink.remove")(function* (workdir: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const file = projectLinkPath(path, workdir); + yield* fs.remove(file, { force: true }).pipe( + Effect.mapError( + (error) => + new ProjectLinkIoError({ + path: file, + message: `failed to remove ${file}: ${error.message}`, + }), + ), + ); +}); + +/** Failures of reading the linked ref in either project-state mode. */ +export type LinkedProjectRefReadError = + | ProjectRefReadError + | ProjectLinkMalformedError + | ProjectLinkIoError + | ExperimentalFeatureFlagError; diff --git a/apps/cli/src/command-internal/project-state-mode.ts b/apps/cli/src/command-internal/project-state-mode.ts new file mode 100644 index 0000000000..c99d6bc01e --- /dev/null +++ b/apps/cli/src/command-internal/project-state-mode.ts @@ -0,0 +1,77 @@ +import { Context, Effect, Exit, type FileSystem, Layer, Option, type Path } from "effect"; + +import { + type ExperimentalFeatureFlagError, + readExperimentalFeatureConfig, + resolveExperimentalFeature, +} from "./experimental-feature.ts"; + +/** Which state model a project uses: the stack runtime, or the legacy `supabase/.temp` files. */ +export type ProjectStateMode = "stack" | "legacy"; + +/** + * Reads the mode before any command runtime exists: `SUPABASE_EXPERIMENTAL_STACK=1|0` wins, and + * an unset or empty value falls back to `[experimental] stack` in the project's config. It + * decodes that single key, so an unreadable or invalid config counts as unset. Any other + * environment value fails. + */ +export const projectStateMode = Effect.fn("ProjectStateMode.resolve")(function* (input: { + readonly args?: ReadonlyArray; + readonly cwd: string; + readonly env: Readonly>; +}): Effect.fn.Return< + ProjectStateMode, + ExperimentalFeatureFlagError, + FileSystem.FileSystem | Path.Path +> { + const enabled = yield* resolveExperimentalFeature({ + feature: "stack", + configValue: readExperimentalFeatureConfig({ + feature: "stack", + args: input.args ?? [], + cwd: input.cwd, + env: input.env, + }), + env: input.env, + }); + const mode: ProjectStateMode = enabled ? "stack" : "legacy"; + yield* Effect.annotateCurrentSpan("project_state.mode", mode); + return mode; +}); + +/** + * The project state mode chosen once at startup. It is an `Exit` so that an invalid + * `SUPABASE_EXPERIMENTAL_STACK` fails only the readers that need the mode. + */ +export class ProjectStateModeContext extends Context.Service< + ProjectStateModeContext, + { readonly mode: Exit.Exit } +>()("supabase/cli/ProjectStateMode") {} + +export const projectStateModeLayer = ( + mode: Exit.Exit | ProjectStateMode, +) => + Layer.succeed(ProjectStateModeContext, { + mode: typeof mode === "string" ? Exit.succeed(mode) : mode, + }); + +/** + * The mode every state reader branches on, `legacy` when none was provided (such as in + * unit-level layers). Fails when the startup environment value was invalid. + */ +export const currentProjectStateMode: Effect.Effect< + ProjectStateMode, + ExperimentalFeatureFlagError +> = Effect.serviceOption(ProjectStateModeContext).pipe( + Effect.flatMap( + Option.match({ + onNone: () => Effect.succeed("legacy"), + onSome: (value) => value.mode, + }), + ), +); + +/** Mode for best-effort readers: an invalid environment value reads as `legacy` state. */ +export const softProjectStateMode: Effect.Effect = currentProjectStateMode.pipe( + Effect.orElseSucceed((): ProjectStateMode => "legacy"), +); diff --git a/apps/cli/src/command-internal/project-target.ts b/apps/cli/src/command-internal/project-target.ts index 9505cc40b3..3e25e6d43e 100644 --- a/apps/cli/src/command-internal/project-target.ts +++ b/apps/cli/src/command-internal/project-target.ts @@ -1,16 +1,14 @@ import type { SupabaseApiError } from "@supabase/api/effect"; import { Data, Effect, Option } from "effect"; -import { ProjectRefResolver } from "../config/project-ref.service.ts"; +import { RemoteTarget, type RemoteTargetError } from "../config/remote-target.service.ts"; import { parentNotLinkedMessage, parentRefInvalidMessage, parentRefTypoHint, - resolveLinkedParentRef, - resolveParentScopedProjectRef, } from "./parent-project-ref.ts"; import { Output } from "../shared/output/output.service.ts"; -import { projectRefPicker } from "../config/project-ref.layer.ts"; +import { projectRefPicker } from "../config/remote-target.layer.ts"; import { branchRef, resolveBranchProjectRef } from "./branch-ref.resolver.ts"; import { compose, withCondition } from "./command-permissions/model.ts"; import { sanitizeInlineName } from "./http-errors.ts"; @@ -173,7 +171,7 @@ export const resolveConfigTarget = Effect.fn("ProjectTarget.resolve")(function* mapResolveError: (cause: SupabaseApiError) => Effect.Effect, ) { const output = yield* Output; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; let ref: string; let branch: string | undefined; @@ -183,11 +181,13 @@ export const resolveConfigTarget = Effect.fn("ProjectTarget.resolve")(function* const byId = BRANCH_UUID_PATTERN.test(target); yield* Effect.annotateCurrentSpan("project_target.kind", byId ? "branch_id" : "branch_name"); - let parentRef: ReturnType; + let parentRef: Effect.Effect; if (byId) { - parentRef = resolveParentScopedProjectRef(Option.none()); + parentRef = remoteTarget + .resolveParentScoped(Option.none()) + .pipe(Effect.map((resolved) => resolved.ref)); } else { - const parent = yield* resolveLinkedParentRef(); + const parent = yield* remoteTarget.parent; if (parent.kind === "absent") { return yield* Effect.fail(errors.notLinked(target)); } @@ -215,7 +215,7 @@ export const resolveConfigTarget = Effect.fn("ProjectTarget.resolve")(function* } } else { yield* Effect.annotateCurrentSpan("project_target.kind", "project"); - ref = yield* resolver.resolve(requested); + ({ ref } = yield* remoteTarget.resolve(requested)); } yield* Effect.annotateCurrentSpan("project.ref", ref); diff --git a/apps/cli/src/command-internal/scratch-paths.integration.test.ts b/apps/cli/src/command-internal/scratch-paths.integration.test.ts new file mode 100644 index 0000000000..ae63a8112e --- /dev/null +++ b/apps/cli/src/command-internal/scratch-paths.integration.test.ts @@ -0,0 +1,42 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect } from "effect"; + +import { mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; +import { pgdeltaScratchDir, scratchDir } from "./scratch-paths.ts"; +import { stackBackendLayer } from "./stack-backend.ts"; + +const resolve = (kind: "legacy" | "stack", workdir: string) => + Effect.all({ + scratch: scratchDir(workdir), + pgdelta: pgdeltaScratchDir(workdir), + }).pipe( + Effect.provide([ + BunServices.layer, + mockCommandSettings({ workdir, supabaseHome: "/home/.supabase" }), + stackBackendLayer(kind), + ]), + ); + +describe("pg-delta scratch directory", () => { + it.effect("gives each project its own root in stack state", () => + Effect.gen(function* () { + const first = yield* resolve("stack", "/work/first"); + const second = yield* resolve("stack", "/work/second"); + + expect(first.pgdelta).not.toBe(second.pgdelta); + for (const { scratch, pgdelta } of [first, second]) { + expect(pgdelta.startsWith(`${scratch}/pgdelta/`)).toBe(true); + } + expect(yield* resolve("stack", "/work/first")).toEqual(first); + }), + ); + + it.effect("stays under the project's supabase/.temp in legacy state", () => + Effect.gen(function* () { + const { pgdelta } = yield* resolve("legacy", "/work/first"); + + expect(pgdelta).toBe("/work/first/supabase/.temp/pgdelta"); + }), + ); +}); diff --git a/apps/cli/src/command-internal/scratch-paths.ts b/apps/cli/src/command-internal/scratch-paths.ts new file mode 100644 index 0000000000..2ea3189c26 --- /dev/null +++ b/apps/cli/src/command-internal/scratch-paths.ts @@ -0,0 +1,30 @@ +import { Crypto, Effect, Path } from "effect"; + +import { CommandSettings } from "../config/command-settings.service.ts"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; +import { usesStackState } from "./stack-backend.ts"; + +/** Host scratch for bundler output and downloads: `$SUPABASE_HOME/tmp` in stack state, else `supabase/.temp`. */ +export const scratchDir = Effect.fnUntraced(function* (projectRoot: string) { + const path = yield* Path.Path; + if (yield* usesStackState) return path.join((yield* CommandSettings).supabaseHome, "tmp"); + return legacyProjectStatePaths(path, projectRoot).tempDir; +}); + +/** + * Catalog snapshots and debug bundles of the pg-delta engines, under {@link scratchDir}. The + * stack-state scratch is shared by every project, so it gains a per-project directory. + */ +export const pgdeltaScratchDir = Effect.fnUntraced(function* (projectRoot: string) { + const path = yield* Path.Path; + const root = path.join(yield* scratchDir(projectRoot), "pgdelta"); + if (!(yield* usesStackState)) return root; + const digest = yield* (yield* Crypto.Crypto).digest( + "SHA-256", + new TextEncoder().encode(path.resolve(projectRoot)), + ); + return path.join( + root, + Array.from(digest.slice(0, 8), (b) => b.toString(16).padStart(2, "0")).join(""), + ); +}); diff --git a/apps/cli/src/command-internal/service-version-overrides.ts b/apps/cli/src/command-internal/service-version-overrides.ts index f2a8c3d656..19ba71b4d9 100644 --- a/apps/cli/src/command-internal/service-version-overrides.ts +++ b/apps/cli/src/command-internal/service-version-overrides.ts @@ -8,7 +8,8 @@ import { type CliErrorActionabilityDeclaration, ErrorActionabilityId, } from "../shared/telemetry/error-actionability.ts"; -import { tempPaths } from "./temp-paths.ts"; +import { legacyProjectStatePaths, type LegacyStateFile } from "./legacy-project-state.paths.ts"; +import { usesStackState } from "./stack-backend.ts"; /** A saved `.temp` pin is not a usable image tag. The file is left unchanged. */ export class InvalidServiceVersionTagError extends Data.TaggedError( @@ -28,18 +29,18 @@ export class InvalidServiceVersionTagError extends Data.TaggedError( * `postgrest` are additionally gated on `majorVersion > 14`. */ const VERSION_FILES = [ - ["auth", "gotrue-version", (majorVersion: number | undefined) => (majorVersion ?? 17) > 14], - ["postgrest", "rest-version", (majorVersion: number | undefined) => (majorVersion ?? 17) > 14], - ["storage", "storage-version"], - ["realtime", "realtime-version"], - ["studio", "studio-version"], - ["pgmeta", "pgmeta-version"], - ["analytics", "logflare-version"], - ["pooler", "pooler-version"], + ["auth", "gotrueVersion", (majorVersion: number | undefined) => (majorVersion ?? 17) > 14], + ["postgrest", "restVersion", (majorVersion: number | undefined) => (majorVersion ?? 17) > 14], + ["storage", "storageVersion"], + ["realtime", "realtimeVersion"], + ["studio", "studioVersion"], + ["pgmeta", "pgmetaVersion"], + ["analytics", "logflareVersion"], + ["pooler", "poolerVersion"], ] as const satisfies ReadonlyArray< readonly [ "auth" | "postgrest" | "storage" | "realtime" | "studio" | "pgmeta" | "analytics" | "pooler", - string, + LegacyStateFile, ((majorVersion: number | undefined) => boolean)?, ] >; @@ -50,28 +51,29 @@ const VERSION_FILES = [ * (including not-exist) resolves to "" for that file. A pin that is not a usable * image tag fails; the file is not rewritten. */ -export const readServiceVersionOverrides = Effect.fnUntraced(function* ( +export const readServiceVersionOverrides = Effect.fn("ServiceVersionOverrides.read")(function* ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, majorVersion: number | undefined, ) { - const paths = tempPaths(path, workdir); const versions: LocalServiceVersionOverrides = {}; + if (yield* usesStackState) return versions; + const paths = legacyProjectStatePaths(path, workdir); - for (const [service, fileName, shouldRead] of VERSION_FILES) { + for (const [service, file, shouldRead] of VERSION_FILES) { if (shouldRead !== undefined && !shouldRead(majorVersion)) { continue; } - const version = yield* fs.readFileString(path.join(paths.tempDir, fileName)).pipe( + const version = yield* fs.readFileString(paths[file]).pipe( Effect.map((content) => content.trim()), Effect.orElseSucceed(() => ""), ); if (version.length === 0) continue; if (!isUsableServiceVersionTag(service, version)) { return yield* new InvalidServiceVersionTagError({ - message: `invalid ${service} image tag "${version}" in supabase/.temp/${fileName}; run supabase link again`, + message: `invalid ${service} image tag "${version}" in ${path.relative(workdir, paths[file])}; run supabase link again`, }); } versions[service] = version; diff --git a/apps/cli/src/command-internal/service-version-overrides.unit.test.ts b/apps/cli/src/command-internal/service-version-overrides.unit.test.ts index 09fe156176..77faf2640a 100644 --- a/apps/cli/src/command-internal/service-version-overrides.unit.test.ts +++ b/apps/cli/src/command-internal/service-version-overrides.unit.test.ts @@ -42,14 +42,12 @@ describe("readServiceVersionOverrides", () => { const workdir = yield* makeWorkdir; const pinPath = path.join(workdir, "supabase", ".temp", "storage-version"); yield* writePin(workdir, "storage-version", "vv1.77.1-versions\n"); - const exit = yield* readPins(workdir).pipe(Effect.exit); - expect(exit).toEqual( - Exit.fail( - new InvalidServiceVersionTagError({ - message: - 'invalid storage image tag "vv1.77.1-versions" in supabase/.temp/storage-version; run supabase link again', - }), - ), + const error = yield* readPins(workdir).pipe(Effect.flip); + expect(error).toEqual( + new InvalidServiceVersionTagError({ + message: + 'invalid storage image tag "vv1.77.1-versions" in supabase/.temp/storage-version; run supabase link again', + }), ); expect(yield* fs.readFileString(pinPath)).toBe("vv1.77.1-versions\n"); }).pipe(Effect.provide(BunServices.layer)), @@ -59,14 +57,12 @@ describe("readServiceVersionOverrides", () => { Effect.gen(function* () { const workdir = yield* makeWorkdir; yield* writePin(workdir, "rest-version", "vv12.2.0"); - const exit = yield* readPins(workdir).pipe(Effect.exit); - expect(exit).toEqual( - Exit.fail( - new InvalidServiceVersionTagError({ - message: - 'invalid postgrest image tag "vv12.2.0" in supabase/.temp/rest-version; run supabase link again', - }), - ), + const error = yield* readPins(workdir).pipe(Effect.flip); + expect(error).toEqual( + new InvalidServiceVersionTagError({ + message: + 'invalid postgrest image tag "vv12.2.0" in supabase/.temp/rest-version; run supabase link again', + }), ); }).pipe(Effect.provide(BunServices.layer)), ); diff --git a/apps/cli/src/command-internal/stack-artifact-selection.ts b/apps/cli/src/command-internal/stack-artifact-selection.ts new file mode 100644 index 0000000000..78a4d8ef35 --- /dev/null +++ b/apps/cli/src/command-internal/stack-artifact-selection.ts @@ -0,0 +1,60 @@ +import { + databaseArtifactPin, + orioledbPostgresVersion, + orioledbVersions, + postgresMajor, + type ArtifactSelection, +} from "@supabase/stack/internal/artifacts"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { Effect, Result, Schema } from "effect"; + +/** A `[db]` config asks for a database release the experimental stack cannot run. */ +export class StackArtifactSelectionError extends Schema.TaggedError()( + "StackArtifactSelectionError", + { message: Schema.String }, +) {} + +type DatabaseSettings = { + readonly major_version: number; + readonly orioledb_version?: string | undefined; +}; + +/** The database release a `[db]` config asks for: its major, or the OrioleDB build it names. */ +const requestedDatabaseVersion = ( + db: DatabaseSettings, + supported: ReadonlyArray, +): Result.Result => { + const orioledb = db.orioledb_version; + if (orioledb === undefined || orioledb.length === 0) + return Result.succeed(String(db.major_version)); + if (!supported.includes(orioledb)) + return Result.fail( + `db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = ${orioledb} is not an OrioleDB build this CLI ships for the experimental stack; it ships: ${ + supported.length === 0 ? "none" : supported.join(", ") + }. A stack keeps the build it was created with, so moving an existing stack to another build means recreating it with supabase stack destroy, which permanently deletes its local database data`, + ); + const major = postgresMajor(orioledb); + if (major !== String(db.major_version)) + return Result.fail( + `db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = ${orioledb} requires db.major_version (or SUPABASE_DB_MAJOR_VERSION) = ${major} for the experimental stack`, + ); + return Result.succeed(orioledbPostgresVersion(orioledb)); +}; + +/** + * The artifact pins a stack command runs: the compiled defaults, with the database pin chosen by + * the project's `[db]` config. Built once per command and written into every creation. + */ +export const stackArtifactSelection = ( + db: DatabaseSettings, + supported: ReadonlyArray = orioledbVersions(), +): Effect.Effect => + Effect.fromResult(requestedDatabaseVersion(db, supported)).pipe( + Effect.mapError((message) => new StackArtifactSelectionError({ message })), + Effect.flatMap((version) => + databaseArtifactPin(version).pipe( + Effect.mapError((error) => new StackArtifactSelectionError({ message: error.message })), + ), + ), + Effect.map((database) => ({ ...defaultArtifactSelection(), database })), + ); diff --git a/apps/cli/src/command-internal/stack-artifact-selection.unit.test.ts b/apps/cli/src/command-internal/stack-artifact-selection.unit.test.ts new file mode 100644 index 0000000000..f9324053bf --- /dev/null +++ b/apps/cli/src/command-internal/stack-artifact-selection.unit.test.ts @@ -0,0 +1,58 @@ +import { describe, expect, it } from "@effect/vitest"; +import { orioledbVersions, postgresMajor } from "@supabase/stack/internal/artifacts"; +import { Effect } from "effect"; + +import { stackArtifactSelection } from "./stack-artifact-selection.ts"; + +const published = ["17.11.0.002"]; + +describe("stackArtifactSelection", () => { + it.effect("selects the major's default database pin when OrioleDB is unset or empty", () => + Effect.gen(function* () { + const seventeen = yield* stackArtifactSelection({ major_version: 17 }, published); + const fifteen = yield* stackArtifactSelection( + { major_version: 15, orioledb_version: "" }, + published, + ); + expect(postgresMajor(seventeen.database.upstreamVersion)).toBe("17"); + expect(fifteen.database.upstreamVersion).toMatch(/^15\./); + expect(fifteen.rest).toEqual(seventeen.rest); + }), + ); + + it.effect("selects every OrioleDB build the artifact catalog pins", () => + Effect.gen(function* () { + const pinned = orioledbVersions(); + expect(pinned.length).toBeGreaterThan(0); + for (const version of pinned) { + const selection = yield* stackArtifactSelection({ + major_version: Number(postgresMajor(version)), + orioledb_version: version, + }); + expect(selection.database.upstreamVersion).toBe(`${version}-orioledb`); + } + }), + ); + + it.effect("rejects an OrioleDB version whose major differs from db.major_version", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + stackArtifactSelection({ major_version: 15, orioledb_version: "17.11.0.002" }, published), + ); + expect(error.message).toBe( + "db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = 17.11.0.002 requires db.major_version (or SUPABASE_DB_MAJOR_VERSION) = 17 for the experimental stack", + ); + }), + ); + + it.effect("fails closed on an unpublished OrioleDB version, listing the published ones", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + stackArtifactSelection({ major_version: 17, orioledb_version: "17.6.1.000" }, published), + ); + expect(error.message).toBe( + "db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = 17.6.1.000 is not an OrioleDB build this CLI ships for the experimental stack; it ships: 17.11.0.002. A stack keeps the build it was created with, so moving an existing stack to another build means recreating it with supabase stack destroy, which permanently deletes its local database data", + ); + }), + ); +}); diff --git a/apps/cli/src/command-internal/stack-backend.ts b/apps/cli/src/command-internal/stack-backend.ts index 80438eb9d6..bbe6354c8c 100644 --- a/apps/cli/src/command-internal/stack-backend.ts +++ b/apps/cli/src/command-internal/stack-backend.ts @@ -1,9 +1,20 @@ -import { Context, Data, Effect, FileSystem, Layer, Option, Path } from "effect"; +import { + Context, + Data, + Effect, + type Exit, + type FileSystem, + Layer, + Option, + type Path, +} from "effect"; import { extractCommandPath, hasRootVersionFlag } from "../shared/cli/run.ts"; +import type { ExperimentalFeatureFlagError } from "./experimental-feature.ts"; import { - readExperimentalFeatureConfig, - resolveExperimentalFeature, -} from "./experimental-feature.ts"; + projectStateMode, + type ProjectStateMode, + softProjectStateMode, +} from "./project-state-mode.ts"; import { actionability, type CliErrorActionabilityDeclaration, @@ -59,20 +70,42 @@ export const currentStackBackend: Effect.Effect<{ readonly kind: StackBackend }, Effect.map((value) => Option.getOrElse(value, () => ({ kind: "legacy" as const }))), ); -/** Resolves the backend, or `undefined` for commands that `experimental.stack` does not route. */ -export const resolveStackBackend = (input: { - readonly args: ReadonlyArray; - readonly cwd: string; - readonly env: Readonly>; -}): Effect.Effect => - Effect.gen(function* () { +/** + * Whether this command keeps project state in `$SUPABASE_HOME` instead of `supabase/.temp`: it was + * routed to the stack backend, or the project is in stack state mode (commands that are never + * routed, such as `functions deploy`). + */ +export const usesStackState: Effect.Effect = Effect.gen(function* () { + if ((yield* currentStackBackend).kind === "stack") return true; + return (yield* softProjectStateMode) === "stack"; +}); + +/** + * Resolves the project state mode and, for the commands `experimental.stack` routes, the backend. + * Routed commands fail on an invalid `SUPABASE_EXPERIMENTAL_STACK`; for the others the failure + * stays in `projectState` for the readers that need it. Both are `undefined` for `--version`. + */ +export const resolveProjectStateSelection = Effect.fn("ProjectState.resolveSelection")( + function* (input: { + readonly args: ReadonlyArray; + readonly cwd: string; + readonly env: Readonly>; + }): Effect.fn.Return< + { + readonly projectState: Exit.Exit | undefined; + readonly stackBackend: StackBackend | undefined; + }, + StackRoutingError, + FileSystem.FileSystem | Path.Path + > { // Completion passes the final token as the cursor word, so it is not part of // the resolved command path. const routingArgs = input.args[0] === "__complete" || input.args[0] === "__completeNoDesc" ? input.args.slice(0, -1) : input.args; - if (hasRootVersionFlag(routingArgs)) return undefined; + if (hasRootVersionFlag(routingArgs)) + return { projectState: undefined, stackBackend: undefined }; const commandPath = extractCommandPath(routingArgs); const completePath = @@ -81,20 +114,20 @@ export const resolveStackBackend = (input: { : commandPath; const routedPath = completePath[0] === "help" ? completePath.slice(1) : completePath; const command = routedPath[0]; - if ( - command !== undefined && - command !== "stack" && - !STACK_BACKEND_COMMANDS.has(command) && - !isFunctionsServePath(routedPath) - ) - return undefined; + const routed = + command === undefined || + command === "stack" || + STACK_BACKEND_COMMANDS.has(command) || + isFunctionsServePath(routedPath); - const enabled = yield* resolveExperimentalFeature({ - feature: "stack", - configValue: readExperimentalFeatureConfig({ feature: "stack", ...input, args: routingArgs }), - env: input.env, - }).pipe( - Effect.mapError((error) => new StackRoutingError({ message: error.message, cause: error })), - ); - return enabled ? "stack" : "legacy"; - }); + const projectState = yield* Effect.exit(projectStateMode({ ...input, args: routingArgs })); + const stackBackend = routed + ? yield* projectState.pipe( + Effect.mapError( + (error) => new StackRoutingError({ message: error.message, cause: error }), + ), + ) + : undefined; + return { projectState, stackBackend }; + }, +); diff --git a/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts b/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts index f6e24993d3..dac1a0cd9e 100644 --- a/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts +++ b/apps/cli/src/command-internal/stack-catalog-setup.integration.test.ts @@ -27,6 +27,7 @@ import { dbConnectionLayer } from "./db-connection.sql-pg.layer.ts"; import { stackCatalogSetupLayer, StackCatalogSetup } from "./stack-catalog-setup.ts"; import { destroyTestStack } from "../../tests/helpers/stack-cleanup.ts"; import { stackArtifactCacheRoot } from "../../tests/helpers/stack-artifacts.ts"; +import { selection } from "../../tests/helpers/pins.ts"; const cacheRoot = stackArtifactCacheRoot; const decodeJson = Schema.decodeSync(Schema.fromJsonString(Schema.Unknown)); @@ -61,8 +62,8 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { const members = yield* stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -71,11 +72,13 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { }, { service: "auth", + pin: selection.auth, config: { jwtSecret }, endpoints: { http: { port: "auto" } }, }, { service: "storage", + pin: selection.storage, config: { jwtSecret, filePath: `${root}/unused-storage`, @@ -84,6 +87,7 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { }, { service: "realtime", + pin: selection.realtime, config: { jwtSecret }, endpoints: { http: { port: "auto" }, rpc: { port: "auto" } }, }, @@ -103,6 +107,7 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { stack, database, databaseServices: ["auth", "storage", "realtime"], + selection, }, overlay: { webhooks: "enabled", @@ -117,36 +122,42 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { expect(callOutput.stderrText).toContain("Seeding globals from roles.sql..."); const rows: Array = []; const errors: Array = []; - const query = yield* stack.commands.run(postgres.psql({ major: 17 }), { - args: ["--dbname", databaseUrl, "-At"], - stdin: Stream.make( - new TextEncoder().encode( - "select coalesce(to_regclass('auth.users')::text,'missing'), coalesce(to_regclass('auth.sessions')::text,'missing'), coalesce(to_regclass('storage.objects')::text,'missing'), coalesce(to_regclass('storage.s3_multipart_uploads')::text,'missing'), coalesce(to_regclass('realtime.messages')::text,'missing'), coalesce(to_regclass('realtime.subscription')::text,'missing'), coalesce(to_regclass('public.catalog_overlay')::text,'missing'), coalesce(to_regprocedure('supabase_functions.http_request()')::text,'missing');", + const query = yield* stack.commands.run( + postgres.psql({ pin: selection.database }), + { + args: ["--dbname", databaseUrl, "-At"], + stdin: Stream.make( + new TextEncoder().encode( + "select coalesce(to_regclass('auth.users')::text,'missing'), coalesce(to_regclass('auth.sessions')::text,'missing'), coalesce(to_regclass('storage.objects')::text,'missing'), coalesce(to_regclass('storage.s3_multipart_uploads')::text,'missing'), coalesce(to_regclass('realtime.messages')::text,'missing'), coalesce(to_regclass('realtime.subscription')::text,'missing'), coalesce(to_regclass('public.catalog_overlay')::text,'missing'), coalesce(to_regprocedure('supabase_functions.http_request()')::text,'missing');", + ), ), - ), - stdout: (bytes) => - Effect.sync(() => rows.push(new TextDecoder().decode(bytes))), - stderr: (bytes) => - Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), - }); + stdout: (bytes) => + Effect.sync(() => rows.push(new TextDecoder().decode(bytes))), + stderr: (bytes) => + Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), + }, + ); expect(query.exitCode, errors.join("")).toBe(0); expect(rows.join("").trim()).toBe( "users|sessions|storage.objects|storage.s3_multipart_uploads|realtime.messages|realtime.subscription|catalog_overlay|supabase_functions.http_request()", ); // pg_net queues webhook requests under the image's function policy. const pgNet: Array = []; - const pgNetQuery = yield* stack.commands.run(postgres.psql({ major: 17 }), { - args: ["--dbname", databaseUrl, "-Atq", "-v", "ON_ERROR_STOP=1"], - stdin: Stream.make( - new TextEncoder().encode( - "begin; create table public.catalog_hook(id int); create trigger catalog_hook after insert on public.catalog_hook for each row execute function supabase_functions.http_request('http://127.0.0.1:9', 'POST', '{}', '{}', '1000'); insert into public.catalog_hook values (1); select (select count(*) from net.http_request_queue) || ';' || string_agg(format('%s:secdef=%s:search_path=%s', proname, prosecdef::text, (exists (select from unnest(proconfig) as setting where setting like 'search_path=%'))::text), ',' order by proname) from pg_proc where pronamespace = 'net'::regnamespace and proname in ('http_get', 'http_post'); rollback;", + const pgNetQuery = yield* stack.commands.run( + postgres.psql({ pin: selection.database }), + { + args: ["--dbname", databaseUrl, "-Atq", "-v", "ON_ERROR_STOP=1"], + stdin: Stream.make( + new TextEncoder().encode( + "begin; create table public.catalog_hook(id int); create trigger catalog_hook after insert on public.catalog_hook for each row execute function supabase_functions.http_request('http://127.0.0.1:9', 'POST', '{}', '{}', '1000'); insert into public.catalog_hook values (1); select (select count(*) from net.http_request_queue) || ';' || string_agg(format('%s:secdef=%s:search_path=%s', proname, prosecdef::text, (exists (select from unnest(proconfig) as setting where setting like 'search_path=%'))::text), ',' order by proname) from pg_proc where pronamespace = 'net'::regnamespace and proname in ('http_get', 'http_post'); rollback;", + ), ), - ), - stdout: (bytes) => - Effect.sync(() => pgNet.push(new TextDecoder().decode(bytes))), - stderr: (bytes) => - Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), - }); + stdout: (bytes) => + Effect.sync(() => pgNet.push(new TextDecoder().decode(bytes))), + stderr: (bytes) => + Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), + }, + ); expect(pgNetQuery.exitCode, errors.join("")).toBe(0); expect(pgNet.join("").trim()).toBe( "1;http_get:secdef=false:search_path=false,http_post:secdef=false:search_path=false", @@ -194,18 +205,21 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { if (credentials === undefined) return yield* Effect.die("stack credentials missing"); const tenantJwks: Array = []; - const tenantJwksQuery = yield* stack.commands.run(postgres.psql({ major: 17 }), { - args: ["--dbname", databaseUrl, "-At"], - stdin: Stream.make( - new TextEncoder().encode( - "SELECT jwt_jwks::text FROM _realtime.tenants WHERE external_id = 'realtime-dev';", + const tenantJwksQuery = yield* stack.commands.run( + postgres.psql({ pin: selection.database }), + { + args: ["--dbname", databaseUrl, "-At"], + stdin: Stream.make( + new TextEncoder().encode( + "SELECT jwt_jwks::text FROM _realtime.tenants WHERE external_id = 'realtime-dev';", + ), ), - ), - stdout: (bytes) => - Effect.sync(() => tenantJwks.push(new TextDecoder().decode(bytes))), - stderr: (bytes) => - Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), - }); + stdout: (bytes) => + Effect.sync(() => tenantJwks.push(new TextDecoder().decode(bytes))), + stderr: (bytes) => + Effect.sync(() => errors.push(new TextDecoder().decode(bytes))), + }, + ); expect(tenantJwksQuery.exitCode, errors.join("")).toBe(0); expect(decodeJson(tenantJwks.join("").trim())).toEqual( decodeJson(credentials.jwks), @@ -233,6 +247,15 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { const releaseStorage = yield* Deferred.make(); const releaseAuth = yield* Deferred.make(); const temporaryDirectory = yield* Ref.make(undefined); + const dispatched: Array<{ type: string; image: string }> = []; + const selected = { + ...selection, + auth: { ...selection.auth, image: "registry.test/auth@sha256:selected" }, + storage: { + ...selection.storage, + image: "registry.test/storage@sha256:selected", + }, + }; const controlledStack: Stack = { ...stack, commands: { @@ -240,6 +263,11 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { run: (invocation: Command) => { if (!("type" in invocation)) return Effect.die("postgres command is not used in this fixture"); + if ("pin" in invocation) + dispatched.push({ + type: invocation.type, + image: invocation.pin.image, + }); if (invocation.type === "storage.initialize") return Ref.set(temporaryDirectory, invocation.filePath).pipe( Effect.andThen(Deferred.succeed(storageReady, undefined)), @@ -271,6 +299,7 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { stack: controlledStack, database, databaseServices: ["auth", "storage"], + selection: selected, }, overlay: { webhooks: "disabled", @@ -299,6 +328,10 @@ describe("stack catalog setup", { timeout: 180_000 }, () => { expect(Exit.hasInterrupts(exit)).toBe(true); } + expect(dispatched.toSorted((a, b) => a.type.localeCompare(b.type))).toEqual([ + { type: "auth.initialize", image: selected.auth.image }, + { type: "storage.initialize", image: selected.storage.image }, + ]); expect(callOutput.stderrText).toBe(beforeOutput); const temporaryPath = yield* Ref.get(temporaryDirectory); if (temporaryPath === undefined) diff --git a/apps/cli/src/command-internal/stack-catalog-setup.ts b/apps/cli/src/command-internal/stack-catalog-setup.ts index feca010521..a55c0a0a3c 100644 --- a/apps/cli/src/command-internal/stack-catalog-setup.ts +++ b/apps/cli/src/command-internal/stack-catalog-setup.ts @@ -1,6 +1,7 @@ import { Context, Data, Effect, FileSystem, Layer, Path } from "effect"; import { type DatabaseInstance, type Stack } from "@supabase/stack/effect"; import { initialization, type InitializationCommand } from "@supabase/stack/commands"; +import { type ArtifactSelection } from "@supabase/stack/internal/artifacts"; import { Output } from "../shared/output/output.service.ts"; import { actionability, @@ -61,6 +62,7 @@ export interface StackCatalogSetupInput { readonly stack: Stack; readonly database: DatabaseInstance; readonly databaseServices: ReadonlyArray; + readonly selection: ArtifactSelection; }; readonly overlay: StackCatalogOverlay; } @@ -91,21 +93,29 @@ const credential = ( const serviceCommand = ( service: DatabaseService, + databaseId: string, credentials: ServiceCredentials, storagePath: string, + selection: ArtifactSelection, ): InitializationCommand => { switch (service) { case "auth": return initialization.auth({ + pin: selection.auth, + databaseId, databaseUrl: credentials.authDatabaseUrl, }); case "storage": return initialization.storage({ + pin: selection.storage, + databaseId, databaseUrl: credentials.storageDatabaseUrl, filePath: storagePath, }); case "realtime": return initialization.realtime({ + pin: selection.realtime, + databaseId, databaseUrl: credentials.databaseUrl, }); } @@ -143,7 +153,15 @@ const applyCatalog = Effect.fn("StackCatalogSetup.apply")(function* ( input.target.databaseServices, (service) => input.target.stack.commands - .run(serviceCommand(service, serviceCredentials, storagePath)) + .run( + serviceCommand( + service, + input.target.database.id, + serviceCredentials, + storagePath, + input.target.selection, + ), + ) .pipe( Effect.asVoid, Effect.mapError(catalogError), diff --git a/apps/cli/src/command-internal/stack-config.ts b/apps/cli/src/command-internal/stack-config.ts index cad545d5a0..2a843e74e1 100644 --- a/apps/cli/src/command-internal/stack-config.ts +++ b/apps/cli/src/command-internal/stack-config.ts @@ -1,3 +1,4 @@ +import { CommandSettings } from "../config/command-settings.service.ts"; import { getDefaultCliConfig, type CliConfig } from "@supabase/config"; import { resolveCliConfigSubtree } from "@supabase/config/internal"; import { validateCliConfig } from "@supabase/config/effect"; @@ -8,6 +9,7 @@ import { DEFAULT_SIGNING_KEY, } from "@supabase/stack/defaults"; import { type ServiceCreationInput as ServiceCreationType } from "@supabase/stack/effect"; +import { type ArtifactSelection } from "@supabase/stack/internal/artifacts"; import { Crypto, Effect, Data, FileSystem, Path, Redacted, Schema, SchemaIssue } from "effect"; import { FetchHttpClient } from "effect/http"; @@ -17,7 +19,12 @@ import { CLI_VERSION } from "../shared/cli/version.ts"; import { resolveAuthConfig } from "./stack-auth-config.ts"; import { parseGoDuration } from "./go-duration.ts"; import { parseFileSizeLimit } from "./storage-bucket-config.ts"; -import { stackDatabaseVersion } from "./stack-database-version.ts"; +import { stackArtifactSelection } from "./stack-artifact-selection.ts"; +import { + type LockRequest, + type LockResolutionError, + rejectFrozenWithoutConfig, +} from "./stack-lock.ts"; import { decryptAuthSecret, @@ -79,6 +86,8 @@ export class StackConfigError extends Data.TaggedError("StackConfigError")<{ } interface StackStartConfig { + /** The artifact pins every creation of this command carries. */ + readonly selection: ArtifactSelection; readonly creations: ( stackId: string, ) => Effect.Effect, StackConfigError>; @@ -102,12 +111,36 @@ interface StackStartConfig { >; } -type StackConfigEffect = Effect.Effect< +type StackConfigEffect = Effect.Effect< StackStartConfig, - StackConfigError, - FileSystem.FileSystem | Path.Path | RuntimeInfo | Crypto.Crypto + StackConfigError | LockResolutionError | E, + FileSystem.FileSystem | Path.Path | RuntimeInfo | Crypto.Crypto | CommandSettings | R >; +/** Host directory holding a stack's Storage objects, beside its runtime state. */ +export const stackUploadsDir = ( + path: Pick, + supabaseHome: string, + stackId: string, +): string => path.join(supabaseHome, "stacks", stackId, "uploads"); + +interface StackConfigOptions { + /** An already-loaded project context to reuse. */ + readonly context?: LocalProjectContext; + /** Chooses the artifact pins instead of the bundled ones; without a `supabase/config.toml` the bundled pins apply, or a frozen run fails. */ + readonly selection?: ( + request: LockRequest, + ) => Effect.Effect< + ArtifactSelection, + StackConfigError | LockResolutionError | E, + FileSystem.FileSystem | Path.Path | Crypto.Crypto | R + >; + /** Fails with `frozen-missing` when there is no `supabase/config.toml` to lock. */ + readonly frozen?: boolean; + /** Marks `selection` as publishing the lock, so the local auth keys must validate before it runs. */ + readonly publishesLock?: boolean; +} + const isRecord = (value: unknown): value is Readonly> => typeof value === "object" && value !== null && !Array.isArray(value); @@ -901,10 +934,12 @@ const endpoint = (port: number | undefined): { readonly port: number | "auto" } port: port === undefined ? "auto" : port, }); -/** Loads and translates the effective project config for all stack commands. - * Pass `opts.context` to reuse an already-loaded project context. */ +/** Loads and translates the effective project config for all stack commands. */ export const loadStackConfig = Effect.fn("StackConfig.load")( - (projectRoot: string, opts?: { readonly context?: LocalProjectContext }): StackConfigEffect => + ( + projectRoot: string, + opts?: StackConfigOptions, + ): StackConfigEffect => Effect.gen(function* () { const context = opts?.context ?? @@ -945,10 +980,8 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( const validationError = configValidationError(validatedConfig); if (validationError !== undefined) return yield* new StackConfigError({ message: validationError }); - const databaseVersion = yield* Effect.fromResult( - stackDatabaseVersion(validatedConfig.db), - ).pipe(Effect.mapError((message) => new StackConfigError({ message }))); - + if (context.loaded === null) + yield* rejectFrozenWithoutConfig(projectRoot, opts?.frozen === true); const externalProviders = yield* Effect.try({ try: () => resolveAuthExternalProviders( @@ -975,6 +1008,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( }, ); const path = yield* Path.Path; + const { supabaseHome } = yield* CommandSettings; const auth = validatedConfig.auth; const issuer = yield* Effect.try({ try: () => { @@ -988,55 +1022,57 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( message: cause instanceof Error ? cause.message : String(cause), }), }); - const localKeys = Effect.try({ - try: () => { - const configured = (value: string | undefined) => - value === undefined || value === "" - ? undefined - : decryptAuthSecret(value, context.projectEnvValues); - const publishableKey = configured(auth.publishable_key); - const secretKey = configured(auth.secret_key); - const configuredAnonKey = configured(auth.anon_key); - const configuredServiceRoleKey = configured(auth.service_role_key); - const configuredSigningKeys = resolveConfiguredSigningKeys( - validatedConfig, - projectRoot, - context.projectEnvValues, - ); - const signingKeys = - configuredSigningKeys ?? - (auth.signing_keys_path === undefined || auth.signing_keys_path.length === 0 - ? undefined - : [DEFAULT_SIGNING_KEY]); - const signingKey = signingKeys?.[0]; - return { - ...(publishableKey === undefined ? {} : { publishableKey }), - ...(secretKey === undefined ? {} : { secretKey }), - ...(configuredAnonKey === undefined - ? signingKey === undefined - ? {} - : { anonKey: generateAsymmetricGoJwt(signingKey, "anon") } - : { anonKey: configuredAnonKey }), - ...(configuredServiceRoleKey === undefined - ? signingKey === undefined + const localKeys = yield* Effect.cached( + Effect.try({ + try: () => { + const configured = (value: string | undefined) => + value === undefined || value === "" + ? undefined + : decryptAuthSecret(value, context.projectEnvValues); + const publishableKey = configured(auth.publishable_key); + const secretKey = configured(auth.secret_key); + const configuredAnonKey = configured(auth.anon_key); + const configuredServiceRoleKey = configured(auth.service_role_key); + const configuredSigningKeys = resolveConfiguredSigningKeys( + validatedConfig, + projectRoot, + context.projectEnvValues, + ); + const signingKeys = + configuredSigningKeys ?? + (auth.signing_keys_path === undefined || auth.signing_keys_path.length === 0 + ? undefined + : [DEFAULT_SIGNING_KEY]); + const signingKey = signingKeys?.[0]; + return { + ...(publishableKey === undefined ? {} : { publishableKey }), + ...(secretKey === undefined ? {} : { secretKey }), + ...(configuredAnonKey === undefined + ? signingKey === undefined + ? {} + : { anonKey: generateAsymmetricGoJwt(signingKey, "anon") } + : { anonKey: configuredAnonKey }), + ...(configuredServiceRoleKey === undefined + ? signingKey === undefined + ? {} + : { serviceRoleKey: generateAsymmetricGoJwt(signingKey, "service_role") } + : { serviceRoleKey: configuredServiceRoleKey }), + anonKeyIsOverride: configuredAnonKey !== undefined, + serviceRoleKeyIsOverride: configuredServiceRoleKey !== undefined, + ...(signingKeys === undefined ? {} - : { serviceRoleKey: generateAsymmetricGoJwt(signingKey, "service_role") } - : { serviceRoleKey: configuredServiceRoleKey }), - anonKeyIsOverride: configuredAnonKey !== undefined, - serviceRoleKeyIsOverride: configuredServiceRoleKey !== undefined, - ...(signingKeys === undefined - ? {} - : { - gotrueJwtKeys: encodeJwkArray(signingKeys), - publicSigningKeys: encodeJwkArray(signingKeys.map(toPublicJwk)), - }), - }; - }, - catch: (cause) => - new StackConfigError({ - message: cause instanceof Error ? cause.message : String(cause), - }), - }); + : { + gotrueJwtKeys: encodeJwkArray(signingKeys), + publicSigningKeys: encodeJwkArray(signingKeys.map(toPublicJwk)), + }), + }; + }, + catch: (cause) => + new StackConfigError({ + message: cause instanceof Error ? cause.message : String(cause), + }), + }), + ); const remoteJwks = Effect.gen(function* () { return issuer === undefined ? undefined @@ -1208,9 +1244,23 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ); const poolMode = validatedConfig.db.pooler.pool_mode === "session" ? ("session" as const) : "transaction"; - const storagePath = `${projectRoot}/supabase/.temp/stack-uploads`; + const selection = yield* context.loaded !== null && opts?.selection !== undefined + ? (opts.publishesLock === true ? localKeys : Effect.void).pipe( + Effect.andThen( + opts.selection({ + projectRoot, + majorVersion: validatedConfig.db.major_version, + orioledbVersion: validatedConfig.db.orioledb_version, + services: validatedConfig.local?.services, + }), + ), + ) + : stackArtifactSelection(validatedConfig.db).pipe( + Effect.mapError((error) => new StackConfigError({ message: error.message })), + ); const pgmetaCreation: ServiceCreationType = { service: "pgmeta", + pin: selection.pgmeta, config: {}, endpoints: { http: endpoint(undefined) }, }; @@ -1221,8 +1271,8 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( return [ { service: "database", + pin: selection.database, config: { - version: databaseVersion, ...(jwtSecret === undefined ? {} : { jwtSecret }), jwtExpiry: validatedConfig.auth.jwt_expiry, settings: validatedConfig.db.settings, @@ -1235,6 +1285,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "analytics" as const, + pin: selection.analytics, config: { backend: "postgres" as const, apiKey: "api-key", @@ -1247,6 +1298,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "pooler" as const, + pin: selection.pooler, config: { ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.value(jwtSecret) }), poolMode, @@ -1263,6 +1315,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "rest" as const, + pin: selection.rest, config: { schemas: validatedConfig.api.schemas.join(","), extraSearchPath: validatedConfig.api.extra_search_path.join(","), @@ -1280,6 +1333,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "auth" as const, + pin: selection.auth, config: { ...authConfig, ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.value(jwtSecret) }), @@ -1292,6 +1346,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "realtime" as const, + pin: selection.realtime, config: { ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.value(jwtSecret) }), ipVersion: @@ -1311,9 +1366,10 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "storage" as const, + pin: selection.storage, config: { ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.value(jwtSecret) }), - filePath: `${storagePath}/${stackId}`, + filePath: stackUploadsDir(path, supabaseHome, stackId), fileSizeLimit: storageFileSizeLimit, s3ProtocolEnabled: validatedConfig.storage.s3_protocol.enabled, s3AccessKeyId: DEFAULT_LOCAL_S3_ACCESS_KEY_ID, @@ -1331,7 +1387,8 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "imgproxy" as const, - config: { filePath: `${storagePath}/${stackId}` }, + pin: selection.imgproxy, + config: { filePath: stackUploadsDir(path, supabaseHome, stackId) }, endpoints: { http: endpoint(undefined) }, } satisfies ServiceCreationType, ] @@ -1340,6 +1397,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "functions" as const, + pin: selection.functions, config: { functionsRoot: `${projectRoot}/supabase/functions`, filesRoot: projectRoot, @@ -1367,6 +1425,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "studio" as const, + pin: selection.studio, config: { snippetsRoot: `${projectRoot}/supabase/snippets`, apiSchemas: validatedConfig.api.schemas.join(","), @@ -1389,6 +1448,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ? [ { service: "mail" as const, + pin: selection.mail, config: {}, endpoints: { http: endpoint(mailPort), @@ -1401,6 +1461,7 @@ export const loadStackConfig = Effect.fn("StackConfig.load")( ]; }); return { + selection, creations: createCreations, source: validatedConfig, projectEnvValues: context.projectEnvValues, diff --git a/apps/cli/src/command-internal/stack-data-conflicts.ts b/apps/cli/src/command-internal/stack-data-conflicts.ts new file mode 100644 index 0000000000..95346802e5 --- /dev/null +++ b/apps/cli/src/command-internal/stack-data-conflicts.ts @@ -0,0 +1,43 @@ +import { findDataConflicts } from "@supabase/stack/effect"; +import type { ArtifactSelection } from "@supabase/stack/internal/artifacts"; +import { Cause, Effect, Path } from "effect"; + +import { CommandSettings } from "../config/command-settings.service.ts"; +import type { Output } from "../shared/output/output.service.ts"; + +/** + * Warns once per data conflict a start at `selection` would meet in the project's saved stacks, and + * returns how many it found; state that cannot be read warns of nothing and is only logged. + */ +export const warnDataConflicts = Effect.fn("StackLock.dataConflicts")( + function* (options: { + readonly projectRoot: string; + readonly selection: ArtifactSelection; + readonly output: Output["Service"]; + }) { + const path = yield* Path.Path; + const { supabaseHome } = yield* CommandSettings; + const conflicts = yield* findDataConflicts({ + stateRoot: path.join(supabaseHome, "stacks"), + projectRoot: options.projectRoot, + requested: options.selection, + }); + yield* Effect.annotateCurrentSpan({ "stack.data_conflicts.count": conflicts.length }); + yield* Effect.forEach( + conflicts, + (conflict) => + options.output.warn( + `stack ${conflict.stackId}: the next supabase start refuses these pins: ${conflict.message}`, + ), + { discard: true }, + ); + return conflicts.length; + }, + Effect.catchCause((cause) => + Cause.hasInterruptsOnly(cause) + ? Effect.interrupt + : Effect.logDebug(`stack data conflict check skipped: ${Cause.pretty(cause)}`).pipe( + Effect.as(0), + ), + ), +); diff --git a/apps/cli/src/command-internal/stack-database-version.ts b/apps/cli/src/command-internal/stack-database-version.ts deleted file mode 100644 index a64ff429fc..0000000000 --- a/apps/cli/src/command-internal/stack-database-version.ts +++ /dev/null @@ -1,31 +0,0 @@ -import { - orioledbPostgresVersion, - orioledbVersions, - postgresMajor, -} from "@supabase/stack/internal/artifacts"; -import { Result } from "effect"; - -/** - * Database artifact version the stack runs for a project's `[db]` config: the major alias, or the - * OrioleDB build named by `db.orioledb_version`, which must be pinned in the artifact catalog. - */ -export const stackDatabaseVersion = ( - db: { readonly major_version: number; readonly orioledb_version?: string | undefined }, - supported: ReadonlyArray = orioledbVersions(), -): Result.Result => { - const orioledb = db.orioledb_version; - if (orioledb === undefined || orioledb.length === 0) - return Result.succeed(String(db.major_version)); - if (!supported.includes(orioledb)) - return Result.fail( - `db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = ${orioledb} is not an OrioleDB build this CLI ships for the experimental stack; it ships: ${ - supported.length === 0 ? "none" : supported.join(", ") - }. A stack keeps the build it was created with, so moving an existing stack to another build means recreating it with supabase stack destroy, which permanently deletes its local database data`, - ); - const major = postgresMajor(orioledb); - if (major !== String(db.major_version)) - return Result.fail( - `db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = ${orioledb} requires db.major_version (or SUPABASE_DB_MAJOR_VERSION) = ${major} for the experimental stack`, - ); - return Result.succeed(orioledbPostgresVersion(orioledb)); -}; diff --git a/apps/cli/src/command-internal/stack-database-version.unit.test.ts b/apps/cli/src/command-internal/stack-database-version.unit.test.ts deleted file mode 100644 index 547a549a7a..0000000000 --- a/apps/cli/src/command-internal/stack-database-version.unit.test.ts +++ /dev/null @@ -1,48 +0,0 @@ -import { describe, expect, it } from "@effect/vitest"; -import { orioledbVersions, postgresMajor } from "@supabase/stack/internal/artifacts"; -import { Result } from "effect"; - -import { stackDatabaseVersion } from "./stack-database-version.ts"; - -const published = ["17.11.0.002"]; - -describe("stackDatabaseVersion", () => { - it("runs the configured major alias when OrioleDB is unset or empty", () => { - expect(stackDatabaseVersion({ major_version: 17 }, published)).toEqual(Result.succeed("17")); - expect(stackDatabaseVersion({ major_version: 15, orioledb_version: "" }, published)).toEqual( - Result.succeed("15"), - ); - }); - - it("routes every OrioleDB version the artifact catalog pins", () => { - const pinned = orioledbVersions(); - expect(pinned.length).toBeGreaterThan(0); - for (const version of pinned) - expect( - stackDatabaseVersion({ - major_version: Number(postgresMajor(version)), - orioledb_version: version, - }), - ).toEqual(Result.succeed(`${version}-orioledb`)); - }); - - it("rejects an OrioleDB version whose major differs from db.major_version", () => { - expect( - stackDatabaseVersion({ major_version: 15, orioledb_version: "17.11.0.002" }, published), - ).toEqual( - Result.fail( - "db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = 17.11.0.002 requires db.major_version (or SUPABASE_DB_MAJOR_VERSION) = 17 for the experimental stack", - ), - ); - }); - - it("fails closed on an unpublished OrioleDB version, listing the published ones", () => { - expect( - stackDatabaseVersion({ major_version: 17, orioledb_version: "17.6.1.000" }, published), - ).toEqual( - Result.fail( - "db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = 17.6.1.000 is not an OrioleDB build this CLI ships for the experimental stack; it ships: 17.11.0.002. A stack keeps the build it was created with, so moving an existing stack to another build means recreating it with supabase stack destroy, which permanently deletes its local database data", - ), - ); - }); -}); diff --git a/apps/cli/src/command-internal/stack-local-database.ts b/apps/cli/src/command-internal/stack-local-database.ts index 76e66357b1..ed610c193c 100644 --- a/apps/cli/src/command-internal/stack-local-database.ts +++ b/apps/cli/src/command-internal/stack-local-database.ts @@ -1,5 +1,6 @@ import { Data, Effect, FileSystem, Option, Path, Redacted } from "effect"; import type { DatabaseInstance, ServiceCreationInput, Stack } from "@supabase/stack/effect"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; import { actionability, type CliErrorActionabilityDeclaration, @@ -12,6 +13,7 @@ import { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; import { currentStackBackend } from "./stack-backend.ts"; import { StackApi } from "./stack-api.ts"; import { loadStackConfig } from "./stack-config.ts"; +import { lockedSelection } from "./stack-lock.ts"; import { readDbToml } from "./db-config.toml-read.ts"; import { catalogDatabaseServices } from "./stack-catalog-setup.ts"; import { resolveExperimentalWithProjectEnv } from "./global-flags.ts"; @@ -144,9 +146,9 @@ export const stackRequireProjectRuntime: Effect.Effect< ), ); -/** Reads the configured database version from the saved database definition. */ -export const stackProjectDatabaseVersion: Effect.Effect< - string | undefined, +/** Reads the database artifact pin from the saved database definition. */ +export const stackProjectDatabasePin: Effect.Effect< + ArtifactPin | undefined, never, CommandSettings | StackApi | Path.Path > = Effect.gen(function* () { @@ -161,7 +163,7 @@ export const stackProjectDatabaseVersion: Effect.Effect< ); if (database === undefined) return undefined; const status = yield* database.status.pipe(Effect.option, Effect.map(Option.getOrUndefined)); - return status?.config.service === "database" ? status.config.config.version : undefined; + return status?.config.service === "database" ? status.config.pin : undefined; }).pipe(Effect.scoped); export class StackNativeEngineError extends Data.TaggedError("StackNativeEngineError")<{ @@ -225,7 +227,9 @@ export const stackEnsurePostgresOnlyStarted = Effect.fn( const path = yield* Path.Path; const fs = yield* FileSystem.FileSystem; const output = yield* Output; - const config = yield* loadStackConfig(settings.workdir).pipe(Effect.mapError(startFailed)); + const config = yield* loadStackConfig(settings.workdir, { + selection: lockedSelection({ frozen: false, output }), + }).pipe(Effect.mapError(startFailed)); const toml = yield* readDbToml(fs, path, settings.workdir).pipe(Effect.mapError(startFailed)); const experimental = yield* resolveExperimentalWithProjectEnv({ ...toml.projectEnv }); const existing = yield* stackForProject(api, settings, path).pipe(Effect.mapError(startFailed)); @@ -281,11 +285,11 @@ export const stackEnsurePostgresOnlyStarted = Effect.fn( (entry) => entry.id === currentDatabase.id && entry.change === "incompatible" && - entry.paths.includes("config.version"), + entry.paths.includes("pin"), ) ) return yield* startFailed({ - message: "The requested database version does not match the saved stack binding", + message: `The requested database version does not match the saved stack binding. Recreate the local database: supabase stack destroy --stack-id ${stack.id}`, }); const savedCredentials = yield* stack.credentials.get.pipe(Effect.mapError(startFailed)); if ( @@ -304,9 +308,28 @@ export const stackEnsurePostgresOnlyStarted = Effect.fn( ); return "already-running" as const; } - yield* currentDatabase.start.pipe(Effect.mapError(startFailed)); - yield* currentDatabase.ready.pipe(Effect.mapError(startFailed)); - yield* applyStackWebhooksOnly(currentDatabase, toml.webhooksEnabled).pipe( + const pinChanged = planned.some( + (entry) => + entry.id === currentDatabase.id && + entry.change === "changed" && + entry.paths.includes("pin"), + ); + // A stopped database starts the selected pin; the instance id and its data stay. + const database = pinChanged + ? databaseFor( + yield* stack.composition + .supabase([databaseCreation], { + reuseIds: [currentDatabase.id], + retainMembers: true, + }) + .pipe(Effect.mapError(startFailed)), + ) + : currentDatabase; + if (database === undefined) + return yield* startFailed({ message: "stack did not rebind the database instance" }); + yield* database.start.pipe(Effect.mapError(startFailed)); + yield* database.ready.pipe(Effect.mapError(startFailed)); + yield* applyStackWebhooksOnly(database, toml.webhooksEnabled).pipe( Effect.mapError(startFailed), ); return "started" as const; @@ -330,7 +353,12 @@ export const stackEnsurePostgresOnlyStarted = Effect.fn( if (credentials === undefined) return yield* startFailed({ message: "stack credentials were not saved" }); yield* initializeStackDatabase({ - target: { stack, database, databaseServices: catalogDatabaseServices(creations) }, + target: { + stack, + database, + databaseServices: catalogDatabaseServices(creations), + selection: config.selection, + }, overlay: projectCatalogOverlay(toml, settings.workdir), migrations: { workdir: settings.workdir, toml, experimental }, }).pipe(Effect.mapError(startFailed)); diff --git a/apps/cli/src/command-internal/stack-lock.integration.test.ts b/apps/cli/src/command-internal/stack-lock.integration.test.ts new file mode 100644 index 0000000000..d5f5aef027 --- /dev/null +++ b/apps/cli/src/command-internal/stack-lock.integration.test.ts @@ -0,0 +1,243 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { LOCK_SCHEMA_URL, writeLock, type Lock } from "@supabase/stack/effect"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; +import { Crypto, Effect, FileSystem, Path } from "effect"; + +import { + database15, + orioledb, + orioledbConfigVersion, + selection, +} from "../../tests/helpers/pins.ts"; +import { StackConfigError } from "./stack-config.ts"; +import { + LockResolutionError, + outdatedServices, + readSelectionOrBundled, + type LockRequest, +} from "./stack-lock.ts"; +import { selectionForRequest } from "./stack-project-selection.ts"; + +const inProject = ( + body: ( + projectRoot: string, + ) => Effect.Effect, +) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + return yield* body(yield* fs.makeTempDirectoryScoped({ prefix: "stack-lock-int-" })); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); + +const requestFor = ( + projectRoot: string, + overrides: Partial> = {}, +): LockRequest => ({ + projectRoot, + majorVersion: 17, + orioledbVersion: undefined, + services: undefined, + ...overrides, +}); + +const orioledbBuild = `${orioledb.upstreamVersion}-r${orioledb.revision}`; + +describe("readSelectionOrBundled without a lock", () => { + it.live("selects the bundled OrioleDB build a [local.services] database version names", () => + inProject((projectRoot) => + Effect.gen(function* () { + const chosen = yield* readSelectionOrBundled( + requestFor(projectRoot, { services: { database: { version: orioledbBuild } } }), + ); + + expect(chosen.database).toEqual(orioledb); + expect(chosen.rest).toEqual(selection.rest); + }), + ), + ); + + it.live( + "fails with the missing-lock error for an upstream version this CLI does not bundle", + () => + inProject((projectRoot) => + Effect.gen(function* () { + const error = yield* Effect.flip( + readSelectionOrBundled( + requestFor(projectRoot, { services: { auth: { version: "v99.0.0" } } }), + ), + ); + + expect(error).toBeInstanceOf(LockResolutionError); + expect(error.message).toBe( + "supabase/supabase.lock not found; run supabase start to create it", + ); + }), + ), + ); + + it.live("fails for a database revision this CLI does not bundle", () => + inProject((projectRoot) => + Effect.gen(function* () { + const error = yield* Effect.flip( + readSelectionOrBundled( + requestFor(projectRoot, { + services: { + database: { version: `${orioledb.upstreamVersion}-r${orioledb.revision + 1}` }, + }, + }), + ), + ); + + expect(error).toBeInstanceOf(LockResolutionError); + }), + ), + ); + + it.live("fails when the OrioleDB alias and the database version name different builds", () => + inProject((projectRoot) => + Effect.gen(function* () { + const error = yield* Effect.flip( + readSelectionOrBundled( + requestFor(projectRoot, { + orioledbVersion: orioledbConfigVersion, + services: { database: { version: "17.6.1.000" } }, + }), + ), + ); + + expect(error).toBeInstanceOf(StackConfigError); + expect(error.message).toContain("conflicts with local.services.database.version"); + }), + ), + ); +}); + +const linkedEntry = (resolved: ArtifactPin) => + ({ specifier: "linked", resolution: "unlinked", resolved }) as const; + +const orioledbLock = (specifier: string, resolved: ArtifactPin): Lock => ({ + $schema: LOCK_SCHEMA_URL, + lockfileVersion: 1, + services: { + database: { specifier, resolution: "resolved", resolved, majorVersion: 17, line: "orioledb" }, + rest: linkedEntry(selection.rest), + auth: linkedEntry(selection.auth), + realtime: linkedEntry(selection.realtime), + storage: linkedEntry(selection.storage), + imgproxy: linkedEntry(selection.imgproxy), + functions: linkedEntry(selection.functions), + studio: linkedEntry(selection.studio), + pgmeta: linkedEntry(selection.pgmeta), + mail: linkedEntry(selection.mail), + analytics: linkedEntry(selection.analytics), + pooler: linkedEntry(selection.pooler), + }, +}); + +describe("a locked OrioleDB build", () => { + it.live("is reported with the bundled OrioleDB pin, not the stock one", () => + inProject((projectRoot) => + Effect.gen(function* () { + yield* writeLock(projectRoot, orioledbLock(orioledbBuild, orioledb)); + + const rows = yield* outdatedServices( + requestFor(projectRoot, { services: { database: { version: orioledbBuild } } }), + ); + + expect(rows.find(({ service }) => service === "database")?.bundled).toBe(orioledbBuild); + }), + ), + ); +}); + +describe("selectionForRequest with a stale lock", () => { + const staleRequest = (projectRoot: string) => + requestFor(projectRoot, { services: { database: { version: orioledbBuild } } }); + + it.live("falls back to the bundled pins of the merged request when a remote applied", () => + inProject((projectRoot) => + Effect.gen(function* () { + yield* writeLock(projectRoot, orioledbLock(orioledb.upstreamVersion, orioledb)); + + const chosen = yield* selectionForRequest({ + ...staleRequest(projectRoot), + appliedRemote: "remote-ref", + }); + + expect(chosen.database).toEqual(orioledb); + expect(chosen.rest).toEqual(selection.rest); + }), + ), + ); + + const pinnedAuth = { + ...selection.auth, + upstreamVersion: "v2.1.0", + image: selection.auth.image.replace(`:${selection.auth.upstreamVersion}-`, ":v2.1.0-"), + }; + const remoteMajorRequest = (projectRoot: string, auth: string) => ({ + ...requestFor(projectRoot, { majorVersion: 15, services: { auth: { version: auth } } }), + appliedRemote: "remote-ref", + }); + const stockLockWithAuth = (auth: Lock["services"]["auth"]): Lock => ({ + ...orioledbLock(orioledb.upstreamVersion, orioledb), + services: { + ...orioledbLock(orioledb.upstreamVersion, orioledb).services, + database: { + specifier: "linked", + resolution: "unlinked", + resolved: selection.database, + majorVersion: 17, + line: "stock", + }, + auth, + }, + }); + + it.live("keeps a fresh locked pin the CLI does not bundle while the database falls back", () => + inProject((projectRoot) => + Effect.gen(function* () { + yield* writeLock( + projectRoot, + stockLockWithAuth({ specifier: "v2.1.0", resolution: "resolved", resolved: pinnedAuth }), + ); + + const chosen = yield* selectionForRequest(remoteMajorRequest(projectRoot, "v2.1.0")); + + expect(chosen.database).toEqual(database15); + expect(chosen.auth).toEqual(pinnedAuth); + }), + ), + ); + + it.live("reports a stale entry the CLI does not bundle as stale", () => + inProject((projectRoot) => + Effect.gen(function* () { + yield* writeLock(projectRoot, stockLockWithAuth(linkedEntry(selection.auth))); + + const error = yield* Effect.flip( + selectionForRequest(remoteMajorRequest(projectRoot, "v2.1.0")), + ); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "frozen-stale" }); + expect(error.message).toBe( + "supabase/supabase.lock is stale for: auth (specifier v2.1.0 ≠ locked linked); run supabase start to refresh it", + ); + }), + ), + ); + + it.live("still fails when no remote applied", () => + inProject((projectRoot) => + Effect.gen(function* () { + yield* writeLock(projectRoot, orioledbLock(orioledb.upstreamVersion, orioledb)); + + const error = yield* Effect.flip( + selectionForRequest({ ...staleRequest(projectRoot), appliedRemote: undefined }), + ); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "frozen-stale" }); + }), + ), + ); +}); diff --git a/apps/cli/src/command-internal/stack-lock.ts b/apps/cli/src/command-internal/stack-lock.ts new file mode 100644 index 0000000000..b44b79db70 --- /dev/null +++ b/apps/cli/src/command-internal/stack-lock.ts @@ -0,0 +1,1431 @@ +import type { CliConfig } from "@supabase/config/effect"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { + holdLock, + isLockBusy, + Lock, + LOCK_SCHEMA_URL, + lockSelection, + readLock, + writeLock, + type LockInvalidError, + type LockIoError, + type LockUnsupportedVersionError, + type NamespaceError, +} from "@supabase/stack/effect"; +import { + artifactServiceKinds, + catalogPins, + databaseArtifactPin, + isOrioledbVersion, + orioledbPostgresVersion, + postgresMajor, + resolvePin, + type ArtifactPin, + type ArtifactSelection, + type ReleaseError, + type ServiceKind, +} from "@supabase/stack/internal/artifacts"; +import { + Clock, + Config, + Crypto, + DateTime, + Effect, + FileSystem, + Option, + Path, + Record, + Result, + Schema, +} from "effect"; +import { FetchHttpClient } from "effect/http"; + +import { ensureGitignore, readProjectLink } from "./project-link.ts"; +import { stackArtifactSelection } from "./stack-artifact-selection.ts"; +import { StackConfigError } from "./stack-config.ts"; +import { warnDataConflicts } from "./stack-data-conflicts.ts"; +import type { HostedObservation } from "./tenant-versions.ts"; +import type { CommandSettings } from "../config/command-settings.service.ts"; +import type { Output } from "../shared/output/output.service.ts"; +import { + actionability, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../shared/telemetry/error-actionability.ts"; + +/** The project lock, relative to the project root. */ +export const LOCK_FILE = "supabase/supabase.lock"; +const LEASE_FILE = ".supabase.lock.lease"; +const LEASE_BUDGET = "60 seconds"; + +type NonDatabaseKind = Exclude; +type DatabaseLock = Lock["services"]["database"]; +type ServiceLock = Lock["services"]["rest"]; +type LocalServices = NonNullable["services"]>; + +/** The config inputs that decide which versions the lock pins. */ +export interface LockRequest { + readonly projectRoot: string; + readonly majorVersion: number; + readonly orioledbVersion: string | undefined; + readonly services: LocalServices | undefined; +} + +/** A parsed `[local.services]` version specifier. */ +type Specifier = + | { readonly kind: "linked" | "bundled" } + | { readonly kind: "release"; readonly upstream: string; readonly revision?: number }; + +interface ServiceRequest { + readonly specifier: string; + readonly parsed: Specifier; +} + +type ServiceRequests = Readonly>; + +const REVISION_SUFFIX = /^(.+)-r(0|[1-9][0-9]*)$/u; + +/** Splits `linked`, `bundled`, `` or `-rN`; the config schema has already validated the form. */ +export const parseSpecifier = (text: string): Specifier => { + if (text === "linked" || text === "bundled") return { kind: text }; + const match = REVISION_SUFFIX.exec(text); + const upstream = match?.[1]; + const revision = match?.[2]; + return upstream === undefined || revision === undefined + ? { kind: "release", upstream: text } + : { kind: "release", upstream, revision: Number(revision) }; +}; + +/** + * The effective specifier of every service: its own entry, else `default`, else `linked`. The + * deprecated `[db] orioledb_version` is a `database` specifier for that OrioleDB build. + */ +const serviceRequests = (request: LockRequest): Result.Result => { + const fallback = request.services?.default?.version ?? "linked"; + const own = request.services?.database?.version; + const alias = + request.orioledbVersion === undefined || request.orioledbVersion.length === 0 + ? undefined + : orioledbPostgresVersion(request.orioledbVersion); + const parsedOwn = own === undefined ? undefined : parseSpecifier(own); + if ( + alias !== undefined && + parsedOwn !== undefined && + !(parsedOwn.kind === "release" && parsedOwn.upstream === alias) + ) + return Result.fail( + `db.orioledb_version (or SUPABASE_DB_ORIOLEDB_VERSION) = ${request.orioledbVersion} conflicts with local.services.database.version = "${own}"; remove one of them`, + ); + const database = own ?? alias ?? fallback; + const parsedDatabase = parseSpecifier(database); + if ( + parsedDatabase.kind === "release" && + postgresMajor(parsedDatabase.upstream) !== String(request.majorVersion) + ) + return Result.fail( + `the database version "${database}" requires db.major_version (or SUPABASE_DB_MAJOR_VERSION) = ${postgresMajor(parsedDatabase.upstream)}, but it is ${request.majorVersion}`, + ); + return Result.succeed( + Record.map(defaultArtifactSelection(), (_pin, kind) => { + const specifier = + kind === "database" ? database : (request.services?.[kind]?.version ?? fallback); + return { specifier, parsed: parseSpecifier(specifier) }; + }), + ); +}; + +const pinLine = (pin: ArtifactPin) => + isOrioledbVersion(pin.upstreamVersion) ? "orioledb" : "stock"; + +/** + * Why a locked entry no longer answers its request, or `undefined` when it still does. Only + * resolution inputs count: the specifier, and for the database its major and line, which a + * hand-edited pin could contradict while the specifier still matches. + */ +const staleReason = ( + kind: ServiceKind, + entry: ServiceLock | DatabaseLock, + request: ServiceRequest, + majorVersion: number, +): string | undefined => { + if (entry.specifier !== request.specifier) + return `specifier ${request.specifier} ≠ locked ${entry.specifier}`; + if (kind !== "database" || !("majorVersion" in entry)) return undefined; + if (entry.majorVersion !== majorVersion) + return `db.major_version ${majorVersion} ≠ locked ${entry.majorVersion}`; + if (postgresMajor(entry.resolved.upstreamVersion) !== String(majorVersion)) + return `locked pin ${entry.resolved.upstreamVersion} is not Postgres ${majorVersion}`; + const line = pinLine(entry.resolved); + const requested = + request.parsed.kind === "release" + ? isOrioledbVersion(request.parsed.upstream) + ? "orioledb" + : "stock" + : request.parsed.kind === "bundled" + ? "stock" + : line; + return entry.line !== line || requested !== line + ? `database line ${requested} ≠ locked ${entry.line}` + : undefined; +}; + +/** The entries whose request changed since the lock was written; every entry when there is no lock. */ +const staleServices = ( + lock: Lock | undefined, + requests: ServiceRequests, + majorVersion: number, +): ReadonlyMap => + new Map( + artifactServiceKinds().flatMap((kind): Array<[ServiceKind, string]> => { + const entry = lock?.services[kind]; + const reason = + entry === undefined + ? "not locked yet" + : staleReason(kind, entry, requests[kind], majorVersion); + return reason === undefined ? [] : [[kind, reason]]; + }), + ); + +/** Why the project lock could not be read, resolved, or written. */ +export class LockResolutionError extends Schema.TaggedError()( + "LockResolutionError", + { + reason: Schema.Literals([ + "unpublished", + "transient", + "integrity", + "invalid-lock", + "frozen-missing", + "frozen-stale", + "lease-busy", + "io", + ]), + message: Schema.String, + }, +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + switch (this.reason) { + case "unpublished": + return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + case "transient": + return { ...actionability.externalNetwork, fingerprint_suffix: "network" }; + case "integrity": + return { ...actionability.externalNetwork, fingerprint_suffix: "asset_checksum" }; + case "invalid-lock": + return { ...actionability.invalidConfig, fingerprint_suffix: "state_file" }; + case "frozen-missing": + return { ...actionability.startStack, fingerprint_suffix: "not_found" }; + case "frozen-stale": + return { ...actionability.startStack, fingerprint_suffix: "conflict" }; + case "lease-busy": + return { ...actionability.invalidInput, fingerprint_suffix: "lease_held" }; + case "io": + return { ...actionability.runtimeCrash, fingerprint_suffix: "filesystem_io" }; + } + } +} + +const lockFailure = ( + error: LockInvalidError | LockUnsupportedVersionError | LockIoError, +): LockResolutionError => + error._tag === "LockIoError" + ? new LockResolutionError({ reason: "io", message: error.message }) + : new LockResolutionError({ + reason: "invalid-lock", + message: + error._tag === "LockInvalidError" + ? `${error.message}; fix it, or delete it and run supabase start to resolve the versions again` + : error.message, + }); + +const RELEASE_OUTCOMES = { + ReleaseUnpublished: { reason: "unpublished", text: "no published slim-services release" }, + ReleaseTransient: { + reason: "transient", + text: "the slim-services release host could not be reached; retry shortly", + }, + ReleaseIntegrity: { reason: "integrity", text: "the published release failed verification" }, +} as const; + +const releaseFailure = ( + kind: ServiceKind, + specifier: string, + error: ReleaseError, +): LockResolutionError => { + const outcome = RELEASE_OUTCOMES[error._tag]; + return new LockResolutionError({ + reason: outcome.reason, + message: `local.services.${kind}.version = "${specifier}": ${outcome.text} (${error.message})`, + }); +}; + +const leaseFailure = (error: NamespaceError): LockResolutionError => + new LockResolutionError({ + reason: isLockBusy(error) ? "lease-busy" : "io", + message: error.message, + }); + +/** The pin compiled into this CLI for a service; the database's depends on the Postgres major. */ +const bundledPin = (kind: ServiceKind, majorVersion: number) => + kind === "database" + ? databaseArtifactPin(String(majorVersion)).pipe( + Effect.mapError((error) => new StackConfigError({ message: error.message })), + ) + : Effect.succeed(defaultArtifactSelection()[kind]); + +/** + * The pin this CLI bundles to compare a locked entry with. A locked database build is matched by + * its exact upstream version, stock or OrioleDB; one the catalog does not pin takes the major's pin. + */ +const bundledPinFor = (kind: ServiceKind, locked: ArtifactPin, majorVersion: number) => + kind === "database" + ? databaseArtifactPin(locked.upstreamVersion).pipe( + Effect.catch(() => bundledPin(kind, majorVersion)), + ) + : bundledPin(kind, majorVersion); + +type LinkedVersion = NonNullable; +type LinkedRemote = NonNullable; + +const OBSERVABLE_KINDS = ["database", "rest", "auth", "storage"] as const; + +/** The services whose version the platform reports; a probe observes these and no others. */ +type ObservableKind = (typeof OBSERVABLE_KINDS)[number]; + +const isObservable = (kind: ServiceKind): kind is ObservableKind => + OBSERVABLE_KINDS.some((observable) => observable === kind); + +/** What a hosted probe learned: its remote, and what the platform reported for each observable service. */ +export interface HostedProbe { + readonly remote: Pick; + readonly observations: Readonly>; +} + +const probedObservation = (probe: HostedProbe, kind: ServiceKind): HostedObservation | undefined => + isObservable(kind) ? probe.observations[kind] : undefined; + +/** Where `linked` specifiers take the hosted versions they follow from. */ +interface LinkedSource { + /** The remote the observations belong to; absent when the project was never linked. */ + readonly remote: LinkedRemote | undefined; + /** Whether the previous lock observed this same remote, so its entries may be kept. */ + readonly sameRemote: boolean; + readonly observation: (kind: ServiceKind) => HostedObservation; + /** Whether the observations come from a fresh probe; entries that do not follow `linked` then take them too. */ + readonly probed: boolean; + /** `stack update` moves fallback pins to this CLI's; `link` and `start` keep a fallback that has not changed. */ + readonly refresh: boolean; +} + +const UNREPORTED: HostedObservation = { _tag: "Unreported" }; + +/** The observations the previous lock stored for its own remote, without probing. */ +const storedSource = (previous: Lock | undefined, refresh: boolean): LinkedSource => ({ + remote: previous?.linked, + sameRemote: true, + observation: (kind) => { + const stored = previous?.services[kind].linked; + return stored === undefined ? UNREPORTED : { _tag: "Observed", ...stored }; + }, + probed: false, + refresh, +}); + +const probedSource = ( + previous: Lock | undefined, + probe: HostedProbe, + observedAt: string, + refresh: boolean, +): LinkedSource => ({ + remote: { ...probe.remote, observedAt }, + sameRemote: + previous?.linked !== undefined && + previous.linked.project === probe.remote.project && + previous.linked.branch.ref === probe.remote.branch.ref, + observation: (kind) => probedObservation(probe, kind) ?? UNREPORTED, + probed: true, + refresh, +}); + +const observedVersion = (observed: HostedObservation): LinkedVersion | undefined => + observed._tag === "Observed" ? { raw: observed.raw, version: observed.version } : undefined; + +/** The hosted version to record on an entry; a failed probe keeps the previous one only for the same remote. */ +const linkedObservation = ( + kind: ServiceKind, + previous: ServiceLock | DatabaseLock | undefined, + source: LinkedSource, +): LinkedVersion | undefined => { + const observed = source.observation(kind); + return observed._tag === "Failed" + ? source.sameRemote + ? previous?.linked + : undefined + : observedVersion(observed); +}; + +interface ResolvedPin { + readonly pin: ArtifactPin; + readonly resolution: ServiceLock["resolution"]; + readonly linked: LinkedVersion | undefined; + /** A fallback or kept entry the user should hear about. */ + readonly notice: string | undefined; + /** Whether the previous entry was kept rather than resolved again. */ + readonly kept: boolean; + /** The release host outage this result stands in for; the pin is what the entry falls back to. */ + readonly outage?: LockResolutionError; +} + +const releasePin = (kind: ServiceKind, upstream: string, revision: number | undefined) => + resolvePin({ + service: kind, + upstream, + ...(revision === undefined ? {} : { revision }), + }).pipe( + Effect.provide(FetchHttpClient.layer), + Effect.mapError((error) => + error._tag === "ConfigError" + ? new StackConfigError({ + message: `failed to read the slim-services endpoints: ${error.message}`, + }) + : error, + ), + ); + +/** + * Resolves a `linked` request from the hosted observation. Without a remote or an observable + * version it takes the bundled pin; a transient failure keeps the previous entry only when the + * remote and every resolution input are unchanged. + */ +const resolveLinked = Effect.fn("StackLock.resolveLinked")(function* ( + kind: ServiceKind, + request: ServiceRequest, + majorVersion: number, + previous: ServiceLock | DatabaseLock | undefined, + source: LinkedSource, +) { + const observed = source.observation(kind); + const linked = linkedObservation(kind, previous, source); + const bundled = yield* bundledPin(kind, majorVersion); + const safe = + source.sameRemote && + previous !== undefined && + staleReason(kind, previous, request, majorVersion) === undefined + ? previous + : undefined; + const keep = (entry: ServiceLock | DatabaseLock, notice: string | undefined): ResolvedPin => ({ + pin: entry.resolved, + resolution: entry.resolution, + linked: entry.linked, + notice, + kept: true, + }); + const fallback = ( + resolution: "unlinked" | "unobserved" | "unpublished", + notice: string | undefined, + unchanged: boolean, + ): ResolvedPin => + safe !== undefined && unchanged && !source.refresh && safe.resolution === resolution + ? keep(safe, notice) + : { pin: bundled, resolution, linked, notice, kept: false }; + const retained = (cause: string, outage?: LockResolutionError): ResolvedPin => { + const fallen: ResolvedPin = + safe === undefined + ? { + pin: bundled, + resolution: "unobserved", + linked, + notice: `${kind}: ${cause}; using the bundled ${version(bundled)}. supabase link or supabase stack update retries.`, + kept: false, + } + : keep(safe, `${kind}: ${cause}; kept ${version(safe.resolved)}`); + return outage === undefined ? fallen : { ...fallen, outage }; + }; + + if (source.remote === undefined) return fallback("unlinked", undefined, true); + if (observed._tag === "Unreported") + return fallback( + "unobserved", + isObservable(kind) + ? `${kind}: the platform does not report its version; using the bundled ${version(bundled)}` + : undefined, + true, + ); + if (observed._tag === "Failed") + return retained(`the hosted version could not be read (${observed.cause})`); + + const sameObservation = safe?.linked?.version === observed.version; + if (kind === "database" && postgresMajor(observed.version) !== String(majorVersion)) + return fallback( + "unpublished", + `database: hosted Postgres ${observed.raw} does not match db.major_version ${majorVersion}; using the bundled ${version(bundled)}`, + sameObservation, + ); + return yield* releasePin(kind, observed.version, undefined).pipe( + Effect.map((pin): ResolvedPin => ({ + pin, + resolution: "resolved", + linked, + notice: undefined, + kept: false, + })), + Effect.catchTags({ + ReleaseUnpublished: () => + Effect.succeed( + fallback( + "unpublished", + `${kind}: hosted ${observed.raw} is not published by slim-services; using the bundled ${version(bundled)}`, + sameObservation, + ), + ), + ReleaseTransient: (error) => + Effect.succeed( + retained( + "the slim-services release host could not be reached", + releaseFailure(kind, request.specifier, error), + ), + ), + ReleaseIntegrity: (error) => Effect.fail(releaseFailure(kind, request.specifier, error)), + }), + ); +}); + +const resolvePinFor = Effect.fn("StackLock.resolvePin")(function* ( + kind: ServiceKind, + request: ServiceRequest, + majorVersion: number, + previous: ServiceLock | DatabaseLock | undefined, + source: LinkedSource, +) { + const { parsed, specifier } = request; + yield* Effect.annotateCurrentSpan({ + "lock.service": kind, + "lock.specifier_kind": parsed.kind, + }); + const resolved: ResolvedPin = + parsed.kind === "linked" + ? yield* resolveLinked(kind, request, majorVersion, previous, source) + : { + pin: + parsed.kind !== "release" + ? yield* bundledPin(kind, majorVersion) + : yield* releasePin(kind, parsed.upstream, parsed.revision).pipe( + Effect.mapError((error) => + error._tag === "StackConfigError" + ? error + : releaseFailure(kind, specifier, error), + ), + ), + resolution: "resolved", + linked: linkedObservation(kind, previous, source), + notice: undefined, + kept: false, + }; + yield* Effect.annotateCurrentSpan({ + "lock.outcome": resolved.resolution, + "lock.kept": resolved.kept, + }); + return resolved; +}); + +/** A resolved entry, the notice it earned, and whether it is the previous entry kept as it was. */ +interface Settled { + readonly entry: Entry; + readonly notice: string | undefined; + readonly kept: boolean; +} + +const resolveService = Effect.fn("StackLock.resolveService")(function* ( + kind: NonDatabaseKind, + request: ServiceRequest, + majorVersion: number, + previous: ServiceLock | undefined, + source: LinkedSource, +) { + const resolved = yield* resolvePinFor(kind, request, majorVersion, previous, source); + return { + entry: { + specifier: request.specifier, + ...(resolved.linked === undefined ? {} : { linked: resolved.linked }), + resolution: resolved.resolution, + resolved: resolved.pin, + }, + notice: resolved.notice, + kept: resolved.kept, + } satisfies Settled; +}); + +const resolveDatabase = Effect.fn("StackLock.resolveDatabase")(function* ( + request: ServiceRequest, + majorVersion: number, + previous: DatabaseLock | undefined, + source: LinkedSource, +) { + const resolved = yield* resolvePinFor("database", request, majorVersion, previous, source); + yield* Effect.annotateCurrentSpan({ "lock.database_line": pinLine(resolved.pin) }); + return { + entry: { + specifier: request.specifier, + ...(resolved.linked === undefined ? {} : { linked: resolved.linked }), + resolution: resolved.resolution, + resolved: resolved.pin, + majorVersion, + line: pinLine(resolved.pin), + }, + notice: resolved.notice, + kept: resolved.kept, + } satisfies Settled; +}); + +const continuousIntegration = Config.option(Config.String("CI")).pipe( + Effect.map((value) => + Option.exists(value, (text) => text !== "" && text !== "false" && text !== "0"), + ), + Effect.mapError( + () => new StackConfigError({ message: "failed to resolve environment variable: CI" }), + ), +); + +/** + * A lock that cannot be used as-is. A frozen `start` is told to commit a lock resolved elsewhere; + * a command that only reads the lock is told to run `supabase start` here, which refreshes it. + */ +const unusableLock = ( + stale: ReadonlyMap, + lock: Lock | undefined, + audience: "frozen" | "reader", +) => + lock === undefined + ? new LockResolutionError({ + reason: "frozen-missing", + message: `${LOCK_FILE} not found; run supabase start ${audience === "frozen" ? "locally and commit it" : "to create it"}`, + }) + : new LockResolutionError({ + reason: "frozen-stale", + message: `${LOCK_FILE} is stale for: ${[...stale].map(([kind, reason]) => `${kind} (${reason})`).join(", ")}; ${audience === "frozen" ? "run supabase start locally and commit the updated lock" : "run supabase start to refresh it"}`, + }); + +const version = (pin: ArtifactPin) => `${pin.upstreamVersion}-r${pin.revision}`; + +const diffLines = ( + previous: Lock | undefined, + lock: Lock, + stale: ReadonlyMap, +) => + [...stale].map(([kind, reason]) => { + const entry = lock.services[kind]; + const before = previous?.services[kind]; + const arrow = before === undefined ? "" : `${version(before.resolved)} → `; + const why = before === undefined ? `specifier ${entry.specifier}` : reason; + const detail = entry.resolution === "resolved" ? why : `${why}; ${entry.resolution}`; + return ` ${kind.padEnd(9)} ${arrow}${version(entry.resolved)} (${detail})`; + }); + +const requestsOf = (request: LockRequest) => + Effect.fromResult(serviceRequests(request)).pipe( + Effect.mapError((message) => new StackConfigError({ message })), + ); + +const inspectLock = Effect.fn("StackLock.inspect")(function* ( + request: LockRequest, + requests: ServiceRequests, +) { + const previous = Option.getOrUndefined( + yield* readLock(request.projectRoot).pipe(Effect.mapError(lockFailure)), + ); + return { previous, stale: staleServices(previous, requests, request.majorVersion) }; +}); + +/** The pins of a fresh lock, without the lease or any resolution; a missing or stale lock fails. */ +export const readLockedSelection = Effect.fn("StackLock.read")(function* (request: LockRequest) { + const { previous, stale } = yield* inspectLock(request, yield* requestsOf(request)); + if (previous === undefined || stale.size > 0) + return yield* unusableLock(stale, previous, "reader"); + return lockSelection(previous); +}); + +/** + * The pins the project lock holds, without loading config or checking staleness; `None` when + * there is no lock. For hosted-only commands that just need the locked release of one service. + */ +export const readLockedPins = Effect.fn("StackLock.readPins")(function* (projectRoot: string) { + const lock = yield* readLock(projectRoot).pipe(Effect.mapError(lockFailure)); + return Option.map(lock, lockSelection); +}); + +const bundledPins = Effect.fnUntraced(function* (request: LockRequest, requests: ServiceRequests) { + const defaults = yield* stackArtifactSelection({ + major_version: request.majorVersion, + orioledb_version: request.orioledbVersion, + }).pipe(Effect.mapError((error) => new StackConfigError({ message: error.message }))); + const bundled = (kind: ServiceKind): ArtifactPin | undefined => { + const { parsed } = requests[kind]; + return parsed.kind !== "release" + ? defaults[kind] + : catalogPins().find( + ({ service, pin }) => + service === kind && + pin.upstreamVersion === parsed.upstream && + (parsed.revision === undefined || parsed.revision === pin.revision), + )?.pin; + }; + return { defaults, bundled }; +}); + +/** The pins this CLI bundles for a request; an explicit upstream version is honoured only when this CLI bundles it. */ +const bundledSelection = Effect.fn("StackLock.bundledSelection")(function* (request: LockRequest) { + const { defaults, bundled } = yield* bundledPins(request, yield* requestsOf(request)); + if (artifactServiceKinds().some((kind) => bundled(kind) === undefined)) + return yield* unusableLock(new Map(), undefined, "reader"); + return Record.map(defaults, (pin, kind) => bundled(kind) ?? pin); +}); + +/** + * The pins of a fresh lock, or the bundled ones when the project has no lock yet. Commands that + * never resolve use this to run the versions `start` locked; a stale lock still fails. + */ +export const readSelectionOrBundled = Effect.fn("StackLock.readOrBundled")(function* ( + request: LockRequest, +) { + const previous = Option.getOrUndefined( + yield* readLock(request.projectRoot).pipe(Effect.mapError(lockFailure)), + ); + if (previous === undefined) { + yield* Effect.annotateCurrentSpan({ "lock.found": false }); + return yield* bundledSelection(request); + } + const stale = staleServices(previous, yield* requestsOf(request), request.majorVersion); + yield* Effect.annotateCurrentSpan({ "lock.found": true, "lock.stale_count": stale.size }); + if (stale.size > 0) return yield* unusableLock(stale, previous, "reader"); + return lockSelection(previous); +}); + +/** + * Like `readSelectionOrBundled`, but stale entries take the bundled pin while fresh ones keep + * their locked pin, for requests an applied `[remotes.]` block changed. + */ +export const readSelectionBundlingStale = Effect.fn("StackLock.readBundlingStale")(function* ( + request: LockRequest, +) { + const requests = yield* requestsOf(request); + const { previous, stale } = yield* inspectLock(request, requests); + if (previous === undefined) return yield* bundledSelection(request); + yield* Effect.annotateCurrentSpan({ "lock.stale_count": stale.size }); + const locked = lockSelection(previous); + if (stale.size === 0) return locked; + const { bundled } = yield* bundledPins(request, requests); + const unbundled = new Map([...stale].filter(([kind]) => bundled(kind) === undefined)); + if (unbundled.size > 0) return yield* unusableLock(unbundled, previous, "reader"); + return Record.map(locked, (pin, kind) => (stale.has(kind) ? (bundled(kind) ?? pin) : pin)); +}); + +/** Takes the project lease for the enclosing scope once git ignores the lease file. */ +const holdLease = Effect.fn("StackLock.lease")(function* (projectRoot: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* ensureGitignore(fs, path, projectRoot).pipe( + Effect.mapError( + (error) => + new LockResolutionError({ + reason: "io", + message: `failed to update supabase/.gitignore: ${error.message}`, + }), + ), + ); + const started = yield* Clock.currentTimeMillis; + yield* holdLock(path.join(projectRoot, "supabase", LEASE_FILE), LEASE_BUDGET, LOCK_FILE).pipe( + Effect.mapError(leaseFailure), + ); + yield* Effect.annotateCurrentSpan({ + "lock.lease_wait_ms": (yield* Clock.currentTimeMillis) - started, + }); +}); + +/** Runs `effect` under the project lease that `supabase link` publishes `.link.json` under. */ +export const withProjectLease = (projectRoot: string, effect: Effect.Effect) => + holdLease(projectRoot).pipe(Effect.andThen(effect), Effect.scoped); + +/** + * Fails a project without `supabase/config.toml` that is frozen, explicitly or by `CI` with a + * lock present: with no config the lock cannot be checked, so the bundled versions must not + * silently stand in for it. An explicitly frozen project with no lock fails on the missing lock. + */ +export const rejectFrozenWithoutConfig = Effect.fn("StackLock.rejectFrozenWithoutConfig")( + function* (projectRoot: string, frozen: boolean) { + if (!frozen && !(yield* continuousIntegration)) return; + const lock = yield* readLock(projectRoot).pipe(Effect.mapError(lockFailure)); + if (Option.isSome(lock)) + return yield* new LockResolutionError({ + reason: "frozen-missing", + message: `supabase/config.toml not found, so ${LOCK_FILE} cannot be checked; commit supabase/config.toml next to it`, + }); + if (frozen) + return yield* new LockResolutionError({ + reason: "frozen-missing", + message: `${LOCK_FILE} not found; run supabase start locally and commit it`, + }); + }, +); + +/** A service the lock gained or moved to another release. */ +interface LockChange { + readonly service: ServiceKind; + /** The release the lock held before; absent when the service had no entry yet. */ + readonly from: string | undefined; + readonly to: string; +} + +/** The pins a command runs and what it changed in the lock to get them. */ +export interface LockedResolution { + readonly selection: ArtifactSelection; + /** Whether `supabase/supabase.lock` was written. */ + readonly written: boolean; + /** Services resolved afresh; empty when the lock was used as-is. */ + readonly changes: ReadonlyArray; + /** Services whose locked entry was kept. */ + readonly unchanged: ReadonlyArray; +} + +const resolutionOf = ( + lock: Lock, + previous: Lock | undefined, + stale: ReadonlyMap, + written: boolean, +): LockedResolution => ({ + selection: lockSelection(lock), + written, + changes: [...stale.keys()].map((service) => ({ + service, + from: previous === undefined ? undefined : version(previous.services[service].resolved), + to: version(lock.services[service].resolved), + })), + unchanged: artifactServiceKinds().filter((kind) => !stale.has(kind)), +}); + +/** The entry of a service that was not re-resolved, with the observation of a fresh probe when it has one. */ +const probedService = ( + kind: ServiceKind, + entry: ServiceLock, + source: LinkedSource, +): ServiceLock => { + const linked = linkedObservation(kind, entry, source); + return { + specifier: entry.specifier, + ...(linked === undefined ? {} : { linked }), + resolution: entry.resolution, + resolved: entry.resolved, + }; +}; + +const probedDatabase = (entry: DatabaseLock, source: LinkedSource): DatabaseLock => { + const linked = linkedObservation("database", entry, source); + return { + specifier: entry.specifier, + ...(linked === undefined ? {} : { linked }), + resolution: entry.resolution, + resolved: entry.resolved, + majorVersion: entry.majorVersion, + line: entry.line, + }; +}; + +interface ResolvedServices { + readonly services: Lock["services"]; + /** What the user should hear about fallbacks and kept entries. */ + readonly notices: ReadonlyArray; + /** Re-resolved services whose previous entry was kept as it was. */ + readonly kept: ReadonlyArray; +} + +/** + * Resolves the `redo` services afresh and keeps every other entry of `previous`. A probing source + * also records its observation on kept entries that do not follow `linked`. + */ +const resolveServices = Effect.fn("StackLock.resolveServices")(function* ( + request: LockRequest, + requests: ServiceRequests, + previous: Lock | undefined, + redo: ReadonlySet, + source: LinkedSource, +) { + yield* Effect.annotateCurrentSpan({ "lock.resolve_count": redo.size }); + const probes = (kind: ServiceKind) => source.probed && requests[kind].parsed.kind !== "linked"; + const service = (kind: NonDatabaseKind) => { + const current = previous?.services[kind]; + return current === undefined || redo.has(kind) + ? resolveService(kind, requests[kind], request.majorVersion, current, source) + : Effect.succeed({ + entry: probes(kind) ? probedService(kind, current, source) : current, + notice: undefined, + kept: true, + } satisfies Settled); + }; + const currentDatabase = previous?.services.database; + const settled = yield* Effect.all( + { + database: + currentDatabase === undefined || redo.has("database") + ? resolveDatabase(requests.database, request.majorVersion, currentDatabase, source) + : Effect.succeed({ + entry: probes("database") ? probedDatabase(currentDatabase, source) : currentDatabase, + notice: undefined, + kept: true, + } satisfies Settled), + rest: service("rest"), + auth: service("auth"), + realtime: service("realtime"), + storage: service("storage"), + imgproxy: service("imgproxy"), + functions: service("functions"), + studio: service("studio"), + pgmeta: service("pgmeta"), + mail: service("mail"), + analytics: service("analytics"), + pooler: service("pooler"), + }, + { concurrency: 4 }, + ); + const services: Lock["services"] = { + database: settled.database.entry, + rest: settled.rest.entry, + auth: settled.auth.entry, + realtime: settled.realtime.entry, + storage: settled.storage.entry, + imgproxy: settled.imgproxy.entry, + functions: settled.functions.entry, + studio: settled.studio.entry, + pgmeta: settled.pgmeta.entry, + mail: settled.mail.entry, + analytics: settled.analytics.entry, + pooler: settled.pooler.entry, + }; + const kinds = artifactServiceKinds(); + return { + services, + notices: kinds.flatMap((kind) => settled[kind].notice ?? []), + kept: kinds.filter((kind) => redo.has(kind) && settled[kind].kept), + } satisfies ResolvedServices; +}); + +const assembleLock = (remote: LinkedRemote | undefined, services: Lock["services"]): Lock => ({ + $schema: LOCK_SCHEMA_URL, + lockfileVersion: 1, + ...(remote === undefined ? {} : { linked: remote }), + services, +}); + +/** Writes `lock` as the project lock; the caller holds the project lease. */ +const publishLock = Effect.fn("StackLock.publish")(function* (request: LockRequest, lock: Lock) { + yield* writeLock(request.projectRoot, lock).pipe(Effect.mapError(lockFailure)); +}); + +const lockEquivalence = Schema.toEquivalence(Lock); + +/** Whether two locks pin and observe the same things; only the time of the observation may differ. */ +const sameLock = (a: Lock, b: Lock) => { + const untimed = (lock: Lock): Lock => + lock.linked === undefined ? lock : { ...lock, linked: { ...lock.linked, observedAt: "" } }; + return lockEquivalence(untimed(a), untimed(b)); +}; + +/** A remote as a project ref, followed by the branch ref unless the branch is the project itself. */ +const remoteLabel = (project: string, branchRef: string) => + branchRef === project ? project : `${project}/${branchRef}`; + +const warnNotices = (output: Output["Service"], notices: ReadonlyArray) => + Effect.forEach(notices, (notice) => output.warn(notice), { discard: true }); + +/** + * Warnings that never block: the checkout is linked to another remote than the lock observed, and + * this CLI bundles newer revisions of locked versions. Reads `.link.json` and bundled pins only. + */ +const lockWarnings = Effect.fn("StackLock.warn")(function* ( + request: LockRequest, + requests: ServiceRequests, + lock: Lock, + output: Output["Service"], +) { + const link = Option.flatten(yield* readProjectLink(request.projectRoot).pipe(Effect.option)); + if (Option.isSome(link)) { + const target = link.value.active_branch; + const observed = lock.linked; + if ( + observed === undefined || + observed.project !== link.value.project.ref || + observed.branch.ref !== target.ref + ) + yield* output.warn( + `${observed === undefined ? `${LOCK_FILE} has no hosted observations` : `lock observed ${remoteLabel(observed.project, observed.branch.ref)}`}, this checkout targets ${remoteLabel(link.value.project.ref, target.ref)}; run supabase stack update`, + ); + } + const newer = yield* Effect.forEach(artifactServiceKinds(), (kind) => + bundledPinFor(kind, lock.services[kind].resolved, request.majorVersion).pipe( + Effect.option, + Effect.map((bundled) => { + const locked = lock.services[kind].resolved; + return Option.filter( + bundled, + (pin) => + !isExactRelease(requests[kind]) && + pin.upstreamVersion === locked.upstreamVersion && + pin.revision > locked.revision, + ).pipe( + Option.map((pin) => `${kind} ${version(locked)} → ${version(pin)}`), + Option.toArray, + ); + }), + ), + ); + const outdated = newer.flat(); + if (outdated.length > 0) + yield* output.warn( + `This CLI bundles newer revisions of locked versions: ${outdated.join(", ")}. Run supabase stack outdated to review, supabase stack update to adopt them.`, + ); +}); + +/** + * Selects the pins a stack command runs from `supabase/supabase.lock`, under the project lease: + * a fresh lock is used as-is; a missing or stale one is resolved and rewritten, unless frozen. + * Frozen is explicit, or implied when `CI` is set and a lock exists, and reads the lock without + * the lease since lock writes are atomic. `linked` entries follow the observations the lock + * stores, never a new probe. + */ +export const lockedResolution = (options: { + readonly frozen: boolean; + readonly output: Output["Service"]; +}) => + Effect.fn("StackLock.select")(function* ( + request: LockRequest, + ): Effect.fn.Return< + LockedResolution, + StackConfigError | LockResolutionError, + FileSystem.FileSystem | Path.Path | Crypto.Crypto + > { + const requests = yield* requestsOf(request); + const ci = yield* continuousIntegration; + + const { resolution, lock } = yield* Effect.gen(function* () { + const unleased = yield* inspectLock(request, requests); + const frozenRead = options.frozen || (ci && unleased.previous !== undefined); + if (!frozenRead) yield* holdLease(request.projectRoot); + const { previous, stale } = frozenRead ? unleased : yield* inspectLock(request, requests); + const frozen = options.frozen || (ci && previous !== undefined); + yield* Effect.annotateCurrentSpan({ + "lock.found": previous !== undefined, + "lock.frozen": frozen, + "lock.stale_count": stale.size, + }); + if (previous !== undefined && stale.size === 0) + return { resolution: resolutionOf(previous, previous, stale, false), lock: previous }; + if (frozen) return yield* unusableLock(stale, previous, "frozen"); + + const resolved = yield* resolveServices( + request, + requests, + previous, + new Set(stale.keys()), + storedSource(previous, false), + ); + const lock = assembleLock(previous?.linked, resolved.services); + yield* publishLock(request, lock); + yield* options.output.info( + [ + "Resolved service versions:", + ...diffLines(previous, lock, stale), + `${previous === undefined ? "Created" : "Updated"} ${LOCK_FILE}.`, + ].join("\n"), + ); + yield* warnNotices(options.output, resolved.notices); + if (previous === undefined && ci) + yield* options.output.warn( + `Commit ${LOCK_FILE} so CI and your team run the same versions.`, + ); + return { resolution: resolutionOf(lock, previous, stale, true), lock }; + }).pipe(Effect.scoped); + yield* lockWarnings(request, requests, lock, options.output); + return resolution; + }); + +/** The pins a stack command runs; see {@link lockedResolution}. */ +export const lockedSelection = (options: Parameters[0]) => { + const resolve = lockedResolution(options); + return (request: LockRequest) => + resolve(request).pipe(Effect.map((resolution) => resolution.selection)); +}; + +/** A service `stack update` locked to another release, and why. */ +export interface UpdatedService extends LockChange { + readonly reason: string; +} + +/** What `stack update` did to `supabase/supabase.lock`. */ +export interface LockUpdate { + /** Whether `supabase/supabase.lock` was written. */ + readonly written: boolean; + readonly changes: ReadonlyArray; + /** Services whose locked entry was kept, excluding `stale` ones. */ + readonly unchanged: ReadonlyArray; + /** Stale services that were not named; the lock still holds their old entries, with the reason. */ + readonly stale: ReadonlyArray<{ readonly service: ServiceKind; readonly reason: string }>; +} + +const isExactRelease = ({ parsed }: ServiceRequest) => + parsed.kind === "release" && parsed.revision !== undefined; + +const sameRelease = (before: ServiceLock | DatabaseLock, after: ServiceLock | DatabaseLock) => + before.specifier === after.specifier && + before.resolution === after.resolution && + version(before.resolved) === version(after.resolved); + +/** The release an entry resolves to; the resolution is added when it is the only thing that differs. */ +const describeMove = ( + before: ServiceLock | DatabaseLock | undefined, + after: Pick, +) => + before !== undefined && + version(before.resolved) === version(after.resolved) && + before.resolution !== after.resolution + ? `${version(after.resolved)} (${after.resolution})` + : version(after.resolved); + +const linkedKinds = (requests: ServiceRequests) => + artifactServiceKinds().filter((kind) => requests[kind].parsed.kind === "linked"); + +/** + * Re-resolves the named services (all when none are named) under the project lease and writes the + * lock when an entry moved. Newest revisions come from the release host, bundled entries take + * this CLI's pin, an exact `-rN` entry that still answers its request is kept, and `linked` + * entries follow a fresh hosted probe when one is available, retrying fallbacks. The probe runs + * only when a `linked` entry is re-resolved; without one the stored observations apply. + * Frozen mode does not apply: asking for newer versions is the point. Once the lease is released, + * a written lock also warns about each saved stack whose data would refuse its pins. + */ +export const updatedLock = (options: { + readonly services: ReadonlyArray; + readonly probe: Effect.Effect, E, R>; + readonly output: Output["Service"]; +}) => + Effect.fn("StackLock.update")(function* ( + request: LockRequest, + ): Effect.fn.Return< + LockUpdate, + StackConfigError | LockResolutionError | E, + FileSystem.FileSystem | Path.Path | Crypto.Crypto | CommandSettings | R + > { + const requests = yield* requestsOf(request); + const { update, published } = yield* Effect.gen(function* () { + yield* holdLease(request.projectRoot); + const { previous, stale } = yield* inspectLock(request, requests); + const named = options.services.length === 0 ? artifactServiceKinds() : options.services; + // A lock has to hold every service, so a missing one is resolved whatever is named. + const requested = new Set( + previous === undefined + ? artifactServiceKinds() + : named.filter((kind) => stale.has(kind) || !isExactRelease(requests[kind])), + ); + const probe = [...requested].some((kind) => requests[kind].parsed.kind === "linked") + ? Option.getOrUndefined(yield* options.probe) + : undefined; + const observedAt = DateTime.formatIso(yield* DateTime.now); + const source = + probe === undefined + ? storedSource(previous, true) + : probedSource(previous, probe, observedAt, true); + // Observations of another remote are no basis for the entries that follow them. + const redo = + probe !== undefined && !source.sameRemote + ? new Set([...requested, ...linkedKinds(requests)]) + : requested; + const resolved = yield* resolveServices(request, requests, previous, redo, source); + const changes = [...redo].flatMap((kind): Array => { + const before = previous?.services[kind]; + const after = resolved.services[kind]; + if (before !== undefined && sameRelease(before, after)) return []; + return [ + { + service: kind, + from: before === undefined ? undefined : version(before.resolved), + to: describeMove(before, after), + reason: stale.get(kind) ?? updateReason(requests[kind].parsed), + }, + ]; + }); + const skipped = [...stale].flatMap(([service, reason]) => + redo.has(service) ? [] : [{ service, reason }], + ); + const unchanged = artifactServiceKinds().filter( + (kind) => + !changes.some(({ service }) => service === kind) && + !skipped.some(({ service }) => service === kind), + ); + const lock = assembleLock(source.remote, resolved.services); + const written = previous === undefined || !sameLock(previous, lock); + yield* Effect.annotateCurrentSpan({ + "lock.found": previous !== undefined, + "lock.probed": probe !== undefined, + "lock.resolve_count": redo.size, + "lock.updated_count": changes.length, + "lock.written": written, + }); + yield* warnNotices(options.output, resolved.notices); + const update: LockUpdate = { written, changes, unchanged, stale: skipped }; + if (!written) return { update, published: undefined }; + yield* publishLock(request, lock); + return { update, published: lock }; + }).pipe(Effect.scoped); + if (published !== undefined) + yield* Effect.annotateCurrentSpan({ + "lock.data_conflict_count": yield* warnDataConflicts({ + projectRoot: request.projectRoot, + selection: lockSelection(published), + output: options.output, + }), + }); + return update; + }); + +const updateReason = (parsed: Specifier) => + parsed.kind === "release" + ? `newest revision of ${parsed.upstream}` + : parsed.kind === "linked" + ? "follows the linked project" + : "bundled with this CLI"; + +/** One service of a hosted probe as `supabase link` reports it. */ +export interface LinkedService { + readonly service: ServiceKind; + /** What the platform reported; absent when it reports nothing or the probe failed. */ + readonly hosted: string | undefined; + readonly locked: string; + readonly resolution: ServiceLock["resolution"]; + /** Whether the entry's pin, resolution or hosted observation differs from the previous lock. */ + readonly changed: boolean; +} + +/** What `supabase link` did to `supabase/supabase.lock`. */ +export interface LinkedLock { + /** Whether the lock was written; false when frozen or when nothing but the observation time changed. */ + readonly written: boolean; + readonly frozen: boolean; + readonly services: ReadonlyArray; + /** Re-resolved `linked` entries whose previous entry survived a transient failure or an unchanged fallback. */ + readonly kept: ReadonlyArray; + /** The remote the lock now observes, when this link changed it. */ + readonly observing: string | undefined; +} + +const hostedRaw = (probe: HostedProbe, kind: ServiceKind) => { + const observed = probedObservation(probe, kind); + return observed?._tag === "Observed" ? observed.raw : undefined; +}; + +const remoteChanged = (previous: Lock | undefined, probe: HostedProbe) => + previous?.linked === undefined || + previous.linked.project !== probe.remote.project || + previous.linked.branch.ref !== probe.remote.branch.ref; + +/** How a frozen link's probe differs from the lock it leaves alone. */ +const differences = (previous: Lock, probe: HostedProbe): ReadonlyArray => [ + ...(previous.linked !== undefined && + previous.linked.project === probe.remote.project && + previous.linked.branch.ref === probe.remote.branch.ref + ? [] + : [ + `the lock observed ${previous.linked === undefined ? "no remote" : remoteLabel(previous.linked.project, previous.linked.branch.ref)}, this link targets ${remoteLabel(probe.remote.project, probe.remote.branch.ref)}`, + ]), + ...artifactServiceKinds().flatMap((kind) => { + if (probedObservation(probe, kind)?._tag === "Failed") return []; + const locked = previous.services[kind].linked?.raw; + const hosted = hostedRaw(probe, kind); + return locked === hosted + ? [] + : [`${kind} locked ${locked ?? "none"}, hosted ${hosted ?? "none"}`]; + }), +]; + +/** + * Records a hosted probe in `supabase/supabase.lock` under the project lease: probes, updates the + * per-service observations, re-resolves every `linked` entry, and sets the lock's remote. The lock + * is written only when something but the time of the observation changed. While frozen (explicit, + * or `CI` with an existing lock) the lock is left untouched and a difference from the probe is + * warned about. Once the services resolve, `publishLink` runs before the lock is written, so a + * lock never observes a remote the checkout does not target; `report` runs last under the lease. + * Once the lease is released, a lock that changed also warns about each saved stack whose data + * would refuse its pins. + */ +export const linkedLock = (options: { + readonly frozen: boolean; + readonly probe: Effect.Effect; + readonly publishLink: Effect.Effect; + readonly report: (result: LinkedLock) => Effect.Effect; + readonly output: Output["Service"]; +}) => + Effect.fn("StackLock.link")(function* ( + request: LockRequest, + ): Effect.fn.Return< + LinkedLock, + StackConfigError | LockResolutionError | E | E2, + FileSystem.FileSystem | Path.Path | Crypto.Crypto | CommandSettings | R | R2 + > { + const requests = yield* requestsOf(request); + const ci = yield* continuousIntegration; + const kinds = artifactServiceKinds(); + const { linked, published } = yield* Effect.gen(function* () { + yield* holdLease(request.projectRoot); + const { previous, stale } = yield* inspectLock(request, requests); + const probe = yield* options.probe; + const frozen = options.frozen || (ci && previous !== undefined); + yield* Effect.annotateCurrentSpan({ + "lock.found": previous !== undefined, + "lock.frozen": frozen, + "lock.stale_count": stale.size, + }); + const row = (kind: ServiceKind, lock: Lock | undefined): LinkedService => { + const entry = (lock ?? previous)?.services[kind]; + const before = previous?.services[kind]; + return { + service: kind, + hosted: hostedRaw(probe, kind), + locked: entry === undefined ? "none" : version(entry.resolved), + resolution: entry?.resolution ?? "unlinked", + changed: + lock !== undefined && + (before === undefined || + !sameRelease(before, lock.services[kind]) || + before.linked?.raw !== lock.services[kind].linked?.raw), + }; + }; + if (frozen) { + const differing = + previous === undefined + ? [`there is no ${LOCK_FILE} to compare with`] + : differences(previous, probe); + if (differing.length > 0) + yield* options.output.warn( + `Hosted versions differ from ${LOCK_FILE}, which a frozen link leaves untouched: ${differing.join("; ")}. Run supabase stack update locally to record them.`, + ); + const result: LinkedLock = { + written: false, + frozen, + services: kinds.map((kind) => row(kind, undefined)), + kept: [], + observing: undefined, + }; + yield* options.report(result); + yield* options.publishLink; + return { linked: result, published: undefined }; + } + + const observedAt = DateTime.formatIso(yield* DateTime.now); + const source = probedSource(previous, probe, observedAt, false); + const redo = new Set([...stale.keys(), ...linkedKinds(requests)]); + const resolved = yield* resolveServices(request, requests, previous, redo, source); + const lock = assembleLock(source.remote, resolved.services); + const written = previous === undefined || !sameLock(previous, lock); + yield* options.publishLink; + if (written) yield* publishLock(request, lock); + yield* warnNotices(options.output, resolved.notices); + const result: LinkedLock = { + written, + frozen, + services: kinds.map((kind) => row(kind, lock)), + kept: resolved.kept, + observing: remoteChanged(previous, probe) + ? remoteLabel(probe.remote.project, probe.remote.branch.ref) + : undefined, + }; + yield* options.report(result); + return { linked: result, published: written ? lock : undefined }; + }).pipe(Effect.scoped); + if (published !== undefined) + yield* Effect.annotateCurrentSpan({ + "lock.data_conflict_count": yield* warnDataConflicts({ + projectRoot: request.projectRoot, + selection: lockSelection(published), + output: options.output, + }), + }); + return linked; + }); + +/** One locked service as `stack outdated` reports it. */ +export interface OutdatedService { + readonly service: ServiceKind; + readonly specifier: string; + readonly resolution: ServiceLock["resolution"]; + readonly locked: string; + /** What the linked project last reported, as the lock stored it; absent when nothing was observed. */ + readonly hosted: string | undefined; + /** The release `stack update` would lock, with its resolution when only that differs; absent when it would keep the entry. */ + readonly update: string | undefined; + /** This CLI's pin for the service. */ + readonly bundled: string; + /** Why the entry no longer answers `config.toml`; absent when it does. */ + readonly stale: string | undefined; + /** Why the newest release could not be determined; absent when it could. */ + readonly error: string | undefined; +} + +/** + * What `stack update` would change, per locked service, without the lease, a write, a download, or + * a hosted probe: `linked` entries follow the stored observation, so a hosted version that moved + * since `supabase link` shows only after `stack update` probes it. A stale entry is reported + * without resolving: `start` or `stack update` resolves it against config. A failed resolution is + * reported on its own row. + */ +export const outdatedServices = Effect.fn("StackLock.outdated")(function* (request: LockRequest) { + const requests = yield* requestsOf(request); + const { previous, stale } = yield* inspectLock(request, requests); + if (previous === undefined) return yield* unusableLock(stale, previous, "reader"); + const lock = previous; + const source = storedSource(lock, true); + + const row = Effect.fnUntraced(function* (kind: ServiceKind) { + const entry = lock.services[kind]; + const requested = requests[kind]; + const reason = stale.get(kind); + const bundled = yield* bundledPinFor(kind, entry.resolved, request.majorVersion); + const locked = version(entry.resolved); + const newest: Effect.Effect< + ResolvedPin | undefined, + StackConfigError | LockResolutionError, + Crypto.Crypto + > = + reason !== undefined || isExactRelease(requested) + ? Effect.undefined + : resolvePinFor(kind, requested, request.majorVersion, entry, source); + const outcome = yield* newest.pipe( + Effect.map((resolved) => ({ resolved, error: undefined })), + Effect.catchTag("LockResolutionError", (error) => + Effect.succeed({ resolved: undefined, error: error.message }), + ), + ); + const resolved = outcome.resolved?.outage === undefined ? outcome.resolved : undefined; + const error = outcome.error ?? outcome.resolved?.outage?.message; + return { + service: kind, + specifier: entry.specifier, + resolution: entry.resolution, + locked, + hosted: entry.linked?.raw, + update: + resolved === undefined || + (version(resolved.pin) === locked && resolved.resolution === entry.resolution) + ? undefined + : describeMove(entry, { resolution: resolved.resolution, resolved: resolved.pin }), + bundled: version(bundled), + stale: reason, + error, + } satisfies OutdatedService; + }); + + const rows = yield* Effect.forEach(artifactServiceKinds(), row, { concurrency: 4 }); + yield* Effect.annotateCurrentSpan({ + "lock.service_count": rows.length, + "lock.stale_count": stale.size, + "lock.outdated_count": rows.filter(({ update }) => update !== undefined).length, + "lock.failed_count": rows.filter(({ error }) => error !== undefined).length, + }); + return rows; +}); diff --git a/apps/cli/src/command-internal/stack-lock.unit.test.ts b/apps/cli/src/command-internal/stack-lock.unit.test.ts new file mode 100644 index 0000000000..6bd41034ef --- /dev/null +++ b/apps/cli/src/command-internal/stack-lock.unit.test.ts @@ -0,0 +1,16 @@ +import { describe, expect, it } from "vitest"; +import { parseSpecifier } from "./stack-lock.ts"; + +describe("parseSpecifier", () => { + it.each([ + ["linked", { kind: "linked" }], + ["bundled", { kind: "bundled" }], + ["v1.2.3", { kind: "release", upstream: "v1.2.3" }], + ["v1.2.3-r4", { kind: "release", upstream: "v1.2.3", revision: 4 }], + ["v2.197.0-r0", { kind: "release", upstream: "v2.197.0", revision: 0 }], + ["15.1.0.000-orioledb", { kind: "release", upstream: "15.1.0.000-orioledb" }], + ["15.1.0.000-orioledb-r12", { kind: "release", upstream: "15.1.0.000-orioledb", revision: 12 }], + ])("reads %s", (text, expected) => { + expect(parseSpecifier(text)).toEqual(expected); + }); +}); diff --git a/apps/cli/src/command-internal/stack-project-selection.ts b/apps/cli/src/command-internal/stack-project-selection.ts new file mode 100644 index 0000000000..80b840e75d --- /dev/null +++ b/apps/cli/src/command-internal/stack-project-selection.ts @@ -0,0 +1,14 @@ +import { + readSelectionBundlingStale, + readSelectionOrBundled, + type LockRequest, +} from "./stack-lock.ts"; + +/** The pins the project's lock selects for a request; an applied `[remotes.]` block can make entries stale, which then take their bundled pins. */ +export const selectionForRequest = ({ + appliedRemote, + ...request +}: LockRequest & { readonly appliedRemote: string | undefined }) => + appliedRemote === undefined + ? readSelectionOrBundled(request) + : readSelectionBundlingStale(request); diff --git a/apps/cli/src/command-internal/stack-shadow-cache.integration.test.ts b/apps/cli/src/command-internal/stack-shadow-cache.integration.test.ts index 8f56ba5fbc..ba86b684f1 100644 --- a/apps/cli/src/command-internal/stack-shadow-cache.integration.test.ts +++ b/apps/cli/src/command-internal/stack-shadow-cache.integration.test.ts @@ -1,11 +1,11 @@ import { CliConfigSchema } from "@supabase/config"; -import { orioledbVersions } from "@supabase/stack/internal/artifacts"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Option, Path, Schema } from "effect"; import type { ShadowSetupInput } from "./db-bootstrap/shadow-database.ts"; import { stackShadowCacheEntry } from "./stack-shadow-cache.ts"; +import { database15, orioledb, selection } from "../../tests/helpers/pins.ts"; const defaultConfig = Schema.decodeSync(CliConfigSchema)({}); @@ -67,7 +67,14 @@ describe("stack shadow cache entry", () => { yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); const base = input(fs, path, root); - const first = yield* stackShadowCacheEntry(base, "17", "native", "darwin", "arm64", "config"); + const first = yield* stackShadowCacheEntry( + base, + selection, + "native", + "darwin", + "arm64", + "config", + ); if (first === undefined) return yield* Effect.die("cache entry unexpectedly disabled"); yield* fs.writeFileString( @@ -76,7 +83,7 @@ describe("stack shadow cache entry", () => { ); const withRoles = yield* stackShadowCacheEntry( base, - "17", + selection, "native", "darwin", "arm64", @@ -87,7 +94,7 @@ describe("stack shadow cache entry", () => { const withWebhooks = yield* stackShadowCacheEntry( input(fs, path, root, { webhooksEnabled: true }), - "17", + selection, "native", "darwin", "arm64", @@ -99,7 +106,7 @@ describe("stack shadow cache entry", () => { const withPassword = yield* stackShadowCacheEntry( input(fs, path, root, {}, { password: "rotated-password" }), - "17", + selection, "native", "darwin", "arm64", @@ -110,12 +117,12 @@ describe("stack shadow cache entry", () => { expect(withPassword.key).not.toBe(withRoles.key); expect( - yield* stackShadowCacheEntry(base, "17", "native", "darwin", "arm64", "config", true), + yield* stackShadowCacheEntry(base, selection, "native", "darwin", "arm64", "config", true), ).toBeUndefined(); expect( yield* stackShadowCacheEntry( input(fs, path, root, { projectEnvValues: { SUPABASE_SHADOW_CACHE: "0" } }), - "17", + selection, "native", "darwin", "arm64", @@ -125,21 +132,35 @@ describe("stack shadow cache entry", () => { }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), ); - it.effect("keys stock and OrioleDB shadows of the same major apart", () => + it.effect("keys shadows of different database or service pins apart", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-shadow-cache-version-" }); - const base = input(fs, path, root); - const entry = (version: string) => - stackShadowCacheEntry(base, version, "native", "darwin", "arm64", "config"); - const [orioledb = ""] = orioledbVersions(); + const base = input(fs, path, root, { + authEnabledForSetup: true, + }); + const key = (chosen: typeof selection) => + stackShadowCacheEntry(base, chosen, "native", "darwin", "arm64", "config").pipe( + Effect.map((entry) => entry?.key), + ); - const stock = yield* entry("17"); - const oriole = yield* entry(`${orioledb}-orioledb`); - expect(stock?.key).toBeDefined(); - expect(oriole?.key).toBeDefined(); - expect(oriole?.key).not.toBe(stock?.key); + const stock = yield* key(selection); + const keys = [ + yield* key({ ...selection, database: database15 }), + yield* key({ ...selection, database: orioledb }), + yield* key({ + ...selection, + database: { ...selection.database, revision: selection.database.revision + 1 }, + }), + yield* key({ + ...selection, + auth: { ...selection.auth, image: `${selection.auth.image}-other` }, + }), + ]; + expect(stock).toBeDefined(); + for (const other of keys) expect(other).toBeDefined(); + expect(new Set([stock, ...keys]).size).toBe(keys.length + 1); }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), ); }); diff --git a/apps/cli/src/command-internal/stack-shadow-cache.ts b/apps/cli/src/command-internal/stack-shadow-cache.ts index cf8f322529..a28669d8b9 100644 --- a/apps/cli/src/command-internal/stack-shadow-cache.ts +++ b/apps/cli/src/command-internal/stack-shadow-cache.ts @@ -1,5 +1,5 @@ import { Effect, Schema } from "effect"; -import { resolveArtifact, postgresVersion } from "@supabase/stack/internal/artifacts"; +import type { ArtifactPin, ArtifactSelection } from "@supabase/stack/internal/artifacts"; import type { ShadowSetupInput } from "./db-bootstrap/shadow-database.ts"; import { SHADOW_CACHE_ENV, @@ -9,15 +9,17 @@ import { import { resolveSetupWebhooksEnabled, type SetupDatabaseOptions } from "./db-bootstrap/db-setup.ts"; import { viperEnvBoolWithProjectFallback } from "./viper-env.ts"; -const encodePostgresImageKey = Schema.encodeEffect( - Schema.fromJsonString(Schema.Array(Schema.String)), -); +const encodeKeyParts = Schema.encodeEffect(Schema.fromJsonString(Schema.Array(Schema.String))); interface StackShadowCacheEntry { readonly key: string; readonly rolesSql: string; } +/** Identifies the exact artifact build, so a version or revision bump never reuses a baseline. */ +const pinKey = (pin: ArtifactPin) => + encodeKeyParts([pin.upstreamVersion, String(pin.revision), pin.image]); + const readRoles = (input: ShadowSetupInput) => Effect.gen(function* () { const file = input.path.join(input.workdir, "supabase", "roles.sql"); @@ -27,7 +29,7 @@ const readRoles = (input: ShadowSetupInput) => export const stackShadowCacheEntry = Effect.fn("StackShadowCache.entry")(function* ( input: ShadowSetupInput, - databaseVersion: string, + selection: ArtifactSelection, runtime: string, platform: string, arch: string, @@ -42,24 +44,15 @@ export const stackShadowCacheEntry = Effect.fn("StackShadowCache.entry")(functio ) return undefined; const rolesSql = yield* readRoles(input); - const postgres = yield* resolveArtifact({ - service: "database", - version: postgresVersion(databaseVersion), - }); - const image = (service: "auth" | "storage" | "realtime", enabled: boolean) => - enabled ? resolveArtifact({ service }) : Effect.succeed({ image: "", version: "" }); - const [auth, storage, realtime] = yield* Effect.all([ - image("auth", input.setup.authEnabledForSetup), - image("storage", input.setup.storageEnabledForSetup), - image("realtime", input.setup.realtimeEnabledForSetup), - ]); + const serviceKey = (pin: ArtifactPin, enabled: boolean) => + enabled ? pinKey(pin) : Effect.succeed(""); const keyInputs: ShadowCacheKeyInputs = { - postgresImage: yield* encodePostgresImageKey([ - "stack-v2", + postgresImage: yield* encodeKeyParts([ + "stack-v3", runtime, platform, arch, - postgres.image, + yield* pinKey(selection.database), ]), majorVersion: input.setup.majorVersion, jwtSecret: input.jwtSecret, @@ -74,9 +67,18 @@ export const stackShadowCacheEntry = Effect.fn("StackShadowCache.entry")(functio vault: input.setup.vault, jwks: "", services: { - auth: { enabled: input.setup.authEnabledForSetup, image: auth.image }, - storage: { enabled: input.setup.storageEnabledForSetup, image: storage.image }, - realtime: { enabled: input.setup.realtimeEnabledForSetup, image: realtime.image }, + auth: { + enabled: input.setup.authEnabledForSetup, + image: yield* serviceKey(selection.auth, input.setup.authEnabledForSetup), + }, + storage: { + enabled: input.setup.storageEnabledForSetup, + image: yield* serviceKey(selection.storage, input.setup.storageEnabledForSetup), + }, + realtime: { + enabled: input.setup.realtimeEnabledForSetup, + image: yield* serviceKey(selection.realtime, input.setup.realtimeEnabledForSetup), + }, }, }; const key = shadowCacheKey(keyInputs); diff --git a/apps/cli/src/command-internal/stack-shadow.integration.test.ts b/apps/cli/src/command-internal/stack-shadow.integration.test.ts index c724b6e1b8..3c75e7cd31 100644 --- a/apps/cli/src/command-internal/stack-shadow.integration.test.ts +++ b/apps/cli/src/command-internal/stack-shadow.integration.test.ts @@ -13,6 +13,7 @@ import { mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; import { DbConnection } from "./db-connection.service.ts"; import { dbConnectionLayer } from "./db-connection.sql-pg.layer.ts"; +import { loadLocalProjectContext } from "./local-project-context.ts"; import { StackApi, stackApiLayer } from "./stack-api.ts"; import { StackCatalogSetup, stackCatalogSetupLayer } from "./stack-catalog-setup.ts"; import { parseConnectionString } from "./db-config.parse.ts"; @@ -22,7 +23,14 @@ import { stackWithShadowDatabase, } from "./stack-shadow.ts"; import type { ShadowSetupInput } from "./db-bootstrap/shadow-database.ts"; -import { orioledbVersions } from "@supabase/stack/internal/artifacts"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; +import { + database15, + orioledb, + orioledbConfigVersion, + selection, +} from "../../tests/helpers/pins.ts"; +import { seedLock, unbundled } from "../../tests/helpers/stack-lock.ts"; const defaultConfig = Schema.decodeSync(CliConfigSchema)({}); @@ -522,60 +530,208 @@ describe("stack shadow databases", () => { 120_000, ); - it.live("creates the shadow database on the pinned OrioleDB build", () => - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-shadow-orioledb-pinned-" }); - const [orioledb = ""] = orioledbVersions(); - const versions: string[] = []; - // Stops at the database creation, so the test asserts the wiring without running OrioleDB. - const capturingApi = Layer.effect( - StackApi, - Effect.gen(function* () { - const api = yield* StackApi; - return StackApi.of({ - ...api, - create: (options) => - api.create(options).pipe( - Effect.map((stack) => ({ - ...stack, - services: { - ...stack.services, - create: (creation) => { - if (creation.service === "database") versions.push(creation.config.version); - return Effect.fail( - new StackError({ operation: "create", message: "captured" }), - ); - }, - }, - })), + for (const target of [ + { + name: "OrioleDB build", + db: { major_version: 17, orioledb_version: orioledbConfigVersion, settings: {} }, + pin: orioledb, + }, + { name: "stock Postgres 15 build", db: { major_version: 15, settings: {} }, pin: database15 }, + { + name: "database build the project lock pins", + db: { major_version: 17, settings: {} }, + pin: unbundled(selection.database), + locked: true, + }, + ]) { + it.live(`creates the shadow database on the selected ${target.name}`, () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-shadow-orioledb-pinned-" }); + const pins: ArtifactPin[] = []; + if (target.locked === true) { + yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(root, "supabase", "config.toml"), + 'project_id = "proj"\n', + ); + yield* seedLock(root, (services) => ({ + ...services, + database: { ...services.database, resolved: target.pin }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: root, supabaseHome: root }), ), - }); - }), - ).pipe(Layer.provide(stackApiLayer), Layer.provide(BunServices.layer)); - const setup = { - ...input(fs, path, root), - db: { major_version: 17, orioledb_version: orioledb, settings: {} }, - }; + ), + ); + } + // Stops at the database creation, so the test asserts the wiring without running OrioleDB. + const capturingApi = Layer.effect( + StackApi, + Effect.gen(function* () { + const api = yield* StackApi; + return StackApi.of({ + ...api, + create: (options) => + api.create(options).pipe( + Effect.map((stack) => ({ + ...stack, + services: { + ...stack.services, + create: (creation) => { + if (creation.service === "database") pins.push(creation.pin); + return Effect.fail( + new StackError({ operation: "create", message: "captured" }), + ); + }, + }, + })), + ), + }); + }), + ).pipe(Layer.provide(stackApiLayer), Layer.provide(BunServices.layer)); + const base = input(fs, path, root); + const setup = { + ...base, + db: target.db, + setup: { ...base.setup, majorVersion: target.db.major_version }, + }; - yield* stackWithShadowDatabase(setup, () => Effect.void, { runtime: "native" }).pipe( - Effect.provide( - Layer.mergeAll( - capturingApi, - stackCatalogSetupLayer, - dbConnectionLayer, - runtimeInfoLayer, - mockCommandSettings({ workdir: root, supabaseHome: root }), - mockOutput().layer, + yield* stackWithShadowDatabase(setup, () => Effect.void, { runtime: "native" }).pipe( + Effect.provide( + Layer.mergeAll( + capturingApi, + stackCatalogSetupLayer, + dbConnectionLayer, + runtimeInfoLayer, + mockCommandSettings({ workdir: root, supabaseHome: root }), + mockOutput().layer, + ), ), - ), - Effect.exit, - ); + Effect.exit, + ); - expect(versions).toEqual([`${orioledb}-orioledb`]); - }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), - ); + expect(pins).toEqual([target.pin]); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + ); + } + + for (const remote of [ + { + title: "a linked diff whose remote block sets major 15 runs the remote major's bundled pin", + major: 15, + remoteDb: "major_version = 15", + db: { major_version: 15, settings: {} }, + pin: database15, + }, + { + title: "a linked diff whose remote block sets major 17 keeps the lock's pin", + major: 17, + remoteDb: "major_version = 17", + db: { major_version: 17, settings: {} }, + pin: unbundled(selection.database), + }, + { + title: + "a linked diff whose remote block sets an OrioleDB version at the same major runs the bundled OrioleDB pin", + major: 17, + remoteDb: `orioledb_version = "${orioledbConfigVersion}"`, + db: { major_version: 17, orioledb_version: orioledbConfigVersion, settings: {} }, + pin: orioledb, + }, + ]) { + it.live(remote.title, () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-shadow-remote-major-" }); + const ref = "abcdefghijklmnopqrst"; + yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(root, "supabase", "config.toml"), + [ + `project_id = "${ref}"`, + "[db]", + "major_version = 17", + `[remotes.${ref.slice(0, 4)}]`, + `project_id = "${ref}"`, + `[remotes.${ref.slice(0, 4)}.db]`, + remote.remoteDb, + "", + ].join("\n"), + ); + const settings = Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: root, supabaseHome: root }), + ); + const locked = unbundled(selection.database); + yield* seedLock(root, (services) => ({ + ...services, + database: { ...services.database, resolved: locked }, + })).pipe(Effect.provide(settings)); + const context = yield* loadLocalProjectContext(root, (message) => message, ref).pipe( + Effect.provide(settings), + ); + expect(context.loaded?.appliedRemote).toBeDefined(); + expect(context.config.db.major_version).toBe(remote.major); + expect(context.config.db.orioledb_version).toBe(remote.db.orioledb_version); + + const pins: ArtifactPin[] = []; + const capturingApi = Layer.effect( + StackApi, + Effect.gen(function* () { + const api = yield* StackApi; + return StackApi.of({ + ...api, + create: (options) => + api.create(options).pipe( + Effect.map((stack) => ({ + ...stack, + services: { + ...stack.services, + create: (creation) => { + if (creation.service === "database") pins.push(creation.pin); + return Effect.fail( + new StackError({ operation: "create", message: "captured" }), + ); + }, + }, + })), + ), + }); + }), + ).pipe(Layer.provide(stackApiLayer), Layer.provide(BunServices.layer)); + const base = input(fs, path, root); + + yield* stackWithShadowDatabase( + { + ...base, + db: remote.db, + context, + setup: { ...base.setup, majorVersion: remote.major }, + }, + () => Effect.void, + { runtime: "native" }, + ).pipe( + Effect.provide( + Layer.mergeAll( + capturingApi, + stackCatalogSetupLayer, + dbConnectionLayer, + settings, + mockOutput().layer, + ), + ), + Effect.exit, + ); + + expect(pins).toEqual([remote.pin]); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + ); + } it.live("refuses an unpinned OrioleDB version before creating a shadow stack", () => Effect.gen(function* () { diff --git a/apps/cli/src/command-internal/stack-shadow.ts b/apps/cli/src/command-internal/stack-shadow.ts index a3de1f9ecf..442d62a99c 100644 --- a/apps/cli/src/command-internal/stack-shadow.ts +++ b/apps/cli/src/command-internal/stack-shadow.ts @@ -1,5 +1,6 @@ import { Effect, FileSystem, Path, Redacted, Result, Schema } from "effect"; -import type { DatabaseInstance, Stack } from "@supabase/stack/effect"; +import { type DatabaseInstance, type Stack } from "@supabase/stack/effect"; +import type { ArtifactPin, ArtifactSelection } from "@supabase/stack/internal/artifacts"; import { CommandSettings } from "../config/command-settings.service.ts"; import { Output } from "../shared/output/output.service.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; @@ -18,7 +19,7 @@ import type { SetupDatabaseOptions } from "./db-bootstrap/db-setup.ts"; import { listLocalMigrationPaths } from "./migration-history.ts"; import { applyMigrations } from "./migration-apply.ts"; import { stackShadowCacheEntry, stackShadowCacheRoles } from "./stack-shadow-cache.ts"; -import { stackDatabaseVersion } from "./stack-database-version.ts"; +import { selectionForRequest } from "./stack-project-selection.ts"; export interface StackShadowAcquiredHandle { readonly stack: Stack; @@ -27,7 +28,7 @@ export interface StackShadowAcquiredHandle { readonly host: string; readonly port: number; readonly runtime: StackRuntime; - readonly version: string; + readonly databasePin: ArtifactPin; /** Cache lineage when a baseline was restored or atomically published by this handle. */ readonly snapshotKey?: string; readonly restoredFromSnapshot: boolean; @@ -81,7 +82,7 @@ const acquireNamespace = Effect.fn("StackShadow.acquireNamespace")(function* (op const initialize = Effect.fn("StackShadow.initialize")(function* ( stack: Stack, runtime: StackRuntime, - version: string, + selection: ArtifactSelection, input: ShadowSetupInput, opts: ShadowOptions, ) { @@ -96,8 +97,8 @@ const initialize = Effect.fn("StackShadow.initialize")(function* ( const createDatabase = () => stack.services.create({ service: "database", + pin: selection.database, config: { - version, databasePassword: Redacted.make(input.password), jwtSecret: Redacted.make(input.jwtSecret), jwtExpiry: input.jwtExpiry, @@ -113,7 +114,7 @@ const initialize = Effect.fn("StackShadow.initialize")(function* ( const cacheResolution = yield* Effect.result( stackShadowCacheEntry( input, - version, + selection, runtime, runtimeInfo.platform, runtimeInfo.arch, @@ -163,6 +164,7 @@ const initialize = Effect.fn("StackShadow.initialize")(function* ( target: { stack, database, + selection, databaseServices: [ ...(input.setup.authEnabledForSetup ? ["auth" as const] : []), ...(input.setup.storageEnabledForSetup ? ["storage" as const] : []), @@ -219,7 +221,7 @@ const initialize = Effect.fn("StackShadow.initialize")(function* ( host: conn.host, port: conn.port, runtime, - version, + databasePin: selection.database, snapshotKey, restoredFromSnapshot: restored, } satisfies StackShadowAcquiredHandle; @@ -234,14 +236,21 @@ export const stackAcquireShadowDatabase = Effect.fn("StackShadow.acquire")(funct opts: ShadowOptions = {}, ) { const output = yield* Output; - // The shadow runs the project's database line, so it matches the local database. - const version = yield* Effect.fromResult( - stackDatabaseVersion({ - major_version: input.setup.majorVersion, - orioledb_version: input.db.orioledb_version, - }), - ).pipe( - Effect.mapError((message) => new ShadowDbError({ message, reason: "container_configuration" })), + const selection = yield* selectionForRequest({ + projectRoot: input.workdir, + majorVersion: input.setup.majorVersion, + orioledbVersion: input.db.orioledb_version, + services: input.context?.config.local?.services, + appliedRemote: input.context?.loaded?.appliedRemote, + }).pipe( + Effect.mapError( + (error) => + new ShadowDbError({ + message: error.message, + reason: "container_configuration", + cause: error, + }), + ), ); const namespace = yield* Effect.acquireRelease(acquireNamespace(opts), ({ stack }) => stack.destroy.pipe( @@ -250,7 +259,7 @@ export const stackAcquireShadowDatabase = Effect.fn("StackShadow.acquire")(funct ), ), ); - return yield* initialize(namespace.stack, namespace.runtime, version, input, opts); + return yield* initialize(namespace.stack, namespace.runtime, selection, input, opts); }, Effect.mapError(shadowError)); /** Runs a command against a fresh shadow, then destroys its owned namespace. */ diff --git a/apps/cli/src/command-internal/stack-storage.native.integration.test.ts b/apps/cli/src/command-internal/stack-storage.native.integration.test.ts index e558a0c4cf..1c29a13e9d 100644 --- a/apps/cli/src/command-internal/stack-storage.native.integration.test.ts +++ b/apps/cli/src/command-internal/stack-storage.native.integration.test.ts @@ -66,6 +66,7 @@ const startFlags = () => ({ stackId: Option.none(), runtime: "native" as const, preparation: "on-demand" as const, + frozen: false, eager: false, }); diff --git a/apps/cli/src/command-internal/start-secrets-cleanup.ts b/apps/cli/src/command-internal/start-secrets-cleanup.ts index 40a0f12235..daf10a24cb 100644 --- a/apps/cli/src/command-internal/start-secrets-cleanup.ts +++ b/apps/cli/src/command-internal/start-secrets-cleanup.ts @@ -1,5 +1,7 @@ import { Effect, FileSystem, Path } from "effect"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; + import type { ContainerIdName } from "./docker-lifecycle.ts"; /** @@ -22,7 +24,7 @@ export const cleanupStartSecrets = Effect.fnUntraced(function* ( containers, (container) => { const workdir = container.workdir.length > 0 ? container.workdir : fallbackWorkdir; - const stagingRoot = path.resolve(workdir, "supabase", ".temp", "start-secrets"); + const stagingRoot = legacyProjectStatePaths({ join: path.resolve }, workdir).startSecretsDir; const target = path.resolve(stagingRoot, container.name); if (target === stagingRoot || !target.startsWith(stagingRoot + path.sep)) { return Effect.void; diff --git a/apps/cli/src/command-internal/storage-runtime.layer.ts b/apps/cli/src/command-internal/storage-runtime.layer.ts index e18b86dbb8..c21642f664 100644 --- a/apps/cli/src/command-internal/storage-runtime.layer.ts +++ b/apps/cli/src/command-internal/storage-runtime.layer.ts @@ -6,8 +6,8 @@ import { commandPlatformApiFactoryLayer } from "../auth/command-platform-api-fac import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; import { commandSettingsLayer } from "../config/command-settings.layer.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; -import { projectRefLayer } from "../config/project-ref.layer.ts"; -import { ProjectRefResolver } from "../config/project-ref.service.ts"; +import { remoteTargetLayer } from "../config/remote-target.layer.ts"; +import { RemoteTarget } from "../config/remote-target.service.ts"; import { debugLoggerLayer } from "./debug-logger.layer.ts"; import { IdentityStitch, identityStitchLayer } from "./identity-stitch.ts"; import { httpClientLayer } from "../auth/http-debug.layer.ts"; @@ -47,7 +47,7 @@ export function storageGatewayRuntimeLayer(subcommand: ReadonlyArray) { cliSettings, platformApiFactory, httpClient, - projectRefLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), + remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), linkedProjectCacheLayer.pipe( Layer.provide(credentials), Layer.provide(cliSettings), @@ -74,7 +74,7 @@ export function storageGatewayRuntimeLayer(subcommand: ReadonlyArray) { type StorageGatewayServices = | CommandPlatformApiFactory | CommandSettings - | ProjectRefResolver + | RemoteTarget | LinkedProjectCache | TelemetryState | IdentityStitch diff --git a/apps/cli/src/command-internal/temp-paths.ts b/apps/cli/src/command-internal/temp-paths.ts deleted file mode 100644 index b77f4f7240..0000000000 --- a/apps/cli/src/command-internal/temp-paths.ts +++ /dev/null @@ -1,81 +0,0 @@ -import { Data, Effect, FileSystem, Option, type Path } from "effect"; -import { - actionability, - type CliErrorActionabilityDeclaration, - ErrorActionabilityId, -} from "../shared/telemetry/error-actionability.ts"; - -/** - * A real failure reading `/supabase/.temp/project-ref` (e.g. the - * path is a directory, or permissions deny access) — distinct from the file - * simply not existing, which means "not linked" rather than an error. - */ -export class ProjectRefReadError extends Data.TaggedError("ProjectRefReadError")<{ - readonly message: string; -}> { - get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { - return actionability.permission; - } -} - -/** - * Absolute paths to the established files under `/supabase/.temp/`. - * `supabase link`/`unlink` own writing and removing this directory; other - * layers only read from it. - */ -export interface TempPaths { - readonly tempDir: string; - readonly projectRef: string; - readonly poolerUrl: string; - readonly postgresVersion: string; - readonly restVersion: string; - readonly gotrueVersion: string; - readonly storageVersion: string; - readonly storageMigration: string; - readonly linkedProjectCache: string; -} - -export function tempPaths(path: Path.Path, workdir: string): TempPaths { - const tempDir = path.join(workdir, "supabase", ".temp"); - return { - tempDir, - projectRef: path.join(tempDir, "project-ref"), - poolerUrl: path.join(tempDir, "pooler-url"), - postgresVersion: path.join(tempDir, "postgres-version"), - restVersion: path.join(tempDir, "rest-version"), - gotrueVersion: path.join(tempDir, "gotrue-version"), - storageVersion: path.join(tempDir, "storage-version"), - storageMigration: path.join(tempDir, "storage-migration"), - linkedProjectCache: path.join(tempDir, "linked-project.json"), - }; -} - -/** - * Reads the linked project ref from `/supabase/.temp/project-ref`, - * returning `None` when the file is absent or blank. A missing file means - * "not linked"; any other read error (a directory, permission denied, …) - * fails instead of being swallowed into an unlinked result. - */ -export const readProjectRefFile = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, -): Effect.Effect, ProjectRefReadError> => - Effect.gen(function* () { - const refPath = tempPaths(path, workdir).projectRef; - // A `NotFound` PlatformError means unlinked (fall through); any other - // read error fails. - const content = yield* fs.readFileString(refPath).pipe( - Effect.catchTag("PlatformError", (error) => - error.reason._tag === "NotFound" - ? Effect.succeed("") - : Effect.fail( - new ProjectRefReadError({ - message: `failed to load project ref: ${error.message}`, - }), - ), - ), - ); - const trimmed = content.trim(); - return trimmed.length === 0 ? Option.none() : Option.some(trimmed); - }); diff --git a/apps/cli/src/command-internal/tenant-versions.integration.test.ts b/apps/cli/src/command-internal/tenant-versions.integration.test.ts new file mode 100644 index 0000000000..abc7bfb57b --- /dev/null +++ b/apps/cli/src/command-internal/tenant-versions.integration.test.ts @@ -0,0 +1,90 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect, Fiber, Layer } from "effect"; +import * as FetchHttpClient from "effect/http/FetchHttpClient"; +import * as HttpClient from "effect/http/HttpClient"; +import * as TestClock from "effect/testing/TestClock"; + +import { fetchGotrueVersion, probeTenantVersions } from "./tenant-versions.ts"; + +const tenant = { + ref: "abcdefghijklmnopqrst", + projectHost: "supabase.test", + serviceKey: "service-role-key", + userAgent: "test", +}; + +const hangingClient = Layer.succeed( + HttpClient.HttpClient, + HttpClient.make(() => Effect.never), +); + +describe("probeTenantVersions", () => { + it.effect("reports every service of a tenant that never answers as failed", () => + Effect.gen(function* () { + const probe = yield* probeTenantVersions(tenant).pipe( + Effect.forkChild({ startImmediately: true }), + ); + + yield* TestClock.adjust("10 seconds"); + + const observed = yield* Fiber.join(probe); + expect(observed.rest._tag).toBe("Failed"); + expect(observed.auth._tag).toBe("Failed"); + expect(observed.storage._tag).toBe("Failed"); + }).pipe(Effect.provide(hangingClient)), + ); +}); + +const serve = (fetch: (request: Request) => Response) => + Effect.acquireRelease( + Effect.try(() => Bun.serve({ hostname: "127.0.0.1", port: 0, fetch })), + (server) => Effect.promise(() => server.stop(true)), + ); + +/** A tenant host that redirects every request to another origin, which records the `apikey` it receives. */ +const redirectingTenant = Effect.gen(function* () { + const leakedKeys: Array = []; + const elsewhere = yield* serve((request) => { + leakedKeys.push(request.headers.get("apikey")); + return Response.json({ version: "v2.0.0", info: { version: "v12.0.0" } }); + }); + const host = yield* serve(() => Response.redirect(`http://localhost:${elsewhere.port}/`, 302)); + const fetch = yield* FetchHttpClient.Fetch; + const toTenantHost: typeof globalThis.fetch = Object.assign( + (input: string | URL | Request, init?: RequestInit) => + fetch(String(input).replace(/^https:\/\/[^/]+/u, `http://127.0.0.1:${host.port}`), init), + { preconnect: () => Promise.resolve() }, + ); + const withTenantHost = (effect: Effect.Effect) => + effect.pipe( + Effect.provide(FetchHttpClient.layer), + Effect.provideService(FetchHttpClient.Fetch, toTenantHost), + ); + return { leakedKeys, withTenantHost }; +}); + +describe("tenant version requests", () => { + it.live("report a tenant redirect as a failed probe without sending the key onward", () => + Effect.gen(function* () { + const { leakedKeys, withTenantHost } = yield* redirectingTenant; + + const observed = yield* withTenantHost(probeTenantVersions(tenant)); + + expect(observed.auth).toEqual({ _tag: "Failed", cause: "HTTP 302" }); + expect(observed.rest).toEqual({ _tag: "Failed", cause: "HTTP 302" }); + expect(observed.storage).toEqual({ _tag: "Failed", cause: "HTTP 302" }); + expect(leakedKeys).toEqual([]); + }).pipe(Effect.scoped), + ); + + it.live("never follow a tenant redirect when link reads a single service version", () => + Effect.gen(function* () { + const { leakedKeys, withTenantHost } = yield* redirectingTenant; + + const version = yield* withTenantHost(fetchGotrueVersion(tenant)); + + expect(version._tag).toBe("None"); + expect(leakedKeys).toEqual([]); + }).pipe(Effect.scoped), + ); +}); diff --git a/apps/cli/src/command-internal/tenant-versions.ts b/apps/cli/src/command-internal/tenant-versions.ts index e5f6b0df78..a7a5f73863 100644 --- a/apps/cli/src/command-internal/tenant-versions.ts +++ b/apps/cli/src/command-internal/tenant-versions.ts @@ -1,6 +1,8 @@ -import { Effect, Option } from "effect"; +import { Duration, Effect, Option, Predicate } from "effect"; +import * as FetchHttpClient from "effect/http/FetchHttpClient"; import * as HttpClient from "effect/http/HttpClient"; import * as HttpClientRequest from "effect/http/HttpClientRequest"; +import type * as HttpClientResponse from "effect/http/HttpClientResponse"; import { tagForServiceVersion } from "../shared/services/services.shared.ts"; @@ -8,12 +10,13 @@ import { tagForServiceVersion } from "../shared/services/services.shared.ts"; * Best-effort probes for the deployed versions of a project's REST, Auth, * and Storage services, used by `supabase link`. Requests use the * service-role key: always `apikey: `, plus `Authorization: - * Bearer ` unless the key is a new-style `sb_…` key. Any - * transport error, non-200 status, parse failure, or missing version - * resolves to `Option.none()` instead of failing the link. + * Bearer ` unless the key is a new-style `sb_…` key. The legacy + * fetchers resolve any transport error, non-200 status, parse failure, or + * missing version to `Option.none()`; {@link probeTenantVersions} keeps + * "the service does not report a version" apart from "the probe failed". */ -interface TenantVersionOptions { +export interface TenantVersionOptions { readonly ref: string; readonly projectHost: string; readonly serviceKey: string; @@ -25,14 +28,17 @@ interface TenantVersionOptions { * whitespace-delimited token becomes the image tag. Empty means not found. */ export function parsePostgrestVersion(body: unknown): Option.Option { + return Option.map(postgrestRaw(body), (raw) => tagForServiceVersion("postgrest", raw)); +} + +function postgrestRaw(body: unknown): Option.Option { if (typeof body !== "object" || body === null) return Option.none(); const info = (body as { info?: unknown }).info; if (typeof info !== "object" || info === null) return Option.none(); const version = (info as { version?: unknown }).version; if (typeof version !== "string" || version.trim().length === 0) return Option.none(); const first = version.trim().split(/\s+/)[0]; - if (first === undefined || first.length === 0) return Option.none(); - return Option.some(tagForServiceVersion("postgrest", first)); + return first === undefined || first.length === 0 ? Option.none() : Option.some(first); } /** @@ -51,9 +57,12 @@ export function parseGotrueVersion(body: unknown): Option.Option { * Any other body becomes the storage image tag. */ export function parseStorageVersion(body: string): Option.Option { + return Option.map(storageRaw(body), (raw) => tagForServiceVersion("storage", raw)); +} + +function storageRaw(body: string): Option.Option { const version = body.trim(); - if (version.length === 0 || version === "0.0.0") return Option.none(); - return Option.some(tagForServiceVersion("storage", version)); + return version.length === 0 || version === "0.0.0" ? Option.none() : Option.some(version); } function tenantRequest(opts: TenantVersionOptions, pathName: string) { @@ -69,18 +78,25 @@ function tenantRequest(opts: TenantVersionOptions, pathName: string) { return request; } -const fetchJson = (request: HttpClientRequest.HttpClientRequest) => +/** Fetch keeps the `apikey` header across a cross-origin redirect, so a tenant 3xx is returned, never followed. */ +const executeTenant = (request: HttpClientRequest.HttpClientRequest) => Effect.gen(function* () { const httpClient = yield* HttpClient.HttpClient; - const response = yield* httpClient.execute(request); + return yield* httpClient + .execute(request) + .pipe(Effect.provideService(FetchHttpClient.RequestInit, { redirect: "manual" })); + }); + +const fetchJson = (request: HttpClientRequest.HttpClientRequest) => + Effect.gen(function* () { + const response = yield* executeTenant(request); if (response.status !== 200) return Option.none(); return Option.some(yield* response.json); }).pipe(Effect.orElseSucceed(() => Option.none())); const fetchText = (request: HttpClientRequest.HttpClientRequest) => Effect.gen(function* () { - const httpClient = yield* HttpClient.HttpClient; - const response = yield* httpClient.execute(request); + const response = yield* executeTenant(request); if (response.status !== 200) return Option.none(); return Option.some(yield* response.text); }).pipe(Effect.orElseSucceed(() => Option.none())); @@ -111,3 +127,101 @@ export const fetchStorageVersion = ( Option.isNone(body) ? Option.none() : parseStorageVersion(body.value), ), ); + +/** What probing a hosted service for its version found. */ +export type HostedObservation = + | { + readonly _tag: "Observed"; + /** The value the platform reported. */ + readonly raw: string; + /** The upstream version, spelled as its release tag. */ + readonly version: string; + } + | { readonly _tag: "Unreported" } + | { readonly _tag: "Failed"; readonly cause: string }; + +type Fetched = + | { readonly _tag: "Body"; readonly body: A } + | { readonly _tag: "Unreported" } + | { readonly _tag: "Failed"; readonly cause: string }; + +export const probeFailure = ( + error: unknown, +): { readonly _tag: "Failed"; readonly cause: string } => ({ + _tag: "Failed", + cause: Predicate.hasProperty(error, "_tag") + ? `request failed (${String(error._tag)})` + : "request failed", +}); + +const PROBE_REQUEST_TIMEOUT = Duration.seconds(10); + +/** A 404 means the service has no version endpoint; any other non-200 status is a failed probe. */ +const fetchBody = ( + request: HttpClientRequest.HttpClientRequest, + read: (response: HttpClientResponse.HttpClientResponse) => Effect.Effect, +): Effect.Effect, never, HttpClient.HttpClient> => + Effect.gen(function* () { + const response = yield* executeTenant(request); + if (response.status === 404) return { _tag: "Unreported" } as const; + if (response.status !== 200) + return { _tag: "Failed", cause: `HTTP ${response.status}` } as const; + return { _tag: "Body", body: yield* read(response) } as const; + }).pipe( + Effect.timeout(PROBE_REQUEST_TIMEOUT), + Effect.catch((error) => Effect.succeed(probeFailure(error))), + ); + +const probeService = ( + opts: TenantVersionOptions, + service: "postgrest" | "auth" | "storage", + pathName: string, + read: (response: HttpClientResponse.HttpClientResponse) => Effect.Effect, + rawOf: (body: A) => Option.Option, +) => + fetchBody(tenantRequest(opts, pathName), read).pipe( + Effect.map((fetched): HostedObservation => { + if (fetched._tag !== "Body") return fetched; + return Option.match(rawOf(fetched.body), { + onNone: () => ({ _tag: "Unreported" }), + onSome: (raw) => ({ _tag: "Observed", raw, version: tagForServiceVersion(service, raw) }), + }); + }), + ); + +/** The hosted versions the tenant endpoints report; Realtime and the other services have no endpoint. */ +interface TenantObservations { + readonly auth: HostedObservation; + readonly rest: HostedObservation; + readonly storage: HostedObservation; +} + +/** + * Probes `rest`, `auth` and `storage` on the project's tenant host, concurrently. A service that + * answers 404 or an empty version is `Unreported`; a transport error or any other status is `Failed`. + */ +export const probeTenantVersions = Effect.fn("TenantVersions.probe")(function* ( + opts: TenantVersionOptions, +) { + const [rest, auth, storage] = yield* Effect.all( + [ + probeService(opts, "postgrest", "/rest/v1/", (response) => response.json, postgrestRaw), + probeService( + opts, + "auth", + "/auth/v1/health", + (response) => response.json, + parseGotrueVersion, + ), + probeService(opts, "storage", "/storage/v1/version", (response) => response.text, storageRaw), + ], + { concurrency: 3 }, + ); + const observations: TenantObservations = { auth, rest, storage }; + const found = Object.values(observations); + yield* Effect.annotateCurrentSpan({ + "tenant.observed_count": found.filter(({ _tag }) => _tag === "Observed").length, + "tenant.failed_count": found.filter(({ _tag }) => _tag === "Failed").length, + }); + return observations; +}); diff --git a/apps/cli/src/command-internal/test-db.handler.ts b/apps/cli/src/command-internal/test-db.handler.ts index df4acb13cd..445cc5cbcf 100644 --- a/apps/cli/src/command-internal/test-db.handler.ts +++ b/apps/cli/src/command-internal/test-db.handler.ts @@ -1,6 +1,11 @@ import { Effect, FileSystem, Option, Path } from "effect"; import { type DatabaseInstance, type Stack } from "@supabase/stack/effect"; import { postgres } from "@supabase/stack/commands"; +import { + databaseArtifactPin, + isOrioledbVersion, + type ArtifactPin, +} from "@supabase/stack/internal/artifacts"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; @@ -58,7 +63,7 @@ type ManagedStack = { readonly stack: Stack; readonly database: DatabaseInstance; readonly connection: PgConnInput; - readonly major: 15 | 17; + readonly pin: ArtifactPin; }; const managedStackFor = Effect.fn("test.db.managedStack")(function* () { @@ -83,11 +88,18 @@ const managedStackFor = Effect.fn("test.db.managedStack")(function* () { return yield* new LocalDbRunningError({ message: "The local stack primary service is not a database.", }); - const major = pgProveMajor(status.config.config.version); + const savedPin = status.config.pin; + const major = pgProveMajor(savedPin.upstreamVersion); if (major === undefined) return yield* new LocalDbRunningError({ - message: `The local database version ${status.config.config.version} is not supported by pg_prove.`, + message: `The local database version ${savedPin.upstreamVersion} is not supported by pg_prove.`, }); + // An OrioleDB database runs pg_prove from the stock build of its major. + const pin = isOrioledbVersion(savedPin.upstreamVersion) + ? yield* databaseArtifactPin(String(major)).pipe( + Effect.mapError((cause) => new LocalDbRunningError({ message: cause.message })), + ) + : savedPin; const credentials = yield* database.credentials({ from: "runtime" }).pipe( Effect.mapError( (cause) => @@ -106,7 +118,7 @@ const managedStackFor = Effect.fn("test.db.managedStack")(function* () { return yield* new LocalDbRunningError({ message: "The local database returned malformed runtime credentials.", }); - return { stack, database, connection, major } satisfies ManagedStack; + return { stack, database, connection, pin } satisfies ManagedStack; }); function sanitizeProjectId(src: string): string { @@ -296,7 +308,7 @@ export const testDb = Effect.fn("test.db")(function* (flags: TestDbFlags) { ? path.dirname(hostPath) : hostPath; return yield* managedStack.stack.commands - .run(postgres.pgProve({ major: managedStack.major }), { + .run(postgres.pgProve({ pin: managedStack.pin }), { args: args.cmd.slice(1), env: runEnv, pgProve: { @@ -317,7 +329,9 @@ export const testDb = Effect.fn("test.db")(function* (flags: TestDbFlags) { if (backend.kind === "stack") { const bundled = yield* BundledPostgresClient; const toml = yield* readDbToml(fs, path, cliSettings.workdir); - const version = String(toml.majorVersion); + const pin = yield* databaseArtifactPin(String(toml.majorVersion)).pipe( + Effect.mapError((cause) => new TestDbRunError({ message: cause.message, cause })), + ); const hostPath = args.hostPaths[0]; const hostWorkingDir = hostPath === undefined @@ -328,7 +342,7 @@ export const testDb = Effect.fn("test.db")(function* (flags: TestDbFlags) { const nativeArgs = ["pg_prove", "--ext", ".pg", "--ext", ".sql", "-r", ...args.hostPaths]; if (debug) nativeArgs.push("--verbose"); return yield* bundled.run({ - version, + pin, runtime: proveRuntime, argv: useNativeProve ? nativeArgs : args.cmd, env: runEnv, diff --git a/apps/cli/src/command-internal/test-db.native.integration.test.ts b/apps/cli/src/command-internal/test-db.native.integration.test.ts index 27583e513e..cea54e8411 100644 --- a/apps/cli/src/command-internal/test-db.native.integration.test.ts +++ b/apps/cli/src/command-internal/test-db.native.integration.test.ts @@ -27,14 +27,15 @@ import type { InitializationCommandOptions, PostgresCommandOptions } from "@supa import type { Stack } from "@supabase/stack/effect"; import type { InitializationCommand, PostgresCommand } from "@supabase/stack/commands"; import { destroyTestStack } from "../../tests/helpers/stack-cleanup.ts"; +import { selection } from "../../tests/helpers/pins.ts"; const runtimes = ["native", "docker"] as const; const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); const databaseCreation = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("cli-pgtap-password"), jwtSecret: Redacted.make("cli-pgtap-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, diff --git a/apps/cli/src/command-internal/upgrade-notice.integration.test.ts b/apps/cli/src/command-internal/upgrade-notice.integration.test.ts index a5584e17c7..34232a2d5c 100644 --- a/apps/cli/src/command-internal/upgrade-notice.integration.test.ts +++ b/apps/cli/src/command-internal/upgrade-notice.integration.test.ts @@ -6,6 +6,7 @@ import { Clock, DateTime, Effect, FileSystem, Layer, Path } from "effect"; import { withConfigEnv } from "../../tests/helpers/command-mocks.ts"; import { runtimeInfoLayer } from "../shared/runtime/runtime-info.layer.ts"; +import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { upgradeNoticeHook } from "./upgrade-notice.ts"; const captureStderr = (body: Effect.Effect) => @@ -34,7 +35,14 @@ describe("upgrade notice user-level cache", () => { const workdir = path.join(root, "outside-project"); const supabaseHome = path.join(root, "supabase-home"); yield* fs.makeDirectory(workdir); - return { fs, workdir, supabaseHome, cacheFile: path.join(supabaseHome, "cli-latest") }; + return { + fs, + path, + root, + workdir, + supabaseHome, + cacheFile: path.join(supabaseHome, "cli-latest"), + }; }); /** Ages the cache against the live clock, which the hook's freshness check reads. */ @@ -53,17 +61,29 @@ describe("upgrade notice user-level cache", () => { const runHook = Effect.fnUntraced(function* ( { workdir, supabaseHome }: Effect.Success, latestTag: string, + options: { + /** `"config"` leaves the env unset so the project's config decides the mode. */ + readonly stack?: boolean | "config"; + readonly args?: ReadonlyArray; + readonly workingDirectory?: string; + } = {}, ) { let fetchCalls = 0; const written = yield* captureStderr( withConfigEnv( - { SUPABASE_HOME: supabaseHome, SUPABASE_NO_UPDATE_NOTIFIER: "0" }, + { + SUPABASE_HOME: supabaseHome, + SUPABASE_NO_UPDATE_NOTIFIER: "0", + ...(options.stack === "config" + ? {} + : { SUPABASE_EXPERIMENTAL_STACK: options.stack === true ? "1" : "0" }), + }, upgradeNoticeHook( - ["projects", "list"], + [...(options.args ?? []), "projects", "list"], { cleanShowHelp: false, delegatedToGo: false, - workingDirectory: workdir, + workingDirectory: options.workingDirectory ?? workdir, isValueTakingFlagToken: () => false, }, Effect.sync(() => { @@ -99,4 +119,88 @@ describe("upgrade notice user-level cache", () => { expect(yield* ctx.fs.readFileString(ctx.cacheFile)).toBe("v99.99.100"); }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))), ); + + describe("inside a project", () => { + const withLegacyCache = Effect.gen(function* () { + const ctx = yield* userCache; + const tempDir = ctx.path.join(ctx.workdir, "supabase", ".temp"); + yield* ctx.fs.makeDirectory(tempDir, { recursive: true }); + const legacyCache = ctx.path.join(tempDir, "cli-latest"); + yield* ctx.fs.writeFileString(legacyCache, "v99.99.99"); + return { ...ctx, legacyCache }; + }); + + it.live("serves the project's supabase/.temp cache in legacy mode", () => + Effect.gen(function* () { + const ctx = yield* withLegacyCache; + + const { fetchCalls, stderr } = yield* runHook(ctx, "v99.99.100"); + + expect(fetchCalls).toBe(0); + expect(stderr).toContain("A new version of Supabase CLI is available: v99.99.99"); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))), + ); + + it.live("keeps the cache in SUPABASE_HOME and leaves supabase/.temp alone in stack mode", () => + Effect.gen(function* () { + const ctx = yield* withLegacyCache; + + const { fetchCalls, stderr } = yield* runHook(ctx, "v99.99.100", { stack: true }); + + expect(fetchCalls).toBe(1); + expect(stderr).toContain("A new version of Supabase CLI is available: v99.99.100"); + expect(yield* ctx.fs.readFileString(ctx.cacheFile)).toBe("v99.99.100"); + expect(yield* ctx.fs.readFileString(ctx.legacyCache)).toBe("v99.99.99"); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))), + ); + + it.live("keeps a stack-mode project out of supabase/.temp when --workdir is relative", () => + Effect.gen(function* () { + const ctx = yield* userCache; + const project = ctx.path.join(ctx.root, "stack-project"); + yield* ctx.fs.makeDirectory(ctx.path.join(project, "supabase"), { recursive: true }); + yield* ctx.fs.writeFileString( + ctx.path.join(project, "supabase", "config.toml"), + 'project_id = "demo"\n\n[experimental]\nstack = true\n', + ); + const runtime = yield* RuntimeInfo; + + const { fetchCalls } = yield* runHook(ctx, "v99.99.100", { + stack: "config", + args: ["--workdir", "stack-project"], + workingDirectory: project, + }).pipe(Effect.provideService(RuntimeInfo, { ...runtime, cwd: ctx.root })); + + expect(fetchCalls).toBe(1); + expect(yield* ctx.fs.readFileString(ctx.cacheFile)).toBe("v99.99.100"); + expect( + yield* ctx.fs.exists(ctx.path.join(project, "supabase", ".temp", "cli-latest")), + ).toBe(false); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))), + ); + + it.live( + "reads the mode from the success working directory, not the directory the command started in", + () => + Effect.gen(function* () { + const ctx = yield* userCache; + const project = ctx.path.join(ctx.root, "bootstrapped-project"); + yield* ctx.fs.makeDirectory(ctx.path.join(project, "supabase"), { recursive: true }); + yield* ctx.fs.writeFileString( + ctx.path.join(project, "supabase", "config.toml"), + 'project_id = "demo"\n\n[experimental]\nstack = true\n', + ); + const runtime = yield* RuntimeInfo; + + const { fetchCalls } = yield* runHook(ctx, "v99.99.100", { + stack: "config", + workingDirectory: project, + }).pipe(Effect.provideService(RuntimeInfo, { ...runtime, cwd: ctx.workdir })); + + expect(fetchCalls).toBe(1); + expect(yield* ctx.fs.readFileString(ctx.cacheFile)).toBe("v99.99.100"); + expect(yield* ctx.fs.exists(ctx.path.join(project, "supabase", ".temp"))).toBe(false); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))), + ); + }); }); diff --git a/apps/cli/src/command-internal/upgrade-notice.ts b/apps/cli/src/command-internal/upgrade-notice.ts index 3ced14525a..57c60cadfb 100644 --- a/apps/cli/src/command-internal/upgrade-notice.ts +++ b/apps/cli/src/command-internal/upgrade-notice.ts @@ -2,8 +2,9 @@ * Post-command upgrade notice: checks GitHub's latest release against the * running version and prints a notice to stderr, honoring * `SUPABASE_NO_UPDATE_NOTIFIER`. The tag is cached in the project's - * `supabase/.temp/cli-latest`, else `/cli-latest`. A failed - * fetch writes an empty cache as an offline backoff. + * `supabase/.temp/cli-latest` in legacy project-state mode, else + * `/cli-latest`. A failed fetch writes an empty cache as an + * offline backoff. */ import { stripVTControlCharacters } from "node:util"; @@ -36,8 +37,10 @@ import { ErrorActionabilityId, } from "../shared/telemetry/error-actionability.ts"; import { bold, yellow } from "./colors.ts"; +import { legacyProjectStatePaths } from "./legacy-project-state.paths.ts"; import { parseDotEnv } from "./dotenv.ts"; import { candidateDotenvFilenames } from "./project-environment.ts"; +import { projectStateMode } from "./project-state-mode.ts"; const LATEST_RELEASE_URL = "https://api.github.com/repos/supabase/cli/releases/latest"; const CACHE_TTL_MS = 10 * 60 * 60 * 1000; @@ -259,8 +262,10 @@ export interface UpgradeNoticeDeps { readonly cwd: string; readonly resolvedCwd?: string; readonly currentVersion: string; - /** The cache directory when `/supabase` is not a real directory. */ + /** The cache directory outside a legacy-mode project. */ readonly supabaseHome: string; + /** Whether the project keeps its state in `supabase/.temp` rather than `$SUPABASE_HOME`. */ + readonly usesLegacyProjectState: Effect.Effect; readonly now: () => number; readonly fetchLatestTag: Effect.Effect; readonly writeStderr: (text: string) => void; @@ -300,10 +305,12 @@ export const runUpgradeNotice = Effect.fnUntraced(function* ( if (updateNotifierDisabled(effectiveEnv("SUPABASE_NO_UPDATE_NOTIFIER"))) return; const debug = debugEnabled(deps, builtin, effectiveEnv("SUPABASE_DEBUG")); const supabaseDir = path.join(base, "supabase"); - const inProject = isDirectoryEntry(yield* lstatEntry(fs, supabaseDir)); + const inProject = + isDirectoryEntry(yield* lstatEntry(fs, supabaseDir)) && (yield* deps.usesLegacyProjectState); /* `resolve` drops a trailing `/` or `/.`, which would make the symlink check follow a symlinked home. */ - const cacheDir = inProject ? path.join(supabaseDir, ".temp") : path.resolve(deps.supabaseHome); - const cacheFile = path.join(cacheDir, "cli-latest"); + const projectState = legacyProjectStatePaths(path, base); + const cacheDir = inProject ? projectState.tempDir : path.resolve(deps.supabaseHome); + const cacheFile = inProject ? projectState.cliLatest : path.join(cacheDir, "cli-latest"); // A hostile checkout can commit a symlink at any level of this well-known // path to clobber an arbitrary user-writable file (CWE-59), so a symlink @@ -431,6 +438,7 @@ const NOTICE_ENV_KEYS = [ "SUPABASE_DEBUG", "SUPABASE_WORKDIR", "SUPABASE_ENV", + "SUPABASE_EXPERIMENTAL_STACK", ]; /** @@ -471,6 +479,18 @@ export const upgradeNoticeHook = ( resolvedCwd: info.workingDirectory, currentVersion: CLI_VERSION, supabaseHome: resolveSupabaseHomeValue(path, supabaseHome, runtime.homeDir), + /* An unreadable mode keeps the cache out of the project; a relative `--workdir` resolves against the process cwd. */ + usesLegacyProjectState: projectStateMode( + info.workingDirectory === undefined + ? { args, cwd: runtime.cwd, env } + : { + cwd: info.workingDirectory, + env: { SUPABASE_EXPERIMENTAL_STACK: env["SUPABASE_EXPERIMENTAL_STACK"] }, + }, + ).pipe( + Effect.map((mode) => mode === "legacy"), + Effect.orElseSucceed(() => false), + ), now: () => clock.currentTimeMillisUnsafe(), /* Runs inside the check span, so the fetch is its child span. */ fetchLatestTag: fetchLatestTag.pipe( diff --git a/apps/cli/src/command-internal/upgrade-notice.unit.test.ts b/apps/cli/src/command-internal/upgrade-notice.unit.test.ts index 1c692c5250..04a152e3e6 100644 --- a/apps/cli/src/command-internal/upgrade-notice.unit.test.ts +++ b/apps/cli/src/command-internal/upgrade-notice.unit.test.ts @@ -144,6 +144,7 @@ layer(BunServices.layer)("runUpgradeNotice", (it) => { cwd: workdir, currentVersion: opts.currentVersion ?? "2.113.0", supabaseHome, + usesLegacyProjectState: Effect.succeed(true), now: () => clock.currentTimeMillisUnsafe(), fetchLatestTag: Effect.suspend(() => { fetchCalls += 1; diff --git a/apps/cli/src/commands/backups/list/list.command.ts b/apps/cli/src/commands/backups/list/list.command.ts index 942b76f10c..eb28f1d634 100644 --- a/apps/cli/src/commands/backups/list/list.command.ts +++ b/apps/cli/src/commands/backups/list/list.command.ts @@ -3,7 +3,7 @@ import { Command, Flag } from "effect/cli"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/backups/list/list.handler.ts b/apps/cli/src/commands/backups/list/list.handler.ts index adfaf43399..0587fe9d73 100644 --- a/apps/cli/src/commands/backups/list/list.handler.ts +++ b/apps/cli/src/commands/backups/list/list.handler.ts @@ -2,7 +2,7 @@ import type { V1ListAllBackupsOutput } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -93,11 +93,11 @@ export const backupsList = Effect.fn("backups.list")(function* (flags: BackupsLi const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { // Spinner is text-mode only; in JSON/stream-json it would leave a dangling `[task] start:` diff --git a/apps/cli/src/commands/backups/restore/restore.command.ts b/apps/cli/src/commands/backups/restore/restore.command.ts index 5a51d25c68..8d72aa8819 100644 --- a/apps/cli/src/commands/backups/restore/restore.command.ts +++ b/apps/cli/src/commands/backups/restore/restore.command.ts @@ -3,7 +3,7 @@ import { Command, Flag } from "effect/cli"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/backups/restore/restore.handler.ts b/apps/cli/src/commands/backups/restore/restore.handler.ts index de62569194..86fde5e2ab 100644 --- a/apps/cli/src/commands/backups/restore/restore.handler.ts +++ b/apps/cli/src/commands/backups/restore/restore.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -25,11 +25,11 @@ export const backupsRestore = Effect.fn("backups.restore")(function* (flags: Bac const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); const recoveryTimeTargetUnix = Option.getOrElse(flags.timestamp, () => 0); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md index e1e17c0bd8..dedd2d4caa 100644 --- a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md @@ -124,6 +124,21 @@ suppressed; a single structured result is emitted for the whole command: } ``` +## Stack mode + +When the new project directory resolves to stack project-state mode (`SUPABASE_EXPERIMENTAL_STACK` +or its `config.toml`), bootstrap writes `supabase/.link.json` (project ref, name and organization from the create +response, active branch ref, normalized pooler URL when available), under the project lease +`supabase/.supabase.lock.lease` that `link` and `unlink` hold, instead of the `supabase/.temp/` +link files, and the push connection reads the pooler URL from that document. The organization and +project `groupIdentify` calls fire from bootstrap, since the linked-project cache fill is a no-op +in this mode. The mode comes from the new project directory and applies to every post-scaffold +step, including the cache finalizer; an invalid `SUPABASE_EXPERIMENTAL_STACK` fails bootstrap +before any project is created. A lease that stays busy fails bootstrap with `LockResolutionError` +reason `lease-busy` and exit code `1`, with no link written. `supabase/.gitignore` gains the lease +and staged-write entries (`.supabase.lock.lease`, `.supabase.lock.*.tmp`) whenever bootstrap takes +the project lease. See `link`'s `SIDE_EFFECTS.md` for the format. + ## Notes - **Native migration push, sharing `db push`'s own core (CLI-1953).** The push step calls @@ -131,7 +146,7 @@ suppressed; a single structured result is emitted for the whole command: — directly with `{ includeAll: false, includeRoles: true, includeSeed: true, dryRun: false }`. Bootstrap never re-resolves the project ref for push: the earlier `create` step already resolved it, and it's reused as-is (passed as a plain value, never re-derived via - `ProjectRefResolver`, which keys off `CommandSettings.workdir`, stale after this handler's + `RemoteTarget`, which keys off `CommandSettings.workdir`, stale after this handler's own `process.chdir` — see the workdir comments in `bootstrap.handler.ts`). The **connection** itself, however, is resolved via its own `resolveLinkedConn` call — dial-direct/ pooler-fallback logic — not reused from the naive `deriveDbConfig(...)` connection already diff --git a/apps/cli/src/commands/bootstrap/bootstrap.handler.ts b/apps/cli/src/commands/bootstrap/bootstrap.handler.ts index 3b06020987..738d806d68 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.handler.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.handler.ts @@ -1,9 +1,11 @@ -import { Effect, FileSystem, Option, Path, Redacted, Ref, Schedule } from "effect"; +import { DateTime, Effect, Exit, FileSystem, Option, Path, Redacted, Ref, Schedule } from "effect"; import type * as HttpClientError from "effect/http/HttpClientError"; import { CommandPlatformApi } from "../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; import { LinkedProjectCache } from "../../telemetry/linked-project-cache.service.ts"; +import { Analytics } from "../../shared/telemetry/analytics.service.ts"; +import { identifyProjectGroups } from "../../shared/telemetry/identify-project-groups.ts"; import { TelemetryState } from "../../telemetry/telemetry-state.service.ts"; import { DnsResolverFlag, @@ -32,9 +34,17 @@ import { warnDeprecatedOrioleDBVersionEnv, } from "../../command-internal/db-config.toml-read.ts"; import { dbPushCore } from "../../command-internal/db-push-core.ts"; -import { linkServicesCore } from "../../command-internal/link-services-core.ts"; +import { fetchPoolerUrl, linkServicesCore } from "../../command-internal/link-services-core.ts"; +import { readLinkSnapshotSoft } from "../../command-internal/link-snapshot.ts"; +import { + type ProjectStateMode, + ProjectStateModeContext, + projectStateMode, +} from "../../command-internal/project-state-mode.ts"; +import { ensureProjectDirectory, writeProjectLink } from "../../command-internal/project-link.ts"; +import { withProjectLease } from "../../command-internal/stack-lock.ts"; import { projectCreateCore } from "../../command-internal/project-create-core.ts"; -import { tempPaths } from "../../command-internal/temp-paths.ts"; +import { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; import { extractServiceKeys } from "../../command-internal/tenant-keys.ts"; import { parseDotEnv } from "../../command-internal/dotenv.ts"; import { @@ -77,9 +87,11 @@ export const bootstrap = Effect.fn("bootstrap")(function* ( const cliSettings = yield* CommandSettings; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; + const legacyState = yield* LegacyProjectState; const templateService = yield* TemplateService; const api = yield* CommandPlatformApi; const linkedProjectCache = yield* LinkedProjectCache; + const analytics = yield* Analytics; const telemetryState = yield* TelemetryState; const workdirFlag = yield* WorkdirFlag; const dnsResolver = yield* DnsResolverFlag; @@ -96,6 +108,7 @@ export const bootstrap = Effect.fn("bootstrap")(function* ( // Hoisted so the linked-project-cache finalizer writes beside the other `supabase/.temp/` // files instead of `cliSettings.workdir`. const resolvedWorkdir = yield* Ref.make(undefined); + const linkModeRef = yield* Ref.make(undefined); yield* Effect.gen(function* () { // Reads the prefixed `SUPABASE_WORKDIR` only (never plain `WORKDIR`). @@ -189,176 +202,218 @@ export const bootstrap = Effect.fn("bootstrap")(function* ( }).pipe(Effect.withSpan("bootstrap.initProject")); } - yield* ensureLogin({ openBrowser: tty.stdinIsTty }); - - const seededPassword = Option.isSome(flags.password) - ? flags.password.value - : Option.isSome(cliSettings.dbPassword) - ? Redacted.value(cliSettings.dbPassword.value) - : ""; - const created = yield* projectCreateCore({ - name: path.basename(workdir), - orgId: "", - dbPassword: seededPassword, - region: undefined, - size: undefined, - highAvailability: undefined, - releaseChannel: undefined, - postgresEngine: undefined, - templateUrl: starter.url.length > 0 ? starter.url : undefined, - emitStructuredResult: false, + // The new project directory's own config can opt into stack mode, so the link target is + // written and read per that directory rather than the directory bootstrap started in. + const linkMode = yield* projectStateMode({ + cwd: workdir, + env: yield* experimentalFeatureEnv("stack"), }); - const projectRef = created.ref; - yield* Ref.set(createdRef, projectRef.length > 0 ? projectRef : undefined); - yield* Effect.annotateCurrentSpan("project.ref", projectRef); + yield* Ref.set(linkModeRef, linkMode); - // A fresh notifier per retry block keeps this step's "Retry (n/8)" counter independent of - // the health-poll and push retries below. - const apiKeysNotify = yield* bootstrapRetryNotify; - const keys = yield* Effect.gen(function* () { - if (isText) yield* output.raw("Linking project...\n", "stderr"); - return yield* getProjectApiKeys(projectRef); - }).pipe(apiKeysNotify, Effect.retry(retry), Effect.withSpan("bootstrap.fetchApiKeys")); - const { anon } = extractServiceKeys(keys); + yield* Effect.gen(function* () { + yield* ensureLogin({ openBrowser: tty.stdinIsTty }); - // Config load must run before link/health/`.env` steps: a malformed config.toml aborts here - // rather than after side effects start. - const { pushYes, toml } = yield* Effect.gen(function* () { - const projectEnv = yield* loadProjectEnv(fs, path, workdir); - return { - pushYes: yield* resolveYesWithProjectEnv(projectEnv), - toml: yield* checkDbToml(fs, path, workdir, projectRef), - }; - }).pipe(Effect.withSpan("bootstrap.loadConfig")); - if (toml.appliedRemote !== undefined) { - yield* output.raw(`Loading config override: [remotes.${toml.appliedRemote}]\n`, "stderr"); - } - yield* warnDeprecatedOrioleDBVersionEnv(toml.envLookup); + const seededPassword = Option.isSome(flags.password) + ? flags.password.value + : Option.isSome(cliSettings.dbPassword) + ? Redacted.value(cliSettings.dbPassword.value) + : ""; + const created = yield* projectCreateCore({ + name: path.basename(workdir), + orgId: "", + dbPassword: seededPassword, + region: undefined, + size: undefined, + highAvailability: undefined, + releaseChannel: undefined, + postgresEngine: undefined, + templateUrl: starter.url.length > 0 ? starter.url : undefined, + emitStructuredResult: false, + }); + const projectRef = created.ref; + yield* Ref.set(createdRef, projectRef.length > 0 ? projectRef : undefined); + yield* Effect.annotateCurrentSpan("project.ref", projectRef); - yield* linkServicesCore({ - ref: projectRef, - serviceKey: anon, - skipPooler: false, - workdir, - }); - const paths = tempPaths(path, workdir); - yield* fs.makeDirectory(path.dirname(paths.projectRef), { recursive: true }); - yield* fs.writeFileString(paths.projectRef, projectRef); + // A fresh notifier per retry block keeps this step's "Retry (n/8)" counter independent of + // the health-poll and push retries below. + const apiKeysNotify = yield* bootstrapRetryNotify; + const keys = yield* Effect.gen(function* () { + if (isText) yield* output.raw("Linking project...\n", "stderr"); + return yield* getProjectApiKeys(projectRef); + }).pipe(apiKeysNotify, Effect.retry(retry), Effect.withSpan("bootstrap.fetchApiKeys")); + const { anon } = extractServiceKeys(keys); - const healthNotify = yield* bootstrapRetryNotify; - yield* Effect.gen(function* () { - if (isText) yield* output.raw("Checking project health...\n", "stderr"); - const services = yield* api.v1.getServicesHealth({ ref: projectRef, services: ["db"] }).pipe( - Effect.catchTags({ - HttpClientError: mapHealthHttpError, - SchemaError: (cause) => - Effect.fail( - new BootstrapHealthError({ message: `Error status 0: ${cause}`, decode: true }), - ), - HttpBodyError: healthTransportError, - SupabaseApiInputError: healthTransportError, - }), - ); - for (const service of services) { - if (!service.healthy) { - return yield* new BootstrapHealthError({ - message: `Service not healthy: ${service.name} (${service.status})`, - }); - } + // Config load must run before link/health/`.env` steps: a malformed config.toml aborts here + // rather than after side effects start. + const { pushYes, toml } = yield* Effect.gen(function* () { + const projectEnv = yield* loadProjectEnv(fs, path, workdir); + return { + pushYes: yield* resolveYesWithProjectEnv(projectEnv), + toml: yield* checkDbToml(fs, path, workdir, projectRef), + }; + }).pipe(Effect.withSpan("bootstrap.loadConfig")); + if (toml.appliedRemote !== undefined) { + yield* output.raw(`Loading config override: [remotes.${toml.appliedRemote}]\n`, "stderr"); } - }).pipe( - Effect.withTracerEnabled(false), - healthNotify, - Effect.retry(retry), - Effect.withSpan("bootstrap.waitHealthy"), - ); + yield* warnDeprecatedOrioleDBVersionEnv(toml.envLookup); - // Uses a naive direct-host db config with no IPv6/pooler fallback, since `.env` is written - // for reference only and, unlike the push connection below, is never used to actually connect. - const dbConfig = deriveDbConfig(projectRef, created.dbPassword, cliSettings.projectHost); - const supabaseUrl = `https://${projectRef}.${cliSettings.projectHost}`; - const envFilePath = path.join(workdir, ".env"); - const envFileWritten = yield* Effect.gen(function* () { - const examplePath = path.join(workdir, ".env.example"); - const hasExample = yield* fs.exists(examplePath); - let example: Record | undefined; - if (hasExample) { - const content = yield* fs.readFileString(examplePath); - example = yield* Effect.try({ - try: () => parseDotEnv(content), - catch: (cause) => - new BootstrapDotEnvParseError({ - message: cause instanceof Error ? cause.message : String(cause), - }), + if (linkMode === "stack") { + const poolerUrl = yield* fetchPoolerUrl(api, projectRef).pipe( + Effect.option, + Effect.map(Option.flatten), + ); + yield* ensureProjectDirectory(workdir); + yield* withProjectLease( + workdir, + writeProjectLink(workdir, { + project: { + ref: projectRef, + name: created.name, + organization_id: created.organizationId, + organization_slug: created.organizationSlug, + }, + active_branch: { + ref: projectRef, + ...(Option.isSome(poolerUrl) ? { pooler_url: poolerUrl.value } : {}), + }, + fetchedAt: DateTime.formatIso(yield* DateTime.now), + }), + ); + yield* identifyProjectGroups(analytics, { + ref: projectRef, + name: created.name, + organization_id: created.organizationId, + organization_slug: created.organizationSlug, + }); + } else { + yield* linkServicesCore({ + ref: projectRef, + serviceKey: anon, + skipPooler: false, + workdir, }); + yield* legacyState.write(workdir, "projectRef", projectRef); } - const env = buildDotEnv(keys, dbConfig, supabaseUrl, example); - yield* fs.writeFileString(envFilePath, marshalDotEnv(env)); - }).pipe( - Effect.as(true), - Effect.catch((cause) => - output - .raw(`Failed to create .env file: ${cause.message}\n`, "stderr") - .pipe(Effect.as(false)), - ), - Effect.withSpan("bootstrap.writeDotEnv"), - ); - // `resolveLinkedConn` doesn't attach a suggestionContext like the full resolver does; build - // one manually so a connect failure inside the push below still gets the established hint. - const suggestionContext: ConnectSuggestionContext = { - dashboardUrl: cliSettings.dashboardUrl, - profileName: cliSettings.profile, - }; - // Falls back to the naive `dbConfig` on `DbConfigIpv6Error` instead of aborting, since a - // freshly created project's pooler metadata may still be propagating. - const resolvedConn = yield* resolveLinkedConn( - projectRef, - workdir, - cliSettings.projectHost, - cliSettings.poolerHost, - dnsResolver, - Option.some(created.dbPassword), - ).pipe( - Effect.catchTag("DbConfigIpv6Error", (error) => - output.raw(`${error.message}\n`, "stderr").pipe(Effect.as(dbConfig)), - ), - ); - const conn = { ...resolvedConn, suggestionContext }; - // Passes workdir/projectRef/toml through directly rather than calling the full `dbPush` - // command, since its CommandSettings-based resolvers would be stale after this handler's - // own chdir above. - const pushNotify = yield* bootstrapRetryNotify; - yield* dbPushCore({ - workdir, - projectRef, - conn, - isLocal: false, - repairSuggestsLocalFlag: false, - dryRun: false, - includeAll: false, - includeRoles: true, - includeSeed: true, - includeVault: true, - dnsResolver, - toml, - yes: pushYes, - emitStructuredResult: false, - }).pipe(pushNotify, Effect.retry(retry), Effect.withSpan("bootstrap.pushMigrations")); + const healthNotify = yield* bootstrapRetryNotify; + yield* Effect.gen(function* () { + if (isText) yield* output.raw("Checking project health...\n", "stderr"); + const services = yield* api.v1 + .getServicesHealth({ ref: projectRef, services: ["db"] }) + .pipe( + Effect.catchTags({ + HttpClientError: mapHealthHttpError, + SchemaError: (cause) => + Effect.fail( + new BootstrapHealthError({ message: `Error status 0: ${cause}`, decode: true }), + ), + HttpBodyError: healthTransportError, + SupabaseApiInputError: healthTransportError, + }), + ); + for (const service of services) { + if (!service.healthy) { + return yield* new BootstrapHealthError({ + message: `Service not healthy: ${service.name} (${service.status})`, + }); + } + } + }).pipe( + Effect.withTracerEnabled(false), + healthNotify, + Effect.retry(retry), + Effect.withSpan("bootstrap.waitHealthy"), + ); - if (isText) { - const suggestion = suggestAppStart(path, runtimeInfo.cwd, workdir, starter.start, aqua); - yield* emitSuccessTrailer(`${suggestion}\n`); - } else { - yield* output.success("", { + // Uses a naive direct-host db config with no IPv6/pooler fallback, since `.env` is written + // for reference only and, unlike the push connection below, is never used to actually connect. + const dbConfig = deriveDbConfig(projectRef, created.dbPassword, cliSettings.projectHost); + const supabaseUrl = `https://${projectRef}.${cliSettings.projectHost}`; + const envFilePath = path.join(workdir, ".env"); + const envFileWritten = yield* Effect.gen(function* () { + const examplePath = path.join(workdir, ".env.example"); + const hasExample = yield* fs.exists(examplePath); + let example: Record | undefined; + if (hasExample) { + const content = yield* fs.readFileString(examplePath); + example = yield* Effect.try({ + try: () => parseDotEnv(content), + catch: (cause) => + new BootstrapDotEnvParseError({ + message: cause instanceof Error ? cause.message : String(cause), + }), + }); + } + const env = buildDotEnv(keys, dbConfig, supabaseUrl, example); + yield* fs.writeFileString(envFilePath, marshalDotEnv(env)); + }).pipe( + Effect.as(true), + Effect.catch((cause) => + output + .raw(`Failed to create .env file: ${cause.message}\n`, "stderr") + .pipe(Effect.as(false)), + ), + Effect.withSpan("bootstrap.writeDotEnv"), + ); + + // `resolveLinkedConn` doesn't attach a suggestionContext like the full resolver does; build + // one manually so a connect failure inside the push below still gets the established hint. + const suggestionContext: ConnectSuggestionContext = { + dashboardUrl: cliSettings.dashboardUrl, + profileName: cliSettings.profile, + }; + // Falls back to the naive `dbConfig` on `DbConfigIpv6Error` instead of aborting, since a + // freshly created project's pooler metadata may still be propagating. + const linkSnapshot = yield* readLinkSnapshotSoft(fs, path, workdir); + const resolvedConn = yield* resolveLinkedConn( + projectRef, workdir, - project_ref: projectRef, - template: starter.name, - start_command: starter.start, - env_file: envFileWritten ? envFilePath : null, - }); - } - yield* setSuccessWorkingDirectory(workdir); + cliSettings.projectHost, + cliSettings.poolerHost, + dnsResolver, + Option.some(created.dbPassword), + { poolerUrl: linkSnapshot.poolerUrl }, + ).pipe( + Effect.catchTag("DbConfigIpv6Error", (error) => + output.raw(`${error.message}\n`, "stderr").pipe(Effect.as(dbConfig)), + ), + ); + const conn = { ...resolvedConn, suggestionContext }; + // Passes workdir/projectRef/toml through directly rather than calling the full `dbPush` + // command, since its CommandSettings-based resolvers would be stale after this handler's + // own chdir above. + const pushNotify = yield* bootstrapRetryNotify; + yield* dbPushCore({ + workdir, + projectRef, + conn, + isLocal: false, + repairSuggestsLocalFlag: false, + dryRun: false, + includeAll: false, + includeRoles: true, + includeSeed: true, + includeVault: true, + dnsResolver, + toml, + yes: pushYes, + emitStructuredResult: false, + }).pipe(pushNotify, Effect.retry(retry), Effect.withSpan("bootstrap.pushMigrations")); + + if (isText) { + const suggestion = suggestAppStart(path, runtimeInfo.cwd, workdir, starter.start, aqua); + yield* emitSuccessTrailer(`${suggestion}\n`); + } else { + yield* output.success("", { + workdir, + project_ref: projectRef, + template: starter.name, + start_command: starter.start, + env_file: envFileWritten ? envFilePath : null, + }); + } + yield* setSuccessWorkingDirectory(workdir); + }).pipe(Effect.provideService(ProjectStateModeContext, { mode: Exit.succeed(linkMode) })); }).pipe( Effect.ensuring( Effect.sync(() => { @@ -371,9 +426,13 @@ export const bootstrap = Effect.fn("bootstrap")(function* ( ), Effect.ensuring( Effect.flatMap( - Effect.all([Ref.get(createdRef), Ref.get(resolvedWorkdir)]), - ([ref, workdir]) => - ref === undefined ? Effect.void : linkedProjectCache.cache(ref, workdir), + Effect.all([Ref.get(createdRef), Ref.get(resolvedWorkdir), Ref.get(linkModeRef)]), + ([ref, workdir, mode]) => + ref === undefined || mode === undefined + ? Effect.void + : linkedProjectCache + .cache(ref, workdir) + .pipe(Effect.provideService(ProjectStateModeContext, { mode: Exit.succeed(mode) })), ), ), Effect.ensuring(telemetryState.flush), diff --git a/apps/cli/src/commands/bootstrap/bootstrap.integration.test.ts b/apps/cli/src/commands/bootstrap/bootstrap.integration.test.ts index c53920506f..3694703aa9 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.integration.test.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.integration.test.ts @@ -1,5 +1,8 @@ +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; +import { holdLock } from "@supabase/stack/effect"; +import { TestClock } from "effect/testing"; import { Cause, ConfigProvider, @@ -44,6 +47,12 @@ import { OutputFlag, } from "../../command-internal/global-flags.ts"; import { CliArgs } from "../../shared/cli/cli-args.service.ts"; +import { readProjectLink } from "../../command-internal/project-link.ts"; +import { + type ProjectStateMode, + softProjectStateMode, +} from "../../command-internal/project-state-mode.ts"; +import { LinkedProjectCache } from "../../telemetry/linked-project-cache.service.ts"; import { DbConnectError } from "../../command-internal/db-connection.errors.ts"; import { DbConnection, type PgConnInput } from "../../command-internal/db-connection.service.ts"; import { debugLoggerLayer } from "../../command-internal/debug-logger.layer.ts"; @@ -223,6 +232,7 @@ function setup(path: Path.Path, opts: SetupOpts = {}) { const layer = Layer.mergeAll( BunServices.layer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), out.layer, api.layer, api.factoryLayer, @@ -320,6 +330,95 @@ describe("bootstrap integration", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.live( + "links through supabase/.link.json instead of .temp when SUPABASE_EXPERIMENTAL_STACK=1", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const s = setup(path, { env: { SUPABASE_EXPERIMENTAL_STACK: "1" } }); + const cacheModes: Array = []; + const modeRecordingCache = Layer.succeed(LinkedProjectCache, { + cache: () => softProjectStateMode.pipe(Effect.map((mode) => void cacheModes.push(mode))), + }); + yield* bootstrap(flags({ template: Option.some("scratch") }), FAST_BACKOFF).pipe( + Effect.provide(Layer.merge(s.layer, modeRecordingCache)), + ); + expect(cacheModes).toEqual(["stack"]); + const link = yield* readProjectLink(s.workdir); + expect(Option.getOrThrow(link)).toMatchObject({ + project: { + ref: VALID_REF, + name: CREATED.name, + organization_id: CREATED.organization_id, + organization_slug: CREATED.organization_slug, + }, + active_branch: { + ref: VALID_REF, + pooler_url: expect.stringContaining("pooler.supabase.com"), + }, + }); + expect(yield* fs.exists(path.join(s.workdir, "supabase", ".temp"))).toBe(false); + expect(s.analytics.groupIdentified).toEqual([ + { + groupType: "organization", + groupKey: CREATED.organization_id, + properties: { organization_slug: CREATED.organization_slug }, + }, + { + groupType: "project", + groupKey: VALID_REF, + properties: { name: CREATED.name, organization_slug: CREATED.organization_slug }, + }, + ]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("fails on a busy project lease without publishing supabase/.link.json", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const s = setup(path, { env: { SUPABASE_EXPERIMENTAL_STACK: "1" } }); + yield* fs.makeDirectory(path.join(s.workdir, "supabase"), { recursive: true }); + yield* holdLock( + path.join(s.workdir, "supabase", ".supabase.lock.lease"), + "1 hour", + "supabase.lock", + ); + + // The lease wait runs on the test clock; the lock stays held, so the outcome is fixed. + const error = yield* bootstrap( + flags({ template: Option.some("scratch") }), + FAST_BACKOFF, + ).pipe( + Effect.provide(s.layer), + Effect.flip, + Effect.raceFirst( + TestClock.adjust("61 seconds").pipe(Effect.andThen(Effect.yieldNow), Effect.forever), + ), + ); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "lease-busy" }); + expect(Option.isNone(yield* readProjectLink(s.workdir))).toBe(true); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("a downloaded template fails closed on an invalid SUPABASE_EXPERIMENTAL_STACK", () => + Effect.gen(function* () { + const path = yield* Path.Path; + const s = setup(path, { + env: { SUPABASE_EXPERIMENTAL_STACK: "yes" }, + samples: [NEXTJS_TEMPLATE], + }); + const error = yield* bootstrap(flags({ template: Option.some("nextjs") }), FAST_BACKOFF).pipe( + Effect.provide(s.layer), + Effect.flip, + ); + expect(error._tag).toBe("ExperimentalFeatureFlagError"); + expect(s.createBodies).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect( "scratch scaffolding fails closed on an invalid SUPABASE_EXPERIMENTAL_STACK before writing config", () => @@ -359,20 +458,6 @@ describe("bootstrap integration", () => { }).pipe(Effect.provide(BunServices.layer)), ); - it.live("ignores SUPABASE_EXPERIMENTAL_STACK on a downloaded template", () => - Effect.gen(function* () { - const path = yield* Path.Path; - const s = setup(path, { - samples: [NEXTJS_TEMPLATE], - env: { SUPABASE_EXPERIMENTAL_STACK: "yes" }, - }); - yield* bootstrap(flags({ template: Option.some("NextJS") }), FAST_BACKOFF).pipe( - Effect.provide(s.layer), - ); - expect(s.downloads).toHaveLength(1); - }).pipe(Effect.provide(BunServices.layer)), - ); - it.effect("rejects an unknown template argument", () => Effect.gen(function* () { const path = yield* Path.Path; diff --git a/apps/cli/src/commands/bootstrap/bootstrap.layers.ts b/apps/cli/src/commands/bootstrap/bootstrap.layers.ts index c1c35c81f6..12b3efe761 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.layers.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.layers.ts @@ -5,7 +5,7 @@ import { httpClientLayer } from "../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryFromApiLayer } from "../../auth/command-platform-api-factory.layer.ts"; import { commandPlatformApiLayer } from "../../auth/command-platform-api.layer.ts"; import { commandSettingsLayer } from "../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../config/remote-target.layer.ts"; import { dbConnectionLayer } from "../../command-internal/db-connection.sql-pg.layer.ts"; import { debugLoggerLayer } from "../../command-internal/debug-logger.layer.ts"; import { identityStitchLayer } from "../../command-internal/identity-stitch.ts"; @@ -45,7 +45,7 @@ export const bootstrapRuntimeLayer = Layer.mergeAll( httpClient, credentials, cliSettings, - projectRefLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), + remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), linkedProjectCacheLayer.pipe( Layer.provide(credentials), Layer.provide(cliSettings), diff --git a/apps/cli/src/commands/bootstrap/bootstrap.layers.unit.test.ts b/apps/cli/src/commands/bootstrap/bootstrap.layers.unit.test.ts index 703485d137..5a5518c27c 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.layers.unit.test.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.layers.unit.test.ts @@ -37,11 +37,11 @@ import { } from "../../command-internal/global-flags.ts"; import { CommandPlatformApiFactory } from "../../auth/command-platform-api-factory.service.ts"; -import { ProjectRefResolver } from "../../config/project-ref.service.ts"; import { IdentityStitch } from "../../command-internal/identity-stitch.ts"; import { TemplateService } from "./bootstrap.templates.ts"; import { bootstrapRuntimeLayer } from "./bootstrap.layers.ts"; +import { stubRemoteTarget } from "../../../tests/helpers/remote-target.ts"; const tempRoot = useTempWorkdir("supabase-bootstrap-layers-"); @@ -62,19 +62,19 @@ function ambientStubs() { ); // These stubs exist only so the Effect type system sees CommandPlatformApi, - // CommandPlatformApiFactory, and ProjectRefResolver as satisfiable in the outer context; the + // CommandPlatformApiFactory, and RemoteTarget as satisfiable in the outer context; the // runtime layer's own provisions take precedence at runtime. const heavyServiceStubs = Layer.mergeAll( mockCommandPlatformApiService().layer, Layer.succeed(CommandPlatformApiFactory, { make: Effect.die("platform-api-factory not needed for layer-exposure test"), }), - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveForLink: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveOptional: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - loadProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - promptProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + load: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + prompt: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), }), Layer.succeed(TemplateService, { listSamples: Effect.die("template-service not needed for layer-exposure test"), diff --git a/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts b/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts index 14ebb3a93c..1d47fcf599 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts @@ -1,3 +1,4 @@ +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer, Option, Path, Schedule } from "effect"; @@ -219,6 +220,7 @@ describe("bootstrap linked-project cache location", () => { const layer = Layer.mergeAll( BunServices.layer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), out.layer, api.layer, api.factoryLayer, diff --git a/apps/cli/src/commands/branches/create/create.command.ts b/apps/cli/src/commands/branches/create/create.command.ts index 4af369ac66..51d4cfa7bb 100644 --- a/apps/cli/src/commands/branches/create/create.command.ts +++ b/apps/cli/src/commands/branches/create/create.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { Layer } from "effect"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/branches/create/create.handler.ts b/apps/cli/src/commands/branches/create/create.handler.ts index 6715693bd0..d268599387 100644 --- a/apps/cli/src/commands/branches/create/create.handler.ts +++ b/apps/cli/src/commands/branches/create/create.handler.ts @@ -13,7 +13,7 @@ import { aqua } from "../../../command-internal/colors.ts"; import { encodeEnv, encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; import { encodeGoToml, encodeGoYaml } from "../../../command-internal/go-struct-output.encoders.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { gateMapError } from "../../../command-internal/upgrade-suggest.ts"; import { GO_BRANCH_RESPONSE } from "../branches.go-payload.ts"; import { @@ -80,7 +80,8 @@ export const branchesCreate = Effect.fn("branches.create")(function* (flags: Bra // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, and the platform 403s on that ref for every branches-management // endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/delete/delete.command.ts b/apps/cli/src/commands/branches/delete/delete.command.ts index 5e1e075de0..473c6d4382 100644 --- a/apps/cli/src/commands/branches/delete/delete.command.ts +++ b/apps/cli/src/commands/branches/delete/delete.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { branchPrompt } from "../branches.prompt.ts"; import { branchRef } from "../../../command-internal/branch-ref.resolver.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; diff --git a/apps/cli/src/commands/branches/delete/delete.handler.ts b/apps/cli/src/commands/branches/delete/delete.handler.ts index d6eefd83d7..949248ccf2 100644 --- a/apps/cli/src/commands/branches/delete/delete.handler.ts +++ b/apps/cli/src/commands/branches/delete/delete.handler.ts @@ -5,7 +5,7 @@ import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.serv import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { BranchesDeleteNetworkError, BranchesDeleteUnexpectedStatusError, @@ -30,7 +30,8 @@ export const branchesDelete = Effect.fn("branches.delete")(function* (flags: Bra // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, and the platform 403s on that ref for every branches-management // endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/disable/disable.command.ts b/apps/cli/src/commands/branches/disable/disable.command.ts index 421b834820..669ca53b3d 100644 --- a/apps/cli/src/commands/branches/disable/disable.command.ts +++ b/apps/cli/src/commands/branches/disable/disable.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/branches/disable/disable.handler.ts b/apps/cli/src/commands/branches/disable/disable.handler.ts index 0ed631772d..b0d35a1664 100644 --- a/apps/cli/src/commands/branches/disable/disable.handler.ts +++ b/apps/cli/src/commands/branches/disable/disable.handler.ts @@ -5,7 +5,7 @@ import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.serv import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { BranchesDisableNetworkError, BranchesDisableUnexpectedStatusError, @@ -30,7 +30,8 @@ export const branchesDisable = Effect.fn("branches.disable")(function* ( // `branches` is PARENT-scoped: after `supabase link `, // `supabase/.temp/project-ref` holds the branch's own ref, and the platform // 403s on that ref for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/get/get.command.ts b/apps/cli/src/commands/branches/get/get.command.ts index ea78c0721f..578a4c658d 100644 --- a/apps/cli/src/commands/branches/get/get.command.ts +++ b/apps/cli/src/commands/branches/get/get.command.ts @@ -6,7 +6,7 @@ import { compose, type EntryCondition, } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { branchPrompt } from "../branches.prompt.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/branches/get/get.handler.ts b/apps/cli/src/commands/branches/get/get.handler.ts index c0f40c3f7f..09c8ef515f 100644 --- a/apps/cli/src/commands/branches/get/get.handler.ts +++ b/apps/cli/src/commands/branches/get/get.handler.ts @@ -20,7 +20,7 @@ import { encodeYaml, } from "../../../command-internal/go-output.encoders.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { BranchesApiKeysNetworkError, BranchesApiKeysUnexpectedStatusError, @@ -83,7 +83,8 @@ export const branchesGet = Effect.fn("branches.get")(function* (flags: BranchesG // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, which the platform 403s on for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/list/SIDE_EFFECTS.md b/apps/cli/src/commands/branches/list/SIDE_EFFECTS.md index a5827db494..bfcbb8e95c 100644 --- a/apps/cli/src/commands/branches/list/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/branches/list/SIDE_EFFECTS.md @@ -13,8 +13,8 @@ > `branches` is PARENT-scoped: after `supabase link `, the on-disk `project-ref` file > holds the BRANCH's own ref, and the Management API returns 403 for a branch ref on every > branches-management endpoint. Every `branches` subcommand therefore resolves the project ref -> via `resolveParentScopedProjectRef` (`command-internal/parent-project-ref.ts`) instead -> of calling `ProjectRefResolver.resolve` directly: an explicit `--project-ref` still wins +> via `RemoteTarget.resolveParentScoped` instead +> of calling `RemoteTarget.resolve` directly: an explicit `--project-ref` still wins > outright; otherwise the PARENT is resolved as env `SUPABASE_PROJECT_ID` → `linked-project.json`'s > `ref` → the `project-ref` file, first ref-shaped candidate wins, falling through to > `resolver.resolve(None)`'s ordinary env/prompt/not-linked behavior when no candidate is @@ -22,7 +22,7 @@ > `project-ref` file then hold the same ref (CLI-2167 follow-up, TS-only divergence). > Pretty-table rendering ALSO makes its own independent, soft read of `/supabase/.temp/project-ref` -> (`ProjectRefResolver.resolveOptional`: env → file, never a prompt, never a failure) to find +> (`RemoteTarget.resolveOptional`: env → file, never a prompt, never a failure) to find > the CURRENTLY linked ref (which may be a branch ref) for the `(active)` marker below — separate > from, and unconditional on, the PARENT-scoped `ref` resolution above. Missing/unreadable → no > marker, never an error (CLI-2167 follow-up, TS-only). diff --git a/apps/cli/src/commands/branches/list/list.command.ts b/apps/cli/src/commands/branches/list/list.command.ts index e87f189c78..f2abb4fbf8 100644 --- a/apps/cli/src/commands/branches/list/list.command.ts +++ b/apps/cli/src/commands/branches/list/list.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/branches/list/list.handler.ts b/apps/cli/src/commands/branches/list/list.handler.ts index cb6f5f5b89..e7fdb4749c 100644 --- a/apps/cli/src/commands/branches/list/list.handler.ts +++ b/apps/cli/src/commands/branches/list/list.handler.ts @@ -2,7 +2,7 @@ import type { V1ListAllBranchesOutput } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -10,7 +10,6 @@ import { Output } from "../../../shared/output/output.service.ts"; import { encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; import { encodeGoToml, encodeGoYaml } from "../../../command-internal/go-struct-output.encoders.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; import { GO_BRANCHES_LIST, GO_BRANCHES_TOML_WRAPPER } from "../branches.go-payload.ts"; import { BranchesEnvNotSupportedError, @@ -33,13 +32,13 @@ export const branchesList = Effect.fn("branches.list")(function* (flags: Branche const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, which the platform 403s on for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { @@ -86,7 +85,12 @@ export const branchesList = Effect.fn("branches.list")(function* (flags: Branche // Marks the linked branch as `(active)`; resolveOptional never prompts or fails // when nothing is linked. - const activeRef = yield* resolver.resolveOptional(Option.none()); - yield* output.raw(renderBranchesListTable(branches, Option.getOrUndefined(activeRef))); + const activeRef = yield* remoteTarget.resolveOptional(Option.none()); + yield* output.raw( + renderBranchesListTable( + branches, + Option.getOrUndefined(Option.map(activeRef, (target) => target.ref)), + ), + ); }).pipe(Effect.ensuring(linkedProjectCache.cache(ref)), Effect.ensuring(telemetryState.flush)); }); diff --git a/apps/cli/src/commands/branches/pause/pause.command.ts b/apps/cli/src/commands/branches/pause/pause.command.ts index dc9645edbd..2ae5a625dc 100644 --- a/apps/cli/src/commands/branches/pause/pause.command.ts +++ b/apps/cli/src/commands/branches/pause/pause.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { branchPrompt } from "../branches.prompt.ts"; import { branchRef } from "../../../command-internal/branch-ref.resolver.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; diff --git a/apps/cli/src/commands/branches/pause/pause.handler.ts b/apps/cli/src/commands/branches/pause/pause.handler.ts index 9d41d1059a..32b030f787 100644 --- a/apps/cli/src/commands/branches/pause/pause.handler.ts +++ b/apps/cli/src/commands/branches/pause/pause.handler.ts @@ -5,7 +5,7 @@ import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.serv import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { BranchesPauseNetworkError, BranchesPauseUnexpectedStatusError, @@ -29,7 +29,8 @@ export const branchesPause = Effect.fn("branches.pause")(function* (flags: Branc // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, which the platform 403s on for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/unpause/unpause.command.ts b/apps/cli/src/commands/branches/unpause/unpause.command.ts index fec9be4e19..a1d5f1419c 100644 --- a/apps/cli/src/commands/branches/unpause/unpause.command.ts +++ b/apps/cli/src/commands/branches/unpause/unpause.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { branchPrompt } from "../branches.prompt.ts"; import { branchRef } from "../../../command-internal/branch-ref.resolver.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; diff --git a/apps/cli/src/commands/branches/unpause/unpause.handler.ts b/apps/cli/src/commands/branches/unpause/unpause.handler.ts index 40fda69422..1ad22b46af 100644 --- a/apps/cli/src/commands/branches/unpause/unpause.handler.ts +++ b/apps/cli/src/commands/branches/unpause/unpause.handler.ts @@ -5,7 +5,7 @@ import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.serv import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { BranchesUnpauseNetworkError, BranchesUnpauseUnexpectedStatusError, @@ -31,7 +31,8 @@ export const branchesUnpause = Effect.fn("branches.unpause")(function* ( // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, which the platform 403s on for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/branches/update/update.command.ts b/apps/cli/src/commands/branches/update/update.command.ts index 2370bd8293..cd4476c402 100644 --- a/apps/cli/src/commands/branches/update/update.command.ts +++ b/apps/cli/src/commands/branches/update/update.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { branchPrompt } from "../branches.prompt.ts"; import { branchRef } from "../../../command-internal/branch-ref.resolver.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; diff --git a/apps/cli/src/commands/branches/update/update.handler.ts b/apps/cli/src/commands/branches/update/update.handler.ts index 2b71d5ef5a..b07c87874b 100644 --- a/apps/cli/src/commands/branches/update/update.handler.ts +++ b/apps/cli/src/commands/branches/update/update.handler.ts @@ -9,7 +9,7 @@ import { Output } from "../../../shared/output/output.service.ts"; import { encodeEnv, encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; import { encodeGoToml, encodeGoYaml } from "../../../command-internal/go-struct-output.encoders.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; -import { resolveParentScopedProjectRef } from "../../../command-internal/parent-project-ref.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { gateMapError } from "../../../command-internal/upgrade-suggest.ts"; import { GO_BRANCH_RESPONSE } from "../branches.go-payload.ts"; import { @@ -39,7 +39,8 @@ export const branchesUpdate = Effect.fn("branches.update")(function* (flags: Bra // `branches` is parent-scoped: after `supabase link `, `supabase/.temp/project-ref` // holds the branch's own ref, which the platform 403s on for every branches-management endpoint. - const ref = yield* resolveParentScopedProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + const { ref } = yield* remoteTarget.resolveParentScoped(flags.projectRef); yield* Effect.annotateCurrentSpan("project.ref", ref); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/config/diff/diff.command.ts b/apps/cli/src/commands/config/diff/diff.command.ts index 9856559bcd..7329ada40b 100644 --- a/apps/cli/src/commands/config/diff/diff.command.ts +++ b/apps/cli/src/commands/config/diff/diff.command.ts @@ -2,7 +2,7 @@ import { Option } from "effect"; import type * as CliCommand from "effect/cli/Command"; import { Command, Flag } from "effect/cli"; -import { PROJECT_REF_PATTERN } from "../../../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../../../config/remote-target.service.ts"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; import { configTarget } from "../../../command-internal/project-target.ts"; diff --git a/apps/cli/src/commands/config/pull/pull.command.ts b/apps/cli/src/commands/config/pull/pull.command.ts index 708f92f7d3..7dc69e586b 100644 --- a/apps/cli/src/commands/config/pull/pull.command.ts +++ b/apps/cli/src/commands/config/pull/pull.command.ts @@ -2,7 +2,7 @@ import { Layer, Option } from "effect"; import type * as CliCommand from "effect/cli/Command"; import { Command, Flag } from "effect/cli"; -import { PROJECT_REF_PATTERN } from "../../../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../../../config/remote-target.service.ts"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; import { configPullRun } from "../../../command-internal/config-pull-run.ts"; diff --git a/apps/cli/src/commands/config/push/push.branch-target.ts b/apps/cli/src/commands/config/push/push.branch-target.ts index 3365b0f1c9..103c8fa2bc 100644 --- a/apps/cli/src/commands/config/push/push.branch-target.ts +++ b/apps/cli/src/commands/config/push/push.branch-target.ts @@ -4,12 +4,8 @@ import * as HttpClientError from "effect/http/HttpClientError"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; import { BRANCH_LOOKUP_TIMEOUT, findBranchName } from "../../../command-internal/branch-target.ts"; -import { - type CachedLinkedProject, - parseCachedLinkedProject, -} from "../../../command-internal/parent-project-ref.ts"; import { BRANCH_PROJECT_REF_PATTERN } from "../../../command-internal/ref-patterns.ts"; -import { readProjectRefFile, tempPaths } from "../../../command-internal/temp-paths.ts"; +import { readLinkSnapshotSoft } from "../../../command-internal/link-snapshot.ts"; import { Output } from "../../../shared/output/output.service.ts"; /** @@ -53,7 +49,7 @@ export type ConfigPushKnownBranch = /** `V1GetProjectOutput.name`/a branch's `name` are unconstrained strings — an * empty (or empty-after-sanitization-adjacent) live value must render as "no - * name available", matching `parseCachedLinkedProject`'s own + * name available", matching the link reader's own * empty-filtering convention for cached names. */ function normalizeApiName(name: string | undefined): string | undefined { return name !== undefined && name.length > 0 ? name : undefined; @@ -82,7 +78,7 @@ const ProbeOutcome = Data.taggedEnum(); * A known branch name/UUID skips the live probe. Otherwise `GET /v1/projects/{ref}` (bounded at * {@link BRANCH_LOOKUP_TIMEOUT}) returns a project on 200, `"unknown"` on any other failure * (never blocking the push), or falls into best-effort recovery on a 404: - * `.temp/linked-project.json` and `.temp/project-ref` supply the parent ref and branch name when + * the link's parent record and linked ref supply the parent ref and branch name when * they can be trusted, and a bare `{ kind: "branch", ref }` otherwise. */ export function resolveConfigPushTarget( @@ -97,11 +93,8 @@ export function resolveConfigPushTarget( const cliSettings = yield* CommandSettings; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const linkedProjectCachePath = tempPaths(path, cliSettings.workdir).linkedProjectCache; - const readCachedParent = fs.readFileString(linkedProjectCachePath).pipe( - Effect.map(parseCachedLinkedProject), - Effect.orElseSucceed(() => Option.none()), - ); + const readSnapshot = readLinkSnapshotSoft(fs, path, cliSettings.workdir); + const readCachedParent = readSnapshot.pipe(Effect.map((snapshot) => snapshot.parent)); // A name target: both fields already known, so just enrich the parent's name from cache // when it happens to match. @@ -161,7 +154,7 @@ export function resolveConfigPushTarget( } // Shared recovery for a confirmed branch whose name/parent aren't known yet. Reads - // .temp/project-ref only once there's a cache candidate to correlate it against, so the + // the linked ref only once there's a cache candidate to correlate it against, so the // common cache-absent path skips a second filesystem read. const cached = yield* readCachedParent; const candidateParentRef = @@ -174,9 +167,7 @@ export function resolveConfigPushTarget( return { kind: "branch", ref }; } - const fileRef = yield* readProjectRefFile(fs, path, cliSettings.workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - ); + const fileRef = (yield* readSnapshot).ref; const branchName = normalizeApiName( yield* findBranchName(candidateParentRef, ref, { spinnerLabel: "Checking branch name...", diff --git a/apps/cli/src/commands/config/push/push.command.ts b/apps/cli/src/commands/config/push/push.command.ts index f4b1626fab..57b9410a9c 100644 --- a/apps/cli/src/commands/config/push/push.command.ts +++ b/apps/cli/src/commands/config/push/push.command.ts @@ -1,7 +1,7 @@ import { Layer, Option } from "effect"; import { Command, Flag } from "effect/cli"; -import { PROJECT_REF_PATTERN } from "../../../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../../../config/remote-target.service.ts"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; diff --git a/apps/cli/src/commands/config/push/push.handler.ts b/apps/cli/src/commands/config/push/push.handler.ts index ba1fb5126d..f957c19be2 100644 --- a/apps/cli/src/commands/config/push/push.handler.ts +++ b/apps/cli/src/commands/config/push/push.handler.ts @@ -5,6 +5,7 @@ import { DateTime, Effect, FileSystem, Option, Path, Ref } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { resolveYesWithProjectEnv } from "../../../command-internal/global-flags.ts"; @@ -18,7 +19,6 @@ import { loadProjectEnv, resolveShellEnv, } from "../../../command-internal/db-config.toml-read.ts"; -import { resolveLinkedParentRef } from "../../../command-internal/parent-project-ref.ts"; import { BRANCH_UUID_PATTERN } from "../../../command-internal/ref-patterns.ts"; import { sanitizeInlineName, @@ -158,6 +158,7 @@ const configTargetErrors = configTargetErrorsFor({ export const configPush = Effect.fn("config.push")(function* (flags: ConfigPushFlags) { const output = yield* Output; + const remoteTarget = yield* RemoteTarget; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; const linkedProjectCache = yield* LinkedProjectCache; @@ -290,7 +291,7 @@ export const configPush = Effect.fn("config.push")(function* (flags: ConfigPushF if (BRANCH_UUID_PATTERN.test(branch)) { knownBranch = { kind: "uuid" }; } else { - const parent = yield* resolveLinkedParentRef(); + const parent = yield* remoteTarget.parent; knownBranch = parent.kind === "resolved" ? { kind: "name", branchName: branch, parentRef: parent.ref } diff --git a/apps/cli/src/commands/config/push/push.integration.test.ts b/apps/cli/src/commands/config/push/push.integration.test.ts index 2617eee123..ed38a5e943 100644 --- a/apps/cli/src/commands/config/push/push.integration.test.ts +++ b/apps/cli/src/commands/config/push/push.integration.test.ts @@ -3379,7 +3379,7 @@ describe("config push branch/project target detection (CLI-2168)", () => { it.effect( "a broken .temp/project-ref (a directory, not a file) degrades gracefully instead of failing", () => { - // Mirrors the established `readProjectRefFile` EISDIR regression + // Mirrors the established `readLegacyProjectRef` EISDIR regression // technique — the target-detection recovery's own best-effort read // must swallow a real read failure (not just a missing file), not // propagate it. A cache candidate (with a branch-list response that diff --git a/apps/cli/src/commands/db/advisors/advisors.handler.ts b/apps/cli/src/commands/db/advisors/advisors.handler.ts index fcf226533c..8bf8ddde8e 100644 --- a/apps/cli/src/commands/db/advisors/advisors.handler.ts +++ b/apps/cli/src/commands/db/advisors/advisors.handler.ts @@ -5,7 +5,7 @@ import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { ProcessControl } from "../../../shared/runtime/process-control.service.ts"; import { CommandCredentials } from "../../../auth/command-credentials.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua } from "../../../command-internal/colors.ts"; import { missingAccessTokenMessage } from "../../../auth/access-token.ts"; import { failsOn } from "../../../command-internal/fail-on.ts"; @@ -116,13 +116,14 @@ const runLinked = Effect.fn("db.advisors.runLinked")(function* ( ) { const resolver = yield* DbConfigResolver; const credentials = yield* CommandCredentials; - const projectRefResolver = yield* ProjectRefResolver; + const projectRefResolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; // The linked-project cache is written whenever the project ref was resolved, even when the // DB-config resolve below fails. Load the ref first (non-prompting, honoring an explicit // `--project-ref`) and wrap everything after it in the cache finalizer. - const ref = yield* projectRefResolver.loadProjectRef(flags.projectRef); + const linkedTarget = yield* projectRefResolver.load(flags.projectRef); + const { ref } = linkedTarget; return yield* Effect.gen(function* () { // The host probe / login-role mint / pooler / IPv6 fallback. The linked lint-gathering path @@ -133,6 +134,7 @@ const runLinked = Effect.fn("db.advisors.runLinked")(function* ( connType: "linked", dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); // The access token is validated against the `sbp_` pattern before calling the API: a diff --git a/apps/cli/src/commands/db/advisors/advisors.integration.test.ts b/apps/cli/src/commands/db/advisors/advisors.integration.test.ts index ca8a0cc973..66b3d868bd 100644 --- a/apps/cli/src/commands/db/advisors/advisors.integration.test.ts +++ b/apps/cli/src/commands/db/advisors/advisors.integration.test.ts @@ -39,7 +39,6 @@ import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { CommandCredentials } from "../../../auth/command-credentials.service.ts"; import { InvalidAccessTokenError } from "../../../auth/errors.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigIpv6Error } from "../../../command-internal/db-config.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; @@ -58,6 +57,7 @@ import { encodeAdvisorLints, scanAdvisorLintRow } from "./advisors.format.ts"; import { dbAdvisors } from "./advisors.handler.ts"; import { splitLintsSql } from "./advisors.lints-sql.ts"; import type { DbAdvisorsFlags } from "./advisors.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LOCAL_CONN: PgConnInput = { host: "127.0.0.1", @@ -157,7 +157,7 @@ function mockConnection(opts: { function mockProjectRef() { const calls: Array = []; - const layer = Layer.succeed(ProjectRefResolver, { + const layer = stubRemoteTarget({ resolve: () => Effect.sync(() => { calls.push("resolve"); @@ -167,12 +167,12 @@ function mockProjectRef() { resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), // Gives an explicit `--project-ref` flag top precedence, mirroring the real resolver so a // test can prove the flag (not just the hardcoded fallback) drives the linked ref. - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.sync(() => { - calls.push("loadProjectRef"); + calls.push("load"); return Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF; }), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); return { layer, @@ -466,7 +466,7 @@ describe("db advisors — local", () => { }); return Effect.gen(function* () { yield* dbAdvisors(flags()); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(cache.cached).toBe(true); }).pipe(Effect.provide(layer)); }); @@ -479,7 +479,7 @@ describe("db advisors — local", () => { }); return Effect.gen(function* () { yield* dbAdvisors(flags()); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(cache.cached).toBe(true); }).pipe(Effect.provide(layer)); }); @@ -737,7 +737,7 @@ describe("db advisors — linked", () => { }); return Effect.gen(function* () { yield* dbAdvisors(flags({ type: Option.some("security") })); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(projectRef.calls).not.toContain("resolve"); }).pipe(Effect.provide(layer)); }); diff --git a/apps/cli/src/commands/db/advisors/advisors.layers.ts b/apps/cli/src/commands/db/advisors/advisors.layers.ts index 99aae70ac5..1ed14713a9 100644 --- a/apps/cli/src/commands/db/advisors/advisors.layers.ts +++ b/apps/cli/src/commands/db/advisors/advisors.layers.ts @@ -5,7 +5,7 @@ import { commandCredentialsLayer } from "../../../auth/command-credentials.layer import { httpClientLayer } from "../../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-api-factory.layer.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; import { debugLoggerLayer } from "../../../command-internal/debug-logger.layer.ts"; @@ -26,7 +26,7 @@ import { telemetryStateLayer } from "../../../telemetry/telemetry-state.layer.ts * * Instead the project-ref resolver is given the lazy `commandPlatformApiFactoryLayer`, whose * `make` is only forced by an interactive project-ref prompt. The linked path resolves the ref - * via the non-prompting `loadProjectRef`, which never forces the factory; the local path never + * via the non-prompting `RemoteTarget.load`, which never forces the factory; the local path never * resolves a project ref at all, so no token is resolved there either. * * `commandSettingsLayer` is provided to each consumer that needs it, since `Layer.provide` does @@ -47,7 +47,7 @@ const platformApiFactory = commandPlatformApiFactoryLayer.pipe( Layer.provide(identityStitchLayer), ); -const projectRef = projectRefLayer.pipe( +const projectRef = remoteTargetLayer.pipe( Layer.provide(platformApiFactory), Layer.provide(cliSettings), ); diff --git a/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md b/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md index 705fe8a15c..1217eaf8a8 100644 --- a/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md @@ -20,7 +20,7 @@ and uses an automatically assigned port; `db.shadow_port` applies only to the le Pg-delta runs in-process. Coverage gaps warn, while `--strict-coverage` makes them fatal, and `PGDELTA_DEBUG` writes diagnostic JSON under -`supabase/.temp/pgdelta/v2/debug//`. The engine may emit transaction-aware +`supabase/.temp/pgdelta/v2/debug//` (`/tmp/pgdelta/v2/debug//` in stack mode). The engine may emit transaction-aware file splits; applicable, convergent SQL is the contract. The bundled formatter defaults to lowercase SQL at width 180; config overrides it, and JSON `null` disables formatting without disabling safe compaction. @@ -48,7 +48,7 @@ it, and JSON `null` disables formatting without disabling safe compaction. | --------------------------------------------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `/supabase/migrations/_.sql` | SQL | non-empty `--file` diff; bundled pg-delta may emit ordered transaction-aware files, while pgAdmin always emits one | | `` (from `--output` / `-o`) | SQL | explicit `--from/--to` mode with `--output`; flattened review representation, not a portable apply script | -| `/supabase/.temp/pgdelta/v2/debug//*.json` | JSON | bundled engine with `PGDELTA_DEBUG` | +| `/supabase/.temp/pgdelta/v2/debug//*.json` | JSON | bundled engine with `PGDELTA_DEBUG`; `/tmp/pgdelta/v2/debug//` in stack mode | | `~/.supabase/cache/shadow-baseline/shadow-baseline-.tar` | tar | cache-enabled COLD shadow provision creates the current key's snapshot (native diff targets + the explicit `--from/--to migrations` shadow; never `--use-pgadmin`/`--use-pg-schema`); a warm hit `touch`es its mtime (LRU); every cache-eligible acquire may delete other keys under LRU keep-3 + 2-day mtime TTL — ~90MB (`SUPABASE_HOME` overrides the root) | | managed snapshot key `` | backend | stack-mode cache-enabled cold shadow provision saves the database baseline; the last completed save replaces the key | | `~/.supabase/cache/shadow-baseline/shadow-baseline-.tar..partial` | tar | during a cold export — the in-flight temp file, `rename`d into the tar above on success and removed on failure; only a crash/SIGKILL leaves it behind, and later cold exports / warm hits sweep leftovers older than 5 minutes | @@ -208,7 +208,7 @@ transaction metadata. mutually-exclusive engine group; `--db-url` / `--linked` / `--local` are a mutually-exclusive target group (default `--local`). - **`--project-ref`** (TS-only, no Go equivalent on any user-facing `db` - command) overrides ONLY the linked-ref resolution `ProjectRefResolver` + command) overrides ONLY the linked-ref resolution `RemoteTarget` performs (flag > `SUPABASE_PROJECT_ID` > `.temp/project-ref`) — unlike `SUPABASE_PROJECT_ID`, it does not affect the shadow container's project id/labels. It never implies `--linked`: passing it with a resolved diff --git a/apps/cli/src/commands/db/diff/diff.handler.ts b/apps/cli/src/commands/db/diff/diff.handler.ts index e6bf7b1078..73e815dd97 100644 --- a/apps/cli/src/commands/db/diff/diff.handler.ts +++ b/apps/cli/src/commands/db/diff/diff.handler.ts @@ -11,7 +11,7 @@ import { branchClause, detectGitBranch } from "../../../shared/git/git-branch.ts import { Output } from "../../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua, yellow } from "../../../command-internal/colors.ts"; import { envOverrideLookup, @@ -250,12 +250,17 @@ export const dbDiff = Effect.fn("db.diff")(function* (flags: DbDiffFlags) { : Option.isSome(flags.linked) ? "linked" : "local"; + const preflightTarget = + preflightConnType === "linked" + ? yield* (yield* RemoteTarget).load(flags.projectRef) + : undefined; const preflight = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType: preflightConnType, dnsResolver, password: Option.none(), linkedProjectRef: flags.projectRef, + target: preflightTarget, }); if (preflightConnType === "linked") { const preflightRef = Option.getOrUndefined(preflight.ref ?? Option.none()); @@ -334,6 +339,7 @@ export const dbDiff = Effect.fn("db.diff")(function* (flags: DbDiffFlags) { dnsResolver, password: Option.none(), linkedProjectRef: flags.projectRef, + target: yield* (yield* RemoteTarget).load(flags.projectRef), }); const ref2 = Option.getOrUndefined(resolved.ref ?? Option.none()); if (ref2 !== undefined) { @@ -502,9 +508,11 @@ export const dbDiff = Effect.fn("db.diff")(function* (flags: DbDiffFlags) { // connection work (TCP probe / temp-role mint). Only the linked path pre-resolves a ref; the // default local/db-url target never merges a remote block. let linkedRef: string | undefined; + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const projectRefResolver = yield* ProjectRefResolver; - linkedRef = yield* projectRefResolver.loadProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + linkedRef = linkedTarget.ref; // Cached the moment the ref is known, not after `cfg`/`localInputs` below (both // fallible) resolve, so the project cache is still written even if a later step fails. yield* Ref.set(linkedRefForCache, Option.some(linkedRef)); @@ -542,6 +550,7 @@ export const dbDiff = Effect.fn("db.diff")(function* (flags: DbDiffFlags) { dnsResolver, password: Option.none(), linkedProjectRef: flags.projectRef, + target: linkedTarget, }); if (linkedRef === undefined) { linkedRef = Option.getOrUndefined(resolved.ref ?? Option.none()); diff --git a/apps/cli/src/commands/db/diff/diff.integration.test.ts b/apps/cli/src/commands/db/diff/diff.integration.test.ts index 464e38df92..ae20831f95 100644 --- a/apps/cli/src/commands/db/diff/diff.integration.test.ts +++ b/apps/cli/src/commands/db/diff/diff.integration.test.ts @@ -69,10 +69,7 @@ import { import { GoProxy } from "../../../command-internal/go-proxy.service.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { - ProjectRefResolver, - PROJECT_NOT_LINKED_MESSAGE, -} from "../../../config/project-ref.service.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../../config/remote-target.service.ts"; import { DbConfigLoadError } from "../../../command-internal/db-config.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { @@ -100,6 +97,7 @@ import { dbDiff } from "./diff.handler.ts"; import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; import { StackNativeEngineError } from "../../../command-internal/stack-local-database.ts"; import { PGADMIN_DESKTOP_NOTE_PREFIX, PGADMIN_DIFF_HEADER } from "./pgadmin-diff.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; interface SetupOpts { readonly format?: OutputFormat; @@ -134,7 +132,7 @@ interface SetupOpts { // `Option.none()` to exercise the config.toml/workdir-basename fallback // (`resolveLocalProjectId`). readonly projectId?: Option.Option; - // Simulates an unlinked workdir: `loadProjectRef` fails with + // Simulates an unlinked workdir: `RemoteTarget.load` fails with // `ProjectRefNotLinkedError` absent an explicit `--project-ref` flag. readonly linkedFails?: boolean; // Per-differ-run `--json-diff` stdout, indexed by run order; falls back to `""` @@ -423,17 +421,17 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Mirrors the same ref `resolver`'s own mock embeds above, and gives an explicit // `--project-ref` flag top precedence over `opts.linkedRef` (mirrors // `reset.integration.test.ts`'s identical mock). - const projectRefResolver = Layer.succeed(ProjectRefResolver, { + const projectRefResolver = stubRemoteTarget({ resolve: () => Effect.succeed(opts.linkedRef ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.linkedRef ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.linkedRef ?? VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true ? Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) : Effect.succeed(opts.linkedRef ?? VALID_REF), - promptProjectRef: () => Effect.succeed(opts.linkedRef ?? VALID_REF), + prompt: () => Effect.succeed(opts.linkedRef ?? VALID_REF), }); const proxyCalls: Array<{ args: ReadonlyArray; env?: Record }> = []; diff --git a/apps/cli/src/commands/db/diff/diff.stack-cache.e2e.test.ts b/apps/cli/src/commands/db/diff/diff.stack-cache.e2e.test.ts index db7c9354f1..06ef9201d1 100644 --- a/apps/cli/src/commands/db/diff/diff.stack-cache.e2e.test.ts +++ b/apps/cli/src/commands/db/diff/diff.stack-cache.e2e.test.ts @@ -6,6 +6,7 @@ import { create as createStack, discover as discoverStacks } from "@supabase/sta import { runSupabaseEffect } from "../../../../tests/helpers/cli.ts"; import { destroyTestStack } from "../../../../tests/helpers/stack-cleanup.ts"; import { stackArtifactCacheRoot } from "../../../../tests/helpers/stack-artifacts.ts"; +import { selection } from "../../../../tests/helpers/pins.ts"; const COMMAND_TIMEOUT_MS = 8 * 60_000; const TEST_TIMEOUT_MS = COMMAND_TIMEOUT_MS * 4 + 2 * 60_000; @@ -46,8 +47,8 @@ const composeStack = Effect.fn("DbDiffStackCacheE2e.composeStack")(function* ( const [database] = yield* stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make("db-diff-stack-cache-e2e-jwt-secret-with-32-chars"), jwtExpiry: 3600, diff --git a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md index 3f5b57cacf..2e43311884 100644 --- a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md @@ -11,7 +11,7 @@ image), to stdout or `--file`. | --------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `supabase/config.toml` | TOML | always (db port/password/major_version, project_id) | | `supabase/.temp/postgres-version` | plain text | always (best-effort) — pins the pg image tag when present | -| `supabase/.temp/pooler-url` | plain text | `--linked` when the direct host is unreachable (pooler URL) | +| `supabase/.temp/pooler-url` | plain text | `--linked` when the direct host is unreachable and the target ref is the linked ref (pooler URL) | | `~/.supabase/access-token` | plain text | `--linked` when `SUPABASE_ACCESS_TOKEN` unset | | `supabase/.temp/project-ref` | plain text | `--linked` (and the default target) ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`/config.toml `project_id`) is set | | `supabase/.env*` | dotenv | always (project env, feeds `SUPABASE_DB_PASSWORD` / `PG*`) | @@ -94,7 +94,10 @@ shell inherits the suppressing variables and is missed. id. It never implies `--linked`: passing it with a resolved `--local`/`--db-url` target is a hard error rather than a silently discarded flag (deliberately stricter than `SUPABASE_PROJECT_ID`, which Go's equivalent - env var simply leaves unused on a non-linked target). + env var simply leaves unused on a non-linked target). The saved `.temp/pooler-url` + applies only when the target ref is the linked ref; a `--project-ref` or + `SUPABASE_PROJECT_ID` naming another project fetches that project's pooler config from the + Management API. - **Container-level pooler fallback.** When a linked dump reaches the direct host from the host process but the `pg_dump` container fails over IPv6, the captured container stderr is classified (`isIPv6ConnectivityError`) and the dump is diff --git a/apps/cli/src/commands/db/dump/dump.handler.ts b/apps/cli/src/commands/db/dump/dump.handler.ts index e6f8a77786..007d1a1dc9 100644 --- a/apps/cli/src/commands/db/dump/dump.handler.ts +++ b/apps/cli/src/commands/db/dump/dump.handler.ts @@ -2,7 +2,7 @@ import { Config, Effect, FileSystem, Option, Path, Predicate, Ref } from "effect import type { Stack } from "@supabase/stack/effect"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -48,10 +48,11 @@ import { } from "../../../command-internal/postgres-client.run.ts"; import { currentStackBackend } from "../../../command-internal/stack-backend.ts"; import { - stackProjectDatabaseVersion, + stackProjectDatabasePin, stackOpenReadyProject, stackRequireProjectRuntime, } from "../../../command-internal/stack-local-database.ts"; +import { databaseArtifactPin } from "@supabase/stack/internal/artifacts"; import { resolveBundledPostgresRuntime } from "../../../command-internal/bundled-postgres-client.ts"; import { viperEnvStringWithProjectFallback } from "../../../command-internal/viper-env.ts"; import { runWithPoolerFallback } from "../shared/pooler-fallback.ts"; @@ -161,12 +162,13 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { } // The project ref is resolved before the connection is built, and the // linked-project cache is refreshed unconditionally afterward, even on a - // connection-resolution failure. `loadProjectRef` validates the ref up front so an + // connection-resolution failure. `RemoteTarget.load` validates the ref up front so an // unvalidated raw `--project-ref` is never stored for the cache finalizer. + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const refResolver = yield* ProjectRefResolver; - const ref = yield* refResolver.loadProjectRef(flags.projectRef); - yield* Ref.set(linkedRefForCache, ref); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + yield* Ref.set(linkedRefForCache, linkedTarget.ref); } const { conn, @@ -178,6 +180,7 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { dnsResolver, password: flags.password, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); const db = isLocal ? "local" : "remote"; // On the linked path, re-read config with the resolved ref so a matching @@ -244,27 +247,32 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { }), } : conn; - const catalogVersion = - backend.kind === "stack" && connType === "local" - ? ((yield* stackProjectDatabaseVersion) ?? String(tomlValues.majorVersion)) - : String(tomlValues.majorVersion); + // A local stack's client matches its saved database build; any other target uses the + // configured major's default pin. + const databasePin = + backend.kind === "stack" + ? yield* Effect.gen(function* () { + const saved = connType === "local" ? yield* stackProjectDatabasePin : undefined; + return saved ?? (yield* databaseArtifactPin(String(tomlValues.majorVersion))); + }).pipe(Effect.mapError((cause) => new DbDumpRunError({ message: cause.message, cause }))) + : undefined; const dumpMajor = tomlValues.majorVersion; const dumpClient = - managedStack !== undefined - ? { - kind: "stack" as const, - stack: managedStack, - command: roleOnly ? ("pg_dumpall" as const) : ("pg_dump" as const), - major: Number(catalogVersion.split(".")[0]) === 15 ? (15 as const) : (17 as const), - } - : backend.kind === "stack" + databasePin === undefined + ? { kind: "container" as const } + : managedStack !== undefined ? { + kind: "stack" as const, + stack: managedStack, + command: roleOnly ? ("pg_dumpall" as const) : ("pg_dump" as const), + pin: databasePin, + } + : { kind: "bundled" as const, command: roleOnly ? ("pg_dumpall" as const) : ("pg_dump" as const), - version: catalogVersion, + pin: databasePin, runtime: bundledRuntime, - } - : { kind: "container" as const }; + }; // 4. Pick the mode-specific script + env. --schema/-s and --exclude/-x arrive here // already CSV-parsed by `parseSchemaFlags`. @@ -436,6 +444,7 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { dnsResolver, password: flags.password, linkedProjectRef: flags.projectRef, + target: linkedTarget, }) .pipe(Effect.orElseSucceed(() => Option.none())), runWithConn: (c) => runContainer(mode.buildEnv(c, opt)), diff --git a/apps/cli/src/commands/db/dump/dump.integration.test.ts b/apps/cli/src/commands/db/dump/dump.integration.test.ts index a4fd320525..d144558b58 100644 --- a/apps/cli/src/commands/db/dump/dump.integration.test.ts +++ b/apps/cli/src/commands/db/dump/dump.integration.test.ts @@ -60,10 +60,9 @@ import { } from "../../../config/project-ref.errors.ts"; import { INVALID_PROJECT_REF_MESSAGE, - ProjectRefResolver, PROJECT_NOT_LINKED_MESSAGE, PROJECT_REF_PATTERN, -} from "../../../config/project-ref.service.ts"; +} from "../../../config/remote-target.service.ts"; import { dbConfig } from "../../../command-internal/db-config.layer.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags } from "../../../command-internal/db-config.types.ts"; @@ -84,6 +83,8 @@ import type { DbDumpFlags } from "./dump.command.ts"; import { dbDump } from "./dump.handler.ts"; import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; import { StackApi } from "../../../command-internal/stack-api.ts"; +import { selection } from "../../../../tests/helpers/pins.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LOCAL_CONN: PgConnInput = { host: "127.0.0.1", @@ -104,8 +105,8 @@ const managedDumpStackApi = (runtime: "native" | "docker") => { const id = "m".repeat(64); const creation: Extract = { service: "database", + pin: selection.database, config: { - version: "17.6.1.173", databasePassword: Redacted.make("secret"), jwtSecret: Redacted.make("secret"), jwtExpiry: 3600, @@ -277,12 +278,12 @@ function mockResolver(opts: { } /** - * Mocks `ProjectRefResolver` for the up-front `loadProjectRef` pre-capture, mirroring + * Mocks `RemoteTarget` for the up-front `RemoteTarget.load` pre-capture, mirroring * push/diff's identical mock: an explicit `--project-ref` flag wins, a malformed value * fails with `InvalidProjectRefError`, and `opts.projectId` stands in for * `CommandSettings.projectId` consulted before falling back to `opts.ref`. */ -function mockProjectRefResolver(opts: { +function mockRemoteTarget(opts: { projectId: Option.Option; ref?: string; linkedFails?: boolean; @@ -291,11 +292,11 @@ function mockProjectRefResolver(opts: { PROJECT_REF_PATTERN.test(ref) ? Effect.succeed(ref) : Effect.fail(new InvalidProjectRefError({ ref, message: INVALID_PROJECT_REF_MESSAGE })); - const layer = Layer.succeed(ProjectRefResolver, { + const layer = stubRemoteTarget({ resolve: () => Effect.succeed(opts.ref ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.ref ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.ref ?? VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => { + load: (flagValue: Option.Option) => { if (Option.isSome(flagValue) && flagValue.value.length > 0) { return validate(flagValue.value); } @@ -306,7 +307,7 @@ function mockProjectRefResolver(opts: { ? Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) : Effect.succeed(opts.ref ?? VALID_REF); }, - promptProjectRef: () => Effect.succeed(opts.ref ?? VALID_REF), + prompt: () => Effect.succeed(opts.ref ?? VALID_REF), }); return { layer }; } @@ -465,7 +466,7 @@ function setup(opts: SetupOpts = {}) { resolveFails: opts.resolveFails, ref: opts.ref, }); - const projectRef = mockProjectRefResolver({ + const projectRef = mockRemoteTarget({ projectId: opts.projectId ?? Option.none(), ref: opts.ref, linkedFails: opts.linkedFails, @@ -1037,7 +1038,7 @@ describe("db dump integration", () => { it.effect( "rejects a malformed --project-ref on the linked path before resolving or caching", () => { - // `loadProjectRef` validates before `resolver.resolve()` runs, so a malformed + // `RemoteTarget.load` validates before `resolver.resolve()` runs, so a malformed // flag fails fast with no connection/API work and no cache write. const { layer, cache, resolver } = setup(); return Effect.gen(function* () { diff --git a/apps/cli/src/commands/db/dump/dump.layers.ts b/apps/cli/src/commands/db/dump/dump.layers.ts index 85229a4aa9..3d7e3f80c9 100644 --- a/apps/cli/src/commands/db/dump/dump.layers.ts +++ b/apps/cli/src/commands/db/dump/dump.layers.ts @@ -4,7 +4,7 @@ import { commandCredentialsLayer } from "../../../auth/command-credentials.layer import { httpClientLayer } from "../../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-api-factory.layer.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; import { dockerRunLayer } from "../../../command-internal/docker-run.layer.ts"; @@ -20,7 +20,7 @@ import { bundledPostgresClientLayer } from "../../../command-internal/bundled-po * Runtime layer for `supabase db dump`. * * Mirrors `test db`'s composition: most of the Management API stack builds lazily - * inside the resolver's `--linked` branch. `ProjectRefResolver` is exposed here (like + * inside the resolver's `--linked` branch. `RemoteTarget` is exposed here (like * `db push`) so the handler can validate `--project-ref` before the linked-project-cache * finalizer sees it. */ @@ -41,9 +41,9 @@ const platformApiFactory = commandPlatformApiFactoryLayer.pipe( Layer.provide(identityStitchLayer), ); -// Exposed so the handler can pre-validate `--project-ref` via `loadProjectRef` +// Exposed so the handler can pre-validate `--project-ref` via `RemoteTarget.load` // before the linked-project-cache finalizer ever sees it. -const projectRef = projectRefLayer.pipe( +const projectRef = remoteTargetLayer.pipe( Layer.provide(platformApiFactory), Layer.provide(cliSettings), ); diff --git a/apps/cli/src/commands/db/lint/lint.handler.ts b/apps/cli/src/commands/db/lint/lint.handler.ts index 7f739bef56..a0f942e71f 100644 --- a/apps/cli/src/commands/db/lint/lint.handler.ts +++ b/apps/cli/src/commands/db/lint/lint.handler.ts @@ -4,7 +4,7 @@ import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { ProcessControl } from "../../../shared/runtime/process-control.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { failsOn } from "../../../command-internal/fail-on.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; @@ -132,6 +132,10 @@ const runLint = Effect.fnUntraced(function* ( // `Flag.mapTryCatch(parseSchemaFlags)` at the command definition. const schemaFlags = flags.schema; + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + target.connType === "linked" ? yield* remoteTarget.load(flags.projectRef) : undefined; + const lintBody = Effect.gen(function* () { // connType passes straight through; the resolver applies db-url > linked > // local precedence and handles `--local`'s default. @@ -140,6 +144,7 @@ const runLint = Effect.fnUntraced(function* ( connType: target.connType ?? "local", dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); yield* Effect.annotateCurrentSpan({ @@ -216,11 +221,9 @@ const runLint = Effect.fnUntraced(function* ( // For `--linked`, the ref is resolved up front (non-prompting) and the // linked-project cache is refreshed on both success and failure; `--local`/`--db-url` // never write it since caching only runs on the linked branch. - if (target.connType === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const ref = yield* projectRef.loadProjectRef(flags.projectRef); - return yield* lintBody.pipe(Effect.ensuring(linkedProjectCache.cache(ref))); + return yield* lintBody.pipe(Effect.ensuring(linkedProjectCache.cache(linkedTarget.ref))); } return yield* lintBody; }); diff --git a/apps/cli/src/commands/db/lint/lint.integration.test.ts b/apps/cli/src/commands/db/lint/lint.integration.test.ts index 3b9e6c8397..b7abf06966 100644 --- a/apps/cli/src/commands/db/lint/lint.integration.test.ts +++ b/apps/cli/src/commands/db/lint/lint.integration.test.ts @@ -33,7 +33,6 @@ import { } from "../../../../tests/helpers/permission-real-layers.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; import { DbExecError } from "../../../command-internal/db-connection.errors.ts"; @@ -47,6 +46,7 @@ import { encodeLintResults, parseLintResult } from "./lint.format.ts"; import { dbLint } from "./lint.handler.ts"; import { CHECK_SCHEMA_SCRIPT, ENABLE_PGSQL_CHECK, LIST_SCHEMAS_SQL } from "./lint.lint-sql.ts"; import type { DbLintFlags } from "./lint.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LOCAL_CONN: PgConnInput = { host: "127.0.0.1", @@ -161,22 +161,22 @@ function mockConnection(opts: { } /** Project-ref resolver mock. `db lint --linked` resolves the ref via the - * non-prompting `loadProjectRef` to write the linked-project cache; the + * non-prompting `RemoteTarget.load` to write the linked-project cache; the * other methods are unused by lint. */ function mockProjectRef() { const calls: Array = []; - const layer = Layer.succeed(ProjectRefResolver, { + const layer = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), // An explicit `--project-ref` flag takes top precedence, mirroring the real // resolver. - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.sync(() => { - calls.push("loadProjectRef"); + calls.push("load"); return Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF; }), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); return { layer, @@ -515,7 +515,7 @@ describe("db lint", () => { }); return Effect.gen(function* () { yield* dbLint(flags({ schema: ["public"] })); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(cache.cached).toBe(true); }).pipe(Effect.provide(layer)); }); @@ -705,7 +705,7 @@ describe("db lint", () => { }); return Effect.gen(function* () { yield* dbLint(flags({ schema: ["public"] })); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(cache.cached).toBe(true); }).pipe(Effect.provide(layer)); }); @@ -718,7 +718,7 @@ describe("db lint", () => { }); return Effect.gen(function* () { yield* dbLint(flags({ schema: ["public"] })); - expect(projectRef.calls).toContain("loadProjectRef"); + expect(projectRef.calls).toContain("load"); expect(cache.cached).toBe(true); }).pipe(Effect.provide(layer)); }); diff --git a/apps/cli/src/commands/db/lint/lint.layers.ts b/apps/cli/src/commands/db/lint/lint.layers.ts index f3c5c291cc..86d90f0b78 100644 --- a/apps/cli/src/commands/db/lint/lint.layers.ts +++ b/apps/cli/src/commands/db/lint/lint.layers.ts @@ -5,7 +5,7 @@ import { commandCredentialsLayer } from "../../../auth/command-credentials.layer import { httpClientLayer } from "../../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-api-factory.layer.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; import { debugLoggerLayer } from "../../../command-internal/debug-logger.layer.ts"; @@ -42,7 +42,7 @@ const platformApiFactory = commandPlatformApiFactoryLayer.pipe( Layer.provide(identityStitchLayer), ); -const projectRef = projectRefLayer.pipe( +const projectRef = remoteTargetLayer.pipe( Layer.provide(platformApiFactory), Layer.provide(cliSettings), ); diff --git a/apps/cli/src/commands/db/lint/lint.layers.unit.test.ts b/apps/cli/src/commands/db/lint/lint.layers.unit.test.ts index 271ce17692..a33ba6e459 100644 --- a/apps/cli/src/commands/db/lint/lint.layers.unit.test.ts +++ b/apps/cli/src/commands/db/lint/lint.layers.unit.test.ts @@ -41,13 +41,13 @@ import { } from "../../../command-internal/global-flags.ts"; import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; import { IdentityStitch } from "../../../command-internal/identity-stitch.ts"; import { dbAdvisorsRuntimeLayer } from "../advisors/advisors.layers.ts"; import { dbLintRuntimeLayer } from "./lint.layers.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const tempRoot = useTempWorkdir("supabase-lint-layers-"); @@ -82,12 +82,12 @@ function ambientStubs() { resolvePoolerFallback: () => Effect.die("db-config-resolver not needed for layer-exposure test"), }), - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveForLink: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveOptional: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - loadProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - promptProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + load: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + prompt: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), }), Layer.succeed(CommandPlatformApiFactory, { make: Effect.die("platform-api-factory not needed for layer-exposure test"), diff --git a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md index a4cc4e29be..da619cd34a 100644 --- a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md @@ -29,7 +29,7 @@ or multi-host `--db-url` without an explicit `sslmode` still requires TLS. Pg-delta runs in-process. Coverage gaps warn; `--strict-coverage` makes them fatal, while `PGDELTA_DEBUG` writes diagnostic JSON under -`supabase/.temp/pgdelta/v2/debug//`. The engine may emit transaction-aware +`supabase/.temp/pgdelta/v2/debug//` (`/tmp/pgdelta/v2/debug//` in stack mode). The engine may emit transaction-aware file splits; applicable, convergent SQL is the contract. Its formatter defaults to lowercase SQL at width 180; config overrides it, and JSON `null` disables formatting without disabling safe compaction. @@ -57,7 +57,7 @@ disables formatting without disabling safe compaction. | `/supabase/schemas/**` | SQL | `--declarative` or deprecated `--experimental` export | | `/supabase/schemas/.pgdelta-export.json` | JSON | bundled declarative / deprecated-`--experimental` export metadata | | `/supabase/config.toml` | TOML | declarative export updates `[db.migrations].schema_paths` when `[experimental.pgdelta] enabled` resolves false (section absent, `enabled` omitted, or `enabled = false` — the default); skipped when `enabled = true` | -| `/supabase/.temp/pgdelta/v2/debug//*.json` | JSON | bundled engine with `PGDELTA_DEBUG` | +| `/supabase/.temp/pgdelta/v2/debug//*.json` | JSON | bundled engine with `PGDELTA_DEBUG`; `/tmp/pgdelta/v2/debug//` in stack mode | | `~/.supabase/cache/shadow-baseline/shadow-baseline-.tar` | tar | cache-enabled COLD shadow provision creates the current key's snapshot, migration-style pull only (never a declarative / deprecated-`--experimental` export, which provisions no shadow); a warm hit `touch`es its mtime (LRU); every cache-eligible acquire may delete other keys under LRU keep-3 + 2-day mtime TTL — ~90MB (`SUPABASE_HOME` overrides the root) | | managed snapshot key `` | backend | stack-mode cache-enabled cold shadow provision saves the database baseline; the last completed save replaces the key | | `~/.supabase/cache/shadow-baseline/shadow-baseline-.tar..partial` | tar | during a cold export — the in-flight temp file, `rename`d into the tar above on success and removed on failure; only a crash/SIGKILL leaves it behind, and later cold exports / warm hits sweep leftovers older than 5 minutes | @@ -185,7 +185,7 @@ Progress strings still go to stderr; stdout carries a single structured envelope - `--declarative` / deprecated `--use-pg-delta` are mutually exclusive with `--diff-engine`; `--db-url` / `--linked` (default) / `--local` are a target group. - **`--project-ref`** (TS-only, no Go equivalent on any user-facing `db` - command) overrides ONLY the linked-ref resolution `ProjectRefResolver` + command) overrides ONLY the linked-ref resolution `RemoteTarget` performs (flag > `SUPABASE_PROJECT_ID` > `.temp/project-ref`) — unlike `SUPABASE_PROJECT_ID`, it does not affect the shadow container's project id/labels. It never implies `--linked`: passing it with a resolved diff --git a/apps/cli/src/commands/db/pull/pull.integration.test.ts b/apps/cli/src/commands/db/pull/pull.integration.test.ts index c2ff521ae2..24f322600f 100644 --- a/apps/cli/src/commands/db/pull/pull.integration.test.ts +++ b/apps/cli/src/commands/db/pull/pull.integration.test.ts @@ -74,10 +74,7 @@ import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { GoProxy } from "../../../command-internal/go-proxy.service.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { - ProjectRefResolver, - PROJECT_NOT_LINKED_MESSAGE, -} from "../../../config/project-ref.service.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../../config/remote-target.service.ts"; import { dbConfig } from "../../../command-internal/db-config.layer.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { type DbSession, DbConnection } from "../../../command-internal/db-connection.service.ts"; @@ -96,6 +93,10 @@ import type { DbRemoteCommitFlags } from "../remote/commit/commit.command.ts"; import type { DbPullFlags } from "./pull.command.ts"; import { dbPull } from "./pull.handler.ts"; import { runDbPull } from "../../../command-internal/db-pull-run.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; +import { database15, selection } from "../../../../tests/helpers/pins.ts"; +import { seedLock, unbundled } from "../../../../tests/helpers/stack-lock.ts"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; const alwaysReadyHttpClientLayer = Layer.succeed( HttpClient.HttpClient, @@ -177,7 +178,7 @@ interface SetupOpts { // `Option.none()` to exercise the config.toml/workdir-basename fallback // (`resolveLocalProjectId`). readonly projectId?: Option.Option; - // Simulates an unlinked workdir: `loadProjectRef` fails with + // Simulates an unlinked workdir: `RemoteTarget.load` fails with // `ProjectRefNotLinkedError` absent an explicit `--project-ref` flag. readonly linkedFails?: boolean; // Swaps in the stateful Docker model (real `stop`/`cp`/`start`), required by the @@ -458,17 +459,17 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Mirrors the same ref `resolver`'s own mock embeds above, and gives an explicit // `--project-ref` flag top precedence over `opts.resolvedRef` (mirrors // `reset.integration.test.ts`'s identical mock). - const projectRefResolver = Layer.succeed(ProjectRefResolver, { + const projectRefResolver = stubRemoteTarget({ resolve: () => Effect.succeed(opts.resolvedRef ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.resolvedRef ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.resolvedRef ?? VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true ? Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) : Effect.succeed(opts.resolvedRef ?? VALID_REF), - promptProjectRef: () => Effect.succeed(opts.resolvedRef ?? VALID_REF), + prompt: () => Effect.succeed(opts.resolvedRef ?? VALID_REF), }); const workdirFiles = Layer.effectDiscard( @@ -1703,6 +1704,72 @@ describe("db pull", () => { }, ); + describe("in stack state mode", () => { + const remoteMajorConfig = [ + "[db]", + "major_version = 15", + "[remotes.staging]", + `project_id = "${VALID_REF}"`, + "[remotes.staging.db]", + "major_version = 17", + "", + ].join("\n"); + + it.effect( + "dumps the initial linked pull with the image of the matching [remotes.] major", + () => { + const s = setup(tmp.current, { + files: { "supabase/config.toml": remoteMajorConfig }, + remoteVersions: [], + dumpStdout: "create table dumped ();\n", + yes: true, + }); + return Effect.gen(function* () { + yield* dbPull(flags()); + expect(s.dumpCalls[0]?.image).toBe(selection.database.image); + expect(s.dumpCalls[0]?.image).not.toBe(database15.image); + }).pipe(Effect.provide(Layer.merge(s.layer, projectStateModeLayer("stack")))); + }, + ); + + it.effect( + "dumps with the remote major's bundled image when the project lock records the local major", + () => { + const s = setup(tmp.current, { + files: { "supabase/config.toml": remoteMajorConfig }, + remoteVersions: [], + dumpStdout: "create table dumped ();\n", + yes: true, + }); + return Effect.gen(function* () { + yield* seedLock(tmp.current, (services) => ({ + ...services, + database: { ...services.database, resolved: unbundled(database15) }, + })); + yield* dbPull(flags()); + expect(s.dumpCalls[0]?.image).toBe(selection.database.image); + }).pipe(Effect.provide(Layer.merge(s.layer, projectStateModeLayer("stack")))); + }, + ); + + it.effect("dumps with the locked image when no remote block applies", () => { + const s = setup(tmp.current, { + files: { "supabase/config.toml": "[db]\nmajor_version = 15\n" }, + remoteVersions: [], + dumpStdout: "create table dumped ();\n", + yes: true, + }); + return Effect.gen(function* () { + yield* seedLock(tmp.current, (services) => ({ + ...services, + database: { ...services.database, resolved: unbundled(database15) }, + })); + yield* dbPull(flags()); + expect(s.dumpCalls[0]?.image).toBe(unbundled(database15).image); + }).pipe(Effect.provide(Layer.merge(s.layer, projectStateModeLayer("stack")))); + }); + }); + it.effect( "resolves the pg_dump network via SUPABASE_NETWORK_ID from supabase/.env when neither the flag nor the ambient env is set", () => { diff --git a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md index 1f5ac260ac..d959a3bfcc 100644 --- a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md @@ -96,7 +96,7 @@ stdout is payload-only. A single `result` object is emitted: - **Targets**: `--db-url`, `--linked` (default), and `--local` are mutually exclusive; with no flag the target defaults to linked. - **`--project-ref`** (TS-only, no Go equivalent on any user-facing `db` - command) overrides ONLY the linked-ref resolution `ProjectRefResolver` + command) overrides ONLY the linked-ref resolution `RemoteTarget` performs (flag > `SUPABASE_PROJECT_ID` > `~/.supabase//project-ref`). It never implies `--linked`: passing it with a resolved `--local`/`--db-url` target is a hard error rather than a silently discarded flag (deliberately diff --git a/apps/cli/src/commands/db/push/push.command.ts b/apps/cli/src/commands/db/push/push.command.ts index 1245f6a140..055f11029f 100644 --- a/apps/cli/src/commands/db/push/push.command.ts +++ b/apps/cli/src/commands/db/push/push.command.ts @@ -51,7 +51,7 @@ const config = { Flag.withDescription("Pushes to the local database."), Flag.withDefault(false), ), - // Feeds `ProjectRefResolver.loadProjectRef`: flag > `SUPABASE_PROJECT_ID` > + // Feeds `RemoteTarget.load`: flag > `SUPABASE_PROJECT_ID` > // `supabase/.temp/project-ref`. Only feeds ref resolution — it does not affect local // container ids or the pg-delta project id (see `db-config.types.ts`'s // `linkedProjectRef` doc) — and is rejected outright on a non-linked target rather diff --git a/apps/cli/src/commands/db/push/push.handler.ts b/apps/cli/src/commands/db/push/push.handler.ts index 4f7f1d57a8..25455ce20c 100644 --- a/apps/cli/src/commands/db/push/push.handler.ts +++ b/apps/cli/src/commands/db/push/push.handler.ts @@ -5,7 +5,7 @@ import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { resolveYesWithProjectEnv } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../../../config/remote-target.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { checkDbToml, @@ -72,9 +72,11 @@ export const dbPush = Effect.fn("db.push")(function* (flags: DbPushFlags) { // matching `[remotes.]` block merges. For `--local` / `--db-url`, // the ref stays empty. let projectRef = ""; + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const refResolver = yield* ProjectRefResolver; - projectRef = yield* refResolver.loadProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + projectRef = linkedTarget.ref; yield* Ref.set(linkedRefForCache, projectRef); } @@ -98,6 +100,7 @@ export const dbPush = Effect.fn("db.push")(function* (flags: DbPushFlags) { password: flags.password, resolveVaultSecrets: !flags.skipVault, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); yield* Effect.annotateCurrentSpan({ diff --git a/apps/cli/src/commands/db/push/push.integration.test.ts b/apps/cli/src/commands/db/push/push.integration.test.ts index c2f5aff5d5..94234b5439 100644 --- a/apps/cli/src/commands/db/push/push.integration.test.ts +++ b/apps/cli/src/commands/db/push/push.integration.test.ts @@ -38,7 +38,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag, YesFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; @@ -51,6 +50,10 @@ import { import { dbPush } from "./push.handler.ts"; import type { DbPushFlags } from "./push.command.ts"; import { DbPushApplyError } from "./push.errors.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; +import { remoteTargetLayer as realRemoteTargetLayer } from "../../../config/remote-target.layer.ts"; +import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; const LIST_MIGRATIONS = "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; @@ -196,8 +199,30 @@ function setup( simulateInitialisingLoginRole?: boolean; /** Runs the real `DbConfigResolver` and linked-project cache against `workdir`. */ realLayers?: boolean; + /** Resolves the hosted target through the real `RemoteTarget` against `workdir`. */ + realRemoteTarget?: boolean; + /** Collects the path of every `.link.json` read made through the layer's file system. */ + linkReads?: Array; }, ) { + const services = + opts.linkReads === undefined + ? BunServices.layer + : Layer.mergeAll( + BunServices.layer, + Layer.effect( + FileSystem.FileSystem, + Effect.map(FileSystem.FileSystem, (real) => + FileSystem.FileSystem.of({ + ...real, + readFileString: (file, encoding) => { + if (file.endsWith(".link.json")) opts.linkReads?.push(file); + return real.readFileString(file, encoding); + }, + }), + ), + ).pipe(Layer.provide(BunServices.layer)), + ); const workdirLayer = Layer.effectDiscard( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -219,13 +244,13 @@ function setup( const telemetry = mockTelemetryStateTracked(); const linkedCache = mockLinkedProjectCacheTracked(); - const projectRefLayer = Layer.succeed(ProjectRefResolver, { + const stubbedRemoteTarget = stubRemoteTarget({ resolve: () => Effect.succeed(opts.projectRef ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.projectRef ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.projectRef ?? VALID_REF)), // An explicit `--project-ref` flag takes top precedence, short-circuiting before // the "not linked" failure. - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true @@ -235,9 +260,26 @@ function setup( }), ) : Effect.succeed(opts.projectRef ?? VALID_REF), - promptProjectRef: () => Effect.succeed(opts.projectRef ?? VALID_REF), + prompt: () => Effect.succeed(opts.projectRef ?? VALID_REF), }); + const remoteTargetLayer = + opts.realRemoteTarget === true + ? realRemoteTargetLayer.pipe( + Layer.provide( + Layer.mergeAll( + mockCommandSettings({ workdir, projectId: Option.none() }), + mockTty({ stdinIsTty: false }), + out.layer, + Layer.succeed(CommandPlatformApiFactory, { + make: Effect.die("the platform API is not used when a link names the target"), + }), + services, + ), + ), + ) + : stubbedRemoteTarget; + const resolver = mockResolver({ isLocal: opts.isLocal ?? true, onResolve: @@ -257,13 +299,14 @@ function setup( workdir, dbConnection: conn.layer, env: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, + services, }) : resolver.layer, mockCommandSettings({ workdir, ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), }), - BunServices.layer, + services, // Prompts are answered through mockOutput's `promptConfirmResponses` (the // TTY/clack path) unless `piped` feeds promptYesNo's non-TTY branch. mockTty({ stdinIsTty: opts.piped === undefined }), @@ -271,7 +314,7 @@ function setup( Layer.succeed(CliArgs, { args: opts.args ?? ["db", "push", "--local"] }), Layer.succeed(YesFlag, opts.yes ?? false), Layer.succeed(DnsResolverFlag, "native"), - projectRefLayer, + remoteTargetLayer, telemetry.layer, opts.realLayers === true ? realLinkedProjectCache({ @@ -916,6 +959,39 @@ describe("db push", () => { }); }); + it.effect("resolves the linked target once across the whole push", () => { + const link = { + project: { ref: VALID_REF }, + active_branch: { + ref: VALID_REF, + pooler_url: `postgres://postgres.${VALID_REF}:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.com:6543/postgres`, + }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }; + const linkReads: Array = []; + const { layer } = setup(tmp.current, { + toml: 'project_id = "test"\n', + files: { "supabase/.link.json": JSON.stringify(link) }, + args: ["db", "push", "--linked"], + isLocal: false, + realLayers: true, + realRemoteTarget: true, + linkReads, + }); + const requests: Array = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, serveLinkedDatabase(VALID_REF)); + yield* dbPush({ ...DEFAULT_FLAGS, local: false, linked: true }).pipe( + Effect.provide(Layer.mergeAll(layer, projectStateModeLayer("stack"))), + ); + expect(linkReads).toHaveLength(1); + }).pipe( + Effect.scoped, + Effect.provide(BunServices.layer), + provideEnv({ SUPABASE_ACCESS_TOKEN: VALID_TOKEN }), + ); + }); + it.effect("mints a login role and fills the cache through the real resolver and cache", () => { // No target flag: push defaults to the linked project. const { layer, projectHttp } = setup(tmp.current, { diff --git a/apps/cli/src/commands/db/push/push.layers.ts b/apps/cli/src/commands/db/push/push.layers.ts index 92008ab9c8..8ef569a655 100644 --- a/apps/cli/src/commands/db/push/push.layers.ts +++ b/apps/cli/src/commands/db/push/push.layers.ts @@ -5,7 +5,7 @@ import { commandCredentialsLayer } from "../../../auth/command-credentials.layer import { httpClientLayer } from "../../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-api-factory.layer.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; import { debugLoggerLayer } from "../../../command-internal/debug-logger.layer.ts"; @@ -41,7 +41,7 @@ const platformApiFactory = commandPlatformApiFactoryLayer.pipe( Layer.provide(identityStitchLayer), ); -const projectRef = projectRefLayer.pipe( +const projectRef = remoteTargetLayer.pipe( Layer.provide(platformApiFactory), Layer.provide(cliSettings), ); diff --git a/apps/cli/src/commands/db/query/query.handler.ts b/apps/cli/src/commands/db/query/query.handler.ts index b64b2d3276..2e572c08bd 100644 --- a/apps/cli/src/commands/db/query/query.handler.ts +++ b/apps/cli/src/commands/db/query/query.handler.ts @@ -5,7 +5,7 @@ import * as HttpClientError from "effect/http/HttpClientError"; import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; import { CommandCredentials } from "../../../auth/command-credentials.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { TelemetryOutputFormat } from "../../../telemetry/telemetry-output-format.service.ts"; @@ -250,11 +250,12 @@ export const dbQuery = Effect.fn("db.query")(function* (flags: DbQueryFlags) { let linkedRef: string | undefined; if (Option.isSome(flags.linked)) { const credentials = yield* CommandCredentials; - const projectRef = yield* ProjectRefResolver; + const projectRef = yield* RemoteTarget; // Config is resolved before the token check, so an unlinked-project/invalid-config/pooler - // failure isn't masked behind a generic "supabase login" error. `loadProjectRef` is the + // failure isn't masked behind a generic "supabase login" error. `RemoteTarget.load` is the // strict loader: a real ref-file read error surfaces instead of being treated as not-linked. - const ref = yield* projectRef.loadProjectRef(flags.projectRef); + const linkedTarget = yield* projectRef.load(flags.projectRef); + const { ref } = linkedTarget; // Recorded now so the cache finalizer still fires if a later step fails. yield* Ref.set(linkedRefForCache, ref); // Loads and validates the remote-merged config and resolves the live DB connection (TCP @@ -265,6 +266,7 @@ export const dbQuery = Effect.fn("db.query")(function* (flags: DbQueryFlags) { connType: "linked", dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); // The Management API still needs a token even when config resolved without minting a login // role. Route through `credentials.getAccessToken`, which validates it against `sbp_...`, diff --git a/apps/cli/src/commands/db/query/query.integration.test.ts b/apps/cli/src/commands/db/query/query.integration.test.ts index d368f3e1d9..d807d9db43 100644 --- a/apps/cli/src/commands/db/query/query.integration.test.ts +++ b/apps/cli/src/commands/db/query/query.integration.test.ts @@ -38,12 +38,9 @@ import { Stdin } from "../../../shared/runtime/stdin.service.ts"; import { AiTool } from "../../../shared/telemetry/ai-tool.service.ts"; import { CommandCredentials } from "../../../auth/command-credentials.service.ts"; import { validateAccessToken } from "../../../auth/access-token.ts"; -import { - ProjectRefResolver, - PROJECT_NOT_LINKED_MESSAGE, -} from "../../../config/project-ref.service.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../../config/remote-target.service.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { ProjectRefReadError } from "../../../command-internal/temp-paths.ts"; +import { ProjectRefReadError } from "../../../command-internal/legacy-project-state.service.ts"; import { TelemetryOutputFormat } from "../../../telemetry/telemetry-output-format.service.ts"; import { DbConfigParseUrlError } from "../../../command-internal/db-config.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -63,6 +60,7 @@ import { import { RLS_CHECK_SQL } from "./query.advisory.ts"; import type { DbQueryFlags } from "./query.command.ts"; import { dbQuery } from "./query.handler.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LOCAL_CONN: PgConnInput = { host: "127.0.0.1", @@ -142,7 +140,7 @@ function mockTelemetryOutputFormat() { } function mockProjectRef(unlinked = false, refReadFails = false) { - // `loadProjectRef` mirrors the real preflight: an explicit `--project-ref` flag wins first, + // `RemoteTarget.load` mirrors the real preflight: an explicit `--project-ref` flag wins first, // then ErrNotLinked when absent, or a read error (ProjectRefReadError) for an unreadable file. const loadProjectRef = (flagValue: Option.Option) => Option.isSome(flagValue) && flagValue.value.length > 0 @@ -156,12 +154,12 @@ function mockProjectRef(unlinked = false, refReadFails = false) { : unlinked ? Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) : Effect.succeed(REF); - return Layer.succeed(ProjectRefResolver, { + return stubRemoteTarget({ resolve: () => Effect.succeed(REF), resolveForLink: () => Effect.succeed(REF), resolveOptional: () => Effect.succeed(unlinked ? Option.none() : Option.some(REF)), - loadProjectRef, - promptProjectRef: () => Effect.succeed(REF), + load: loadProjectRef, + prompt: () => Effect.succeed(REF), }); } diff --git a/apps/cli/src/commands/db/remote/changes/SIDE_EFFECTS.md b/apps/cli/src/commands/db/remote/changes/SIDE_EFFECTS.md index 619062b43d..106e7c4dae 100644 --- a/apps/cli/src/commands/db/remote/changes/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/remote/changes/SIDE_EFFECTS.md @@ -51,3 +51,4 @@ Not applicable. - Deprecated: use `db diff --use-migra --linked` instead. - `--schema` / `-s` restricts the diff to specific schemas. - `--db-url` and `--linked` (default true) are mutually exclusive. +- In stack mode the linked ref is resolved in TypeScript and passed to the Go command as `--project-ref`; no `supabase/.temp` file is read. diff --git a/apps/cli/src/commands/db/remote/changes/changes.handler.ts b/apps/cli/src/commands/db/remote/changes/changes.handler.ts index b7b6db5eb5..118b643562 100644 --- a/apps/cli/src/commands/db/remote/changes/changes.handler.ts +++ b/apps/cli/src/commands/db/remote/changes/changes.handler.ts @@ -1,5 +1,6 @@ import { Effect, Option } from "effect"; import { GoProxy } from "../../../../command-internal/go-proxy.service.ts"; +import { goProjectTarget } from "../../../../command-internal/go-proxy-target-env.ts"; import type { DbRemoteChangesFlags } from "./changes.command.ts"; export const dbRemoteChanges = Effect.fn("db.remote.changes")(function* ( @@ -13,5 +14,9 @@ export const dbRemoteChanges = Effect.fn("db.remote.changes")(function* ( if (Option.isSome(flags.dbUrl)) args.push("--db-url", flags.dbUrl.value); if (flags.linked) args.push("--linked"); if (Option.isSome(flags.password)) args.push("--password", flags.password.value); - yield* proxy.exec(args); + const target = Option.isNone(flags.dbUrl) + ? yield* goProjectTarget(Option.none()) + : { args: [], env: {} }; + args.push(...target.args); + yield* proxy.exec(args, { env: target.env }); }); diff --git a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md index 4a4eee0f95..5b7b0a9515 100644 --- a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md @@ -263,7 +263,7 @@ to those defaults (the usual outcome for an interactive terminal). stack, not the flag name: a `--db-url` pointing at the local stack is treated as a local reset. - **`--project-ref`** (TS-only, no Go equivalent on any user-facing `db` - command) overrides ONLY the linked-ref resolution `ProjectRefResolver` + command) overrides ONLY the linked-ref resolution `RemoteTarget` performs (flag > `SUPABASE_PROJECT_ID` > `~/.supabase//project-ref`) — unlike `SUPABASE_PROJECT_ID`, it does not affect the local container id. It never implies `--linked`: passing it with a resolved `--local`/`--db-url` diff --git a/apps/cli/src/commands/db/reset/reset.handler.ts b/apps/cli/src/commands/db/reset/reset.handler.ts index e86008eca9..ebb5e08ec6 100644 --- a/apps/cli/src/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/commands/db/reset/reset.handler.ts @@ -10,7 +10,7 @@ import { promptYesNo } from "../../../command-internal/prompt-yes-no.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../shared/output/errors.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua, yellow } from "../../../command-internal/colors.ts"; import { resolveResetSeedConfig } from "../../../command-internal/db-bootstrap/db-setup.ts"; import { resetLocalDatabase } from "../../../command-internal/db-bootstrap/reset-local-database.ts"; @@ -179,10 +179,11 @@ export const dbReset = Effect.fn("db.reset")(function* (flags: DbResetFlags) { // Loaded before the fallible linked resolution, so the post-run cache finalizer still fires // even when `resolver.resolve` fails mid-way (merged config, temp-role mint, connection). + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const refResolver = yield* ProjectRefResolver; - const ref = yield* refResolver.loadProjectRef(flags.projectRef); - yield* Ref.set(linkedRefForCache, Option.some(ref)); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + yield* Ref.set(linkedRefForCache, Option.some(linkedTarget.ref)); } const cfg = yield* resolver.resolve({ @@ -190,6 +191,7 @@ export const dbReset = Effect.fn("db.reset")(function* (flags: DbResetFlags) { connType, dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); yield* Effect.annotateCurrentSpan({ diff --git a/apps/cli/src/commands/db/reset/reset.integration.test.ts b/apps/cli/src/commands/db/reset/reset.integration.test.ts index 9d97038f6d..112f460b41 100644 --- a/apps/cli/src/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/commands/db/reset/reset.integration.test.ts @@ -62,13 +62,15 @@ import { type StubbedRequest, } from "../../../../tests/helpers/permission-real-layers.ts"; import { unusedGateway } from "../../../../tests/helpers/unused-stack.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../tests/helpers/legacy-state-guard.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { - ProjectRefResolver, - PROJECT_NOT_LINKED_MESSAGE, -} from "../../../config/project-ref.service.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../../config/remote-target.service.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DebugFlag, @@ -107,6 +109,11 @@ import { } from "../../../command-internal/db-connection.service.ts"; import { dbReset } from "./reset.handler.ts"; import type { DbResetFlags } from "./reset.command.ts"; +import { database15, selection } from "../../../../tests/helpers/pins.ts"; +import { seedLock, unbundled } from "../../../../tests/helpers/stack-lock.ts"; +import { classifyCliErrorActionability } from "../../../shared/telemetry/error-actionability.ts"; +import { runtimeInfoLayer } from "../../../shared/runtime/runtime-info.layer.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LIST_MIGRATIONS = "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; @@ -545,8 +552,8 @@ const serviceCreation = ( case "database": return { service, + pin: database15, config: { - version: "15", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(RESET_JWT), jwtExpiry: 3600, @@ -554,17 +561,33 @@ const serviceCreation = ( endpoints: {}, }; case "auth": - return { service, config: { databaseUrl, jwtSecret: RESET_JWT }, endpoints: {} }; + return { + service, + pin: selection.auth, + config: { databaseUrl, jwtSecret: RESET_JWT }, + endpoints: {}, + }; case "storage": return { service, + pin: selection.storage, config: { databaseUrl, filePath: "/tmp/storage", jwtSecret: RESET_JWT }, endpoints: {}, }; case "realtime": - return { service, config: { databaseUrl, jwtSecret: RESET_JWT }, endpoints: {} }; + return { + service, + pin: selection.realtime, + config: { databaseUrl, jwtSecret: RESET_JWT }, + endpoints: {}, + }; case "pooler": - return { service, config: { databaseUrl, jwtSecret: RESET_JWT }, endpoints: {} }; + return { + service, + pin: selection.pooler, + config: { databaseUrl, jwtSecret: RESET_JWT }, + endpoints: {}, + }; } }; @@ -846,7 +869,7 @@ function mockResetStackApi(opts: { runtime: "native" as const, instances: members.map((member) => ({ id: member.id, - creation: { service: "mail" as const, config: {} }, + creation: { service: "mail" as const, pin: selection.mail, config: {} }, })), lifetime: "detached" as const, composition: { members: [], dependencies: [] }, @@ -908,7 +931,7 @@ function setup( replicationSlotCounts?: ReadonlyArray; replicationSlotQueryFails?: boolean; failStatement?: { readonly sql: string; readonly code?: string; readonly message: string }; - // Simulates an unlinked workdir: `loadProjectRef` fails with `ProjectRefNotLinkedError` + // Simulates an unlinked workdir: `RemoteTarget.load` fails with `ProjectRefNotLinkedError` // absent an explicit `--project-ref` flag, instead of falling back to `opts.ref`. linkedFails?: boolean; stackBackend?: boolean; @@ -1011,6 +1034,7 @@ function setup( }), ); const projectHttp = recordingProjectHttpClient(VALID_REF); + const recorder = recordingFileSystem(); const layer = Layer.mergeAll( out.layer, conn.layer, @@ -1023,6 +1047,7 @@ function setup( : resolver.layer, mockCommandSettings({ workdir }), BunServices.layer, + recorder.layer, child.layer, mockLocalDockerEngineUnavailableLayer, mockRuntimeInfo({ platform: "linux" }), @@ -1035,19 +1060,19 @@ function setup( // real pipe for the non-interactive bucket-seed prompts instead. mockTty({ stdinIsTty: opts.stdinIsTty ?? true }), mockStdin(opts.stdinIsTty ?? true, opts.pipedStdin), - // `loadProjectRef` gives an explicit `--project-ref` flag top precedence, mirrored here so a + // `RemoteTarget.load` gives an explicit `--project-ref` flag top precedence, mirrored here so a // test can prove the flag (not just `opts.ref`) drives the linked ref. - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.succeed(opts.ref ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.ref ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.ref ?? VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true ? Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) : Effect.succeed(opts.ref ?? VALID_REF), - promptProjectRef: () => Effect.succeed(opts.ref ?? VALID_REF), + prompt: () => Effect.succeed(opts.ref ?? VALID_REF), }), Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(platformApi.layer)), @@ -1080,6 +1105,7 @@ function setup( catalogApplied: catalog?.applied ?? [], requests, projectHttp, + accessed: recorder.accessed, }; } @@ -1282,6 +1308,98 @@ describe("db reset", () => { }); }); + it.effect("a stack reset never reads or writes supabase/.temp", () => { + seedLegacyStateDecoys(tmp.current); + const { layer, accessed } = setup(tmp.current, { + toml: 'project_id = "test"\n', + args: ["db", "reset", "--local"], + isLocal: true, + stackBackend: true, + }); + return Effect.gen(function* () { + yield* dbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + expect(accessed.some((entry) => entry.endsWith("config.toml"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + }); + }); + + it.effect("runs the catalog one-shots of a postgres-only stack on the lock's pins", () => { + const toml = 'project_id = "test"\n'; + const lockedAuth = unbundled(selection.auth); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(tmp.current, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(tmp.current, "supabase", "config.toml"), toml); + yield* seedLock(tmp.current, (services) => ({ + ...services, + auth: { ...services.auth, resolved: lockedAuth }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: tmp.current, supabaseHome: tmp.current }), + ), + ), + ); + const { layer, catalogApplied } = setup(tmp.current, { + toml, + args: ["db", "reset", "--local"], + isLocal: true, + stackBackend: true, + stackPostgresOnly: true, + }); + + yield* dbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + + expect(catalogApplied[0]?.target.selection.auth).toEqual(lockedAuth); + }).pipe(Effect.provide(BunServices.layer)); + }); + + it.effect( + "tells a stale lock to run supabase start and classifies it as user-actionable", + () => { + const toml = 'project_id = "test"\n'; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(tmp.current, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(tmp.current, "supabase", "config.toml"), toml); + yield* seedLock(tmp.current, (services) => ({ + ...services, + rest: { ...services.rest, specifier: "v99.0.0" }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: tmp.current, supabaseHome: tmp.current }), + ), + ), + ); + const { layer } = setup(tmp.current, { + toml, + args: ["db", "reset", "--local"], + isLocal: true, + stackBackend: true, + }); + + const error = yield* dbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer), Effect.flip); + + expect(error).toMatchObject({ + message: expect.stringContaining( + "rest (specifier linked ≠ locked v99.0.0); run supabase start to refresh it", + ), + }); + expect(error.message).not.toContain("commit"); + expect(classifyCliErrorActionability(error)).toMatchObject({ + error_kind: "user_actionable", + suggested_command: "supabase start", + error_fingerprint: "tag:ResetLocalDbFailedError:conflict", + }); + }).pipe(Effect.provide(BunServices.layer)); + }, + ); + it.effect("preserves the existing stack composition while rebuilding the database", () => { const { layer, catalogApplied } = setup(tmp.current, { toml: 'project_id = "test"\n', diff --git a/apps/cli/src/commands/db/reset/reset.layers.ts b/apps/cli/src/commands/db/reset/reset.layers.ts index 5de6afff43..9fc5a325e8 100644 --- a/apps/cli/src/commands/db/reset/reset.layers.ts +++ b/apps/cli/src/commands/db/reset/reset.layers.ts @@ -6,7 +6,7 @@ import { commandCredentialsLayer } from "../../../auth/command-credentials.layer import { httpClientLayer } from "../../../auth/http-debug.layer.ts"; import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-api-factory.layer.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; import { debugLoggerLayer } from "../../../command-internal/debug-logger.layer.ts"; @@ -41,7 +41,7 @@ const platformApiFactory = commandPlatformApiFactoryLayer.pipe( Layer.provide(identityStitchLayer), ); -const projectRef = projectRefLayer.pipe( +const projectRef = remoteTargetLayer.pipe( Layer.provide(platformApiFactory), Layer.provide(cliSettings), ); diff --git a/apps/cli/src/commands/db/reset/reset.stack.e2e.test.ts b/apps/cli/src/commands/db/reset/reset.stack.e2e.test.ts index b06a5eda4b..718996241e 100644 --- a/apps/cli/src/commands/db/reset/reset.stack.e2e.test.ts +++ b/apps/cli/src/commands/db/reset/reset.stack.e2e.test.ts @@ -7,6 +7,7 @@ import { create as createStack } from "@supabase/stack/effect"; import { runSupabaseEffect } from "../../../../tests/helpers/cli.ts"; import { destroyTestStack } from "../../../../tests/helpers/stack-cleanup.ts"; import { stackArtifactCacheRoot } from "../../../../tests/helpers/stack-artifacts.ts"; +import { selection } from "../../../../tests/helpers/pins.ts"; const COMMAND_TIMEOUT_MS = 8 * 60_000; const TEST_TIMEOUT_MS = COMMAND_TIMEOUT_MS + 2 * 60_000; @@ -124,8 +125,8 @@ const composeStack = Effect.fn("DbResetStackE2e.composeStack")(function* ( yield* stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(JWT_SECRET), jwtExpiry: 3600, @@ -134,6 +135,7 @@ const composeStack = Effect.fn("DbResetStackE2e.composeStack")(function* ( }, { service: "rest", + pin: selection.rest, config: { jwtSecret: JWT_SECRET, }, @@ -141,6 +143,7 @@ const composeStack = Effect.fn("DbResetStackE2e.composeStack")(function* ( }, { service: "auth", + pin: selection.auth, config: { jwtSecret: JWT_SECRET, }, diff --git a/apps/cli/src/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts b/apps/cli/src/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts index b0330f6ac6..a5a5c04bfa 100644 --- a/apps/cli/src/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts +++ b/apps/cli/src/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts @@ -41,7 +41,6 @@ const toml: DbTomlValues = { port: 54322, shadowPort: 54320, password: "postgres", - poolerConnectionString: Option.none(), projectId: Option.none(), majorVersion: 17, orioledbVersion: Option.none(), diff --git a/apps/cli/src/commands/db/schema/declarative/declarative.smart-target.ts b/apps/cli/src/commands/db/schema/declarative/declarative.smart-target.ts index 413a99378b..55a9e16f6e 100644 --- a/apps/cli/src/commands/db/schema/declarative/declarative.smart-target.ts +++ b/apps/cli/src/commands/db/schema/declarative/declarative.smart-target.ts @@ -7,7 +7,10 @@ import { import { promptYesNo } from "../../../../command-internal/prompt-yes-no.ts"; import { Output } from "../../../../shared/output/output.service.ts"; import { resetLocalDatabase } from "../../../../command-internal/db-bootstrap/reset-local-database.ts"; -import { PROJECT_REF_PATTERN } from "../../../../config/project-ref.service.ts"; +import { + PROJECT_REF_PATTERN, + type ResolvedRemoteTarget, +} from "../../../../config/remote-target.service.ts"; import { currentStackBackend } from "../../../../command-internal/stack-backend.ts"; import { DbConfigResolver } from "../../../../command-internal/db-config.service.ts"; import { loadProjectEnv } from "../../../../command-internal/db-config.toml-read.ts"; @@ -91,8 +94,14 @@ export const resolveLocalTargetEndpoint = Effect.fnUntraced(function* ( } satisfies PgDeltaDatabaseEndpoint; }); -/** Resolves a remote target without discarding TLS and connection options. */ -export const resolveRemoteEndpoint = Effect.fnUntraced(function* (flags: SmartTargetFlags) { +/** + * Resolves a remote target without discarding TLS and connection options. A caller that already + * resolved the linked target passes it, so the connection does not resolve it again. + */ +export const resolveRemoteEndpoint = Effect.fnUntraced(function* ( + flags: SmartTargetFlags, + target?: ResolvedRemoteTarget, +) { const resolver = yield* DbConfigResolver; const dnsResolver = yield* DnsResolverFlag; const resolved = yield* resolver.resolve({ @@ -100,6 +109,7 @@ export const resolveRemoteEndpoint = Effect.fnUntraced(function* (flags: SmartTa connType: Option.isSome(flags.dbUrl) ? "db-url" : "linked", dnsResolver, password: flags.password, + ...(target === undefined ? {} : { target }), }); return { kind: "database", @@ -121,7 +131,7 @@ export const resolveSmartTargetEndpoint = Effect.fn("DeclarativeSchema.smartTarg fs: FileSystem.FileSystem, path: Path.Path, workdir: string, - linkedRef: Option.Option, + linkedTarget: Option.Option, beforeLocalTarget: Effect.Effect = Effect.void, ) { if (!hasMigrations) { @@ -139,15 +149,16 @@ export const resolveSmartTargetEndpoint = Effect.fn("DeclarativeSchema.smartTarg const yes = yield* resolveYesWithProjectEnv(projectEnv); // Inserts "Linked project" between local and custom when the workdir is linked with a valid // ref; an invalid on-disk ref hides the choice rather than showing it and failing later. - const showLinked = Option.isSome(linkedRef) && PROJECT_REF_PATTERN.test(linkedRef.value); + const showLinked = + Option.isSome(linkedTarget) && PROJECT_REF_PATTERN.test(linkedTarget.value.ref); const choice = yield* output.promptSelect("Generate declarative schema from:", [ { value: "local", label: "Local database", hint: "generate from local Postgres" }, - ...(showLinked && Option.isSome(linkedRef) + ...(showLinked && Option.isSome(linkedTarget) ? [ { value: "linked", label: "Linked project", - hint: `generate from remote linked project (${linkedRef.value})`, + hint: `generate from remote linked project (${linkedTarget.value.ref})`, }, ] : []), @@ -157,7 +168,10 @@ export const resolveSmartTargetEndpoint = Effect.fn("DeclarativeSchema.smartTarg if (choice === "linked") { // Same path as an explicit `--linked`: login-role mint + pooler fallback, then the resolved URL. - return yield* resolveRemoteEndpoint({ ...flags, linked: Option.some(true) }); + return yield* resolveRemoteEndpoint( + { ...flags, linked: Option.some(true) }, + Option.getOrUndefined(linkedTarget), + ); } if (choice === "custom") { diff --git a/apps/cli/src/commands/db/schema/declarative/generate/generate.handler.ts b/apps/cli/src/commands/db/schema/declarative/generate/generate.handler.ts index 229e3603d2..202b4705cb 100644 --- a/apps/cli/src/commands/db/schema/declarative/generate/generate.handler.ts +++ b/apps/cli/src/commands/db/schema/declarative/generate/generate.handler.ts @@ -9,8 +9,11 @@ import { promptYesNo } from "../../../../../command-internal/prompt-yes-no.ts"; import { Output } from "../../../../../shared/output/output.service.ts"; import { Tty } from "../../../../../shared/runtime/tty.service.ts"; import { CommandSettings } from "../../../../../config/command-settings.service.ts"; +import { + RemoteTarget, + type ResolvedRemoteTarget, +} from "../../../../../config/remote-target.service.ts"; import { bold } from "../../../../../command-internal/colors.ts"; -import { readProjectRefFile } from "../../../../../command-internal/temp-paths.ts"; import { loadProjectEnv, readDbToml, @@ -61,6 +64,7 @@ export const dbSchemaDeclarativeGenerate = Effect.fn("db.schema.declarative.gene const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; const linkedProjectCache = yield* LinkedProjectCache; + const remoteTarget = yield* RemoteTarget; const dnsResolver = yield* DnsResolverFlag; // The project env is loaded and resolved before the gate below, so a `SUPABASE_EXPERIMENTAL` // set only in `supabase/.env` opens the gate too. @@ -102,17 +106,14 @@ export const dbSchemaDeclarativeGenerate = Effect.fn("db.schema.declarative.gene // block overrides `experimental.pgdelta.*` downstream only, not the gate above. Smart-mode's // "Linked project" choice does not re-load, so only `flags.linked` triggers this. let toml = baseToml; + let explicitLinkedTarget: ResolvedRemoteTarget | undefined; // The resolved linked ref (explicit `--linked` only) is threaded into the raw-shadow export // source, so its platform setup uses the remote-merged config, and into the post-run // linked-project cache finalizer below. if (Option.isSome(flags.linked)) { - const linkedRef = Option.isSome(cliSettings.projectId) - ? cliSettings.projectId - : yield* readProjectRefFile(fs, path, cliSettings.workdir); - if (Option.isSome(linkedRef)) { - yield* Ref.set(linkedProjectRef, linkedRef.value); - toml = yield* readDbToml(fs, path, cliSettings.workdir, linkedRef.value); - } + explicitLinkedTarget = yield* remoteTarget.load(Option.none()); + yield* Ref.set(linkedProjectRef, explicitLinkedTarget.ref); + toml = yield* readDbToml(fs, path, cliSettings.workdir, explicitLinkedTarget.ref); } // Preserve the selected value for user-facing output: invocation-local `--output` wins, @@ -186,7 +187,7 @@ export const dbSchemaDeclarativeGenerate = Effect.fn("db.schema.declarative.gene } target = yield* resolveLocalTargetEndpoint(local, dnsResolver); } else { - target = yield* resolveRemoteEndpoint(flags); + target = yield* resolveRemoteEndpoint(flags, explicitLinkedTarget); } overwrite = flags.overwrite; } else { @@ -212,21 +213,17 @@ export const dbSchemaDeclarativeGenerate = Effect.fn("db.schema.declarative.gene } } const hasMigrations = yield* hasMigrationFiles(fs, path, migrationsDir); - // Only when migrations exist, resolve the ref (config `project_id` → `.temp/project-ref`) - // and record it for the post-run cache finalizer, so smart generate in a linked workdir - // caches the project regardless of which target the user picks. - let linkedRef = Option.none(); + // Only when migrations exist, resolve the target (env, then link) and record its ref for + // the post-run cache finalizer, so smart generate in a linked workdir caches the project + // regardless of which target the user picks. + let linkedTarget = Option.none(); if (hasMigrations) { - // Only decides whether to offer the linked choice, so swallow a broken - // `.temp/project-ref` here (omit the choice) rather than aborting; the explicit - // `--linked` branch above still propagates a real failure. - linkedRef = Option.isSome(cliSettings.projectId) - ? cliSettings.projectId - : yield* readProjectRefFile(fs, path, cliSettings.workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - ); - if (Option.isSome(linkedRef)) { - yield* Ref.set(linkedProjectRef, linkedRef.value); + // Only decides whether to offer the linked choice, so a broken link omits the choice + // rather than aborting; the explicit `--linked` branch above still propagates a real + // failure. + linkedTarget = yield* remoteTarget.resolveOptional(Option.none()); + if (Option.isSome(linkedTarget)) { + yield* Ref.set(linkedProjectRef, linkedTarget.value.ref); } } target = yield* resolveSmartTargetEndpoint( @@ -236,7 +233,7 @@ export const dbSchemaDeclarativeGenerate = Effect.fn("db.schema.declarative.gene fs, path, cliSettings.workdir, - linkedRef, + linkedTarget, (yield* DeclarativeSeam).ensureLocalPostgresImageCurrent, ); overwrite = true; diff --git a/apps/cli/src/commands/db/schema/declarative/generate/generate.integration.test.ts b/apps/cli/src/commands/db/schema/declarative/generate/generate.integration.test.ts index c2455eb972..102ab7a744 100644 --- a/apps/cli/src/commands/db/schema/declarative/generate/generate.integration.test.ts +++ b/apps/cli/src/commands/db/schema/declarative/generate/generate.integration.test.ts @@ -67,6 +67,7 @@ import { import { GoProxy } from "../../../../../command-internal/go-proxy.service.ts"; import { CommandPlatformApi } from "../../../../../auth/command-platform-api.service.ts"; import { CommandPlatformApiFactory } from "../../../../../auth/command-platform-api-factory.service.ts"; +import { remoteTargetLayer as realRemoteTargetLayer } from "../../../../../config/remote-target.layer.ts"; import { dockerRunLayer } from "../../../../../command-internal/docker-run.layer.ts"; import { stackBackendLayer } from "../../../../../command-internal/stack-backend.ts"; import { StackApi } from "../../../../../command-internal/stack-api.ts"; @@ -81,6 +82,7 @@ import { DeclarativeSeam } from "../../../shared/pgdelta.seam.service.ts"; import { DeclarativeNotEnabledError } from "../declarative.errors.ts"; import type { DbSchemaDeclarativeGenerateFlags } from "./generate.command.ts"; import { dbSchemaDeclarativeGenerate } from "./generate.handler.ts"; +import { selection } from "../../../../../../tests/helpers/pins.ts"; interface SetupOpts { experimental?: boolean; @@ -139,8 +141,8 @@ function generateStackApi(workdir: string) { id: "primary", config: { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make("secret"), jwtExpiry: 3600, @@ -341,6 +343,24 @@ function setup(workdir: string, opts: SetupOpts = {}) { Layer.provide(processControl.layer), ); const projectHttp = recordingProjectHttpClient(VALID_REF); + const settings = mockCommandSettings({ + workdir, + projectId: opts.projectId ?? Option.some("test"), + }); + const tty = mockTty({ stdinIsTty: opts.stdinIsTty ?? false, stdoutIsTty: false }); + const remoteTarget = realRemoteTargetLayer.pipe( + Layer.provide( + Layer.mergeAll( + settings, + tty, + out.layer, + Layer.succeed(CommandPlatformApiFactory, { + make: CommandPlatformApi.pipe(Effect.provide(platformApi.layer)), + }), + BunServices.layer, + ), + ), + ); const layer = Layer.mergeAll( out.layer, telemetry.layer, @@ -362,8 +382,9 @@ function setup(workdir: string, opts: SetupOpts = {}) { : resolver, proxy, dbConn, - mockCommandSettings({ workdir, projectId: opts.projectId ?? Option.some("test") }), - mockTty({ stdinIsTty: opts.stdinIsTty ?? false, stdoutIsTty: false }), + settings, + tty, + remoteTarget, mockStdin(opts.stdinIsTty ?? false), experimentalFlag, cliArgs, @@ -851,7 +872,11 @@ describe("db schema declarative generate integration", () => { it.effect("--linked=false is an explicit linked target (Go gates on flag.Changed)", () => { // Non-interactive (no TTY, no --yes), so a smart-mode fall-through would fail with "specify // a target" instead. - const s = setup(tmp.current, { experimental: true, stdinIsTty: false }); + const s = setup(tmp.current, { + experimental: true, + stdinIsTty: false, + projectId: Option.some("abcdefghijklmnopqrst"), + }); return Effect.gen(function* () { const exit = yield* Effect.exit( dbSchemaDeclarativeGenerate(flags({ linked: Option.some(false) })), diff --git a/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md b/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md index 55188fe177..4d47b6e92d 100644 --- a/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md @@ -15,7 +15,7 @@ cache described below; `--no-cache` bypasses it as well as the legacy backend ca Pg-delta runs in-process and uses two scoped shadow databases. Coverage gaps warn; `--strict-coverage` makes them fatal, while `PGDELTA_DEBUG` writes diagnostic JSON under -`supabase/.temp/pgdelta/v2/debug//`. The engine may emit ordered +`supabase/.temp/pgdelta/v2/debug//` (`/tmp/pgdelta/v2/debug//` in stack mode). The engine may emit ordered transaction-aware files; applicable, convergent SQL is the contract. `--no-cache` bypasses the engine's shadow baseline cache. The bundled formatter defaults to lowercase SQL @@ -129,7 +129,7 @@ existing SQL or creates an export manifest. codes (a non-interactive run still fails with "no declarative schema found"). - The migration apply is native (connects to the local DB and records migration history). On apply failure a debug bundle is written under - `supabase/.temp/pgdelta/debug/` and, in a TTY, a reset-and-reapply is offered + `supabase/.temp/pgdelta/debug/` (`/tmp/pgdelta/debug/` in stack mode) and, in a TTY, a reset-and-reapply is offered (the reset itself is native too — `resetLocalDatabase` — run in-process, sharing this command's own telemetry/linked-project-cache finalizer cycle rather than firing a second one from a child process). diff --git a/apps/cli/src/commands/db/schema/declarative/sync/sync.handler.ts b/apps/cli/src/commands/db/schema/declarative/sync/sync.handler.ts index 9f91ea660c..d4ab199c13 100644 --- a/apps/cli/src/commands/db/schema/declarative/sync/sync.handler.ts +++ b/apps/cli/src/commands/db/schema/declarative/sync/sync.handler.ts @@ -1,3 +1,4 @@ +import { pgdeltaScratchDir } from "../../../../../command-internal/scratch-paths.ts"; import { Clock, DateTime, Effect, FileSystem, Option, Path, Ref, Result } from "effect"; import { withSingleOrioleDBVersionEnvWarning } from "../../../../../command-internal/local-config-values.ts"; @@ -24,11 +25,13 @@ import { import { makeDir } from "../../../../../command-internal/make-dir.ts"; import { applyMigrationFile } from "../../../../../command-internal/migration-apply.ts"; import { ENABLE_LOCAL_WEBHOOKS_SUGGESTION } from "../../../../../command-internal/pg-net-guidance.ts"; -import { readProjectRefFile } from "../../../../../command-internal/temp-paths.ts"; +import { + RemoteTarget, + type ResolvedRemoteTarget, +} from "../../../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../../telemetry/telemetry-state.service.ts"; import { listLocalMigrations } from "../../../../../command-internal/migration-list.ts"; -import { pgDeltaTempPath } from "../../../../../command-internal/pgdelta.paths.ts"; import { isPgDeltaDebugEnabled, resolvePgDeltaProjectId, @@ -95,6 +98,7 @@ const runDbSchemaDeclarativeSync = Effect.fn("db.schema.declarative.sync")(funct const path = yield* Path.Path; const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; + const remoteTarget = yield* RemoteTarget; // The project env is loaded and resolved before the gate below, so a `SUPABASE_EXPERIMENTAL` // set only in `supabase/.env` opens the gate too. const projectEnv = yield* loadProjectEnv(fs, path, cliSettings.workdir); @@ -141,7 +145,7 @@ const runDbSchemaDeclarativeSync = Effect.fn("db.schema.declarative.sync")(funct // a bare `extension.sql` is ambiguous in a tree with nested schema folders. const extensionSqlRel = path.join(declarativeDirRel, "extension.sql"); const migrationsDir = path.join(cliSettings.workdir, "supabase", "migrations"); - const tempDir = pgDeltaTempPath(path, cliSettings.workdir); + const tempDir = yield* pgdeltaScratchDir(cliSettings.workdir); const run: DeclarativeRunContext = { pgDelta: { // `resolvePgDeltaProjectId` resolves `SUPABASE_PROJECT_ID` env → config.toml's @@ -205,21 +209,15 @@ const runDbSchemaDeclarativeSync = Effect.fn("db.schema.declarative.sync")(funct (yield* listLocalMigrations(fs, path, migrationsDir).pipe( Effect.orElseSucceed((): ReadonlyArray => []), )).length > 0; - // Only when migrations exist, resolve the ref (config `project_id` → `.temp/project-ref`) - // and record it for the finalizer, so a linked-workdir bootstrap caches regardless of the - // chosen target. - let linkedRef = Option.none(); + // Only when migrations exist, resolve the target (env, then link) and record its ref for + // the finalizer, so a linked-workdir bootstrap caches regardless of the chosen target. + let linkedTarget = Option.none(); if (hasMigrations) { - // Only decides whether to offer the linked choice, so swallow a broken - // `.temp/project-ref` here; `linkedProjectRef` then stays unset so the post-run cache - // correctly does not fire. - linkedRef = Option.isSome(cliSettings.projectId) - ? cliSettings.projectId - : yield* readProjectRefFile(fs, path, cliSettings.workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - ); - if (Option.isSome(linkedRef)) { - yield* Ref.set(linkedProjectRef, linkedRef.value); + // Only decides whether to offer the linked choice, so a broken link omits the choice; + // `linkedProjectRef` then stays unset so the post-run cache correctly does not fire. + linkedTarget = yield* remoteTarget.resolveOptional(Option.none()); + if (Option.isSome(linkedTarget)) { + yield* Ref.set(linkedProjectRef, linkedTarget.value.ref); } } // sync has no target flags, so reset stays interactive (the prompt fires under the local @@ -231,7 +229,7 @@ const runDbSchemaDeclarativeSync = Effect.fn("db.schema.declarative.sync")(funct fs, path, cliSettings.workdir, - linkedRef, + linkedTarget, ensureLocalPostgresImageCurrent, ); const generated = yield* generateDeclarativeOutput(run, target); diff --git a/apps/cli/src/commands/db/schema/declarative/sync/sync.integration.test.ts b/apps/cli/src/commands/db/schema/declarative/sync/sync.integration.test.ts index 135d0acd27..80da2461ef 100644 --- a/apps/cli/src/commands/db/schema/declarative/sync/sync.integration.test.ts +++ b/apps/cli/src/commands/db/schema/declarative/sync/sync.integration.test.ts @@ -39,6 +39,11 @@ import { type StubbedRequest, } from "../../../../../../tests/helpers/permission-real-layers.ts"; import { unusedGateway } from "../../../../../../tests/helpers/unused-stack.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../../../tests/helpers/legacy-state-guard.ts"; import { alwaysReadyHttpClientLayer, defaultLocalResetRoute, @@ -76,6 +81,7 @@ import { } from "../../../../../command-internal/global-flags.ts"; import { CommandPlatformApi } from "../../../../../auth/command-platform-api.service.ts"; import { CommandPlatformApiFactory } from "../../../../../auth/command-platform-api-factory.service.ts"; +import { remoteTargetLayer as realRemoteTargetLayer } from "../../../../../config/remote-target.layer.ts"; import { dockerRunLayer } from "../../../../../command-internal/docker-run.layer.ts"; import { stackBackendLayer } from "../../../../../command-internal/stack-backend.ts"; import { StackApi } from "../../../../../command-internal/stack-api.ts"; @@ -98,6 +104,7 @@ import { DeclarativeSeam } from "../../../shared/pgdelta.seam.service.ts"; import { DeclarativeNotEnabledError } from "../declarative.errors.ts"; import type { DbSchemaDeclarativeSyncFlags } from "./sync.command.ts"; import { dbSchemaDeclarativeSync } from "./sync.handler.ts"; +import { selection } from "../../../../../../tests/helpers/pins.ts"; interface SetupOpts { experimental?: boolean; @@ -152,8 +159,8 @@ function syncStackApi(workdir: string, port: number) { id: "primary", config: { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make("secret"), jwtExpiry: 3600, @@ -384,6 +391,25 @@ function setup(workdir: string, opts: SetupOpts = {}) { }), ); const projectHttp = recordingProjectHttpClient(VALID_REF); + const settings = mockCommandSettings({ + workdir, + projectId: opts.projectId ?? Option.some("test"), + }); + const tty = mockTty({ stdinIsTty: opts.stdinIsTty ?? false, stdoutIsTty: false }); + const remoteTarget = realRemoteTargetLayer.pipe( + Layer.provide( + Layer.mergeAll( + settings, + tty, + out.layer, + Layer.succeed(CommandPlatformApiFactory, { + make: CommandPlatformApi.pipe(Effect.provide(platformApi.layer)), + }), + BunServices.layer, + ), + ), + ); + const recorder = recordingFileSystem(); const layer = Layer.mergeAll( out.layer, telemetry.layer, @@ -404,8 +430,9 @@ function setup(workdir: string, opts: SetupOpts = {}) { env: { SUPABASE_ACCESS_TOKEN: VALID_TOKEN }, }) : resolver, - mockCommandSettings({ workdir, projectId: opts.projectId ?? Option.some("test") }), - mockTty({ stdinIsTty: opts.stdinIsTty ?? false, stdoutIsTty: false }), + settings, + tty, + remoteTarget, mockStdin(opts.stdinIsTty ?? false), experimentalFlag, cliArgs, @@ -419,6 +446,7 @@ function setup(workdir: string, opts: SetupOpts = {}) { make: CommandPlatformApi.pipe(Effect.provide(platformApi.layer)), }), BunServices.layer, + recorder.layer, // `child.layer` must be listed after `BunServices.layer` — `Layer.mergeAll` resolves a // duplicate service tag to whichever layer is listed last, so this mock overrides Bun's // real `ChildProcessSpawner` instead of the reverse. @@ -434,6 +462,7 @@ function setup(workdir: string, opts: SetupOpts = {}) { ); return { layer, + accessed: recorder.accessed, out, child, dbExec, @@ -1049,6 +1078,33 @@ describe("db schema declarative sync integration", () => { }).pipe(Effect.provide(s.layer)); }); + it.effect( + "on the stack backend a failed apply saves its debug bundle outside supabase/.temp", + () => { + const s = setup(tmp.current, { + experimental: true, + diffSql: "ALTER TABLE a ADD COLUMN b int;\n", + stackBackend: true, + applyFails: true, + }); + seedLegacyStateDecoys(tmp.current); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* seedDeclarative(tmp.current); + const error = yield* dbSchemaDeclarativeSync(flags({ apply: Option.some(true) })).pipe( + Effect.flip, + ); + expect(error._tag).toBe("DeclarativeApplyError"); + const pgdeltaRoot = path.join(tmp.current, ".supabase", "tmp", "pgdelta"); + const projectRoots = yield* fs.readDirectory(pgdeltaRoot); + expect(projectRoots).toHaveLength(1); + expect(yield* fs.exists(path.join(pgdeltaRoot, projectRoots[0] ?? "", "debug"))).toBe(true); + expect(legacyStateAccesses(s.accessed)).toEqual([]); + }).pipe(Effect.provide(s.layer)); + }, + ); + it.effect("refuses a known implicit-extension load failure under --yes", () => { const s = setup(tmp.current, { yes: true, diff --git a/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.integration.test.ts b/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.integration.test.ts index 75ade774b2..8295649669 100644 --- a/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.integration.test.ts +++ b/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.integration.test.ts @@ -2,6 +2,7 @@ import * as BunServices from "@effect/platform-bun/BunServices"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Layer, Option } from "effect"; +import { mockCommandSettings } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput } from "../../../../tests/helpers/mocks.ts"; import { DebugLogger } from "../../../command-internal/debug-logger.service.ts"; import { DeclarativeShadowDbError } from "./pgdelta.errors.ts"; @@ -31,7 +32,6 @@ const toml: DbTomlValues = { port: 54322, shadowPort: 54320, password: "postgres", - poolerConnectionString: Option.none(), projectId: Option.none(), majorVersion: 17, orioledbVersion: Option.none(), @@ -99,6 +99,7 @@ function setup() { unusedAdapter, debug, mockOutput().layer, + mockCommandSettings({ workdir: "/unused-pgdelta-next-workdir" }), ); return { state, diff --git a/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.ts b/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.ts index 6815739a1f..061360769c 100644 --- a/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.ts +++ b/apps/cli/src/commands/db/shared/pgdelta-engine.next.layer.ts @@ -1,6 +1,7 @@ -import { Clock, Effect, FileSystem, Layer, Path, Ref } from "effect"; +import { Clock, Crypto, Effect, FileSystem, Layer, Path, Ref } from "effect"; import type { Pool } from "pg"; +import { CommandSettings } from "../../../config/command-settings.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { layeredParseEnv, @@ -31,6 +32,7 @@ import { savePgDeltaNextDebugArtifacts, type PgDeltaNextDebugArtifacts, } from "./pgdelta-next-artifacts.ts"; +import { pgdeltaScratchDir } from "../../../command-internal/scratch-paths.ts"; import { pgDeltaNextDiagnosticReport, reportPgDeltaNextDiagnostics, @@ -132,7 +134,9 @@ export const pgDeltaNextEngineLayer = Layer.effect( const shadowService = yield* PgDeltaNextShadow; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; + const crypto = yield* Crypto.Crypto; const debugLogger = yield* DebugLogger; + const settings = yield* CommandSettings; const output = yield* Output; const feedbackInvitationShown = yield* Ref.make(false); @@ -146,7 +150,10 @@ export const pgDeltaNextEngineLayer = Layer.effect( const debugDir = yield* savePgDeltaNextDebugArtifacts( fs, path, - workdir, + yield* pgdeltaScratchDir(workdir).pipe( + Effect.provideService(CommandSettings, settings), + Effect.provideService(Crypto.Crypto, crypto), + ), id, operation, artifacts, diff --git a/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.ts b/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.ts index 14cc4105e1..ea88c78669 100644 --- a/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.ts +++ b/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.ts @@ -1,6 +1,5 @@ import { DateTime, Effect, type FileSystem, type Path, Schema } from "effect"; -import { pgDeltaTempPath } from "../../../command-internal/pgdelta.paths.ts"; import type { PgDeltaNextDiagnostic, PgDeltaNextOperation, @@ -14,8 +13,8 @@ export interface PgDeltaNextDebugArtifacts { } /** Explicit cache/artifact generation for the bundled pg-delta implementation. */ -export function pgDeltaNextTempPath(path: Path.Path, workdir: string): string { - return path.join(pgDeltaTempPath(path, workdir), "v2"); +export function pgDeltaNextTempPath(path: Path.Path, pgdeltaDir: string): string { + return path.join(pgdeltaDir, "v2"); } const PrettyJsonString = Schema.fromJsonString(Schema.Unknown, { space: 2 }); @@ -42,12 +41,12 @@ interface PgDeltaNextArtifactMetadata { export const savePgDeltaNextDebugArtifacts = Effect.fn("PgDeltaNextArtifacts.save")(function* ( fs: FileSystem.FileSystem, path: Path.Path, - workdir: string, + pgdeltaDir: string, id: string, operation: PgDeltaNextOperation, artifacts: PgDeltaNextDebugArtifacts, ) { - const debugDir = path.join(pgDeltaNextTempPath(path, workdir), "debug", id); + const debugDir = path.join(pgDeltaNextTempPath(path, pgdeltaDir), "debug", id); yield* fs.makeDirectory(debugDir, { recursive: true }); const files: Array = []; diff --git a/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.unit.test.ts b/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.unit.test.ts index f4d1a8dae7..c90ba8601a 100644 --- a/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.unit.test.ts +++ b/apps/cli/src/commands/db/shared/pgdelta-next-artifacts.unit.test.ts @@ -7,7 +7,7 @@ import { pgDeltaNextTempPath, savePgDeltaNextDebugArtifacts, } from "./pgdelta-next-artifacts.ts"; -import { pgDeltaTempPath } from "../../../command-internal/pgdelta.paths.ts"; +import { legacyProjectStatePaths } from "../../../command-internal/legacy-project-state.paths.ts"; const UnknownFromJsonString = Schema.fromJsonString(Schema.Unknown); @@ -18,7 +18,8 @@ describe("pg-delta next artifact generation", () => { const path = yield* Path.Path; const root = yield* fs.makeTempDirectoryScoped({ prefix: "pgdelta-next-artifacts-" }); const debugId = formatPgDeltaNextDebugId(Date.UTC(2024, 0, 2, 3, 4, 5, 678), "diff"); - const debugDir = yield* savePgDeltaNextDebugArtifacts(fs, path, root, debugId, "diff", { + const pgdeltaDir = legacyProjectStatePaths(path, root).pgdeltaDir; + const debugDir = yield* savePgDeltaNextDebugArtifacts(fs, path, pgdeltaDir, debugId, "diff", { sourceSnapshot: '{"source":true}\n', desiredSnapshot: '{"desired":true}\n', plan: '{"plan":true}\n', @@ -30,8 +31,8 @@ describe("pg-delta next artifact generation", () => { .pipe(Effect.flatMap(Schema.decodeEffect(UnknownFromJsonString))); expect(debugId).toBe("20240102-030405-678-diff"); - expect(pgDeltaNextTempPath(path, root)).not.toBe(pgDeltaTempPath(path, root)); - expect(debugDir).toBe(path.join(pgDeltaNextTempPath(path, root), "debug", debugId)); + expect(pgDeltaNextTempPath(path, pgdeltaDir)).not.toBe(pgdeltaDir); + expect(debugDir).toBe(path.join(pgDeltaNextTempPath(path, pgdeltaDir), "debug", debugId)); expect(yield* readJson("metadata.json")).toEqual({ version: 1, generation: "v2", diff --git a/apps/cli/src/commands/db/shared/pgdelta-next-shadow.stack.integration.test.ts b/apps/cli/src/commands/db/shared/pgdelta-next-shadow.stack.integration.test.ts index 6351fcd3b0..216cd484fe 100644 --- a/apps/cli/src/commands/db/shared/pgdelta-next-shadow.stack.integration.test.ts +++ b/apps/cli/src/commands/db/shared/pgdelta-next-shadow.stack.integration.test.ts @@ -30,7 +30,6 @@ const toml = { port: 54321, shadowPort: 54320, password: "postgres", - poolerConnectionString: Option.none(), projectId: Option.some("test"), majorVersion: 17, orioledbVersion: Option.none(), diff --git a/apps/cli/src/commands/db/start/SIDE_EFFECTS.md b/apps/cli/src/commands/db/start/SIDE_EFFECTS.md index bd093e1e8e..10b9290306 100644 --- a/apps/cli/src/commands/db/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/start/SIDE_EFFECTS.md @@ -195,6 +195,17 @@ service identities. Postgres-only first create skips analytics and pooler artifact downloads. If first initialization fails after the database is created, the new database is destroyed so the command can be retried after fixing the cause. If cleanup itself fails, stderr identifies the incomplete instance and recommends `supabase stack destroy`. +Resuming a stopped database whose saved pin differs from the selected one in anything but a +slim-services revision rebinds the same instance to the selected pin before starting; its id and +data stay. Initialized data accepts only a revision of its own build, so another build, line or +major fails with advice to run `supabase stack destroy`. Auth, Storage and Realtime must not be +selected at an older upstream than one that already migrated the database (recorded in +`schema-writers.json` in the instance data directory); such a start fails with the same advice. +The database version comes from `supabase/supabase.lock`, resolved and written under the +project lease exactly as `supabase start` does (see its `SIDE_EFFECTS.md`, "Service versions and +the lock"), except that there is no `--frozen` flag: a missing or stale lock is resolved, unless +`CI` is set and a lock exists, in which case a stale lock fails. The lock covers every service, +not only the database. The resident owner process manages the database beyond the CLI invocation. Stack mode rejects `--from-backup` with exit code 1. diff --git a/apps/cli/src/commands/db/start/start.integration.test.ts b/apps/cli/src/commands/db/start/start.integration.test.ts index 2c80574d04..2d4791b7d2 100644 --- a/apps/cli/src/commands/db/start/start.integration.test.ts +++ b/apps/cli/src/commands/db/start/start.integration.test.ts @@ -50,7 +50,7 @@ import type { DbStartFlags } from "./start.command.ts"; import { stackLocalDatabaseConn } from "../../../command-internal/stack-local-database.ts"; import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; import { StackApi } from "../../../command-internal/stack-api.ts"; -import { postgresVersion } from "@supabase/stack/internal/artifacts"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; import { StackCatalogSetup, StackCatalogSetupError, @@ -62,6 +62,7 @@ import { type ServiceCreationInput, type Stack, } from "@supabase/stack/effect"; +import { database15, selection } from "../../../../tests/helpers/pins.ts"; const DEFAULT_FLAGS: DbStartFlags = { fromBackup: Option.none() }; const PG_NET_CREATE_FINGERPRINT = "create extension if not exists pg_net schema extensions"; @@ -1628,11 +1629,8 @@ describe("db start stack backend", () => { const stackId = "b".repeat(64); const databaseCreation: Extract = { service: "database", + pin: selection.database, config: { - // Derived from the real catalog's major-17 pin, not hardcoded: `dbStart`'s stack-backend - // path resolves the desired version through the same catalog, so a bump would otherwise - // make the "resumes the existing database" test below see a spurious version mismatch. - version: postgresVersion("17"), databasePassword: Redacted.make("secret"), jwtSecret: Redacted.make("secret"), jwtExpiry: 3600, @@ -1645,14 +1643,16 @@ describe("db start stack backend", () => { running: boolean; destroyed: boolean; credentialsCalled?: boolean; + pin: ArtifactPin; + startedPin?: ArtifactPin; }, onDestroy: () => void = () => {}, - version = databaseCreation.config.version, ): DatabaseInstance => ({ id: "database-primary", service: "database", start: Effect.sync(() => { state.running = true; + state.startedPin = state.pin; }), ready: Effect.void, stop: Effect.sync(() => { @@ -1669,10 +1669,7 @@ describe("db start stack backend", () => { endpoints: state.running ? [{ name: "sql", protocol: "tcp" as const, host: "127.0.0.1", port: 54329 }] : [], - config: { - ...databaseCreation, - config: { ...databaseCreation.config, version }, - }, + config: { ...databaseCreation, pin: state.pin }, lifecycle: state.running ? ("running" as const) : ("stopped" as const), health: state.running ? ("healthy" as const) : undefined, error: undefined, @@ -1696,22 +1693,23 @@ describe("db start stack backend", () => { existing: boolean, running = false, standalone = false, - version = databaseCreation.config.version, + pin: ArtifactPin = databaseCreation.pin, ) => { - const state: { running: boolean; destroyed: boolean; credentialsCalled?: boolean } = { - running, - destroyed: false, - }; + const state: { + running: boolean; + destroyed: boolean; + credentialsCalled?: boolean; + pin: ArtifactPin; + startedPin?: ArtifactPin; + reusedIds?: ReadonlyArray; + } = { running, destroyed: false, pin }; + let otherMemberIds: ReadonlyArray = []; let members: ReadonlyArray = []; let registered: ReadonlyArray = []; - const database = databaseInstance( - state, - () => { - members = []; - registered = []; - }, - version, - ); + const database = databaseInstance(state, () => { + members = []; + registered = []; + }); members = existing && !standalone ? [database] : []; registered = existing ? [database] : []; const stack: Stack = { @@ -1721,7 +1719,9 @@ describe("db start stack backend", () => { get: (id: string) => id === database.id ? Effect.succeed(database) - : Effect.fail(new StackError({ operation: "get", message: `unknown ${id}` })), + : otherMemberIds.includes(id) + ? Effect.succeed({ ...database, id }) + : Effect.fail(new StackError({ operation: "get", message: `unknown ${id}` })), list: Effect.sync(() => registered), }, credentials: { @@ -1746,7 +1746,7 @@ describe("db start stack backend", () => { creations.flatMap((creation) => creation.service === "database" && registered.includes(database) ? [ - postgresVersion(creation.config.version) === postgresVersion(version) + creation.pin.image === state.pin.image ? { id: database.id, service: "database" as const, @@ -1757,8 +1757,11 @@ describe("db start stack backend", () => { id: database.id, service: "database" as const, member: members.includes(database), - change: "incompatible" as const, - paths: ["config.version"], + change: + creation.pin.upstreamVersion === state.pin.upstreamVersion + ? ("changed" as const) + : ("incompatible" as const), + paths: ["pin"], endpointsOnly: false, }, ] @@ -1766,12 +1769,22 @@ describe("db start stack backend", () => { ), ), describe: Effect.sync(() => ({ - members: members.map(({ id }) => ({ id, activation: "eager" as const })), + members: [...members.map(({ id }) => id), ...otherMemberIds].map((id) => ({ + id, + activation: "eager" as const, + })), dependencies: [], })), - supabase: (creations: ReadonlyArray) => + supabase: ( + creations: ReadonlyArray, + options?: { readonly reuseIds?: ReadonlyArray; readonly retainMembers?: boolean }, + ) => Effect.sync(() => { + state.reusedIds = options?.reuseIds; + const [creation] = creations; + if (options?.reuseIds !== undefined && creation !== undefined) state.pin = creation.pin; members = creations.map(() => database); + if (options?.retainMembers !== true) otherMemberIds = []; registered = [database]; return members; }), @@ -1786,7 +1799,13 @@ describe("db start stack backend", () => { gateway: unusedGateway, commands: { run: () => Effect.die("unused") }, }; - return { stack, state }; + return { + stack, + state, + withOtherMembers: (ids: ReadonlyArray) => { + otherMemberIds = ids; + }, + }; }; const stackLayer = (root: string, fixture: ReturnType, existing: boolean) => @@ -1875,7 +1894,7 @@ describe("db start stack backend", () => { it.effect("rejects a different database major when resuming the saved stack", () => { const { layer, catalogApplied } = setup({ recordCatalog: true }); - const fixture = stackFixture(true, false, false, "15.13.0.161"); + const fixture = stackFixture(true, false, false, database15); return Effect.gen(function* () { const exit = yield* dbStart(DEFAULT_FLAGS).pipe( Effect.provide( @@ -1893,6 +1912,58 @@ describe("db start stack backend", () => { }); }); + it.effect("starts a stopped database on the selected pin revision and keeps its instance", () => { + const { layer } = setup({ recordCatalog: true }); + const savedPin = { + ...selection.database, + revision: selection.database.revision - 1, + image: `${selection.database.image}-previous`, + }; + const fixture = stackFixture(true, false, false, savedPin); + return Effect.gen(function* () { + yield* dbStart(DEFAULT_FLAGS).pipe( + Effect.provide( + Layer.mergeAll( + layer, + stackBackendLayer("stack"), + stackLayer(tempRoot.current, fixture, true), + ), + ), + ); + expect(fixture.state.reusedIds).toEqual(["database-primary"]); + expect(fixture.state.startedPin).toEqual(selection.database); + expect(fixture.state.destroyed).toBe(false); + }); + }); + + it.effect("rebinds the database without detaching the other members of the saved stack", () => { + const { layer } = setup({ recordCatalog: true }); + const savedPin = { + ...selection.database, + revision: selection.database.revision - 1, + image: `${selection.database.image}-previous`, + }; + const fixture = stackFixture(true, false, false, savedPin); + fixture.withOtherMembers(["auth-primary", "rest-primary"]); + return Effect.gen(function* () { + yield* dbStart(DEFAULT_FLAGS).pipe( + Effect.provide( + Layer.mergeAll( + layer, + stackBackendLayer("stack"), + stackLayer(tempRoot.current, fixture, true), + ), + ), + ); + expect(fixture.state.startedPin).toEqual(selection.database); + expect((yield* fixture.stack.composition.describe).members.map(({ id }) => id)).toEqual([ + "database-primary", + "auth-primary", + "rest-primary", + ]); + }); + }); + it.effect("rejects a standalone database outside the saved composition", () => { const { layer, catalogApplied } = setup({ recordCatalog: true }); const fixture = stackFixture(true, false, true); diff --git a/apps/cli/src/commands/domains/activate/activate.command.ts b/apps/cli/src/commands/domains/activate/activate.command.ts index e5df45dfb6..499035c3ad 100644 --- a/apps/cli/src/commands/domains/activate/activate.command.ts +++ b/apps/cli/src/commands/domains/activate/activate.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/domains/activate/activate.handler.ts b/apps/cli/src/commands/domains/activate/activate.handler.ts index e3844d3436..069480c447 100644 --- a/apps/cli/src/commands/domains/activate/activate.handler.ts +++ b/apps/cli/src/commands/domains/activate/activate.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -18,13 +18,13 @@ export const domainsActivate = Effect.fn("domains.activate")(function* ( ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* warnIncludeRawOutputDeprecated(flags.includeRawOutput); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const activating = diff --git a/apps/cli/src/commands/domains/create/create.command.ts b/apps/cli/src/commands/domains/create/create.command.ts index 08fd97a3fa..d6d0b5cf20 100644 --- a/apps/cli/src/commands/domains/create/create.command.ts +++ b/apps/cli/src/commands/domains/create/create.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/domains/create/create.handler.ts b/apps/cli/src/commands/domains/create/create.handler.ts index 76336a82aa..f80a91ea1e 100644 --- a/apps/cli/src/commands/domains/create/create.handler.ts +++ b/apps/cli/src/commands/domains/create/create.handler.ts @@ -3,7 +3,7 @@ import * as HttpClient from "effect/http/HttpClient"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -21,13 +21,13 @@ export const domainsCreate = Effect.fn("domains.create")(function* (flags: Domai const httpClient = yield* HttpClient.HttpClient; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* warnIncludeRawOutputDeprecated(flags.includeRawOutput); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); // Write the linked-project cache and persist the telemetry state file on // success and failure. diff --git a/apps/cli/src/commands/domains/delete/delete.command.ts b/apps/cli/src/commands/domains/delete/delete.command.ts index 8e5e6ca90e..446fd58556 100644 --- a/apps/cli/src/commands/domains/delete/delete.command.ts +++ b/apps/cli/src/commands/domains/delete/delete.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/domains/delete/delete.handler.ts b/apps/cli/src/commands/domains/delete/delete.handler.ts index c8286c3017..889088ef34 100644 --- a/apps/cli/src/commands/domains/delete/delete.handler.ts +++ b/apps/cli/src/commands/domains/delete/delete.handler.ts @@ -1,7 +1,7 @@ import { Effect } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -18,13 +18,13 @@ const DELETE_SUCCESS_MESSAGE = "Deleted custom hostname config successfully."; export const domainsDelete = Effect.fn("domains.delete")(function* (flags: DomainsDeleteFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* warnIncludeRawOutputDeprecated(flags.includeRawOutput); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const deleting = diff --git a/apps/cli/src/commands/domains/get/get.command.ts b/apps/cli/src/commands/domains/get/get.command.ts index 1c3880c715..21dc077de6 100644 --- a/apps/cli/src/commands/domains/get/get.command.ts +++ b/apps/cli/src/commands/domains/get/get.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/domains/get/get.handler.ts b/apps/cli/src/commands/domains/get/get.handler.ts index 2d752e8098..b6f040004d 100644 --- a/apps/cli/src/commands/domains/get/get.handler.ts +++ b/apps/cli/src/commands/domains/get/get.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -16,13 +16,13 @@ const mapGetError = mapDomainsHttpError("get"); export const domainsGet = Effect.fn("domains.get")(function* (flags: DomainsGetFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* warnIncludeRawOutputDeprecated(flags.includeRawOutput); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); // Write the linked-project cache and persist the telemetry state file on // success and failure. diff --git a/apps/cli/src/commands/domains/reverify/reverify.command.ts b/apps/cli/src/commands/domains/reverify/reverify.command.ts index 8ff4d02fee..6edac1b1c5 100644 --- a/apps/cli/src/commands/domains/reverify/reverify.command.ts +++ b/apps/cli/src/commands/domains/reverify/reverify.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/domains/reverify/reverify.handler.ts b/apps/cli/src/commands/domains/reverify/reverify.handler.ts index a7b10b9dee..91f5643c77 100644 --- a/apps/cli/src/commands/domains/reverify/reverify.handler.ts +++ b/apps/cli/src/commands/domains/reverify/reverify.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -18,13 +18,13 @@ export const domainsReverify = Effect.fn("domains.reverify")(function* ( ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* warnIncludeRawOutputDeprecated(flags.includeRawOutput); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const reverifying = diff --git a/apps/cli/src/commands/encryption/get-root-key/get-root-key.command.ts b/apps/cli/src/commands/encryption/get-root-key/get-root-key.command.ts index 1456321a96..ececf00c12 100644 --- a/apps/cli/src/commands/encryption/get-root-key/get-root-key.command.ts +++ b/apps/cli/src/commands/encryption/get-root-key/get-root-key.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/encryption/get-root-key/get-root-key.handler.ts b/apps/cli/src/commands/encryption/get-root-key/get-root-key.handler.ts index c2fc3e1b0c..8994e941d4 100644 --- a/apps/cli/src/commands/encryption/get-root-key/get-root-key.handler.ts +++ b/apps/cli/src/commands/encryption/get-root-key/get-root-key.handler.ts @@ -1,7 +1,7 @@ import { Effect } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -15,11 +15,11 @@ export const encryptionGetRootKey = Effect.fn("encryption.get-root-key")(functio ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); // Write the linked-project cache and persist the telemetry state file on // success and failure. diff --git a/apps/cli/src/commands/encryption/update-root-key/update-root-key.command.ts b/apps/cli/src/commands/encryption/update-root-key/update-root-key.command.ts index 48771f6ea9..6ede9edb6b 100644 --- a/apps/cli/src/commands/encryption/update-root-key/update-root-key.command.ts +++ b/apps/cli/src/commands/encryption/update-root-key/update-root-key.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { stdinLayer } from "../../../shared/runtime/stdin.layer.ts"; diff --git a/apps/cli/src/commands/encryption/update-root-key/update-root-key.handler.ts b/apps/cli/src/commands/encryption/update-root-key/update-root-key.handler.ts index 0a776e7914..cefcbd9110 100644 --- a/apps/cli/src/commands/encryption/update-root-key/update-root-key.handler.ts +++ b/apps/cli/src/commands/encryption/update-root-key/update-root-key.handler.ts @@ -2,7 +2,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { aqua } from "../../../command-internal/colors.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { Stdin } from "../../../shared/runtime/stdin.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; @@ -20,12 +20,12 @@ export const encryptionUpdateRootKey = Effect.fn("encryption.update-root-key")(f ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const stdin = yield* Stdin; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); // In text mode, the piped-input path writes the prompt to stderr and echoes a // trailing newline to stdout after reading; on a TTY the masked prompt uses diff --git a/apps/cli/src/commands/experimental/compute/compute.output.ts b/apps/cli/src/commands/experimental/compute/compute.output.ts index cd2dc04aa3..29db58a7b6 100644 --- a/apps/cli/src/commands/experimental/compute/compute.output.ts +++ b/apps/cli/src/commands/experimental/compute/compute.output.ts @@ -100,7 +100,7 @@ export const rejectComputeEnvOutput = Effect.fnUntraced(function* () { * re-resolve to whichever project is currently linked — for `delete --yes`, * a same-named compute deleted without a prompt. Keyed off the flag, not the * resolved ref, so a link-supplied ref isn't echoed back as noise; an empty - * flag value counts as "not supplied", matching `ProjectRefResolver`. + * flag value counts as "not supplied", matching `RemoteTarget`. */ export const computeProjectRefSuffix = (projectRef: Option.Option): string => Option.isSome(projectRef) && projectRef.value.length > 0 diff --git a/apps/cli/src/commands/experimental/compute/delete/delete.command.ts b/apps/cli/src/commands/experimental/compute/delete/delete.command.ts index 2c942a3215..c6aa73426b 100644 --- a/apps/cli/src/commands/experimental/compute/delete/delete.command.ts +++ b/apps/cli/src/commands/experimental/compute/delete/delete.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withKind } from "../../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../../telemetry/linked-project-cache.layer.ts"; import { computeRead } from "../../../../shared/compute/compute-api.ts"; import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/experimental/compute/delete/delete.handler.ts b/apps/cli/src/commands/experimental/compute/delete/delete.handler.ts index c73878224e..102a1fc924 100644 --- a/apps/cli/src/commands/experimental/compute/delete/delete.handler.ts +++ b/apps/cli/src/commands/experimental/compute/delete/delete.handler.ts @@ -19,7 +19,7 @@ import { ComputeApiUnexpectedStatusError, } from "../../../../shared/compute/compute.errors.ts"; import { resolveYes } from "../../../../command-internal/global-flags.ts"; -import { ProjectRefResolver } from "../../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../../config/remote-target.service.ts"; import { Tty } from "../../../../shared/runtime/tty.service.ts"; import { LinkedProjectCache } from "../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; @@ -41,7 +41,7 @@ export const computeDelete = Effect.fn("compute.delete")(function* (flags: Compu const output = yield* Output; const path = yield* Path.Path; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const tty = yield* Tty; @@ -51,7 +51,7 @@ export const computeDelete = Effect.fn("compute.delete")(function* (flags: Compu // them; everything that can fail on its own — loading `config.toml`, // validating the name, resolving the compute — belongs inside, so those // failures still flush telemetry. Same shape as `config/push`. - const projectRef = yield* resolver.resolve(flags.projectRef); + const { ref: projectRef } = yield* resolver.resolve(flags.projectRef); // Every retry this command suggests is for a *destructive* re-run, so the ref // has to survive the copy-paste. const refSuffix = computeProjectRefSuffix(flags.projectRef); diff --git a/apps/cli/src/commands/experimental/compute/list/list.command.ts b/apps/cli/src/commands/experimental/compute/list/list.command.ts index c59799f059..e4b9a17738 100644 --- a/apps/cli/src/commands/experimental/compute/list/list.command.ts +++ b/apps/cli/src/commands/experimental/compute/list/list.command.ts @@ -2,7 +2,7 @@ import { Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/experimental/compute/list/list.handler.ts b/apps/cli/src/commands/experimental/compute/list/list.handler.ts index 05e68e5766..cb2814d65a 100644 --- a/apps/cli/src/commands/experimental/compute/list/list.handler.ts +++ b/apps/cli/src/commands/experimental/compute/list/list.handler.ts @@ -9,7 +9,7 @@ import { CommandSettings } from "../../../../config/command-settings.service.ts" import { formatApiSize } from "../../../../shared/compute/compute-runtimes.ts"; import { computeUrl } from "../../../../shared/compute/compute-url.ts"; import { listCompute, type ComputeRecord } from "../../../../shared/compute/compute-api.ts"; -import { ProjectRefResolver } from "../../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { discoverComputeNames, loadComputeProject } from "../compute.shared.ts"; @@ -96,7 +96,7 @@ export const computeList = Effect.fn("compute.list")(function* (flags: ComputeLi const output = yield* Output; const path = yield* Path.Path; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const settings = yield* CommandSettings; @@ -104,7 +104,7 @@ export const computeList = Effect.fn("compute.list")(function* (flags: ComputeLi // Resolved here, outside the block below, since caching it is one of that // block's own finalizers — everything else that can fail belongs inside so // those failures still flush telemetry. - const projectRef = yield* resolver.resolve(flags.projectRef); + const { ref: projectRef } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const project = yield* loadComputeProject; diff --git a/apps/cli/src/commands/experimental/compute/logs/logs.command.ts b/apps/cli/src/commands/experimental/compute/logs/logs.command.ts index 7221e4701c..ed17d71eb5 100644 --- a/apps/cli/src/commands/experimental/compute/logs/logs.command.ts +++ b/apps/cli/src/commands/experimental/compute/logs/logs.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withContext } from "../../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../../telemetry/linked-project-cache.layer.ts"; import { computeRead } from "../../../../shared/compute/compute-api.ts"; import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/experimental/compute/logs/logs.handler.ts b/apps/cli/src/commands/experimental/compute/logs/logs.handler.ts index b4f3f79d59..d2ca029893 100644 --- a/apps/cli/src/commands/experimental/compute/logs/logs.handler.ts +++ b/apps/cli/src/commands/experimental/compute/logs/logs.handler.ts @@ -30,7 +30,7 @@ import { ComputeApiNetworkError, ComputeApiUnexpectedStatusError, } from "../../../../shared/compute/compute.errors.ts"; -import { ProjectRefResolver } from "../../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { validateComputeName } from "../compute.shared.ts"; @@ -142,7 +142,7 @@ export const computeLogs = Effect.fn("compute.logs")(function* ( ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const processControl = yield* ProcessControl; @@ -151,7 +151,7 @@ export const computeLogs = Effect.fn("compute.logs")(function* ( // checkout fails inside `resolve` after the command has already run. Only the // linked-project cache stays under the ref, since it has nothing to write without one. yield* Effect.gen(function* () { - const projectRef = yield* resolver.resolve(flags.projectRef); + const { ref: projectRef } = yield* resolver.resolve(flags.projectRef); const refSuffix = computeProjectRefSuffix(flags.projectRef); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/experimental/compute/push/push.command.ts b/apps/cli/src/commands/experimental/compute/push/push.command.ts index 1e24fd4ad5..a59d689019 100644 --- a/apps/cli/src/commands/experimental/compute/push/push.command.ts +++ b/apps/cli/src/commands/experimental/compute/push/push.command.ts @@ -6,7 +6,7 @@ import { withCondition, withContext, } from "../../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../../telemetry/linked-project-cache.layer.ts"; import { computeRead } from "../../../../shared/compute/compute-api.ts"; import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/experimental/compute/push/push.handler.ts b/apps/cli/src/commands/experimental/compute/push/push.handler.ts index 4551975f86..8d4e9513cf 100644 --- a/apps/cli/src/commands/experimental/compute/push/push.handler.ts +++ b/apps/cli/src/commands/experimental/compute/push/push.handler.ts @@ -50,7 +50,7 @@ import { ComputeSourceMissingError, MissingComputeExposureError, } from "../../../../shared/compute/compute.errors.ts"; -import { ProjectRefResolver } from "../../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { @@ -544,14 +544,14 @@ export const computePush = Effect.fn("compute.push")(function* ( ) { const output = yield* Output; const path = yield* Path.Path; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // Resolved here, outside the block below, since caching it is one of that // block's own finalizers — everything else that can fail belongs inside so // those failures still flush telemetry. - const projectRef = yield* resolver.resolve(flags.projectRef); + const { ref: projectRef } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const project = yield* loadComputeProject; diff --git a/apps/cli/src/commands/experimental/compute/status/status.command.ts b/apps/cli/src/commands/experimental/compute/status/status.command.ts index 97f7a756b2..2fbcc75d9c 100644 --- a/apps/cli/src/commands/experimental/compute/status/status.command.ts +++ b/apps/cli/src/commands/experimental/compute/status/status.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../../telemetry/linked-project-cache.layer.ts"; import { computeRead } from "../../../../shared/compute/compute-api.ts"; import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/experimental/compute/status/status.handler.ts b/apps/cli/src/commands/experimental/compute/status/status.handler.ts index c3c7648976..2aab16c2d3 100644 --- a/apps/cli/src/commands/experimental/compute/status/status.handler.ts +++ b/apps/cli/src/commands/experimental/compute/status/status.handler.ts @@ -15,7 +15,7 @@ import { formatApiSize } from "../../../../shared/compute/compute-runtimes.ts"; import { computeUrl } from "../../../../shared/compute/compute-url.ts"; import { getCompute } from "../../../../shared/compute/compute-api.ts"; import { ComputeNotDeployedError } from "../../../../shared/compute/compute.errors.ts"; -import { ProjectRefResolver } from "../../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { @@ -36,7 +36,7 @@ export const computeStatus = Effect.fn("compute.status")(function* (flags: Compu const output = yield* Output; const path = yield* Path.Path; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const settings = yield* CommandSettings; @@ -44,7 +44,7 @@ export const computeStatus = Effect.fn("compute.status")(function* (flags: Compu // Resolved here, outside the block below, since caching it is one of that // block's own finalizers — everything else that can fail belongs inside so // those failures still flush telemetry. - const projectRef = yield* resolver.resolve(flags.projectRef); + const { ref: projectRef } = yield* resolver.resolve(flags.projectRef); const refSuffix = computeProjectRefSuffix(flags.projectRef); yield* Effect.gen(function* () { diff --git a/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md index 8e06c6b960..16b76c3d49 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/destroy/SIDE_EFFECTS.md @@ -1,8 +1,7 @@ # `supabase stack destroy` Permanently removes each selected managed namespace, its service registrations, -owned database data, port claims, and attached jobs. Caller-owned Storage upload -files remain. The experimental feature flag controls command registration. +owned database data, Storage upload files, port claims, and attached jobs. The experimental feature flag controls command registration. ## Selection and confirmation @@ -13,10 +12,9 @@ found fail together, each named, before anything is destroyed, and ids naming th same stack select it once. Explicit legacy `-o/--output` is rejected in favor of `--output-format`. -Interactive text mode asks one question for all selected stacks and states that -Storage uploads are preserved. Non-interactive and machine-output runs require +Interactive text mode asks one question for all selected stacks. Non-interactive and machine-output runs require `--yes`. With `--yes` the command prints no question; stderr states which stacks and -data are destroyed and that Storage uploads are preserved. Rejection or cancellation +data are destroyed. Rejection or cancellation does not open or destroy a stack. Discovery may create/chmod the registry directory to 0700 but does not launch an owner. @@ -66,8 +64,12 @@ in place. Reads identity/state under `/stacks//` and, for implicit selection, the project's canonical Git context. Removes only the -selected namespace's owned state/data; other stack namespaces and caller-owned -uploads are preserved. The independent artifact cache is retained. Communicates +selected namespace's owned state/data and its `uploads/` directory +(`/stacks//uploads`), after the namespace is destroyed and +including for the containers of a deleted stack; a failure to remove it fails the command naming +the directory. This leaves the stack directory holding only an empty `data/`, which a later +registry sweep prunes. Other stack namespaces and files outside the stack directory +(a caller-supplied Storage path) are preserved. The independent artifact cache is retained. Communicates with the local owner and selected runtime, with no hosted API calls. Shared routing/settings can read project configuration and profiles. diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.command.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.command.ts index 91a1f25277..53f6644c69 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.command.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.command.ts @@ -27,7 +27,7 @@ export type StackDestroyFlags = CliCommand.Command.Config.Infer; export const stackDestroyCommand = Command.make("destroy", config).pipe( Command.withDescription( - "Permanently destroy a managed local stack and its owned data, preserving Storage uploads.", + "Permanently destroy a managed local stack and its owned data and Storage uploads.", ), Command.withShortDescription("Destroy a managed local stack"), Command.withExamples([ diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts index 2f4c73fe27..4cbe9848d3 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts @@ -8,7 +8,14 @@ import { } from "../../../../shared/telemetry/error-actionability.ts"; export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStackDestroyError")<{ - readonly reason: "flags" | "confirmation" | "cancelled" | "invalid-config" | "runtime" | "stack"; + readonly reason: + | "flags" + | "confirmation" + | "cancelled" + | "invalid-config" + | "runtime" + | "stack" + | "uploads"; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -26,6 +33,8 @@ export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStac return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; case "runtime": return { ...actionability.dockerNotRunning, fingerprint_suffix: "docker_not_running" }; + case "uploads": + return { ...actionability.runtimeCrash, fingerprint_suffix: "filesystem_io" }; case "stack": return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; } diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.handler.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.handler.ts index 37615fb82d..fe070e4446 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.handler.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.handler.ts @@ -1,4 +1,4 @@ -import { Cause, Effect, Exit, Option, Path, Result } from "effect"; +import { Cause, Effect, Exit, FileSystem, Option, Path, Result } from "effect"; import type { StackError } from "@supabase/stack/effect"; import { MachineErrorContext } from "../../../../shared/output/machine-error-context.service.ts"; import { Output } from "../../../../shared/output/output.service.ts"; @@ -6,6 +6,7 @@ import { OutputFlag, resolveYes } from "../../../../command-internal/global-flag import { promptYesNo } from "../../../../command-internal/prompt-yes-no.ts"; import { Tty } from "../../../../shared/runtime/tty.service.ts"; import { CommandSettings } from "../../../../config/command-settings.service.ts"; +import { stackUploadsDir } from "../../../../command-internal/stack-config.ts"; import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { isStackId, @@ -51,6 +52,7 @@ export const stackDestroy = Effect.fn("experimental.stack.destroy")(function* ( const resolver = yield* StackTargetResolver; const path = yield* Path.Path; + const fs = yield* FileSystem.FileSystem; const locations = { stateRoot: path.join(settings.supabaseHome, "stacks"), cacheRoot: path.join(settings.supabaseHome, "cache", "stack"), @@ -131,15 +133,9 @@ export const stackDestroy = Effect.fn("experimental.stack.destroy")(function* ( suggestion: "Pass --yes when running non-interactively or in a machine-readable format.", }); const scope = selected.map((entry) => entry.scope).join("; "); - const preserved = "Storage upload files will be preserved."; - if (yes) yield* output.raw(`Permanently destroying ${scope}. ${preserved}\n`, "stderr"); + if (yes) yield* output.raw(`Permanently destroying ${scope}.\n`, "stderr"); else { - const confirmed = yield* promptYesNo( - output, - false, - `Permanently destroy ${scope}? ${preserved}`, - false, - ); + const confirmed = yield* promptYesNo(output, false, `Permanently destroy ${scope}?`, false); if (!confirmed) return yield* new StackCommandDestroyError({ reason: "cancelled", @@ -163,6 +159,18 @@ export const stackDestroy = Effect.fn("experimental.stack.destroy")(function* ( ), Effect.mapError(destroyError(id)), ); + const uploads = stackUploadsDir(path, settings.supabaseHome, id); + yield* fs.remove(uploads, { recursive: true, force: true }).pipe( + Effect.mapError( + (cause) => + new StackCommandDestroyError({ + reason: "uploads", + message: `Stack ${id} was destroyed, but its Storage uploads could not be removed: ${cause.message}`, + suggestion: `Remove ${uploads} manually.`, + cause, + }), + ), + ); if (!batch) yield* Effect.annotateCurrentSpan({ "stack.prompted": !yes }); if (output.format === "text") yield* output.raw( @@ -186,18 +194,32 @@ export const stackDestroy = Effect.fn("experimental.stack.destroy")(function* ( const [failure] = failed; if (failure !== undefined) { if (!batch) return yield* failure.error; + /* A stack whose registration is gone counts as destroyed and cannot be retried. */ + const leftover = failed.filter(({ error }) => error.reason === "uploads"); + const retryable = failed.filter(({ error }) => error.reason !== "uploads"); const machineErrorContext = yield* Effect.serviceOption(MachineErrorContext); if (Option.isSome(machineErrorContext)) - yield* machineErrorContext.value.set({ destroyed_stacks: destroyed }); - const retry = `"supabase stack destroy ${failed.map(({ id }) => `--stack-id ${id}`).join(" ")} --yes"`; - const runtime = failed.every(({ error }) => error.reason === "runtime"); + yield* machineErrorContext.value.set({ + destroyed_stacks: [...destroyed, ...leftover.map(({ id }) => ({ id }))], + }); + const manual = + leftover.length === 0 ? undefined : leftover.map(({ error }) => error.suggestion).join(" "); + const retry = `"supabase stack destroy ${retryable.map(({ id }) => `--stack-id ${id}`).join(" ")} --yes"`; + const runtime = + retryable.length > 0 && retryable.every(({ error }) => error.reason === "runtime"); + const retrySuggestion = runtime + ? `Start the container engine, then run ${retry} again; nothing was removed for those stacks.` + : `Resolve each error, then run ${retry} to retry the stacks that failed.`; return yield* new StackCommandDestroyError({ - reason: runtime ? "runtime" : "stack", + reason: retryable.length === 0 ? "uploads" : runtime ? "runtime" : "stack", message: `Failed to destroy ${failed.length} managed stack(s).`, detail: failed.map(({ id, error }) => `${id}: ${error.message}`).join("\n"), - suggestion: runtime - ? `Start the container engine, then run ${retry} again; nothing was removed for those stacks.` - : `Resolve each error, then run ${retry} to retry the stacks that failed.`, + suggestion: + retryable.length === 0 + ? manual + : manual === undefined + ? retrySuggestion + : `${retrySuggestion} ${manual}`, // Telemetry classifies a batch by its first failure; `detail` lists every stack. cause: failure.error, }); diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.integration.test.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.integration.test.ts index be0332c4dd..a1ef37b003 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.integration.test.ts @@ -1,7 +1,18 @@ import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { StackError } from "@supabase/stack/effect"; -import { Effect, FileSystem, Layer, Option, Path, Schema, Sink, Stdio, Stream } from "effect"; +import { + Effect, + FileSystem, + Layer, + Option, + Path, + PlatformError, + Schema, + Sink, + Stdio, + Stream, +} from "effect"; import { CliOutput, Command } from "effect/cli"; import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; import { jsonOutputLayer } from "../../../../shared/output/output.layer.ts"; @@ -22,6 +33,8 @@ import { import { StackApi, stackApiLayer, stackTargetResolverLayer } from "../stack.shared.ts"; import { stackDestroyCommand } from "./destroy.command.ts"; import { stackDestroy } from "./destroy.handler.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; +import { stackUploadsDir } from "../../../../command-internal/stack-config.ts"; const live = Layer.provideMerge(stackApiLayer, BunServices.layer); const refuseDestroy = ( @@ -50,6 +63,24 @@ const refuseDestroy = ( }), ), }); +const refuseUploadsRemoval = Layer.effect( + FileSystem.FileSystem, + Effect.map(FileSystem.FileSystem, (fs) => ({ + ...fs, + remove: (target: string, options?: Parameters[1]) => + target.endsWith("uploads") + ? Effect.fail( + PlatformError.systemError({ + _tag: "PermissionDenied", + module: "FileSystem", + method: "remove", + description: "denied", + pathOrDescriptor: target, + }), + ) + : fs.remove(target, options), + })), +).pipe(Layer.provide(BunServices.layer)); const runDestroy = (layer: Layer.Layer, argv: ReadonlyArray) => Command.runWith(stackDestroyCommand.pipe(Command.provide(layer)), { version: "0.0.0-test" })( argv, @@ -125,49 +156,52 @@ describe("stack destroy", () => { const error = yield* stackDestroy(f.flags).pipe(Effect.provide(f.layer), Effect.flip); expect(error.reason).toBe("cancelled"); expect(f.output.promptConfirmCalls.map(({ message }) => message)).toEqual([ - `Permanently destroy stack ${f.stack.id} at ${f.projectRoot} and its owned data? Storage upload files will be preserved.`, + `Permanently destroy stack ${f.stack.id} at ${f.projectRoot} and its owned data?`, ]); const saved = yield* f.api.discover(f.locations); expect(saved.map(({ definition }) => definition.id)).toEqual([f.stack.id]); }).pipe(Effect.provide(live)), ); - it.live("destroys an offline namespace without affecting a different stack", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const other = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "other", - runtime: "native", - }); - yield* Effect.acquireUseRelease( - Effect.succeed(f.stack), - () => stackDestroy(f.flags).pipe(Effect.provide(f.layer)), - () => - f.api - .discover(f.locations) - .pipe( - Effect.flatMap((entries) => - entries.some( - ({ definition, host }) => definition.id === f.stack.id && host !== undefined, - ) - ? f.stack.stop - : Effect.void, + it.live( + "destroys an offline namespace without affecting a different stack", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const other = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "other", + runtime: "native", + }); + yield* Effect.acquireUseRelease( + Effect.succeed(f.stack), + () => stackDestroy(f.flags).pipe(Effect.provide(f.layer)), + () => + f.api + .discover(f.locations) + .pipe( + Effect.flatMap((entries) => + entries.some( + ({ definition, host }) => definition.id === f.stack.id && host !== undefined, + ) + ? f.stack.stop + : Effect.void, + ), ), - ), - ); - expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ - other.id, - ]); - expect(f.output.stderrText).toContain( - `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data. Storage upload files will be preserved.\n`, - ); - expect(f.output.stderrText).not.toContain("[y/N]"); - expect(f.output.promptConfirmCalls).toEqual([]); - expect(f.output.stdoutText).toContain(`Stack ${f.stack.id} destroyed.`); - expect(f.telemetry.flushed).toBe(true); - }).pipe(Effect.provide(live)), + ); + expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual( + [other.id], + ); + expect(f.output.stderrText).toContain( + `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data.\n`, + ); + expect(f.output.stderrText).not.toContain("[y/N]"); + expect(f.output.promptConfirmCalls).toEqual([]); + expect(f.output.stdoutText).toContain(`Stack ${f.stack.id} destroyed.`); + expect(f.telemetry.flushed).toBe(true); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); it.effect("refuses a stack with an unknown service kind and names its directory", () => @@ -189,31 +223,37 @@ describe("stack destroy", () => { }).pipe(Effect.provide(live)), ); - it.live("destroys only the stack addressed by the short ID that stack list shows", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const other = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "other", - runtime: "native", - }); - - yield* stackDestroy({ ...f.flags, stackId: [f.stack.id.slice(0, 8)] }).pipe( - Effect.provide(f.layer), - ); + it.live( + "destroys only the stack addressed by the short ID that stack list shows", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const other = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "other", + runtime: "native", + }); + + yield* stackDestroy({ ...f.flags, stackId: [f.stack.id.slice(0, 8)] }).pipe( + Effect.provide(f.layer), + ); - expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ - other.id, - ]); - expect(f.output.stdoutText).toContain(`Stack ${f.stack.id} destroyed.`); - }).pipe(Effect.provide(live)), + expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual( + [other.id], + ); + expect(f.output.stdoutText).toContain(`Stack ${f.stack.id} destroyed.`); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); it.effect("dispatches a full ID with no registration to its deleted-stack cleanup", () => Effect.gen(function* () { const f = yield* fixture(true); const id = "d".repeat(64); + const uploads = stackUploadsDir(f.path, f.root, id); + yield* f.fs.makeDirectory(uploads, { recursive: true }); + yield* f.fs.writeFileString(f.path.join(uploads, "object.txt"), "upload"); let destroyed = 0; const api = Layer.succeed(StackApi, { ...f.api, @@ -235,8 +275,9 @@ describe("stack destroy", () => { ); expect(destroyed).toBe(1); + expect(yield* f.fs.exists(uploads)).toBe(false); expect(f.output.stderrText).toContain( - `Permanently destroying the containers deleted stack ${id} left behind. Storage upload files will be preserved.\n`, + `Permanently destroying the containers deleted stack ${id} left behind.\n`, ); expect(f.output.stdoutText).toContain(`Removed the containers stack ${id} left behind.`); expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ @@ -280,7 +321,24 @@ describe("stack destroy", () => { ); it.live( - "destroys a live namespace and standalone services while preserving caller-owned uploads", + "removes the stack's Storage uploads, leaving only the empty data directory to prune", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const uploads = stackUploadsDir(f.path, f.root, f.stack.id); + yield* f.fs.makeDirectory(f.path.join(uploads, "bucket"), { recursive: true }); + yield* f.fs.writeFileString(f.path.join(uploads, "bucket", "object.txt"), "upload"); + yield* stackDestroy(f.flags).pipe(Effect.provide(f.layer)); + expect(yield* f.fs.exists(uploads)).toBe(false); + expect(yield* f.fs.readDirectory(f.path.join(f.locations.stateRoot, f.stack.id))).toEqual([ + "data", + ]); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, + ); + + it.live( + "destroys a live namespace and standalone services while leaving files outside the stack directory", () => Effect.gen(function* () { const f = yield* fixture(true); @@ -290,6 +348,7 @@ describe("stack destroy", () => { yield* Effect.acquireUseRelease( f.stack.services.create({ service: "storage", + pin: selection.storage, config: { jwtSecret: "test-secret", filePath: uploads, @@ -298,7 +357,12 @@ describe("stack destroy", () => { }), () => Effect.gen(function* () { - yield* f.stack.services.create({ service: "mail", config: {}, endpoints: {} }); + yield* f.stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + endpoints: {}, + }); yield* stackDestroy(f.flags).pipe(Effect.provide(f.layer)); expect(yield* f.api.discover(f.locations)).toEqual([]); expect(yield* f.fs.readFileString(f.path.join(uploads, "object.txt"))).toBe( @@ -315,71 +379,78 @@ describe("stack destroy", () => { ), ); }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); - it.live("destroys each stack named by repeated --stack-id flags once", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const second = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "second", - runtime: "native", - }); - const third = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "third", - runtime: "native", - }); - const output = mockOutput({ format: "json" }); - - yield* runDestroy(Layer.merge(f.layer, output.layer), [ - "--stack-id", - f.stack.id, - "--stack-id", - second.id.slice(0, 8), - "--stack-id", - f.stack.id.slice(0, 8), - ]); - - expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ - third.id, - ]); - expect(output.stderrText).toBe( - `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data; stack ${second.id} at ${f.projectRoot} and its owned data. Storage upload files will be preserved.\n`, - ); - expect(output.messages).toContainEqual( - expect.objectContaining({ - type: "success", - data: { - destroyed: true, - stacks: [{ id: f.stack.id }, { id: second.id }], - }, - }), - ); - }).pipe(Effect.provide(live)), + it.live( + "destroys each stack named by repeated --stack-id flags once", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const second = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "second", + runtime: "native", + }); + const third = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "third", + runtime: "native", + }); + const output = mockOutput({ format: "json" }); + + yield* runDestroy(Layer.merge(f.layer, output.layer), [ + "--stack-id", + f.stack.id, + "--stack-id", + second.id.slice(0, 8), + "--stack-id", + f.stack.id.slice(0, 8), + ]); + + expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual( + [third.id], + ); + expect(output.stderrText).toBe( + `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data; stack ${second.id} at ${f.projectRoot} and its owned data.\n`, + ); + expect(output.messages).toContainEqual( + expect.objectContaining({ + type: "success", + data: { + destroyed: true, + stacks: [{ id: f.stack.id }, { id: second.id }], + }, + }), + ); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); - it.live("keeps the batch result when repeated --stack-id flags name one stack", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const output = mockOutput({ format: "json" }); - - yield* runDestroy(Layer.merge(f.layer, output.layer), [ - "--stack-id", - f.stack.id, - "--stack-id", - f.stack.id.slice(0, 8), - ]); - - expect(output.messages).toContainEqual( - expect.objectContaining({ - type: "success", - data: { destroyed: true, stacks: [{ id: f.stack.id }] }, - }), - ); - }).pipe(Effect.provide(live)), + it.live( + "keeps the batch result when repeated --stack-id flags name one stack", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const output = mockOutput({ format: "json" }); + + yield* runDestroy(Layer.merge(f.layer, output.layer), [ + "--stack-id", + f.stack.id, + "--stack-id", + f.stack.id.slice(0, 8), + ]); + + expect(output.messages).toContainEqual( + expect.objectContaining({ + type: "success", + data: { destroyed: true, stacks: [{ id: f.stack.id }] }, + }), + ); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); it.effect("rejects --stack combined with repeated --stack-id flags", () => @@ -422,7 +493,7 @@ describe("stack destroy", () => { expect(error.reason).toBe("cancelled"); expect(f.output.promptConfirmCalls.map(({ message }) => message)).toEqual([ - `Permanently destroy stack ${f.stack.id} at ${f.projectRoot} and its owned data; stack ${second.id} at ${f.projectRoot} and its owned data? Storage upload files will be preserved.`, + `Permanently destroy stack ${f.stack.id} at ${f.projectRoot} and its owned data; stack ${second.id} at ${f.projectRoot} and its owned data?`, ]); expect( (yield* f.api.discover(f.locations)).map(({ definition }) => definition.id).toSorted(), @@ -455,142 +526,215 @@ describe("stack destroy", () => { }).pipe(Effect.provide(live)), ); - it.effect("fails a single stack with its own destroy error", () => - Effect.gen(function* () { - const f = yield* fixture(true); + it.effect( + "fails a single stack with its own destroy error", + () => + Effect.gen(function* () { + const f = yield* fixture(true); - const error = yield* stackDestroy(f.flags).pipe( - Effect.provide(Layer.merge(f.layer, refuseDestroy(f.api, [f.stack.id]))), - Effect.flip, - ); + const error = yield* stackDestroy(f.flags).pipe( + Effect.provide(Layer.merge(f.layer, refuseDestroy(f.api, [f.stack.id]))), + Effect.flip, + ); - expect(error).toMatchObject({ reason: "stack", message: "engine refused" }); - expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ - f.stack.id, - ]); - }).pipe(Effect.provide(live)), + expect(error).toMatchObject({ reason: "stack", message: "engine refused" }); + expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual( + [f.stack.id], + ); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); - it.effect("points a batch at the engine only when it was unreachable for every stack", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const second = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "second", - runtime: "native", - }); - const ids = [f.stack.id, second.id]; - const retry = `"supabase stack destroy --stack-id ${f.stack.id} --stack-id ${second.id} --yes"`; - const destroyWith = (unreachable: ReadonlyArray) => - stackDestroy({ ...f.flags, stackId: ids }).pipe( - Effect.provide(Layer.merge(f.layer, refuseDestroy(f.api, ids, unreachable))), - Effect.flip, + it.effect( + "points a batch at the engine only when it was unreachable for every stack", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const second = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "second", + runtime: "native", + }); + const ids = [f.stack.id, second.id]; + const retry = `"supabase stack destroy --stack-id ${f.stack.id} --stack-id ${second.id} --yes"`; + const destroyWith = (unreachable: ReadonlyArray) => + stackDestroy({ ...f.flags, stackId: ids }).pipe( + Effect.provide(Layer.merge(f.layer, refuseDestroy(f.api, ids, unreachable))), + Effect.flip, + ); + + expect(yield* destroyWith([f.stack.id])).toMatchObject({ + reason: "stack", + suggestion: `Resolve each error, then run ${retry} to retry the stacks that failed.`, + }); + expect(yield* destroyWith(ids)).toMatchObject({ + reason: "runtime", + suggestion: `Start the container engine, then run ${retry} again; nothing was removed for those stacks.`, + }); + expect( + (yield* f.api.discover(f.locations)).map(({ definition }) => definition.id).toSorted(), + ).toEqual(ids.toSorted()); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, + ); + + it.live( + "destroys a registered stack and a deleted stack's containers in one batch", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const id = "d".repeat(64); + let removed = 0; + const api = Layer.succeed(StackApi, { + ...f.api, + findDeleted: (options) => + Effect.succeed( + options.id === id + ? Option.some({ + id, + destroy: Effect.sync(() => { + removed += 1; + }), + }) + : Option.none(), + ), + }); + + yield* stackDestroy({ ...f.flags, stackId: [f.stack.id, id] }).pipe( + Effect.provide(Layer.merge(f.layer, api)), ); - expect(yield* destroyWith([f.stack.id])).toMatchObject({ - reason: "stack", - suggestion: `Resolve each error, then run ${retry} to retry the stacks that failed.`, - }); - expect(yield* destroyWith(ids)).toMatchObject({ - reason: "runtime", - suggestion: `Start the container engine, then run ${retry} again; nothing was removed for those stacks.`, - }); - expect( - (yield* f.api.discover(f.locations)).map(({ definition }) => definition.id).toSorted(), - ).toEqual(ids.toSorted()); - }).pipe(Effect.provide(live)), + expect(removed).toBe(1); + expect(f.output.stderrText).toBe( + `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data; the containers deleted stack ${id} left behind.\n`, + ); + expect(f.output.stdoutText).toBe( + `Stack ${f.stack.id} destroyed.\nRemoved the containers stack ${id} left behind.\n`, + ); + expect(yield* f.api.discover(f.locations)).toEqual([]); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); - it.live("destroys a registered stack and a deleted stack's containers in one batch", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const id = "d".repeat(64); - let removed = 0; - const api = Layer.succeed(StackApi, { - ...f.api, - findDeleted: (options) => - Effect.succeed( - options.id === id - ? Option.some({ - id, - destroy: Effect.sync(() => { - removed += 1; - }), - }) - : Option.none(), + it.live( + "keeps destroying after a failure and fails once with the stacks it destroyed", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const second = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "second", + runtime: "native", + }); + const stdout: Array = []; + const stdio = Layer.succeed( + Stdio.Stdio, + Stdio.make({ + args: Effect.succeed([]), + stdin: Stream.empty, + stdout: () => + Sink.forEach((item: string | Uint8Array) => + Effect.sync(() => { + stdout.push(typeof item === "string" ? item : new TextDecoder().decode(item)); + }), + ), + stderr: () => Sink.forEach(() => Effect.void), + }), + ); + const processControl = mockProcessControl(); + + yield* runDestroy( + Layer.merge( + f.layer, + Layer.mergeAll( + refuseDestroy(f.api, [f.stack.id]), + jsonOutputLayer.pipe(Layer.provide(stdio)), + processControl.layer, + ), ), - }); - - yield* stackDestroy({ ...f.flags, stackId: [f.stack.id, id] }).pipe( - Effect.provide(Layer.merge(f.layer, api)), - ); + ["--stack-id", f.stack.id, "--stack-id", second.id], + ); - expect(removed).toBe(1); - expect(f.output.stderrText).toBe( - `Permanently destroying stack ${f.stack.id} at ${f.projectRoot} and its owned data; the containers deleted stack ${id} left behind. Storage upload files will be preserved.\n`, - ); - expect(f.output.stdoutText).toBe( - `Stack ${f.stack.id} destroyed.\nRemoved the containers stack ${id} left behind.\n`, - ); - expect(yield* f.api.discover(f.locations)).toEqual([]); - }).pipe(Effect.provide(live)), + expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual( + [f.stack.id], + ); + expect( + yield* Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown))(stdout.join("")), + ).toEqual({ + destroyed_stacks: [{ id: second.id }], + _tag: "Error", + error: { + code: "ExperimentalStackDestroyError", + message: "Failed to destroy 1 managed stack(s).", + detail: `${f.stack.id}: engine refused`, + suggestion: `Resolve each error, then run "supabase stack destroy --stack-id ${f.stack.id} --yes" to retry the stacks that failed.`, + }, + }); + expect(processControl.exitCode).toBe(1); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); - it.live("keeps destroying after a failure and fails once with the stacks it destroyed", () => - Effect.gen(function* () { - const f = yield* fixture(true); - const second = yield* f.api.create({ - ...f.locations, - projectRoot: f.root, - name: "second", - runtime: "native", - }); - const stdout: Array = []; - const stdio = Layer.succeed( - Stdio.Stdio, - Stdio.make({ - args: Effect.succeed([]), - stdin: Stream.empty, - stdout: () => - Sink.forEach((item: string | Uint8Array) => - Effect.sync(() => { - stdout.push(typeof item === "string" ? item : new TextDecoder().decode(item)); - }), + it.live( + "counts a stack as destroyed when only its uploads directory could not be removed", + () => + Effect.gen(function* () { + const f = yield* fixture(true); + const second = yield* f.api.create({ + ...f.locations, + projectRoot: f.root, + name: "second", + runtime: "native", + }); + const stdout: Array = []; + const stdio = Layer.succeed( + Stdio.Stdio, + Stdio.make({ + args: Effect.succeed([]), + stdin: Stream.empty, + stdout: () => + Sink.forEach((item: string | Uint8Array) => + Effect.sync(() => { + stdout.push(typeof item === "string" ? item : new TextDecoder().decode(item)); + }), + ), + stderr: () => Sink.forEach(() => Effect.void), + }), + ); + const processControl = mockProcessControl(); + + yield* runDestroy( + Layer.merge( + f.layer, + Layer.mergeAll( + refuseUploadsRemoval, + jsonOutputLayer.pipe(Layer.provide(stdio)), + processControl.layer, ), - stderr: () => Sink.forEach(() => Effect.void), - }), - ); - const processControl = mockProcessControl(); - - yield* runDestroy( - Layer.merge( - f.layer, - Layer.mergeAll( - refuseDestroy(f.api, [f.stack.id]), - jsonOutputLayer.pipe(Layer.provide(stdio)), - processControl.layer, ), - ), - ["--stack-id", f.stack.id, "--stack-id", second.id], - ); + ["--stack-id", f.stack.id, "--stack-id", second.id], + ); - expect((yield* f.api.discover(f.locations)).map(({ definition }) => definition.id)).toEqual([ - f.stack.id, - ]); - expect( - yield* Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown))(stdout.join("")), - ).toEqual({ - destroyed_stacks: [{ id: second.id }], - _tag: "Error", - error: { - code: "ExperimentalStackDestroyError", - message: "Failed to destroy 1 managed stack(s).", - detail: `${f.stack.id}: engine refused`, - suggestion: `Resolve each error, then run "supabase stack destroy --stack-id ${f.stack.id} --yes" to retry the stacks that failed.`, - }, - }); - expect(processControl.exitCode).toBe(1); - }).pipe(Effect.provide(live)), + expect(yield* f.api.discover(f.locations)).toEqual([]); + const uploads = stackUploadsDir(f.path, f.root, f.stack.id); + const secondUploads = stackUploadsDir(f.path, f.root, second.id); + expect( + yield* Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown))(stdout.join("")), + ).toMatchObject({ + destroyed_stacks: expect.arrayContaining([{ id: f.stack.id }, { id: second.id }]), + _tag: "Error", + error: { + code: "ExperimentalStackDestroyError", + message: "Failed to destroy 2 managed stack(s).", + suggestion: expect.stringContaining(`Remove ${uploads} manually.`), + }, + }); + expect(stdout.join("")).toContain(`Remove ${secondUploads} manually.`); + expect(stdout.join("")).not.toContain("to retry the stacks that failed"); + expect(processControl.exitCode).toBe(1); + }).pipe(Effect.provide(live)), + { timeout: 60_000 }, ); }); diff --git a/apps/cli/src/commands/experimental/stack/install/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/install/SIDE_EFFECTS.md new file mode 100644 index 0000000000..840c37fa21 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/install/SIDE_EFFECTS.md @@ -0,0 +1,100 @@ +# `supabase stack install` + +The command is available when the experimental stack family is enabled. It resolves the project's +service versions into `supabase/supabase.lock` and downloads everything the lock pins for the chosen +runtime. It creates, starts and registers no stack, and reads no stack state or database data, so +it needs no stack selector. + +## Service versions and the lock + +The lock is read, resolved and written exactly as `supabase stack start` does it (see "Service +versions and the lock" in [`../start/SIDE_EFFECTS.md`](../start/SIDE_EFFECTS.md)), under the same +project lease `supabase/.supabase.lock.lease`: + +- A lock that answers the current `[local.services]` request is used as-is, with no network. +- A missing lock, or a service whose specifier changed, is resolved again and the lock is written. + The `Resolved service versions` report lists the diff. +- `--frozen`, or the `CI` environment variable (not empty, `false` or `0`) with an existing lock, + fails instead of writing: a missing lock fails with `supabase/supabase.lock not found`, a stale one + with `is stale for: ()`. Nothing is downloaded after a lock failure. A frozen + run reads the lock without the lease and writes nothing in the project. +- A `linked` service follows the hosted version `supabase link` recorded, resolved on the release + host only; install never calls the Management API or needs a login. When the release host cannot + be reached for it, the service takes the bundled version as `unobserved` and warns instead of + failing; only an explicit `` specifier fails with transient (`network`). A `.link.json` + that targets another remote than the lock observed, or a newer bundled revision of a locked + version, prints a warning (see `start`). +- A project without `supabase/config.toml` uses the versions bundled with the CLI and writes no lock. + Frozen, by `--frozen` or by `CI` with an existing lock, the lock cannot be checked, so it fails + with `supabase/config.toml not found` rather than using the bundled versions; `--frozen` without + a lock fails with `supabase/supabase.lock not found`. Both fail before anything is downloaded. + Without a lock `CI` changes nothing. + +To move a fresh entry to a newer release, use `supabase stack update`; `supabase stack outdated` +reports which entries would move (see [`../update/SIDE_EFFECTS.md`](../update/SIDE_EFFECTS.md) and +[`../outdated/SIDE_EFFECTS.md`](../outdated/SIDE_EFFECTS.md)). + +## Download + +`--runtime` defaults to `auto`, which selects Docker, then Podman, then native, as `stack start` +does for a new stack; an explicit value has no fallback. Explicit `--runtime native` on a platform +with no native artifacts fails with reason `flags`. Reason `runtime` applies only to `auto` when no +engine is found; an explicit `docker` or `podman` with an unreachable engine fails with reason `stack` +(kind `engine-unavailable`). + +- `native` downloads the locked archive of each of the twelve services for the current platform + into `$SUPABASE_HOME/cache/stack` (`~/.supabase/cache/stack` by default), verifying the pinned + archive and manifest digests. An artifact already in the cache is not downloaded again. +- `docker` and `podman` pull each locked image by digest through the selected engine; an image the + engine already holds is not pulled again. + +Up to four services are fetched at a time. A download failure fails the command with reason `stack` +and keeps the package's failure kind; archives that completed stay cached. + +## Files read + +- `/supabase/config.toml`, project dotenv files, supported `SUPABASE_*` overrides, and + `/supabase/supabase.lock`. +- Shared profile and telemetry state under `/`. + +## Files written + +- `/supabase/supabase.lock` (JSON), only when it is missing or stale and not frozen. +- `/supabase/.supabase.lock.lease` and `.supabase.lock.*.tmp` while the lock is written; + `supabase/.gitignore` gains both entries whenever the project lease is taken. A frozen run takes + no lease and writes neither. +- Native archives under `$SUPABASE_HOME/cache/stack`, keyed by content digest. +- Container images in the selected engine's image store. +- Telemetry state after success or failure. + +## Network and subprocesses + +Resolving a release specifier probes the slim-services GitHub releases and registry +(`SUPABASE_INTERNAL_SLIM_RELEASES_URL` and `SUPABASE_INTERNAL_SLIM_REGISTRY_URL` redirect them). Native +downloads fetch from the release host, falling back to a public S3 mirror. Container runtimes run +the engine's CLI to inspect and pull images. No service process is started. + +## Flags and output + +`--runtime auto|docker|podman|native` and `--frozen`. The legacy `-o/--output` flag is rejected; +use `--output-format`. + +Text output prints the lock diff when it changed, then whether `supabase/supabase.lock` was written +or is up to date, a `Downloaded N of M archives|images for the runtime; K already present.` +summary, and one ` (downloaded|cached)` line per service. + +JSON output returns `runtime`, `lock` (`file`, `written`, `added`, `changed`, `unchanged`; `null` +without a project config), `artifacts` (`service`, `release`, `outcome`), `downloaded` and `cached`, +and an empty message. Errors retain typed actionability and package diagnostics. + +## Exit codes + +- `0` when the lock is current and every artifact is present. +- `130` when interrupted. +- `1` for a rejected output flag, invalid configuration, a stale or missing lock under `--frozen`, a + failed resolution, an unavailable runtime, or a failed download. + +## Telemetry + +The command emits the standard `cli_command_executed` event through command telemetry and flushes +telemetry state after success or failure. It emits no custom event. diff --git a/apps/cli/src/commands/experimental/stack/install/install.command.ts b/apps/cli/src/commands/experimental/stack/install/install.command.ts new file mode 100644 index 0000000000..b658f64016 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/install/install.command.ts @@ -0,0 +1,41 @@ +import { Command, Flag } from "effect/cli"; +import type * as CliCommand from "effect/cli/Command"; +import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withCommandTelemetry } from "../../../../telemetry/command-telemetry.ts"; +import { frozenFlag } from "../stack.flags.ts"; +import { stackInstall } from "./install.handler.ts"; + +const config = { + runtime: Flag.Literals("runtime", ["auto", "docker", "podman", "native"] as const).pipe( + Flag.withDescription( + "Runtime to download artifacts for. auto selects Docker, then Podman, then native, based on what is available. native downloads archives; docker and podman pull images.", + ), + Flag.withDefault("auto" as const), + ), + frozen: frozenFlag, +} as const; + +export type StackInstallFlags = CliCommand.Command.Config.Infer; + +export const stackInstallCommand = Command.make("install", config).pipe( + Command.withDescription( + "Resolve the local service versions in supabase/supabase.lock and download everything the lock pins for the chosen runtime, without creating or starting a stack. " + + "A missing or out-of-date lock is resolved and written; --frozen (or CI with an existing lock) fails instead.", + ), + Command.withShortDescription("Resolve and download local stack artifacts"), + Command.withExamples([ + { + command: "supabase stack install", + description: "Resolve the lock and download artifacts for the available runtime", + }, + { + command: "supabase stack install --frozen --runtime native", + description: "Download the locked native archives, failing if the lock is stale", + }, + ]), + withPermissions({ operations: [], noApiEffectFlags: ["runtime", "frozen"] }), + Command.withHandler((flags) => + stackInstall(flags).pipe(withCommandTelemetry({ flags, config }), withJsonErrorHandling), + ), +); diff --git a/apps/cli/src/commands/experimental/stack/install/install.errors.ts b/apps/cli/src/commands/experimental/stack/install/install.errors.ts new file mode 100644 index 0000000000..9ef0b0d4d6 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/install/install.errors.ts @@ -0,0 +1,52 @@ +import type { StackError } from "@supabase/stack/effect"; +import { Data } from "effect"; +import type { StackRuntimeSelectionError } from "../../../../command-internal/stack-runtime.ts"; +import { + actionability, + causeDeclaration, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, + unclassifiedStackFailureActionability, +} from "../../../../shared/telemetry/error-actionability.ts"; + +export class StackCommandInstallError extends Data.TaggedError("ExperimentalStackInstallError")<{ + readonly reason: "flags" | "invalid-config" | "lock" | "runtime" | "stack"; + readonly message: string; + readonly suggestion?: string; + readonly cause?: unknown; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + switch (this.reason) { + case "flags": + return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; + case "invalid-config": + return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + case "lock": + return ( + causeDeclaration(this.cause) ?? { + ...actionability.invalidConfig, + fingerprint_suffix: "invalid_config", + } + ); + case "runtime": + return { ...actionability.dockerNotRunning, fingerprint_suffix: "docker_not_running" }; + case "stack": + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + } + } +} + +/** Maps a failed download to the install command's stable error boundary. */ +export const stackInstallError = (error: StackError): StackCommandInstallError => + new StackCommandInstallError({ reason: "stack", message: error.message, cause: error }); + +/** Maps a failed runtime selection to the install command's stable error boundary. */ +export const stackInstallRuntimeError = ( + error: StackRuntimeSelectionError, +): StackCommandInstallError => + new StackCommandInstallError({ + reason: error.reason === "native-unsupported" ? "flags" : "runtime", + message: error.message, + suggestion: error.suggestion, + cause: error, + }); diff --git a/apps/cli/src/commands/experimental/stack/install/install.handler.ts b/apps/cli/src/commands/experimental/stack/install/install.handler.ts new file mode 100644 index 0000000000..77b121db0d --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/install/install.handler.ts @@ -0,0 +1,154 @@ +import { prepareArtifactSelection } from "@supabase/stack/internal/artifacts"; +import type { PreparedRelease } from "@supabase/stack/internal/artifacts"; +import { Effect, Path, Ref, Schema } from "effect"; +import { FetchHttpClient } from "effect/http"; +import { OutputFlag } from "../../../../command-internal/global-flags.ts"; +import { loadLocalProjectContext } from "../../../../command-internal/local-project-context.ts"; +import { loadStackConfig, StackConfigError } from "../../../../command-internal/stack-config.ts"; +import { + LockResolutionError, + lockedResolution, + type LockedResolution, + LOCK_FILE, +} from "../../../../command-internal/stack-lock.ts"; +import { + selectStackRuntime, + type StackRuntime, +} from "../../../../command-internal/stack-runtime.ts"; +import { CommandSettings } from "../../../../config/command-settings.service.ts"; +import { Output } from "../../../../shared/output/output.service.ts"; +import { RuntimeInfo } from "../../../../shared/runtime/runtime-info.service.ts"; +import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; +import { mapTargetError, rejectStackOutput } from "../stack.shared.ts"; +import type { StackInstallFlags } from "./install.command.ts"; +import { + StackCommandInstallError, + stackInstallError, + stackInstallRuntimeError, +} from "./install.errors.ts"; + +const lockReport = (resolution: LockedResolution | undefined) => + resolution === undefined + ? null + : { + file: LOCK_FILE, + written: resolution.written, + added: resolution.changes.flatMap(({ service, from, to }) => + from === undefined ? [{ service, to }] : [], + ), + changed: resolution.changes.flatMap(({ service, from, to }) => + from === undefined ? [] : [{ service, from, to }], + ), + unchanged: resolution.unchanged, + }; + +const count = (releases: ReadonlyArray, outcome: PreparedRelease["outcome"]) => + releases.filter((release) => release.outcome === outcome).length; + +const render = ( + runtime: StackRuntime, + resolution: LockedResolution | undefined, + releases: ReadonlyArray, +) => { + const lock = + resolution === undefined + ? "No supabase/config.toml: using the versions bundled with the CLI; no lock written." + : resolution.written + ? `Wrote ${LOCK_FILE}.` + : `${LOCK_FILE} is up to date.`; + const downloaded = count(releases, "downloaded"); + const lines = [ + lock, + `Downloaded ${downloaded} of ${releases.length} ${runtime === "native" ? "archives" : "images"} for the ${runtime} runtime; ${releases.length - downloaded} already present.`, + ...releases.map(({ service, release, outcome }) => ` ${service} ${release} (${outcome})`), + ]; + return `${lines.join("\n")}\n`; +}; + +/** Resolves and locks the service versions, then downloads what the lock pins without starting a stack. */ +export const stackInstall = Effect.fn("experimental.stack.install")(function* ( + flags: StackInstallFlags, +) { + const telemetryState = yield* TelemetryState; + const body = Effect.gen(function* () { + const output = yield* Output; + const settings = yield* CommandSettings; + const path = yield* Path.Path; + const runtimeInfo = yield* RuntimeInfo; + const outputFlag = yield* Effect.serviceOption(OutputFlag); + yield* rejectStackOutput(outputFlag).pipe( + Effect.mapError(mapTargetError((props) => new StackCommandInstallError(props))), + ); + + const pickRuntime = selectStackRuntime( + flags.runtime === "auto" ? undefined : flags.runtime, + ).pipe(Effect.mapError(stackInstallRuntimeError), Effect.withSpan("install.runtime")); + const picked = yield* Ref.make(undefined); + const resolved = yield* Ref.make(undefined); + const resolve = lockedResolution({ frozen: flags.frozen, output }); + const config = yield* Effect.gen(function* () { + const context = yield* loadLocalProjectContext( + settings.workdir, + (message) => new StackConfigError({ message }), + ); + return yield* loadStackConfig(settings.workdir, { + context, + frozen: flags.frozen, + publishesLock: true, + // The runtime is checked after the config validates and before the lock is touched. + selection: (request) => + pickRuntime.pipe( + Effect.tap((runtime) => Ref.set(picked, runtime)), + Effect.andThen(resolve(request)), + Effect.tap((resolution) => Ref.set(resolved, resolution)), + Effect.map((resolution) => resolution.selection), + ), + }); + }).pipe( + Effect.mapError((error) => + error instanceof StackCommandInstallError + ? error + : new StackCommandInstallError({ + reason: Schema.is(LockResolutionError)(error) ? "lock" : "invalid-config", + message: error.message, + cause: error, + }), + ), + Effect.withSpan("install.resolve"), + ); + const resolution = yield* Ref.get(resolved); + const runtime = (yield* Ref.get(picked)) ?? (yield* pickRuntime); + + const task = + output.format === "text" ? yield* output.task("Downloading artifacts...") : undefined; + const releases = yield* prepareArtifactSelection(config.selection, { + runtime, + cacheRoot: path.join(settings.supabaseHome, "cache", "stack"), + platform: { os: runtimeInfo.platform, arch: runtimeInfo.arch }, + }).pipe( + Effect.provide(FetchHttpClient.layer), + Effect.mapError(stackInstallError), + Effect.tapError(() => task?.fail() ?? Effect.void), + Effect.withSpan("install.download"), + ); + yield* task?.clear ?? Effect.void; + + const downloaded = count(releases, "downloaded"); + yield* Effect.annotateCurrentSpan({ + "install.runtime": runtime, + "install.downloaded_count": downloaded, + "install.cached_count": releases.length - downloaded, + "install.lock_written": resolution?.written ?? false, + }); + if (output.format === "text") yield* output.raw(render(runtime, resolution, releases)); + else + yield* output.success("", { + runtime, + lock: lockReport(resolution), + artifacts: releases, + downloaded, + cached: releases.length - downloaded, + }); + }); + return yield* body.pipe(Effect.ensuring(telemetryState.flush)); +}); diff --git a/apps/cli/src/commands/experimental/stack/install/install.integration.test.ts b/apps/cli/src/commands/experimental/stack/install/install.integration.test.ts new file mode 100644 index 0000000000..c6c7d3c920 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/install/install.integration.test.ts @@ -0,0 +1,350 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { readLock } from "@supabase/stack/effect"; +import { + artifactServiceKinds, + catalogPins, + resolveArtifact, + type ServiceKind, +} from "@supabase/stack/internal/artifacts"; +import { Effect, FileSystem, Layer, Option } from "effect"; +import { + mockCommandSettings, + mockTelemetryStateTracked, + withConfigEnv, +} from "../../../../../tests/helpers/command-mocks.ts"; +import { mockOutput, mockRuntimeInfo } from "../../../../../tests/helpers/mocks.ts"; +import { releaseArchive } from "../../../../../tests/helpers/release-archive.ts"; +import { + publishRelease, + releaseHostEnv, + startReleaseFixture, + type ReleaseFixture, +} from "../../../../../tests/helpers/release-fixture.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; +import { ExperimentalFlag } from "../../../../command-internal/global-flags.ts"; +import { StackCommandInstallError } from "./install.errors.ts"; +import { stackInstall } from "./install.handler.ts"; + +const UPSTREAM = (kind: ServiceKind) => (kind === "database" ? "17.99.0.0" : "v99.0.0"); + +const requestedVersions = (upstream: (kind: ServiceKind) => string = UPSTREAM) => + artifactServiceKinds() + .map((kind) => `[local.services.${kind}]\nversion = "${upstream(kind)}"\n`) + .join("\n"); + +/** Publishes a release of every service, each with an archive holding what that service requires. */ +const publishAll = (host: ReleaseFixture) => + Effect.forEach(artifactServiceKinds(), (kind) => + Effect.gen(function* () { + const producer = catalogPins().find( + (entry) => entry.service === kind && entry.isDefault, + )?.sourceService; + if (producer === undefined) return yield* Effect.die(`no producer for ${kind}`); + const { requiredRuntimePaths, executablePath } = yield* resolveArtifact({ + service: kind, + pin: selection[kind], + }); + publishRelease(host, { + producer, + upstream: UPSTREAM(kind), + revision: 0, + provenanceSource: `docker.io/upstream/${producer}:${UPSTREAM(kind)}@sha256:${"a".repeat(64)}`, + archive: releaseArchive([...new Set([...requiredRuntimePaths, executablePath])]), + }); + }), + ); + +const project = (config = requestedVersions()) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-install-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + `project_id = "stack-install"\n${config}`, + ); + return root; + }); + +const lockPath = (root: string) => `${root}/supabase/supabase.lock`; +const leasePath = (root: string) => `${root}/supabase/.supabase.lock.lease`; + +const install = ( + root: string, + host: ReleaseFixture, + options: { + readonly format?: "text" | "json"; + readonly frozen?: boolean; + readonly ci?: boolean; + readonly platform?: "linux" | "darwin"; + } = {}, +) => { + const output = mockOutput({ format: options.format ?? "text" }); + const run = withConfigEnv( + { CI: options.ci === true ? "true" : "", ...releaseHostEnv(host) }, + stackInstall({ runtime: "native", frozen: options.frozen ?? false }), + ).pipe( + Effect.provide( + Layer.mergeAll( + BunServices.layer, + mockRuntimeInfo({ platform: options.platform ?? "linux", arch: "x64" }), + output.layer, + mockTelemetryStateTracked().layer, + mockCommandSettings({ workdir: root }), + Layer.succeed(ExperimentalFlag, false), + ), + ), + ); + return { output, run }; +}; + +const cachedFiles = (root: string) => + Effect.flatMap(FileSystem.FileSystem, (fs) => + fs.readDirectory(`${root}/.supabase/cache/stack`, { recursive: true }), + ).pipe(Effect.map((entries) => entries.toSorted())); + +const archiveRequests = (host: ReleaseFixture) => + host.requests.filter((path) => path.endsWith(".tar.zst")); + +describe("experimental stack install", () => { + it.live( + "resolves the lock, downloads every native archive, and finds them all cached on the next run", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + + const first = install(root, host); + yield* first.run; + + const lock = yield* readLock(root).pipe(Effect.map(Option.getOrThrow)); + expect(lock.services.rest.resolved.upstreamVersion).toBe("v99.0.0"); + expect(lock.services.database.resolved.upstreamVersion).toBe("17.99.0.0"); + expect( + first.output.messages.flatMap((entry) => + entry.type === "info" && entry.message.startsWith("Resolved service versions:") + ? [entry.message] + : [], + ), + ).toHaveLength(1); + expect(first.output.stdoutText).toContain( + "Downloaded 12 of 12 archives for the native runtime; 0 already present.", + ); + expect(first.output.stdoutText).toContain("rest v99.0.0-r0 (downloaded)"); + expect(archiveRequests(host)).toHaveLength(12); + expect(yield* cachedFiles(root)).not.toHaveLength(0); + expect(yield* fs.readFileString(`${root}/supabase/.gitignore`)).toContain( + ".supabase.lock.lease", + ); + + const written = yield* fs.readFileString(lockPath(root)); + const cached = yield* cachedFiles(root); + host.requests.length = 0; + const second = install(root, host, { format: "json" }); + yield* second.run; + + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(yield* cachedFiles(root)).toEqual(cached); + expect(archiveRequests(host)).toEqual([]); + const result = second.output.messages.find((entry) => entry.type === "success"); + expect(result?.type === "success" ? result.data : undefined).toMatchObject({ + runtime: "native", + lock: { file: "supabase/supabase.lock", written: false, added: [], changed: [] }, + downloaded: 0, + cached: 12, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails a frozen install naming the stale service and leaves the project untouched", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + yield* install(root, host).run; + const written = yield* fs.readFileString(lockPath(root)); + const cached = yield* cachedFiles(root); + const ignored = yield* fs.readFileString(`${root}/supabase/.gitignore`); + yield* fs.remove(leasePath(root)); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + `project_id = "stack-install"\n${requestedVersions((kind) => (kind === "rest" ? "v99.1.0" : UPSTREAM(kind)))}`, + ); + host.requests.length = 0; + + const error = yield* install(root, host, { frozen: true }).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ reason: "lock" }); + expect(error.message).toContain("rest"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(yield* fs.readFileString(`${root}/supabase/.gitignore`)).toBe(ignored); + expect(yield* fs.exists(leasePath(root))).toBe(false); + expect(yield* cachedFiles(root)).toEqual(cached); + expect(archiveRequests(host)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("installs under a frozen lock without taking the lease or writing into the project", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + yield* install(root, host).run; + const written = yield* fs.readFileString(lockPath(root)); + yield* fs.remove(`${root}/supabase/.gitignore`); + yield* fs.remove(leasePath(root)); + + yield* install(root, host, { frozen: true }).run; + + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(yield* fs.exists(leasePath(root))).toBe(false); + expect(yield* fs.exists(`${root}/supabase/.gitignore`)).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails a frozen install of a project without config.toml, downloading nothing", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-install-bare-" }); + + const error = yield* install(root, host, { frozen: true }).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-missing" }, + }); + expect(yield* fs.readDirectory(root)).toEqual([]); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails an install in CI with a lock but no config.toml, downloading nothing", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + yield* install(root, host).run; + const written = yield* fs.readFileString(lockPath(root)); + yield* fs.remove(`${root}/supabase/config.toml`); + host.requests.length = 0; + + const error = yield* install(root, host, { ci: true }).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-missing" }, + }); + expect(error.message).toContain("supabase/config.toml not found"); + expect(error.message).not.toContain("supabase.lock not found"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails a frozen install with a lock but no config.toml on the missing config", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + yield* install(root, host).run; + const written = yield* fs.readFileString(lockPath(root)); + yield* fs.remove(`${root}/supabase/config.toml`); + host.requests.length = 0; + + const error = yield* install(root, host, { frozen: true }).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-missing" }, + }); + expect(error.message).toContain("supabase/config.toml not found"); + expect(error.message).not.toContain("supabase.lock not found"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("ignores the lease file when it takes the lease for a lock that is already fresh", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + yield* install(root, host).run; + const written = yield* fs.readFileString(lockPath(root)); + yield* fs.writeFileString(`${root}/supabase/.gitignore`, "node_modules\n"); + + yield* install(root, host).run; + + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + const ignored = (yield* fs.readFileString(`${root}/supabase/.gitignore`)).split("\n"); + expect(ignored).toEqual(expect.arrayContaining(["node_modules", ".supabase.lock.lease"])); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails with reason invalid-config on a malformed config.toml, writing no lock", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project("this is = not [valid toml\n"); + + const error = yield* install(root, host).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ reason: "invalid-config" }); + expect(yield* fs.exists(lockPath(root))).toBe(false); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live( + "fails with reason invalid-config on an escaping function entrypoint, writing no lock", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project( + `${requestedVersions()}\n[functions.x]\nentrypoint = "../../outside.ts"\n`, + ); + + const error = yield* install(root, host).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ reason: "invalid-config" }); + expect(error.message).toContain("must remain within the project directory"); + expect(yield* fs.exists(lockPath(root))).toBe(false); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails with reason flags when the platform has no native artifacts", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + yield* publishAll(host); + const root = yield* project(); + + const error = yield* install(root, host, { platform: "darwin" }).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandInstallError); + expect(error).toMatchObject({ reason: "flags" }); + expect(host.requests).toEqual([]); + expect(yield* fs.exists(lockPath(root))).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/experimental/stack/logs/logs.integration.test.ts b/apps/cli/src/commands/experimental/stack/logs/logs.integration.test.ts index b5e6cff6a7..4d52c520f2 100644 --- a/apps/cli/src/commands/experimental/stack/logs/logs.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/logs/logs.integration.test.ts @@ -32,24 +32,30 @@ import { import { StackApi, stackApiLayer, stackTargetResolverLayer } from "../stack.shared.ts"; import type { StackLogsFlags } from "./logs.command.ts"; import { stackLogs } from "./logs.handler.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; const live = Layer.provideMerge(stackApiLayer, BunServices.layer); type SavedInstance = SavedStack["instances"][number]; const mail = (id: string): SavedInstance => ({ id, - creation: { service: "mail", config: {}, endpoints: {} }, + creation: { service: "mail", pin: selection.mail, config: {}, endpoints: {} }, }); const rest = (id: string): SavedInstance => ({ id, - creation: { service: "rest", config: {}, endpoints: { http: { port: "auto" } } }, + creation: { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: "auto" } }, + }, }); const database = (id: string): SavedInstance => ({ id, creation: { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("stack-logs-test"), jwtSecret: Redacted.make("stack-logs-test-jwt-secret"), jwtExpiry: 3600, diff --git a/apps/cli/src/commands/experimental/stack/outdated/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/outdated/SIDE_EFFECTS.md new file mode 100644 index 0000000000..d48360cb26 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/outdated/SIDE_EFFECTS.md @@ -0,0 +1,67 @@ +# `supabase stack outdated` + +The command is available when the experimental stack family is enabled. It reports, for each of the +twelve services in `supabase/supabase.lock`, the locked release and the release +`supabase stack update` would lock. It changes nothing: it takes no lease, writes no file and +downloads no artifact. + +## What it reports + +Each row has the service, the specifier from the lock, the hosted version the lock last observed +(`-` when none), the locked release, the update (absent when +`stack update` would keep the entry), the release bundled with this CLI, and a status: + +- `up to date`: `stack update` would keep the entry. +- `update available`: the specifier is `` and a newer revision is published, or the + specifier is `bundled` and this CLI bundles a different release, or the specifier is `linked` and + its stored observation now resolves to a different release or resolution (for example a version + slim-services has since published). +- `stale`: the entry no longer answers `config.toml`; the reason is listed under the table. It is + not probed, because `supabase start` re-resolves it. +- `probe failed`: the newest release could not be determined, for example when the release host + answered with a server error. The reason is listed under the table, and the command still + succeeds. + +An exact `-r` specifier is never probed. A missing lock, or one that cannot be read, +fails the command with reason `lock`; `invalid-config` is only for `config.toml` problems, such as a +project without `supabase/config.toml`. + +## Files read + +- `/supabase/config.toml`, project dotenv files, supported `SUPABASE_*` overrides, and + `/supabase/supabase.lock`. +- Shared profile and telemetry state under `/`. + +## Files written + +- Telemetry state after success or failure. Nothing under `supabase/`. + +## Network and subprocesses + +Each `` entry, and each `linked` entry with an observation, that is not stale is probed +against the slim-services GitHub releases and registry (`SUPABASE_INTERNAL_SLIM_RELEASES_URL` and +`SUPABASE_INTERNAL_SLIM_REGISTRY_URL` redirect them), up to four at a time. It never calls the +Management API or a tenant host and needs no login, so a hosted version that moved since +`supabase link` shows only after `supabase stack update`. No archive or image is downloaded and no +subprocess runs. + +## Flags and output + +The legacy `-o/--output` flag is rejected; use `--output-format`. + +Text output prints the table, one line per stale or failed row, and a hint naming +`supabase stack update` or `supabase start` when either applies. + +JSON output returns `file`, `services` (`service`, `specifier`, `resolution`, `hosted`, `locked`, `update`, +`bundled`, `stale`, `error`; `null` when absent), and the `updatable`, `stale` and `failed` counts. + +## Exit codes + +- `0` whenever the report was produced, including when updates are available or a row failed. +- `130` when interrupted. +- `1` for a rejected output flag, invalid configuration, or a missing or unreadable lock. + +## Telemetry + +The command emits the standard `cli_command_executed` event through command telemetry and flushes +telemetry state after success or failure. It emits no custom event. diff --git a/apps/cli/src/commands/experimental/stack/outdated/outdated.command.ts b/apps/cli/src/commands/experimental/stack/outdated/outdated.command.ts new file mode 100644 index 0000000000..bf755c6307 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/outdated/outdated.command.ts @@ -0,0 +1,23 @@ +import { Command } from "effect/cli"; +import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withCommandTelemetry } from "../../../../telemetry/command-telemetry.ts"; +import { stackOutdated } from "./outdated.handler.ts"; + +export const stackOutdatedCommand = Command.make("outdated").pipe( + Command.withDescription( + "Report, for each service in supabase/supabase.lock, the release supabase stack update would lock instead, this CLI's bundled pin, and whether the entry is out of date with config.toml. " + + "Read-only: it writes nothing and downloads no artifacts.", + ), + Command.withShortDescription("Show locked service versions with newer releases"), + Command.withExamples([ + { + command: "supabase stack outdated", + description: "List what supabase stack update would change", + }, + ]), + withPermissions({ operations: [] }), + Command.withHandler(() => + stackOutdated().pipe(withCommandTelemetry({ flags: {} }), withJsonErrorHandling), + ), +); diff --git a/apps/cli/src/commands/experimental/stack/outdated/outdated.errors.ts b/apps/cli/src/commands/experimental/stack/outdated/outdated.errors.ts new file mode 100644 index 0000000000..15dd7f40c1 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/outdated/outdated.errors.ts @@ -0,0 +1,30 @@ +import { Data } from "effect"; +import { + actionability, + causeDeclaration, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../../../../shared/telemetry/error-actionability.ts"; + +export class StackCommandOutdatedError extends Data.TaggedError("ExperimentalStackOutdatedError")<{ + readonly reason: "flags" | "invalid-config" | "lock"; + readonly message: string; + readonly suggestion?: string; + readonly cause?: unknown; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + switch (this.reason) { + case "flags": + return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; + case "invalid-config": + return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + case "lock": + return ( + causeDeclaration(this.cause) ?? { + ...actionability.invalidConfig, + fingerprint_suffix: "invalid_config", + } + ); + } + } +} diff --git a/apps/cli/src/commands/experimental/stack/outdated/outdated.handler.ts b/apps/cli/src/commands/experimental/stack/outdated/outdated.handler.ts new file mode 100644 index 0000000000..b3918460f0 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/outdated/outdated.handler.ts @@ -0,0 +1,133 @@ +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { Effect, Ref, Schema } from "effect"; +import { OutputFlag } from "../../../../command-internal/global-flags.ts"; +import { loadLocalProjectContext } from "../../../../command-internal/local-project-context.ts"; +import { loadStackConfig, StackConfigError } from "../../../../command-internal/stack-config.ts"; +import { + LockResolutionError, + outdatedServices, + type OutdatedService, + LOCK_FILE, +} from "../../../../command-internal/stack-lock.ts"; +import { CommandSettings } from "../../../../config/command-settings.service.ts"; +import { renderGlamourTable } from "../../../../output/glamour-table.ts"; +import { Output } from "../../../../shared/output/output.service.ts"; +import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; +import { mapTargetError, rejectStackOutput } from "../stack.shared.ts"; +import { StackCommandOutdatedError } from "./outdated.errors.ts"; + +const status = (row: OutdatedService) => + row.error !== undefined + ? "probe failed" + : row.stale !== undefined + ? "stale" + : row.update !== undefined + ? "update available" + : "up to date"; + +const render = (rows: ReadonlyArray) => { + const table = renderGlamourTable( + ["SERVICE", "SPECIFIER", "HOSTED", "LOCKED", "UPDATE", "BUNDLED", "STATUS"], + rows.map((row) => [ + row.service, + row.specifier, + row.hosted ?? "-", + row.locked, + row.update ?? "-", + row.bundled, + status(row), + ]), + ).trimEnd(); + const details = rows.flatMap(({ service, stale, error }) => [ + ...(stale === undefined ? [] : [` ${service}: ${LOCK_FILE} is stale (${stale})`]), + ...(error === undefined ? [] : [` ${service}: ${error}`]), + ]); + const hints = [ + ...(rows.some(({ update }) => update !== undefined) + ? ["Run supabase stack update to adopt the updates."] + : []), + ...(rows.some(({ stale }) => stale !== undefined) + ? ["Run supabase start or supabase stack update to re-resolve stale entries."] + : []), + ]; + return `${[table, ...(details.length === 0 ? [] : ["", ...details]), ...(hints.length === 0 ? [] : ["", ...hints])].join("\n")}\n`; +}; + +/** + * Reports which locked service versions a `stack update` would change, without writing or downloading + * anything. Hosted versions are the stored observations; only `stack update` re-probes the project. + */ +export const stackOutdated = Effect.fn("experimental.stack.outdated")(function* () { + const telemetryState = yield* TelemetryState; + const body = Effect.gen(function* () { + const output = yield* Output; + const settings = yield* CommandSettings; + const outputFlag = yield* Effect.serviceOption(OutputFlag); + yield* rejectStackOutput(outputFlag).pipe( + Effect.mapError(mapTargetError((props) => new StackCommandOutdatedError(props))), + ); + + const reported = yield* Ref.make | undefined>(undefined); + yield* Effect.gen(function* () { + const context = yield* loadLocalProjectContext( + settings.workdir, + (message) => new StackConfigError({ message }), + ); + return yield* loadStackConfig(settings.workdir, { + context, + selection: (request) => + outdatedServices(request).pipe( + Effect.tap((rows) => Ref.set(reported, rows)), + Effect.as(defaultArtifactSelection()), + ), + }); + }).pipe( + Effect.mapError( + (error) => + new StackCommandOutdatedError({ + reason: Schema.is(LockResolutionError)(error) ? "lock" : "invalid-config", + message: error.message, + cause: error, + }), + ), + Effect.withSpan("outdated.probe"), + ); + const rows = yield* Ref.get(reported); + if (rows === undefined) + return yield* new StackCommandOutdatedError({ + reason: "invalid-config", + message: "No supabase/config.toml in this project, so there is no lock to inspect.", + suggestion: "Run supabase init, then supabase start to create supabase/supabase.lock.", + }); + + const outdated = rows.filter(({ update }) => update !== undefined).length; + const stale = rows.filter((row) => row.stale !== undefined).length; + const failed = rows.filter((row) => row.error !== undefined).length; + yield* Effect.annotateCurrentSpan({ + "outdated.service_count": rows.length, + "outdated.updatable_count": outdated, + "outdated.stale_count": stale, + "outdated.failed_count": failed, + }); + if (output.format === "text") yield* output.raw(render(rows)); + else + yield* output.success("", { + file: LOCK_FILE, + services: rows.map((row) => ({ + service: row.service, + specifier: row.specifier, + resolution: row.resolution, + hosted: row.hosted ?? null, + locked: row.locked, + update: row.update ?? null, + bundled: row.bundled, + stale: row.stale ?? null, + error: row.error ?? null, + })), + updatable: outdated, + stale, + failed, + }); + }); + return yield* body.pipe(Effect.ensuring(telemetryState.flush)); +}); diff --git a/apps/cli/src/commands/experimental/stack/outdated/outdated.integration.test.ts b/apps/cli/src/commands/experimental/stack/outdated/outdated.integration.test.ts new file mode 100644 index 0000000000..33f5131170 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/outdated/outdated.integration.test.ts @@ -0,0 +1,218 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { holdLock, readLock, writeLock } from "@supabase/stack/effect"; +import { catalogPins, type ServiceKind } from "@supabase/stack/internal/artifacts"; +import { Effect, FileSystem, Layer, Option } from "effect"; +import { + mockCommandSettings, + mockTelemetryStateTracked, + withConfigEnv, +} from "../../../../../tests/helpers/command-mocks.ts"; +import { mockOutput, mockRuntimeInfo } from "../../../../../tests/helpers/mocks.ts"; +import { + publishRelease, + releaseHostEnv, + startReleaseFixture, + type ReleaseFixture, +} from "../../../../../tests/helpers/release-fixture.ts"; +import { seedLock } from "../../../../../tests/helpers/stack-lock.ts"; +import { ExperimentalFlag } from "../../../../command-internal/global-flags.ts"; +import { StackCommandOutdatedError } from "./outdated.errors.ts"; +import { stackOutdated } from "./outdated.handler.ts"; + +const UPSTREAM = "v99.0.0"; + +const publish = (host: ReleaseFixture, kind: ServiceKind, revision: number) => { + const producer = catalogPins().find( + (entry) => entry.service === kind && entry.isDefault, + )?.sourceService; + if (producer === undefined) throw new Error(`no producer for ${kind}`); + publishRelease(host, { producer, upstream: UPSTREAM, revision }); +}; + +const configToml = (requested: Partial>) => + [ + 'project_id = "stack-outdated"', + '[local.services.default]\nversion = "bundled"', + ...Object.entries(requested).map( + ([kind, version]) => `[local.services.${kind}]\nversion = "${version}"`, + ), + ].join("\n\n"); + +const project = (requested: Partial>) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-outdated-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString(`${root}/supabase/config.toml`, configToml(requested)); + return root; + }); + +const layer = (root: string, output: ReturnType) => + Layer.mergeAll( + BunServices.layer, + mockRuntimeInfo({ platform: "linux", arch: "x64" }), + output.layer, + mockTelemetryStateTracked().layer, + mockCommandSettings({ workdir: root }), + Layer.succeed(ExperimentalFlag, false), + ); + +const seed = (root: string, host: ReleaseFixture) => + withConfigEnv(releaseHostEnv(host), seedLock(root)).pipe( + Effect.provide(layer(root, mockOutput())), + ); + +const outdated = (root: string, host: ReleaseFixture, format: "text" | "json") => { + const output = mockOutput({ format }); + const run = withConfigEnv(releaseHostEnv(host), stackOutdated()).pipe( + Effect.provide(layer(root, output)), + ); + return { output, run }; +}; + +const lockPath = (root: string) => `${root}/supabase/supabase.lock`; +const leasePath = (root: string) => `${root}/supabase/.supabase.lock.lease`; + +describe("experimental stack outdated", () => { + it.live( + "reports a newer revision and a stale entry without writing, waiting on the lease or downloading", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM, rest: UPSTREAM }); + publish(host, "auth", 0); + publish(host, "rest", 0); + yield* seed(root, host); + publish(host, "auth", 1); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + configToml({ auth: UPSTREAM, rest: "v98.0.0" }), + ); + yield* holdLock(leasePath(root), "1 second", "test lock"); + const written = yield* fs.readFileString(lockPath(root)); + host.requests.length = 0; + + const { output, run } = outdated(root, host, "json"); + yield* run; + + const result = output.messages.find((entry) => entry.type === "success"); + const data = result?.type === "success" ? result.data : undefined; + expect(data).toMatchObject({ + file: "supabase/supabase.lock", + updatable: 1, + stale: 1, + failed: 0, + }); + expect(data).toMatchObject({ + services: expect.arrayContaining([ + { + service: "auth", + specifier: UPSTREAM, + resolution: "resolved", + locked: `${UPSTREAM}-r0`, + update: `${UPSTREAM}-r1`, + bundled: expect.stringMatching(/-r\d+$/u), + hosted: null, + stale: null, + error: null, + }, + expect.objectContaining({ + service: "rest", + update: null, + stale: expect.stringContaining("v98.0.0"), + }), + ]), + }); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(host.requests.filter((path) => path.endsWith(".tar.zst"))).toEqual([]); + expect(host.requests.filter((path) => path.includes("/releases/rest-"))).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("reports a transient release failure on its row and still succeeds", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM, rest: UPSTREAM }); + publish(host, "auth", 0); + publish(host, "rest", 0); + yield* seed(root, host); + publish(host, "rest", 1); + host.failing.set("/releases/auth-v99.0.0-r0/SHA256SUMS", 503); + + const { output, run } = outdated(root, host, "text"); + yield* run; + + expect(output.stdoutText).toContain("probe failed"); + expect(output.stdoutText).toContain("auth:"); + expect(output.stdoutText).toContain(`${UPSTREAM}-r1`); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("reports a release outage on a linked entry as a failed probe and keeps its pin", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM }); + publish(host, "auth", 0); + yield* seed(root, host); + yield* fs.writeFileString(`${root}/supabase/config.toml`, configToml({ auth: "linked" })); + const lock = yield* readLock(root).pipe(Effect.map(Option.getOrThrow)); + yield* writeLock(root, { + ...lock, + linked: { + project: "projectrefabcdefghijk", + branch: { ref: "projectrefabcdefghijk", isDefault: true }, + observedAt: "2026-01-01T00:00:00.000Z", + }, + services: { + ...lock.services, + auth: { + ...lock.services.auth, + specifier: "linked", + linked: { raw: UPSTREAM, version: UPSTREAM }, + }, + }, + }).pipe(Effect.orDie); + const written = yield* fs.readFileString(lockPath(root)); + host.failing.set("/releases/auth-v99.0.0-r0/SHA256SUMS", 503); + + const { output, run } = outdated(root, host, "json"); + yield* run; + + const result = output.messages.find((entry) => entry.type === "success"); + const data = result?.type === "success" ? result.data : undefined; + expect(data).toMatchObject({ + updatable: 0, + failed: 1, + services: expect.arrayContaining([ + expect.objectContaining({ + service: "auth", + specifier: "linked", + hosted: UPSTREAM, + locked: `${UPSTREAM}-r0`, + update: null, + error: expect.stringContaining("the slim-services release host could not be reached"), + }), + ]), + }); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails with guidance when the project has no lock yet", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({}); + + const error = yield* outdated(root, host, "text").run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandOutdatedError); + expect(error).toMatchObject({ reason: "lock" }); + expect(yield* fs.exists(lockPath(root))).toBe(false); + expect(yield* fs.exists(leasePath(root))).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/experimental/stack/prepare/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/prepare/SIDE_EFFECTS.md deleted file mode 100644 index d6021f9344..0000000000 --- a/apps/cli/src/commands/experimental/stack/prepare/SIDE_EFFECTS.md +++ /dev/null @@ -1,58 +0,0 @@ -# `supabase stack prepare` - -The command is available when the experimental stack family is enabled. It resolves the selected -project and stack, loads the effective project configuration, and asks the stack package to prepare -the selected service artifacts without starting or changing the saved composition. A new target gets -a managed descriptor so it can later be opened, started, listed, or destroyed. - -For each selected creation, the CLI creates a temporary standalone service instance in the target -namespace, runs that instance's `prepare` operation, and destroys that same instance in a scoped -finalizer. Existing composition members and unrelated standalone instances are never adopted or -modified. A preparation failure still runs cleanup for every acquired temporary instance. - -## Files read - -- `/supabase/config.toml`, project dotenv files, and supported `SUPABASE_*` overrides - used by the shared stack config loader. -- Managed stack state under `/stacks//` when opening an - existing target. -- Shared profile and telemetry state under `/`. - -## Files written - -- A managed stack descriptor/state record when the target does not exist. Existing records are - updated while temporary instances and their instance directories are created and removed. -- Runtime artifacts through the selected native or container runtime's package-owned cache. -- Telemetry state after success or failure once the handler starts. - -## Network and subprocesses - -The resident owner process may be launched and remains available after preparation; `stack stop` -shuts it down. No composition service is started. Native preparation may download runtime artifacts. Container preparation may pull images from the -configured registry and invoke the selected container engine. The stack package owns artifact -caching, preparation, and cancellation; the CLI owns the temporary instance cleanup boundary. - -## Flags and output - -`--stack` and `--stack-id` are mutually exclusive. `--runtime` defaults to `auto` for new stacks, -which selects the same runtime as `stack start` (Docker, then Podman, then native); -existing stacks reuse their persisted runtime without probing, and an explicit mismatch fails. With no -`--capability`, every enabled creation from the effective project configuration is prepared. -Repeated `--capability` includes each capability's configured companion services (Storage/Imgproxy, -Studio/Pgmeta); requesting a disabled service fails before instance preparation. The legacy `-o/--output` flag is rejected; use -`--output-format`. - -Text output lists the stack ID and each prepared service and version. JSON output returns the stack -ID and prepared capabilities. Errors retain typed actionability and package diagnostics. - -## Exit codes - -- `0` when all selected temporary instances prepare and are cleaned up. -- `130` when preparation is interrupted. -- `1` for invalid targets/configuration, unavailable runtime or registry, preparation failures, or - rejected output flags. - -## Telemetry - -The command emits the standard `cli_command_executed` event through command telemetry and flushes -telemetry state after successful and failed handler execution. It emits no custom event. diff --git a/apps/cli/src/commands/experimental/stack/prepare/prepare.command.ts b/apps/cli/src/commands/experimental/stack/prepare/prepare.command.ts deleted file mode 100644 index ab0e46b31b..0000000000 --- a/apps/cli/src/commands/experimental/stack/prepare/prepare.command.ts +++ /dev/null @@ -1,48 +0,0 @@ -import { Command, Flag } from "effect/cli"; -import type * as CliCommand from "effect/cli/Command"; -import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; -import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; -import { withCommandTelemetry } from "../../../../telemetry/command-telemetry.ts"; -import { stackPrepare } from "./prepare.handler.ts"; -import { STACK_PREPARABLE_CAPABILITIES } from "../start/start.options.ts"; - -const config = { - stack: Flag.String("stack").pipe(Flag.withDescription("Name this stack."), Flag.optional), - stackId: Flag.String("stack-id").pipe( - Flag.withDescription("Open an existing stack by id or unique id prefix."), - Flag.optional, - ), - runtime: Flag.Literals("runtime", ["auto", "docker", "podman", "native"] as const).pipe( - Flag.withDescription( - "Runtime to use for a new stack. auto selects Docker, then Podman, then native, based on what is available.", - ), - Flag.withDefault("auto" as const), - ), - capability: Flag.atMost( - Flag.Literals("capability", STACK_PREPARABLE_CAPABILITIES), - STACK_PREPARABLE_CAPABILITIES.length, - ).pipe(Flag.withDescription("Capability to prepare (repeatable).")), -} as const; - -export type StackPrepareFlags = CliCommand.Command.Config.Infer; - -export const stackPrepareCommand = Command.make("prepare", config).pipe( - Command.withDescription( - "Download artifacts for a managed local Supabase stack without starting its services. If the target does not exist, it is created and registered for later listing or destruction.", - ), - Command.withShortDescription("Prepare a managed local stack"), - Command.withExamples([ - { command: "supabase stack prepare", description: "Prepare all enabled stack capabilities" }, - { - command: "supabase stack prepare --stack feature-a --capability rest", - description: "Prepare one capability in a named stack", - }, - ]), - withPermissions({ - operations: [], - noApiEffectFlags: ["stack", "stack-id", "runtime", "capability"], - }), - Command.withHandler((flags) => - stackPrepare(flags).pipe(withCommandTelemetry({ flags, config }), withJsonErrorHandling), - ), -); diff --git a/apps/cli/src/commands/experimental/stack/prepare/prepare.errors.ts b/apps/cli/src/commands/experimental/stack/prepare/prepare.errors.ts deleted file mode 100644 index 5d149fe83f..0000000000 --- a/apps/cli/src/commands/experimental/stack/prepare/prepare.errors.ts +++ /dev/null @@ -1,39 +0,0 @@ -import type { StackError } from "@supabase/stack/effect"; -import { Data } from "effect"; -import { - actionability, - causeDeclaration, - type CliErrorActionabilityDeclaration, - ErrorActionabilityId, - unclassifiedStackFailureActionability, -} from "../../../../shared/telemetry/error-actionability.ts"; - -export class StackCommandPrepareError extends Data.TaggedError("ExperimentalStackPrepareError")<{ - readonly reason: "flags" | "invalid-config" | "runtime" | "lifecycle" | "stack"; - readonly message: string; - readonly suggestion?: string; - readonly cause?: unknown; -}> { - get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { - switch (this.reason) { - case "flags": - return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; - case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; - case "runtime": - return { ...actionability.dockerNotRunning, fingerprint_suffix: "docker_not_running" }; - case "lifecycle": - return { ...actionability.invalidConfig, fingerprint_suffix: "lifecycle" }; - case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; - } - } -} - -/** Maps an internal stack failure to the prepare command's stable error boundary. */ -export const stackPrepareError = (error: StackError): StackCommandPrepareError => - new StackCommandPrepareError({ - reason: "stack", - message: error.message, - cause: error, - }); diff --git a/apps/cli/src/commands/experimental/stack/prepare/prepare.handler.ts b/apps/cli/src/commands/experimental/stack/prepare/prepare.handler.ts deleted file mode 100644 index 4bbbb88ca1..0000000000 --- a/apps/cli/src/commands/experimental/stack/prepare/prepare.handler.ts +++ /dev/null @@ -1,185 +0,0 @@ -import { Cause, Effect, Exit, Option, Path } from "effect"; -import { Output } from "../../../../shared/output/output.service.ts"; -import { OutputFlag } from "../../../../command-internal/global-flags.ts"; -import { CommandSettings } from "../../../../config/command-settings.service.ts"; -import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; -import { loadStackConfig } from "../../../../command-internal/stack-config.ts"; -import { - StackApi, - stackCapabilityForService, - StackTargetError, - rejectStackOutput, - StackTargetResolver, - validateStackTarget, -} from "../stack.shared.ts"; -import type { StackPrepareFlags } from "./prepare.command.ts"; -import { StackCommandPrepareError, stackPrepareError } from "./prepare.errors.ts"; -import { - automaticRuntimeNotice, - selectStackRuntime, -} from "../../../../command-internal/stack-runtime.ts"; - -type PreparedCapability = { - readonly capability: string; - readonly version?: string; - readonly outcome: "prepared"; -}; - -const mapTargetError = (error: StackTargetError) => - new StackCommandPrepareError({ - reason: error.reason, - message: error.message, - ...(error.suggestion === undefined ? {} : { suggestion: error.suggestion }), - cause: error, - }); - -const render = (id: string, capabilities: ReadonlyArray) => { - const lines = [`Stack ${id} prepared.`]; - if (capabilities.length > 0) { - lines.push("Capabilities:"); - for (const capability of capabilities) - lines.push( - ` ${capability.capability} ${capability.version ?? "catalog default"} (${capability.outcome})`, - ); - } - return `${lines.join("\n")}\n`; -}; - -/** Prepares selected stack artifacts without starting service processes. */ -export const stackPrepare = Effect.fn("experimental.stack.prepare")(function* ( - flags: StackPrepareFlags, -) { - const telemetryState = yield* TelemetryState; - const body = Effect.gen(function* () { - const output = yield* Output; - const settings = yield* CommandSettings; - const path = yield* Path.Path; - const resolver = yield* StackTargetResolver; - const api = yield* StackApi; - const outputFlag = yield* Effect.serviceOption(OutputFlag); - yield* rejectStackOutput(outputFlag).pipe(Effect.mapError(mapTargetError)); - yield* validateStackTarget({ - stack: Option.getOrUndefined(flags.stack), - stackId: Option.getOrUndefined(flags.stackId), - }).pipe(Effect.mapError(mapTargetError)); - const target = yield* resolver - .resolve({ - projectRoot: settings.workdir, - ...(Option.isSome(flags.stack) ? { name: flags.stack.value } : {}), - ...(Option.isSome(flags.stackId) ? { id: flags.stackId.value } : {}), - runtime: flags.runtime, - }) - .pipe(Effect.mapError(mapTargetError)); - const config = yield* loadStackConfig(target.projectRoot).pipe( - Effect.mapError( - (error) => - new StackCommandPrepareError({ - reason: "invalid-config", - message: error.message, - cause: error, - }), - ), - ); - const stateRoot = path.join(settings.supabaseHome, "stacks"); - const cacheRoot = path.join(settings.supabaseHome, "cache", "stack"); - const createStack = Effect.gen(function* () { - const runtime = yield* selectStackRuntime(target.runtime).pipe( - Effect.mapError( - (error) => - new StackCommandPrepareError({ - reason: error.reason === "native-unsupported" ? "flags" : "runtime", - message: error.message, - suggestion: error.suggestion, - cause: error, - }), - ), - ); - const created = yield* api - .create({ - projectRoot: target.projectRoot, - stateRoot, - cacheRoot, - runtime, - ...(target.name === undefined ? {} : { name: target.name }), - }) - .pipe(Effect.mapError(stackPrepareError)); - const notice = automaticRuntimeNotice(target.runtime, runtime); - if (notice !== undefined) yield* output.info(notice); - return created; - }); - const stack = - target.id === undefined - ? yield* createStack - : yield* api - .open({ id: target.id, stateRoot, cacheRoot }) - .pipe(Effect.mapError(stackPrepareError)); - const creations = yield* config.creations(stack.id).pipe( - Effect.mapError( - (error) => - new StackCommandPrepareError({ - reason: "invalid-config", - message: error.message, - cause: error, - }), - ), - ); - const selectedServices = flags.capability.length === 0 ? undefined : new Set(flags.capability); - for (const capability of flags.capability) { - if (!creations.some(({ service }) => stackCapabilityForService(service) === capability)) - return yield* new StackCommandPrepareError({ - reason: "invalid-config", - message: `Service ${capability} is disabled in the project configuration`, - suggestion: "Enable the service in supabase/config.toml before preparing it.", - }); - } - const requested = - selectedServices === undefined - ? creations - : creations.filter(({ service }) => - selectedServices.has(stackCapabilityForService(service)), - ); - yield* Effect.annotateCurrentSpan({ "stack.capability_count": requested.length }); - const task = yield* output.task("Preparing local Supabase stack..."); - const capabilities = yield* Effect.forEach(requested, (creation) => - Effect.acquireUseRelease( - stack.services.create(creation).pipe(Effect.mapError(stackPrepareError)), - (instance) => - instance.prepare.pipe(Effect.mapError(stackPrepareError), Effect.as(creation)), - (instance) => - instance.destroy.pipe( - Effect.mapError( - (cause) => - new StackCommandPrepareError({ - reason: "lifecycle", - message: `Failed to remove temporary ${instance.service} instance ${instance.id}: ${cause.message}`, - suggestion: `Inspect stack ${stack.id}; run supabase stack destroy --stack-id ${stack.id} to remove it if no existing data must be retained.`, - cause, - }), - ), - ), - ), - ).pipe( - Effect.onExit((exit) => - Exit.isSuccess(exit) - ? task.clear - : Option.match(Cause.findErrorOption(exit.cause), { - onNone: () => (Cause.hasInterruptsOnly(exit.cause) ? task.cancel() : task.fail()), - onSome: (error) => task.fail(error.message), - }), - ), - ); - const prepared = capabilities.map((creation): PreparedCapability => ({ - capability: creation.service, - ...(creation.service === "database" - ? { version: creation.config.version } - : creation.version === undefined - ? {} - : { version: creation.version }), - outcome: "prepared", - })); - if (output.format === "text") yield* output.raw(render(stack.id, prepared)); - else yield* output.success("", { id: stack.id, capabilities: prepared }); - return prepared; - }); - return yield* body.pipe(Effect.ensuring(telemetryState.flush)); -}); diff --git a/apps/cli/src/commands/experimental/stack/prepare/prepare.integration.test.ts b/apps/cli/src/commands/experimental/stack/prepare/prepare.integration.test.ts deleted file mode 100644 index 4589a2e673..0000000000 --- a/apps/cli/src/commands/experimental/stack/prepare/prepare.integration.test.ts +++ /dev/null @@ -1,483 +0,0 @@ -import { BunServices } from "@effect/platform-bun"; -import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Option, Path, Redacted, Stream } from "effect"; -import { - StackError, - type Stack, - type ServiceCreationInput, - type ServiceInstances, -} from "@supabase/stack/effect"; -import { runtimeInfoLayer } from "../../../../shared/runtime/runtime-info.layer.ts"; -import type { RuntimeInfo } from "../../../../shared/runtime/runtime-info.service.ts"; -import { - StackRuntimeSelectionError, - type StackRuntime, -} from "../../../../command-internal/stack-runtime.ts"; -import { - containerEngineSpawner, - type ContainerEngineState, -} from "../../../../../tests/helpers/child-process-spawner.ts"; -import { unusedGateway } from "../../../../../tests/helpers/unused-stack.ts"; -import { - mockCommandSettings, - mockTelemetryStateTracked, -} from "../../../../../tests/helpers/command-mocks.ts"; -import { mockOutput, mockRuntimeInfo } from "../../../../../tests/helpers/mocks.ts"; -import { CommandTelemetryAttributes } from "../../../../telemetry/command-telemetry-attributes.ts"; -import { StackApi, StackTargetResolver } from "../stack.shared.ts"; -import { stackPrepare } from "./prepare.handler.ts"; -import type { StackPrepareFlags } from "./prepare.command.ts"; -import { StackCommandPrepareError } from "./prepare.errors.ts"; - -const id = "a".repeat(64); -const flags = (overrides: Partial = {}): StackPrepareFlags => ({ - stack: Option.none(), - stackId: Option.none(), - runtime: "native", - capability: ["database"], - ...overrides, -}); - -const makeProject = (config: string) => - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-prepare-" }); - yield* fs.makeDirectory(path.join(root, "supabase")); - yield* fs.writeFileString(path.join(root, "supabase", "config.toml"), config); - return root; - }); - -interface FixtureOptions { - readonly failPreparation?: boolean; - readonly savedRuntime?: StackRuntime; - readonly engines?: { - readonly docker: ContainerEngineState; - readonly podman: ContainerEngineState; - }; - readonly runtimeInfo?: Layer.Layer; -} - -const makeFixture = (root: string, options: FixtureOptions = {}) => { - const { failPreparation = false } = options; - const engines = containerEngineSpawner( - options.engines ?? { docker: "missing", podman: "missing" }, - ); - const createdRuntimes: Array = []; - const recordedRuntimes: Array = []; - let openCount = 0; - let prepareCount = 0; - let startCount = 0; - let destroyCount = 0; - const database = { - id: "database-id", - service: "database" as const, - start: Effect.sync(() => { - startCount += 1; - }), - ready: Effect.void, - stop: Effect.void, - restart: () => Effect.void, - destroy: Effect.sync(() => { - destroyCount += 1; - }), - prepare: Effect.sync(() => { - prepareCount += 1; - }).pipe( - Effect.andThen(() => - failPreparation - ? Effect.fail( - new StackError({ - operation: "prepareService", - message: "download failed", - kind: "artifact-download", - }), - ) - : Effect.void, - ), - ), - status: Effect.succeed({ - id: "database-id", - endpoints: [], - config: { - service: "database" as const, - config: { - version: "17", - databasePassword: Redacted.make("password"), - jwtSecret: Redacted.make("jwt-secret"), - jwtExpiry: 3600, - }, - }, - lifecycle: "stopped" as const, - health: undefined, - error: undefined, - exit: undefined, - currentOperation: undefined, - wakeEnabled: false, - }), - followStatus: Stream.empty, - readLogs: () => Stream.empty, - credentials: () => Effect.succeed({}), - saveSnapshot: () => Effect.die("unused"), - restoreSnapshot: () => Effect.die("unused"), - resetData: Effect.die("unused"), - }; - function create( - creation: Input, - ): Effect.Effect; - function create( - creation: ServiceCreationInput, - ): Effect.Effect { - switch (creation.service) { - case "database": - return Effect.succeed(database); - case "studio": - return Effect.succeed({ ...database, service: "studio" as const }); - case "pgmeta": - return Effect.succeed({ ...database, service: "pgmeta" as const }); - default: - return Effect.die(`fixture does not support ${creation.service}`); - } - } - - const stack: Stack = { - id, - services: { - create, - get: () => Effect.succeed(database), - list: Effect.succeed([]), - }, - credentials: { get: Effect.die("unused") }, - composition: { - plan: () => Effect.succeed([]), - supabase: () => Effect.die("prepare must not change the composition"), - configure: () => Effect.void, - describe: Effect.succeed({ - members: [], - dependencies: [], - }), - start: Effect.succeed([]), - stop: Effect.succeed([]), - restart: Effect.succeed([]), - }, - startupEndpointChanges: Effect.succeed([]), - stop: Effect.void, - destroy: Effect.void, - gateway: unusedGateway, - commands: { run: () => Effect.die("unused") }, - }; - const output = mockOutput(); - const telemetry = mockTelemetryStateTracked(); - const layer = Layer.mergeAll( - BunServices.layer, - options.runtimeInfo ?? runtimeInfoLayer, - engines.layer, - mockCommandSettings({ workdir: root, supabaseHome: root }), - output.layer, - telemetry.layer, - Layer.succeed(CommandTelemetryAttributes, { - record: (values) => - Effect.sync(() => { - if (values.stack_runtime !== undefined) recordedRuntimes.push(values.stack_runtime); - }), - }), - Layer.succeed(StackTargetResolver, { - resolve: (input) => - Effect.succeed({ - projectRoot: root, - hostRunning: false, - ...(options.savedRuntime === undefined - ? input.runtime === "auto" - ? {} - : { runtime: input.runtime } - : { id, runtime: options.savedRuntime }), - }), - }), - Layer.succeed(StackApi, { - create: (input) => - Effect.sync(() => { - createdRuntimes.push(input.runtime); - return stack; - }), - open: () => - Effect.sync(() => { - openCount += 1; - return stack; - }), - discover: () => Effect.succeed([]), - find: () => Effect.die("unused"), - findDeleted: () => Effect.die("unused"), - }), - ); - return { - layer, - get createdRuntimes() { - return createdRuntimes; - }, - get recordedRuntimes() { - return recordedRuntimes; - }, - get openCount() { - return openCount; - }, - get runtimeNotices() { - return output.messages - .filter( - ({ type, message }) => type === "info" && message?.startsWith("Docker didn't answer"), - ) - .map(({ message }) => message); - }, - get probes() { - return [...new Set(engines.spawned.map(({ command }) => command))]; - }, - get prepareCount() { - return prepareCount; - }, - get startCount() { - return startCount; - }, - get destroyCount() { - return destroyCount; - }, - }; -}; - -const databaseOnlyConfig = `project_id = "prepare-test"\n\n[api]\nenabled = false\n`; - -describe("stack prepare", () => { - it.effect("prepares a temporary Database without changing the composition", () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root); - return stackPrepare(flags()).pipe( - Effect.provide(fixture.layer), - Effect.tap((prepared) => - Effect.sync(() => { - expect(prepared.length).toBeGreaterThan(0); - expect(fixture.prepareCount).toBe(prepared.length); - expect(fixture.startCount).toBe(0); - expect(fixture.destroyCount).toBe(prepared.length); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); - - it.effect("prepares Studio together with its configured Pgmeta companion", () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root); - return stackPrepare(flags({ capability: ["studio"] })).pipe( - Effect.provide(fixture.layer), - Effect.tap((prepared) => - Effect.sync(() => { - expect(prepared.map((result) => result.capability).sort()).toEqual([ - "pgmeta", - "studio", - ]); - expect(prepared.every((result) => result.version !== "catalog default")).toBe(true); - expect(fixture.prepareCount).toBe(2); - expect(fixture.destroyCount).toBe(2); - expect(fixture.startCount).toBe(0); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); - - it.effect("removes its temporary instance after a preparation failure", () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root, { failPreparation: true }); - return stackPrepare(flags()).pipe( - Effect.provide(fixture.layer), - Effect.flip, - Effect.tap((error) => - Effect.sync(() => { - expect(error.message).toContain("download failed"); - expect(error.reason).toBe("stack"); - expect(fixture.destroyCount).toBe(1); - expect(fixture.startCount).toBe(0); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); - - it.effect("rejects a disabled requested service before preparing an instance", () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root); - return stackPrepare(flags({ capability: ["rest"] })).pipe( - Effect.provide(fixture.layer), - Effect.flip, - Effect.tap((error) => - Effect.sync(() => { - expect(error.message).toContain("rest is disabled"); - expect(fixture.prepareCount).toBe(0); - expect(fixture.destroyCount).toBe(0); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); - - it.effect("rejects malformed config before creating a stack", () => - makeProject("project_id = [\n").pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root); - return stackPrepare(flags()).pipe( - Effect.provide(fixture.layer), - Effect.flip, - Effect.tap((error) => - Effect.sync(() => { - expect(error).toBeInstanceOf(StackCommandPrepareError); - expect(error.reason).toBe("invalid-config"); - expect(fixture.prepareCount).toBe(0); - expect(fixture.startCount).toBe(0); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); -}); - -describe("stack prepare automatic runtime selection", () => { - const selectRuntime = (options: FixtureOptions) => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root, options); - return stackPrepare(flags({ runtime: "auto" })).pipe( - Effect.provide(fixture.layer), - Effect.as(fixture), - ); - }), - Effect.provide(BunServices.layer), - ); - - it.effect("creates a Docker stack when the Docker engine answers", () => - selectRuntime({ engines: { docker: "running", podman: "running" } }).pipe( - Effect.tap((fixture) => - Effect.sync(() => { - expect(fixture.createdRuntimes).toEqual(["docker"]); - expect(fixture.probes).toEqual(["docker"]); - expect(fixture.runtimeNotices).toEqual([]); - }), - ), - ), - ); - - it.effect("creates a Podman stack when Docker is not installed and Podman answers", () => - selectRuntime({ - engines: { docker: "missing", podman: "running" }, - runtimeInfo: mockRuntimeInfo({ platform: "linux", arch: "x64" }), - }).pipe( - Effect.tap((fixture) => - Effect.sync(() => { - expect(fixture.createdRuntimes).toEqual(["podman"]); - expect(fixture.probes).toEqual(["docker", "podman"]); - expect(fixture.runtimeNotices).toHaveLength(1); - expect(fixture.runtimeNotices[0]).toContain("uses the Podman runtime"); - expect(fixture.runtimeNotices[0]).toContain("supabase stack destroy"); - }), - ), - ), - ); - - it.effect( - "creates a native stack on Linux when the Docker daemon is down and Podman is absent", - () => - selectRuntime({ - engines: { docker: "stopped", podman: "missing" }, - runtimeInfo: mockRuntimeInfo({ platform: "linux", arch: "x64" }), - }).pipe( - Effect.tap((fixture) => - Effect.sync(() => { - expect(fixture.createdRuntimes).toEqual(["native"]); - expect(fixture.probes).toEqual(["docker", "podman"]); - expect(fixture.runtimeNotices).toHaveLength(1); - expect(fixture.runtimeNotices[0]).toContain("uses the native runtime"); - }), - ), - ), - ); - - for (const [platform, arch] of [ - ["win32", "x64"], - ["darwin", "x64"], - ] as const) { - it.effect(`fails without creating a stack when no engine answers on ${platform}/${arch}`, () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root, { - engines: { docker: "stopped", podman: "stopped" }, - runtimeInfo: mockRuntimeInfo({ platform, arch }), - }); - return stackPrepare(flags({ runtime: "auto" })).pipe( - Effect.provide(fixture.layer), - Effect.flip, - Effect.tap((error) => - Effect.sync(() => { - expect(error).toBeInstanceOf(StackCommandPrepareError); - expect(error.reason).toBe("runtime"); - expect(error.cause).toBeInstanceOf(StackRuntimeSelectionError); - expect(error.message).toContain(`not supported on ${platform}/${arch}`); - expect(error.suggestion).toBe("Start Docker or Podman, then rerun the command."); - expect(fixture.createdRuntimes).toEqual([]); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); - } - - it.effect("reuses a saved runtime without probing any engine", () => - selectRuntime({ - savedRuntime: "podman", - engines: { docker: "running", podman: "running" }, - }).pipe( - Effect.tap((fixture) => - Effect.sync(() => { - expect(fixture.openCount).toBe(1); - expect(fixture.createdRuntimes).toEqual([]); - expect(fixture.probes).toEqual([]); - expect(fixture.runtimeNotices).toEqual([]); - }), - ), - ), - ); - - it.effect( - "honors an explicit runtime without probing any engine and records it on telemetry", - () => - makeProject(databaseOnlyConfig).pipe( - Effect.flatMap((root) => { - const fixture = makeFixture(root, { engines: { docker: "running", podman: "running" } }); - return stackPrepare(flags({ runtime: "podman" })).pipe( - Effect.provide(fixture.layer), - Effect.tap(() => - Effect.sync(() => { - expect(fixture.createdRuntimes).toEqual(["podman"]); - expect(fixture.recordedRuntimes).toEqual(["podman"]); - expect(fixture.probes).toEqual([]); - expect(fixture.runtimeNotices).toEqual([]); - }), - ), - ); - }), - Effect.provide(BunServices.layer), - ), - ); -}); diff --git a/apps/cli/src/commands/experimental/stack/restart/restart.integration.test.ts b/apps/cli/src/commands/experimental/stack/restart/restart.integration.test.ts index 24537493a0..90c1898a6e 100644 --- a/apps/cli/src/commands/experimental/stack/restart/restart.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/restart/restart.integration.test.ts @@ -12,6 +12,7 @@ import { StackApi, stackApiLayer, stackTargetResolverLayer } from "../stack.shar import { stackRestart } from "./restart.handler.ts"; import { destroyTestStack } from "../../../../../tests/helpers/stack-cleanup.ts"; import { stackArtifactCacheRoot } from "../../../../../tests/helpers/stack-artifacts.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; const live = Layer.provideMerge( stackApiLayer, @@ -85,11 +86,13 @@ describe("stack restart", () => { Effect.gen(function* () { const member = yield* stack.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); const standalone = yield* stack.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); diff --git a/apps/cli/src/commands/experimental/stack/stack-backend.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-backend.integration.test.ts index 7c82c1835c..bca56c0cf7 100644 --- a/apps/cli/src/commands/experimental/stack/stack-backend.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-backend.integration.test.ts @@ -1,16 +1,22 @@ import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Cause, Effect, Exit, FileSystem, Option, Path } from "effect"; +import { ExperimentalFeatureFlagError } from "../../../command-internal/experimental-feature.ts"; +import { readLinkSnapshot } from "../../../command-internal/link-snapshot.ts"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; import { respondToComplete } from "../../../cli/complete.ts"; import { rootCommandForFeatures } from "../../../cli/root.ts"; import { type StackBackend, StackRoutingError, - resolveStackBackend, + resolveProjectStateSelection, } from "../../../command-internal/stack-backend.ts"; -const resolve = (input: Parameters[0]) => - resolveStackBackend(input).pipe(Effect.provide(BunServices.layer)); +const select = (input: Parameters[0]) => + resolveProjectStateSelection(input).pipe(Effect.provide(BunServices.layer)); + +const resolve = (input: Parameters[0]) => + select(input).pipe(Effect.map(({ stackBackend }) => stackBackend)); const project = (config: string, format: "toml" | "json" = "toml") => Effect.gen(function* () { @@ -32,7 +38,7 @@ const completionFlags = (backend: StackBackend | undefined, command: string) => "--", ])?.candidates.map(({ name }) => name); -describe("resolveStackBackend", () => { +describe("resolveProjectStateSelection", () => { it.effect("keeps the explicit stack namespace disabled without the feature flag", () => Effect.gen(function* () { expect(yield* resolve({ args: ["stack", "start"], cwd: "/missing", env: {} })).toBe("legacy"); @@ -216,6 +222,44 @@ stack = true ); }); + it.effect("keeps an invalid environment value from failing an unrouted command", () => + withServices( + Effect.gen(function* () { + const root = yield* project("[experimental]\nstack = true\n"); + const selection = yield* select({ + args: ["login"], + cwd: root, + env: { SUPABASE_EXPERIMENTAL_STACK: "garbage" }, + }); + expect(selection.stackBackend).toBeUndefined(); + expect(selection.projectState).toBeDefined(); + }), + ), + ); + + it.effect("fails a link-state read with the typed error once the mode is invalid", () => + withServices( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* project("[experimental]\nstack = true\n"); + const selection = yield* select({ + args: ["link"], + cwd: root, + env: { SUPABASE_EXPERIMENTAL_STACK: "garbage" }, + }); + if (selection.projectState === undefined) return yield* Effect.die("no selection"); + + const read = yield* readLinkSnapshot(fs, path, root).pipe( + Effect.map((snapshot) => snapshot.ref), + Effect.provide(projectStateModeLayer(selection.projectState)), + Effect.flip, + ); + expect(read).toBeInstanceOf(ExperimentalFeatureFlagError); + }), + ), + ); + it.effect("honors explicit workdir and separated global boolean values", () => { return withServices( Effect.gen(function* () { @@ -360,7 +404,18 @@ stack = true ])?.candidates.map(({ name }) => name); expect(stackCommands).toEqual( backend === "stack" - ? ["destroy", "list", "logs", "prepare", "restart", "start", "status", "stop"] + ? [ + "destroy", + "install", + "list", + "logs", + "outdated", + "restart", + "start", + "status", + "stop", + "update", + ] : [], ); } diff --git a/apps/cli/src/commands/experimental/stack/stack-command-telemetry.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-command-telemetry.integration.test.ts index 0aabefa418..2927d798ed 100644 --- a/apps/cli/src/commands/experimental/stack/stack-command-telemetry.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-command-telemetry.integration.test.ts @@ -140,26 +140,6 @@ describe("stack command telemetry", () => { ); }); - it.effect("records the prepare command identity on invalid target input", () => { - return setup().pipe( - Effect.flatMap((fixture) => { - const command = stackCommand.pipe(Command.provide(fixture.layer)); - return Effect.gen(function* () { - yield* Command.runWith(command, { version: "0.0.0-test" })([ - "prepare", - "--stack-id", - "invalid", - ]).pipe(Effect.flip); - const event = fixture.analytics.captured.find( - (candidate) => candidate.event === EventCommandExecuted, - ); - expect(event?.properties[PropCommand]).toBe("stack prepare"); - }).pipe(Effect.provide(fixture.layer)); - }), - Effect.provide(BunServices.layer), - ); - }); - it.effect("records the logs command identity when no stack exists", () => { return setup().pipe( Effect.flatMap((fixture) => { diff --git a/apps/cli/src/commands/experimental/stack/stack-config-environment.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-config-environment.integration.test.ts index cb84a2c98f..d528e0809d 100644 --- a/apps/cli/src/commands/experimental/stack/stack-config-environment.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-config-environment.integration.test.ts @@ -5,7 +5,7 @@ import type { ServiceCreationInput } from "@supabase/stack/effect"; import { orioledbVersions } from "@supabase/stack/internal/artifacts"; import { runtimeInfoLayer } from "../../../shared/runtime/runtime-info.layer.ts"; -import { withEnvVar } from "../../../../tests/helpers/command-mocks.ts"; +import { mockCommandSettings, withEnvVar } from "../../../../tests/helpers/command-mocks.ts"; import { loadStackConfig } from "../../../command-internal/stack-config.ts"; import { createStackConfigProject } from "../../../../tests/helpers/stack-config.ts"; @@ -20,7 +20,13 @@ const project = ( const load = (projectRoot: string) => loadStackConfig(projectRoot).pipe( - Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer)), + Effect.provide( + Layer.mergeAll( + BunServices.layer, + runtimeInfoLayer, + mockCommandSettings({ workdir: projectRoot }), + ), + ), ); const withEnvironment = ( diff --git a/apps/cli/src/commands/experimental/stack/stack-config.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-config.integration.test.ts index 553b3c03be..e483585608 100644 --- a/apps/cli/src/commands/experimental/stack/stack-config.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-config.integration.test.ts @@ -7,8 +7,15 @@ import { importJWK, jwtVerify } from "jose"; import { ServiceCreationInput } from "../../../../../../packages/stack/src/services/Catalog.ts"; import { runtimeInfoLayer } from "../../../shared/runtime/runtime-info.layer.ts"; import { CLI_VERSION } from "../../../shared/cli/version.ts"; +import { + database15, + orioledb, + orioledbConfigVersion, + selection, +} from "../../../../tests/helpers/pins.ts"; import { renderCliConfigTemplate } from "../../../shared/init/project-init.templates.ts"; +import { mockCommandSettings } from "../../../../tests/helpers/command-mocks.ts"; import { loadStackConfig } from "../../../command-internal/stack-config.ts"; import { createStackConfigProject } from "../../../../tests/helpers/stack-config.ts"; import { @@ -18,7 +25,13 @@ import { const load = (projectRoot: string) => loadStackConfig(projectRoot).pipe( - Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer)), + Effect.provide( + Layer.mergeAll( + BunServices.layer, + runtimeInfoLayer, + mockCommandSettings({ workdir: projectRoot }), + ), + ), ); const project = (contents: string, options: Parameters[1] = {}) => @@ -49,6 +62,30 @@ describe("loadStackConfig", () => { vi.unstubAllEnvs(); }); + it.effect( + "pins the database to the line selected by [db] major_version or orioledb_version", + () => + Effect.gen(function* () { + const databasePin = (contents: string, name: string) => + Effect.gen(function* () { + const root = yield* project(contents); + const config = yield* load(root); + const creations = yield* config.creations(name); + return creations.find((creation) => creation.service === "database")?.pin; + }); + + expect( + yield* databasePin('project_id = "pin-15"\n[db]\nmajor_version = 15\n', "pin-15"), + ).toEqual(database15); + expect( + yield* databasePin( + `project_id = "pin-oriole"\n[db]\norioledb_version = "${orioledbConfigVersion}"\n`, + "pin-oriole", + ), + ).toEqual(orioledb); + }).pipe(Effect.scoped), + ); + it.effect("decodes the default recipe and leaves listeners automatic", () => Effect.gen(function* () { const root = yield* project(`project_id = "stack-config-defaults" @@ -66,7 +103,7 @@ enabled = true expect(keys.gotrueJwtKeys).toBeUndefined(); expect(keys.publicSigningKeys).toBeUndefined(); const database = recipes.get("database"); - expect(database?.service === "database" ? database.config.version : undefined).toBe("17"); + for (const service of services) expect(service.pin).toEqual(selection[service.service]); expect( database?.service === "database" ? database.config.rootKey : undefined, ).toBeUndefined(); diff --git a/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts index e9cf18c46c..8631177d53 100644 --- a/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-forwarding.integration.test.ts @@ -8,11 +8,16 @@ import { makeSpec as poolerSpec } from "../../../../../../packages/stack/src/ser import { makeSpec as realtimeSpec } from "../../../../../../packages/stack/src/services/Realtime.ts"; import { makeSpec as storageSpec } from "../../../../../../packages/stack/src/services/Storage.ts"; import { makeSpec as studioSpec } from "../../../../../../packages/stack/src/services/Studio.ts"; +import { mockCommandSettings } from "../../../../tests/helpers/command-mocks.ts"; import { loadStackConfig } from "../../../command-internal/stack-config.ts"; import { runtimeInfoLayer } from "../../../shared/runtime/runtime-info.layer.ts"; import { createStackConfigProject } from "../../../../tests/helpers/stack-config.ts"; -const layer = Layer.merge(BunServices.layer, runtimeInfoLayer); +const layer = Layer.mergeAll( + BunServices.layer, + runtimeInfoLayer, + mockCommandSettings({ workdir: "/unused-stack-forwarding-workdir" }), +); describe("stack service configuration", () => { it.effect("carries editable TOML settings into native and container service environments", () => diff --git a/apps/cli/src/commands/experimental/stack/stack.command.ts b/apps/cli/src/commands/experimental/stack/stack.command.ts index 7665e0366e..c2f54790e7 100644 --- a/apps/cli/src/commands/experimental/stack/stack.command.ts +++ b/apps/cli/src/commands/experimental/stack/stack.command.ts @@ -17,7 +17,9 @@ import { stackDestroyCommand as stackDestroyCommandBase } from "./destroy/destro import { stackLogsCommand as stackLogsCommandBase } from "./logs/logs.command.ts"; import { stackListCommand as stackListCommandBase } from "./list/list.command.ts"; import { stackRestartCommand as stackRestartCommandBase } from "./restart/restart.command.ts"; -import { stackPrepareCommand as stackPrepareCommandBase } from "./prepare/prepare.command.ts"; +import { stackInstallCommand as stackInstallCommandBase } from "./install/install.command.ts"; +import { stackOutdatedCommand as stackOutdatedCommandBase } from "./outdated/outdated.command.ts"; +import { stackUpdateCommand as stackUpdateCommandBase } from "./update/update.command.ts"; import { stackApiLayer, stackTargetResolverLayer } from "./stack.shared.ts"; import { configureLoopbackProxyBypass } from "../../../command-internal/hostname.ts"; @@ -81,8 +83,15 @@ const stackListCommand = stackListCommandBase.pipe( const stackRestartCommand = stackRestartCommandBase.pipe( Command.provide(commandRuntimeLayer(["stack", "restart"])), ); -const stackPrepareCommand = stackPrepareCommandBase.pipe( - Command.provide(commandRuntimeLayer(["stack", "prepare"])), +const stackInstallCommand = stackInstallCommandBase.pipe( + Command.provide(commandRuntimeLayer(["stack", "install"])), +); +const stackOutdatedCommand = stackOutdatedCommandBase.pipe( + Command.provide(commandRuntimeLayer(["stack", "outdated"])), +); +const stackUpdateCommand = stackUpdateCommandBase.pipe( + Command.provide(commandRuntimeLayer(["stack", "update"])), + Command.provide(stackStartRuntimeLayer), ); export const stackCommand = Command.make("stack").pipe( @@ -92,13 +101,15 @@ export const stackCommand = Command.make("stack").pipe( Command.withShortDescription("Manage experimental local stacks"), Command.withSubcommands([ stackDestroyCommand, + stackInstallCommand, stackListCommand, stackLogsCommand, - stackPrepareCommand, + stackOutdatedCommand, stackRestartCommand, stackStartCommand, stackStatusCommand, stackStopCommand, + stackUpdateCommand, ]), Command.provide(stackRuntimeLayer), ); diff --git a/apps/cli/src/commands/experimental/stack/stack.flags.ts b/apps/cli/src/commands/experimental/stack/stack.flags.ts new file mode 100644 index 0000000000..0ca89073a1 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/stack.flags.ts @@ -0,0 +1,9 @@ +import { Flag } from "effect/cli"; + +/** Fails instead of resolving when `supabase/supabase.lock` is missing or stale. */ +export const frozenFlag = Flag.Boolean("frozen").pipe( + Flag.withDescription( + "Fail instead of updating supabase/supabase.lock when it is missing or out of date with config.toml. Implied when CI is set and the lock exists.", + ), + Flag.withDefault(false), +); diff --git a/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md index f8ad598caa..646341ec16 100644 --- a/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/start/SIDE_EFFECTS.md @@ -32,15 +32,16 @@ credentials when present. Starting with Studio creates `supabase/snippets/`, whe snippets. Email template `content_path` values and third-party identity providers remain unsupported: they require template serving and shared external JWKS verification respectively. -`db.orioledb_version` (or `SUPABASE_DB_ORIOLEDB_VERSION`) runs the catalog's OrioleDB build of that -version instead of the stock major. Loading fails unless the catalog pins that OrioleDB version and -`db.major_version` matches its major; the error lists the pinned OrioleDB versions. The -`experimental.s3_*` OrioleDB settings are not forwarded, so loading fails when they are set. +`db.orioledb_version` (or `SUPABASE_DB_ORIOLEDB_VERSION`) is a `local.services.database.version` +request for that OrioleDB build instead of the stock major, resolved like any other version (see +"Service versions and the lock"); the two keys must agree when both are set, and `db.major_version` +must match the build's major. The `experimental.s3_*` OrioleDB settings are not forwarded, so +loading fails when they are set. Secrets needed by enabled services are passed to the runtime. State and service data live under `$SUPABASE_HOME/stacks//` (`~/.supabase/stacks//` by default); native artifacts use `$SUPABASE_HOME/cache/stack`, keyed by content digest and retired automatically once unused for -30 days. Storage files use the caller-owned project directory `supabase/.temp/stack-uploads//`. +30 days. Storage files use `/stacks//uploads/`; the command reads and writes nothing under `supabase/.temp`. Functions preparation may build the project's source. The owner persists each service's output under `$SUPABASE_HOME/stacks//logs/`, keeping at most about 10 MiB (plus the segment being written) per service instance. Destroying an instance or @@ -125,6 +126,99 @@ The inspector port is retained as an endpoint intent and does not enable debuggi Running start calls do not refresh Functions from changed project files; stop the stack and start it again to apply those changes. +## Service versions and the lock + +When the project has a `supabase/config.toml`, the versions a new or stopped stack runs come from +`supabase/supabase.lock`, a JSON file the CLI writes and the project commits. For each of the +twelve services it records the specifier requested under `[local.services]` (`.version`, +else `default.version`, else `linked`), how it resolved, and the exact pin: the release revision, +each target's archive and manifest digests, and the image digest. Its +[JSON Schema](https://supabase.com/docs/cli/supabase-lock/v1.schema.json) is published with the +docs. A project without `supabase/config.toml` runs the versions bundled with the CLI and has no +lock, unless `--frozen` is set, which fails the start. + +A specifier is `linked`, `bundled`, `` (the highest published revision) or +`-r` (exactly that revision). `linked` and `bundled` pin the version bundled with the +CLI. `linked` follows the hosted version that `supabase link` recorded in the entry's `linked` field +for the project it observed, and takes the bundled pin, recorded as `unlinked`, when the project was +never linked; a service the platform did not report keeps the bundled pin as `unobserved`, and a +hosted version slim-services does not publish as `unpublished`. Start never probes the hosted +project, needs no login and never calls the Management API: removing an override of a service +that was observed restores the linked version from the stored observation. A release +specifier probes the slim-services GitHub releases and the registry, verifies the published +digests, and records them. `SUPABASE_INTERNAL_SLIM_RELEASES_URL` and +`SUPABASE_INTERNAL_SLIM_REGISTRY_URL` redirect those hosts. + +A start reads the lock under the project lease `supabase/.supabase.lock.lease`, an exclusive lock +held while the lock is read, resolved and written. A concurrent +start waits up to 60 seconds for the lease and then fails with a `lock` error that names the lock +file; the wait is recorded as `lock.lease_wait_ms` on the `StackLock.lease` span. A lock that +answers the current request is used as-is, with no network or login. A service whose specifier +changed since the lock was written is resolved again, and the lock is rewritten with a `Resolved +service versions` report. A changed `db.major_version` re-resolves the database. Services that are +unchanged keep their pins, so starting never moves a pinned version on its own. The CLI adds +`.supabase.lock.lease` and the lock's staged writes (`.supabase.lock.*.tmp`) to +`supabase/.gitignore` whenever it takes the project lease. A frozen start, explicit or implied by +`CI` (see `--frozen` below), reads the lock without the lease and writes nothing in the project: +lock writes are atomic renames, so a read never sees a partial lock. + +`supabase stack install` runs this same resolution under the same lease, then downloads what the lock +pins without starting a stack; see [`../install/SIDE_EFFECTS.md`](../install/SIDE_EFFECTS.md). +`supabase stack update` re-resolves chosen services even when fresh, and `supabase stack outdated` +reports what it would change; see [`../update/SIDE_EFFECTS.md`](../update/SIDE_EFFECTS.md) and +[`../outdated/SIDE_EFFECTS.md`](../outdated/SIDE_EFFECTS.md). + +When `supabase/.link.json` targets a project or branch other than the one the lock observed, start +and `stack install` warn `lock observed /, this checkout targets +/; run supabase stack update`; this is a warning, never staleness. They also warn, +without any network, when this CLI bundles a newer revision of a version the lock holds +(`supabase stack outdated` and `supabase stack update` review and adopt it), excluding exact +`-r` specifiers. + +Resolving a release specifier needs the network: a first start with an explicit upstream version +(`` or `-r`, even one the CLI bundles) or with `[db] orioledb_version` (even +a bundled build) fetches the release manifests and fails when the host is unreachable, because a +bare upstream version never takes a shortcut through the bundled catalog. A first start whose +specifiers are all `linked` or `bundled` (the default) needs no network, unless the project was +linked, when a `linked` service resolves its observed release from the release host. A `linked` +service whose observed release cannot be fetched because the host is unreachable does not fail the +start: it takes the bundled version, recorded as `unobserved` (or keeps its previous entry when +nothing it depends on changed), and warns, naming `supabase link` and `supabase stack update` as +the retries. Any other failed resolution fails the start with reason `lock` and writes nothing; the +cause is unpublished (`invalid_config`) or transient (`network`) for an explicit `` or +`-r` specifier, or an integrity mismatch (`asset_checksum`) for any specifier. A saved +database whose pin comes from `[local.services]` reports its changes as +`[local.services.database] version`. + +`supabase db reset`, the database shadow of `supabase db diff`, the temporary Functions instance +`supabase functions serve` creates when the stack excludes Functions, and the local services listing +read the same lock without the lease or any resolution; serving through an existing Functions member +runs the build its stack already started with. When the project has +no lock they fall back to the versions bundled with the CLI, honouring `[db] major_version` and +`orioledb_version` and a `[local.services]` database version the CLI bundles; any other explicit +upstream-version specifier fails with `supabase/supabase.lock not found; run supabase start to create +it`. A stale lock fails with `is stale for: ()`. The services listing instead lists +the bundled versions in both cases. + +`--frozen` makes the start fail instead of resolving: a missing lock fails with +`supabase/supabase.lock not found`, and a stale one with `is stale for: ()`, both +advising to run `supabase start` locally and commit the lock. When the `CI` environment variable is +set (not empty, `false` or `0`) and a lock exists, the start is frozen without the flag. A frozen +start, explicit or by `CI`, of a project with a lock but no `supabase/config.toml` fails with +`supabase/config.toml not found`, since the lock cannot be checked, instead of using the bundled +versions; with no lock and no config, `--frozen` fails with `supabase/supabase.lock not found`. A CI +start with no lock resolves and writes it, then warns to commit it. A lock that is not valid JSON, does +not match the format or is of a newer format version always fails, frozen or not; the first of +these advises deleting the file and starting again. + +Initialized database data stays bound to its build. A stopped stack whose database holds no +initialized data (no readiness marker and no PostgreSQL data directory) adopts a changed +PostgreSQL major or release line; one with initialized data fails, advising to keep the recorded +build by pinning it under `[local.services]` or to run `supabase stack destroy` (see below). +`supabase link` and `supabase stack update` warn when the versions they lock would be refused by +the data of a saved stack of the project. `supabase stack status` reads the lock without the lease or any resolution; a +missing or stale lock makes its configuration drift check unavailable. + ## Service selection `--exclude` accepts repeated or comma-separated capability names: `rest`, `auth`, `realtime`, @@ -186,17 +280,41 @@ add the same changes to the success payload. Any other incompatible path blocks whole composition, even for a member whose own change is purely a changed endpoint: a changed `config.toml`-backed or env-var-backed setting (such as a changed PostgreSQL major version) still fails before modifying the stopped composition, naming the key or env var behind the change with its -saved and requested values and suggesting reverting it; a changed catalog-pinned artifact version or -a same-major PostgreSQL build mismatch, which no `config.toml` key or env var controls, instead uses -a plain label with no revert advice. Either way the failure suggests running the stack's exact -`supabase stack destroy` command to recreate it. -Switching between stock PostgreSQL and OrioleDB is an artifact version change, reported as +saved and requested values and suggesting reverting it. Only a database release-line change (the +stock PostgreSQL major, or the exact OrioleDB build; path `pin`) or an endpoint change is +incompatible; any other pin change, such as another service's catalog build, restarts that +service and does not block the re-plan, except that a schema-owning service never moves to an older +upstream than one that already migrated the database (see below). A database pin change that keeps +the release line, such as a different stock build within the same major, is also planned as a +restart, but a database with initialized data then fails its start (see below). +A saved OrioleDB build the catalog no longer carries is reported as non-editable, with no revert +advice. Either way the failure suggests running the stack's exact `supabase stack destroy` command +to recreate it. +Switching between stock PostgreSQL and OrioleDB is a release-line change, reported as `db.orioledb_version` (or `SUPABASE_DB_ORIOLEDB_VERSION`) with `unset` standing for stock. -Initialized database data is reused only on its own release line (the PostgreSQL major for stock -builds, the exact build for OrioleDB, since a newer OrioleDB build may refuse older data). A -first start records the requested line before initializing data, so a first start interrupted -before readiness resumes on that line; unmarked data without a recorded line can prove only its -major, so it is never reused for OrioleDB. +Initialized database data is reused only by a slim-services revision of the build that created +it: the same release line, PostgreSQL major and upstream version, with any `-rN` revision. Every +start that finds initialized data checks this before preparing or launching the database. A +different line, major or upstream version fails the start, naming the recorded and requested builds +and advising to keep the data by pinning `[local.services.database] version = ""` in +`supabase/config.toml`, with `[db] major_version` set to the recorded major when the major differs and +any `[db] orioledb_version` removed when an OrioleDB build is requested, or to run `supabase stack destroy`, which permanently deletes the +stack's local database data; data with no recorded identity offers only the destroy. A +first start records its identity in `.supabase-database-identity.json` before initializing data, so +a first start interrupted before readiness resumes only on that build; initialized data with no +recorded identity is never reused. +Auth, Storage, Realtime, Analytics and the pooler own schemas in the database. The database +instance root records the newest upstream version of each in `schema-writers.json`, raised under a +per-database lock before a service or its one-shot initialization command migrates. A start whose +selected upstream is older than the recorded one fails, naming the version and advising to keep the +data by pinning `[local.services.] version = ""` in +`supabase/config.toml`, or to run `supabase stack destroy` to recreate the stack. Versions order by their leading dotted number +(`v2.196.0`, `2026.10.05-sha-...`, `17.6.1.054`); a suffix is ignored and an unparseable version +sorts oldest. Services without schemas (REST, imgproxy, Functions, Studio, pg-meta, mail) accept any +version change. A missing record means no schema owner has migrated yet. Saving a snapshot stores +the record with it; restoring raises the record to the newer of the two, publishes the data, then +sets the snapshot's record, so a crash between those steps never leaves older writers allowed. A +reset of database data removes the record. A service on an external database has no local record. ## First startup and retries diff --git a/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts b/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts index 2f0f7f92e2..706f606743 100644 --- a/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/start/start-export-pointer.integration.test.ts @@ -28,6 +28,7 @@ import { CommandSettings } from "../../../../config/command-settings.service.ts" import { StackApi, StackTargetResolver } from "../stack.shared.ts"; import { stackStatusCommand } from "../status/status.command.ts"; import { statusEnvPointer } from "./start-summary.format.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; /** Distinguishable credentials, keyed by a short tag, for one fake target stack. */ const credentialsFor = (tag: string): StackCredentials => ({ @@ -47,8 +48,8 @@ const credentialsFor = (tag: string): StackCredentials => ({ const databaseCreation = (sqlPort: number, credentials: StackCredentials): ServiceCreation => ({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(credentials.databasePassword), jwtSecret: Redacted.make(credentials.jwtSecret), jwtExpiry: 3600, diff --git a/apps/cli/src/commands/experimental/stack/start/start.command.ts b/apps/cli/src/commands/experimental/stack/start/start.command.ts index f7a39945c4..5b6392e174 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.command.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.command.ts @@ -7,6 +7,7 @@ import { machineErrorContextLayer } from "../../../../shared/output/machine-erro import { withCommandTelemetry } from "../../../../telemetry/command-telemetry.ts"; import { stringSliceFlag } from "../../../../command-internal/string-slice-flag.ts"; import { stdinLayer } from "../../../../shared/runtime/stdin.layer.ts"; +import { frozenFlag } from "../stack.flags.ts"; import { stackStart } from "./start.handler.ts"; import { STACK_START_EXCLUDABLE_CAPABILITIES } from "./start.options.ts"; @@ -33,6 +34,7 @@ const config = { Flag.withDescription("Artifact preparation policy."), Flag.withDefault("background" as const), ), + frozen: frozenFlag, eager: Flag.Boolean("eager").pipe( Flag.withDescription( "Activate all enabled capabilities before returning and disable automatic idle stops.", @@ -63,7 +65,7 @@ export const stackStartCommand = Command.make("start", config).pipe( ]), withPermissions({ operations: [], - noApiEffectFlags: ["exclude", "stack", "stack-id", "runtime", "preparation", "eager"], + noApiEffectFlags: ["exclude", "stack", "stack-id", "runtime", "preparation", "frozen", "eager"], }), Command.withHandler((flags) => stackStart(flags).pipe( diff --git a/apps/cli/src/commands/experimental/stack/start/start.errors.ts b/apps/cli/src/commands/experimental/stack/start/start.errors.ts index 15d7edea98..b4b3d07c04 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.errors.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.errors.ts @@ -8,7 +8,7 @@ import { } from "../../../../shared/telemetry/error-actionability.ts"; export class StackCommandStartError extends Data.TaggedError("ExperimentalStackStartError")<{ - readonly reason: "invalid-config" | "flags" | "runtime" | "lifecycle" | "seed" | "stack"; + readonly reason: "invalid-config" | "lock" | "flags" | "runtime" | "lifecycle" | "seed" | "stack"; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -18,6 +18,13 @@ export class StackCommandStartError extends Data.TaggedError("ExperimentalStackS switch (this.reason) { case "invalid-config": return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + case "lock": + return ( + causeDeclaration(this.cause) ?? { + ...actionability.invalidConfig, + fingerprint_suffix: "invalid_config", + } + ); case "flags": return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; case "runtime": diff --git a/apps/cli/src/commands/experimental/stack/start/start.handler.ts b/apps/cli/src/commands/experimental/stack/start/start.handler.ts index 20cbe2b796..ad58bb8163 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.handler.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.handler.ts @@ -3,7 +3,6 @@ import { isOrioledbVersion, orioledbConfigVersion, orioledbVersions, - postgresVersion, } from "@supabase/stack/internal/artifacts"; import { connectionEnv, @@ -24,7 +23,7 @@ import { type StackRuntime, } from "../../../../command-internal/stack-runtime.ts"; import { RuntimeInfo } from "../../../../shared/runtime/runtime-info.service.ts"; -import { Effect, FileSystem, Fiber, Match, Option, Path, Redacted, Ref } from "effect"; +import { Effect, FileSystem, Fiber, Match, Option, Path, Redacted, Ref, Schema } from "effect"; import { apiRoute, resolveNativePostgresUser, @@ -56,6 +55,7 @@ import { hasConfiguredBuckets, seedBucketsRun } from "../../../../command-intern import { loadLocalProjectContext } from "../../../../command-internal/local-project-context.ts"; import { loadStackConfig, + stackUploadsDir, StackConfigError, stackEndpointSetting, stackMajorVersionSetting, @@ -63,6 +63,12 @@ import { type StackEndpointSetting, } from "../../../../command-internal/stack-config.ts"; import { envOverride } from "../../../../command-internal/local-config-values.ts"; +import type { CliConfig } from "@supabase/config/effect"; +import { + LockResolutionError, + lockedSelection, + parseSpecifier, +} from "../../../../command-internal/stack-lock.ts"; import { StackApi, stackCapabilityForService, @@ -196,12 +202,23 @@ const stackAcquireError = ( }; const loadStartConfig = Effect.fn("experimental.stack.start.loadConfig")( - function* (projectRoot: string, fs: FileSystem.FileSystem, path: Path.Path) { + function* ( + projectRoot: string, + fs: FileSystem.FileSystem, + path: Path.Path, + selection: ReturnType, + frozen: boolean, + ) { const context = yield* loadLocalProjectContext( projectRoot, (message) => new StackConfigError({ message }), ); - const config = yield* loadStackConfig(projectRoot, { context }); + const config = yield* loadStackConfig(projectRoot, { + context, + selection, + frozen, + publishesLock: true, + }); const keys = yield* config.keys; const toml = yield* readDbToml(fs, path, projectRoot); return { context, config, keys, toml }; @@ -209,7 +226,7 @@ const loadStartConfig = Effect.fn("experimental.stack.start.loadConfig")( Effect.mapError( (error) => new StackCommandStartError({ - reason: "invalid-config", + reason: Schema.is(LockResolutionError)(error) ? "lock" : "invalid-config", message: error.message, cause: error, }), @@ -240,9 +257,13 @@ const endpointPortLabel = (endpoints: unknown, name: string): string => { return port === "auto" ? "automatic" : typeof port === "number" ? String(port) : "unset"; }; -const databaseVersionOf = ( - creation: ServiceCreation | ServiceCreationInput | undefined, -): string | undefined => (creation?.service === "database" ? creation.config.version : undefined); +const databaseVersionOf = (creation: ServiceCreation | ServiceCreationInput | undefined): string => + creation?.service === "database" ? creation.pin.upstreamVersion : "unknown"; + +const databaseReleaseOf = (creation: ServiceCreation | ServiceCreationInput | undefined): string => + creation?.service === "database" + ? `${creation.pin.upstreamVersion}-r${creation.pin.revision}` + : "unknown"; const majorVersionOf = (version: string): string => version.split(".")[0] ?? version; @@ -269,8 +290,7 @@ const settingKeyLabel = ( /** * One setting behind an incompatible path, reported as the JSON/stream-json error envelope's * `stack_changes` entries (contract documented in `SIDE_EFFECTS.md`). `editable` marks whether `key` is a - * `config.toml` key or env var the user can revert, or plain wording for a catalog-pinned - * artifact or Postgres build. + * `config.toml` key or env var the user can revert, or plain wording otherwise. */ interface StructuredSettingChange { readonly service: string; @@ -287,27 +307,26 @@ const describeSettingChange = ( savedCreation: ServiceCreation | undefined, requestedCreation: ServiceCreationInput | undefined, projectEnvValues: Readonly>, + databaseFromServices: boolean, ): StructuredSettingChange | ReadonlyArray => { - if (service === "database" && path === "config.version") { - // `postgresVersion` resolves a bare major alias (e.g. "17") to the pinned build the - // composition plan actually compared, so the saved/requested pair reflects what changed. - const savedVersion = postgresVersion(databaseVersionOf(savedCreation) ?? "unknown"); - const requestedVersion = postgresVersion(databaseVersionOf(requestedCreation) ?? "unknown"); + if (service === "database" && path === "pin") { + const savedVersion = databaseVersionOf(savedCreation); + const requestedVersion = databaseVersionOf(requestedCreation); const savedMajor = majorVersionOf(savedVersion); const requestedMajor = majorVersionOf(requestedVersion); const savedOrioledb = orioledbVersionOf(savedVersion); const requestedOrioledb = orioledbVersionOf(requestedVersion); - // Same major and engine but different pinned build: no setting controls this, so name the - // actual (unpinnable) versions instead. - if (savedMajor === requestedMajor && savedOrioledb === requestedOrioledb) - return { - service, - path, - key: "Postgres build", - saved: savedVersion, - requested: requestedVersion, - editable: false, - }; + if (databaseFromServices) + return [ + { + service, + path, + key: "[local.services.database] version", + saved: databaseReleaseOf(savedCreation), + requested: databaseReleaseOf(requestedCreation), + editable: true, + }, + ]; const changes: Array = []; if (savedMajor !== requestedMajor) changes.push({ @@ -342,24 +361,32 @@ const describeSettingChange = ( requested: endpointPortLabel(requestedCreation?.endpoints, endpointName), editable: true, }; - // No config.toml key or env var covers this path (e.g. the catalog-pinned artifact `version`): - // name it plainly instead of implying a setting the user could edit. + // No config.toml key or env var covers this path: name it plainly instead of implying a + // setting the user could edit. return { service, path, - key: path === "version" ? `${service} artifact version` : `${service} ${path}`, - saved: path === "version" ? (savedCreation?.version ?? "unknown") : "changed", - requested: path === "version" ? (requestedCreation?.version ?? "unknown") : "changed", + key: `${service} ${path}`, + saved: "changed", + requested: "changed", editable: false, }; }; +/** Whether `[local.services]` names a published database release, which `[db]` settings no longer select. */ +const databasePublishedBy = (source: CliConfig): boolean => { + const services = source.local?.services; + const version = services?.database?.version ?? services?.default?.version; + return version !== undefined && parseSpecifier(version).kind === "release"; +}; + /** Every incompatible path across every rejected saved member, as one structured list. */ const incompatibleSettingChanges = ( planned: ReadonlyArray, savedConfigById: ReadonlyMap, requested: ReadonlyArray, projectEnvValues: Readonly>, + databaseFromServices: boolean, ): ReadonlyArray => planned .filter((entry) => entry.member && entry.change === "incompatible") @@ -373,6 +400,7 @@ const incompatibleSettingChanges = ( savedConfigById.get(entry.id), requested.find((creation) => creation.service === entry.service), projectEnvValues, + databaseFromServices, ), ) : [], @@ -411,12 +439,13 @@ const revertAdvice = (editableKeys: ReadonlyArray): string | undefined = ? `Revert ${editableKeys[0]} to its saved value` : "Revert the settings listed to their saved values"; -/** Rejects every saved member whose endpoints or artifact versions the request would change. */ +/** Rejects every saved member whose endpoints or database line or major the request would change. */ const incompatibleChange = ( planned: ReadonlyArray, savedConfigById: ReadonlyMap, requested: ReadonlyArray, projectEnvValues: Readonly>, + databaseFromServices: boolean, stackIdentity: { readonly id: string; readonly name?: string }, ownerLive: boolean, ): @@ -427,7 +456,13 @@ const incompatibleChange = ( readonly command: string | undefined; } | undefined => { - const changes = incompatibleSettingChanges(planned, savedConfigById, requested, projectEnvValues); + const changes = incompatibleSettingChanges( + planned, + savedConfigById, + requested, + projectEnvValues, + databaseFromServices, + ); if (changes.length === 0) return undefined; const lines = settingChangeLines(changes); const command = destroyCommandFor(stackIdentity.id); @@ -561,7 +596,15 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags NotNeeded: () => Effect.void, }); // Every project-context load prints the config deprecation WARNs, so it happens at most once. - const startConfig = yield* Effect.cached(loadStartConfig(target.projectRoot, fs, path)); + const startConfig = yield* Effect.cached( + loadStartConfig( + target.projectRoot, + fs, + path, + lockedSelection({ frozen: flags.frozen, output }), + flags.frozen, + ), + ); if (target.id === undefined || !target.hostRunning) yield* startConfig; if (target.id === undefined) yield* ensurePostgresUser; const stateRoot = path.join(settings.supabaseHome, "stacks"); @@ -600,7 +643,8 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags // before it registers endpoint namespaces from the saved state. An owner that is already live // ignores them and keeps applying endpoint changes only after stop and start, so the liveness // observed above never decides whether a replacement owner that wins a stop race re-plans. A - // configuration that cannot resolve only fails a start whose owner was not running. + // configuration that cannot resolve only fails a start whose owner was not running; a lock + // failure fails either way. const resolveForReplan = (stackId: string) => Effect.gen(function* () { const loaded = yield* startConfig; @@ -611,8 +655,10 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags ? undefined : target.hostRunning ? yield* resolveForReplan(target.id).pipe( - Effect.option, - Effect.map(Option.getOrUndefined), + Effect.catchIf( + (error) => error.reason !== "lock", + () => Effect.void, + ), ) : yield* resolveForReplan(target.id); const startupComplete = yield* Ref.make(false); @@ -823,12 +869,9 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags } if (requested.some(({ service }) => service === "storage")) yield* fs - .makeDirectory( - path.join(target.projectRoot, "supabase", ".temp", "stack-uploads", stack.id), - { - recursive: true, - }, - ) + .makeDirectory(stackUploadsDir(path, settings.supabaseHome, stack.id), { + recursive: true, + }) .pipe( Effect.mapError( (cause) => @@ -882,6 +925,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags savedConfigById, requested, config.projectEnvValues, + databasePublishedBy(config.source), stackIdentity, target.hostRunning, ); @@ -1014,7 +1058,12 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags } : undefined; yield* initializeStackDatabase({ - target: { stack, database, databaseServices: catalogDatabaseServices(requested) }, + target: { + stack, + database, + databaseServices: catalogDatabaseServices(requested), + selection: config.selection, + }, overlay: projectCatalogOverlay(toml, target.projectRoot), ...(migrations === undefined ? {} : { migrations }), }).pipe( diff --git a/apps/cli/src/commands/experimental/stack/start/start.integration.test.ts b/apps/cli/src/commands/experimental/stack/start/start.integration.test.ts index 2409dcba09..ca3c828c75 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.integration.test.ts @@ -3,6 +3,7 @@ import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { vi } from "vitest"; import { + ConfigProvider, Deferred, Effect, Fiber, @@ -20,9 +21,15 @@ import { DEFAULT_LOCAL_JWT_SECRET, DEFAULT_POSTGRES_ROOT_KEY, } from "@supabase/stack/defaults"; -import { orioledbVersions, postgresVersion } from "@supabase/stack/internal/artifacts"; +import { + artifactServiceKinds, + orioledbVersions, + type ArtifactPin, +} from "@supabase/stack/internal/artifacts"; import { apiRoute, + lockSelection, + readLock, StackError, type EndpointPortChange, type PlanOptions, @@ -32,14 +39,33 @@ import { type ServiceInstances, type StackCredentials, type Stack, + writeLock, } from "@supabase/stack/effect"; import { planSupabaseComposition } from "@supabase/stack/internal/composition"; import { mockCommandSettings, mockTelemetryStateTracked, + withConfigEnv, withEnvVar, } from "../../../../../tests/helpers/command-mocks.ts"; import { containerEngineSpawner } from "../../../../../tests/helpers/child-process-spawner.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../../tests/helpers/legacy-state-guard.ts"; +import { + database15, + orioledb as orioledbPin, + orioledbConfigVersion, + selection, +} from "../../../../../tests/helpers/pins.ts"; +import { + publishRelease, + releaseHostEnv, + startReleaseFixture, + type ReleaseFixture, +} from "../../../../../tests/helpers/release-fixture.ts"; import { mockOutput, mockProcessControl, @@ -51,9 +77,11 @@ import { DbConnection, type DbSession, } from "../../../../command-internal/db-connection.service.ts"; +import { writeProjectLink } from "../../../../command-internal/project-link.ts"; import { StackCatalogSetup } from "../../../../command-internal/stack-catalog-setup.ts"; import { ExperimentalFlag, YesFlag } from "../../../../command-internal/global-flags.ts"; import { CommandPlatformApiFactory } from "../../../../auth/command-platform-api-factory.service.ts"; +import { causeDeclaration } from "../../../../shared/telemetry/error-actionability.ts"; import { stdinLayer } from "../../../../shared/runtime/stdin.layer.ts"; import * as HttpClient from "effect/http/HttpClient"; import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; @@ -70,6 +98,7 @@ const flags = (exclude: ReadonlyArray = []) => ({ stackId: Option.none(), runtime: "native" as const, preparation: "background" as const, + frozen: false, eager: false, }); @@ -172,12 +201,14 @@ const instance = ( if (input === undefined) return; current = { ...current, - ...("version" in input ? { version: input.version } : {}), ...("endpoints" in input ? { endpoints: input.endpoints } : {}), config: { ...current.config, ...input.config }, }; }), - status: Effect.sync(() => status(current)), + status: Effect.sync(() => { + const savedPin = view().savedPin; + return status(savedPin === undefined ? current : { ...current, pin: savedPin }); + }), credentials: () => Effect.succeed({ databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres" }), saveSnapshot: () => Effect.die("unused"), @@ -259,6 +290,7 @@ interface MemberView { readonly wakeEnabled: boolean; readonly health: "starting" | "healthy" | "unhealthy" | undefined; readonly ready: Effect.Effect; + readonly savedPin: ArtifactPin | undefined; } interface MemberStatus { @@ -306,6 +338,7 @@ const fakeStack = (compositionStart?: Stack["composition"]["start"]) => { let activations = new Map(); const memberStatuses = new Map(); const memberReadiness = new Map>(); + const savedPins = new Map(); /** The endpoint changes a simulated owner startup re-planned, until the next `applyStartupReplan`. */ let pendingStartupChanges: ReadonlyArray = []; let savedCredentials = credentialsFor(undefined, undefined); @@ -318,6 +351,7 @@ const fakeStack = (compositionStart?: Stack["composition"]["start"]) => { status?.wakeEnabled ?? (lifecycle === "running" && activations.get(id) === "lazy"), health: status?.health ?? (memberLifecycle === "running" ? "healthy" : undefined), ready: memberReadiness.get(id) ?? Effect.void, + savedPin: savedPins.get(id), }; }; /** Builds a member from a creation, tracking it for a later `reassignEndpoint`. */ @@ -454,6 +488,10 @@ const fakeStack = (compositionStart?: Stack["composition"]["start"]) => { get savedCredentials() { return savedCredentials; }, + /** Replaces a member's saved artifact pin, which a real restart cannot change. */ + replaceSavedPin(id: string, pin: ArtifactPin) { + savedPins.set(id, pin); + }, setMemberStatus(id: string, status: MemberStatus) { memberStatuses.set(id, status); }, @@ -531,6 +569,12 @@ const fakeStack = (compositionStart?: Stack["composition"]["start"]) => { }; }; +/** A CI runner sets `CI`, which would freeze the lock between the starts of one test. */ +const ciUnsetLayer = ConfigProvider.layerAdd( + ConfigProvider.fromEnvRecord({ CI: "" }, { preserveEmptyStrings: true }), + { asPrimary: true }, +); + const layers = ( root: string, fixture: ReturnType, @@ -570,6 +614,7 @@ const layers = ( target, api, Layer.succeed(ExperimentalFlag, false), + ciUnsetLayer, Layer.succeed(CliArgs, { args: ["stack", "start"] }), Layer.succeed(StackCatalogSetup, { apply: () => Effect.sync(() => fixture.applyCatalog()), @@ -952,6 +997,26 @@ describe("experimental stack start", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.effect("keeps storage uploads under SUPABASE_HOME and never touches supabase/.temp", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-uploads-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString(`${root}/supabase/config.toml`, 'project_id = "start-uploads"\n'); + seedLegacyStateDecoys(root); + const excluded = ["auth", "realtime", "functions", "mail", "analytics", "pooler"]; + const fixture = fakeStack(); + const recorder = recordingFileSystem(); + + yield* stackStart(flags(excluded)).pipe( + Effect.provide(Layer.mergeAll(layers(root, fixture), recorder.layer)), + ); + + expect(yield* fs.exists(`${root}/.supabase/stacks/${fixture.stack.id}/uploads`)).toBe(true); + expect(legacyStateAccesses(recorder.accessed)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect("returns the connection env for an already-running stack in JSON", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -2062,56 +2127,51 @@ describe("experimental stack start", () => { ); it.effect( - "names the full Postgres build, not major_version, when only the pinned build differs", + "launches the database pin selected by [db] major_version and the stock pin elsewhere", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-pg-build-" }); + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-selected-pin-" }); yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); yield* fs.writeFileString( `${root}/supabase/config.toml`, - 'project_id = "pg-build"\n[db]\nmajor_version = 17\n', + 'project_id = "selected-pin"\n[db]\nmajor_version = 15\n', ); const fixture = fakeStack(); yield* stackStart(flags()).pipe(Effect.provide(layers(root, fixture))); - const database = fixture.members.find(({ service }) => service === "database"); - if (database?.service !== "database") return yield* Effect.die("Database missing"); - const observed = yield* database.status; - if (observed.config.service !== "database") - return yield* Effect.die("Database config missing"); - const pinnedVersion = postgresVersion("17"); - // A saved build the current catalog no longer pins (`postgresVersion` only normalizes a - // recognized alias): same major as the requested `17`, different full build. - yield* database.restart({ - config: { ...observed.config.config, version: "17.0.0-stale-build" }, - }); - yield* fixture.stack.composition.stop; - const error = yield* stackStart(flags()).pipe( - Effect.provide(layers(root, fixture)), - Effect.flip, - ); - expect(error).toMatchObject({ - reason: "invalid-config", - message: expect.stringContaining( - `Postgres build: saved 17.0.0-stale-build, requested ${pinnedVersion}`, - ), - }); - expect(error).toBeInstanceOf(StackCommandStartError); - if (error instanceof StackCommandStartError) { - expect(error.message).not.toContain("major_version"); - expect(error.suggestion).not.toContain("Revert"); - expect(error.suggestion).toContain( - "This CLI release starts a different Postgres build than the saved stack.", - ); - expect(error.suggestion).toContain( - `supabase stack destroy --stack-id ${fixture.stack.id}`, - ); + const pins = new Map(); + for (const member of fixture.members) { + const observed = yield* member.status; + if (observed.config.service === "database") pins.set("database", observed.config.pin); + if (observed.config.service === "rest") pins.set("rest", observed.config.pin); } + expect(pins.get("database")).toEqual(database15); + expect(pins.get("rest")).toEqual(selection.rest); }).pipe(Effect.provide(BunServices.layer)), ); + const retiredOrioledbPin = (pin: ArtifactPin): ArtifactPin => ({ + ...pin, + upstreamVersion: "17.0.0.000-orioledb", + image: "registry.invalid/postgres@sha256:retired", + }); + const [orioledb = ""] = orioledbVersions(); + + /** `supabase start` of `root` with the release host reachable, so an OrioleDB specifier resolves. */ + const startOnHost = (root: string, fixture: ReturnType, host: ReleaseFixture) => + withConfigEnv(releaseHostEnv(host), stackStart(flags())).pipe( + Effect.provide(layers(root, fixture)), + ); + const publishOrioledb = (host: ReleaseFixture) => { + for (let revision = 0; revision <= orioledbPin.revision; revision++) + publishRelease(host, { + producer: "postgres", + upstream: orioledbPin.upstreamVersion, + revision, + }); + }; for (const target of [ { name: "both [db] major_version and [db] orioledb_version when stock 15 becomes OrioleDB 17", @@ -2141,9 +2201,11 @@ describe("experimental stack start", () => { revert: "Revert SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION to its saved value", }, ]) - it.effect(`names ${target.name}`, () => + it.live(`names ${target.name}`, () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + publishOrioledb(host); const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-orioledb-" }); yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); const configPath = `${root}/supabase/config.toml`; @@ -2153,10 +2215,7 @@ describe("experimental stack start", () => { yield* fs.writeFileString(configPath, `project_id = "orioledb"\n${target.after}`); yield* fixture.stack.composition.stop; - const restart = stackStart(flags()).pipe( - Effect.provide(layers(root, fixture)), - Effect.flip, - ); + const restart = startOnHost(root, fixture, host).pipe(Effect.flip); const error = yield* target.env === undefined ? restart : withEnvVar(target.envVar, target.env, restart); @@ -2171,11 +2230,13 @@ describe("experimental stack start", () => { }).pipe(Effect.provide(BunServices.layer)), ); - it.effect( + it.live( "warns once about SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION on a first start that includes storage", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + publishOrioledb(host); const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-orioledb-alias-" }); yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); yield* fs.writeFileString(`${root}/supabase/config.toml`, 'project_id = "alias"\n'); @@ -2188,8 +2249,7 @@ describe("experimental stack start", () => { const errorSpy = vi.spyOn(console, "error").mockImplementation((...args) => { consoleErrors.push(args.map((a) => String(a)).join(" ")); }); - yield* stackStart(flags()).pipe( - Effect.provide(layers(root, fixture)), + yield* startOnHost(root, fixture, host).pipe( Effect.ensuring(Effect.sync(() => errorSpy.mockRestore())), ); const services = yield* fixture.stack.services.list; @@ -2216,10 +2276,7 @@ describe("experimental stack start", () => { if (database?.service !== "database") return yield* Effect.die("database missing"); const observed = yield* database.status; if (observed.config.service !== "database") return yield* Effect.die("config missing"); - yield* database.restart({ - ...observed.config, - config: { ...observed.config.config, version: "17.0.0.000-orioledb" }, - }); + fixture.replaceSavedPin(database.id, retiredOrioledbPin(observed.config.pin)); yield* fixture.stack.composition.stop; const error = yield* stackStart(flags()).pipe( Effect.provide(layers(root, fixture)), @@ -2238,47 +2295,6 @@ describe("experimental stack start", () => { }).pipe(Effect.provide(BunServices.layer)), ); - it.effect( - "drops the revert sentence for an artifact-version-only mismatch and explains the fix in plain language", - () => - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-artifact-version-" }); - yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); - yield* fs.writeFileString( - `${root}/supabase/config.toml`, - 'project_id = "artifact-version"\n', - ); - const fixture = fakeStack(); - yield* stackStart(flags()).pipe(Effect.provide(layers(root, fixture))); - const rest = fixture.members.find(({ service }) => service === "rest"); - if (rest?.service !== "rest") return yield* Effect.die("REST missing"); - const restObserved = yield* rest.status; - if (restObserved.config.service !== "rest") return yield* Effect.die("REST config missing"); - yield* rest.restart({ ...restObserved.config, version: "rest-v1-stale" }); - yield* fixture.stack.composition.stop; - const error = yield* stackStart(flags()).pipe( - Effect.provide(layers(root, fixture)), - Effect.flip, - ); - - expect(error).toMatchObject({ - reason: "invalid-config", - message: expect.stringContaining("rest artifact version"), - }); - expect(error).toBeInstanceOf(StackCommandStartError); - if (error instanceof StackCommandStartError) { - expect(error.suggestion).not.toContain("Revert"); - expect(error.suggestion).toContain( - "This CLI release starts a different rest artifact version than the saved stack.", - ); - expect(error.suggestion).toContain( - `supabase stack destroy --stack-id ${fixture.stack.id}`, - ); - } - }).pipe(Effect.provide(BunServices.layer)), - ); - it.effect( "uses the destroy-only wording when a non-editable change accompanies an editable one", () => @@ -2292,11 +2308,12 @@ describe("experimental stack start", () => { ); const fixture = fakeStack(); yield* stackStart(flags()).pipe(Effect.provide(layers(root, fixture))); - const rest = fixture.members.find(({ service }) => service === "rest"); - if (rest?.service !== "rest") return yield* Effect.die("REST missing"); - const restObserved = yield* rest.status; - if (restObserved.config.service !== "rest") return yield* Effect.die("REST config missing"); - yield* rest.restart({ ...restObserved.config, version: "rest-v1-stale" }); + const database = fixture.members.find(({ service }) => service === "database"); + if (database?.service !== "database") return yield* Effect.die("Database missing"); + const observed = yield* database.status; + if (observed.config.service !== "database") + return yield* Effect.die("Database config missing"); + fixture.replaceSavedPin(database.id, retiredOrioledbPin(observed.config.pin)); yield* fixture.stack.composition.stop; yield* fs.writeFileString( @@ -2311,14 +2328,14 @@ describe("experimental stack start", () => { expect(error).toMatchObject({ reason: "invalid-config", // The editable change still appears in the message even though reverting it alone - // can't unblock start: the non-editable artifact-version change still would. + // can't unblock start: the retired OrioleDB build still would. message: expect.stringContaining("[studio] port: saved automatic, requested 12345"), }); expect(error).toBeInstanceOf(StackCommandStartError); if (error instanceof StackCommandStartError) { expect(error.suggestion).not.toContain("Revert"); expect(error.suggestion).toContain( - "This CLI release starts a different rest artifact version than the saved stack.", + "This CLI release starts a different [db] orioledb_version than the saved stack.", ); expect(error.suggestion).toContain( `supabase stack destroy --stack-id ${fixture.stack.id}`, @@ -2354,7 +2371,7 @@ describe("experimental stack start", () => { expect(envelope.stack_changes).toEqual([ { service: "database", - path: "config.version", + path: "pin", key: "[db] major_version", saved: "17", requested: "15", @@ -2393,7 +2410,7 @@ describe("experimental stack start", () => { expect(event.stack_changes).toEqual([ { service: "database", - path: "config.version", + path: "pin", key: "[db] major_version", saved: "17", requested: "15", @@ -2446,25 +2463,19 @@ describe("experimental stack start", () => { }).pipe(Effect.provide(BunServices.layer)), ); - it.effect("matches a Postgres major alias to the saved pinned database version", () => + it.effect("reuses the saved database member when the selected pin is unchanged", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; - const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-postgres-alias-" }); + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-same-pin-" }); yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); yield* fs.writeFileString( `${root}/supabase/config.toml`, - 'project_id = "start-postgres-alias"\n[db]\nmajor_version = 17\n[edge_runtime]\nenabled = false\n', + 'project_id = "start-same-pin"\n[db]\nmajor_version = 17\n[edge_runtime]\nenabled = false\n', ); const fixture = fakeStack(); yield* stackStart(flags()).pipe(Effect.provide(layers(root, fixture))); const database = fixture.members.find(({ service }) => service === "database"); if (database?.service !== "database") return yield* Effect.die("Database missing"); - const observed = yield* database.status; - if (observed.config.service !== "database") - return yield* Effect.die("Database config missing"); - const pinnedVersion = postgresVersion("17"); - expect(pinnedVersion).not.toBe("17"); - yield* database.restart({ config: { ...observed.config.config, version: pinnedVersion } }); yield* fixture.stack.composition.stop; yield* stackStart(flags()).pipe(Effect.provide(layers(root, fixture))); @@ -2600,3 +2611,616 @@ describe("experimental stack start", () => { }).pipe(Effect.provide(BunServices.layer)), ); }); + +const lockPath = (root: string) => `${root}/supabase/supabase.lock`; +const leasePath = (root: string) => `${root}/supabase/.supabase.lock.lease`; + +const REST_OVERRIDE = '[local.services.rest]\nversion = "v99.0.0"\n'; + +const setConfig = (root: string, config: string) => + Effect.flatMap(FileSystem.FileSystem, (fs) => + fs.writeFileString(`${root}/supabase/config.toml`, `project_id = "start-lock"\n${config}`), + ); + +const lockProject = (config = "") => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-lock-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* setConfig(root, config); + return root; + }); + +const lockText = (root: string) => + Effect.flatMap(FileSystem.FileSystem, (fs) => fs.readFileString(lockPath(root))); + +const readProjectLock = (root: string) => readLock(root).pipe(Effect.map(Option.getOrThrow)); + +/** Publishes `postgrest` `v99.0.0` revisions 0 and 1, so the unpinned specifier resolves to 1. */ +const publishRest = (host: ReleaseFixture) => { + publishRelease(host, { producer: "postgrest", upstream: "v99.0.0", revision: 0 }); + return publishRelease(host, { producer: "postgrest", upstream: "v99.0.0", revision: 1 }); +}; + +/** The artifact pin the fake stack's member of `service` was created with. */ +const memberPin = (stack: ReturnType, service: ServiceCreation["service"]) => + stack.members.flatMap((member) => + member.service === service ? [Effect.runSync(member.status).config.pin] : [], + )[0]; + +/** One `supabase start` of `root`, with `CI` pinned and the release host reachable only when given. */ +const lockedStart = ( + root: string, + options: { + readonly stack?: ReturnType; + readonly host?: ReleaseFixture; + readonly ci?: boolean; + readonly frozen?: boolean; + readonly existing?: boolean; + } = {}, +) => { + const output = mockOutput(); + const stack = options.stack ?? fakeStack(); + const run = withConfigEnv( + { + CI: options.ci === true ? "true" : "", + ...(options.host === undefined ? {} : releaseHostEnv(options.host)), + }, + stackStart({ ...flags(), frozen: options.frozen ?? false }), + ).pipe(Effect.provide(layers(root, stack, output, options.existing))); + return { output, stack, run }; +}; + +const infoMessages = (output: ReturnType) => + output.messages.flatMap((entry) => (entry.type === "info" ? [entry.message] : [])); + +const resolvedReports = (output: ReturnType) => + infoMessages(output).filter((message) => message.startsWith("Resolved service versions:")); + +const LINKED_BRANCH_REF = "branchprojectrefabcd"; + +const LINKED_MAIN = { + project: "projectrefabcdefghijk", + branch: { ref: "projectrefabcdefghijk", isDefault: true }, +}; + +const COMMIT_WARNING = "Commit supabase/supabase.lock so CI and your team run the same versions."; + +describe("experimental stack start service lock", () => { + it.effect("writes the default lock on the first start and keeps it on later starts", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* lockProject(); + const first = lockedStart(root); + yield* first.run; + + const lock = yield* readProjectLock(root); + expect(lockSelection(lock)).toEqual(selection); + expect(lock.services.database).toMatchObject({ majorVersion: 17, line: "stock" }); + expect(Object.values(lock.services).map(({ resolution }) => resolution)).toEqual( + Array.from({ length: 12 }, () => "unlinked"), + ); + expect(resolvedReports(first.output)).toHaveLength(1); + expect(resolvedReports(first.output)[0]).toContain("Created supabase/supabase.lock."); + const gitignore = yield* fs.readFileString(`${root}/supabase/.gitignore`); + expect(gitignore).toContain(".supabase.lock.lease"); + expect(gitignore).toContain(".supabase.lock.*.tmp"); + + const written = yield* lockText(root); + yield* first.stack.stack.composition.stop; + const second = lockedStart(root, { stack: first.stack }); + yield* second.run; + expect(yield* lockText(root)).toBe(written); + expect(resolvedReports(second.output)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("resolves a version override through the release host and runs the stack on it", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const latest = publishRest(host); + const root = yield* lockProject(REST_OVERRIDE); + const { output, stack, run } = lockedStart(root, { host }); + yield* run; + + const lock = yield* readProjectLock(root); + expect(lock.services.rest).toMatchObject({ + specifier: "v99.0.0", + resolution: "resolved", + resolved: { upstreamVersion: "v99.0.0", revision: 1 }, + }); + expect(lock.services.rest.resolved.image).toContain(latest.digest); + expect(lock.services.auth.resolved).toEqual(selection.auth); + expect(memberPin(stack, "rest")).toEqual(lock.services.rest.resolved); + expect(resolvedReports(output)[0]).toContain("v99.0.0-r1"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("goes back to the bundled version when the override is removed", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(REST_OVERRIDE); + yield* lockedStart(root, { host }).run; + const overridden = yield* readProjectLock(root); + + yield* setConfig(root, ""); + const { output, run } = lockedStart(root, { host, existing: false }); + yield* run; + + const lock = yield* readProjectLock(root); + expect(lock.services.rest).toMatchObject({ specifier: "linked", resolution: "unlinked" }); + expect(lock.services.rest.resolved).toEqual(selection.rest); + expect({ ...lock.services, rest: undefined }).toEqual({ + ...overridden.services, + rest: undefined, + }); + expect(resolvedReports(output)[0]).toContain("specifier linked ≠ locked v99.0.0"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("restores the linked version when the override is removed, without a new probe", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(REST_OVERRIDE); + yield* lockedStart(root, { host }).run; + const overridden = yield* readProjectLock(root); + yield* writeLock(root, { + ...overridden, + linked: { ...LINKED_MAIN, observedAt: "2026-01-01T00:00:00.000Z" }, + services: { + ...overridden.services, + rest: { ...overridden.services.rest, linked: { raw: "99.0.0", version: "v99.0.0" } }, + }, + }); + + yield* setConfig(root, ""); + yield* lockedStart(root, { host, existing: false }).run; + + const lock = yield* readProjectLock(root); + expect(lock.linked).toEqual({ ...LINKED_MAIN, observedAt: "2026-01-01T00:00:00.000Z" }); + expect(lock.services.rest).toMatchObject({ + specifier: "linked", + resolution: "resolved", + linked: { version: "v99.0.0" }, + resolved: { upstreamVersion: "v99.0.0", revision: 1 }, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("warns when the checkout targets another remote than the lock observed", () => + Effect.gen(function* () { + const root = yield* lockProject(); + yield* lockedStart(root).run; + const lock = yield* readProjectLock(root); + yield* writeLock(root, { + ...lock, + linked: { ...LINKED_MAIN, observedAt: "2026-01-01T00:00:00.000Z" }, + }); + yield* writeProjectLink(root, { + project: { ref: LINKED_MAIN.project }, + active_branch: { ref: LINKED_BRANCH_REF, is_default: false }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const { output, run } = lockedStart(root, { existing: false }); + yield* run; + + expect(output.messages).toContainEqual({ + type: "warn", + message: `lock observed ${LINKED_MAIN.project}, this checkout targets ${LINKED_MAIN.project}/${LINKED_BRANCH_REF}; run supabase stack update`, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("warns when this CLI bundles a newer revision of a locked version, offline", () => + Effect.gen(function* () { + const root = yield* lockProject(); + yield* lockedStart(root).run; + const lock = yield* readProjectLock(root); + const kind = artifactServiceKinds().find( + (service) => lock.services[service].resolved.revision > 0, + ); + if (kind === undefined) return expect.fail("no bundled pin has a revision above r0"); + const entry = lock.services[kind]; + yield* writeLock(root, { + ...lock, + services: { + ...lock.services, + [kind]: { + ...entry, + resolved: { + ...entry.resolved, + revision: 0, + image: entry.resolved.image.replace(/-r\d+@/u, "-r0@"), + }, + }, + }, + }); + + const { output, run } = lockedStart(root, { existing: false }); + yield* run; + + const { upstreamVersion, revision } = entry.resolved; + expect(output.messages).toContainEqual({ + type: "warn", + message: `This CLI bundles newer revisions of locked versions: ${kind} ${upstreamVersion}-r0 → ${upstreamVersion}-r${revision}. Run supabase stack outdated to review, supabase stack update to adopt them.`, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails with invalid-config on a missing signing keys file, writing no lock", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* lockProject('[auth]\nsigning_keys_path = "missing-keys.json"\n'); + + const error = yield* lockedStart(root).run.pipe(Effect.flip); + + expect(error).toMatchObject({ reason: "invalid-config" }); + expect(yield* fs.exists(lockPath(root))).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live( + "names [local.services.database] version when a service version swaps the database build", + () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* lockProject(); + const stack = fakeStack(); + yield* lockedStart(root, { stack }).run; + for (let revision = 0; revision <= orioledbPin.revision; revision++) + publishRelease(host, { + producer: "postgres", + upstream: orioledbPin.upstreamVersion, + revision, + }); + + yield* setConfig( + root, + `[local.services.database]\nversion = "${orioledbPin.upstreamVersion}"\n`, + ); + yield* stack.stack.composition.stop; + const error = yield* lockedStart(root, { stack, host }).run.pipe(Effect.flip); + + expect(error).toMatchObject({ + reason: "invalid-config", + message: expect.stringContaining( + `[local.services.database] version: saved ${selection.database.upstreamVersion}-r${selection.database.revision}, requested ${orioledbPin.upstreamVersion}-r${orioledbPin.revision}`, + ), + }); + expect(error.message).not.toContain("[db]"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live( + "starts an OrioleDB build the CLI does not bundle once the release host publishes it", + () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRelease(host, { + producer: "postgres", + upstream: "17.0.0.000-orioledb", + revision: 0, + }); + const root = yield* lockProject('[db]\norioledb_version = "17.0.0.000"\n'); + const { stack, run } = lockedStart(root, { host }); + yield* run; + + const lock = yield* readProjectLock(root); + expect(lock.services.database).toMatchObject({ + line: "orioledb", + resolved: { upstreamVersion: "17.0.0.000-orioledb", revision: 0 }, + }); + expect(memberPin(stack, "database")).toEqual(lock.services.database.resolved); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("re-resolves the database when db.major_version changes", () => + Effect.gen(function* () { + const root = yield* lockProject(); + yield* lockedStart(root).run; + + yield* setConfig(root, "[db]\nmajor_version = 15\n"); + const { output, run } = lockedStart(root, { existing: false }); + yield* run; + + const lock = yield* readProjectLock(root); + expect(lock.services.database).toMatchObject({ + majorVersion: 15, + line: "stock", + resolved: database15, + }); + expect(resolvedReports(output)[0]).toContain("db.major_version 15 ≠ locked 17"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails a frozen start on a missing or stale lock and leaves the project untouched", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(); + + const missing = lockedStart(root, { host, frozen: true }); + const notFound = yield* missing.run.pipe(Effect.flip); + expect(notFound).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-missing" }, + }); + expect(notFound.message).toContain( + "supabase/supabase.lock not found; run supabase start locally and commit it", + ); + expect(yield* fs.exists(lockPath(root))).toBe(false); + expect(yield* fs.exists(`${root}/supabase/.gitignore`)).toBe(false); + expect(yield* fs.exists(leasePath(root))).toBe(false); + + const stack = fakeStack(); + yield* lockedStart(root, { stack }).run; + const written = yield* lockText(root); + const ignored = yield* fs.readFileString(`${root}/supabase/.gitignore`); + yield* stack.stack.composition.stop; + yield* fs.remove(leasePath(root)); + yield* lockedStart(root, { stack, frozen: true }).run; + expect(yield* lockText(root)).toBe(written); + expect(yield* fs.exists(leasePath(root))).toBe(false); + + yield* setConfig(root, REST_OVERRIDE); + const composed = stack.composed; + yield* stack.stack.composition.stop; + const stale = yield* lockedStart(root, { stack, host, frozen: true }).run.pipe(Effect.flip); + expect(stale).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-stale" }, + }); + expect(stale.message).toContain( + "supabase/supabase.lock is stale for: rest (specifier v99.0.0 ≠ locked linked); run supabase start locally and commit the updated lock", + ); + expect(yield* lockText(root)).toBe(written); + expect(yield* fs.readFileString(`${root}/supabase/.gitignore`)).toBe(ignored); + expect(yield* fs.exists(leasePath(root))).toBe(false); + expect(stack.composed).toBe(composed); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails a frozen start on a stale lock while the stack is running", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(); + const stack = fakeStack(); + yield* lockedStart(root, { stack }).run; + const written = yield* lockText(root); + const composed = stack.composed; + + yield* setConfig(root, REST_OVERRIDE); + const stale = yield* lockedStart(root, { stack, host, frozen: true }).run.pipe(Effect.flip); + + expect(stale).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "frozen-stale" }, + }); + expect(yield* lockText(root)).toBe(written); + expect(stack.composed).toBe(composed); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live.each([ + { + name: "an unpublished release", + reason: "unpublished", + message: "no published slim-services release", + suffix: "invalid_config", + arrange: (_host: ReleaseFixture) => undefined, + }, + { + name: "a release host that is unavailable", + reason: "transient", + message: "could not be reached; retry shortly", + suffix: "network", + arrange: (host: ReleaseFixture) => { + publishRest(host); + host.failing.set("/releases/postgrest-v99.0.0-r0/SHA256SUMS", 503); + }, + }, + { + name: "a release that fails verification", + reason: "integrity", + message: "the published release failed verification", + suffix: "asset_checksum", + arrange: (host: ReleaseFixture) => { + publishRelease(host, { + producer: "postgrest", + upstream: "v99.0.0", + revision: 0, + tamperedManifest: true, + }); + }, + }, + ])("fails the start on $name and writes no lock", (row) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + row.arrange(host); + const root = yield* lockProject(REST_OVERRIDE); + const { stack, run } = lockedStart(root, { host }); + const error = yield* run.pipe(Effect.flip); + + expect(error).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: row.reason }, + }); + expect(error.message).toContain('local.services.rest.version = "v99.0.0"'); + expect(error.message).toContain(row.message); + expect(causeDeclaration(error)).toMatchObject({ fingerprint_suffix: row.suffix }); + expect(yield* fs.exists(lockPath(root))).toBe(false); + expect(stack.composed).toBe(0); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("resolves and warns to commit when CI starts without a lock, then freezes it", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(); + + const first = lockedStart(root, { ci: true }); + yield* first.run; + expect(first.output.messages).toContainEqual({ type: "warn", message: COMMIT_WARNING }); + const written = yield* lockText(root); + + yield* first.stack.stack.composition.stop; + const second = lockedStart(root, { stack: first.stack, ci: true }); + yield* second.run; + expect(second.output.messages.filter(({ type }) => type === "warn")).toEqual([]); + expect(resolvedReports(second.output)).toEqual([]); + + yield* setConfig(root, REST_OVERRIDE); + yield* first.stack.stack.composition.stop; + const stale = yield* lockedStart(root, { stack: first.stack, host, ci: true }).run.pipe( + Effect.flip, + ); + expect(stale.message).toContain("supabase/supabase.lock is stale for: rest"); + expect(yield* lockText(root)).toBe(written); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("uses a teammate's committed lock as-is, with no probe, link or login", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + publishRest(host); + const author = yield* lockProject(REST_OVERRIDE); + yield* lockedStart(author, { host }).run; + const requests = host.requests.length; + expect(requests).toBeGreaterThan(0); + + const teammate = yield* lockProject(REST_OVERRIDE); + yield* fs.copyFile(lockPath(author), lockPath(teammate)); + const { output, stack, run } = lockedStart(teammate, { host }); + yield* run; + + expect(host.requests).toHaveLength(requests); + expect(resolvedReports(output)).toEqual([]); + expect(yield* lockText(teammate)).toBe(yield* lockText(author)); + expect(memberPin(stack, "rest")).toMatchObject({ upstreamVersion: "v99.0.0", revision: 1 }); + expect(yield* fs.exists(`${teammate}/supabase/.link.json`)).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("serializes concurrent starts on the project lease so the versions resolve once", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject(REST_OVERRIDE); + const first = lockedStart(root, { host }); + const second = lockedStart(root, { host }); + yield* Effect.all([first.run, second.run], { concurrency: 2 }); + + const sums = host.requests.filter((path) => path.endsWith("/SHA256SUMS")); + expect(sums.length).toBeGreaterThan(0); + expect(new Set(sums).size).toBe(sums.length); + expect(resolvedReports(first.output).length + resolvedReports(second.output).length).toBe(1); + expect(memberPin(first.stack, "rest")).toEqual(memberPin(second.stack, "rest")); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("applies default to every service and lets an entry override it", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + publishRest(host); + const root = yield* lockProject( + `[local.services.default]\nversion = "bundled"\n[local.services.rest]\nversion = "v99.0.0-r1"\n`, + ); + yield* lockedStart(root, { host }).run; + + const lock = yield* readProjectLock(root); + expect(lock.services.auth).toMatchObject({ specifier: "bundled", resolution: "resolved" }); + expect(lock.services.rest).toMatchObject({ + specifier: "v99.0.0-r1", + resolved: { upstreamVersion: "v99.0.0", revision: 1 }, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("rejects orioledb_version next to a different database version", () => + Effect.gen(function* () { + const root = yield* lockProject( + `[db]\norioledb_version = "${orioledbConfigVersion}"\n[local.services.database]\nversion = "bundled"\n`, + ); + const error = yield* lockedStart(root).run.pipe(Effect.flip); + + expect(error).toMatchObject({ + reason: "invalid-config", + message: expect.stringContaining("conflicts with local.services.database.version"), + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("rejects a database version of another Postgres major than db.major_version", () => + Effect.gen(function* () { + const root = yield* lockProject( + `[db]\nmajor_version = 15\n[local.services.database]\nversion = "${selection.database.upstreamVersion}"\n`, + ); + const error = yield* lockedStart(root).run.pipe(Effect.flip); + + expect(error).toMatchObject({ + reason: "invalid-config", + message: expect.stringContaining("requires db.major_version"), + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("re-resolves a lock whose database pin was edited to contradict its own entry", () => + Effect.gen(function* () { + const root = yield* lockProject(); + yield* lockedStart(root).run; + const lock = yield* readProjectLock(root); + const { database } = lock.services; + const edits = [ + [{ ...database, majorVersion: 15 }, "db.major_version 17 ≠ locked 15"], + [{ ...database, line: "orioledb" }, "database line stock ≠ locked orioledb"], + [ + { ...database, resolved: database15 }, + `locked pin ${database15.upstreamVersion} is not Postgres 17`, + ], + ] as const; + for (const [edited, reason] of edits) { + yield* writeLock(root, { ...lock, services: { ...lock.services, database: edited } }); + const { output, run } = lockedStart(root, { existing: false }); + yield* run; + + expect(resolvedReports(output)[0]).toContain(reason); + expect((yield* readProjectLock(root)).services.database.resolved).toEqual( + selection.database, + ); + } + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("fails on a malformed or newer lock whether or not the start is frozen", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* lockProject(); + for (const [contents, advice] of [ + ["{ not json", "delete it and run supabase start to resolve the versions again"], + ['{"lockfileVersion":2}', "upgrade the CLI"], + ] as const) { + yield* fs.writeFileString(lockPath(root), contents); + for (const frozen of [false, true]) { + const error = yield* lockedStart(root, { frozen }).run.pipe(Effect.flip); + expect(error).toMatchObject({ + reason: "lock", + cause: { _tag: "LockResolutionError", reason: "invalid-lock" }, + }); + expect(error.message).toContain(advice); + expect(yield* lockText(root)).toBe(contents); + } + } + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/experimental/stack/start/start.native.integration.test.ts b/apps/cli/src/commands/experimental/stack/start/start.native.integration.test.ts index ee6b7fa2c2..f0cf154f4e 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.native.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.native.integration.test.ts @@ -20,7 +20,7 @@ import { stdinLayer } from "../../../../shared/runtime/stdin.layer.ts"; import { runtimeInfoLayer } from "../../../../shared/runtime/runtime-info.layer.ts"; import { ExperimentalFlag, YesFlag } from "../../../../command-internal/global-flags.ts"; import { stackStart } from "./start.handler.ts"; -import { stackPrepare } from "../prepare/prepare.handler.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; import { destroyTestStacks } from "../../../../../tests/helpers/stack-cleanup.ts"; const excluded = [ @@ -40,16 +40,10 @@ const flags = (exclude: ReadonlyArray) => ({ stackId: Option.none(), runtime: "native" as const, preparation: "on-demand" as const, + frozen: false, eager: false, }); -const prepareFlags = () => ({ - stack: Option.none(), - stackId: Option.none(), - runtime: "native" as const, - capability: [], -}); - const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); const captureOwnerPid = (capture: (pid: number) => void) => @@ -230,8 +224,16 @@ describe("experimental stack start native lifecycle", () => { const fixture = makeLayers(root); yield* Effect.ensuring( Effect.gen(function* () { - yield* Effect.scoped(stackPrepare(prepareFlags())); const api = yield* StackApi; + yield* Effect.scoped( + api.create({ + projectRoot: root, + stateRoot: path.join(root, "stacks"), + cacheRoot: path.join(root, "cache"), + runtime: "native", + startOwner: true, + }), + ); const preparedDefinitions = yield* api.discover({ stateRoot: path.join(root, "stacks"), }); @@ -671,4 +673,66 @@ describe("experimental stack start native lifecycle", () => { }).pipe(Effect.provide(BunServices.layer)), { timeout: 120_000 }, ); + + it.live( + "refuses an Auth older than the one that migrated the data, and accepts a newer one", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-start-downgrade-" }); + yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(root, "supabase", "config.toml"), + n2ProjectConfig(undefined), + ); + const locations = { + stateRoot: path.join(root, "stacks"), + cacheRoot: path.join(root, "cache"), + }; + const fixture = makeLayers(root); + yield* Effect.ensuring( + Effect.gen(function* () { + const api = yield* StackApi; + const stackId = yield* stackStart(flags(onlyRestAndAuth)); + const stack = yield* api.open({ id: stackId, ...locations }); + const database = (yield* stack.services.list).find( + (instance) => instance.service === "database", + ); + if (database === undefined) return yield* Effect.die("database missing"); + const record = path.join( + locations.stateRoot, + stackId, + "data", + database.id, + "schema-writers.json", + ); + expect(yield* fs.readFileString(record)).toContain(selection.auth.upstreamVersion); + yield* stack.stop; + + yield* fs.writeFileString(record, '{"auth":"v999.0.0"}'); + const refused = yield* Effect.scoped(Effect.exit(stackStart(flags(onlyRestAndAuth)))); + expect(Exit.isFailure(refused)).toBe(true); + if (!Exit.isFailure(refused)) return; + const message = Cause.pretty(refused.cause); + expect(message).toContain( + `auth ${selection.auth.upstreamVersion} is older than v999.0.0`, + ); + expect(message).toContain(`supabase stack destroy --stack-id ${stackId}`); + expect(yield* fs.readFileString(record)).toContain("v999.0.0"); + + yield* fs.writeFileString(record, '{"auth":"v0.0.1"}'); + expect(yield* stackStart({ ...flags(onlyRestAndAuth), eager: true })).toBe(stackId); + const raised = yield* fs.readFileString(record); + expect(raised).toContain(selection.auth.upstreamVersion); + expect(raised).not.toContain("v0.0.1"); + }), + Effect.gen(function* () { + const api = yield* StackApi; + yield* destroyTestStacks(api, locations.stateRoot, locations.cacheRoot); + }), + ).pipe(Effect.provide(fixture.layer)); + }).pipe(Effect.provide(BunServices.layer)), + { timeout: 180_000 }, + ); }); diff --git a/apps/cli/src/commands/experimental/stack/start/start.options.ts b/apps/cli/src/commands/experimental/stack/start/start.options.ts index dd89edeedd..13b810c47b 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.options.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.options.ts @@ -9,8 +9,3 @@ export const STACK_START_EXCLUDABLE_CAPABILITIES = [ "analytics", "pooler", ] as const; - -export const STACK_PREPARABLE_CAPABILITIES = [ - "database", - ...STACK_START_EXCLUDABLE_CAPABILITIES, -] as const; diff --git a/apps/cli/src/commands/experimental/stack/status/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/status/SIDE_EFFECTS.md index 1c2b7673c6..547ac793d7 100644 --- a/apps/cli/src/commands/experimental/stack/status/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/experimental/stack/status/SIDE_EFFECTS.md @@ -27,9 +27,11 @@ composition members through the stack package's composition plan. It reads Values that the composition or stack credentials supply are ignored, as are changes to membership; an excluded service is not considered drift. The comparison reads saved state only, so it is available while the owner is unavailable. It reports `config_drift.status` as `unchanged` or -`changed`, with `services..` paths, when the comparison is possible. A configuration -loading failure or unreadable saved state keeps the saved stack report available and is shown as -`config_drift.status: "unavailable"` with a message in JSON. Status does not apply current configuration. The `services` list may include +`changed`, with `services..` paths, when the comparison is possible. The requested +versions come from `supabase/supabase.lock`, read without the project lease and never resolved; a +missing or stale lock, like a configuration loading failure or unreadable saved state, keeps the +saved stack report available and is shown as `config_drift.status: "unavailable"` with a message +in JSON. Status does not apply current configuration. The `services` list may include saved standalone instances; composition members identify the services used for primary database, environment export, and drift comparisons. diff --git a/apps/cli/src/commands/experimental/stack/status/status.handler.ts b/apps/cli/src/commands/experimental/stack/status/status.handler.ts index 918462bcc1..5671768029 100644 --- a/apps/cli/src/commands/experimental/stack/status/status.handler.ts +++ b/apps/cli/src/commands/experimental/stack/status/status.handler.ts @@ -8,6 +8,7 @@ import { CommandSettings } from "../../../../config/command-settings.service.ts" import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import type { StackRuntime } from "../../../../command-internal/stack-runtime.ts"; import { loadStackConfig } from "../../../../command-internal/stack-config.ts"; +import { readLockedSelection } from "../../../../command-internal/stack-lock.ts"; import { withProjectFunctionsEnv } from "../../../../command-internal/stack-functions-env.ts"; import { bold, gray, green, red, yellow } from "../../../../command-internal/colors.ts"; import { @@ -313,9 +314,12 @@ const unavailableDrift = (message: string): StackReport["config_drift"] => ({ message, }); +const projectConfigUnavailable = (error: { readonly message: string }) => + Effect.succeed(unavailableDrift(`Project configuration could not be compared: ${error.message}`)); + const configDrift = Effect.fn("experimental.stack.status.configDrift")( function* (stack: Stack, projectRoot: string, functionsIsMember: boolean) { - const loaded = yield* loadStackConfig(projectRoot); + const loaded = yield* loadStackConfig(projectRoot, { selection: readLockedSelection }); const creations = yield* loaded.creations(stack.id); // Drift ignores non-members, so a Functions dotenv only matters when Functions is a member. const requested = functionsIsMember @@ -326,14 +330,9 @@ const configDrift = Effect.fn("experimental.stack.status.configDrift")( (effect) => effect.pipe( Effect.catchTags({ - StackConfigError: (error) => - Effect.succeed( - unavailableDrift(`Project configuration could not be compared: ${error.message}`), - ), - StackFunctionsEnvError: (error) => - Effect.succeed( - unavailableDrift(`Project configuration could not be compared: ${error.message}`), - ), + StackConfigError: projectConfigUnavailable, + LockResolutionError: projectConfigUnavailable, + StackFunctionsEnvError: projectConfigUnavailable, StackError: (error) => Effect.succeed( unavailableDrift(`Saved configuration could not be compared: ${error.message}`), diff --git a/apps/cli/src/commands/experimental/stack/status/status.integration.test.ts b/apps/cli/src/commands/experimental/stack/status/status.integration.test.ts index a188f02817..03857032ba 100644 --- a/apps/cli/src/commands/experimental/stack/status/status.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/status/status.integration.test.ts @@ -18,6 +18,8 @@ import { runtimeInfoLayer } from "../../../../shared/runtime/runtime-info.layer. import { StackApi, StackTargetResolver } from "../stack.shared.ts"; import type { StackStatusFlags } from "./status.command.ts"; import { stackStatus } from "./status.handler.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; +import { seedLock } from "../../../../../tests/helpers/stack-lock.ts"; const stackId = "a".repeat(64); type StatusOutputFormat = "text" | "json" | "stream-json"; @@ -40,8 +42,8 @@ const savedCredentials: StackCredentials = { }; const database: ServiceCreation = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -50,21 +52,25 @@ const database: ServiceCreation = { }; const rest: ServiceCreation = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: 54321 } }, }; const auth: ServiceCreation = { service: "auth", + pin: selection.auth, config: { jwtSecret }, endpoints: { http: { port: 54321 } }, }; const studio: ServiceCreation = { service: "studio", + pin: selection.studio, config: {}, endpoints: { http: { port: 54323 } }, }; const functions: ServiceCreation = { service: "functions", + pin: selection.functions, config: { functionsRoot: "/project/supabase/functions", bootstrap: "export default {};", @@ -74,8 +80,9 @@ const functions: ServiceCreation = { }; const storage = (s3ProtocolEnabled: boolean): ServiceCreation => ({ service: "storage", + pin: selection.storage, config: { - filePath: "/project/supabase/.temp/stack-uploads", + filePath: "/home/stacks/stack-id/uploads", jwtSecret, s3ProtocolEnabled, s3AccessKeyId: "local-access-key", @@ -272,6 +279,8 @@ const runStatus = (input: { BunServices.layer, runtimeInfoLayer, ); + if (input.config === "explicit" || input.config === "multiline-functions-env") + yield* seedLock(root).pipe(Effect.provide(layer)); const effect = stackStatus(input.flags ?? flags()).pipe(Effect.provide(layer)); return { effect, out, root }; }).pipe(Effect.provide(BunServices.layer)); @@ -668,7 +677,8 @@ it.effect("omits Storage S3 details for a Storage member saved without S3 keys", const env = yield* runStatus({ services: storageServices({ service: "storage", - config: { filePath: "/project/supabase/.temp/stack-uploads", jwtSecret }, + pin: selection.storage, + config: { filePath: "/home/stacks/stack-id/uploads", jwtSecret }, endpoints: { http: { port: 54321 } }, }), reachable: true, @@ -796,8 +806,8 @@ it.effect("derives DB_URL using the postgres role and a non-default saved passwo const password = "p@ss w/ord!"; const customDatabase: ServiceCreation = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(password), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, diff --git a/apps/cli/src/commands/experimental/stack/stop/stop.integration.test.ts b/apps/cli/src/commands/experimental/stack/stop/stop.integration.test.ts index e7b8f7b11c..b1ff80586c 100644 --- a/apps/cli/src/commands/experimental/stack/stop/stop.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stop/stop.integration.test.ts @@ -10,6 +10,7 @@ import { import * as StackNamespace from "../../../../../../../packages/stack/src/StackNamespace.ts"; import { StackApi, stackApiLayer, stackTargetResolverLayer } from "../stack.shared.ts"; import { stackStop } from "./stop.handler.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; const flags = (id?: string) => ({ all: Option.none(), @@ -42,7 +43,12 @@ describe("stack stop", () => { const f = yield* fixture(); const stopped = yield* Ref.make(false); yield* Effect.acquireUseRelease( - f.stack.services.create({ service: "mail", config: {}, endpoints: {} }), + f.stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + endpoints: {}, + }), (mail) => Effect.gen(function* () { expect((yield* f.api.discover(f.locations))[0]?.host).toBeDefined(); diff --git a/apps/cli/src/commands/experimental/stack/update/SIDE_EFFECTS.md b/apps/cli/src/commands/experimental/stack/update/SIDE_EFFECTS.md new file mode 100644 index 0000000000..6aca800265 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/update/SIDE_EFFECTS.md @@ -0,0 +1,114 @@ +# `supabase stack update [service...]` + +The command is available when the experimental stack family is enabled. It re-resolves the named +services of `supabase/supabase.lock`, or every service when none is named, and writes the lock. It +creates, starts and registers no stack and downloads no artifact, so it needs no stack selector. It +reads saved stack state only to warn about database data a new pin would be refused by (see +"Refused data"). + +## Service versions and the lock + +The lock is read and written as `supabase stack start` does it (see "Service versions and the lock" +in [`../start/SIDE_EFFECTS.md`](../start/SIDE_EFFECTS.md)), under the same project lease +`supabase/.supabase.lock.lease`. Each named service is resolved again even when its entry is fresh: + +- `` (a version without a revision) moves to the newest published revision. +- `-r` names one exact release and keeps its pin. +- `bundled` moves to the version bundled with this CLI. +- `linked` probes the hosted project or branch named by `supabase/.link.json` again (the + `update.probe` span) and resolves to the version it reports, retrying a service that fell back as + `unobserved` or `unpublished`. Entries that follow another specifier take the new observation in + their `linked` field. When the target differs from the remote the lock observed, every `linked` + entry is resolved again against the new target and nothing from the old remote is kept. A probe + that fails transiently keeps the previous entry only when nothing it depends on changed, else + takes the bundled version as `unobserved`, with a warning. Without `supabase/.link.json`, or + when no `linked` entry is re-resolved, no probe runs and the stored observations apply; an + unlinked project resolves `linked` to the bundled version as `unlinked`. +- A named service whose specifier changed since the lock was written is resolved again. A stale + service that was not named keeps its entry; the command reports it as skipped and leaves it for + `supabase start` or `supabase stack update `. +- A missing lock is resolved in full. + +Services that are not named keep their entries byte for byte. The lock is written only when an +entry changed. The lease file is created on every run, including one that changes nothing, and +`supabase/.gitignore` gains the lease and staged-write entries whenever the lease is taken. When +nothing changed and no stale service was skipped, the command reports +`supabase/supabase.lock is already up to date.`. + +`update` takes no `--frozen`, and `CI` does not freeze it: running it is the request to rewrite +the lock. A project without `supabase/config.toml` fails with reason `invalid-config`. A service +name that is not one of the twelve services fails with reason `flags` and lists the valid names; +nothing is resolved or written. + +## Refused data + +When the command writes the lock and releases the project lease, it checks the new pins against +the database data of every saved stack of this project, by the rules the next `supabase stack +start` of that stack applies. The lock is written either way. Each refusal gets one +warning, `stack : the next supabase start refuses these pins:` followed by the +failure that start reports: a database build other than a new `-rN` revision of the one that +initialized the data (on Docker and Podman, also the one an interrupted first start recorded), or a schema-owning service the stack runs against that database that is +older than one that already migrated it. The failure names the recorded and refused versions and +the remedies: pinning the recorded version in `supabase/config.toml` +(`[local.services.] version`, with `[db] major_version` set back when the database major +differs and any `[db] orioledb_version` removed when an OrioleDB build is requested), or +`supabase stack destroy --stack-id `, which permanently deletes the stack's local database +data. Data with no recorded identity offers only the destroy. The check is skipped when nothing was +written, the span records `lock.data_conflict_count`, and saved state that cannot be read is logged +at debug level without failing the command. + +## Files read + +- `/supabase/config.toml`, project dotenv files, supported `SUPABASE_*` overrides, + `/supabase/supabase.lock`, and `/supabase/.link.json`. +- Shared profile and telemetry state under `/`. +- When the lock is written, after the lease is released: the saved stack definitions under + `/stacks/` and, in each database's data directory, its readiness + marker, `data/PG_VERSION`, and build and schema-writer records. No stack lease is taken, no owner + is probed, and a missing `stacks/` directory is not created. + +## Files written + +- `/supabase/supabase.lock` (JSON), only when an entry changed. +- `/supabase/.supabase.lock.lease` on every run, and `.supabase.lock.*.tmp` while the lock + is written; `supabase/.gitignore` gains both entries on every run, when the lease is taken. +- Telemetry state after success or failure. + +## Network and subprocesses + +Resolving a release specifier probes the slim-services GitHub releases and registry +(`SUPABASE_INTERNAL_SLIM_RELEASES_URL` and `SUPABASE_INTERNAL_SLIM_REGISTRY_URL` redirect them). +Only the named services of the `` or `linked` form are probed there. + +Only when a named service has the `linked` specifier and `supabase/.link.json` exists does the +command also need a login (the same as `supabase link`; `start` never does) and call the Management +API (`getProject`, or `getABranchConfig` for a branch; `getProjectApiKeys` with `reveal`) and the +target's tenant host (`/rest/v1/`, `/auth/v1/health`, `/storage/v1/version`, with the service-role +key, which is never logged). A missing login, or a failed Management API call, fails with reason +`hosted` and writes nothing. No archive or image is downloaded and no subprocess runs. + +## Flags and output + +Positional `service` names; the legacy `-o/--output` flag is rejected, use `--output-format`. + +A change that moves only the resolution (for example `unobserved` to `resolved`) shows the +version with the new resolution in parentheses, and a `linked` entry's reason is `follows the +linked project`. Text output lists each changed service as ` → ()`, a summary of +changed and unchanged entries, and one ` is stale (); run supabase start or +supabase stack update ` line per skipped stale service. + +JSON output returns `lock` (`file`, `written`, `added`, `changed`, `unchanged`, `stale` as the +names of skipped stale services) and an empty +message. Errors retain typed actionability. + +## Exit codes + +- `0` when the lock was updated or left as it was, including when stale services were skipped. +- `130` when interrupted. +- `1` for a rejected output flag, an unknown service, invalid configuration, a lock that cannot be + read, a failed resolution, or a failed hosted probe (reason `hosted`). + +## Telemetry + +The command emits the standard `cli_command_executed` event through command telemetry and flushes +telemetry state after success or failure. It emits no custom event. diff --git a/apps/cli/src/commands/experimental/stack/update/update.command.ts b/apps/cli/src/commands/experimental/stack/update/update.command.ts new file mode 100644 index 0000000000..bfa250d993 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/update/update.command.ts @@ -0,0 +1,61 @@ +import { artifactServiceKinds } from "@supabase/stack/internal/artifacts"; +import { Argument, Command } from "effect/cli"; +import type * as CliCommand from "effect/cli/Command"; +import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withCommandTelemetry } from "../../../../telemetry/command-telemetry.ts"; +import { stackUpdate } from "./update.handler.ts"; + +const config = { + services: Argument.String("service").pipe( + Argument.withDescription( + `Services to update; every service when omitted. One of: ${artifactServiceKinds().join(", ")}.`, + ), + Argument.variadic(), + ), +} as const; + +export type StackUpdateFlags = CliCommand.Command.Config.Infer; + +export const stackUpdateCommand = Command.make("update", config).pipe( + Command.withDescription( + "Re-resolve the named local service versions (all when none are named) in supabase/supabase.lock, even when their entries are up to date: " + + "a release specifier moves to its newest published revision, a bundled entry to this CLI's pin, and an exact revision stays. " + + "A linked entry re-probes the linked project or branch, which needs a login, and observed or published versions are retried. " + + "Nothing is downloaded; supabase start and supabase stack install fetch what the lock names.", + ), + Command.withShortDescription("Update locked local service versions"), + Command.withExamples([ + { + command: "supabase stack update", + description: "Move every service to what its specifier now resolves to", + }, + { + command: "supabase stack update auth studio", + description: "Update only auth and studio", + }, + ]), + withPermissions({ + operations: [ + { + operationId: "v1-get-project", + kind: "best-effort", + context: "supabase/.link.json exists and a linked service is re-resolved", + }, + { + operationId: "v1-get-project-api-keys", + kind: "best-effort", + request: { query: { reveal: "true" } }, + context: "supabase/.link.json exists and a linked service is re-resolved", + }, + { + operationId: "v1-get-a-branch-config", + kind: "best-effort", + context: "the linked ref is a branch", + }, + ], + }), + Command.withHandler((flags) => + stackUpdate(flags).pipe(withCommandTelemetry({ flags }), withJsonErrorHandling), + ), +); diff --git a/apps/cli/src/commands/experimental/stack/update/update.errors.ts b/apps/cli/src/commands/experimental/stack/update/update.errors.ts new file mode 100644 index 0000000000..12643cb2b8 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/update/update.errors.ts @@ -0,0 +1,37 @@ +import { Data } from "effect"; +import { + actionability, + causeDeclaration, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../../../../shared/telemetry/error-actionability.ts"; + +export class StackCommandUpdateError extends Data.TaggedError("ExperimentalStackUpdateError")<{ + readonly reason: "flags" | "invalid-config" | "lock" | "hosted"; + readonly message: string; + readonly suggestion?: string; + readonly cause?: unknown; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + switch (this.reason) { + case "flags": + return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; + case "invalid-config": + return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + case "hosted": + return ( + causeDeclaration(this.cause) ?? { + ...actionability.externalNetwork, + fingerprint_suffix: "network", + } + ); + case "lock": + return ( + causeDeclaration(this.cause) ?? { + ...actionability.invalidConfig, + fingerprint_suffix: "invalid_config", + } + ); + } + } +} diff --git a/apps/cli/src/commands/experimental/stack/update/update.handler.ts b/apps/cli/src/commands/experimental/stack/update/update.handler.ts new file mode 100644 index 0000000000..393c519f3e --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/update/update.handler.ts @@ -0,0 +1,143 @@ +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { artifactServiceKinds, type ServiceKind } from "@supabase/stack/internal/artifacts"; +import { Effect, Option, Ref, Schema } from "effect"; +import { CommandPlatformApiFactory } from "../../../../auth/command-platform-api-factory.service.ts"; +import { probeLinkedTarget } from "../../../../command-internal/hosted-versions.ts"; +import { readProjectLink } from "../../../../command-internal/project-link.ts"; +import { OutputFlag } from "../../../../command-internal/global-flags.ts"; +import { loadLocalProjectContext } from "../../../../command-internal/local-project-context.ts"; +import { loadStackConfig, StackConfigError } from "../../../../command-internal/stack-config.ts"; +import { + LOCK_FILE, + LockResolutionError, + updatedLock, + type LockUpdate, +} from "../../../../command-internal/stack-lock.ts"; +import { CommandSettings } from "../../../../config/command-settings.service.ts"; +import { Output } from "../../../../shared/output/output.service.ts"; +import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; +import { mapTargetError, rejectStackOutput } from "../stack.shared.ts"; +import { StackCommandUpdateError } from "./update.errors.ts"; +import type { StackUpdateFlags } from "./update.command.ts"; + +const serviceKinds = (names: ReadonlyArray) => { + const valid = artifactServiceKinds(); + const unknown = names.filter((name) => !valid.some((kind) => kind === name)); + return unknown.length === 0 + ? Effect.succeed(valid.filter((kind) => names.includes(kind))) + : Effect.fail( + new StackCommandUpdateError({ + reason: "flags", + message: `Unknown service ${unknown.map((name) => `"${name}"`).join(", ")}; valid services are ${valid.join(", ")}.`, + }), + ); +}; + +const render = (update: LockUpdate) => { + const skipped = update.stale.map( + ({ service, reason }) => + `${service} is stale (${reason}); run supabase start or supabase stack update ${service}`, + ); + if (!update.written) + return skipped.length === 0 + ? `${LOCK_FILE} is already up to date.\n` + : `${skipped.join("\n")}\n`; + const lines = update.changes.map(({ service, from, to, reason }) => { + const arrow = from === undefined ? "" : `${from} → `; + return ` ${service.padEnd(9)} ${arrow}${to} (${reason})`; + }); + return `${["Updated service versions:", ...lines, `Updated ${LOCK_FILE} (${update.changes.length} changed, ${update.unchanged.length} unchanged).`, ...skipped].join("\n")}\n`; +}; + +const lockReport = (update: LockUpdate) => ({ + file: LOCK_FILE, + written: update.written, + added: update.changes.flatMap(({ service, from, to, reason }) => + from === undefined ? [{ service, to, reason }] : [], + ), + changed: update.changes.flatMap(({ service, from, to, reason }) => + from === undefined ? [] : [{ service, from, to, reason }], + ), + unchanged: update.unchanged, + stale: update.stale.map(({ service }) => service), +}); + +/** Re-resolves the named service versions in the project lock and writes it, without downloading anything. */ +export const stackUpdate = Effect.fn("experimental.stack.update")(function* ( + flags: StackUpdateFlags, +) { + const telemetryState = yield* TelemetryState; + const body = Effect.gen(function* () { + const output = yield* Output; + const settings = yield* CommandSettings; + const outputFlag = yield* Effect.serviceOption(OutputFlag); + yield* rejectStackOutput(outputFlag).pipe( + Effect.mapError(mapTargetError((props) => new StackCommandUpdateError(props))), + ); + const services: ReadonlyArray = yield* serviceKinds(flags.services); + + const updated = yield* Ref.make(undefined); + // The hosted probe needs a login, so it runs only when a linked entry is re-resolved. + const probe = Effect.gen(function* () { + const link = yield* readProjectLink(settings.workdir); + if (Option.isNone(link)) return Option.none(); + const api = yield* (yield* CommandPlatformApiFactory).make; + return Option.some(yield* probeLinkedTarget(api, link.value)); + }).pipe( + Effect.mapError( + (error) => + new StackCommandUpdateError({ + reason: "hosted", + message: `Could not read the hosted versions of the linked project: ${error.message}`, + suggestion: "Run supabase login, then supabase link, and retry.", + cause: error, + }), + ), + ); + const update = updatedLock({ services, probe, output }); + yield* Effect.gen(function* () { + const context = yield* loadLocalProjectContext( + settings.workdir, + (message) => new StackConfigError({ message }), + ); + return yield* loadStackConfig(settings.workdir, { + context, + publishesLock: true, + selection: (request) => + update(request).pipe( + Effect.tap((result) => Ref.set(updated, result)), + Effect.as(defaultArtifactSelection()), + ), + }); + }).pipe( + Effect.mapError((error) => + error instanceof StackCommandUpdateError + ? error + : new StackCommandUpdateError({ + reason: Schema.is(LockResolutionError)(error) ? "lock" : "invalid-config", + message: error.message, + cause: error, + }), + ), + Effect.withSpan("update.resolve"), + ); + const result = yield* Ref.get(updated); + if (result === undefined) + return yield* new StackCommandUpdateError({ + reason: "invalid-config", + message: "No supabase/config.toml in this project, so there is no lock to update.", + suggestion: "Run supabase init, then supabase start to create supabase/supabase.lock.", + }); + + yield* Effect.annotateCurrentSpan({ + "update.requested_count": services.length, + "update.updated_count": result.changes.length, + "update.unchanged_count": result.unchanged.length, + "update.stale_count": result.stale.length, + "update.lock_written": result.written, + }); + if (output.format === "text") yield* output.raw(render(result)); + else yield* output.success("", { lock: lockReport(result) }); + }); + return yield* body.pipe(Effect.ensuring(telemetryState.flush)); +}); diff --git a/apps/cli/src/commands/experimental/stack/update/update.integration.test.ts b/apps/cli/src/commands/experimental/stack/update/update.integration.test.ts new file mode 100644 index 0000000000..6649d42485 --- /dev/null +++ b/apps/cli/src/commands/experimental/stack/update/update.integration.test.ts @@ -0,0 +1,441 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { readLock } from "@supabase/stack/effect"; +import { + artifactServiceKinds, + catalogPins, + postgresMajor, + type ServiceKind, +} from "@supabase/stack/internal/artifacts"; +import { Effect, FileSystem, Layer, Option } from "effect"; +import * as HttpClient from "effect/http/HttpClient"; +import { CommandPlatformApiFactory } from "../../../../auth/command-platform-api-factory.service.ts"; +import { + mockCommandSettings, + mockTelemetryStateTracked, + withConfigEnv, +} from "../../../../../tests/helpers/command-mocks.ts"; +import { mockOutput, mockRuntimeInfo } from "../../../../../tests/helpers/mocks.ts"; +import { selection } from "../../../../../tests/helpers/pins.ts"; +import { + publishRelease, + releaseHostEnv, + startReleaseFixture, + type ReleaseFixture, +} from "../../../../../tests/helpers/release-fixture.ts"; +import { + rebuilt, + seedLock, + seedProjectStack, + unbundled, +} from "../../../../../tests/helpers/stack-lock.ts"; +import { ExperimentalFlag } from "../../../../command-internal/global-flags.ts"; +import { StackCommandUpdateError } from "./update.errors.ts"; +import { stackUpdate } from "./update.handler.ts"; + +const UPSTREAM = "v99.0.0"; + +const publish = (host: ReleaseFixture, kind: ServiceKind, revision: number) => { + const producer = catalogPins().find( + (entry) => entry.service === kind && entry.isDefault, + )?.sourceService; + if (producer === undefined) throw new Error(`no producer for ${kind}`); + publishRelease(host, { producer, upstream: UPSTREAM, revision }); +}; + +/** Every service follows the bundled pin except those named, which request a published release. */ +const project = (requested: Partial>) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-update-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + [ + 'project_id = "stack-update"', + '[local.services.default]\nversion = "bundled"', + ...Object.entries(requested).map( + ([kind, version]) => `[local.services.${kind}]\nversion = "${version}"`, + ), + ].join("\n\n"), + ); + return root; + }); + +const layer = (root: string, output: ReturnType) => + Layer.mergeAll( + BunServices.layer, + mockRuntimeInfo({ platform: "linux", arch: "x64" }), + output.layer, + mockTelemetryStateTracked().layer, + mockCommandSettings({ workdir: root }), + Layer.succeed(ExperimentalFlag, false), + // No linked entry is re-resolved here, so nothing asks the hosted project. + Layer.succeed(CommandPlatformApiFactory, { make: Effect.die("unused") }), + Layer.succeed( + HttpClient.HttpClient, + HttpClient.make(() => Effect.die("unused")), + ), + ); + +const seed = (root: string, host: ReleaseFixture, edit?: Parameters[1]) => + withConfigEnv(releaseHostEnv(host), seedLock(root, edit)).pipe( + Effect.provide(layer(root, mockOutput())), + ); + +const update = ( + root: string, + host: ReleaseFixture, + services: ReadonlyArray, + options: { readonly format?: "text" | "json"; readonly ci?: boolean } = {}, +) => { + const output = mockOutput({ format: options.format ?? "text" }); + const run = withConfigEnv( + { CI: options.ci === true ? "true" : "", ...releaseHostEnv(host) }, + stackUpdate({ services }), + ).pipe(Effect.provide(layer(root, output))); + return { output, run }; +}; + +const lockPath = (root: string) => `${root}/supabase/supabase.lock`; + +const readServices = (root: string) => + readLock(root).pipe( + Effect.map(Option.getOrThrow), + Effect.map((lock) => lock.services), + ); + +describe("experimental stack update", () => { + it.live("moves the named service to its newest revision and keeps every other entry", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM, rest: UPSTREAM }); + publish(host, "auth", 0); + publish(host, "rest", 0); + yield* seed(root, host); + const before = yield* readServices(root); + publish(host, "auth", 1); + publish(host, "rest", 1); + host.requests.length = 0; + + const { output, run } = update(root, host, ["auth"]); + yield* run; + + const after = yield* readServices(root); + expect(after.auth.resolved.revision).toBe(1); + const { auth: _updated, ...others } = after; + const { auth: _previous, ...unchanged } = before; + expect(others).toEqual(unchanged); + expect(output.stdoutText).toContain("auth"); + expect(output.stdoutText).toContain(`${UPSTREAM}-r0 → ${UPSTREAM}-r1`); + expect(output.stdoutText).toContain("1 changed, 11 unchanged"); + expect(host.requests.filter((path) => path.includes("/releases/rest-"))).toEqual([]); + expect(host.requests.filter((path) => path.endsWith(".tar.zst"))).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live( + "updates every service when none is named, moving bundled to this CLI and keeping an exact revision", + () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM, rest: UPSTREAM, storage: `${UPSTREAM}-r1` }); + for (const kind of ["auth", "rest"] as const) publish(host, kind, 0); + for (const revision of [0, 1]) publish(host, "storage", revision); + yield* seed(root, host, (services) => ({ + ...services, + studio: { ...services.studio, resolved: unbundled(services.studio.resolved) }, + })); + for (const kind of ["auth", "rest"] as const) publish(host, kind, 1); + publish(host, "storage", 2); + host.requests.length = 0; + + const { output, run } = update(root, host, [], { format: "json" }); + yield* run; + + const after = yield* readServices(root); + expect(after.auth.resolved.revision).toBe(1); + expect(after.rest.resolved.revision).toBe(1); + expect(after.storage.resolved.revision).toBe(1); + expect(after.studio.resolved).toMatchObject({ + upstreamVersion: selection.studio.upstreamVersion, + revision: selection.studio.revision, + image: selection.studio.image, + }); + expect(host.requests.filter((path) => path.includes("/releases/storage-"))).toEqual([]); + const result = output.messages.find((entry) => entry.type === "success"); + expect(result?.type === "success" ? result.data : undefined).toMatchObject({ + lock: { + file: "supabase/supabase.lock", + written: true, + added: [], + changed: [ + { service: "rest", from: `${UPSTREAM}-r0`, to: `${UPSTREAM}-r1` }, + { service: "auth", from: `${UPSTREAM}-r0`, to: `${UPSTREAM}-r1` }, + { service: "studio" }, + ], + }, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("rewrites the lock under CI, which does not freeze an explicit update", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM }); + publish(host, "auth", 0); + yield* seed(root, host); + publish(host, "auth", 1); + + yield* update(root, host, ["auth"], { ci: true }).run; + + expect((yield* readServices(root)).auth.resolved.revision).toBe(1); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("resolves every service when the project has no lock yet", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM }); + publish(host, "auth", 0); + + yield* update(root, host, ["auth"]).run; + + const services = yield* readServices(root); + expect(Object.keys(services).toSorted()).toEqual([...artifactServiceKinds()].toSorted()); + expect(services.auth.resolved.revision).toBe(0); + expect(yield* fs.readFileString(`${root}/supabase/.gitignore`)).toContain( + ".supabase.lock.lease", + ); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("leaves the lock byte for byte alone when every entry already answers the config", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: `${UPSTREAM}-r1` }); + publish(host, "auth", 1); + yield* seed(root, host); + const written = yield* fs.readFileString(lockPath(root)); + + const { output, run } = update(root, host, ["auth"]); + yield* run; + + expect(output.stdoutText).toContain("already up to date"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + const json = update(root, host, ["auth"], { format: "json" }); + yield* json.run; + const result = json.output.messages.find((entry) => entry.type === "success"); + expect(result?.type === "success" ? result.data : undefined).toMatchObject({ + lock: { written: false, changed: [], stale: [] }, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + describe("a service whose specifier changed since the lock was written", () => { + const staleProject = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM, rest: UPSTREAM }); + publish(host, "auth", 0); + publish(host, "rest", 0); + yield* seed(root, host); + publish(host, "rest", 1); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + `project_id = "stack-update"\n\n[local.services.default]\nversion = "bundled"\n\n[local.services.auth]\nversion = "${UPSTREAM}"\n\n[local.services.rest]\nversion = "${UPSTREAM}-r1"\n`, + ); + return { fs, host, root }; + }); + + it.live("is resolved again when named", () => + Effect.gen(function* () { + const { host, root } = yield* staleProject; + + const { output, run } = update(root, host, ["rest"]); + yield* run; + + expect((yield* readServices(root)).rest.resolved.revision).toBe(1); + expect(output.stdoutText).toContain(`${UPSTREAM}-r0 → ${UPSTREAM}-r1`); + expect(output.stdoutText).not.toContain("is stale"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("is reported and left as written when another service is named", () => + Effect.gen(function* () { + const { fs, host, root } = yield* staleProject; + const written = yield* fs.readFileString(lockPath(root)); + + const text = update(root, host, ["auth"]); + yield* text.run; + const json = update(root, host, ["auth"], { format: "json" }); + yield* json.run; + + expect(text.output.stdoutText).toContain("rest is stale ("); + expect(text.output.stdoutText).toContain("supabase stack update rest"); + expect(text.output.stdoutText).not.toContain("already up to date"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + const result = json.output.messages.find((entry) => entry.type === "success"); + const lock = result?.type === "success" ? result.data : undefined; + expect(lock).toMatchObject({ lock: { written: false, stale: ["rest"] } }); + expect(lock).toEqual({ + lock: expect.objectContaining({ unchanged: expect.not.arrayContaining(["rest"]) }), + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + }); + + it.live("fails on an escaping function entrypoint, leaving the lock byte for byte alone", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM }); + publish(host, "auth", 0); + yield* seed(root, host); + publish(host, "auth", 1); + const written = yield* fs.readFileString(lockPath(root)); + const config = yield* fs.readFileString(`${root}/supabase/config.toml`); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + `${config}\n\n[functions.x]\nentrypoint = "../../outside.ts"\n`, + ); + + const error = yield* update(root, host, ["auth"]).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandUpdateError); + expect(error).toMatchObject({ reason: "invalid-config" }); + expect(error.message).toContain("must remain within the project directory"); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails on an unknown service naming the valid ones, and writes nothing", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({ auth: UPSTREAM }); + publish(host, "auth", 0); + yield* seed(root, host); + const written = yield* fs.readFileString(lockPath(root)); + host.requests.length = 0; + + const error = yield* update(root, host, ["auth", "nope"]).run.pipe(Effect.flip); + + expect(error).toBeInstanceOf(StackCommandUpdateError); + expect(error).toMatchObject({ reason: "flags" }); + expect(error.message).toContain('"nope"'); + expect(error.message).toContain(artifactServiceKinds().join(", ")); + expect(yield* fs.readFileString(lockPath(root))).toBe(written); + expect(host.requests).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); + +describe("experimental stack update and saved stack data", () => { + const BUILT = selection.database.upstreamVersion; + const OLDER_BUILD = `${postgresMajor(BUILT)}.0.0.001`; + + const stackWarnings = (output: ReturnType) => + output.messages + .filter(({ type }) => type === "warn") + .map(({ message }) => message) + .filter((message) => message.startsWith("stack ")); + + const destroy = (stackId: string) => + `run \`supabase stack destroy --stack-id ${stackId}\` to recreate the stack — this permanently deletes its local database data`; + + const lockedAt = + (databaseUpstream: string) => + (services: Parameters[1]>>[0]) => ({ + ...services, + database: { + ...services.database, + resolved: rebuilt(services.database.resolved, databaseUpstream), + }, + }); + + it.live( + "offers pinning the recorded build when the bundled database moves to another build", + () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({}); + yield* seed(root, host, lockedAt(OLDER_BUILD)); + const stackId = yield* seedProjectStack({ + projectRoot: root, + supabaseHome: `${root}/.supabase`, + built: OLDER_BUILD, + }); + + const { output, run } = update(root, host, []); + yield* run; + + expect(stackWarnings(output)).toEqual([ + `stack ${stackId}: the next supabase start refuses these pins: PostgreSQL data was initialized by stock PostgreSQL build ${OLDER_BUILD}, but stock PostgreSQL build ${BUILT} was requested; keep this data by pinning \`[local.services.database] version = "${OLDER_BUILD}"\` in supabase/config.toml, or ${destroy(stackId)}`, + ]); + expect((yield* readServices(root)).database.resolved.upstreamVersion).toBe(BUILT); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("stays quiet when only a revision of the build that initialized the data moves", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({}); + yield* seed(root, host, (services) => ({ + ...services, + database: { ...services.database, resolved: unbundled(services.database.resolved) }, + })); + yield* seedProjectStack({ + projectRoot: root, + supabaseHome: `${root}/.supabase`, + built: BUILT, + }); + + const { output, run } = update(root, host, []); + yield* run; + + expect(stackWarnings(output)).toEqual([]); + expect((yield* readServices(root)).database.resolved.revision).toBe( + selection.database.revision, + ); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("stays quiet when the lock is already up to date", () => + Effect.gen(function* () { + const host = yield* startReleaseFixture; + const root = yield* project({}); + yield* seed(root, host); + yield* seedProjectStack({ + projectRoot: root, + supabaseHome: `${root}/.supabase`, + built: OLDER_BUILD, + }); + + const { output, run } = update(root, host, []); + yield* run; + + expect(output.stdoutText).toContain("already up to date"); + expect(stackWarnings(output)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("still updates the lock when the saved stacks cannot be read", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + const root = yield* project({}); + yield* seed(root, host, lockedAt(OLDER_BUILD)); + yield* fs.makeDirectory(`${root}/.supabase`, { recursive: true }); + yield* fs.writeFileString(`${root}/.supabase/stacks`, "not a directory"); + + const { output, run } = update(root, host, []); + yield* run; + + expect(stackWarnings(output)).toEqual([]); + expect((yield* readServices(root)).database.resolved.upstreamVersion).toBe(BUILT); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/feedback/add/SIDE_EFFECTS.md b/apps/cli/src/commands/feedback/add/SIDE_EFFECTS.md index 844fbba57f..b51f331a51 100644 --- a/apps/cli/src/commands/feedback/add/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/feedback/add/SIDE_EFFECTS.md @@ -138,7 +138,7 @@ event); the shared stream-json layer emits these for every `output.task`. (written by `supabase link`) → `null`, the same order as `feedback delete`. The flag and env values are validated like every other command's explicit ref (a malformed value fails with the shared invalid-ref error); the file is the - soft half of `ProjectRefResolver.resolveOptional`, read directly so the + soft half of `RemoteTarget.resolveOptional`, read directly so the command has no auth dependency and works when the user is not logged in — a missing, unreadable, or malformed file degrades to `null`. Note `CommandSettings.projectId` alone is only the env var — it is NOT diff --git a/apps/cli/src/commands/feedback/add/add.handler.ts b/apps/cli/src/commands/feedback/add/add.handler.ts index fe0e67a71b..51f7a803ba 100644 --- a/apps/cli/src/commands/feedback/add/add.handler.ts +++ b/apps/cli/src/commands/feedback/add/add.handler.ts @@ -127,7 +127,7 @@ export const feedbackAdd = Effect.fn("feedback.add")(function* (args: FeedbackAd const isAgent = resolveAgentMode(agentFlag, aiTool.name); const agentName = isAgent ? Option.getOrUndefined(aiTool.name) : undefined; // Flag → SUPABASE_PROJECT_ID → the linked ref file, the same order as - // `feedback delete` and `ProjectRefResolver`. An empty `--project-ref ""` + // `feedback delete` and `RemoteTarget`. An empty `--project-ref ""` // counts as unset; a malformed flag or env value fails as invalid input. const projectRef = yield* resolveFeedbackProjectRef( cliSettings.workdir, diff --git a/apps/cli/src/commands/feedback/add/add.integration.test.ts b/apps/cli/src/commands/feedback/add/add.integration.test.ts index b4a604a3f1..c4c78f245d 100644 --- a/apps/cli/src/commands/feedback/add/add.integration.test.ts +++ b/apps/cli/src/commands/feedback/add/add.integration.test.ts @@ -28,7 +28,7 @@ import { useTempWorkdir, } from "../../../../tests/helpers/command-mocks.ts"; import { invalidOutputFormatMessage } from "../../../command-internal/go-output-flag.ts"; -import { INVALID_PROJECT_REF_MESSAGE } from "../../../config/project-ref.service.ts"; +import { INVALID_PROJECT_REF_MESSAGE } from "../../../config/remote-target.service.ts"; import { FEEDBACK_OUTPUT_FORMATS } from "../feedback-output.ts"; import type { FeedbackAddArgs } from "./add.command.ts"; import { feedbackAddHandler } from "./add.command.ts"; @@ -320,7 +320,7 @@ describe("feedback add", () => { }).pipe(Effect.provide(layer)); }); - it.effect("treats an empty --project-ref as unset, like ProjectRefResolver", () => { + it.effect("treats an empty --project-ref as unset, like RemoteTarget", () => { const { layer, submitter } = setupFeedback({ projectIdEnv: "envenvenvenvenvenvre" }); return Effect.gen(function* () { yield* feedbackAdd(addArgs(["empty flag feedback"], { projectRef: Option.some("") })); @@ -402,7 +402,7 @@ describe("feedback add", () => { }); it.effect("rejects a malformed SUPABASE_PROJECT_ID instead of submitting unlinked", () => { - // A value the user supplied is validated the way `ProjectRefResolver` does + // A value the user supplied is validated the way `RemoteTarget` does // for every command: a typo fails as invalid input rather than silently // falling through to the linked ref file (or to "unlinked"). const { layer, submitter } = setupFeedback({ projectIdEnv: "not-a-valid-ref!" }); diff --git a/apps/cli/src/commands/feedback/delete/delete.handler.ts b/apps/cli/src/commands/feedback/delete/delete.handler.ts index 0989bebc19..04627ef9f5 100644 --- a/apps/cli/src/commands/feedback/delete/delete.handler.ts +++ b/apps/cli/src/commands/feedback/delete/delete.handler.ts @@ -51,7 +51,7 @@ export const feedbackDelete = Effect.fn("feedback.delete")(function* (args: Feed // when the same ref is presented. Flag → SUPABASE_PROJECT_ID → the linked // ref file; extra context against a context-free row is ignored server-side, // so sending whatever resolves is always safe. An empty `--project-ref ""` - // counts as unset, as in `ProjectRefResolver`; a malformed flag or env value + // counts as unset, as in `RemoteTarget`; a malformed flag or env value // fails as invalid input instead of falling through. const projectRef = yield* resolveFeedbackProjectRef( cliSettings.workdir, diff --git a/apps/cli/src/commands/feedback/delete/delete.integration.test.ts b/apps/cli/src/commands/feedback/delete/delete.integration.test.ts index aef1d52237..9c7f0f6561 100644 --- a/apps/cli/src/commands/feedback/delete/delete.integration.test.ts +++ b/apps/cli/src/commands/feedback/delete/delete.integration.test.ts @@ -23,7 +23,7 @@ import { } from "../../../../tests/helpers/command-mocks.ts"; import type { FeedbackDeleteArgs } from "./delete.command.ts"; import { feedbackDeleteHandler } from "./delete.command.ts"; -import { INVALID_PROJECT_REF_MESSAGE } from "../../../config/project-ref.service.ts"; +import { INVALID_PROJECT_REF_MESSAGE } from "../../../config/remote-target.service.ts"; import { FEEDBACK_INVALID_TOKEN_MESSAGE, FEEDBACK_NOT_FOUND_MESSAGE } from "./delete.errors.ts"; import { feedbackDelete } from "./delete.handler.ts"; @@ -336,7 +336,7 @@ describe("feedback delete", () => { }).pipe(Effect.provide(layer)); }); - it.effect("treats an empty --project-ref as unset, like ProjectRefResolver", () => { + it.effect("treats an empty --project-ref as unset, like RemoteTarget", () => { const { layer, client } = setupFeedbackDelete({ yes: true, projectIdEnv: "envenvenvenvenvenvre", diff --git a/apps/cli/src/commands/feedback/feedback-project-ref.ts b/apps/cli/src/commands/feedback/feedback-project-ref.ts index 2f5710e2bc..495af841ba 100644 --- a/apps/cli/src/commands/feedback/feedback-project-ref.ts +++ b/apps/cli/src/commands/feedback/feedback-project-ref.ts @@ -3,25 +3,24 @@ import { InvalidProjectRefError } from "../../config/project-ref.errors.ts"; import { INVALID_PROJECT_REF_MESSAGE, PROJECT_REF_PATTERN, -} from "../../config/project-ref.service.ts"; -import { readProjectRefFile } from "../../command-internal/temp-paths.ts"; +} from "../../config/remote-target.service.ts"; +import { readLinkSnapshotSoft } from "../../command-internal/link-snapshot.ts"; -// Mirrors `ProjectRefResolver` (`project-ref.layer.ts`) source by source: +// Mirrors `RemoteTarget` (`remote-target.layer.ts`) source by source: // // - The caller-supplied override (`--project-ref` and/or `SUPABASE_PROJECT_ID`, // captured by `CommandSettings`) is validated against `PROJECT_REF_PATTERN` // and fails with the shared `InvalidProjectRefError` when malformed — the -// same hard check every `resolve`/`loadProjectRef` caller applies to a value +// same hard check every `resolve`/`load` caller applies to a value // the user typed, so a typo is reported instead of silently falling through // to a different project's context. -// - `/supabase/.temp/project-ref`, the file `supabase link` writes, is -// the soft half (`resolveOptional`): read directly rather than via the full -// resolver because that layer requires `CommandPlatformApiFactory` for its -// prompt path, and feedback must keep working when the user isn't logged in. -// A broken or malformed ref file degrades to "unlinked" instead of failing -// the command — the workdir can be an untrusted checkout where the file is a -// symlink to a local secret, and anything that isn't a well-formed ref is -// discarded rather than sent to the feedback backend. +// - The link `supabase link` records is the soft half (`resolveOptional`): it is +// read directly rather than through `RemoteTarget`, because that layer requires +// `CommandPlatformApiFactory` for its prompt path, and feedback must keep working +// when the user isn't logged in. A broken or malformed link degrades to +// "unlinked" instead of failing the command — the workdir can be an untrusted +// checkout where the file is a symlink to a local secret, and anything that +// isn't a well-formed ref is discarded rather than sent to the feedback backend. export const resolveFeedbackProjectRef = Effect.fnUntraced(function* ( workdir: string, override: Option.Option, @@ -37,8 +36,7 @@ export const resolveFeedbackProjectRef = Effect.fnUntraced(function* ( } const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - return yield* readProjectRefFile(fs, path, workdir).pipe( - Effect.orElseSucceed(() => Option.none()), - Effect.map(Option.filter((ref) => PROJECT_REF_PATTERN.test(ref))), + return yield* readLinkSnapshotSoft(fs, path, workdir).pipe( + Effect.map((snapshot) => Option.filter(snapshot.ref, (ref) => PROJECT_REF_PATTERN.test(ref))), ); }); diff --git a/apps/cli/src/commands/functions/delete/delete.command.ts b/apps/cli/src/commands/functions/delete/delete.command.ts index 9157344f5b..fca4201e6e 100644 --- a/apps/cli/src/commands/functions/delete/delete.command.ts +++ b/apps/cli/src/commands/functions/delete/delete.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { FUNCTIONS_PROJECT_REF_SAFE_FLAGS } from "../../../shared/functions/functions.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/functions/delete/delete.handler.ts b/apps/cli/src/commands/functions/delete/delete.handler.ts index ddd47691b4..26115ba9b7 100644 --- a/apps/cli/src/commands/functions/delete/delete.handler.ts +++ b/apps/cli/src/commands/functions/delete/delete.handler.ts @@ -2,7 +2,7 @@ import { Effect, Option, Ref } from "effect"; import { deleteFunction } from "../../../shared/functions/delete.ts"; import { aqua } from "../../../command-internal/colors.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import type { FunctionsDeleteFlags } from "./delete.command.ts"; @@ -11,7 +11,7 @@ export const functionsDelete = Effect.fn("functions.delete")(function* ( flags: FunctionsDeleteFlags, ) { const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const resolvedProjectRef = yield* Ref.make(Option.none()); @@ -21,9 +21,10 @@ export const functionsDelete = Effect.fn("functions.delete")(function* ( { api, resolveProjectRef: (projectRef) => - resolver - .resolve(projectRef) - .pipe(Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref)))), + remoteTarget.resolve(projectRef).pipe( + Effect.map((target) => target.ref), + Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref))), + ), // The "Deleted Function" line is stdout-bound, so the TTY gate must check stdout. styleIdentifier: (text) => aqua(text, process.stdout), }, diff --git a/apps/cli/src/commands/functions/deploy/deploy.command.ts b/apps/cli/src/commands/functions/deploy/deploy.command.ts index eb80c99408..2c861f0af0 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.command.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.command.ts @@ -3,7 +3,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { FUNCTIONS_PROJECT_REF_SAFE_FLAGS } from "../../../shared/functions/functions.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/functions/deploy/deploy.handler.ts b/apps/cli/src/commands/functions/deploy/deploy.handler.ts index 805e8033d6..143b3ff8a8 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.handler.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.handler.ts @@ -5,7 +5,7 @@ import { aqua, bold, yellow } from "../../../command-internal/colors.ts"; import { functionsGoConfigCompat } from "../../../command-internal/functions-go-config.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { resolveYes } from "../../../command-internal/global-flags.ts"; import { dashboardUrl } from "../../../command-internal/profile.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; @@ -18,7 +18,7 @@ export const functionsDeploy = Effect.fn("functions.deploy")(function* ( ) { const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; // Also honors `SUPABASE_YES`, not just the `--yes` flag, for the `--prune` confirm. const yes = yield* resolveYes; const linkedProjectCache = yield* LinkedProjectCache; @@ -27,9 +27,7 @@ export const functionsDeploy = Effect.fn("functions.deploy")(function* ( const stdio = yield* Stdio.Stdio; const path = yield* Path.Path; const rawArgs = yield* stdio.args; - const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin( - path.join(cliSettings.workdir, "supabase"), - ); + const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin(cliSettings.workdir); const resolvedProjectRef = yield* Ref.make(Option.none()); yield* deployFunctions(flags, { @@ -44,9 +42,10 @@ export const functionsDeploy = Effect.fn("functions.deploy")(function* ( rawArgs, edgeRuntimeVersion, resolveProjectRef: (projectRef) => - resolver - .resolve(projectRef) - .pipe(Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref)))), + remoteTarget.resolve(projectRef).pipe( + Effect.map((target) => target.ref), + Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref))), + ), // Written to stdout, so the TTY color gate must check stdout. styleIdentifier: (text) => aqua(text, process.stdout), // Written to stderr, matching `bold`'s default TTY gate. diff --git a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts index 99f71f7de0..198c545795 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts @@ -21,6 +21,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { TestClock } from "effect/testing"; import { YesFlag } from "../../../command-internal/global-flags.ts"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; import { stripControlSequences } from "../../../shared/output/strip-control-sequences.ts"; import { type ApiHandler, @@ -36,6 +37,7 @@ import { withEnvVar, } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput, mockRuntimeInfo, mockTty } from "../../../../tests/helpers/mocks.ts"; +import { seedLegacyStateDecoys } from "../../../../tests/helpers/legacy-state-guard.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; import { projectResponse, @@ -43,6 +45,8 @@ import { } from "../../../../tests/helpers/permission-real-layers.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mockChildProcessSpawner } from "../../../../tests/helpers/child-process-spawner.ts"; +import { seedLock } from "../../../../tests/helpers/stack-lock.ts"; +import { runtimeInfoLayer } from "../../../shared/runtime/runtime-info.layer.ts"; import { deployFunctions, shouldChmodBundleOutputDirectory, @@ -2258,6 +2262,126 @@ describe("functions deploy", () => { }, ); + it.live( + "bundles on the locked edge-runtime release in stack state mode even when the lock is stale", + () => { + const lockedImage = `ghcr.io/supabase/cli/edge-runtime:v1.99.0-r1@sha256:${"a".repeat(64)}`; + const out = mockOutput({ format: "text" }); + const api = mockFunctionCreateApi(); + const child = mockDockerBundleSpawner(); + const layer = Layer.mergeAll( + buildTestRuntime({ + out, + api, + cliSettings: mockCommandSettings({ workdir: tempRoot.current }), + runtimeInfo: mockRuntimeInfo({ cwd: tempRoot.current }), + }), + projectStateModeLayer("stack"), + Layer.succeed(YesFlag, false), + child.layer, + Stdio.layerTest({ + args: Effect.succeed(["functions", "deploy", "hello-world", "--use-api=false"]), + }), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + // A stale lock fails the local stack commands; a hosted deploy only reads the pin. + yield* seedLock(tempRoot.current, (services) => ({ + ...services, + rest: { ...services.rest, specifier: "v99.0.0" }, + functions: { + ...services.functions, + resolved: { + ...services.functions.resolved, + upstreamVersion: "v1.99.0", + revision: 1, + image: lockedImage, + upstreamImage: "supabase/edge-runtime:v1.99.0", + }, + }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: tempRoot.current, supabaseHome: tempRoot.current }), + ), + ), + ); + seedLegacyStateDecoys(tempRoot.current); + + yield* functionsDeploy({ ...baseFlags, useApi: false, useDocker: true }); + + expect(child.spawned[1]).toEqual({ + command: "docker", + args: ["image", "inspect", "public.ecr.aws/supabase/edge-runtime:v1.99.0"], + }); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }, + ); + + it.live( + "bundles on the locked slim image, revision and digest included, in stack state mode when slim images are on", + () => { + const lockedImage = `ghcr.io/supabase/cli/edge-runtime:v1.99.0-r1@sha256:${"a".repeat(64)}`; + const out = mockOutput({ format: "text" }); + const api = mockFunctionCreateApi(); + const child = mockDockerBundleSpawner(); + const layer = Layer.mergeAll( + buildTestRuntime({ + out, + api, + cliSettings: mockCommandSettings({ workdir: tempRoot.current }), + runtimeInfo: mockRuntimeInfo({ cwd: tempRoot.current }), + }), + projectStateModeLayer("stack"), + Layer.succeed(YesFlag, false), + child.layer, + Stdio.layerTest({ + args: Effect.succeed(["functions", "deploy", "hello-world", "--use-api=false"]), + }), + ); + + return Effect.gen(function* () { + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction(tempRoot.current, "hello-world"); + // A stale lock fails the local stack commands; a hosted deploy only reads the pin. + yield* seedLock(tempRoot.current, (services) => ({ + ...services, + rest: { ...services.rest, specifier: "v99.0.0" }, + functions: { + ...services.functions, + resolved: { + ...services.functions.resolved, + upstreamVersion: "v1.99.0", + revision: 1, + image: lockedImage, + upstreamImage: "supabase/edge-runtime:v1.99.0", + }, + }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: tempRoot.current, supabaseHome: tempRoot.current }), + ), + ), + ); + seedLegacyStateDecoys(tempRoot.current); + + yield* functionsDeploy({ ...baseFlags, useApi: false, useDocker: true }); + + expect(child.spawned[1]).toEqual({ + command: "docker", + args: ["image", "inspect", lockedImage], + }); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot), (body) => + withEnvVar("SUPABASE_USE_SLIM_IMAGES", "true", body), + ); + }, + ); + it.live("passes --verbose to the bundler container when DEBUG=true", () => { const out = mockOutput({ format: "text" }); const api = mockFunctionCreateApi(); diff --git a/apps/cli/src/commands/functions/download/SIDE_EFFECTS.md b/apps/cli/src/commands/functions/download/SIDE_EFFECTS.md index b8570af678..e2d013c49c 100644 --- a/apps/cli/src/commands/functions/download/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/functions/download/SIDE_EFFECTS.md @@ -8,7 +8,7 @@ | `/profile` | plain text | when `--profile` and `SUPABASE_PROFILE` are both unset | | `.yaml` | YAML | when `SUPABASE_PROFILE` or `--profile` points to a file | | `/supabase/.temp/project-ref` | plain text | when `--project-ref` and `SUPABASE_PROJECT_ID` are both unset | -| `/supabase/.temp/edge-runtime-version` | plain text | Read unconditionally by `resolveEdgeRuntimeVersionPin()` in the handler, before the shared downloader chooses `--use-api` vs Docker — only affects the resolved edge-runtime image tag on the Docker-unbundle path | +| `/supabase/.temp/edge-runtime-version` | plain text | Read by `resolveEdgeRuntimeVersionPin()` in the handler in legacy project-state mode only (stack mode takes the tag from the locked Functions pin), before the shared downloader chooses `--use-api` vs Docker — only affects the resolved edge-runtime image tag on the Docker-unbundle path | | `/supabase/{.env,.env.local,.env.,.env..local}` and the same four at the project root | dotenv | Docker-unbundle path only, before resolving config.toml — project dotenv (`resolveProjectEnvironmentValues`), merged into the `SUPABASE_*` overrides below and threaded into registry resolution | | `/supabase/config.toml` | TOML | Read unconditionally after resolving the project ref, before checking `--use-api`/`--use-docker` or whether Docker is running — resolves `edge_runtime.deno_version` and `project_id` (`loadCliConfig`) for the Docker-unbundle path. `goViperCompat`'s `tomlOnly: true` means `config.json` is never read here, unlike other `loadCliConfig` callers. A malformed config fails here even on the `--use-api` invocation. Also runs the full `Config.Validate` pipeline (`resolveLocalConfigValues`, same one `start`/`stop`/`status` already use) — an invalid config (bad `db.major_version`, malformed auth hook, etc.) now fails the Docker-unbundle path up front, even for fields this command never otherwise reads. | | `/supabase/`, `[api.tls]` cert/key paths, email template `content_path` — when configured | varies | Docker-unbundle path only, as part of the `Config.Validate` pipeline above — read even though this command never uses their contents; a `content_path` resolved path is CONFINED to the project root (symlinks dereferenced with `realpathSync`) before it is read, aborting with `resolves outside the project root` before any other work | @@ -19,7 +19,7 @@ | Path | Format | When | | --------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `/supabase/functions//` | bytes | for each source file returned by the API (server-side path: `--use-api`, `--use-docker=false`, or the Docker-unbundle fallback when Docker isn't running) | -| `/supabase/.temp/output_.eszip` | bytes | Docker-unbundle path (default): downloaded eszip, extracted into `supabase/functions//...` by the edge-runtime container; removed after the attempt unless `--debug` is set | +| `/supabase/.temp/output_.eszip` | bytes | Docker-unbundle path (default; under `/tmp/` in stack mode): downloaded eszip, extracted into `supabase/functions//...` by the edge-runtime container; removed after the attempt unless `--debug` is set | | `/supabase/.temp/linked-project.json` | JSON | post-run, once a project ref has resolved (success or failure), only when no such file exists, `/supabase/.temp/project-ref` doesn't name another ref, an access token exists and `GET /v1/projects/{ref}` returns 200 | | `/telemetry.json` | JSON | after command completion, flushed on both success and failure paths | diff --git a/apps/cli/src/commands/functions/download/download.command.ts b/apps/cli/src/commands/functions/download/download.command.ts index 5e22ea65c0..31ef3d052a 100644 --- a/apps/cli/src/commands/functions/download/download.command.ts +++ b/apps/cli/src/commands/functions/download/download.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { functionsDownload as functionsDownloadCalls } from "../../../shared/functions/download.ts"; import { FUNCTIONS_PROJECT_REF_SAFE_FLAGS } from "../../../shared/functions/functions.shared.ts"; diff --git a/apps/cli/src/commands/functions/download/download.handler.ts b/apps/cli/src/commands/functions/download/download.handler.ts index 964257d4f5..f6251feec6 100644 --- a/apps/cli/src/commands/functions/download/download.handler.ts +++ b/apps/cli/src/commands/functions/download/download.handler.ts @@ -1,4 +1,4 @@ -import { Effect, Option, Path, Ref, Stdio } from "effect"; +import { Effect, Option, Ref, Stdio } from "effect"; import { downloadFunctions, makeGoProxyLegacyBundleArgs, @@ -10,7 +10,7 @@ import { aqua, bold, yellow } from "../../../command-internal/colors.ts"; import { functionsGoConfigCompat } from "../../../command-internal/functions-go-config.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import type { FunctionsDownloadFlags } from "./download.command.ts"; @@ -20,17 +20,14 @@ export const functionsDownload = Effect.fn("functions.download")(function* ( ) { const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const proxy = yield* GoProxy; const output = yield* Output; const stdio = yield* Stdio.Stdio; - const path = yield* Path.Path; const rawArgs = yield* stdio.args; - const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin( - path.join(cliSettings.workdir, "supabase"), - ); + const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin(cliSettings.workdir); const resolvedProjectRef = yield* Ref.make(Option.none()); yield* Effect.gen(function* () { @@ -47,9 +44,10 @@ export const functionsDownload = Effect.fn("functions.download")(function* ( // Written to stderr, matching `yellow`'s default TTY gate. styleWarning: (text) => yellow(text), resolveProjectRef: (projectRef) => - resolver - .resolve(projectRef) - .pipe(Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref)))), + remoteTarget.resolve(projectRef).pipe( + Effect.map((target) => target.ref), + Effect.tap((ref) => Ref.set(resolvedProjectRef, Option.some(ref))), + ), // Suppresses the delegated binary's own `cli_command_executed` so a // proxied invocation fires exactly one event. In machine-output mode its // stdout is captured and discarded instead of inherited, since diff --git a/apps/cli/src/commands/functions/download/download.integration.test.ts b/apps/cli/src/commands/functions/download/download.integration.test.ts index df0461afb9..2e98a221a9 100644 --- a/apps/cli/src/commands/functions/download/download.integration.test.ts +++ b/apps/cli/src/commands/functions/download/download.integration.test.ts @@ -21,6 +21,7 @@ import { import { ChildProcess, ChildProcessSpawner } from "effect/process"; import * as HttpClientResponse from "effect/http/HttpClientResponse"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; import { commandRuntimeLayer } from "../../../shared/runtime/command-runtime.layer.ts"; import { type ApiHandler, @@ -1491,6 +1492,59 @@ describe("functions download", () => { }, ); + it.live( + "unbundles from a per-run directory under SUPABASE_HOME/tmp in stack state mode and removes it afterwards", + () => { + const out = mockOutput({ format: "text" }); + const api = mockCommandPlatformApi(); + const proxy = mockProxy(); + const child = mockChildProcessSpawner({ exitCode: 0 }); + const supabaseHome = `${tempRoot.current}/home`; + const layer = Layer.mergeAll( + buildTestRuntime({ + out, + api, + cliSettings: mockCommandSettings({ workdir: tempRoot.current, supabaseHome }), + }), + projectStateModeLayer("stack"), + proxy.layer, + child.layer, + Stdio.layerTest({ + args: Effect.succeed([ + "functions", + "download", + "hello-world", + "--use-docker", + "--project-ref", + PROJECT_ID, + ]), + }), + ); + const eszipBinds = (args: ReadonlyArray | undefined) => + (args ?? []).filter((arg) => arg.endsWith(":/root/eszips/output_hello-world.eszip:ro")); + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* functionsDownload({ ...baseFlags, useDocker: true }); + yield* functionsDownload({ ...baseFlags, useDocker: true }); + + const hostEszipDirs = child.spawned + .filter((spawned) => spawned.args[0] === "run") + .flatMap((spawned) => eszipBinds(spawned.args)) + .map((bind) => path.dirname(bind.slice(0, bind.indexOf(":/root/eszips/")))); + expect(hostEszipDirs).toHaveLength(2); + expect(hostEszipDirs[0]).not.toBe(hostEszipDirs[1]); + for (const dir of hostEszipDirs) { + expect(path.dirname(dir)).toBe(path.join(supabaseHome, "tmp")); + expect(path.basename(dir)).toMatch(/^\.supabase-eszip-hello-world-/); + expect(yield* fs.exists(dir)).toBe(false); + } + expect(yield* fs.exists(path.join(tempRoot.current, "supabase", ".temp"))).toBe(false); + }).pipe(Effect.provide(layer)); + }, + ); + it.effect( "fails on an invalid project config before falling back when Docker is not running", () => { diff --git a/apps/cli/src/commands/functions/list/list.command.ts b/apps/cli/src/commands/functions/list/list.command.ts index da56d6ba8c..bafba916be 100644 --- a/apps/cli/src/commands/functions/list/list.command.ts +++ b/apps/cli/src/commands/functions/list/list.command.ts @@ -2,7 +2,7 @@ import { Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { FUNCTIONS_PROJECT_REF_SAFE_FLAGS } from "../../../shared/functions/functions.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/functions/list/list.handler.ts b/apps/cli/src/commands/functions/list/list.handler.ts index ea310010d1..438865fb8a 100644 --- a/apps/cli/src/commands/functions/list/list.handler.ts +++ b/apps/cli/src/commands/functions/list/list.handler.ts @@ -4,7 +4,7 @@ import { Effect, Option, Ref, Result } from "effect"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { mapHttpError, sanitizeErrorBody } from "../../../command-internal/http-errors.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -34,15 +34,16 @@ export const functionsList = Effect.fn("functions.list")(function* (flags: Funct const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const resolvedProjectRef = yield* Ref.make(Option.none()); yield* Effect.gen(function* () { - const ref = yield* resolver - .resolve(flags.projectRef) - .pipe(Effect.tap((projectRef) => Ref.set(resolvedProjectRef, Option.some(projectRef)))); + const ref = yield* remoteTarget.resolve(flags.projectRef).pipe( + Effect.map((target) => target.ref), + Effect.tap((projectRef) => Ref.set(resolvedProjectRef, Option.some(projectRef))), + ); yield* Effect.annotateCurrentSpan("project.ref", ref); const fetching = diff --git a/apps/cli/src/commands/functions/list/list.integration.test.ts b/apps/cli/src/commands/functions/list/list.integration.test.ts index e6a3e88df5..192c2c0e65 100644 --- a/apps/cli/src/commands/functions/list/list.integration.test.ts +++ b/apps/cli/src/commands/functions/list/list.integration.test.ts @@ -19,10 +19,10 @@ import { realLinkedProjectCache, } from "../../../../tests/helpers/permission-real-layers.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { mockOutput, mockTty } from "../../../../tests/helpers/mocks.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { functionsList } from "./list.handler.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; type Functions = typeof V1ListAllFunctionsOutput.Type; @@ -489,7 +489,7 @@ describe("functions list integration", () => { telemetry: telemetry.layer, linkedProjectCache: cache.layer, }), - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.fail( new ProjectRefNotLinkedError({ @@ -498,8 +498,8 @@ describe("functions list integration", () => { ), resolveForLink: () => Effect.die("not used in functions list test"), resolveOptional: () => Effect.die("not used in functions list test"), - loadProjectRef: () => Effect.die("not used in functions list test"), - promptProjectRef: () => Effect.die("not used in functions list test"), + load: () => Effect.die("not used in functions list test"), + prompt: () => Effect.die("not used in functions list test"), }), ); return Effect.gen(function* () { diff --git a/apps/cli/src/commands/functions/serve/serve.stack.e2e.test.ts b/apps/cli/src/commands/functions/serve/serve.stack.e2e.test.ts index 3f1661117e..c4445956b9 100644 --- a/apps/cli/src/commands/functions/serve/serve.stack.e2e.test.ts +++ b/apps/cli/src/commands/functions/serve/serve.stack.e2e.test.ts @@ -26,6 +26,7 @@ import { spawnSupabase } from "../../../../tests/helpers/cli.ts"; import { generateGoJwt } from "../../../command-internal/go-jwt.ts"; import { destroyTestStack } from "../../../../tests/helpers/stack-cleanup.ts"; import { stackArtifactCacheRoot } from "../../../../tests/helpers/stack-artifacts.ts"; +import { selection } from "../../../../tests/helpers/pins.ts"; const jwtSecret = "functions-serve-stack-e2e-secret-at-least-32-characters"; const nativeSupported = @@ -85,8 +86,8 @@ const fixture = Effect.fn("FunctionsServeE2e.fixture")(function* ( yield* stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -95,6 +96,7 @@ const fixture = Effect.fn("FunctionsServeE2e.fixture")(function* ( }, { service: "rest", + pin: selection.rest, config: { jwtSecret }, endpoints: { http: { port: "auto" } }, }, @@ -102,6 +104,7 @@ const fixture = Effect.fn("FunctionsServeE2e.fixture")(function* ( ? [ { service: "functions" as const, + pin: selection.functions, config: { functionsRoot, jwtSecret, @@ -112,6 +115,7 @@ const fixture = Effect.fn("FunctionsServeE2e.fixture")(function* ( }, { service: "studio" as const, + pin: selection.studio, config: { jwtSecret }, endpoints: { http: { port: "auto" as const } }, }, @@ -133,6 +137,7 @@ const fixture = Effect.fn("FunctionsServeE2e.fixture")(function* ( if (httpPort === undefined) return yield* Effect.die("API port missing"); yield* stack.services.create({ service: "functions", + pin: selection.functions, config: { functionsRoot, jwtSecret, @@ -226,7 +231,7 @@ const postmasterStartTime = Effect.fn("FunctionsServeE2e.postmasterStartTime")(f ) { const decoder = new TextDecoder(); let stdout = ""; - const result = yield* stack.commands.run(postgres.psql({ major: 17 }), { + const result = yield* stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "-tA", "--command", "SELECT pg_postmaster_start_time()"], stdout: (bytes) => Effect.sync(() => (stdout += decoder.decode(bytes))), stderr: () => Effect.void, diff --git a/apps/cli/src/commands/functions/serve/serve.stack.handler.ts b/apps/cli/src/commands/functions/serve/serve.stack.handler.ts index 0568e32e4b..d2a0b8f41b 100644 --- a/apps/cli/src/commands/functions/serve/serve.stack.handler.ts +++ b/apps/cli/src/commands/functions/serve/serve.stack.handler.ts @@ -5,6 +5,10 @@ import { CommandSettings } from "../../../config/command-settings.service.ts"; import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; import { ProcessControl } from "../../../shared/runtime/process-control.service.ts"; import { loadStackConfig, StackConfigError } from "../../../command-internal/stack-config.ts"; +import { + LockResolutionError, + readSelectionOrBundled, +} from "../../../command-internal/stack-lock.ts"; import { readStackFunctionsEnv, StackFunctionsEnvError, @@ -24,28 +28,21 @@ type FunctionsCreation = Extract { + if (Schema.is(StackError)(cause)) return "stack"; + if (Schema.is(LockResolutionError)(cause)) + return cause.reason === "io" ? "runtime" : "invalid-config"; + if (cause instanceof StackFunctionsEnvError || cause instanceof StackConfigError) + return "invalid-config"; + // The session only decodes the stack's saved credentials. + if (cause instanceof Schema.SchemaError) return "state"; + return cause instanceof LocalDbRunningError ? "lifecycle" : "runtime"; +}; + const runtimeError = (cause: { readonly message: string }) => cause instanceof FunctionsServeStackError ? cause - : Schema.is(StackError)(cause) - ? new FunctionsServeStackError({ - reason: "stack", - message: cause.message, - cause, - }) - : new FunctionsServeStackError({ - reason: - cause instanceof StackFunctionsEnvError || cause instanceof StackConfigError - ? "invalid-config" - : // The session only decodes the stack's saved credentials. - Schema.isSchemaError(cause) - ? "state" - : cause instanceof LocalDbRunningError - ? "lifecycle" - : "runtime", - message: cause.message, - cause, - }); + : new FunctionsServeStackError({ reason: serveReason(cause), message: cause.message, cause }); const invalidConfig = (message: string) => new FunctionsServeStackError({ reason: "invalid-config", message }); @@ -213,7 +210,7 @@ const session = Effect.fn("functions.serve.session")(function* (flags: Functions return yield* invalidConfig("Invalid saved database configuration."); const identity = yield* stack.credentials.get; if (identity === undefined) return yield* invalidConfig("The stack has no saved credentials."); - const config = yield* loadStackConfig(settings.workdir); + const config = yield* loadStackConfig(settings.workdir, { selection: readSelectionOrBundled }); yield* Effect.annotateCurrentSpan("functions.instance", "temporary"); const refreshedJwks = yield* config.remoteJwks.pipe( Effect.catch((cause) => diff --git a/apps/cli/src/commands/functions/serve/serve.stack.integration.test.ts b/apps/cli/src/commands/functions/serve/serve.stack.integration.test.ts index 48c0c8fa56..00570c44e2 100644 --- a/apps/cli/src/commands/functions/serve/serve.stack.integration.test.ts +++ b/apps/cli/src/commands/functions/serve/serve.stack.integration.test.ts @@ -27,6 +27,11 @@ import { StackApi } from "../../../command-internal/stack-api.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { mockCommandSettings } from "../../../../tests/helpers/command-mocks.ts"; import { unusedGateway } from "../../../../tests/helpers/unused-stack.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../tests/helpers/legacy-state-guard.ts"; import { mockOutput, mockProcessControl, @@ -35,6 +40,8 @@ import { import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; import { functionsServeStack } from "./serve.stack.handler.ts"; +import { database15, selection } from "../../../../tests/helpers/pins.ts"; +import { seedLock, unbundled } from "../../../../tests/helpers/stack-lock.ts"; type DatabaseInstance = Extract< Effect.Success>, @@ -56,7 +63,6 @@ const functionsConfig = (): Extract[" }); const databaseConfig: Extract["config"] = { - version: "15", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make("jwt-secret"), jwtExpiry: 3600, @@ -143,11 +149,13 @@ const fixture = ( let destroyed = false; const databaseCreation: Extract = { service: "database", + pin: database15, config: databaseConfig, endpoints: { sql: { port: 54322 } }, }; const functionsCreation = (): Extract => ({ service: "functions", + pin: selection.functions, config: currentFunctions, endpoints: { http: { port: 54321 } }, }); @@ -310,8 +318,10 @@ const fixture = ( findDeleted: () => Effect.die("unused"), } satisfies StackApi["Service"]; const api = Layer.succeed(StackApi, apiService); + const recorder = recordingFileSystem(); const layer = Layer.mergeAll( BunServices.layer, + recorder.layer, api, mockCommandSettings({ workdir: options.standaloneProjectRoot ?? "/project", @@ -326,6 +336,7 @@ const fixture = ( ); return { layer, + accessed: recorder.accessed, signal, started, output, @@ -386,6 +397,69 @@ describe("experimental Stack Functions serve", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.live("runs standalone Functions on the pin the project lock selects", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "functions-locked-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + 'project_id = "functions-locked"\n\n[edge_runtime]\nenabled = true\n', + ); + const lockedFunctions = unbundled(selection.functions); + yield* seedLock(root, (services) => ({ + ...services, + functions: { ...services.functions, resolved: lockedFunctions }, + })).pipe( + Effect.provide( + Layer.mergeAll( + mockCommandSettings({ workdir: root, supabaseHome: "/home" }), + mockRuntimeInfo({ cwd: root, homeDir: "/home" }), + ), + ), + ); + const state = yield* fixture({ standaloneProjectRoot: root }); + const run = yield* functionsServeStack(flags()).pipe( + Effect.provide(state.layer), + Effect.forkChild({ startImmediately: true }), + ); + yield* Deferred.await(state.started); + + expect(state.createdFunctions?.pin).toEqual(lockedFunctions); + + yield* Deferred.succeed(state.signal, undefined); + yield* Fiber.join(run); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live( + "serves on the bundled pin without reading supabase/.temp, whatever a legacy pin says", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "functions-no-legacy-state-" }); + yield* fs.makeDirectory(`${root}/supabase`, { recursive: true }); + yield* fs.writeFileString( + `${root}/supabase/config.toml`, + 'project_id = "functions-no-legacy-state"\n\n[edge_runtime]\nenabled = true\n', + ); + seedLegacyStateDecoys(root); + const state = yield* fixture({ standaloneProjectRoot: root }); + const run = yield* functionsServeStack(flags()).pipe( + Effect.provide(state.layer), + Effect.forkChild({ startImmediately: true }), + ); + yield* Deferred.await(state.started); + + expect(state.createdFunctions?.pin).toEqual(selection.functions); + expect(state.accessed.some((entry) => entry.endsWith("config.toml"))).toBe(true); + expect(legacyStateAccesses(state.accessed)).toEqual([]); + + yield* Deferred.succeed(state.signal, undefined); + yield* Fiber.join(run); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.live("refreshes remote JWKS before creating standalone Functions", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; diff --git a/apps/cli/src/commands/gen/bearer-jwt/SIDE_EFFECTS.md b/apps/cli/src/commands/gen/bearer-jwt/SIDE_EFFECTS.md index fca885a827..ea891c6e58 100644 --- a/apps/cli/src/commands/gen/bearer-jwt/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/gen/bearer-jwt/SIDE_EFFECTS.md @@ -94,7 +94,7 @@ kid of your signing key (or leave blank to use the first one): ` on stderr, echo - `[auth].signing_keys_path` **configured**, **real TTY**: presents an interactive picker (`Select a signing key:`) built from each key's `kid`/`alg`/`key_ops`, then prints `Selected key ID: ` to stderr. Does not byte-match the old interactive - picker UI (an accepted divergence — see `project-ref.layer.ts` for the + picker UI (an accepted divergence — see `remote-target.layer.ts` for the established precedent of only matching the observable "Selected ..." line). - **`[auth].enabled = false` quirk:** the `signing_keys_path` file is only read when `auth.enabled` is `true` — but which prompt is shown depends purely on diff --git a/apps/cli/src/commands/gen/keys/SIDE_EFFECTS.md b/apps/cli/src/commands/gen/keys/SIDE_EFFECTS.md index 6b2f9d460b..fff6dcfd23 100644 --- a/apps/cli/src/commands/gen/keys/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/gen/keys/SIDE_EFFECTS.md @@ -51,5 +51,6 @@ Not applicable. - **Deprecated**: use `gen signing-key` instead. - `--project-ref` flag specifies the project ref. +- In stack mode the project resolves from `--project-ref`, `SUPABASE_PROJECT_ID`, the link, then (on a TTY) an interactive project picker, and the resolved ref is passed to the delegated command. - `--override-name` overrides specific variable names in the output. - Experimental command for generating preview branch keys. diff --git a/apps/cli/src/commands/gen/keys/keys.handler.ts b/apps/cli/src/commands/gen/keys/keys.handler.ts index 620768e490..b8fc125bf1 100644 --- a/apps/cli/src/commands/gen/keys/keys.handler.ts +++ b/apps/cli/src/commands/gen/keys/keys.handler.ts @@ -1,13 +1,15 @@ -import { Effect, Option } from "effect"; +import { Effect } from "effect"; import { GoProxy } from "../../../command-internal/go-proxy.service.ts"; +import { goProjectTarget } from "../../../command-internal/go-proxy-target-env.ts"; import type { GenKeysFlags } from "./keys.command.ts"; export const genKeys = Effect.fn("gen.keys")(function* (flags: GenKeysFlags) { const proxy = yield* GoProxy; const args: string[] = ["gen", "keys"]; - if (Option.isSome(flags.projectRef)) args.push("--project-ref", flags.projectRef.value); + const target = yield* goProjectTarget(flags.projectRef, { picker: true }); + args.push(...target.args); for (const name of flags.overrideName) { args.push("--override-name", name); } - yield* proxy.exec(args); + yield* proxy.exec(args, { env: target.env }); }); diff --git a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md index 8db2584e95..e97cb639ae 100644 --- a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md @@ -25,7 +25,7 @@ is on, `--local` resolves the project stack through `DbConfigResolver` | `{/supabase}/.env*` | dotenv | `--local`; resolves the same nested environment overrides as the CLI. Every `--db-url` resolution also reads it (same `.env..local`/`.env.local`/`.env.`/`.env` search) to populate the `PG*` fallbacks used while parsing the DSN. | | `~/.pg_service.conf` (or a `PGSERVICEFILE`/DSN `servicefile=` override) | libpq service file | `--db-url`, only when the DSN or `PGSERVICE` names a `service` — its settings fill in for connection fields the DSN itself omits | | `Package.resolved` (nearest) | JSON | `--lang swift` without `SUPABASE_TYPEGEN`, on macOS or Linux x86_64: the first one found walking up from the working directory, including the copies inside an `.xcodeproj` or `.xcworkspace`; the supabase-swift version it pins picks the `supabase-typegen` release | -| `/supabase/.temp/rest-version` | plain text | `--local` only, when `db.major_version > 14` — forces v9 compat if the tag contains `v9` | +| `/supabase/.temp/rest-version` | plain text | `--local` on the legacy backend only, when `db.major_version > 14` — forces v9 compat if the tag contains `v9`; the stack backend never reads it | ## Files Written diff --git a/apps/cli/src/commands/gen/types/types.command.ts b/apps/cli/src/commands/gen/types/types.command.ts index fdf68fd631..3e9cc11395 100644 --- a/apps/cli/src/commands/gen/types/types.command.ts +++ b/apps/cli/src/commands/gen/types/types.command.ts @@ -11,7 +11,7 @@ import { dbConfigLinked, dbConfigPoolerFallbackLinked, } from "../../../command-internal/db-config.layer.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { withCommandTelemetry } from "../../../telemetry/command-telemetry.ts"; diff --git a/apps/cli/src/commands/gen/types/types.handler.ts b/apps/cli/src/commands/gen/types/types.handler.ts index 7d600165da..8ceecbcd1c 100644 --- a/apps/cli/src/commands/gen/types/types.handler.ts +++ b/apps/cli/src/commands/gen/types/types.handler.ts @@ -13,9 +13,10 @@ import { } from "../../../shared/cli/cobra-flag-groups.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; import { - ProjectRefResolver, + RemoteTarget, PROJECT_NOT_LINKED_MESSAGE, -} from "../../../config/project-ref.service.ts"; + type ResolvedRemoteTarget, +} from "../../../config/remote-target.service.ts"; import { spawnContainerCli } from "../../../command-internal/container-cli.ts"; import { isIPv6ConnectivityErrorCause } from "../../../command-internal/connect-errors.ts"; import { isDockerDaemonUnreachable } from "../../../command-internal/docker-suggest.ts"; @@ -33,7 +34,7 @@ import { import { getHostname } from "../../../command-internal/hostname.ts"; import type { DbConnectError } from "../../../command-internal/db-connection.errors.ts"; import type { PgConnInput } from "../../../command-internal/db-connection.service.ts"; -import { tempPaths } from "../../../command-internal/temp-paths.ts"; +import { LegacyProjectState } from "../../../command-internal/legacy-project-state.service.ts"; import { missingProjectConfigMessageEffect, relativeConfigPath, @@ -229,12 +230,13 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) const telemetryState = yield* TelemetryState; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; + const legacyState = yield* LegacyProjectState; const stdio = yield* Stdio.Stdio; const dnsResolver = yield* DnsResolverFlag; const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; const rawArgs = yield* stdio.args; const platformApi = yield* CommandPlatformApiFactory; - const projectRef = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const dbConfig = yield* DbConfigResolver; const generator = yield* GenTypesGenerator; @@ -384,13 +386,14 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) }); const runProjectTypes = ( - projectRef: string, + target: ResolvedRemoteTarget, includedSchemas: ReadonlyArray, // True for an explicit `--project-id ` (an ad-hoc remote project that may // differ from the current workdir); false for `--linked` / the linked fallback. adHocProjectRef: boolean, ) => Effect.gen(function* () { + const projectRef = target.ref; yield* Effect.annotateCurrentSpan("project.ref", projectRef); const api = yield* platformApi.make; @@ -409,6 +412,7 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) connType: "linked", dnsResolver, linkedProjectRef: Option.some(projectRef), + target, adHocProjectRef, }; const resolved = yield* dbConfig.resolve(resolveFlags); @@ -437,7 +441,7 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) .pipe(Effect.catch(mapProjectTypesError)); yield* output.raw(response.types); - }).pipe(Effect.ensuring(linkedProjectCache.cache(projectRef))); + }).pipe(Effect.ensuring(linkedProjectCache.cache(target.ref))); const runPreviewBranchTypes = (branchRef: string, includedSchemas: ReadonlyArray) => Effect.gen(function* () { @@ -617,13 +621,13 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) path.basename(cliSettings.workdir), ); - const paths = tempPaths(path, cliSettings.workdir); // Only forces v9 compat from the rest-version file's image tag when the database's - // major version is > 14, so older databases aren't forced into v9 mode. + // major version is > 14, so older databases aren't forced into v9 mode. A stack runs + // the PostgREST release it locked, so it has no pin file to consult. const restVersion = - config.majorVersion > 14 - ? (yield* fs - .readFileString(paths.restVersion) + backend.kind === "legacy" && config.majorVersion > 14 + ? (yield* legacyState + .read(cliSettings.workdir, "restVersion") .pipe(Effect.orElseSucceed(() => ""))).trim() : ""; const forcedV9 = restVersion.length > 0 && restVersion.includes("v9"); @@ -696,10 +700,10 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) if (flags.linked) { yield* Effect.annotateCurrentSpan("typegen.source", "linked"); - const ref = yield* projectRef.resolve(Option.none()); - const loaded = schemas.length > 0 ? null : yield* loadConfig(ref); + const target = yield* remoteTarget.resolve(Option.none()); + const loaded = schemas.length > 0 ? null : yield* loadConfig(target.ref); yield* runProjectTypes( - ref, + target, schemas.length > 0 ? schemas : schemasFromConfig(loaded?.config.api.schemas), false, ); @@ -708,10 +712,10 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) if (Option.isSome(flags.projectId)) { yield* Effect.annotateCurrentSpan("typegen.source", "project_id"); - const ref = yield* projectRef.resolve(flags.projectId); - const loaded = schemas.length > 0 ? null : yield* loadConfig(ref); + const target = yield* remoteTarget.resolve(flags.projectId); + const loaded = schemas.length > 0 ? null : yield* loadConfig(target.ref); yield* runProjectTypes( - ref, + target, schemas.length > 0 ? schemas : schemasFromConfig(loaded?.config.api.schemas), true, ); @@ -719,7 +723,7 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) } yield* Effect.annotateCurrentSpan("typegen.source", "linked"); - const resolvedRef = yield* projectRef.resolve(Option.none()).pipe( + const resolvedTarget = yield* remoteTarget.resolve(Option.none()).pipe( Effect.catchTag("ProjectRefNotLinkedError", (cause) => Effect.fail( cause.message === PROJECT_NOT_LINKED_MESSAGE @@ -730,9 +734,9 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) ), ), ); - const loaded = schemas.length > 0 ? null : yield* loadConfig(resolvedRef); + const loaded = schemas.length > 0 ? null : yield* loadConfig(resolvedTarget.ref); yield* runProjectTypes( - resolvedRef, + resolvedTarget, schemas.length > 0 ? schemas : schemasFromConfig(loaded?.config.api.schemas), false, ); diff --git a/apps/cli/src/commands/gen/types/types.integration.test.ts b/apps/cli/src/commands/gen/types/types.integration.test.ts index ac78a25ef3..ac46209254 100644 --- a/apps/cli/src/commands/gen/types/types.integration.test.ts +++ b/apps/cli/src/commands/gen/types/types.integration.test.ts @@ -69,6 +69,11 @@ import { GenTypesLocalDbInspectError, GenTypesLocalDbNotRunningError } from "./t import { genTypes } from "./types.handler.ts"; import { localDbContainerId, parseQueryTimeoutMillis, rootCaBundle } from "./types.shared.ts"; import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../tests/helpers/legacy-state-guard.ts"; import { type GenTypesGenerateError, GenTypesGenerationError, @@ -505,6 +510,7 @@ const setup = Effect.fnUntraced(function* ( }, }); + const recorder = recordingFileSystem(); const runtime = buildTestRuntime({ out, api, @@ -515,11 +521,13 @@ const setup = Effect.fnUntraced(function* ( }), telemetry: telemetry.layer, linkedProjectCache: linkedProjectCache.layer, + fileSystem: recorder.layer, }); const layer = Layer.mergeAll( runtime, BunServices.layer, + recorder.layer, child.layer, Stdio.layerTest({ args: Effect.succeed(opts.args ?? ["gen", "types"]) }), Layer.succeed(OutputFlag, opts.goOutput ?? Option.none()), @@ -541,6 +549,7 @@ const setup = Effect.fnUntraced(function* ( api, generator, layer, + accessed: recorder.accessed, }; }); @@ -2718,6 +2727,35 @@ describe("gen types", () => { }); describe("Local generation: stack backend (no Docker inspection at all)", () => { + it.effect("ignores a legacy rest-version pin and never touches supabase/.temp", () => + Effect.gen(function* () { + const workdir = yield* makeWorkdir("supabase-gen-types-stack-decoy-"); + const { layer, generator, accessed } = yield* setup({ + workdir, + dbConfigResolve: () => + Effect.succeed( + localResolvedConfig({ + host: "127.0.0.1", + port: 54321, + user: "postgres", + password: "postgres", + database: "postgres", + }), + ), + }); + seedLegacyStateDecoys(workdir); + + yield* genTypes(defaultFlags({ local: true })).pipe( + Effect.provide(Layer.mergeAll(layer, stackBackendLayer("stack"))), + ); + + expect(generator.calls).toHaveLength(1); + expect(generator.calls[0]?.options["detect-one-to-one-relationships"]).toBe(true); + expect(accessed.some((entry) => entry.endsWith("config.toml"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect("resolves the stack local database directly, without inspecting any container", () => Effect.gen(function* () { const workdir = yield* makeWorkdir("supabase-gen-types-stack-local-"); diff --git a/apps/cli/src/commands/gen/types/types.layers.ts b/apps/cli/src/commands/gen/types/types.layers.ts index f30f4be646..37a4b5c8d1 100644 --- a/apps/cli/src/commands/gen/types/types.layers.ts +++ b/apps/cli/src/commands/gen/types/types.layers.ts @@ -5,8 +5,8 @@ import { commandPlatformApiFactoryLayer } from "../../../auth/command-platform-a import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; import { commandSettingsLayer } from "../../../config/command-settings.layer.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { projectRefLayer } from "../../../config/project-ref.layer.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { remoteTargetLayer } from "../../../config/remote-target.layer.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { dbConfigLayer } from "../../../command-internal/db-config.layer.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { dbConnectionLayer } from "../../../command-internal/db-connection.sql-pg.layer.ts"; @@ -57,7 +57,7 @@ export const genTypesRuntimeLayer = (() => { generator, cliSettings, platformApiFactory, - projectRefLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), + remoteTargetLayer.pipe(Layer.provide(platformApiFactory), Layer.provide(cliSettings)), linkedProjectCacheLayer.pipe( Layer.provide(credentials), Layer.provide(cliSettings), @@ -84,7 +84,7 @@ export const genTypesRuntimeLayer = (() => { type GenTypesServices = | CommandPlatformApiFactory | CommandSettings - | ProjectRefResolver + | RemoteTarget | DbConfigResolver | DbConnection | GenTypesGenerator diff --git a/apps/cli/src/commands/gen/types/types.layers.unit.test.ts b/apps/cli/src/commands/gen/types/types.layers.unit.test.ts index 2e270777de..9fc6e03515 100644 --- a/apps/cli/src/commands/gen/types/types.layers.unit.test.ts +++ b/apps/cli/src/commands/gen/types/types.layers.unit.test.ts @@ -33,10 +33,10 @@ import { } from "../../../command-internal/global-flags.ts"; import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { IdentityStitch } from "../../../command-internal/identity-stitch.ts"; import { genTypesRuntimeLayer } from "./types.layers.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const tempRoot = useTempWorkdir("supabase-gen-types-layers-"); @@ -59,12 +59,12 @@ function ambientStubs() { ); const heavyServiceStubs = Layer.mergeAll( - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveForLink: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), resolveOptional: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - loadProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), - promptProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + load: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + prompt: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), }), Layer.succeed(CommandPlatformApiFactory, { make: Effect.die("platform-api-factory not needed for layer-exposure test"), diff --git a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md index 62d266c43c..13828c0ed9 100644 --- a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md @@ -30,7 +30,7 @@ no new config reads. | Method | Path | Auth | When | | -------- | ------------------------------------------- | ------------ | ------------------------------------------------------------------------------------ | | `POST` | `/v1/projects/{ref}/cli/login-role` | Bearer token | `--linked` with no database password, to create a temp login role | -| `GET` | `/v1/projects/{ref}/config/database/pooler` | Bearer token | `--linked` with `--project-ref`, direct host unreachable, no usable saved pooler URL | +| `GET` | `/v1/projects/{ref}/config/database/pooler` | Bearer token | `--linked`, direct host unreachable, no saved pooler URL recorded for the target ref | | `POST` | `/v1/projects/{ref}/network-bans/retrieve` | Bearer token | `--linked`, from the 3rd failed temp-role connect (list bans) | | `DELETE` | `/v1/projects/{ref}/network-bans` | Bearer token | `--linked`, only when bans are listed (clear a self-ban) | @@ -131,4 +131,6 @@ emit `Command "" is deprecated and will be removed in v3, use "" i error rather than a silently discarded flag (deliberately stricter than `SUPABASE_PROJECT_ID`, which Go's equivalent env var simply leaves unused on a non-linked target). Shared verbatim by every `inspect db` subcommand via - `INSPECT_DB_FLAGS`. + `INSPECT_DB_FLAGS`. The saved `.temp/pooler-url` applies only when the target ref is the + linked ref; a `--project-ref` or `SUPABASE_PROJECT_ID` naming another project fetches that + project's pooler config from the Management API. diff --git a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md index 2714c1fa17..a914fded01 100644 --- a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md @@ -10,7 +10,7 @@ validating those CSVs. | ------------------------------------------ | ---------- | -------------------------------------------------------------------------------------- | | `/supabase/config.toml` | TOML | always — `[experimental.inspect.rules]` (custom rules) + `[db]` subtree | | `/supabase/.env*` (nested) | dotenv | always — `env(VAR)` expansion for `[db]` and rule string fields | -| `/supabase/.temp/pooler-url` | plain text | `--linked` path (pooler connection string) | +| `/supabase/.temp/pooler-url` | plain text | `--linked` path, only when the target ref is the linked ref (pooler connection string) | | `/supabase/.temp/project-ref` | plain text | `--linked` path, without `--project-ref` or `SUPABASE_PROJECT_ID` (linked project ref) | | `~/.supabase/access-token` | plain text | `--linked` path, when `SUPABASE_ACCESS_TOKEN` unset and keyring unavailable | | `//.csv` ×14 | CSV | read back in-memory for rule evaluation | diff --git a/apps/cli/src/commands/link/SIDE_EFFECTS.md b/apps/cli/src/commands/link/SIDE_EFFECTS.md index 1bbb86f0b9..462d3b2722 100644 --- a/apps/cli/src/commands/link/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/link/SIDE_EFFECTS.md @@ -10,18 +10,19 @@ counterpart exists for this behavior. ## Files Read -| Path | Format | When | -| ------------------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `supabase/config.toml` | TOML (`project_id`) | NOT read for ref resolution itself — read by `CommandSettings` for workdir/project-id discovery generally (`--workdir` resolution, `SUPABASE_PROJECT_ID` passthrough) | -| `supabase/.temp/linked-project.json` | JSON (`ref` field) | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 2nd parent-project candidate for branch-name resolution (CLI-2167, TS-only) | -| `supabase/.temp/project-ref` | plain text | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 3rd (last) parent-project candidate for branch-name resolution (CLI-2167, TS-only) | -| `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` is unset and the keyring is unavailable | +| Path | Format | When | +| ------------------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `supabase/config.toml` | TOML (`project_id`) | NOT read for ref resolution itself — read by `CommandSettings` for workdir/project-id discovery generally (`--workdir` resolution, `SUPABASE_PROJECT_ID` passthrough) | +| `supabase/.temp/linked-project.json` | JSON (`ref` field) | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 2nd parent-project candidate for branch-name resolution (CLI-2167, TS-only) | +| `supabase/.temp/project-ref` | plain text | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 3rd (last) parent-project candidate for branch-name resolution (CLI-2167, TS-only) | +| `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` is unset and the keyring is unavailable | +| `$SUPABASE_HOME/stacks//` | JSON | stack mode only, when the lock is written and the lease is released: saved stack state and database data records, to warn about data a new pin would be refused by; nothing is created, locked or probed (see "Hosted versions in `supabase/supabase.lock`") | > For resolving the final linked ref, the on-disk `supabase/.temp/project-ref` file is **not** > read — `link` never falls back to it there. It **is** read for the TS-only branch-name lookup above, which resolves > the currently-linked _parent_ project (env `SUPABASE_PROJECT_ID` → `linked-project.json` → > `project-ref` file, first ref-shaped candidate wins) before searching its branches. This -> deliberately does NOT reuse `ProjectRefResolver.resolveOptional` — that resolves the +> deliberately does NOT reuse `RemoteTarget.resolveOptional` — that resolves the > FINAL linked ref, which right after linking a branch would be the branch's own ref, breaking a > second `link ` (CLI-2167 follow-up). `linked-project.json` works as the parent > candidate because `link`'s plain-project arm writes it for a real (non-404) project, and the 404 @@ -32,8 +33,8 @@ counterpart exists for this behavior. > lost until a real (non-404) `link` run. `LinkedProjectCache.cache` (the post-run > PersistentPostRun-parity fill) never overwrites an EXISTING file either way, so between the two, > the cache reliably tracks the last known-good parent project even across subsequent branch links. -> This parent resolution is hoisted into `command-internal/parent-project-ref.ts` -> (`resolveLinkedParentRef`), shared with the `branches` command family, which is +> This parent resolution is `RemoteTarget.parent` +> in `config/remote-target.layer.ts`, shared with the `branches` command family, which is > PARENT-scoped for the same reason — see `branches/list/SIDE_EFFECTS.md`. ## Files Written @@ -88,7 +89,8 @@ Management API (base `CommandSettings.apiUrl`, `Authorization: Bearer .`, `apikey: ` + `Authorization: Bearer `): +Tenant service gateway (`https://.`, `apikey: ` + `Authorization: Bearer `; +a redirect is never followed and counts as a failed probe): | Method | Path | When | | ------ | --------------------- | ----------- | @@ -102,11 +104,12 @@ Tenant service gateway (`https://.`, `apikey: ` + ## Environment Variables -| Variable | Purpose | -| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `SUPABASE_PROJECT_ID` | link-target resolution: `[ref-or-branch]` positional → `--project-ref` → env → TTY prompt (CLI-2167 adds the positional ahead of the flag). Also the 1st parent-project candidate for a TS-only branch-name lookup (CLI-2167). | -| `SUPABASE_ACCESS_TOKEN` | Management API bearer auth (env → keyring → `~/.supabase/access-token`) | -| `SUPABASE_DB_PASSWORD` | **no-op** for `link`: neither this variable nor `--password` is read (the DB-connection path that would consume them is dead code in Go) | +| Variable | Purpose | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `SUPABASE_PROJECT_ID` | link-target resolution: `[ref-or-branch]` positional → `--project-ref` → env → TTY prompt (CLI-2167 adds the positional ahead of the flag). Also the 1st parent-project candidate for a TS-only branch-name lookup (CLI-2167). | +| `SUPABASE_ACCESS_TOKEN` | Management API bearer auth (env → keyring → `~/.supabase/access-token`) | +| `SUPABASE_DB_PASSWORD` | **no-op** for `link`: neither this variable nor `--password` is read (the DB-connection path that would consume them is dead code in Go) | +| `SUPABASE_EXPERIMENTAL_STACK` | `1` or `0` selects stack project-state mode over `[experimental] stack`; unset or empty uses the config. Any other value fails the command (see Exit Codes) | ## Exit Codes @@ -125,6 +128,7 @@ Tenant service gateway (`https://.`, `apikey: ` + | `1` | branch name/UUID not found on the linked parent project (CLI-2167, TS-only) | | `1` | branch resolved but has no `project_ref` yet (still provisioning) (CLI-2167, TS-only) | | `1` | branch listing failure (network/status) during branch-name resolution (CLI-2167, TS-only) | +| `1` | `SUPABASE_EXPERIMENTAL_STACK` is set to a non-empty value other than `0` or `1` (`must be 0 or 1 when set`); nothing is written | > Best-effort service-link and telemetry errors never affect the exit code. @@ -150,6 +154,113 @@ project-ref link). The `Resolved branch ""...` line above still goes out v in these modes (stderr in `json`; a structured `log` event in `stream-json`) rather than `output.raw`. Warnings still go to stderr. +## Stack mode + +When stack project-state mode is active (`SUPABASE_EXPERIMENTAL_STACK=1`, or `[experimental] stack` +in `config.toml` with the variable unset), the link target is one gitignored document, +`/supabase/.link.json`, and nothing is written under `supabase/.temp/`: + +```json +{ + "project": { "ref": "…", "name": "…", "organization_id": "…", "organization_slug": "…" }, + "active_branch": { "ref": "…", "name": "…", "is_default": false, "pooler_url": "…" }, + "fetchedAt": "" +} +``` + +- `project` is the parent project; `active_branch.ref` is the ref commands act on, which is the + project's own ref unless a branch was linked (`name` and `is_default` appear for a branch + resolved by name; a raw branch ref whose parent is unknown records only `is_default: false`). When the ref is a branch project that `getProject` does not return, the + parent comes from the branch resolution, else from the previous document's project when that + project still lists the ref as a branch, else the ref itself. +- `pooler_url` uses the same normalization as `.temp/pooler-url`; it is omitted with + `--skip-pooler` or when the project reports no primary pooler. +- The document is written atomically after `supabase/.gitignore` gains `.link.json`, + `..link.json.*.tmp` (the staged write), `.supabase.lock.lease`, and `.supabase.lock.*.tmp` (the + staged lock write). The Postgres version and `linked-project.json` are not written. +- A document that does not decode fails reads with `ProjectLinkMalformedError`, which suggests + running `supabase link` again; a read or write failure other than absence is + `ProjectLinkIoError`. +- The organization and project `groupIdentify` calls fire from `link` itself, since the post-run + linked-project cache fill is a no-op in this mode. For other commands the fill writes + `$SUPABASE_HOME/project-cache/.json` (atomic, once per ref; see `docs/supabase-home.md`) + instead of `.temp/linked-project.json`. + +### Hosted versions in `supabase/supabase.lock` + +When the project has a `supabase/config.toml`, stack-mode `link` also records what the linked +project or branch runs in `supabase/supabase.lock`. The config is validated first, so an invalid +`supabase/config.toml` fails the link before any probe or write. Without a `config.toml` there is no +lock, only `.link.json` is written, under the same project lease, and nothing below runs. A lease that +stays busy fails the link with `LockResolutionError` reason `lease-busy` before anything is written. +`supabase link` always needs a login. Under the project lease `supabase/.supabase.lock.lease` it: + +- Probes the target (the `link.probe` span; `stack update` runs the same probe under `update.probe`): + the database version from `getProject` (`database.version`), or from `getABranchConfig` + (`postgres_version`) when the ref is a branch the project lookup does not return, and the `rest`, `auth` and `storage` versions from the target's + tenant host (`/rest/v1/`, `/auth/v1/health`, `/storage/v1/version`) with the service-role key. + The key is sent as `apikey`, and as a bearer token unless it is an `sb_` key; it is never logged + or recorded, and a redirect is never followed (the probe counts it as failed). Services without a + version endpoint are not probed. +- Resolves the entries whose specifier is `linked` (the default) against the observations (the + `link.lock` span), looking each hosted version up on the slim-services release host + (`SUPABASE_INTERNAL_SLIM_RELEASES_URL` and `SUPABASE_INTERNAL_SLIM_REGISTRY_URL` redirect it), and + records every observation in its entry's `linked` field even when a specifier overrides it. The + lock's top-level `linked` gains the project, the branch (`ref`, `isDefault`; a project link + without a branch records the project ref as the branch ref, default) and `observedAt`. A branch + linked by ref or by name records the same identity; its name is kept only in `.link.json`. +- A service the platform does not report, or a hosted version slim-services does not publish + (`resolution` `unobserved` or `unpublished`), takes the version bundled with the CLI. Services + with no version endpoint (`realtime`, `imgproxy`, `functions`, `studio`, `pgmeta`, `mail`, + `analytics`, `pooler`) are recorded as `unobserved` without a warning; one the probe should have + read (`database`, `rest`, `auth`, `storage`) warns. A database + whose hosted major differs from `db.major_version` does the same. A hosted or release-host + failure that is transient keeps the previous entry only when the remote, the specifier and + `db.major_version` are unchanged, else takes the bundled version as `unobserved`; both print a + warning with the cause. An entry that fell back and whose inputs are unchanged keeps its pin. + Linking to another project or branch keeps nothing observed on the old remote. An explicit + `` or `-r` specifier that cannot be resolved fails the link, and a release + integrity mismatch always does. +- Once every entry has resolved, writes `.link.json`, then the lock (sorted-key JSON, atomically) + only when it differs from the stored one, ignoring `linked.observedAt`, all while holding the + lease; `supabase/.gitignore` gains its entries when the lease is taken. A resolution failure + writes neither. The link goes first so a lock never observes a remote the checkout does not + target: if the lock write fails, `.link.json` already targets the new remote while the lock still + observes the previous one, the next `start` warns that the lock observed another remote, and + `supabase stack update` or running `supabase link` again repairs it. The command then prints the + hosted and locked version of each service and, when the lock changed, + `Updated supabase/supabase.lock[: now observing ] ( version changes). Review and commit it.`, + else `supabase/supabase.lock is already up to date.`; the report is printed after both writes, + still under the lease. In JSON output it adds `lock` (`file`, `written`, `services` with + `service`, `hosted`, `locked`, `resolution`). When the lock was written, the lease is then + released and the new pins are checked against the database data of every saved stack of this + project (the `StackLock.dataConflicts` span, which records `lock.data_conflict_count`), by the + rules the next `supabase stack start` of that stack applies. Each refusal gets one + warning, `stack : the next supabase start refuses these pins:` followed by + the failure that start reports: a database build other than a new `-rN` revision of the one that + initialized the data (on Docker and Podman, also the one an interrupted first start recorded), or a schema-owning service the stack runs against that database that is + older than one that already migrated it. The failure names the recorded and refused versions and + the remedies: pinning the recorded version in `supabase/config.toml` + (`[local.services.] version`, with `[db] major_version` set back when the database major + differs and any `[db] orioledb_version` removed when an OrioleDB build is requested), or + `supabase stack destroy --stack-id `, which permanently deletes the stack's local database + data (data with no recorded identity offers only the destroy). The lock stays written, the link + never fails on this check, and saved state that cannot be read is logged at debug level and + warns of nothing. + +`--frozen`, or `CI` set (not empty, `false` or `0`) with an existing lock, makes `link` write only +`.link.json`: the lock is untouched, no data check runs, and a warning lists the services whose +hosted versions differ from it, advising `supabase stack update`. A missing lock is created even under `CI`. The probes +still run. In legacy mode the flag is accepted and does nothing. `supabase unlink` never touches the +lock. + +Network by command: `supabase link` reaches the Management API, the target's tenant host and the +release host. `supabase start` and `supabase stack install` reach only the release host, only when +they resolve a release, and never need a login or the Management API; `supabase stack update` +reaches the Management API and tenant host only when a `linked` entry is re-resolved and +`supabase/.link.json` exists (it then needs a login), plus the release host; `supabase stack +outdated` reaches only the release host. + ## Known divergence - The cosmetic `WARNING: Local database version differs from the linked project.` message is diff --git a/apps/cli/src/commands/link/link.command.ts b/apps/cli/src/commands/link/link.command.ts index da2ba02398..720235e62c 100644 --- a/apps/cli/src/commands/link/link.command.ts +++ b/apps/cli/src/commands/link/link.command.ts @@ -2,11 +2,11 @@ import { Option } from "effect"; import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; -import { PROJECT_REF_PATTERN } from "../../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../../config/remote-target.service.ts"; import { withPermissions } from "../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withContext } from "../../command-internal/command-permissions/model.ts"; import { linkServices } from "../../command-internal/link-services-core.ts"; -import { projectRefPicker } from "../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../command-internal/management-api-runtime.layer.ts"; @@ -35,6 +35,11 @@ const config = { Flag.withDescription("Use direct connection instead of pooler."), Flag.withDefault(false), ), + frozen: Flag.Boolean("frozen").pipe( + Flag.withDescription("Leave supabase/supabase.lock untouched (stack mode only)."), + Flag.withDefault(false), + Flag.withHidden, + ), } as const; export type LinkFlags = CliCommand.Command.Config.Infer; @@ -82,6 +87,12 @@ export const linkCommand = Command.make("link", config).pipe( kind: "required", request: { query: { reveal: "true" } }, }, + { + operationId: "v1-get-a-branch-config", + kind: "best-effort", + context: + "stack mode, supabase/config.toml exists, and the project lookup returns 404 (the ref is a branch)", + }, { operationId: "v1-list-all-branches", kind: "best-effort", @@ -89,7 +100,7 @@ export const linkCommand = Command.make("link", config).pipe( "the project lookup returns 404 (the ref is a branch), the ref was not resolved from a branch name, and supabase/.temp/linked-project.json names a different project", }, ], - noApiEffectFlags: ["password"], + noApiEffectFlags: ["password", "frozen"], }, withContext(projectRefPicker, "no [ref-or-branch] argument"), linkServices, diff --git a/apps/cli/src/commands/link/link.handler.ts b/apps/cli/src/commands/link/link.handler.ts index 1ab9394bba..05dee044ec 100644 --- a/apps/cli/src/commands/link/link.handler.ts +++ b/apps/cli/src/commands/link/link.handler.ts @@ -1,37 +1,54 @@ import type { V1ListAllBranchesOutput } from "@supabase/api/effect"; -import { Duration, Effect, FileSystem, Option, Path, Ref, Schema } from "effect"; -import type { PlatformError } from "effect/PlatformError"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { DateTime, Duration, Effect, Option, Ref, Schema } from "effect"; import { CommandPlatformApi } from "../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; -import { ProjectRefResolver, PROJECT_REF_PATTERN } from "../../config/project-ref.service.ts"; +import { RemoteTarget, PROJECT_REF_PATTERN } from "../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../telemetry/telemetry-state.service.ts"; import { Output } from "../../shared/output/output.service.ts"; import { Analytics } from "../../shared/telemetry/analytics.service.ts"; import { withAnalyticsContext } from "../../shared/telemetry/analytics-context.ts"; +import { identifyProjectGroups } from "../../shared/telemetry/identify-project-groups.ts"; import { EventProjectLinked, - GroupOrganization, - GroupProject, PropLinkedVia, PropParentProjectRef, } from "../../shared/telemetry/event-catalog.ts"; import { classifyProjectLookupError } from "../../command-internal/branch-target.ts"; import { - type CachedLinkedProject, parentNotLinkedMessage, parentRefInvalidMessage, parentRefTypoHint, - parseCachedLinkedProject, - resolveLinkedParentRef, } from "../../command-internal/parent-project-ref.ts"; +import { + type CachedLinkedProject, + parseCachedLinkedProject, +} from "../../command-internal/link-snapshot.ts"; import { dashboardUrl } from "../../command-internal/profile.ts"; import { mapTenantApiKeysError } from "../../command-internal/get-tenant-api-keys.ts"; import { sanitizeInlineName, mapHttpError } from "../../command-internal/http-errors.ts"; -import { linkServicesCore } from "../../command-internal/link-services-core.ts"; +import { fetchPoolerUrl, linkServicesCore } from "../../command-internal/link-services-core.ts"; +import { currentProjectStateMode } from "../../command-internal/project-state-mode.ts"; +import { + type ProjectLink, + readProjectLink, + ensureProjectDirectory, + writeProjectLink, +} from "../../command-internal/project-link.ts"; import { extractServiceKeys } from "../../command-internal/tenant-keys.ts"; -import { tempPaths } from "../../command-internal/temp-paths.ts"; +import { linkRemote, observeHostedVersions } from "../../command-internal/hosted-versions.ts"; +import { loadLocalProjectContext } from "../../command-internal/local-project-context.ts"; +import { loadStackConfig, StackConfigError } from "../../command-internal/stack-config.ts"; +import { + LOCK_FILE, + linkedLock, + type LinkedLock, + type LockRequest, + withProjectLease, +} from "../../command-internal/stack-lock.ts"; +import { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; import { LinkApiKeysNetworkError, LinkAuthTokenError, @@ -57,6 +74,7 @@ type LinkBranch = LinkBranches[number]; interface LinkBranchResolution { readonly projectRef: string; readonly branchName: string; + readonly isDefault: boolean; readonly parentRef: string; } @@ -70,8 +88,6 @@ const classifyProjectError = classifyProjectLookupError({ networkMessage: (cause) => `failed to retrieve remote project status: ${String(cause)}`, }); -type WriteTempFile = (filePath: string, content: string) => Effect.Effect; - const mapApiKeysError = mapTenantApiKeysError({ networkError: LinkApiKeysNetworkError, statusError: LinkAuthTokenError, @@ -142,8 +158,9 @@ function linkBranchNotFoundMessage( const resolveLinkBranchRef = Effect.fn("link.resolveBranch")(function* (value: string) { const output = yield* Output; const api = yield* CommandPlatformApi; + const remoteTarget = yield* RemoteTarget; - const parent = yield* resolveLinkedParentRef(); + const parent = yield* remoteTarget.parent; if (parent.kind === "absent") { return yield* new LinkBranchNotLinkedError({ message: parentNotLinkedMessage(value) }); } @@ -196,19 +213,149 @@ const resolveLinkBranchRef = Effect.fn("link.resolveBranch")(function* (value: s const line = `Resolved branch "${sanitizeInlineName(found.name)}" of project ${parentRef} to project ref ${found.project_ref}.`; yield* output.format === "text" ? output.raw(`${line}\n`, "stderr") : output.info(line); - return { projectRef: found.project_ref, branchName: found.name, parentRef }; + return { + projectRef: found.project_ref, + branchName: found.name, + isDefault: found.is_default, + parentRef, + }; +}); + +/** + * The parent project recorded in a stack-mode link written for a branch-project ref (the 404 + * arm). A resolved branch names its parent, keeping an agreeing document's richer metadata; a + * raw ref keeps a divergent document's project only when that project still lists `ref` as a + * branch, and otherwise has no known parent, so the project is the ref itself. + */ +const resolveStackLinkParent = Effect.fn("link.resolveStackParent")(function* (input: { + readonly ref: string; + readonly workdir: string; + readonly branchResolution: Option.Option; +}) { + const output = yield* Output; + const api = yield* CommandPlatformApi; + const existing = (yield* readProjectLink(input.workdir).pipe(Effect.option)).pipe(Option.flatten); + const existingProject = Option.map(existing, (link) => link.project); + + if (Option.isSome(input.branchResolution)) { + const parentRef = input.branchResolution.value.parentRef; + return Option.isSome(existingProject) && existingProject.value.ref === parentRef + ? existingProject.value + : { ref: parentRef }; + } + if (Option.isNone(existingProject) || existingProject.value.ref === input.ref) { + return { ref: input.ref }; + } + if (!PROJECT_REF_PATTERN.test(existingProject.value.ref)) return { ref: input.ref }; + + const correlating = + output.format === "text" ? yield* output.task("Checking branch parent...") : undefined; + const verified = yield* api.v1.listAllBranches({ ref: existingProject.value.ref }).pipe( + Effect.timeout(LINK_CACHE_CORRELATION_TIMEOUT), + Effect.map((branches) => branches.some((branch) => branch.project_ref === input.ref)), + Effect.orElseSucceed(() => false), + Effect.ensuring(correlating?.clear ?? Effect.void), + Effect.tap((result) => Effect.annotateCurrentSpan("link.parent_verified", result)), + ); + return verified ? existingProject.value : { ref: input.ref }; +}); + +const renderLinkedLock = (result: LinkedLock) => { + const rows = result.services.map(({ service, hosted, locked, resolution }) => [ + service, + hosted ?? "-", + locked, + resolution, + ]); + const header = ["service", "hosted", "locked", "resolution"]; + const widths = header.map((title, column) => + Math.max(title.length, ...rows.map((row) => row[column]?.length ?? 0)), + ); + const line = (cells: ReadonlyArray) => + ` ${cells.map((cell, column) => cell.padEnd(widths[column] ?? 0)).join(" ")}`.trimEnd(); + const changed = result.services.filter(({ changed }) => changed).length; + return [ + "Hosted versions:", + line(header), + ...rows.map(line), + result.frozen + ? `${LOCK_FILE} was left untouched.` + : result.written + ? `Updated ${LOCK_FILE}${result.observing === undefined ? "" : `: now observing ${result.observing}`} (${changed} ${changed === 1 ? "version change" : "version changes"}). Review and commit it.` + : `${LOCK_FILE} is already up to date.`, + ].join("\n"); +}; + +/** + * Publishes a stack-mode link and records its hosted versions in `supabase/supabase.lock`: the + * project config is validated first, then, under the project lease, the project or branch is + * probed, `.link.json` published, and the lock written. `undefined` when the project has no + * `supabase/config.toml`, so there is no lock to keep and only `.link.json` is published. + */ +const publishStackLink = Effect.fn("link.lock")(function* (input: { + readonly ref: string; + readonly link: ProjectLink; + readonly serviceKey: string; + /** The version `getProject` reported; absent when the ref is a branch. */ + readonly databaseVersion: Option.Option; + readonly frozen: boolean; +}) { + const output = yield* Output; + const api = yield* CommandPlatformApi; + const { workdir } = yield* CommandSettings; + const publishLink = writeProjectLink(workdir, input.link); + const context = yield* loadLocalProjectContext( + workdir, + (message) => new StackConfigError({ message }), + ); + if (context.loaded === null) { + yield* ensureProjectDirectory(workdir); + yield* withProjectLease(workdir, publishLink); + return undefined; + } + + // The loader offers the lock request only part-way through validating the config, so the + // request is taken from it and the lock is resolved once the whole config has validated. + const requested = yield* Ref.make(Option.none()); + yield* loadStackConfig(workdir, { + context, + selection: (request) => + Ref.set(requested, Option.some(request)).pipe(Effect.as(defaultArtifactSelection())), + }); + const request = yield* Ref.get(requested).pipe(Effect.flatMap(Effect.fromOption), Effect.orDie); + const result = yield* linkedLock({ + frozen: input.frozen, + probe: observeHostedVersions({ + api, + ref: input.ref, + serviceKey: input.serviceKey, + databaseVersion: input.databaseVersion, + }).pipe(Effect.map((observations) => ({ remote: linkRemote(input.link), observations }))), + publishLink, + report: (locked) => + output.format === "text" ? output.info(renderLinkedLock(locked)) : Effect.void, + output, + })(request); + yield* Effect.annotateCurrentSpan({ + "link.resolved_count": result.services.length, + "link.unobserved_count": result.services.filter(({ resolution }) => resolution === "unobserved") + .length, + "link.kept_count": result.kept.length, + "link.lock_written": result.written, + }); + return result; }); export const link = Effect.fn("link")(function* (flags: LinkFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const analytics = yield* Analytics; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; + const legacyState = yield* LegacyProjectState; + const stackState = (yield* currentProjectStateMode) === "stack"; // Captured once ref/branch resolution succeeds; `undefined` means no ref was ever resolved, // so the post-run cache fill below stays a no-op. @@ -243,18 +390,13 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { ? Option.some(branchResolution.value.projectRef) : requested; - const ref = yield* resolver.resolveForLink(resolvedRefOrBranch); + const { ref } = yield* resolver.resolveForLink(resolvedRefOrBranch); yield* Ref.set(resolvedRef, ref); yield* Effect.annotateCurrentSpan({ "project.ref": ref, "link.via_branch": Option.isSome(branchResolution), }); - const paths = tempPaths(path, cliSettings.workdir); - - const writeTempFile: WriteTempFile = (filePath, content) => - fs - .makeDirectory(path.dirname(filePath), { recursive: true }) - .pipe(Effect.andThen(() => fs.writeFileString(filePath, content))); + const workdir = cliSettings.workdir; // 1. Check remote project status (404 tolerated for branch projects). const project = yield* api.v1 @@ -283,8 +425,8 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { } // Updates the postgres image version to match the remote project. const version = project.value.database.version; - if (version.length > 0) { - yield* writeTempFile(paths.postgresVersion, version); + if (!stackState && version.length > 0) { + yield* legacyState.write(workdir, "postgresVersion", version); } } @@ -297,45 +439,79 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { return yield* new LinkMissingKeyError({ message: "Anon key not found." }); } - // 3. Link services — best-effort, using the service-role key for tenant probes. - yield* linkServicesCore({ - ref, - serviceKey: serviceRole, - skipPooler: flags.skipPooler, - workdir: cliSettings.workdir, - }); - - // 4. Save project ref (mandatory — a write failure fails the command). - yield* writeTempFile(paths.projectRef, ref); + // 3. Link services — best-effort, using the service-role key for tenant probes. Stack mode + // keeps only the pooler URL; versions come from the service lock. + const poolerUrl = stackState + ? flags.skipPooler + ? Option.none() + : yield* fetchPoolerUrl(api, ref).pipe(Effect.option, Effect.map(Option.flatten)) + : yield* linkServicesCore({ + ref, + serviceKey: serviceRole, + skipPooler: flags.skipPooler, + workdir, + }).pipe(Effect.as(Option.none())); + + // 4. Save the link (mandatory — a write failure fails the command). In stack mode the link + // and the lock are published together under the lock lease, the link first. + let stackLock: LinkedLock | undefined; + if (stackState) { + const parent = Option.isSome(project) + ? { + ref: project.value.ref, + name: project.value.name, + organization_id: project.value.organization_id, + organization_slug: project.value.organization_slug, + } + : yield* resolveStackLinkParent({ ref, workdir, branchResolution }); + const link: ProjectLink = { + project: parent, + active_branch: { + ref, + ...(Option.isSome(branchResolution) + ? { + name: branchResolution.value.branchName, + is_default: branchResolution.value.isDefault, + } + : // A ref that is not a project is a non-default branch, even when its parent is unknown. + Option.isNone(project) + ? { is_default: false } + : {}), + ...(Option.isSome(poolerUrl) ? { pooler_url: poolerUrl.value } : {}), + }, + fetchedAt: DateTime.formatIso(yield* DateTime.now), + }; + stackLock = yield* publishStackLink({ + ref, + link, + serviceKey: serviceRole, + databaseVersion: Option.map(project, (found) => found.database.version), + frozen: flags.frozen, + }); + } else { + yield* legacyState.write(workdir, "projectRef", ref); + } // 5. Telemetry + linked-project cache (only for resolvable projects, not the 404 branch path). if (Option.isSome(project)) { const p = project.value; // Best-effort: if the rewrite fails while a stale cache for a different project // survives, delete it rather than leave the parent chain trusting the old project. - yield* encodeLinkedProjectCache({ - ref: p.ref, - name: p.name, - organization_id: p.organization_id, - organization_slug: p.organization_slug, - }).pipe( - Effect.flatMap((content) => writeTempFile(paths.linkedProjectCache, content)), - Effect.catch(() => fs.remove(paths.linkedProjectCache, { force: true })), - Effect.ignore, - ); - - const groups = { organization: p.organization_id, project: p.ref } as const; - if (p.organization_id.length > 0) { - yield* analytics.groupIdentify(GroupOrganization, p.organization_id, { - organization_slug: p.organization_slug, - }); - } - if (p.ref.length > 0) { - yield* analytics.groupIdentify(GroupProject, p.ref, { + if (!stackState) { + yield* encodeLinkedProjectCache({ + ref: p.ref, name: p.name, + organization_id: p.organization_id, organization_slug: p.organization_slug, - }); + }).pipe( + Effect.flatMap((content) => legacyState.write(workdir, "linkedProjectCache", content)), + Effect.catch(() => legacyState.remove(workdir, "linkedProjectCache")), + Effect.ignore, + ); } + + const groups = { organization: p.organization_id, project: p.ref } as const; + yield* identifyProjectGroups(analytics, p); // A default branch's `project_ref` is the parent's own ref, so `link main` still hits // this 200 arm (never the 404 arm below); check `branchResolution` here too or a // default-branch link loses `linked_via`. @@ -352,12 +528,14 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { // 404 path: `ref` is a branch. The plain-project arm above never writes // `linked-project.json` for this ref, and the post-run cache fill (`Effect.ensuring` // below) also GETs `ref` and 404s on a branch — so without persisting parent evidence - // here, `resolveLinkedParentRef`'s chain can lose it or go stale. Both branches below + // here, `RemoteTarget.parent`'s chain can lose it or go stale. Both branches below // are best-effort and never affect `link`'s own outcome. - const cachedParent = yield* fs.readFileString(paths.linkedProjectCache).pipe( - Effect.map(parseCachedLinkedProject), - Effect.orElseSucceed(() => Option.none()), - ); + const cachedParent = stackState + ? Option.none() + : yield* legacyState.read(workdir, "linkedProjectCache").pipe( + Effect.map(parseCachedLinkedProject), + Effect.orElseSucceed(() => Option.none()), + ); if (Option.isSome(branchResolution)) { // (a) The branch's parent is known here, so persist it durably even when the @@ -365,12 +543,12 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { // it may be richer (name/org) than the ref-only record below, e.g. from a real // `link ` run. const parentRef = branchResolution.value.parentRef; - if (Option.isNone(cachedParent) || cachedParent.value.ref !== parentRef) { + if (!stackState && (Option.isNone(cachedParent) || cachedParent.value.ref !== parentRef)) { // If the replacement write fails, delete the stale cache instead of leaving a // wrong parent trusted — no parent info beats wrong parent info. yield* encodeLinkedParentCache({ ref: parentRef }).pipe( - Effect.flatMap((content) => writeTempFile(paths.linkedProjectCache, content)), - Effect.catch(() => fs.remove(paths.linkedProjectCache, { force: true })), + Effect.flatMap((content) => legacyState.write(workdir, "linkedProjectCache", content)), + Effect.catch(() => legacyState.remove(workdir, "linkedProjectCache")), Effect.ignore, ); } @@ -409,7 +587,7 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { Effect.tap((result) => Effect.annotateCurrentSpan("link.parent_verified", result)), ); if (!verified) { - yield* fs.remove(paths.linkedProjectCache, { force: true }).pipe(Effect.ignore); + yield* legacyState.remove(workdir, "linkedProjectCache").pipe(Effect.ignore); } } } @@ -420,6 +598,20 @@ export const link = Effect.fn("link")(function* (flags: LinkFlags) { } else { yield* output.success("", { project_ref: ref, + ...(stackLock === undefined + ? {} + : { + lock: { + file: LOCK_FILE, + written: stackLock.written, + services: stackLock.services.map(({ service, hosted, locked, resolution }) => ({ + service, + hosted: hosted ?? null, + locked, + resolution, + })), + }, + }), // Present only when a branch name/UUID was resolved above; absent for a plain // project-ref link. ...(Option.isSome(branchResolution) diff --git a/apps/cli/src/commands/link/link.integration.test.ts b/apps/cli/src/commands/link/link.integration.test.ts index e57e936012..922a43a4a3 100644 --- a/apps/cli/src/commands/link/link.integration.test.ts +++ b/apps/cli/src/commands/link/link.integration.test.ts @@ -269,6 +269,7 @@ const flags = (overrides: Partial = {}): LinkFlags => ({ projectRef: Option.some(VALID_REF), password: Option.none(), skipPooler: false, + frozen: false, ...overrides, }); diff --git a/apps/cli/src/commands/link/link.lock.integration.test.ts b/apps/cli/src/commands/link/link.lock.integration.test.ts new file mode 100644 index 0000000000..51cc5b8b12 --- /dev/null +++ b/apps/cli/src/commands/link/link.lock.integration.test.ts @@ -0,0 +1,636 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import type { V1ListAllBranchesOutput } from "@supabase/api/effect"; +import { readLock, writeLock } from "@supabase/stack/effect"; +import { catalogPins, postgresMajor, type ServiceKind } from "@supabase/stack/internal/artifacts"; +import { Effect, FileSystem, Layer, Option, PlatformError } from "effect"; +import * as HttpClient from "effect/http/HttpClient"; +import * as HttpClientResponse from "effect/http/HttpClientResponse"; + +import { + VALID_REF, + buildTestRuntime, + mockCommandPlatformApiService, + mockCommandSettings, + mockTelemetryStateTracked, + statusCodeFailure, + transportFailure, + useTempWorkdir, + withConfigEnv, +} from "../../../tests/helpers/command-mocks.ts"; +import { mockAnalytics, mockOutput, mockRuntimeInfo } from "../../../tests/helpers/mocks.ts"; +import { selection } from "../../../tests/helpers/pins.ts"; +import { seedProjectStack, unbundled } from "../../../tests/helpers/stack-lock.ts"; +import { + publishRelease, + releaseHostEnv, + startReleaseFixture, + type ReleaseFixture, +} from "../../../tests/helpers/release-fixture.ts"; +import { projectStateModeLayer } from "../../command-internal/project-state-mode.ts"; +import { ExperimentalFlag } from "../../command-internal/global-flags.ts"; +import { stackUpdate } from "../experimental/stack/update/update.handler.ts"; +import { link } from "./link.handler.ts"; +import type { LinkFlags } from "./link.command.ts"; + +const BRANCH_REF = "branchprojectrefabcd"; +const POOLER_URL = "postgresql://postgres.ref@pooler.example.co:5432/postgres"; +const HOSTED = "v99.0.0"; + +const PROJECT = { + id: VALID_REF, + ref: VALID_REF, + name: "My Project", + organization_id: "org_123", + organization_slug: "acme", + status: "ACTIVE_HEALTHY", + region: "us-east-1", + created_at: "2026-01-01T00:00:00Z", + database: { + host: "db.example.co", + version: "15.1.0.117", + postgres_engine: "15", + release_channel: "ga", + }, +}; + +const SERVICE_KEYS = [ + { + name: "service_role", + api_key: "service-role-key", + type: "secret", + secret_jwt_template: { role: "service_role" }, + }, + { name: "anon", api_key: "anon-key", type: "publishable" }, +]; + +const POOLER = [ + { + identifier: "primary", + database_type: "PRIMARY", + db_user: "postgres", + db_host: "pooler.example.co", + db_port: 6543, + db_name: "postgres", + connection_string: "postgresql://postgres.ref:[YOUR-PASSWORD]@pooler.example.co:6543/postgres", + connectionString: "", + default_pool_size: null, + max_client_conn: null, + pool_mode: "transaction", + }, +]; + +const FEATURE_BRANCH: (typeof V1ListAllBranchesOutput.Type)[number] = { + id: "11111111-2222-4333-8444-555555555555", + name: "feature-branch", + project_ref: BRANCH_REF, + parent_project_ref: VALID_REF, + is_default: false, + persistent: false, + status: "MIGRATIONS_PASSED", + created_at: "2026-05-27T01:02:03Z", + updated_at: "2026-05-27T01:02:04Z", + with_data: true, +}; + +type Reply = { readonly status: number; readonly body: string } | "fail"; + +/** What the hosted project's tenant endpoints answer; tests reassign a service to change it. */ +interface Tenant { + rest: Reply; + auth: Reply; + storage: Reply; + readonly requests: Array; +} + +const hostedTenant = (): Tenant => ({ + rest: { status: 200, body: JSON.stringify({ info: { version: "99.0.0 (abc123)" } }) }, + auth: { status: 200, body: JSON.stringify({ version: HOSTED }) }, + storage: { status: 200, body: "99.0.0" }, + requests: [], +}); + +const tenantLayer = (tenant: Tenant) => + Layer.succeed( + HttpClient.HttpClient, + HttpClient.make((request) => { + const { pathname } = new URL(request.url); + tenant.requests.push(pathname); + const reply = pathname.startsWith("/rest/") + ? tenant.rest + : pathname.startsWith("/auth/") + ? tenant.auth + : tenant.storage; + return reply === "fail" + ? Effect.fail(transportFailure(request)) + : Effect.succeed( + HttpClientResponse.fromWeb(request, new Response(reply.body, { status: reply.status })), + ); + }), + ); + +const producerOf = (kind: ServiceKind) => { + const producer = catalogPins().find( + (entry) => entry.service === kind && entry.isDefault, + )?.sourceService; + if (producer === undefined) throw new Error(`no producer for ${kind}`); + return producer; +}; + +const publish = (host: ReleaseFixture, kind: ServiceKind, upstream: string) => + publishRelease(host, { producer: producerOf(kind), upstream, revision: 0 }); + +const tempRoot = useTempWorkdir("supabase-link-lock-int-"); + +interface World { + readonly host: ReleaseFixture; + readonly tenant: Tenant; + readonly workdir: string; + /** Switches the link target between the project and its feature branch, and sets the hosted database version. */ + readonly target: { onBranch: boolean; database: string }; +} + +const startWorld = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const host = yield* startReleaseFixture; + for (const kind of ["rest", "auth", "storage"] as const) publish(host, kind, HOSTED); + const workdir = tempRoot.current; + yield* fs.makeDirectory(`${workdir}/supabase`, { recursive: true }); + yield* fs.writeFileString(`${workdir}/supabase/config.toml`, 'project_id = "link-lock"\n'); + return { + host, + tenant: hostedTenant(), + workdir, + target: { onBranch: false, database: PROJECT.database.version }, + } satisfies World; +}); + +const layerFor = ( + world: World, + out: ReturnType, + fileSystem?: Layer.Layer, +) => { + const apiMock = mockCommandPlatformApiService({ + v1: { + getProject: () => + world.target.onBranch + ? Effect.fail(statusCodeFailure(404)) + : Effect.succeed({ + ...PROJECT, + database: { ...PROJECT.database, version: world.target.database }, + }), + getABranchConfig: () => + Effect.succeed({ postgres_version: "15.1.0.117", postgres_engine: "15" }), + getProjectApiKeys: () => Effect.succeed(SERVICE_KEYS), + getPoolerConfig: () => Effect.succeed(POOLER), + listAllBranches: () => Effect.succeed(world.target.onBranch ? [FEATURE_BRANCH] : []), + }, + }); + return Layer.mergeAll( + buildTestRuntime({ + out, + api: { layer: apiMock.layer, httpClientLayer: tenantLayer(world.tenant) }, + cliSettings: mockCommandSettings({ workdir: world.workdir, projectId: Option.none() }), + analytics: mockAnalytics(), + telemetry: mockTelemetryStateTracked().layer, + ...(fileSystem === undefined ? {} : { fileSystem }), + }), + projectStateModeLayer("stack"), + mockRuntimeInfo({ platform: "linux", arch: "x64" }), + Layer.succeed(ExperimentalFlag, false), + ); +}; + +const flags = (overrides: Partial = {}): LinkFlags => ({ + refOrBranch: Option.none(), + projectRef: Option.some(VALID_REF), + password: Option.none(), + skipPooler: false, + frozen: false, + ...overrides, +}); + +const toBranch = (world: World, overrides: Partial = {}) => { + world.target.onBranch = true; + return flags({ + refOrBranch: Option.some("feature-branch"), + projectRef: Option.none(), + ...overrides, + }); +}; + +const env = (world: World) => ({ CI: "", ...releaseHostEnv(world.host) }); + +/** Runs `supabase link` against the world and returns what it printed. */ +const runLink = ( + world: World, + linkFlags: LinkFlags = flags(), + format: "text" | "json" = "text", +) => { + const out = mockOutput({ format }); + return withConfigEnv(env(world), link(linkFlags)).pipe( + Effect.provide(layerFor(world, out)), + Effect.as(out), + ); +}; + +const lockWriteFails = Layer.effect( + FileSystem.FileSystem, + Effect.map( + FileSystem.FileSystem, + (real) => + new Proxy(real, { + get(target, property, receiver) { + const value: unknown = Reflect.get(target, property, receiver); + if (property !== "rename" || typeof value !== "function") return value; + return (from: string, to: string) => + to.endsWith("supabase/supabase.lock") + ? Effect.fail( + PlatformError.systemError({ + _tag: "PermissionDenied", + module: "FileSystem", + method: "rename", + description: `EACCES: permission denied, rename '${from}' -> '${to}'`, + pathOrDescriptor: to, + }), + ) + : Reflect.apply(value, target, [from, to]); + }, + }), + ), +).pipe(Layer.provide(BunServices.layer)); + +const runLinkWithLockWriteFailing = (world: World, linkFlags: LinkFlags) => { + const out = mockOutput({ format: "text" }); + return withConfigEnv(env(world), link(linkFlags)).pipe( + Effect.provide(layerFor(world, out, lockWriteFails)), + ); +}; + +const runUpdate = (world: World) => { + const out = mockOutput({ format: "text" }); + return withConfigEnv(env(world), stackUpdate({ services: [] })).pipe( + Effect.provide(layerFor(world, out)), + Effect.as(out), + ); +}; + +const warnings = (out: ReturnType) => + out.messages.filter(({ type }) => type === "warn").map(({ message }) => message); + +const readServices = (workdir: string) => + readLock(workdir).pipe( + Effect.map(Option.getOrThrow), + Effect.map((lock) => lock.services), + ); + +const writeConfig = (world: World, services: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.writeFileString( + `${world.workdir}/supabase/config.toml`, + `project_id = "link-lock"\n${services}`, + ); + }); + +const lockText = (workdir: string) => + FileSystem.FileSystem.pipe( + Effect.flatMap((fs) => fs.readFileString(`${workdir}/supabase/supabase.lock`)), + ); + +describe("link records hosted versions in the stack lock", () => { + it.live("observes the hosted services and resolves linked entries to their releases", () => + Effect.gen(function* () { + const world = yield* startWorld; + + const out = yield* runLink(world); + + const lock = yield* readLock(world.workdir).pipe(Effect.map(Option.getOrThrow)); + expect(lock.linked).toEqual({ + project: VALID_REF, + branch: { ref: VALID_REF, isDefault: true }, + observedAt: expect.any(String), + }); + for (const kind of ["rest", "auth", "storage"] as const) { + expect(lock.services[kind]).toMatchObject({ + specifier: "linked", + resolution: "resolved", + linked: { version: HOSTED }, + resolved: { upstreamVersion: HOSTED, revision: 0 }, + }); + } + expect( + yield* FileSystem.FileSystem.pipe( + Effect.flatMap((fs) => fs.exists(`${world.workdir}/supabase/.link.json`)), + ), + ).toBe(true); + expect(out.messages.map(({ message }) => message).join("\n")).toContain( + "Updated supabase/supabase.lock", + ); + expect(lock.services.realtime.resolution).toBe("unobserved"); + expect(warnings(out)).toEqual([ + expect.stringMatching(/^database: hosted Postgres 15\.1\.0\.117/), + ]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("takes the bundled pin as unobserved for a service the platform does not report", () => + Effect.gen(function* () { + const world = yield* startWorld; + world.tenant.storage = { status: 404, body: "" }; + + const out = yield* runLink(world, flags(), "json"); + + const services = yield* readServices(world.workdir); + expect(services.storage).toMatchObject({ + resolution: "unobserved", + resolved: { + upstreamVersion: selection.storage.upstreamVersion, + revision: selection.storage.revision, + }, + }); + expect(services.storage.linked).toBeUndefined(); + expect(services.rest.resolution).toBe("resolved"); + expect(warnings(out).join("\n")).toContain("storage: the platform does not report"); + const success = out.messages.find(({ type }) => type === "success"); + expect(success?.data).toMatchObject({ + project_ref: VALID_REF, + lock: { + file: "supabase/supabase.lock", + written: true, + services: expect.arrayContaining([ + { service: "rest", hosted: "99.0.0", locked: `${HOSTED}-r0`, resolution: "resolved" }, + { + service: "storage", + hosted: null, + locked: `${selection.storage.upstreamVersion}-r${selection.storage.revision}`, + resolution: "unobserved", + }, + ]), + }, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("does not rewrite the lock when linking again changes nothing", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world); + const written = yield* lockText(world.workdir); + + const out = yield* runLink(world); + + expect(yield* lockText(world.workdir)).toBe(written); + const printed = out.messages.map(({ message }) => message).join("\n"); + expect(printed).toContain("supabase/supabase.lock is already up to date"); + expect(printed).not.toContain("Updated supabase/supabase.lock"); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("keeps a fallback entry's pin when linking again changes nothing", () => + Effect.gen(function* () { + const world = yield* startWorld; + world.tenant.storage = { status: 404, body: "" }; + yield* runLink(world); + const lock = yield* readLock(world.workdir).pipe(Effect.map(Option.getOrThrow)); + const pinned = unbundled(lock.services.storage.resolved); + yield* writeLock(world.workdir, { + ...lock, + services: { ...lock.services, storage: { ...lock.services.storage, resolved: pinned } }, + }); + + yield* runLink(world); + + expect((yield* readServices(world.workdir)).storage).toMatchObject({ + resolution: "unobserved", + resolved: pinned, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("records the same branch identity whether it was linked by ref or by name", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const world = yield* startWorld; + const branchOf = () => + readLock(world.workdir).pipe(Effect.map((lock) => Option.getOrThrow(lock).linked?.branch)); + + yield* runLink(world); + yield* runLink(world, toBranch(world)); + const byName = yield* branchOf(); + yield* fs.remove(`${world.workdir}/supabase/supabase.lock`); + yield* fs.remove(`${world.workdir}/supabase/.link.json`); + yield* runLink(world, flags({ projectRef: Option.some(BRANCH_REF) })); + + expect(byName).toEqual({ ref: BRANCH_REF, isDefault: false }); + expect(yield* branchOf()).toEqual(byName); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("fails the link, leaving no lock or link, when an explicit pin is not published", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const world = yield* startWorld; + yield* writeConfig(world, '[local.services.rest]\nversion = "v98.0.0"\n'); + + const error = yield* runLink(world).pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "unpublished" }); + expect(yield* fs.exists(`${world.workdir}/supabase/supabase.lock`)).toBe(false); + expect(yield* fs.exists(`${world.workdir}/supabase/.link.json`)).toBe(false); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("keeps the previous entry when a hosted probe fails and nothing else changed", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world); + const before = yield* readServices(world.workdir); + world.tenant.rest = "fail"; + + const out = yield* runLink(world); + + const after = yield* readServices(world.workdir); + expect(after.rest).toEqual(before.rest); + expect(warnings(out).join("\n")).toContain(`rest: the hosted version could not be read`); + expect(warnings(out).join("\n")).toContain(`kept ${HOSTED}-r0`); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("takes nothing from the old remote when a probe fails after the link moved", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world); + world.tenant.rest = "fail"; + + const out = yield* runLink(world, toBranch(world)); + + const lock = yield* readLock(world.workdir).pipe(Effect.map(Option.getOrThrow)); + expect(lock.linked?.branch).toEqual({ ref: BRANCH_REF, isDefault: false }); + expect(out.messages.map(({ message }) => message).join("\n")).toContain( + `now observing ${lock.linked?.project}/${BRANCH_REF}`, + ); + expect(lock.services.rest).toMatchObject({ + resolution: "unobserved", + resolved: { + upstreamVersion: selection.rest.upstreamVersion, + revision: selection.rest.revision, + }, + }); + expect(lock.services.rest.linked).toBeUndefined(); + expect(lock.services.auth.linked?.version).toBe(HOSTED); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("targets the new remote in .link.json when the lock cannot be written", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const world = yield* startWorld; + yield* runLink(world); + const written = yield* lockText(world.workdir); + + const error = yield* runLinkWithLockWriteFailing(world, toBranch(world)).pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "io" }); + const document = yield* fs.readFileString(`${world.workdir}/supabase/.link.json`); + expect(document).toContain(BRANCH_REF); + expect(yield* lockText(world.workdir)).toBe(written); + + yield* runLink(world, toBranch(world)); + + const lock = yield* readLock(world.workdir).pipe(Effect.map(Option.getOrThrow)); + expect(lock.linked?.branch.ref).toBe(BRANCH_REF); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("a frozen link records the target in .link.json and leaves the lock untouched", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world, flags({ skipPooler: true })); + const written = yield* lockText(world.workdir); + world.tenant.rest = { status: 200, body: '{"info":{"version":"98.0.0"}}' }; + + const out = yield* runLink(world, flags({ frozen: true })); + + expect(yield* lockText(world.workdir)).toBe(written); + const document = yield* FileSystem.FileSystem.pipe( + Effect.flatMap((fs) => fs.readFileString(`${world.workdir}/supabase/.link.json`)), + ); + expect(document).toContain(POOLER_URL); + expect(warnings(out).join("\n")).toContain( + "Hosted versions differ from supabase/supabase.lock", + ); + expect(out.messages.map(({ message }) => message).join("\n")).toContain( + "supabase/supabase.lock was left untouched", + ); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); + +describe("stack update follows the recorded observations", () => { + it.live("stack update asks the linked project again and moves linked entries", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world); + publish(world.host, "rest", "v99.1.0"); + world.tenant.rest = { status: 200, body: '{"info":{"version":"99.1.0"}}' }; + + yield* runUpdate(world); + + const services = yield* readServices(world.workdir); + expect(services.rest).toMatchObject({ + linked: { version: "v99.1.0" }, + resolved: { upstreamVersion: "v99.1.0", revision: 0 }, + }); + expect(services.auth.resolved.upstreamVersion).toBe(HOSTED); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); + +describe("link warns when a saved stack would refuse the versions it locks", () => { + const HOSTED_DATABASE = `${postgresMajor(selection.database.upstreamVersion)}.99.0.001`; + const BUILT = selection.database.upstreamVersion; + + const seed = ( + world: World, + data: Omit[0], "projectRoot" | "supabaseHome">, + ) => + seedProjectStack({ + ...data, + projectRoot: world.workdir, + supabaseHome: `${world.workdir}/.supabase`, + }); + + const hostDatabase = (world: World) => { + publish(world.host, "database", HOSTED_DATABASE); + world.target.database = HOSTED_DATABASE; + }; + + const stackWarnings = (out: ReturnType) => + warnings(out).filter((message) => message.startsWith("stack ")); + + const destroy = (stackId: string) => + `run \`supabase stack destroy --stack-id ${stackId}\` to recreate the stack — this permanently deletes its local database data`; + + it.live( + "offers pinning the recorded build when linking moves off the build that initialized the data", + () => + Effect.gen(function* () { + const world = yield* startWorld; + hostDatabase(world); + const stackId = yield* seed(world, { built: BUILT }); + + const out = yield* runLink(world); + + expect(stackWarnings(out)).toEqual([ + `stack ${stackId}: the next supabase start refuses these pins: PostgreSQL data was initialized by stock PostgreSQL build ${BUILT}, but stock PostgreSQL build ${HOSTED_DATABASE} was requested; keep this data by pinning \`[local.services.database] version = "${BUILT}"\` in supabase/config.toml, or ${destroy(stackId)}`, + ]); + expect((yield* readServices(world.workdir)).database.resolved.upstreamVersion).toBe( + HOSTED_DATABASE, + ); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("stays quiet when the data was built by the version the link locks", () => + Effect.gen(function* () { + const world = yield* startWorld; + hostDatabase(world); + yield* seed(world, { + built: HOSTED_DATABASE, + composed: ["auth"], + migrated: { auth: HOSTED }, + }); + + const out = yield* runLink(world); + + expect(stackWarnings(out)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("leaves a frozen link silent about saved stacks", () => + Effect.gen(function* () { + const world = yield* startWorld; + yield* runLink(world); + hostDatabase(world); + yield* seed(world, { built: BUILT }); + + const out = yield* runLink(world, flags({ frozen: true })); + + expect(stackWarnings(out)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.live("still links when the saved stacks cannot be read", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const world = yield* startWorld; + yield* fs.makeDirectory(`${world.workdir}/.supabase`, { recursive: true }); + yield* fs.writeFileString(`${world.workdir}/.supabase/stacks`, "not a directory"); + + const out = yield* runLink(world); + + expect(stackWarnings(out)).toEqual([]); + expect((yield* readServices(world.workdir)).auth.resolved.upstreamVersion).toBe(HOSTED); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/link/link.stack-mode.integration.test.ts b/apps/cli/src/commands/link/link.stack-mode.integration.test.ts new file mode 100644 index 0000000000..b3e9cdd585 --- /dev/null +++ b/apps/cli/src/commands/link/link.stack-mode.integration.test.ts @@ -0,0 +1,371 @@ +import { describe, expect, it } from "@effect/vitest"; +import type { V1ListAllBranchesOutput } from "@supabase/api/effect"; +import { holdLock } from "@supabase/stack/effect"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import * as HttpClient from "effect/http/HttpClient"; +import { TestClock } from "effect/testing"; + +import { mockAnalytics, mockOutput } from "../../../tests/helpers/mocks.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../tests/helpers/legacy-state-guard.ts"; +import { + VALID_REF, + buildTestRuntime, + mockCommandPlatformApiService, + mockCommandSettings, + mockTelemetryStateTracked, + statusCodeFailure, + useTempWorkdir, +} from "../../../tests/helpers/command-mocks.ts"; +import { projectStateModeLayer } from "../../command-internal/project-state-mode.ts"; +import { + ProjectLinkIoError, + ProjectLinkMalformedError, + readProjectLink, + writeProjectLink, +} from "../../command-internal/project-link.ts"; +import { RemoteTarget } from "../../config/remote-target.service.ts"; +import { link } from "./link.handler.ts"; +import type { LinkFlags } from "./link.command.ts"; + +const BRANCH_REF = "branchprojectrefabcd"; +const POOLER_URL = "postgresql://postgres.ref@pooler.example.co:5432/postgres"; + +const PROJECT = { + id: VALID_REF, + ref: VALID_REF, + name: "My Project", + organization_id: "org_123", + organization_slug: "acme", + status: "ACTIVE_HEALTHY", + region: "us-east-1", + created_at: "2026-01-01T00:00:00Z", + database: { + host: "db.example.co", + version: "15.1.0.117", + postgres_engine: "15", + release_channel: "ga", + }, +}; + +const SERVICE_KEYS = [ + { + name: "service_role", + api_key: "service-role-key", + type: "secret", + secret_jwt_template: { role: "service_role" }, + }, + { name: "anon", api_key: "anon-key", type: "publishable" }, +]; + +const POOLER = [ + { + identifier: "primary", + database_type: "PRIMARY", + db_user: "postgres", + db_host: "pooler.example.co", + db_port: 6543, + db_name: "postgres", + connection_string: "postgresql://postgres.ref:[YOUR-PASSWORD]@pooler.example.co:6543/postgres", + connectionString: "", + default_pool_size: null, + max_client_conn: null, + pool_mode: "transaction", + }, +]; + +const FEATURE_BRANCH: (typeof V1ListAllBranchesOutput.Type)[number] = { + id: "11111111-2222-4333-8444-555555555555", + name: "feature-branch", + project_ref: BRANCH_REF, + parent_project_ref: VALID_REF, + is_default: false, + persistent: false, + status: "MIGRATIONS_PASSED", + created_at: "2026-05-27T01:02:03Z", + updated_at: "2026-05-27T01:02:04Z", + with_data: true, +}; + +// These projects have no supabase/config.toml, so there is no lock to record and no tenant probe. +const noTenantHttp = Layer.succeed( + HttpClient.HttpClient, + HttpClient.make(() => + Effect.die("stack-mode link without a config must not call tenant endpoints"), + ), +); + +const tempRoot = useTempWorkdir("supabase-link-stack-int-"); + +interface SetupOpts { + readonly projectFails?: boolean; + readonly branches?: typeof V1ListAllBranchesOutput.Type; +} + +function setup(opts: SetupOpts = {}) { + const recorder = recordingFileSystem(); + const out = mockOutput({ format: "text" }); + const apiMock = mockCommandPlatformApiService({ + v1: { + getProject: () => + opts.projectFails === true ? Effect.fail(statusCodeFailure(404)) : Effect.succeed(PROJECT), + getProjectApiKeys: () => Effect.succeed(SERVICE_KEYS), + getPoolerConfig: () => Effect.succeed(POOLER), + listAllBranches: () => Effect.succeed(opts.branches ?? []), + }, + }); + const layer = Layer.mergeAll( + buildTestRuntime({ + out, + api: { layer: apiMock.layer, httpClientLayer: noTenantHttp }, + cliSettings: mockCommandSettings({ workdir: tempRoot.current, projectId: Option.none() }), + analytics: mockAnalytics(), + telemetry: mockTelemetryStateTracked().layer, + fileSystem: recorder.layer, + }), + projectStateModeLayer("stack"), + ); + return { layer, workdir: tempRoot.current, accessed: recorder.accessed }; +} + +const flags = (overrides: Partial = {}): LinkFlags => ({ + refOrBranch: Option.none(), + projectRef: Option.some(VALID_REF), + password: Option.none(), + skipPooler: false, + frozen: false, + ...overrides, +}); + +const readSupabaseFile = Effect.fnUntraced(function* (workdir: string, ...segments: string[]) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* fs.readFileString(path.join(workdir, "supabase", ...segments)); +}); + +const tempDirExists = Effect.fnUntraced(function* (workdir: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* fs.exists(path.join(workdir, "supabase", ".temp")); +}); + +const seedLink = writeProjectLink; + +describe("link integration in stack mode", () => { + it.effect("writes the whole link into supabase/.link.json and nothing under .temp", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + yield* link(flags()); + + const document = yield* readProjectLink(workdir); + expect(Option.getOrUndefined(document)).toEqual({ + project: { + ref: VALID_REF, + name: "My Project", + organization_id: "org_123", + organization_slug: "acme", + }, + active_branch: { ref: VALID_REF, pooler_url: POOLER_URL }, + fetchedAt: expect.any(String), + }); + expect(yield* tempDirExists(workdir)).toBe(false); + }).pipe(Effect.provide(layer)); + }); + + it.effect("never reads or writes supabase/.temp and ignores a legacy link left there", () => { + const { layer, workdir, accessed } = setup(); + seedLegacyStateDecoys(workdir); + return Effect.gen(function* () { + yield* link(flags()); + + const document = yield* readProjectLink(workdir); + expect(Option.map(document, (value) => value.project.ref)).toEqual(Option.some(VALID_REF)); + expect(accessed.some((entry) => entry.includes(".link.json"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + it.effect("keeps the link, its staged writes, and the stack lease out of version control", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + yield* link(flags()); + const ignored = (yield* readSupabaseFile(workdir, ".gitignore")).split("\n"); + expect(ignored).toContain(".link.json"); + expect(ignored).toContain("..link.json.*.tmp"); + expect(ignored).toContain(".supabase.lock.lease"); + expect(ignored).toContain(".supabase.lock.*.tmp"); + }).pipe(Effect.provide(layer)); + }); + + it.effect("leaves a symlinked supabase/.gitignore and its target untouched", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const outside = path.join(workdir, "outside-gitconfig"); + yield* fs.writeFileString(outside, "[user]\n\tname = kept\n"); + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + const gitignore = path.join(workdir, "supabase", ".gitignore"); + yield* fs.symlink(outside, gitignore); + + yield* link(flags()); + + expect(yield* fs.readFileString(outside)).toBe("[user]\n\tname = kept\n"); + expect(yield* fs.readLink(gitignore)).toBe(outside); + expect(Option.isSome(yield* readProjectLink(workdir))).toBe(true); + }).pipe(Effect.provide(layer)); + }); + + it.effect("fails on a busy project lease without replacing the existing link", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + const path = yield* Path.Path; + yield* seedLink(workdir, { + project: { ref: "previousprojectrefabc" }, + active_branch: { ref: "previousprojectrefabc" }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + yield* holdLock( + path.join(workdir, "supabase", ".supabase.lock.lease"), + "1 hour", + "supabase.lock", + ); + + // The lease wait runs on the test clock; the lock stays held, so the outcome is fixed. + const error = yield* link(flags()).pipe( + Effect.flip, + Effect.raceFirst( + TestClock.adjust("61 seconds").pipe(Effect.andThen(Effect.yieldNow), Effect.forever), + ), + ); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "lease-busy" }); + const document = yield* readProjectLink(workdir); + expect(Option.map(document, (value) => value.project.ref)).toEqual( + Option.some("previousprojectrefabc"), + ); + }).pipe(Effect.provide(layer)); + }); + + it.effect("omits the pooler URL with --skip-pooler", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + yield* link(flags({ skipPooler: true })); + const document = yield* readProjectLink(workdir); + expect(Option.map(document, (value) => value.active_branch)).toEqual( + Option.some({ ref: VALID_REF }), + ); + }).pipe(Effect.provide(layer)); + }); + + it.effect("records a name-resolved branch as the active branch of its parent", () => { + const { layer, workdir } = setup({ projectFails: true, branches: [FEATURE_BRANCH] }); + return Effect.gen(function* () { + yield* seedLink(workdir, { + project: { ref: VALID_REF, name: "My Project", organization_id: "org_123" }, + active_branch: { ref: VALID_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + yield* link(flags({ refOrBranch: Option.some("feature-branch"), projectRef: Option.none() })); + + const document = yield* readProjectLink(workdir); + expect(Option.getOrUndefined(document)).toEqual({ + project: { ref: VALID_REF, name: "My Project", organization_id: "org_123" }, + active_branch: { + ref: BRANCH_REF, + name: "feature-branch", + is_default: false, + pooler_url: POOLER_URL, + }, + fetchedAt: expect.any(String), + }); + }).pipe(Effect.provide(layer)); + }); + + it.effect("treats a raw branch ref unknown to the previous project as its own project", () => { + const { layer, workdir } = setup({ projectFails: true, branches: [] }); + return Effect.gen(function* () { + yield* seedLink(workdir, { + project: { ref: VALID_REF }, + active_branch: { ref: VALID_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + yield* link(flags({ projectRef: Option.some(BRANCH_REF) })); + + const document = yield* readProjectLink(workdir); + expect(Option.map(document, (value) => value.project.ref)).toEqual(Option.some(BRANCH_REF)); + expect(Option.map(document, (value) => value.active_branch.ref)).toEqual( + Option.some(BRANCH_REF), + ); + }).pipe(Effect.provide(layer)); + }); + + it.effect("keeps the previous project for a raw branch ref the project still lists", () => { + const { layer, workdir } = setup({ projectFails: true, branches: [FEATURE_BRANCH] }); + return Effect.gen(function* () { + yield* seedLink(workdir, { + project: { ref: VALID_REF }, + active_branch: { ref: VALID_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + yield* link(flags({ projectRef: Option.some(BRANCH_REF) })); + + const document = yield* readProjectLink(workdir); + expect(Option.map(document, (value) => value.project.ref)).toEqual(Option.some(VALID_REF)); + expect(Option.map(document, (value) => value.active_branch.ref)).toEqual( + Option.some(BRANCH_REF), + ); + }).pipe(Effect.provide(layer)); + }); + + it.effect("later commands resolve the linked ref from the active branch", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + yield* seedLink(workdir, { + project: { ref: VALID_REF }, + active_branch: { ref: BRANCH_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const resolver = yield* RemoteTarget; + + expect((yield* resolver.resolve(Option.none())).ref).toBe(BRANCH_REF); + }).pipe(Effect.provide(layer)); + }); + + it.effect("a malformed link document fails ref resolution with the relink hint", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(workdir, "supabase", ".link.json"), "{not json"); + const resolver = yield* RemoteTarget; + + const error = yield* resolver.resolve(Option.none()).pipe(Effect.flip); + + expect(error).toBeInstanceOf(ProjectLinkMalformedError); + expect(error).toHaveProperty("suggestion", expect.stringContaining("supabase link")); + }).pipe(Effect.provide(layer)); + }); + + it.effect("an unreadable link document fails ref resolution with an I/O error", () => { + const { layer, workdir } = setup(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(workdir, "supabase", ".link.json"), { recursive: true }); + const resolver = yield* RemoteTarget; + + const error = yield* resolver.resolve(Option.none()).pipe(Effect.flip); + + expect(error).toBeInstanceOf(ProjectLinkIoError); + }).pipe(Effect.provide(layer)); + }); +}); diff --git a/apps/cli/src/commands/migration/down/down.handler.ts b/apps/cli/src/commands/migration/down/down.handler.ts index 3a58e5abc5..73dab3be5f 100644 --- a/apps/cli/src/commands/migration/down/down.handler.ts +++ b/apps/cli/src/commands/migration/down/down.handler.ts @@ -8,7 +8,7 @@ import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../shared/output/errors.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua, bold, yellow } from "../../../command-internal/colors.ts"; import { loadProjectEnv, @@ -68,11 +68,15 @@ const runDown = Effect.fnUntraced(function* ( // Resolves before `--last` validation, so an unlinked/invalid target error // surfaces before the `--last must be greater than 0` error. + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + connType === "linked" ? yield* remoteTarget.load(flags.projectRef) : undefined; const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType, dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); // Loads after the flag-group check above, so a flag conflict surfaces before @@ -148,11 +152,9 @@ const runDown = Effect.fnUntraced(function* ( // Attached to the whole flow via `Effect.ensuring` so the cache write still // runs on the `--last`/cancel failure paths. - if (connType === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const linkedRef = yield* projectRef.loadProjectRef(flags.projectRef); - return yield* downFlow.pipe(Effect.ensuring(linkedProjectCache.cache(linkedRef))); + return yield* downFlow.pipe(Effect.ensuring(linkedProjectCache.cache(linkedTarget.ref))); } return yield* downFlow; }); diff --git a/apps/cli/src/commands/migration/down/down.integration.test.ts b/apps/cli/src/commands/migration/down/down.integration.test.ts index f0f14badda..85dac89957 100644 --- a/apps/cli/src/commands/migration/down/down.integration.test.ts +++ b/apps/cli/src/commands/migration/down/down.integration.test.ts @@ -36,7 +36,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag, YesFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; @@ -48,6 +47,7 @@ import { MigrationTargetFlagsError, OperationCanceledError } from "../migration. import { MigrationLastTooLargeError, MigrationLastZeroError } from "./down.errors.ts"; import { migrationDown } from "./down.handler.ts"; import type { MigrationDownFlags } from "./down.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LIST_SQL = "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; @@ -135,15 +135,15 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Gives an explicit --project-ref flag precedence over the VALID_REF fallback, so a // test can prove the flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF, ), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); const http = recordingProjectHttpClient(VALID_REF); diff --git a/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts b/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts index 5f4c809bab..ebec1025c3 100644 --- a/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts +++ b/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts @@ -33,7 +33,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag, YesFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigLoadError } from "../../../command-internal/db-config.errors.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; @@ -42,6 +41,7 @@ import { MigrationTargetFlagsError, OperationCanceledError } from "../migration. import { MigrationFetchWriteError } from "./fetch.errors.ts"; import { migrationFetch } from "./fetch.handler.ts"; import type { MigrationFetchFlags } from "./fetch.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const SELECT_SQL = "SELECT version, coalesce(name, '') as name, statements FROM supabase_migrations.schema_migrations"; @@ -115,15 +115,15 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Gives an explicit --project-ref flag precedence over the VALID_REF fallback, so a // test can prove the flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF, ), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); const http = recordingProjectHttpClient(VALID_REF); diff --git a/apps/cli/src/commands/migration/list/list.handler.ts b/apps/cli/src/commands/migration/list/list.handler.ts index 308e64ea7b..9822c122cd 100644 --- a/apps/cli/src/commands/migration/list/list.handler.ts +++ b/apps/cli/src/commands/migration/list/list.handler.ts @@ -4,7 +4,7 @@ import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { renderGlamourTable } from "../../../output/glamour-table.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; @@ -56,6 +56,12 @@ const runList = Effect.fnUntraced(function* ( }); } + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + (target.connType ?? "linked") === "linked" + ? yield* remoteTarget.load(flags.projectRef) + : undefined; + const listBody = Effect.gen(function* () { const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, @@ -63,6 +69,7 @@ const runList = Effect.fnUntraced(function* ( dnsResolver, password: flags.password, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); const remote = yield* Effect.scoped( @@ -99,11 +106,9 @@ const runList = Effect.fnUntraced(function* ( // `--linked` resolves the project ref and writes the linked-project cache; `--local` // / `--db-url` leave the ref empty. - if ((target.connType ?? "linked") === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const ref = yield* projectRef.loadProjectRef(flags.projectRef); - return yield* listBody.pipe(Effect.ensuring(linkedProjectCache.cache(ref))); + return yield* listBody.pipe(Effect.ensuring(linkedProjectCache.cache(linkedTarget.ref))); } return yield* listBody; }); diff --git a/apps/cli/src/commands/migration/list/list.integration.test.ts b/apps/cli/src/commands/migration/list/list.integration.test.ts index b92233b30d..f421827dd2 100644 --- a/apps/cli/src/commands/migration/list/list.integration.test.ts +++ b/apps/cli/src/commands/migration/list/list.integration.test.ts @@ -19,7 +19,7 @@ import { mockTelemetryStateTracked, useTempWorkdir, } from "../../../../tests/helpers/command-mocks.ts"; -import { mockOutput } from "../../../../tests/helpers/mocks.ts"; +import { mockOutput, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; import { asRecordedRequests, @@ -34,7 +34,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { MigrationsReadError } from "../../../command-internal/migration.errors.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; @@ -43,6 +42,9 @@ import { DbConnection } from "../../../command-internal/db-connection.service.ts import { MigrationPasswordFlagsError, MigrationTargetFlagsError } from "../migration.errors.ts"; import { migrationList } from "./list.handler.ts"; import type { MigrationListFlags } from "./list.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; +import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-factory.service.ts"; +import { remoteTargetLayer as realRemoteTargetLayer } from "../../../config/remote-target.layer.ts"; const LIST_SQL = "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; @@ -101,15 +103,15 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Gives an explicit --project-ref flag precedence over the VALID_REF fallback, so a // test can prove the flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF, ), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); const http = recordingProjectHttpClient(VALID_REF); @@ -398,4 +400,32 @@ describe("migration list", () => { provideEnv({ SUPABASE_ACCESS_TOKEN: VALID_TOKEN }), ); }); + + it.effect("reports an unlinked project before it reads an invalid config.toml", () => { + const ctx = setup(tmp.current, { realLayers: true }); + const unlinkedTarget = realRemoteTargetLayer.pipe( + Layer.provide( + Layer.mergeAll( + mockCommandSettings({ workdir: tmp.current, projectId: Option.none() }), + mockTty({ stdinIsTty: false }), + mockOutput().layer, + Layer.succeed(CommandPlatformApiFactory, { + make: Effect.die("the platform API is not used without a link"), + }), + BunServices.layer, + ), + ), + ); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(tmp.current, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(tmp.current, "supabase", "config.toml"), "= not toml\n"); + const error = yield* Effect.flip(migrationList(flags())); + expect(error._tag).toBe("ProjectRefNotLinkedError"); + }).pipe( + Effect.provide(Layer.mergeAll(ctx.layer, unlinkedTarget)), + provideEnv({ SUPABASE_ACCESS_TOKEN: VALID_TOKEN }), + ); + }); }); diff --git a/apps/cli/src/commands/migration/repair/repair.handler.ts b/apps/cli/src/commands/migration/repair/repair.handler.ts index 530168638f..7ae9e0cd12 100644 --- a/apps/cli/src/commands/migration/repair/repair.handler.ts +++ b/apps/cli/src/commands/migration/repair/repair.handler.ts @@ -9,7 +9,7 @@ import { emitSuccessTrailer } from "../../../shared/cli/success-trailer.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../shared/output/errors.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua } from "../../../command-internal/colors.ts"; import { loadProjectEnv } from "../../../command-internal/db-config.toml-read.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -146,12 +146,16 @@ const runRepair = Effect.fnUntraced(function* ( // Resolves the DB config (and, for the linked default, the project ref) before the // version parse and any prompt, so an invalid target surfaces first. + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + connType === "linked" ? yield* remoteTarget.load(input.projectRef) : undefined; const cfg = yield* resolver.resolve({ dbUrl: input.dbUrl, connType, dnsResolver, password: input.password, linkedProjectRef: input.projectRef, + target: linkedTarget, }); yield* Effect.annotateCurrentSpan({ "db.conn_type": connType, @@ -228,11 +232,9 @@ const runRepair = Effect.fnUntraced(function* ( // Attached to the whole flow via `Effect.ensuring` below so the cache write still // runs even when the version parse fails or the repair-all prompt is declined. - if (connType === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const ref = yield* projectRef.loadProjectRef(input.projectRef); - return yield* repairFlow.pipe(Effect.ensuring(linkedProjectCache.cache(ref))); + return yield* repairFlow.pipe(Effect.ensuring(linkedProjectCache.cache(linkedTarget.ref))); } return yield* repairFlow; }); diff --git a/apps/cli/src/commands/migration/repair/repair.integration.test.ts b/apps/cli/src/commands/migration/repair/repair.integration.test.ts index 389b025e68..4ce69a2da5 100644 --- a/apps/cli/src/commands/migration/repair/repair.integration.test.ts +++ b/apps/cli/src/commands/migration/repair/repair.integration.test.ts @@ -37,7 +37,6 @@ import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag, YesFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; import { DbExecError } from "../../../command-internal/db-connection.errors.ts"; @@ -50,6 +49,7 @@ import { } from "../migration.errors.ts"; import { MigrationRepairUpdateError } from "./repair.errors.ts"; import { migrationRepair, type MigrationRepairInput } from "./repair.handler.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; interface SetupOpts { /** Runs the real `DbConfigResolver` and linked-project cache against `workdir`. */ @@ -126,15 +126,15 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Gives an explicit --project-ref flag precedence over the VALID_REF fallback, so a // test can prove the flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF, ), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); const http = recordingProjectHttpClient(VALID_REF); diff --git a/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md index 6755704f8d..904c587020 100644 --- a/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md @@ -28,7 +28,7 @@ and uses an automatically assigned port; `db.shadow_port` applies only to the le | `/supabase/roles.sql` | SQL | shadow `SetupDatabase` (custom-roles seed); missing file tolerated | | managed snapshot key `` | backend | warm stack-shadow baseline; managed retention may evict entries and snapshots survive stack destruction | | `/supabase/.env`, `.env.local`, `SUPABASE_ENV`-selected dotenv | dotenv | always (`--yes`/registry/network-id overrides) | -| `/supabase/.temp/{project-ref,postgres-version,pooler-url}` | plain text | `--linked` / linked path — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `/supabase/.temp/{project-ref,postgres-version,pooler-url}` | plain text | `--linked` / linked path — the ref is not taken from `project-ref` when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set; `pooler-url` is used only when the target ref is the linked ref | | `~/.supabase/access-token` | plain text | `--linked` without `--password`/`SUPABASE_ACCESS_TOKEN` | | `~/.docker/config.json` + Docker context store | JSON | resolving the Docker hostname for shadow/pg_dump containers | diff --git a/apps/cli/src/commands/migration/squash/squash.handler.ts b/apps/cli/src/commands/migration/squash/squash.handler.ts index c003b6fac8..4d5d804e1c 100644 --- a/apps/cli/src/commands/migration/squash/squash.handler.ts +++ b/apps/cli/src/commands/migration/squash/squash.handler.ts @@ -13,7 +13,7 @@ import { import { Output } from "../../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget, type ResolvedRemoteTarget } from "../../../config/remote-target.service.ts"; import { aqua, bold } from "../../../command-internal/colors.ts"; import { buildLocalDbContainerInputs, @@ -137,7 +137,7 @@ const squashMigrations = Effect.fn("MigrationSquash.squashMigrations")(function* kind: "stack" as const, stack: handle.stack, command: "pg_dump" as const, - major: toml.majorVersion === 15 ? (15 as const) : (17 as const), + pin: handle.databasePin, }; const session = yield* connectShadowDatabase(stackConn); const before = yield* squashDumpSchemaToString({ @@ -499,9 +499,11 @@ const runSquash = Effect.fnUntraced(function* ( // resolver.resolve() below (like db diff --linked); read unconditionally since the // shadow is provisioned locally either way. let linkedRef: string | undefined; + let linkedTarget: ResolvedRemoteTarget | undefined; if (connType === "linked") { - const projectRefResolver = yield* ProjectRefResolver; - linkedRef = yield* projectRefResolver.loadProjectRef(flags.projectRef); + const remoteTarget = yield* RemoteTarget; + linkedTarget = yield* remoteTarget.load(flags.projectRef); + linkedRef = linkedTarget.ref; yield* Ref.set(linkedRefForCache, linkedRef); } const toml = yield* readDbToml(fs, path, cliSettings.workdir, linkedRef); @@ -529,6 +531,7 @@ const runSquash = Effect.fnUntraced(function* ( dnsResolver, password: flags.password, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); if (linkedRef === undefined) { linkedRef = Option.getOrUndefined(cfg.ref ?? Option.none()); diff --git a/apps/cli/src/commands/migration/squash/squash.integration.test.ts b/apps/cli/src/commands/migration/squash/squash.integration.test.ts index 9824ba4bba..60745fec4f 100644 --- a/apps/cli/src/commands/migration/squash/squash.integration.test.ts +++ b/apps/cli/src/commands/migration/squash/squash.integration.test.ts @@ -59,7 +59,6 @@ import { } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { INTERNAL_SCHEMAS } from "../../../command-internal/pg-dump.env.ts"; import { dumpSchemaScript } from "../../../command-internal/pg-dump.scripts.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -91,6 +90,7 @@ import { MigrationSquashWriteError, } from "./squash.errors.ts"; import { migrationSquash } from "./squash.handler.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; // Distinguishes squash's three pg_dump containers from the shadow's setup jobs by env: // only a pg_dump invocation carries PGDATABASE. Among dump calls, the first two sharing @@ -345,17 +345,17 @@ function setup(workdir: string, opts: SetupOpts = {}) { // Gives an explicit --project-ref flag precedence over the opts.linkedRef/VALID_REF // fallback, so a test can prove the flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(opts.linkedRef ?? VALID_REF), resolveForLink: () => Effect.succeed(opts.linkedRef ?? VALID_REF), resolveOptional: () => Effect.succeed(Option.some(opts.linkedRef ?? VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : (opts.linkedRef ?? VALID_REF), ), - promptProjectRef: () => Effect.succeed(opts.linkedRef ?? VALID_REF), + prompt: () => Effect.succeed(opts.linkedRef ?? VALID_REF), }); const debugLogs: Array = []; diff --git a/apps/cli/src/commands/migration/squash/squash.native.integration.test.ts b/apps/cli/src/commands/migration/squash/squash.native.integration.test.ts index 0049071ee0..a0e3830828 100644 --- a/apps/cli/src/commands/migration/squash/squash.native.integration.test.ts +++ b/apps/cli/src/commands/migration/squash/squash.native.integration.test.ts @@ -25,10 +25,10 @@ import { StackApi, stackApiLayer } from "../../../command-internal/stack-api.ts" import { stackBackendLayer } from "../../../command-internal/stack-backend.ts"; import { stackCatalogSetupLayer } from "../../../command-internal/stack-catalog-setup.ts"; import { BundledPostgresClient } from "../../../command-internal/bundled-postgres-client.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { migrationSquash } from "./squash.handler.ts"; import type { MigrationSquashFlags } from "./squash.command.ts"; import { destroyTestStack } from "../../../../tests/helpers/stack-cleanup.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const runtimes = ["native", "docker"] as const; const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); @@ -80,12 +80,12 @@ const testLayer = (root: string) => Layer.succeed(NetworkIdFlag, Option.none()), Layer.succeed(YesFlag, true), Layer.succeed(DebugLogger, { debug: () => Effect.void, http: () => Effect.void }), - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: () => Effect.succeed("abcdefghijklmnopqrst"), resolveForLink: () => Effect.succeed("abcdefghijklmnopqrst"), resolveOptional: () => Effect.succeed(Option.some("abcdefghijklmnopqrst")), - loadProjectRef: () => Effect.succeed("abcdefghijklmnopqrst"), - promptProjectRef: () => Effect.succeed("abcdefghijklmnopqrst"), + load: () => Effect.succeed("abcdefghijklmnopqrst"), + prompt: () => Effect.succeed("abcdefghijklmnopqrst"), }), mockStdin(false), Layer.succeed(BundledPostgresClient, { run: () => Effect.die("unused") }), diff --git a/apps/cli/src/commands/migration/up/up.handler.ts b/apps/cli/src/commands/migration/up/up.handler.ts index bfd54f5450..c34e4d3efb 100644 --- a/apps/cli/src/commands/migration/up/up.handler.ts +++ b/apps/cli/src/commands/migration/up/up.handler.ts @@ -4,7 +4,7 @@ import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { bold } from "../../../command-internal/colors.ts"; import { readDbToml, @@ -64,12 +64,19 @@ const runUp = Effect.fnUntraced(function* ( const migrationsDir = path.join(cliSettings.workdir, "supabase", "migrations"); + const remoteTarget = yield* RemoteTarget; + const linkedTarget = + (target.connType ?? "local") === "linked" + ? yield* remoteTarget.load(flags.projectRef) + : undefined; + const upBody = Effect.gen(function* () { const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType: target.connType ?? "local", dnsResolver, linkedProjectRef: flags.projectRef, + target: linkedTarget, }); const ref = Option.getOrUndefined(cfg.ref ?? Option.none()); const toml = yield* readDbToml(fs, path, cliSettings.workdir, ref); @@ -149,11 +156,9 @@ const runUp = Effect.fnUntraced(function* ( ); }); - if ((target.connType ?? "local") === "linked") { - const projectRef = yield* ProjectRefResolver; + if (linkedTarget !== undefined) { const linkedProjectCache = yield* LinkedProjectCache; - const linkedRef = yield* projectRef.loadProjectRef(flags.projectRef); - return yield* upBody.pipe(Effect.ensuring(linkedProjectCache.cache(linkedRef))); + return yield* upBody.pipe(Effect.ensuring(linkedProjectCache.cache(linkedTarget.ref))); } return yield* upBody; }); diff --git a/apps/cli/src/commands/migration/up/up.integration.test.ts b/apps/cli/src/commands/migration/up/up.integration.test.ts index 94811c3f5d..2e266fddea 100644 --- a/apps/cli/src/commands/migration/up/up.integration.test.ts +++ b/apps/cli/src/commands/migration/up/up.integration.test.ts @@ -35,7 +35,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; import { DbExecError } from "../../../command-internal/db-connection.errors.ts"; @@ -46,6 +45,7 @@ import { MigrationTargetFlagsError } from "../migration.errors.ts"; import { MigrationMissingLocalError, MigrationMissingRemoteError } from "./up.errors.ts"; import { migrationUp } from "./up.handler.ts"; import type { MigrationUpFlags } from "./up.command.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; const LIST_SQL = "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; const READ_VAULT = "SELECT id, name FROM vault.secrets WHERE name = ANY($1)"; @@ -119,15 +119,15 @@ function setup(workdir: string, opts: SetupOpts = {}) { // The flag value takes precedence over the VALID_REF fallback, so tests can assert the // flag drives the linked ref. - const projectRef = Layer.succeed(ProjectRefResolver, { + const projectRef = stubRemoteTarget({ resolve: () => Effect.succeed(VALID_REF), resolveForLink: () => Effect.succeed(VALID_REF), resolveOptional: () => Effect.succeed(Option.some(VALID_REF)), - loadProjectRef: (flagValue: Option.Option) => + load: (flagValue: Option.Option) => Effect.succeed( Option.isSome(flagValue) && flagValue.value.length > 0 ? flagValue.value : VALID_REF, ), - promptProjectRef: () => Effect.succeed(VALID_REF), + prompt: () => Effect.succeed(VALID_REF), }); const http = recordingProjectHttpClient(VALID_REF); diff --git a/apps/cli/src/commands/network-bans/get/get.command.ts b/apps/cli/src/commands/network-bans/get/get.command.ts index 9ca0064702..1417714948 100644 --- a/apps/cli/src/commands/network-bans/get/get.command.ts +++ b/apps/cli/src/commands/network-bans/get/get.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/network-bans/get/get.handler.ts b/apps/cli/src/commands/network-bans/get/get.handler.ts index 97e101fd99..9b3365c35a 100644 --- a/apps/cli/src/commands/network-bans/get/get.handler.ts +++ b/apps/cli/src/commands/network-bans/get/get.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -27,12 +27,12 @@ export const networkBansGet = Effect.fn("network-bans.get")(function* (flags: Ne const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/network-bans/remove/remove.command.ts b/apps/cli/src/commands/network-bans/remove/remove.command.ts index cce5d16728..3ce03882aa 100644 --- a/apps/cli/src/commands/network-bans/remove/remove.command.ts +++ b/apps/cli/src/commands/network-bans/remove/remove.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/network-bans/remove/remove.handler.ts b/apps/cli/src/commands/network-bans/remove/remove.handler.ts index 99fff0bf30..7a5dbf237d 100644 --- a/apps/cli/src/commands/network-bans/remove/remove.handler.ts +++ b/apps/cli/src/commands/network-bans/remove/remove.handler.ts @@ -2,7 +2,7 @@ import { isIP } from "node:net"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -28,13 +28,13 @@ export const networkBansRemove = Effect.fn("network-bans.remove")(function* ( const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { // Resolves before validating --db-unban-ip, so a bad ref surfaces before a bad IP. - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { for (const ip of flags.dbUnbanIp) { diff --git a/apps/cli/src/commands/network-restrictions/get/get.command.ts b/apps/cli/src/commands/network-restrictions/get/get.command.ts index 5e6201d8f4..646fd5f8f7 100644 --- a/apps/cli/src/commands/network-restrictions/get/get.command.ts +++ b/apps/cli/src/commands/network-restrictions/get/get.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/network-restrictions/get/get.handler.ts b/apps/cli/src/commands/network-restrictions/get/get.handler.ts index 5b207123df..732654f902 100644 --- a/apps/cli/src/commands/network-restrictions/get/get.handler.ts +++ b/apps/cli/src/commands/network-restrictions/get/get.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -35,12 +35,12 @@ export const networkRestrictionsGet = Effect.fn("network-restrictions.get")(func const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/network-restrictions/update/update.command.ts b/apps/cli/src/commands/network-restrictions/update/update.command.ts index f54ba55776..7f356d9ece 100644 --- a/apps/cli/src/commands/network-restrictions/update/update.command.ts +++ b/apps/cli/src/commands/network-restrictions/update/update.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/network-restrictions/update/update.handler.ts b/apps/cli/src/commands/network-restrictions/update/update.handler.ts index 441549b952..bf679451c8 100644 --- a/apps/cli/src/commands/network-restrictions/update/update.handler.ts +++ b/apps/cli/src/commands/network-restrictions/update/update.handler.ts @@ -5,7 +5,7 @@ import type { import { Effect, Option, Result } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -43,7 +43,7 @@ export const networkRestrictionsUpdate = Effect.fn("network-restrictions.update" const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -60,7 +60,7 @@ export const networkRestrictionsUpdate = Effect.fn("network-restrictions.update" } const { v4, v6 } = validation.success; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const updating = diff --git a/apps/cli/src/commands/notebooks/pull/pull.command.ts b/apps/cli/src/commands/notebooks/pull/pull.command.ts index e0b34a738c..f9c74789cd 100644 --- a/apps/cli/src/commands/notebooks/pull/pull.command.ts +++ b/apps/cli/src/commands/notebooks/pull/pull.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withContext } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/notebooks/pull/pull.handler.ts b/apps/cli/src/commands/notebooks/pull/pull.handler.ts index 16f1319dc7..e2f9903424 100644 --- a/apps/cli/src/commands/notebooks/pull/pull.handler.ts +++ b/apps/cli/src/commands/notebooks/pull/pull.handler.ts @@ -2,7 +2,7 @@ import { Effect, Option } from "effect"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { @@ -46,7 +46,7 @@ export const notebooksPull = Effect.fn("notebooks.pull")(function* (flags: Noteb const output = yield* Output; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -61,7 +61,7 @@ export const notebooksPull = Effect.fn("notebooks.pull")(function* (flags: Noteb yield* rejectNotebooksEnvOutput(); const machineOutput = yield* notebooksMachineOutputRequested(); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const pulled: Array = []; diff --git a/apps/cli/src/commands/notebooks/push/push.command.ts b/apps/cli/src/commands/notebooks/push/push.command.ts index e473a861f5..6d7eb38fd7 100644 --- a/apps/cli/src/commands/notebooks/push/push.command.ts +++ b/apps/cli/src/commands/notebooks/push/push.command.ts @@ -2,7 +2,7 @@ import { Argument, Command, Flag } from "effect/cli"; import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withContext } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/notebooks/push/push.handler.ts b/apps/cli/src/commands/notebooks/push/push.handler.ts index fcedd7acbe..406d75b111 100644 --- a/apps/cli/src/commands/notebooks/push/push.handler.ts +++ b/apps/cli/src/commands/notebooks/push/push.handler.ts @@ -2,7 +2,7 @@ import { Effect, Option } from "effect"; import { Output } from "../../../shared/output/output.service.ts"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { NotebookNotFoundError } from "../notebooks.errors.ts"; @@ -43,7 +43,7 @@ export const notebooksPush = Effect.fn("notebooks.push")(function* (flags: Noteb const output = yield* Output; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -58,7 +58,7 @@ export const notebooksPush = Effect.fn("notebooks.push")(function* (flags: Noteb yield* rejectNotebooksEnvOutput(); const machineOutput = yield* notebooksMachineOutputRequested(); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const local = yield* listLocalNotebooks(workdir); diff --git a/apps/cli/src/commands/postgres-config/delete/delete.command.ts b/apps/cli/src/commands/postgres-config/delete/delete.command.ts index d5defd7020..a068b8a294 100644 --- a/apps/cli/src/commands/postgres-config/delete/delete.command.ts +++ b/apps/cli/src/commands/postgres-config/delete/delete.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { postgresConfigRead, postgresConfigWrite } from "../postgres-config.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/postgres-config/delete/delete.handler.ts b/apps/cli/src/commands/postgres-config/delete/delete.handler.ts index aa4db6cd79..eb940c13ed 100644 --- a/apps/cli/src/commands/postgres-config/delete/delete.handler.ts +++ b/apps/cli/src/commands/postgres-config/delete/delete.handler.ts @@ -1,6 +1,6 @@ import { Effect } from "effect"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -21,12 +21,12 @@ export const postgresConfigDelete = Effect.fn("postgres-config.delete")(function flags: PostgresConfigDeleteFlags, ) { const output = yield* Output; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const deleting = diff --git a/apps/cli/src/commands/postgres-config/get/get.command.ts b/apps/cli/src/commands/postgres-config/get/get.command.ts index 7cabf46495..0b08a40bd5 100644 --- a/apps/cli/src/commands/postgres-config/get/get.command.ts +++ b/apps/cli/src/commands/postgres-config/get/get.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { postgresConfigRead } from "../postgres-config.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/postgres-config/get/get.handler.ts b/apps/cli/src/commands/postgres-config/get/get.handler.ts index db64d5f380..c5e4315b6b 100644 --- a/apps/cli/src/commands/postgres-config/get/get.handler.ts +++ b/apps/cli/src/commands/postgres-config/get/get.handler.ts @@ -1,6 +1,6 @@ import { Effect } from "effect"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -14,12 +14,12 @@ export const postgresConfigGet = Effect.fn("postgres-config.get")(function* ( flags: PostgresConfigGetFlags, ) { const output = yield* Output; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/postgres-config/update/update.command.ts b/apps/cli/src/commands/postgres-config/update/update.command.ts index b8fd3f1b59..29b4049a4f 100644 --- a/apps/cli/src/commands/postgres-config/update/update.command.ts +++ b/apps/cli/src/commands/postgres-config/update/update.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withCondition } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { postgresConfigRead, postgresConfigWrite } from "../postgres-config.shared.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/postgres-config/update/update.handler.ts b/apps/cli/src/commands/postgres-config/update/update.handler.ts index 42b0585b8c..e8f6c9b417 100644 --- a/apps/cli/src/commands/postgres-config/update/update.handler.ts +++ b/apps/cli/src/commands/postgres-config/update/update.handler.ts @@ -1,6 +1,6 @@ import { Effect } from "effect"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -24,7 +24,7 @@ export const postgresConfigUpdate = Effect.fn("postgres-config.update")(function flags: PostgresConfigUpdateFlags, ) { const output = yield* Output; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -38,7 +38,7 @@ export const postgresConfigUpdate = Effect.fn("postgres-config.update")(function nextOverrides[splits[0] ?? ""] = splits[1] ?? ""; } - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const updating = diff --git a/apps/cli/src/commands/projects/api-keys/api-keys.command.ts b/apps/cli/src/commands/projects/api-keys/api-keys.command.ts index 66a51081b6..b6fcfe96b3 100644 --- a/apps/cli/src/commands/projects/api-keys/api-keys.command.ts +++ b/apps/cli/src/commands/projects/api-keys/api-keys.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose, withCondition } from "../../../command-internal/command-permissions/model.ts"; import { projectApiKeys } from "../../../command-internal/get-api-keys.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/projects/api-keys/api-keys.handler.ts b/apps/cli/src/commands/projects/api-keys/api-keys.handler.ts index b5767a9814..f2bf076938 100644 --- a/apps/cli/src/commands/projects/api-keys/api-keys.handler.ts +++ b/apps/cli/src/commands/projects/api-keys/api-keys.handler.ts @@ -1,7 +1,7 @@ import type { V1GetProjectApiKeysOutput } from "@supabase/api/effect"; import { Effect, Option } from "effect"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -53,12 +53,12 @@ export const projectsApiKeys = Effect.fn("projects.api-keys")(function* ( ) { const output = yield* Output; const goOutputFlag = yield* OutputFlag; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // `--project-ref` resolution prompts on a TTY and fails when unlinked. - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/projects/delete/SIDE_EFFECTS.md b/apps/cli/src/commands/projects/delete/SIDE_EFFECTS.md index 9994a41dc7..1826fb46c4 100644 --- a/apps/cli/src/commands/projects/delete/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/projects/delete/SIDE_EFFECTS.md @@ -28,21 +28,23 @@ ## Environment Variables -| Variable | Purpose | Required? | -| ----------------------- | ---------------------------------------------------- | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| Variable | Purpose | Required? | +| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | +| `SUPABASE_EXPERIMENTAL_STACK` | `1` or `0` selects stack project-state mode over `[experimental] stack`; any other non-empty value fails before the project is deleted | no (unset or empty uses `[experimental] stack`) | ## Exit Codes -| Code | Condition | -| ---- | ---------------------------------------------------------- | -| `0` | success — project deleted | -| `1` | authentication error — no valid token found | -| `1` | project not found — 404 response from `/v1/projects/{ref}` | -| `1` | API error — non-2xx/404 response from `/v1/projects/{ref}` | -| `1` | network / connection failure | -| `1` | declined confirmation (cancellation) / no ref on a non-TTY | +| Code | Condition | +| ---- | -------------------------------------------------------------------------------------------------------- | +| `0` | success — project deleted | +| `1` | authentication error — no valid token found | +| `1` | project not found — 404 response from `/v1/projects/{ref}` | +| `1` | API error — non-2xx/404 response from `/v1/projects/{ref}` | +| `1` | network / connection failure | +| `1` | declined confirmation (cancellation) / no ref on a non-TTY | +| `1` | `SUPABASE_EXPERIMENTAL_STACK` is set to a non-empty value other than `0` or `1` (before the DELETE call) | ## Telemetry Events Fired @@ -86,6 +88,22 @@ On failure, an `error` event is emitted instead: {"type":"error","code":"ApiError","message":"…"} ``` +## Stack mode + +In stack project-state mode the best-effort cleanup reads `supabase/.link.json` and removes it +only when its `project.ref` or `active_branch.ref` equals the deleted ref; `supabase/.temp/` is +not consulted. The mode is resolved before the DELETE call, so an invalid +`SUPABASE_EXPERIMENTAL_STACK` fails before the project is deleted. + +When the link matches, the file is removed under the project lease `supabase/.supabase.lock.lease` +(the lease `supabase link` publishes `.link.json` under), after reading the link again and +re-checking the ref, so a concurrent `link` to another project keeps its link. A lease that stays +busy fails the command with `LockResolutionError` reason `lease-busy` and exit code `1` after the +project was deleted; the link document is left in place. The lease file is created, and +`supabase/.gitignore` gains the lease and staged-write entries (`.supabase.lock.lease`, +`.supabase.lock.*.tmp`), whenever the command takes the project lease, which is only when the link +matches. + ## Notes - In interactive mode (when stdin is a TTY and no ref is provided), the CLI prompts the diff --git a/apps/cli/src/commands/projects/delete/delete.handler.ts b/apps/cli/src/commands/projects/delete/delete.handler.ts index ff743b6d66..a9f66c38aa 100644 --- a/apps/cli/src/commands/projects/delete/delete.handler.ts +++ b/apps/cli/src/commands/projects/delete/delete.handler.ts @@ -1,5 +1,5 @@ import type { V1DeleteAProjectOutput } from "@supabase/api/effect"; -import { Effect, FileSystem, Option, Path, Ref } from "effect"; +import { Effect, Option, Ref } from "effect"; import * as HttpClientError from "effect/http/HttpClientError"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; @@ -7,11 +7,15 @@ import { CommandSettings } from "../../../config/command-settings.service.ts"; import { InvalidProjectRefError } from "../../../config/project-ref.errors.ts"; import { INVALID_PROJECT_REF_MESSAGE, - ProjectRefResolver, + RemoteTarget, PROJECT_REF_PATTERN, -} from "../../../config/project-ref.service.ts"; +} from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; +import { LegacyProjectState } from "../../../command-internal/legacy-project-state.service.ts"; +import { currentProjectStateMode } from "../../../command-internal/project-state-mode.ts"; +import { readProjectLink, removeProjectLink } from "../../../command-internal/project-link.ts"; +import { withProjectLease } from "../../../command-internal/stack-lock.ts"; import { resolveYes } from "../../../command-internal/global-flags.ts"; import { promptYesNo } from "../../../command-internal/prompt-yes-no.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../shared/output/errors.ts"; @@ -33,15 +37,14 @@ type DeletedProject = typeof V1DeleteAProjectOutput.Type; export const projectsDelete = Effect.fn("projects.delete")(function* (flags: ProjectsDeleteFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const cliSettings = yield* CommandSettings; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // `resolveYes` also honors `SUPABASE_YES`, not just the `--yes` flag. const yes = yield* resolveYes; const tty = yield* Tty; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; + const legacyState = yield* LegacyProjectState; // Captured so the `Effect.ensuring` cache write below can use whichever ref resolved from // the arg or prompt. @@ -54,7 +57,7 @@ export const projectsDelete = Effect.fn("projects.delete")(function* (flags: Pro if (Option.isSome(flags.ref) && flags.ref.value.length > 0) { ref = flags.ref.value; } else if (tty.stdinIsTty && output.interactive) { - ref = yield* resolver.promptProjectRef("Which project do you want to delete?"); + ref = yield* resolver.prompt("Which project do you want to delete?"); } else { return yield* new ProjectsDeleteRefRequiredError({ message: "accepts 1 arg(s), received 0", @@ -66,6 +69,8 @@ export const projectsDelete = Effect.fn("projects.delete")(function* (flags: Pro return yield* new InvalidProjectRefError({ ref, message: INVALID_PROJECT_REF_MESSAGE }); } + const stackState = (yield* currentProjectStateMode) === "stack"; + const title = `Do you want to delete project ${aqua(ref)}? This action is irreversible.`; // `promptYesNo` echoes ` [y/N] <answer>` to stderr even on a non-TTY: it scans one // piped line (100ms timeout) so `echo y | supabase projects delete <ref>` confirms; @@ -103,17 +108,36 @@ export const projectsDelete = Effect.fn("projects.delete")(function* (flags: Pro // No per-ref keyring credential delete: the access token is stored under the profile // name, not the ref, so there is nothing to remove here. - // Best-effort: removes `supabase/.temp` only when the linked ref matches the deleted - // project. - const tempDir = path.join(cliSettings.workdir, "supabase", ".temp"); - const refPath = path.join(tempDir, "project-ref"); + // Best-effort: removes `supabase/.temp` (`supabase/.link.json` in stack mode) only when the + // linked ref matches the deleted project. // Compare trimmed content so a corrupt/multi-ref file can't trigger an unintended removal. - const matches = yield* fs.readFileString(refPath).pipe( - Effect.map((content) => content.trim() === ref), - Effect.orElseSucceed(() => false), - ); - if (matches) { - yield* fs.remove(tempDir, { recursive: true }).pipe(Effect.ignore); + if (stackState) { + const linkedTo = readProjectLink(cliSettings.workdir).pipe( + Effect.map( + Option.filter((value) => value.project.ref === ref || value.active_branch.ref === ref), + ), + Effect.orElseSucceed(() => Option.none()), + ); + // A concurrent `link` publishes under the project lease, so the match is checked again + // under it before the file is removed. + if (Option.isSome(yield* linkedTo)) { + yield* withProjectLease( + cliSettings.workdir, + Effect.gen(function* () { + if (Option.isSome(yield* linkedTo)) { + yield* removeProjectLink(cliSettings.workdir).pipe(Effect.ignore); + } + }), + ); + } + } else { + const matches = yield* legacyState.read(cliSettings.workdir, "projectRef").pipe( + Effect.map((content) => content.trim() === ref), + Effect.orElseSucceed(() => false), + ); + if (matches) { + yield* legacyState.removeAll(cliSettings.workdir).pipe(Effect.ignore); + } } if (output.format === "json" || output.format === "stream-json") { diff --git a/apps/cli/src/commands/projects/delete/delete.integration.test.ts b/apps/cli/src/commands/projects/delete/delete.integration.test.ts index 2fce7458a4..290187509d 100644 --- a/apps/cli/src/commands/projects/delete/delete.integration.test.ts +++ b/apps/cli/src/commands/projects/delete/delete.integration.test.ts @@ -1,6 +1,8 @@ import type { V1ListAllProjectsOutput } from "@supabase/api/effect"; import { describe, expect, it } from "@effect/vitest"; +import { holdLock } from "@supabase/stack/effect"; import { Cause, Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; +import { TestClock } from "effect/testing"; import { mockOutput, mockStdin, mockTty } from "../../../../tests/helpers/mocks.ts"; import { @@ -19,6 +21,8 @@ import { import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; import { realLinkedProjectCache } from "../../../../tests/helpers/permission-real-layers.ts"; import { YesFlag } from "../../../command-internal/global-flags.ts"; +import { writeProjectLink } from "../../../command-internal/project-link.ts"; +import { projectStateModeLayer } from "../../../command-internal/project-state-mode.ts"; import { projectsDelete } from "./delete.handler.ts"; const OTHER_REF = "qrstuvwxyzabcdefghij"; @@ -55,6 +59,7 @@ interface SetupOpts { readonly network?: "fail"; readonly promptConfirmResponses?: ReadonlyArray<boolean>; readonly promptSelectResponses?: ReadonlyArray<string>; + readonly stack?: boolean; } function setup(opts: SetupOpts = {}) { @@ -88,6 +93,7 @@ function setup(opts: SetupOpts = {}) { linkedProjectCache: cache.layer, }), Layer.succeed(YesFlag, opts.yes ?? false), + ...(opts.stack === true ? [projectStateModeLayer("stack")] : []), ); return { layer, out, api, telemetry, cache }; } @@ -330,6 +336,83 @@ describe("projects delete integration", () => { }).pipe(Effect.provide(layer)); }); + describe("in stack mode", () => { + const writeLinkDocument = (projectRef: string, activeRef = "branchprojectrefabcd") => + writeProjectLink(tempRoot.current, { + project: { ref: projectRef }, + active_branch: { ref: activeRef }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const linkDocumentExists = Effect.fnUntraced(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* fs.exists(path.join(tempRoot.current, "supabase", ".link.json")); + }); + + it.effect("removes the link document when the deleted ref is the linked project", () => { + const { layer } = setup({ yes: true, stack: true }); + return Effect.gen(function* () { + yield* writeLinkDocument(VALID_REF); + yield* projectsDelete({ ref: Option.some(VALID_REF) }); + expect(yield* linkDocumentExists()).toBe(false); + }).pipe(Effect.provide(layer)); + }); + + it.effect("removes the link document when the deleted ref is the linked branch", () => { + const { layer } = setup({ yes: true, stack: true }); + return Effect.gen(function* () { + yield* writeLinkDocument(OTHER_REF, VALID_REF); + yield* projectsDelete({ ref: Option.some(VALID_REF) }); + expect(yield* linkDocumentExists()).toBe(false); + }).pipe(Effect.provide(layer)); + }); + + it.effect("fails on a busy project lease and keeps the link document", () => { + const { layer } = setup({ yes: true, stack: true }); + return Effect.gen(function* () { + yield* writeLinkDocument(VALID_REF); + yield* holdLock( + `${tempRoot.current}/supabase/.supabase.lock.lease`, + "1 hour", + "supabase.lock", + ); + // The lease wait runs on the test clock; the lock stays held, so the outcome is fixed. + yield* TestClock.adjust("1 second").pipe( + Effect.andThen(Effect.yieldNow), + Effect.forever, + Effect.forkScoped, + ); + + const error = yield* projectsDelete({ ref: Option.some(VALID_REF) }).pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "lease-busy" }); + expect(yield* linkDocumentExists()).toBe(true); + }).pipe(Effect.provide(layer)); + }); + + it.effect("deletes the project without creating supabase/ when the project has none", () => { + const { layer } = setup({ yes: true, stack: true }); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + yield* projectsDelete({ ref: Option.some(VALID_REF) }); + + expect(yield* fs.exists(path.join(tempRoot.current, "supabase"))).toBe(false); + }).pipe(Effect.provide(layer)); + }); + + it.effect("keeps the link document when another project is deleted", () => { + const { layer } = setup({ yes: true, stack: true }); + return Effect.gen(function* () { + yield* writeLinkDocument(OTHER_REF); + yield* projectsDelete({ ref: Option.some(VALID_REF) }); + expect(yield* linkDocumentExists()).toBe(true); + }).pipe(Effect.provide(layer)); + }); + }); + it.effect("maps HTTP 404 to project-does-not-exist", () => { const { layer } = setup({ yes: true, byMethod: { DELETE: { status: 404, body: {} } } }); return Effect.gen(function* () { diff --git a/apps/cli/src/commands/projects/list/list.handler.ts b/apps/cli/src/commands/projects/list/list.handler.ts index b85ff1f614..d563500763 100644 --- a/apps/cli/src/commands/projects/list/list.handler.ts +++ b/apps/cli/src/commands/projects/list/list.handler.ts @@ -2,13 +2,12 @@ import { operationDefinitions } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; -import { resolveLinkedParentRef } from "../../../command-internal/parent-project-ref.ts"; import { type GoType, encodeGoToml, @@ -66,12 +65,15 @@ export const projectsList = Effect.fn("projects.list")(function* (_flags: Projec const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const remoteTarget = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // Loaded purely as a marker for the "linked" column; `resolveOptional` never fails or prompts. - const linkedRef = yield* resolver.resolveOptional(Option.none()); + const linkedRef = Option.map( + yield* remoteTarget.resolveOptional(Option.none()), + (target) => target.ref, + ); yield* Effect.gen(function* () { const fetching = @@ -126,14 +128,14 @@ export const projectsList = Effect.fn("projects.list")(function* (_flags: Projec // `markerRef` decides which row shows as linked, since a linked branch's own ref never // matches a project row here. An exact match on `linkedRef` wins outright; only when it - // misses do we fall back to the parent chain (env → linked-project.json → project-ref file). + // misses do we fall back to the parent chain (env, then the recorded parent, then the linked ref). let markerRef = linkedRef; if (Option.isSome(linkedRef)) { const hasExactMatch = parsed.some( (project) => readProjectField(project, "id") === linkedRef.value, ); if (!hasExactMatch) { - const parent = yield* resolveLinkedParentRef(); + const parent = yield* remoteTarget.parent; markerRef = parent.kind === "resolved" ? Option.some(parent.ref) : Option.none(); } } diff --git a/apps/cli/src/commands/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/pull/SIDE_EFFECTS.md index 9c5c4338be..3cc2cef0e0 100644 --- a/apps/cli/src/commands/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/pull/SIDE_EFFECTS.md @@ -36,7 +36,7 @@ Each executed sub-step also performs its own file reads exactly as documented in threads the already-resolved `ref` into each step's existing `--project-ref`-shaped input rather than having each step re-resolve it (ADR 0024's "resolve once" decision) — most of a sub-step's own target-resolution file reads are therefore skipped in practice (the ref is already ref-shaped, so -`ProjectRefResolver.loadProjectRef` short-circuits with no file/network access). +`RemoteTarget.load` short-circuits with no file/network access). ## Files Written diff --git a/apps/cli/src/commands/pull/pull.command.ts b/apps/cli/src/commands/pull/pull.command.ts index d7e02b6862..422937d3ad 100644 --- a/apps/cli/src/commands/pull/pull.command.ts +++ b/apps/cli/src/commands/pull/pull.command.ts @@ -10,7 +10,7 @@ import { dbConfigPoolerFallbackLinked, } from "../../command-internal/db-config.layer.ts"; import { configTarget } from "../../command-internal/project-target.ts"; -import { PROJECT_REF_PATTERN } from "../../config/project-ref.service.ts"; +import { PROJECT_REF_PATTERN } from "../../config/remote-target.service.ts"; import { functionsDownload } from "../../shared/functions/download.ts"; import { linkedProjectCache } from "../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/pull/pull.layers.ts b/apps/cli/src/commands/pull/pull.layers.ts index 06ef3a1543..2cc71f41a1 100644 --- a/apps/cli/src/commands/pull/pull.layers.ts +++ b/apps/cli/src/commands/pull/pull.layers.ts @@ -4,7 +4,7 @@ import type * as HttpClient from "effect/http/HttpClient"; import { CommandCredentials } from "../../auth/command-credentials.service.ts"; import { CommandPlatformApi } from "../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../config/remote-target.service.ts"; import { DbConfigResolver } from "../../command-internal/db-config.service.ts"; import { DbConnection } from "../../command-internal/db-connection.service.ts"; import { DebugLogger } from "../../command-internal/debug-logger.service.ts"; @@ -52,7 +52,7 @@ type PullServices = | HttpClient.HttpClient | CommandCredentials | CommandSettings - | ProjectRefResolver + | RemoteTarget | LinkedProjectCache | TelemetryState | CommandRuntime diff --git a/apps/cli/src/commands/pull/pull.steps.ts b/apps/cli/src/commands/pull/pull.steps.ts index 842bb7d62a..2de47504f3 100644 --- a/apps/cli/src/commands/pull/pull.steps.ts +++ b/apps/cli/src/commands/pull/pull.steps.ts @@ -1,4 +1,4 @@ -import { Effect, Option, Path, Stdio } from "effect"; +import { Effect, Option, Stdio } from "effect"; import { CommandPlatformApi } from "../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; @@ -126,12 +126,9 @@ export const pullDbStep = Effect.fnUntraced(function* (context: PullStepContext) export const pullFunctionsStep = Effect.fnUntraced(function* (context: PullStepContext) { const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const path = yield* Path.Path; const stdio = yield* Stdio.Stdio; const rawArgs = yield* stdio.args; - const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin( - path.join(cliSettings.workdir, "supabase"), - ); + const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersionPin(cliSettings.workdir); const result: DownloadFunctionsResult = yield* downloadFunctions( { diff --git a/apps/cli/src/commands/secrets/list/list.command.ts b/apps/cli/src/commands/secrets/list/list.command.ts index 26224a044c..4f4c30b22c 100644 --- a/apps/cli/src/commands/secrets/list/list.command.ts +++ b/apps/cli/src/commands/secrets/list/list.command.ts @@ -3,7 +3,7 @@ import { Command, Flag } from "effect/cli"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/secrets/list/list.handler.ts b/apps/cli/src/commands/secrets/list/list.handler.ts index c148a71acc..79810ba7ae 100644 --- a/apps/cli/src/commands/secrets/list/list.handler.ts +++ b/apps/cli/src/commands/secrets/list/list.handler.ts @@ -2,7 +2,7 @@ import type { V1ListAllSecretsOutput } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -54,11 +54,11 @@ export const secretsList = Effect.fn("secrets.list")(function* (flags: SecretsLi const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/secrets/set/set.command.ts b/apps/cli/src/commands/secrets/set/set.command.ts index 144f06953d..74d28ed719 100644 --- a/apps/cli/src/commands/secrets/set/set.command.ts +++ b/apps/cli/src/commands/secrets/set/set.command.ts @@ -3,7 +3,7 @@ import { Argument, Command, Flag } from "effect/cli"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/secrets/set/set.handler.ts b/apps/cli/src/commands/secrets/set/set.handler.ts index 2c1d0449ec..934a7176eb 100644 --- a/apps/cli/src/commands/secrets/set/set.handler.ts +++ b/apps/cli/src/commands/secrets/set/set.handler.ts @@ -10,7 +10,7 @@ import { parse as parseDotenv } from "dotenv"; import { Effect, FileSystem, Option, Path, Predicate, Redacted, Result, Schema } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { DebugLogger } from "../../../command-internal/debug-logger.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -86,7 +86,7 @@ function filterDecodableSecrets(secrets: Record<string, unknown>): Record<string export const secretsSet = Effect.fn("secrets.set")(function* (flags: SecretsSetFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const debugLogger = yield* DebugLogger; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -94,7 +94,7 @@ export const secretsSet = Effect.fn("secrets.set")(function* (flags: SecretsSetF const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { // Source 1: `[edge_runtime.secrets]` from `supabase/config.toml`. Only resolved values are diff --git a/apps/cli/src/commands/secrets/unset/unset.command.ts b/apps/cli/src/commands/secrets/unset/unset.command.ts index 5b07cf19fc..7cd8911567 100644 --- a/apps/cli/src/commands/secrets/unset/unset.command.ts +++ b/apps/cli/src/commands/secrets/unset/unset.command.ts @@ -4,7 +4,7 @@ import { Argument, Command, Flag } from "effect/cli"; import { Layer } from "effect"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { stdinLayer } from "../../../shared/runtime/stdin.layer.ts"; diff --git a/apps/cli/src/commands/secrets/unset/unset.handler.ts b/apps/cli/src/commands/secrets/unset/unset.handler.ts index ccc1f462ac..6aff86b877 100644 --- a/apps/cli/src/commands/secrets/unset/unset.handler.ts +++ b/apps/cli/src/commands/secrets/unset/unset.handler.ts @@ -2,7 +2,7 @@ import type { V1ListAllSecretsOutput } from "@supabase/api/effect"; import { Effect } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { resolveYes } from "../../../command-internal/global-flags.ts"; @@ -40,13 +40,13 @@ const mapUnsetError = mapHttpError({ export const secretsUnset = Effect.fn("secrets.unset")(function* (flags: SecretsUnsetFlags) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; // `resolveYes` also honors `SUPABASE_YES`, not just the `--yes` flag. const yes = yield* resolveYes; - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { let names: ReadonlyArray<string> = flags.names; diff --git a/apps/cli/src/commands/seed/buckets/buckets.handler.ts b/apps/cli/src/commands/seed/buckets/buckets.handler.ts index 49b0eeb8f5..c1cd58a909 100644 --- a/apps/cli/src/commands/seed/buckets/buckets.handler.ts +++ b/apps/cli/src/commands/seed/buckets/buckets.handler.ts @@ -2,7 +2,7 @@ import { Effect, FileSystem, Option, Ref } from "effect"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { changedLinkedLocalFlags } from "../../../command-internal/db-target-flags.ts"; import { seedBucketsRun } from "../../../command-internal/seed-buckets.ts"; import { requireExplicitWorkdirProject } from "../../../command-internal/workdir-project.ts"; @@ -68,8 +68,8 @@ export const seedBuckets = Effect.fn("seed.buckets")(function* ( Effect.mapError((error) => new SeedMissingProjectConfigError({ message: error.message })), ); - const projectRefResolver = yield* ProjectRefResolver; - const projectRef = isLinked ? yield* projectRefResolver.loadProjectRef(flags.projectRef) : ""; + const projectRefResolver = yield* RemoteTarget; + const projectRef = isLinked ? (yield* projectRefResolver.load(flags.projectRef)).ref : ""; yield* Ref.set(linkedRef, projectRef); yield* Effect.annotateCurrentSpan({ "db.conn_type": isLinked ? "linked" : "local" }); diff --git a/apps/cli/src/commands/seed/buckets/buckets.integration.test.ts b/apps/cli/src/commands/seed/buckets/buckets.integration.test.ts index c11e2ad493..6cf44c990f 100644 --- a/apps/cli/src/commands/seed/buckets/buckets.integration.test.ts +++ b/apps/cli/src/commands/seed/buckets/buckets.integration.test.ts @@ -33,7 +33,6 @@ import { import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { YesFlag } from "../../../command-internal/global-flags.ts"; import type { OutputFormat } from "../../../shared/output/types.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; import { ProjectRefNotLinkedError } from "../../../config/project-ref.errors.ts"; import { generateGoJwt } from "../../../command-internal/go-jwt.ts"; import { seedBucketsRun } from "../../../command-internal/seed-buckets.ts"; @@ -53,11 +52,17 @@ import { type SetupStorageStackApiOptions, } from "../../../../tests/helpers/storage.ts"; import { unusedStackServices } from "../../../../tests/helpers/unused-stack.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../../tests/helpers/legacy-state-guard.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; import { projectResponse, realLinkedProjectCache, } from "../../../../tests/helpers/permission-real-layers.ts"; +import { stubRemoteTarget } from "../../../../tests/helpers/remote-target.ts"; interface MockRoute { readonly method: string; @@ -92,7 +97,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( readonly pipedAnswers?: ReadonlyArray<string>; readonly args?: ReadonlyArray<string>; readonly yes?: boolean; - /** Project ref returned by loadProjectRef for --linked tests. */ + /** Project ref returned by RemoteTarget.load for --linked tests. */ readonly projectRef?: string; /** API keys response for Management API mock. */ readonly apiKeys?: ReadonlyArray<{ @@ -101,7 +106,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( type?: string | null; secret_jwt_template?: Record<string, unknown> | null; }>; - /** When true, loadProjectRef fails with ProjectRefNotLinkedError. */ + /** When true, RemoteTarget.load fails with ProjectRefNotLinkedError. */ readonly linkedFails?: boolean; /** When set, the Management API `getProjectApiKeys` call fails with this error. */ readonly apiKeysFail?: HttpClientError.HttpClientError; @@ -176,7 +181,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( const linkedCache = mockLinkedProjectCacheTracked(); const projectRefRef = opts.projectRef ?? VALID_REF; - const projectRefLayer = Layer.succeed(ProjectRefResolver, { + const remoteTargetLayer = stubRemoteTarget({ resolve: () => opts.linkedFails === true ? Effect.fail( @@ -197,7 +202,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( // An explicit `--project-ref` flag takes precedence and short-circuits // before `linkedFails`, so a test can prove the flag resolves a ref even // for an "unlinked" workdir. - loadProjectRef: (flagValue: Option.Option<string>) => + load: (flagValue: Option.Option<string>) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true @@ -207,7 +212,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( }), ) : Effect.succeed(projectRefRef), - promptProjectRef: () => Effect.succeed(projectRefRef), + prompt: () => Effect.succeed(projectRefRef), }); const defaultApiKeys = [ @@ -243,6 +248,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( : undefined; const stackApi = buildStorageStackApi(workdir, opts.stackApi); + const recorder = recordingFileSystem(); const layer = Layer.mergeAll( out.layer, @@ -251,13 +257,14 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( ConfigProvider.layer(ConfigProvider.fromEnvRecord(process.env, { preserveEmptyStrings: true })), mockCommandSettings({ workdir, explicitWorkdir: opts.explicitWorkdir ?? false }), BunServices.layer, + recorder.layer, runtimeInfoLayer, // Seed-bucket prompts model an interactive user answering via `confirm`. mockTty({ stdinIsTty: true, stdoutIsTty: false }), mockStdin(true, opts.pipedAnswers ? `${opts.pipedAnswers.join("\n")}\n` : undefined), Layer.succeed(CliArgs, { args: opts.args ?? ["seed", "buckets"] }), Layer.succeed(YesFlag, opts.yes ?? false), - projectRefLayer, + remoteTargetLayer, recorded === undefined ? Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(managementApi.layer)), @@ -281,6 +288,7 @@ const setupSeedBuckets = Effect.fnUntraced(function* ( linkedCache, stackCalls: stackApi.stackCalls, managementRequests: recorded?.requests ?? [], + accessed: recorder.accessed, }; }); @@ -3055,6 +3063,27 @@ describe("stack backend", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.effect("seeds buckets without reading or writing supabase/.temp", () => + Effect.gen(function* () { + const { layer, accessed } = yield* setupSeedBuckets(tmp.current, { + toml: '[storage.buckets.images]\npublic = true\nobjects_path = "./assets"\n', + files: { "supabase/assets/a.txt": "hello" }, + stackBackend: true, + routes: [ + { method: "GET", match: "/storage/v1/bucket", body: [] }, + { method: "POST", match: "/storage/v1/bucket", body: { name: "images" } }, + { method: "POST", match: "/storage/v1/object/", body: {} }, + ], + }); + seedLegacyStateDecoys(tmp.current); + + yield* seedBuckets(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + + expect(accessed.some((entry) => entry.endsWith("a.txt"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect( "short-circuits with the empty summary despite a malformed SUPABASE_API_PORT, since the stack backend never reads it", () => diff --git a/apps/cli/src/commands/services/SIDE_EFFECTS.md b/apps/cli/src/commands/services/SIDE_EFFECTS.md index 1588e49944..80f9b8eb7a 100644 --- a/apps/cli/src/commands/services/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/services/SIDE_EFFECTS.md @@ -88,9 +88,9 @@ TS-only NDJSON success event with the same `{ services: [...] }` payload. ## Notes - Backend selection follows canonical experimental-feature routing: `SUPABASE_EXPERIMENTAL_STACK=1|0` takes precedence over `experimental.stack`; unset or empty uses project config, and an invalid environment value fails. Output fields and serializers stay the same. -- The legacy backend uses its baked-in service matrix and honors its existing config/version overrides. The stack backend lists services from the installed CLI artifact catalog using canonical `ghcr.io/supabase/cli/...` image names and catalog versions, reported as the upstream version without a slim revision suffix (`-rN`). Native and Docker runtimes use the same catalog versions. Since the catalog belongs to the installed CLI, an older launched CLI or a newer/mirrored stack image may differ from this inventory. The command does not inspect running containers, image pulls, service health, or live stack state. -- For stack mode, PostgreSQL uses the configured major version or `SUPABASE_DB_MAJOR_VERSION` (15 or 17). Invalid configuration or an unsupported PostgreSQL major warns with the cause and falls back to default catalog versions; absent config uses defaults. Legacy image pins, slim-image rewriting, and remote image overrides do not affect stack results. +- The legacy backend uses its baked-in service matrix and honors its existing config/version overrides. The stack backend lists services using canonical `ghcr.io/supabase/cli/...` image names and the versions pinned in `supabase/supabase.lock` (read without the lease or any resolution), falling back to the installed CLI artifact catalog when the project has no lock, reported as the upstream version without a slim revision suffix (`-rN`). Native and Docker runtimes use the same versions. Since the lock and catalog belong to the project and the installed CLI, an older launched CLI or a newer/mirrored stack image may differ from this inventory. The command does not inspect running containers, image pulls, service health, or live stack state. +- For stack mode, PostgreSQL uses the configured major version or `SUPABASE_DB_MAJOR_VERSION` (15 or 17). Invalid configuration, an unsupported PostgreSQL major, or a stale or unreadable lock warns with the cause and falls back to default catalog versions; absent config uses defaults. Legacy image pins, slim-image rewriting, and remote image overrides do not affect stack results. - Linked-version checks are best-effort. Remote lookup failures do not change the exit code; they only leave the `LINKED` column empty for unavailable services. - A malformed linked ref is the one lookup failure that prints an explicit stderr warning (see API Routes above); every other remote failure (network error, expired token, etc.) still fails silently and just leaves `LINKED` empty. Most real-world malformed refs come from an untrimmed `SUPABASE_PROJECT_ID` env var (e.g. a trailing newline from a secrets manager or `.env` file) rather than actual file tampering — the env var is read raw and unlike the on-disk `project-ref` file is never trimmed. -- Version mismatches are reported to stderr as a warning. In stack mode, the warning explains that catalog versions cannot be changed with `supabase link`. +- Version mismatches are reported to stderr as a warning. In stack mode, the warning is headed "The stack service versions differ from your linked project:" and recommends running `supabase link` to update services set to `linked` (the default) in `supabase/supabase.lock`; services pinned in `[local.services]` keep their version. - `telemetry.json` is written on every invocation, including `--output env` failures. diff --git a/apps/cli/src/commands/services/services-local-stack.ts b/apps/cli/src/commands/services/services-local-stack.ts index 902fd7ec94..a2dfc007fd 100644 --- a/apps/cli/src/commands/services/services-local-stack.ts +++ b/apps/cli/src/commands/services/services-local-stack.ts @@ -1,16 +1,14 @@ -import { - artifactServiceKinds, - postgresVersion, - resolveArtifact, -} from "@supabase/stack/internal/artifacts"; +import { artifactServiceKinds, type ArtifactSelection } from "@supabase/stack/internal/artifacts"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; import { Effect, Result } from "effect"; import { loadLocalProjectContext } from "../../command-internal/local-project-context.ts"; import { envOverrideMajorVersion, envOverrideOrioleDBVersion, } from "../../command-internal/local-config-values.ts"; -import { stackDatabaseVersion } from "../../command-internal/stack-database-version.ts"; -import { upstreamVersionFromTag } from "../../shared/services/services.shared.ts"; +import { stackArtifactSelection } from "../../command-internal/stack-artifact-selection.ts"; +import { StackConfigError } from "../../command-internal/stack-config.ts"; +import { readSelectionOrBundled } from "../../command-internal/stack-lock.ts"; import type { ServiceVersionRow } from "../../shared/services/services.shared.ts"; import type { RemoteServiceName } from "../../shared/services/services.shared.ts"; @@ -27,7 +25,7 @@ export const stackServiceVersions = Effect.fn("services.stackServiceVersions")(f ) { const context = yield* loadLocalProjectContext(workdir, (message) => message).pipe(Effect.result); let configError: string | undefined; - let databaseVersion: string | undefined; + let selection: ArtifactSelection = defaultArtifactSelection(); if (Result.isFailure(context)) configError = context.failure; else { const resolved = yield* Effect.try({ @@ -38,36 +36,48 @@ export const stackServiceVersions = Effect.fn("services.stackServiceVersions")(f ); if (value !== 15 && value !== 17) throw new Error(`unsupported PostgreSQL major version: ${value}`); - return stackDatabaseVersion({ + const db = { major_version: value, orioledb_version: envOverrideOrioleDBVersion( context.success.config.db.orioledb_version, context.success.projectEnvValues, ), - }); + }; + // A listing never resolves: it shows the locked pins, else the bundled ones. + return readSelectionOrBundled({ + projectRoot: workdir, + majorVersion: db.major_version, + orioledbVersion: db.orioledb_version, + services: context.success.config.local?.services, + }).pipe( + Effect.catchTag("LockResolutionError", (error) => + Effect.sync(() => { + configError = error.message; + }).pipe( + Effect.andThen(stackArtifactSelection(db)), + Effect.mapError((cause) => new StackConfigError({ message: cause.message })), + ), + ), + ); }, catch: (cause) => (cause instanceof Error ? cause.message : String(cause)), - }).pipe(Effect.flatMap(Effect.fromResult), Effect.result); + }).pipe( + Effect.flatMap((selected) => Effect.mapError(selected, (error) => error.message)), + Effect.result, + ); if (Result.isFailure(resolved)) configError = resolved.failure; - else databaseVersion = resolved.success; + else selection = resolved.success; } yield* Effect.annotateCurrentSpan({ "config.load_failed": configError !== undefined }); - return yield* Effect.forEach(artifactServiceKinds(), (service) => - Effect.gen(function* () { - const artifact = yield* resolveArtifact({ - service, - ...(service === "database" && databaseVersion !== undefined - ? { version: postgresVersion(databaseVersion) } - : {}), - }); - const name = artifact.image.split("@")[0]?.replace(/:[^/:]+$/, "") ?? artifact.image; - const remoteName = remoteNames[service]; - return { - name, - // A slim revision suffix (`-rN`) repackages the same upstream release. - local: upstreamVersionFromTag(artifact.version), - remote: remoteName === undefined ? "" : (remote[remoteName] ?? ""), - } satisfies ServiceVersionRow; - }), - ).pipe(Effect.map((rows) => ({ rows, configError }))); + const rows = artifactServiceKinds().map((service) => { + const pin = selection[service]; + const name = pin.image.split("@")[0]?.replace(/:[^/:]+$/, "") ?? pin.image; + const remoteName = remoteNames[service]; + return { + name, + local: pin.upstreamVersion, + remote: remoteName === undefined ? "" : (remote[remoteName] ?? ""), + } satisfies ServiceVersionRow; + }); + return { rows, configError }; }); diff --git a/apps/cli/src/commands/services/services.handler.ts b/apps/cli/src/commands/services/services.handler.ts index 648fb10b5a..52608ba28e 100644 --- a/apps/cli/src/commands/services/services.handler.ts +++ b/apps/cli/src/commands/services/services.handler.ts @@ -1,10 +1,11 @@ +import { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; import { Effect, FileSystem, Option, Path } from "effect"; import { CommandSettings } from "../../config/command-settings.service.ts"; import { CommandCredentials } from "../../auth/command-credentials.service.ts"; import { INVALID_PROJECT_REF_MESSAGE, PROJECT_REF_PATTERN, -} from "../../config/project-ref.service.ts"; +} from "../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../telemetry/telemetry-state.service.ts"; import { @@ -14,6 +15,8 @@ import { import { resolveDbImage } from "../../command-internal/db-image.ts"; import { resolveEdgeRuntimeImage } from "../../command-internal/edge-runtime-image.ts"; import { readServiceVersionOverrides } from "../../command-internal/service-version-overrides.ts"; +import { currentProjectStateMode } from "../../command-internal/project-state-mode.ts"; +import { readLinkSnapshot } from "../../command-internal/link-snapshot.ts"; import { currentStackBackend } from "../../command-internal/stack-backend.ts"; import { OutputFlag } from "../../command-internal/global-flags.ts"; import { Output } from "../../shared/output/output.service.ts"; @@ -64,14 +67,27 @@ export const services = Effect.fn("services")(function* (_flags: ServicesFlags) const telemetryState = yield* TelemetryState; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; + const legacyState = yield* LegacyProjectState; - const projectRefPath = path.join(cliSettings.workdir, "supabase", ".temp", "project-ref"); const linkedProjectRef = yield* Effect.gen(function* () { if (Option.isSome(cliSettings.projectId)) { return cliSettings.projectId; } - const exists = yield* fs.exists(projectRefPath).pipe(Effect.orElseSucceed(() => false)); + if ((yield* currentProjectStateMode) === "stack") { + return yield* readLinkSnapshot(fs, path, cliSettings.workdir).pipe( + Effect.map((snapshot) => snapshot.ref), + Effect.catch((cause) => + output + .raw(`failed to load project ref: ${cause.message}\n`, "stderr") + .pipe(Effect.as(Option.none<string>())), + ), + ); + } + + const exists = yield* legacyState + .exists(cliSettings.workdir, "projectRef") + .pipe(Effect.orElseSucceed(() => false)); if (!exists) { return Option.none<string>(); } @@ -79,7 +95,7 @@ export const services = Effect.fn("services")(function* (_flags: ServicesFlags) // Warns on a ref-file read error, but treats a NotFound race between the // exists() check and this read as simply unlinked (silent, no warning). // Only the "failed to load project ref: " prefix is compatibility-bearing. - const content = yield* fs.readFileString(projectRefPath).pipe( + const content = yield* legacyState.read(cliSettings.workdir, "projectRef").pipe( Effect.catchReason("PlatformError", "NotFound", () => Effect.succeed("")), Effect.catch((cause) => output.raw(`failed to load project ref: ${String(cause)}\n`, "stderr").pipe(Effect.as("")), @@ -207,9 +223,9 @@ export const services = Effect.fn("services")(function* (_flags: ServicesFlags) rows, backend.kind === "stack" ? { - heading: "The CLI stack catalog versions differ from your linked project:", + heading: "The stack service versions differ from your linked project:", recommendation: - "These versions come from the CLI stack catalog and cannot be changed with supabase link.", + "Run supabase link to update services set to linked (the default) in supabase/supabase.lock; services pinned in [local.services] keep their version.", } : {}, ); diff --git a/apps/cli/src/commands/services/services.integration.test.ts b/apps/cli/src/commands/services/services.integration.test.ts index ba86633639..3cdff4aa24 100644 --- a/apps/cli/src/commands/services/services.integration.test.ts +++ b/apps/cli/src/commands/services/services.integration.test.ts @@ -1,3 +1,4 @@ +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; import { describe, expect, it } from "@effect/vitest"; import { catalogPins, orioledbVersions } from "@supabase/stack/internal/artifacts"; import { BunServices } from "@effect/platform-bun"; @@ -20,7 +21,7 @@ import { import { FetchHttpClient } from "effect/http"; import { CommandCredentials } from "../../auth/command-credentials.service.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; -import { INVALID_PROJECT_REF_MESSAGE } from "../../config/project-ref.service.ts"; +import { INVALID_PROJECT_REF_MESSAGE } from "../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../telemetry/linked-project-cache.service.ts"; import { GLOBAL_FLAGS, OutputFlag } from "../../command-internal/global-flags.ts"; import { stackBackendLayer } from "../../command-internal/stack-backend.ts"; @@ -102,6 +103,7 @@ function setup( cachedRefs, layer: Layer.mergeAll( BunServices.layer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), FetchHttpClient.layer, mockRuntimeInfo({ cwd: opts.workdir ?? defaultWorkdir.current, @@ -266,6 +268,7 @@ describe("services", () => { const args = ["services"]; const layer = Layer.mergeAll( BunServices.layer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), processControlLayer, CliOutput.layer(textCliOutputFormatter()), out.layer, @@ -472,6 +475,27 @@ describe("services", () => { }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), ); + it.effect("reports an invalid supabase.lock before falling back to the bundled versions", () => + Effect.gen(function* () { + const workdir = yield* makeProjectWithDbMajorVersion(17); + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.writeFileString(path.join(workdir, "supabase", "supabase.lock"), "{ not json"); + const { layer, out } = setup({ goOutput: Option.some("json"), workdir }); + + yield* services({}).pipe(Effect.provide(Layer.mergeAll(layer, stackBackendLayer("stack")))); + + expect(yield* decodeServiceRows(out.stdoutText)).toContainEqual( + expect.objectContaining({ + name: "ghcr.io/supabase/cli/postgres", + local: catalogUpstreamVersion("postgres"), + }), + ); + expect(out.stderrText).toContain("supabase/supabase.lock"); + expect(out.stderrText).toContain("using default stack catalog versions"); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + ); + it.effect("ignores config.json and reads legacy config.toml for local image selection", () => Effect.gen(function* () { const workdir = yield* makeProjectWithConfigFiles({ @@ -670,7 +694,9 @@ major_version = 15 remote: "17.6.1.200", }), ); - expect(stack.out.stderrText).toContain("cannot be changed with supabase link"); + expect(stack.out.stderrText).toContain( + "Run supabase link to update services set to linked (the default) in supabase/supabase.lock", + ); expect(stack.out.stderrText).not.toContain("You are running different service versions"); }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), ); diff --git a/apps/cli/src/commands/snippets/download/download.command.ts b/apps/cli/src/commands/snippets/download/download.command.ts index 9bb2ae8201..7239df4734 100644 --- a/apps/cli/src/commands/snippets/download/download.command.ts +++ b/apps/cli/src/commands/snippets/download/download.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/snippets/download/download.handler.ts b/apps/cli/src/commands/snippets/download/download.handler.ts index 2b5e2609ad..7b6123fb8d 100644 --- a/apps/cli/src/commands/snippets/download/download.handler.ts +++ b/apps/cli/src/commands/snippets/download/download.handler.ts @@ -3,7 +3,7 @@ import { Effect, Predicate, Result, Schema } from "effect"; import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; import { goQuote } from "../../../command-internal/go-quote.ts"; @@ -121,12 +121,12 @@ export const snippetsDownload = Effect.fn("snippets.download")(function* ( ) { const output = yield* Output; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const snippetId = yield* Effect.fromResult(parseSnippetUuid(flags.snippetId)).pipe( diff --git a/apps/cli/src/commands/snippets/list/list.command.ts b/apps/cli/src/commands/snippets/list/list.command.ts index f19f376d28..ccecee4ffc 100644 --- a/apps/cli/src/commands/snippets/list/list.command.ts +++ b/apps/cli/src/commands/snippets/list/list.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/snippets/list/list.handler.ts b/apps/cli/src/commands/snippets/list/list.handler.ts index 3207bfcd2b..67a1d8d2e3 100644 --- a/apps/cli/src/commands/snippets/list/list.handler.ts +++ b/apps/cli/src/commands/snippets/list/list.handler.ts @@ -3,7 +3,7 @@ import { Effect, Option, Predicate, Schema } from "effect"; import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; @@ -93,7 +93,7 @@ export const snippetsList = Effect.fn("snippets.list")(function* (flags: Snippet const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -103,7 +103,7 @@ export const snippetsList = Effect.fn("snippets.list")(function* (flags: Snippet // write linked-project cache (needs `ref`) // flush telemetry (no `ref` required) yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { if (Option.getOrUndefined(goOutputFlag) === "env") { diff --git a/apps/cli/src/commands/ssl-enforcement/get/get.command.ts b/apps/cli/src/commands/ssl-enforcement/get/get.command.ts index 73268ef99b..a4e2dedf8a 100644 --- a/apps/cli/src/commands/ssl-enforcement/get/get.command.ts +++ b/apps/cli/src/commands/ssl-enforcement/get/get.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/ssl-enforcement/get/get.handler.ts b/apps/cli/src/commands/ssl-enforcement/get/get.handler.ts index 0bf9a7f158..4b254246c1 100644 --- a/apps/cli/src/commands/ssl-enforcement/get/get.handler.ts +++ b/apps/cli/src/commands/ssl-enforcement/get/get.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -30,7 +30,7 @@ export const sslEnforcementGet = Effect.fn("ssl-enforcement.get")(function* ( const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -38,7 +38,7 @@ export const sslEnforcementGet = Effect.fn("ssl-enforcement.get")(function* ( // emission fails. `linkedProjectCache.cache` requires a resolved ref, so // it wraps the inner sub-effect only. yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/ssl-enforcement/update/update.command.ts b/apps/cli/src/commands/ssl-enforcement/update/update.command.ts index e0e3e95f36..8f1b0bc632 100644 --- a/apps/cli/src/commands/ssl-enforcement/update/update.command.ts +++ b/apps/cli/src/commands/ssl-enforcement/update/update.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/ssl-enforcement/update/update.handler.ts b/apps/cli/src/commands/ssl-enforcement/update/update.handler.ts index cd92a3f378..d3084d8b0b 100644 --- a/apps/cli/src/commands/ssl-enforcement/update/update.handler.ts +++ b/apps/cli/src/commands/ssl-enforcement/update/update.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -33,7 +33,7 @@ export const sslEnforcementUpdate = Effect.fn("ssl-enforcement.update")(function const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; @@ -48,7 +48,7 @@ export const sslEnforcementUpdate = Effect.fn("ssl-enforcement.update")(function return yield* new SslEnforcementNoEnableDisableFlagError(); } - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const updating = diff --git a/apps/cli/src/commands/sso/add/add.command.ts b/apps/cli/src/commands/sso/add/add.command.ts index b431c07105..d6dbbbf919 100644 --- a/apps/cli/src/commands/sso/add/add.command.ts +++ b/apps/cli/src/commands/sso/add/add.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/sso/add/add.handler.ts b/apps/cli/src/commands/sso/add/add.handler.ts index 10709a6f34..7ae676619d 100644 --- a/apps/cli/src/commands/sso/add/add.handler.ts +++ b/apps/cli/src/commands/sso/add/add.handler.ts @@ -4,7 +4,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { cobraMutuallyExclusiveErrorMessage, @@ -145,7 +145,7 @@ export const ssoAdd = Effect.fn("sso.add")(function* (flags: SsoAddFlags) { const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const stdio = yield* Stdio.Stdio; @@ -231,7 +231,7 @@ export const ssoAdd = Effect.fn("sso.add")(function* (flags: SsoAddFlags) { const attributeMappingFile = pflagStringValue(occurrences, "attribute-mapping-file"); const domains = pflagSliceValue(occurrences, "domains", flags.domains); - const ref = yield* resolver.resolve(projectRef); + const { ref } = yield* resolver.resolve(projectRef); // Use the pflag-reconciled profile's host when it disagreed with // `--profile`, otherwise the config layer's — applies to the POST and diff --git a/apps/cli/src/commands/sso/info/info.command.ts b/apps/cli/src/commands/sso/info/info.command.ts index 9cb148d261..3c62174fd6 100644 --- a/apps/cli/src/commands/sso/info/info.command.ts +++ b/apps/cli/src/commands/sso/info/info.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/sso/info/info.handler.ts b/apps/cli/src/commands/sso/info/info.handler.ts index df6f3d1b8c..e984adae4f 100644 --- a/apps/cli/src/commands/sso/info/info.handler.ts +++ b/apps/cli/src/commands/sso/info/info.handler.ts @@ -1,6 +1,6 @@ import { Effect, Option } from "effect"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { @@ -17,12 +17,12 @@ import type { SsoInfoFlags } from "./info.command.ts"; export const ssoInfo = Effect.fn("sso.info")(function* (flags: SsoInfoFlags) { const output = yield* Output; const goOutputFlag = yield* OutputFlag; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const payload = buildInfoPayload(ref); diff --git a/apps/cli/src/commands/sso/list/list.command.ts b/apps/cli/src/commands/sso/list/list.command.ts index 13905e7e16..4c4ef32e80 100644 --- a/apps/cli/src/commands/sso/list/list.command.ts +++ b/apps/cli/src/commands/sso/list/list.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/sso/list/list.handler.ts b/apps/cli/src/commands/sso/list/list.handler.ts index aed93e2aed..53a38f4f3c 100644 --- a/apps/cli/src/commands/sso/list/list.handler.ts +++ b/apps/cli/src/commands/sso/list/list.handler.ts @@ -2,7 +2,7 @@ import type { SupabaseApiError } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { encodeEnv, encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; @@ -61,12 +61,12 @@ export const ssoList = Effect.fn("sso.list")(function* (flags: SsoListFlags) { const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/sso/remove/remove.command.ts b/apps/cli/src/commands/sso/remove/remove.command.ts index 95a41eb24e..c5490ba066 100644 --- a/apps/cli/src/commands/sso/remove/remove.command.ts +++ b/apps/cli/src/commands/sso/remove/remove.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/sso/remove/remove.handler.ts b/apps/cli/src/commands/sso/remove/remove.handler.ts index 4356533dac..a14198767d 100644 --- a/apps/cli/src/commands/sso/remove/remove.handler.ts +++ b/apps/cli/src/commands/sso/remove/remove.handler.ts @@ -2,7 +2,7 @@ import type { SupabaseApiError } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; @@ -58,14 +58,14 @@ export const ssoRemove = Effect.fn("sso.remove")(function* (flags: SsoRemoveFlag const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { const providerId = yield* Effect.fromResult(validateUuid(flags.providerId)); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const removing = diff --git a/apps/cli/src/commands/sso/show/show.command.ts b/apps/cli/src/commands/sso/show/show.command.ts index 58d90a54f8..79a2a73350 100644 --- a/apps/cli/src/commands/sso/show/show.command.ts +++ b/apps/cli/src/commands/sso/show/show.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { managementApiRuntimeLayer } from "../../../command-internal/management-api-runtime.layer.ts"; diff --git a/apps/cli/src/commands/sso/show/show.handler.ts b/apps/cli/src/commands/sso/show/show.handler.ts index 4367c95c4f..fe58d467f7 100644 --- a/apps/cli/src/commands/sso/show/show.handler.ts +++ b/apps/cli/src/commands/sso/show/show.handler.ts @@ -2,7 +2,7 @@ import type { SupabaseApiError } from "@supabase/api/effect"; import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { encodeGoJson } from "../../../command-internal/go-output.encoders.ts"; @@ -47,14 +47,14 @@ export const ssoShow = Effect.fn("sso.show")(function* (flags: SsoShowFlags) { const output = yield* Output; const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { const providerId = yield* Effect.fromResult(validateUuid(flags.providerId)); - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/commands/sso/update/update.command.ts b/apps/cli/src/commands/sso/update/update.command.ts index 7be5c75e55..0361be8c3b 100644 --- a/apps/cli/src/commands/sso/update/update.command.ts +++ b/apps/cli/src/commands/sso/update/update.command.ts @@ -3,7 +3,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/sso/update/update.handler.ts b/apps/cli/src/commands/sso/update/update.handler.ts index 042fca0962..7927a86314 100644 --- a/apps/cli/src/commands/sso/update/update.handler.ts +++ b/apps/cli/src/commands/sso/update/update.handler.ts @@ -4,7 +4,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; import { cobraMutuallyExclusiveErrorMessage, @@ -210,7 +210,7 @@ export const ssoUpdate = Effect.fn("sso.update")(function* (flags: SsoUpdateFlag const goOutputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; const cliSettings = yield* CommandSettings; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; const stdio = yield* Stdio.Stdio; @@ -301,7 +301,7 @@ export const ssoUpdate = Effect.fn("sso.update")(function* (flags: SsoUpdateFlag const providerId = yield* Effect.fromResult(validateUuid(flags.providerId)); - const ref = yield* resolver.resolve(projectRefFlag); + const { ref } = yield* resolver.resolve(projectRefFlag); // Use the pflag-reconciled profile's host when it disagreed with // `--profile`, otherwise the config layer's. diff --git a/apps/cli/src/commands/status/status.integration.test.ts b/apps/cli/src/commands/status/status.integration.test.ts index c33b4f45e8..0859cd518f 100644 --- a/apps/cli/src/commands/status/status.integration.test.ts +++ b/apps/cli/src/commands/status/status.integration.test.ts @@ -24,6 +24,11 @@ import * as HttpClientRequestModule from "effect/http/HttpClientRequest"; import { vi } from "vitest"; import { mockOutput, mockProcessControl } from "../../../tests/helpers/mocks.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../tests/helpers/legacy-state-guard.ts"; import { statusCodeFailure, transportFailure, @@ -41,6 +46,11 @@ import { CommandPlatformApi } from "../../auth/command-platform-api.service.ts"; import { runtimeInfoLayer } from "../../shared/runtime/runtime-info.layer.ts"; import { AccessTokenRequiredError } from "../../auth/errors.ts"; import { OutputFlag } from "../../command-internal/global-flags.ts"; +import { writeProjectLink } from "../../command-internal/project-link.ts"; +import { + projectStateModeLayer, + type ProjectStateMode, +} from "../../command-internal/project-state-mode.ts"; import { withJsonErrorHandling } from "../../shared/output/json-error-handling.ts"; import { machineErrorContextLayer } from "../../shared/output/machine-error-context.layer.ts"; import { jsonOutputLayer, streamJsonOutputLayer } from "../../shared/output/output.layer.ts"; @@ -325,6 +335,8 @@ interface SetupOpts { readonly apiFactoryLayer?: Layer.Layer<CommandPlatformApiFactory>; /** `SUPABASE_PROJECT_ID` for the linked-state chain — defaults to unset. */ readonly projectId?: Option.Option<string>; + /** Where the link state lives; defaults to the legacy `supabase/.temp` files. */ + readonly projectStateMode?: ProjectStateMode; } function setup(opts: SetupOpts = {}) { @@ -353,20 +365,32 @@ function setup(opts: SetupOpts = {}) { ? undefined : mockCommandPlatformApiFactoryDirect(opts.apiFactory); + const recorder = recordingFileSystem(); const layer = Layer.mergeAll( BunServices.layer, + recorder.layer, runtimeInfoLayer, out.layer, cliSettings, telemetry.layer, child.layer, + projectStateModeLayer(opts.projectStateMode ?? "legacy"), Layer.succeed(OutputFlag, opts.goOutput ?? Option.none()), ...(apiMock === undefined ? [] : [apiMock.layer]), ...(apiFactoryMock === undefined ? [] : [apiFactoryMock.layer]), ...(opts.apiFactoryLayer === undefined ? [] : [opts.apiFactoryLayer]), ); - return { workdir, out, telemetry, child, layer, apiMock, apiFactoryMock }; + return { + workdir, + out, + telemetry, + child, + layer, + apiMock, + apiFactoryMock, + accessed: recorder.accessed, + }; } /** @@ -1253,6 +1277,52 @@ content_path = "./supabase/templates/password_changed_notification.html" }, ); + it.effect( + "stack mode: prints the Linked Project block from .link.json with zero Management API calls", + () => { + const { layer, out, workdir, apiMock } = setup({ + branches: { ok: [] }, + projectStateMode: "stack", + }); + return Effect.gen(function* () { + yield* writeConfig(); + yield* writeProjectLink(workdir, { + project: { + ref: LINKED_PLAIN_REF, + name: "My Project", + organization_id: "org_1", + organization_slug: "acme", + }, + active_branch: { ref: LINKED_PLAIN_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + yield* status(flags()); + expect( + out.stdoutText.startsWith( + `Linked Project:\n Org: acme (org_1)\n Project: My Project (${LINKED_PLAIN_REF})\n`, + ), + ).toBe(true); + expect(apiMock?.requests).toEqual([]); + }).pipe(Effect.provide(layer)); + }, + ); + + it.effect( + "stack mode: never reads or writes supabase/.temp and ignores a legacy link there", + () => { + const { layer, out, workdir, accessed } = setup({ projectStateMode: "stack" }); + seedLegacyStateDecoys(workdir); + return Effect.gen(function* () { + yield* writeConfig(); + yield* status(flags()); + expect(out.stdoutText).not.toContain("Linked Project"); + expect(out.stdoutText).not.toContain("decoy"); + expect(accessed.some((entry) => entry.endsWith("config.toml"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + }).pipe(Effect.provide(layer)); + }, + ); + it.effect( "branch-linked with an api mock: prints the full block (Org + parent Project + resolved Branch) with a Checking linked branch... spinner", () => { diff --git a/apps/cli/src/commands/storage/cp/cp.handler.ts b/apps/cli/src/commands/storage/cp/cp.handler.ts index 883a9fafa0..99207d4bdf 100644 --- a/apps/cli/src/commands/storage/cp/cp.handler.ts +++ b/apps/cli/src/commands/storage/cp/cp.handler.ts @@ -4,7 +4,7 @@ import { Effect, FileSystem, Option, Path, Ref, Result, Stream } from "effect"; import type { PlatformError } from "effect/PlatformError"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -63,7 +63,7 @@ export const storageCp = Effect.fn("storage.cp")(function* (flags: StorageCpFlag const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; const linkedProjectCache = yield* LinkedProjectCache; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const posixPath = yield* Effect.provide(Path.Path, BunPath.layerPosix); @@ -91,7 +91,7 @@ export const storageCp = Effect.fn("storage.cp")(function* (flags: StorageCpFlag }); } - const projectRef = flags.local ? "" : yield* resolver.loadProjectRef(flags.projectRef); + const projectRef = flags.local ? "" : (yield* resolver.load(flags.projectRef)).ref; if (projectRef !== "") { yield* Ref.set(linkedRef, Option.some(projectRef)); } diff --git a/apps/cli/src/commands/storage/ls/ls.handler.ts b/apps/cli/src/commands/storage/ls/ls.handler.ts index 76093562a4..34cb2f95d8 100644 --- a/apps/cli/src/commands/storage/ls/ls.handler.ts +++ b/apps/cli/src/commands/storage/ls/ls.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option, Ref } from "effect"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -27,7 +27,7 @@ export const storageLs = Effect.fn("storage.ls")(function* (flags: StorageLsFlag const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; const linkedProjectCache = yield* LinkedProjectCache; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedRef = yield* Ref.make(Option.none<string>()); @@ -44,7 +44,7 @@ export const storageLs = Effect.fn("storage.ls")(function* (flags: StorageLsFlag // `--local` clears the ref; otherwise the linked path resolves it. No network access yet, // safe before the URL parse below. - const projectRef = flags.local ? "" : yield* resolver.loadProjectRef(flags.projectRef); + const projectRef = flags.local ? "" : (yield* resolver.load(flags.projectRef)).ref; if (projectRef !== "") { yield* Ref.set(linkedRef, Option.some(projectRef)); } diff --git a/apps/cli/src/commands/storage/mv/mv.handler.ts b/apps/cli/src/commands/storage/mv/mv.handler.ts index 65b5e7d982..1177df71b6 100644 --- a/apps/cli/src/commands/storage/mv/mv.handler.ts +++ b/apps/cli/src/commands/storage/mv/mv.handler.ts @@ -2,7 +2,7 @@ import { BunPath } from "@effect/platform-bun"; import { Effect, Option, Path, Ref } from "effect"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; @@ -34,7 +34,7 @@ export const storageMv = Effect.fn("storage.mv")(function* (flags: StorageMvFlag const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; const linkedProjectCache = yield* LinkedProjectCache; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const posixPath = yield* Effect.provide(Path.Path, BunPath.layerPosix); const linkedRef = yield* Ref.make(Option.none<string>()); @@ -50,7 +50,7 @@ export const storageMv = Effect.fn("storage.mv")(function* (flags: StorageMvFlag }); } - const projectRef = flags.local ? "" : yield* resolver.loadProjectRef(flags.projectRef); + const projectRef = flags.local ? "" : (yield* resolver.load(flags.projectRef)).ref; if (projectRef !== "") { yield* Ref.set(linkedRef, Option.some(projectRef)); } diff --git a/apps/cli/src/commands/storage/rm/rm.handler.ts b/apps/cli/src/commands/storage/rm/rm.handler.ts index f5ad9a587e..8f2e542ec8 100644 --- a/apps/cli/src/commands/storage/rm/rm.handler.ts +++ b/apps/cli/src/commands/storage/rm/rm.handler.ts @@ -1,7 +1,7 @@ import { Effect, FileSystem, Option, Path, Ref } from "effect"; import { CommandSettings } from "../../../config/command-settings.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { resolveYesWithProjectEnv } from "../../../command-internal/global-flags.ts"; @@ -55,7 +55,7 @@ export const storageRm = Effect.fn("storage.rm")(function* (flags: StorageRmFlag const cliSettings = yield* CommandSettings; const telemetryState = yield* TelemetryState; const linkedProjectCache = yield* LinkedProjectCache; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; @@ -73,7 +73,7 @@ export const storageRm = Effect.fn("storage.rm")(function* (flags: StorageRmFlag }); } - const projectRef = flags.local ? "" : yield* resolver.loadProjectRef(flags.projectRef); + const projectRef = flags.local ? "" : (yield* resolver.load(flags.projectRef)).ref; if (projectRef !== "") { yield* Ref.set(linkedRef, Option.some(projectRef)); } diff --git a/apps/cli/src/commands/test/db/SIDE_EFFECTS.md b/apps/cli/src/commands/test/db/SIDE_EFFECTS.md index 3fbdd04e34..b85b60921b 100644 --- a/apps/cli/src/commands/test/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/test/db/SIDE_EFFECTS.md @@ -7,7 +7,7 @@ | `<cwd>/supabase/tests/**/*.{sql,pg}` | SQL | default test discovery when no `[path]` given | | `<path...>` | SQL | when explicit test files/dirs are passed | | `<workdir>/supabase/config.toml` | TOML | always: `db.port`, `db.shadow_port`, `db.password`, `project_id`. Absent → defaults; **present but malformed → command fails** | -| `<workdir>/supabase/.temp/pooler-url` | text | `--linked` pooler fallback only — the connection-pooler URL written by `supabase link` (read from this file rather than from config.toml) | +| `<workdir>/supabase/.temp/pooler-url` | text | `--linked` pooler fallback, linked ref only — the connection-pooler URL written by `supabase link` (read from this file, not config.toml) | | `~/.supabase/access-token` | text | `--linked` only, when `SUPABASE_ACCESS_TOKEN` unset | | `<workdir>/supabase/.temp/project-ref` | text | `--linked` only, to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | | `<supabaseHome>/stacks/*/state.json` | JSON | managed `--local`: discover the project stack and open its saved definition | @@ -149,7 +149,10 @@ command (exit 1). - The linked connection pooler URL is read from `supabase/.temp/pooler-url` (written by `supabase link`) rather than from config.toml — the `[db.pooler]` config.toml field is intentionally ignored. The pooler's `?options=reference=<ref>` startup param is - carried through to the connection for the legacy pooler-URL format. + carried through to the connection for the legacy pooler-URL format. The saved URL is used + only when the target ref is the linked project's ref (`.temp/project-ref`); a `--project-ref` + or `SUPABASE_PROJECT_ID` naming another project fetches that project's pooler config from the + Management API instead. - Compose pg_prove image is fixed at `supabase/pg_prove:3.36`; the old Go CLI's `[images] pgprove` config override is not modeled by the TS config schema (documented divergence). - The old Go CLI's hidden `--network-id` override is not declared on the TS command (documented divergence). diff --git a/apps/cli/src/commands/unlink/SIDE_EFFECTS.md b/apps/cli/src/commands/unlink/SIDE_EFFECTS.md index afcbd4b907..95308d474e 100644 --- a/apps/cli/src/commands/unlink/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/unlink/SIDE_EFFECTS.md @@ -27,7 +27,10 @@ None. ## Environment Variables -None beyond `--workdir` / `SUPABASE_WORKDIR` resolution. +| Variable | Purpose | +| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `SUPABASE_WORKDIR` | `--workdir` resolution | +| `SUPABASE_EXPERIMENTAL_STACK` | `1` or `0` selects stack project-state mode over `[experimental] stack`; unset or empty uses the config. Any other value fails (see Exit Codes) | ## Exit Codes @@ -38,6 +41,7 @@ None beyond `--workdir` / `SUPABASE_WORKDIR` resolution. | `1` | project-ref read error | | `1` | temp-dir removal error | | `1` | keyring delete error other than not-found (e.g. permission denied) | +| `1` | `SUPABASE_EXPERIMENTAL_STACK` is set to a non-empty value other than `0` or `1`; nothing is removed | ## Output @@ -50,6 +54,19 @@ None beyond `--workdir` / `SUPABASE_WORKDIR` resolution. Emits a structured success (`{ project_ref }`) and suppresses the human `Finished` line. +## Stack mode + +In stack project-state mode, `unlink` reads the ref from `supabase/.link.json` +(`active_branch.ref`), removes that file instead of `supabase/.temp/`, and deletes the same +keyring credential. A missing document fails with `ProjectRefNotLinkedError`; the legacy `.temp` +directory is left untouched. The file is removed under the project lease +`supabase/.supabase.lock.lease` (created if absent), the lease `supabase link` publishes +`.link.json` under; `supabase/.gitignore` gains the lease entries whenever the lease is taken. A +lease that stays busy fails with `LockResolutionError` reason `lease-busy` +and exit code `1` before the link or the credential is touched. A document that does not decode is still removed, so a damaged link +can be cleared; the ref is then unknown, so no credential is deleted and the structured success +carries `project_ref: null`. + ## Known divergence The `Finished supabase unlink.` line is emitted as **plain text**; the old Go CLI rendered diff --git a/apps/cli/src/commands/unlink/unlink.errors.ts b/apps/cli/src/commands/unlink/unlink.errors.ts index 8c997ad4f1..61915d50cb 100644 --- a/apps/cli/src/commands/unlink/unlink.errors.ts +++ b/apps/cli/src/commands/unlink/unlink.errors.ts @@ -7,9 +7,9 @@ import { } from "../../shared/telemetry/error-actionability.ts"; /** - * Reading `supabase/.temp/project-ref` failed for a reason other than the file - * being absent (which maps to `ProjectRefNotLinkedError`). Message format: - * `"failed to load project ref: " + err`. + * Reading the linked project ref (`supabase/.temp/project-ref`, or `supabase/.link.json` in stack + * mode) failed for a reason other than the file being absent (which maps to + * `ProjectRefNotLinkedError`). Message format: `"failed to load project ref: " + err`. */ export class UnlinkRefReadError extends Data.TaggedError("UnlinkRefReadError")<{ readonly message: string; @@ -19,7 +19,7 @@ export class UnlinkRefReadError extends Data.TaggedError("UnlinkRefReadError")<{ } } -/** Removing the `supabase/.temp` directory failed. */ +/** Removing the `supabase/.temp` directory, or `supabase/.link.json` in stack mode, failed. */ export class UnlinkTempRemovalError extends Data.TaggedError("UnlinkTempRemovalError")<{ readonly message: string; }> { diff --git a/apps/cli/src/commands/unlink/unlink.handler.ts b/apps/cli/src/commands/unlink/unlink.handler.ts index 7049256544..d3a38ff567 100644 --- a/apps/cli/src/commands/unlink/unlink.handler.ts +++ b/apps/cli/src/commands/unlink/unlink.handler.ts @@ -1,62 +1,115 @@ -import { Effect, FileSystem, Path, Predicate, Result } from "effect"; +import { Effect, FileSystem, Option, Path, Predicate, Result } from "effect"; import { CommandCredentials } from "../../auth/command-credentials.service.ts"; import { CredentialDeleteError } from "../../auth/errors.ts"; import { CommandSettings } from "../../config/command-settings.service.ts"; import { ProjectRefNotLinkedError } from "../../config/project-ref.errors.ts"; -import { PROJECT_NOT_LINKED_MESSAGE } from "../../config/project-ref.service.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../config/remote-target.service.ts"; import { TelemetryState } from "../../telemetry/telemetry-state.service.ts"; import { Output } from "../../shared/output/output.service.ts"; -import { tempPaths } from "../../command-internal/temp-paths.ts"; +import { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; +import { currentProjectStateMode } from "../../command-internal/project-state-mode.ts"; +import { readProjectLink, removeProjectLink } from "../../command-internal/project-link.ts"; +import { withProjectLease } from "../../command-internal/stack-lock.ts"; import { UnlinkRefReadError, UnlinkTempRemovalError } from "./unlink.errors.ts"; +const readLegacyLinkRef = Effect.fn("unlink.readLegacyRef")(function* ( + legacyState: LegacyProjectState["Service"], + workdir: string, +) { + const exists = yield* legacyState + .exists(workdir, "projectRef") + .pipe(Effect.orElseSucceed(() => false)); + if (!exists) { + return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); + } + // No trimming needed: `link` writes the ref with no trailing newline, so the raw + // bytes round-trip exactly for both the stderr message and the keyring key. + return yield* legacyState.read(workdir, "projectRef").pipe( + Effect.mapError( + (cause) => + new UnlinkRefReadError({ + message: `failed to load project ref: ${String(cause)}`, + }), + ), + ); +}); + +/** The linked ref, or `undefined` when the link document is malformed and only its removal is possible. */ +const readStackLinkRef = Effect.fn("unlink.readStackRef")(function* (workdir: string) { + const link = yield* readProjectLink(workdir).pipe( + Effect.catchTags({ + ProjectLinkIoError: (cause) => + Effect.fail( + new UnlinkRefReadError({ + message: `failed to load project ref: ${cause.message}`, + }), + ), + ProjectLinkMalformedError: () => Effect.undefined, + }), + ); + if (link === undefined) return undefined; + if (Option.isNone(link)) { + return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); + } + return link.value.active_branch.ref; +}); + export const unlink = Effect.fn("unlink")(function* () { const output = yield* Output; const cliSettings = yield* CommandSettings; const credentials = yield* CommandCredentials; const telemetryState = yield* TelemetryState; + const legacyState = yield* LegacyProjectState; + const workdir = cliSettings.workdir; + const stackState = (yield* currentProjectStateMode) === "stack"; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const paths = tempPaths(path, cliSettings.workdir); - - yield* Effect.gen(function* () { + const run = Effect.gen(function* () { // 1. Load the linked project ref. An absent file means not-linked; any other - // read failure surfaces verbatim. - const exists = yield* fs.exists(paths.projectRef).pipe(Effect.orElseSucceed(() => false)); - if (!exists) { - return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); - } - // No trimming needed: `link` writes the ref with no trailing newline, so the raw - // bytes round-trip exactly for both the stderr message and the keyring key. - const projectRef = yield* fs.readFileString(paths.projectRef).pipe( - Effect.mapError( - (cause) => - new UnlinkRefReadError({ - message: `failed to load project ref: ${String(cause)}`, - }), - ), - ); + // read failure surfaces verbatim. In stack mode the read, the removal, and the credential + // delete share one project lease, so a concurrent `link` cannot swap the target in between. + const projectRef = stackState + ? yield* readStackLinkRef(workdir) + : yield* readLegacyLinkRef(legacyState, workdir); - yield* output.raw(`Unlinking project: ${projectRef}\n`, "stderr"); + yield* output.raw( + projectRef === undefined + ? "Unlinking project: unreadable link document, its credential is left in place\n" + : `Unlinking project: ${projectRef}\n`, + "stderr", + ); // 2. Best-effort: both the temp-dir removal and the credential delete are // attempted regardless of either failing; their errors are joined below. const collected: Array<UnlinkTempRemovalError | CredentialDeleteError> = []; - const removed = yield* fs.remove(paths.tempDir, { recursive: true, force: true }).pipe( - Effect.mapError( - (cause) => - new UnlinkTempRemovalError({ - message: `failed to remove temp directory: ${String(cause)}`, - }), - ), - Effect.result, - ); + const removed = yield* stackState + ? removeProjectLink(workdir).pipe( + Effect.mapError( + (cause) => + new UnlinkTempRemovalError({ + message: `failed to remove project link: ${cause.message}`, + }), + ), + Effect.result, + ) + : legacyState.removeAll(workdir).pipe( + Effect.mapError( + (cause) => + new UnlinkTempRemovalError({ + message: `failed to remove temp directory: ${String(cause)}`, + }), + ), + Effect.result, + ); if (Result.isFailure(removed)) collected.push(removed.failure); - const deleted = yield* credentials.deleteProjectCredential(projectRef).pipe(Effect.result); - if (Result.isFailure(deleted)) collected.push(deleted.failure); + if (projectRef !== undefined) { + const deleted = yield* credentials.deleteProjectCredential(projectRef).pipe(Effect.result); + if (Result.isFailure(deleted)) collected.push(deleted.failure); + } const [first, ...rest] = collected; if (first !== undefined) { @@ -76,7 +129,20 @@ export const unlink = Effect.fn("unlink")(function* () { if (output.format === "text") { yield* output.raw("Finished supabase unlink.\n"); } else { - yield* output.success("", { project_ref: projectRef }); + yield* output.success("", { project_ref: projectRef ?? null }); } - }).pipe(Effect.ensuring(telemetryState.flush)); + }); + + // The lease file lives in `supabase/`; without it no link exists, and nothing is read unleased. + const hasProjectDir = + stackState && + (yield* fs.exists(path.join(workdir, "supabase")).pipe(Effect.orElseSucceed(() => true))); + // A busy project lease fails the command before anything is read or removed. + yield* ( + !stackState + ? run + : hasProjectDir + ? withProjectLease(workdir, run) + : Effect.fail(new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE })) + ).pipe(Effect.ensuring(telemetryState.flush)); }); diff --git a/apps/cli/src/commands/unlink/unlink.integration.test.ts b/apps/cli/src/commands/unlink/unlink.integration.test.ts index 9282b3e04a..ddf73cc1f8 100644 --- a/apps/cli/src/commands/unlink/unlink.integration.test.ts +++ b/apps/cli/src/commands/unlink/unlink.integration.test.ts @@ -1,10 +1,31 @@ +import { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Cause, Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; +import { holdLock } from "@supabase/stack/effect"; +import { + Cause, + Clock, + Effect, + Exit, + FileSystem, + Fiber, + Layer, + Option, + Path, + Queue, + Scope, +} from "effect"; import { badArgument } from "effect/PlatformError"; +import { TestClock } from "effect/testing"; import * as HttpClient from "effect/http/HttpClient"; import { mockAnalytics, mockOutput } from "../../../tests/helpers/mocks.ts"; +import { + legacyStateAccesses, + recordingFileSystem, + seedLegacyStateDecoys, +} from "../../../tests/helpers/legacy-state-guard.ts"; import { VALID_REF, buildTestRuntime, @@ -14,6 +35,11 @@ import { mockTelemetryStateTracked, useTempWorkdir, } from "../../../tests/helpers/command-mocks.ts"; +import { writeProjectLink } from "../../command-internal/project-link.ts"; +import { + projectStateMode, + projectStateModeLayer, +} from "../../command-internal/project-state-mode.ts"; import { unlink } from "./unlink.handler.ts"; const tempRoot = useTempWorkdir("supabase-unlink-int-"); @@ -27,17 +53,18 @@ interface SetupOpts { format?: "text" | "json" | "stream-json"; deleteFails?: boolean; removeFails?: boolean; + stack?: boolean; } -// Wraps the real Bun FileSystem but forces `remove` to fail, so the temp-dir-removal +// Wraps the real state service but forces `removeAll` to fail, so the temp-dir-removal // error branch is exercised deterministically, independent of filesystem permissions. -const failingRemoveFsLayer = Layer.effect( - FileSystem.FileSystem, +const failingRemoveStateLayer = Layer.effect( + LegacyProjectState, Effect.gen(function* () { - const real = yield* FileSystem.FileSystem; - return FileSystem.FileSystem.of({ + const real = yield* LegacyProjectState; + return LegacyProjectState.of({ ...real, - remove: () => + removeAll: () => Effect.fail( badArgument({ module: "FileSystem", @@ -47,7 +74,7 @@ const failingRemoveFsLayer = Layer.effect( ), }); }), -).pipe(Layer.provide(BunServices.layer)); +).pipe(Layer.provide(legacyProjectStateLayer), Layer.provide(BunServices.layer)); const seedProjectRef = (workdir: string, ref: string) => Effect.gen(function* () { @@ -66,6 +93,7 @@ const tempDirExists = (workdir: string) => }); function setup(opts: SetupOpts = {}) { + const recorder = recordingFileSystem(); const out = mockOutput({ format: opts.format ?? "text" }); const telemetry = mockTelemetryStateTracked(); const credentials = mockCommandCredentialsTracked({ deleteFails: opts.deleteFails }); @@ -81,11 +109,20 @@ function setup(opts: SetupOpts = {}) { cliSettings, analytics: mockAnalytics(), telemetry: telemetry.layer, + fileSystem: recorder.layer, }), credentials.layer, - ...(opts.removeFails === true ? [failingRemoveFsLayer] : []), + ...(opts.removeFails === true ? [failingRemoveStateLayer] : []), + ...(opts.stack === true ? [projectStateModeLayer("stack")] : []), ); - return { layer, out, telemetry, credentials, workdir: tempRoot.current }; + return { + layer, + out, + telemetry, + credentials, + workdir: tempRoot.current, + accessed: recorder.accessed, + }; } describe("unlink integration", () => { @@ -205,4 +242,183 @@ describe("unlink integration", () => { expect(success?.data).toMatchObject({ project_ref: VALID_REF }); }).pipe(Effect.provide(layer)); }); + + it.effect("fails when SUPABASE_EXPERIMENTAL_STACK is not 0 or 1, removing nothing", () => { + const { layer, credentials, workdir } = setup(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const mode = yield* projectStateMode({ + cwd: workdir, + env: { SUPABASE_EXPERIMENTAL_STACK: "maybe" }, + }).pipe(Effect.exit); + yield* seedProjectRef(workdir, VALID_REF); + yield* writeProjectLink(workdir, { + project: { ref: VALID_REF }, + active_branch: { ref: VALID_REF }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const error = yield* unlink().pipe(Effect.provide(projectStateModeLayer(mode)), Effect.flip); + + expect(error).toMatchObject({ + _tag: "ExperimentalFeatureFlagError", + message: "SUPABASE_EXPERIMENTAL_STACK must be 0 or 1 when set", + }); + expect(yield* tempDirExists(workdir)).toBe(true); + expect(yield* fs.exists(path.join(workdir, "supabase", ".link.json"))).toBe(true); + expect(credentials.deletedRefs).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + describe("in stack mode", () => { + const seedLinkDocument = (workdir: string, ref: string) => + writeProjectLink(workdir, { + project: { ref: VALID_REF }, + active_branch: { ref }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + + const linkDocumentExists = (workdir: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* fs.exists(path.join(workdir, "supabase", ".link.json")); + }); + + it.effect("removes the link document and the credential of the active branch", () => { + const { layer, out, credentials, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + yield* seedLinkDocument(workdir, "branchprojectrefabcd"); + yield* unlink(); + expect(yield* linkDocumentExists(workdir)).toBe(false); + expect(credentials.deletedRefs).toEqual(["branchprojectrefabcd"]); + expect(out.stdoutText).toContain("Finished supabase unlink."); + }).pipe(Effect.provide(layer)); + }); + + it.effect("reports not linked without creating supabase/ when the project has none", () => { + const { layer, credentials, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + const error = yield* unlink().pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "ProjectRefNotLinkedError" }); + expect(yield* fs.exists(path.join(workdir, "supabase"))).toBe(false); + expect(credentials.deletedRefs).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + it.effect("fails on a busy project lease before removing the link or its credential", () => { + const { layer, credentials, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + yield* seedLinkDocument(workdir, VALID_REF); + yield* holdLock(`${workdir}/supabase/.supabase.lock.lease`, "1 hour", "supabase.lock"); + // The lease wait runs on the test clock; the lock stays held, so the outcome is fixed. + yield* TestClock.adjust("1 second").pipe( + Effect.andThen(Effect.yieldNow), + Effect.forever, + Effect.forkScoped, + ); + + const error = yield* unlink().pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "lease-busy" }); + expect(yield* linkDocumentExists(workdir)).toBe(true); + expect(credentials.deletedRefs).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + it.effect("acts on the link document it reads under the lease, not one read before it", () => { + const { layer, out, credentials, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + const concurrentLink = yield* Scope.make(); + yield* seedLinkDocument(workdir, "firstprojectrefabcdef"); + yield* holdLock(`${workdir}/supabase/.supabase.lock.lease`, "1 hour", "supabase.lock").pipe( + Scope.provide(concurrentLink), + ); + // Each sleep the lease wait registers on the test clock is announced here. + const sleeps = yield* Queue.unbounded<void>(); + const unlinking = yield* Clock.clockWith((clock) => + unlink().pipe( + Effect.provideService(Clock.Clock, { + currentTimeMillisUnsafe: () => clock.currentTimeMillisUnsafe(), + currentTimeMillis: clock.currentTimeMillis, + currentTimeNanosUnsafe: () => clock.currentTimeNanosUnsafe(), + currentTimeNanos: clock.currentTimeNanos, + monotonicTimeNanosUnsafe: () => clock.monotonicTimeNanosUnsafe(), + monotonicTimeNanos: clock.monotonicTimeNanos, + sleep: (duration) => + Queue.offer(sleeps, undefined).pipe(Effect.andThen(clock.sleep(duration))), + }), + Effect.forkScoped, + ), + ); + + yield* Queue.take(sleeps); + + yield* seedLinkDocument(workdir, "secondprojectrefabcde"); + yield* Scope.close(concurrentLink, Exit.void); + const wakeOnEachSleep = TestClock.adjust("50 millis").pipe( + Effect.andThen(Queue.take(sleeps)), + Effect.forever, + ); + yield* Effect.race(Fiber.join(unlinking), wakeOnEachSleep); + + expect(out.stderrText).toContain("Unlinking project: secondprojectrefabcde"); + expect(credentials.deletedRefs).toEqual(["secondprojectrefabcde"]); + expect(yield* linkDocumentExists(workdir)).toBe(false); + }).pipe(Effect.provide(layer)); + }); + + it.effect("leaves supabase/supabase.lock untouched", () => { + const { layer, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const lock = path.join(workdir, "supabase", "supabase.lock"); + yield* seedLinkDocument(workdir, VALID_REF); + yield* fs.writeFileString(lock, '{"linked":"kept"}\n'); + yield* unlink(); + expect(yield* fs.readFileString(lock)).toBe('{"linked":"kept"}\n'); + }).pipe(Effect.provide(layer)); + }); + + it.effect("leaves the legacy temp dir alone without reading or writing it", () => { + const { layer, workdir, accessed } = setup({ stack: true }); + seedLegacyStateDecoys(workdir); + return Effect.gen(function* () { + yield* seedLinkDocument(workdir, VALID_REF); + yield* unlink(); + expect(accessed.some((entry) => entry.includes(".link.json"))).toBe(true); + expect(legacyStateAccesses(accessed)).toEqual([]); + expect(yield* tempDirExists(workdir)).toBe(true); + }).pipe(Effect.provide(layer)); + }); + + it.effect("fails with ProjectRefNotLinkedError when only legacy state exists", () => { + const { layer, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + yield* seedProjectRef(workdir, VALID_REF); + const error = yield* unlink().pipe(Effect.flip); + expect(error._tag).toBe("ProjectRefNotLinkedError"); + }).pipe(Effect.provide(layer)); + }); + + it.effect("removes a malformed link document and leaves credentials alone", () => { + const { layer, credentials, out, workdir } = setup({ stack: true }); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(workdir, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(workdir, "supabase", ".link.json"), "{not json"); + yield* unlink(); + expect(yield* linkDocumentExists(workdir)).toBe(false); + expect(credentials.deletedRefs).toEqual([]); + expect(out.stdoutText).toContain("Finished supabase unlink."); + }).pipe(Effect.provide(layer)); + }); + }); }); diff --git a/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts b/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts index 745b0c45fb..21fb312bf8 100644 --- a/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/activate/activate.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts b/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts index 574cf45783..c92ee4dd27 100644 --- a/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/activate/activate.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { gateResponse, suggestUpgrade } from "../../../command-internal/upgrade-suggest.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -38,12 +38,12 @@ export const vanitySubdomainsActivate = Effect.fn("vanity-subdomains.activate")( const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { // This check sits inside both `Effect.ensuring` wrappers so telemetry and the diff --git a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts index c8c1fda87f..654004bb42 100644 --- a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { upgradeSuggestion } from "../../../command-internal/upgrade-suggest.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; diff --git a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts index 656decf53d..6b1309ff1f 100644 --- a/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/check-availability/check-availability.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { gateResponse, suggestUpgrade } from "../../../command-internal/upgrade-suggest.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -37,12 +37,12 @@ export const vanitySubdomainsCheckAvailability = Effect.fn("vanity-subdomains.ch const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { // This check sits inside both `Effect.ensuring` wrappers so telemetry and the diff --git a/apps/cli/src/commands/vanity-subdomains/delete/delete.command.ts b/apps/cli/src/commands/vanity-subdomains/delete/delete.command.ts index 4a72421cae..7aef7734ab 100644 --- a/apps/cli/src/commands/vanity-subdomains/delete/delete.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/delete/delete.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/vanity-subdomains/delete/delete.handler.ts b/apps/cli/src/commands/vanity-subdomains/delete/delete.handler.ts index a3c012d68c..87088a385a 100644 --- a/apps/cli/src/commands/vanity-subdomains/delete/delete.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/delete/delete.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -26,12 +26,12 @@ export const vanitySubdomainsDelete = Effect.fn("vanity-subdomains.delete")(func const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const deleting = diff --git a/apps/cli/src/commands/vanity-subdomains/get/get.command.ts b/apps/cli/src/commands/vanity-subdomains/get/get.command.ts index 81c5a869f8..8183872d87 100644 --- a/apps/cli/src/commands/vanity-subdomains/get/get.command.ts +++ b/apps/cli/src/commands/vanity-subdomains/get/get.command.ts @@ -4,7 +4,7 @@ import type * as CliCommand from "effect/cli/Command"; import { withPermissions } from "../../../command-internal/command-permissions/command-permissions.annotation.ts"; import { compose } from "../../../command-internal/command-permissions/model.ts"; -import { projectRefPicker } from "../../../config/project-ref.layer.ts"; +import { projectRefPicker } from "../../../config/remote-target.layer.ts"; import { linkedProjectCache } from "../../../telemetry/linked-project-cache.layer.ts"; import { withJsonErrorHandling } from "../../../shared/output/json-error-handling.ts"; import { requireExperimental } from "../../../command-internal/experimental-gate.ts"; diff --git a/apps/cli/src/commands/vanity-subdomains/get/get.handler.ts b/apps/cli/src/commands/vanity-subdomains/get/get.handler.ts index 26974c69f7..bffb59fbc5 100644 --- a/apps/cli/src/commands/vanity-subdomains/get/get.handler.ts +++ b/apps/cli/src/commands/vanity-subdomains/get/get.handler.ts @@ -1,7 +1,7 @@ import { Effect, Option } from "effect"; import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; -import { ProjectRefResolver } from "../../../config/project-ref.service.ts"; +import { RemoteTarget } from "../../../config/remote-target.service.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { OutputFlag } from "../../../command-internal/global-flags.ts"; @@ -41,12 +41,12 @@ export const vanitySubdomainsGet = Effect.fn("vanity-subdomains.get")(function* const output = yield* Output; const outputFlag = yield* OutputFlag; const api = yield* CommandPlatformApi; - const resolver = yield* ProjectRefResolver; + const resolver = yield* RemoteTarget; const linkedProjectCache = yield* LinkedProjectCache; const telemetryState = yield* TelemetryState; yield* Effect.gen(function* () { - const ref = yield* resolver.resolve(flags.projectRef); + const { ref } = yield* resolver.resolve(flags.projectRef); yield* Effect.gen(function* () { const fetching = diff --git a/apps/cli/src/config/project-ref.layer.integration.test.ts b/apps/cli/src/config/project-ref.layer.integration.test.ts deleted file mode 100644 index 4d49ab1daa..0000000000 --- a/apps/cli/src/config/project-ref.layer.integration.test.ts +++ /dev/null @@ -1,123 +0,0 @@ -import type { V1ListAllProjectsOutput } from "@supabase/api/effect"; -import { BunServices } from "@effect/platform-bun"; -import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Option, Path } from "effect"; - -import { - VALID_REF, - mockCommandPlatformApi, - mockCommandSettings, - useTempWorkdir, -} from "../../tests/helpers/command-mocks.ts"; -import { mockOutput, mockTty } from "../../tests/helpers/mocks.ts"; -import { assertBlockDrift } from "../../tests/helpers/permission-drift.ts"; -import { projectRefLayer, projectRefPicker } from "./project-ref.layer.ts"; -import { ProjectRefResolver } from "./project-ref.service.ts"; - -type Projects = typeof V1ListAllProjectsOutput.Type; - -const PROJECT: Projects[number] = { - id: VALID_REF, - ref: VALID_REF, - organization_id: "org-123", - organization_slug: "acme", - name: "alpha", - region: "us-east-1", - created_at: "2026-05-27T01:02:03Z", - status: "ACTIVE_HEALTHY", - database: { - host: "db.alpha.supabase.co", - version: "15.1", - postgres_engine: "15", - release_channel: "ga", - }, -}; - -describe("projectRefLayer Management API requests", () => { - const workdir = useTempWorkdir("project-ref-api-"); - - function setup(opts: { readonly projectId?: string }) { - const api = mockCommandPlatformApi({ response: { status: 200, body: [PROJECT] } }); - const output = mockOutput({ promptSelectResponses: [VALID_REF] }); - const layer = projectRefLayer.pipe( - Layer.provide( - mockCommandSettings({ - workdir: workdir.current, - projectId: opts.projectId === undefined ? Option.none() : Option.some(opts.projectId), - }), - ), - Layer.provide(mockTty({ stdinIsTty: true })), - Layer.provide(output.layer), - Layer.provide(api.factoryLayer), - Layer.provide(BunServices.layer), - ); - return { api, layer }; - } - - it.effect("lists projects for the interactive picker when no ref is configured", () => { - const { api, layer } = setup({}); - return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); - expect(ref).toBe(VALID_REF); - expect(api.requests.map(({ method, url }) => `${method} ${url}`)).toEqual([ - "GET https://api.supabase.com/v1/projects", - ]); - assertBlockDrift(projectRefPicker, { - activeFlags: {}, - requests: api.requests, - }); - }).pipe(Effect.provide(layer)); - }); - - it.effect("lists projects for the picker when --project-ref is empty", () => { - const { api, layer } = setup({}); - return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.some("")); - expect(ref).toBe(VALID_REF); - expect(api.requests.map(({ method, url }) => `${method} ${url}`)).toEqual([ - "GET https://api.supabase.com/v1/projects", - ]); - assertBlockDrift(projectRefPicker, { - activeFlags: { "project-ref": "" }, - requests: api.requests, - }); - }).pipe(Effect.provide(layer)); - }); - - it.effect("requests nothing when --project-ref names the project", () => { - const { api, layer } = setup({}); - return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.some(VALID_REF)); - expect(ref).toBe(VALID_REF); - expect(api.requests).toEqual([]); - }).pipe(Effect.provide(layer)); - }); - - it.effect("requests nothing when SUPABASE_PROJECT_ID names the project", () => { - const { api, layer } = setup({ projectId: VALID_REF }); - return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); - expect(ref).toBe(VALID_REF); - expect(api.requests).toEqual([]); - }).pipe(Effect.provide(layer)); - }); - - it.effect("requests nothing when supabase/.temp/project-ref links the project", () => { - const { api, layer } = setup({}); - return Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const tempDir = path.join(workdir.current, "supabase", ".temp"); - yield* fs.makeDirectory(tempDir, { recursive: true }); - yield* fs.writeFileString(path.join(tempDir, "project-ref"), VALID_REF); - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); - expect(ref).toBe(VALID_REF); - expect(api.requests).toEqual([]); - }).pipe(Effect.provide([layer, BunServices.layer])); - }); -}); diff --git a/apps/cli/src/config/project-ref.layer.ts b/apps/cli/src/config/project-ref.layer.ts deleted file mode 100644 index cef3f5c181..0000000000 --- a/apps/cli/src/config/project-ref.layer.ts +++ /dev/null @@ -1,155 +0,0 @@ -import { Effect, FileSystem, Layer, Option, Path } from "effect"; - -import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; -import { Output } from "../shared/output/output.service.ts"; -import { Tty } from "../shared/runtime/tty.service.ts"; -import { readProjectRefFile } from "../command-internal/temp-paths.ts"; -import { CommandSettings } from "./command-settings.service.ts"; -import { - InvalidProjectRefError, - ProjectRefNotLinkedError, - ProjectRefRequiredError, -} from "./project-ref.errors.ts"; -import { - INVALID_PROJECT_REF_MESSAGE, - ProjectRefResolver, - PROJECT_NOT_LINKED_MESSAGE, - PROJECT_REF_PATTERN, -} from "./project-ref.service.ts"; -import { block } from "../command-internal/command-permissions/model.ts"; - -/** `ProjectRefResolver`'s interactive picker, reached only when no ref comes from a flag, env or the linked workdir. */ -export const projectRefPicker = block("project-ref", [ - { - operationId: "v1-list-all-projects", - kind: "required", - when: [{ flag: "project-ref", present: false }], - context: - "an interactive terminal (stdin and stdout are TTYs, output format not json), no SUPABASE_PROJECT_ID and no supabase/.temp/project-ref (`link` ignores that file)", - }, -]); - -function assertValid(ref: string): Effect.Effect<string, InvalidProjectRefError> { - if (PROJECT_REF_PATTERN.test(ref)) { - return Effect.succeed(ref); - } - return Effect.fail(new InvalidProjectRefError({ ref, message: INVALID_PROJECT_REF_MESSAGE })); -} - -export const projectRefLayer = Layer.effect( - ProjectRefResolver, - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const cliSettings = yield* CommandSettings; - const tty = yield* Tty; - const output = yield* Output; - const platformApi = yield* CommandPlatformApiFactory; - - const readRefFile = readProjectRefFile(fs, path, cliSettings.workdir); - - const promptForProjectRef = Effect.fnUntraced(function* (title: string) { - const api = yield* platformApi.make.pipe( - Effect.mapError( - (cause) => - new ProjectRefNotLinkedError({ - message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: failed to retrieve projects: ${String( - cause, - )}`, - }), - ), - ); - const projects = yield* api.v1.listAllProjects().pipe( - Effect.mapError( - (cause) => - new ProjectRefNotLinkedError({ - message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: failed to retrieve projects: ${String( - cause, - )}`, - }), - ), - ); - const options = projects.map((project) => ({ - value: project.id, - label: project.id, - hint: `name: ${project.name}, org: ${project.organization_slug}, region: ${project.region}`, - })); - const chosen = yield* output.promptSelect(title, options).pipe( - Effect.mapError( - (cause) => - new ProjectRefNotLinkedError({ - message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: ${cause.detail}`, - }), - ), - ); - // In text mode `output.info` writes to stderr; in json/stream-json modes it's a no-op. - yield* output.info(`Selected project: ${chosen}`); - return chosen; - }); - - return ProjectRefResolver.of({ - resolve: (flagValue) => - Effect.gen(function* () { - if (Option.isSome(flagValue) && flagValue.value.length > 0) { - return yield* assertValid(flagValue.value); - } - if (Option.isSome(cliSettings.projectId)) { - return yield* assertValid(cliSettings.projectId.value); - } - const fileValue = yield* readRefFile; - if (Option.isSome(fileValue)) { - return yield* assertValid(fileValue.value); - } - if (tty.stdinIsTty && output.interactive) { - const chosen = yield* promptForProjectRef("Select a project:"); - return yield* assertValid(chosen); - } - return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); - }), - resolveForLink: (flagValue) => - Effect.gen(function* () { - if (Option.isSome(flagValue) && flagValue.value.length > 0) { - return yield* assertValid(flagValue.value); - } - if (Option.isSome(cliSettings.projectId)) { - return yield* assertValid(cliSettings.projectId.value); - } - // `resolveForLink` skips the ref-file fallback that `resolve` uses. - if (tty.stdinIsTty && output.interactive) { - const chosen = yield* promptForProjectRef("Select a project:"); - return yield* assertValid(chosen); - } - return yield* new ProjectRefRequiredError({ - message: `required flag(s) "project-ref" not set`, - }); - }), - resolveOptional: (flagValue) => - Effect.gen(function* () { - if (Option.isSome(flagValue) && flagValue.value.length > 0) { - return Option.some(flagValue.value); - } - if (Option.isSome(cliSettings.projectId)) { - return cliSettings.projectId; - } - // A ref-file read error degrades to "not linked" here, unlike `resolve`/ - // `loadProjectRef`, since `projects list` only uses the value as a display marker. - return yield* readRefFile.pipe(Effect.orElseSucceed(() => Option.none<string>())); - }), - loadProjectRef: (flagValue) => - Effect.gen(function* () { - if (Option.isSome(flagValue) && flagValue.value.length > 0) { - return yield* assertValid(flagValue.value); - } - if (Option.isSome(cliSettings.projectId)) { - return yield* assertValid(cliSettings.projectId.value); - } - const fileValue = yield* readRefFile; - if (Option.isSome(fileValue)) { - return yield* assertValid(fileValue.value); - } - return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); - }), - promptProjectRef: promptForProjectRef, - }); - }), -); diff --git a/apps/cli/src/config/project-ref.service.ts b/apps/cli/src/config/project-ref.service.ts deleted file mode 100644 index 7a1783bc1a..0000000000 --- a/apps/cli/src/config/project-ref.service.ts +++ /dev/null @@ -1,77 +0,0 @@ -import type { Effect, Option } from "effect"; -import { Context } from "effect"; - -import { BRANCH_PROJECT_REF_PATTERN } from "../command-internal/ref-patterns.ts"; -import type { ProjectRefReadError } from "../command-internal/temp-paths.ts"; -import type { - InvalidProjectRefError, - ProjectRefNotLinkedError, - ProjectRefRequiredError, -} from "./project-ref.errors.ts"; - -interface ProjectRefResolverShape { - readonly resolve: ( - flagValue: Option.Option<string>, - ) => Effect.Effect< - string, - ProjectRefNotLinkedError | InvalidProjectRefError | ProjectRefReadError, - never - >; - /** - * Resolution chain used by `supabase link`, skipping the on-disk `project-ref` file: - * flag → `cliSettings.projectId` (env `SUPABASE_PROJECT_ID`) → (TTY) prompt. Fails with - * `ProjectRefRequiredError` on a non-TTY when neither is set, matching cobra's - * `required flag(s) "project-ref" not set` wording. - */ - readonly resolveForLink: ( - flagValue: Option.Option<string>, - ) => Effect.Effect< - string, - ProjectRefNotLinkedError | InvalidProjectRefError | ProjectRefRequiredError, - never - >; - /** - * Soft resolution chain (flag → `cliSettings.projectId` → ref file), with no prompt and no - * failure; returns `None` when nothing resolves. Unlike `resolve`, the value is not - * format-validated — safe only because every caller treats it as a display marker, never - * an API input; a caller needing validation must do it itself. - */ - readonly resolveOptional: ( - flagValue: Option.Option<string>, - ) => Effect.Effect<Option.Option<string>, never, never>; - /** - * Non-prompting resolution chain (flag → `cliSettings.projectId` → ref file) that fails - * hard with `ProjectRefNotLinkedError` when nothing resolves, with ref-format validation. - * Used by the `--linked` PreRun of the `db` command family, so a run with a token but no - * linked-project file fails fast instead of opening a project picker. - */ - readonly loadProjectRef: ( - flagValue: Option.Option<string>, - ) => Effect.Effect< - string, - ProjectRefNotLinkedError | InvalidProjectRefError | ProjectRefReadError, - never - >; - /** - * Lists all projects and prompts the user to select one with the given title, - * writing "Selected project: <ref>" to stderr (text mode). The `title` lets - * callers set their own per-command prompt label (e.g. `projects delete` - * uses "Which project do you want to delete?"). Used on a TTY when no - * positional ref is supplied; never reads the linked ref file. - */ - readonly promptProjectRef: ( - title: string, - ) => Effect.Effect<string, ProjectRefNotLinkedError, never>; -} - -export class ProjectRefResolver extends Context.Service< - ProjectRefResolver, - ProjectRefResolverShape ->()("supabase/cli/ProjectRefResolver") {} - -export const PROJECT_REF_PATTERN = BRANCH_PROJECT_REF_PATTERN; - -export const PROJECT_NOT_LINKED_MESSAGE = "Cannot find project ref. Have you run supabase link?"; - -export const INVALID_PROJECT_REF_MESSAGE = - "Invalid project ref format. Must be like `abcdefghijklmnopqrst`."; diff --git a/apps/cli/src/config/remote-target.layer.integration.test.ts b/apps/cli/src/config/remote-target.layer.integration.test.ts new file mode 100644 index 0000000000..11312fb0ad --- /dev/null +++ b/apps/cli/src/config/remote-target.layer.integration.test.ts @@ -0,0 +1,255 @@ +import type { V1ListAllProjectsOutput } from "@supabase/api/effect"; +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; + +import { + VALID_REF, + mockCommandPlatformApi, + mockCommandSettings, + useTempWorkdir, +} from "../../tests/helpers/command-mocks.ts"; +import { mockOutput, mockTty } from "../../tests/helpers/mocks.ts"; +import { assertBlockDrift } from "../../tests/helpers/permission-drift.ts"; +import { writeProjectLink } from "../command-internal/project-link.ts"; +import { projectStateModeLayer } from "../command-internal/project-state-mode.ts"; +import { remoteTargetLayer, projectRefPicker } from "./remote-target.layer.ts"; +import { RemoteTarget } from "./remote-target.service.ts"; + +type Projects = typeof V1ListAllProjectsOutput.Type; + +const PROJECT: Projects[number] = { + id: VALID_REF, + ref: VALID_REF, + organization_id: "org-123", + organization_slug: "acme", + name: "alpha", + region: "us-east-1", + created_at: "2026-05-27T01:02:03Z", + status: "ACTIVE_HEALTHY", + database: { + host: "db.alpha.supabase.co", + version: "15.1", + postgres_engine: "15", + release_channel: "ga", + }, +}; + +describe("remoteTargetLayer Management API requests", () => { + const workdir = useTempWorkdir("project-ref-api-"); + + function setup(opts: { readonly projectId?: string }) { + const api = mockCommandPlatformApi({ response: { status: 200, body: [PROJECT] } }); + const output = mockOutput({ promptSelectResponses: [VALID_REF] }); + const layer = remoteTargetLayer.pipe( + Layer.provide( + mockCommandSettings({ + workdir: workdir.current, + projectId: opts.projectId === undefined ? Option.none() : Option.some(opts.projectId), + }), + ), + Layer.provide(mockTty({ stdinIsTty: true })), + Layer.provide(output.layer), + Layer.provide(api.factoryLayer), + Layer.provide(BunServices.layer), + ); + return { api, layer }; + } + + it.effect("lists projects for the interactive picker when no ref is configured", () => { + const { api, layer } = setup({}); + return Effect.gen(function* () { + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); + expect(ref).toBe(VALID_REF); + expect(api.requests.map(({ method, url }) => `${method} ${url}`)).toEqual([ + "GET https://api.supabase.com/v1/projects", + ]); + assertBlockDrift(projectRefPicker, { + activeFlags: {}, + requests: api.requests, + }); + }).pipe(Effect.provide(layer)); + }); + + it.effect("lists projects for the picker when --project-ref is empty", () => { + const { api, layer } = setup({}); + return Effect.gen(function* () { + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.some("")); + expect(ref).toBe(VALID_REF); + expect(api.requests.map(({ method, url }) => `${method} ${url}`)).toEqual([ + "GET https://api.supabase.com/v1/projects", + ]); + assertBlockDrift(projectRefPicker, { + activeFlags: { "project-ref": "" }, + requests: api.requests, + }); + }).pipe(Effect.provide(layer)); + }); + + it.effect("requests nothing when --project-ref names the project", () => { + const { api, layer } = setup({}); + return Effect.gen(function* () { + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.some(VALID_REF)); + expect(ref).toBe(VALID_REF); + expect(api.requests).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + it.effect("requests nothing when SUPABASE_PROJECT_ID names the project", () => { + const { api, layer } = setup({ projectId: VALID_REF }); + return Effect.gen(function* () { + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); + expect(ref).toBe(VALID_REF); + expect(api.requests).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + + it.effect("requests nothing when supabase/.temp/project-ref links the project", () => { + const { api, layer } = setup({}); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const tempDir = path.join(workdir.current, "supabase", ".temp"); + yield* fs.makeDirectory(tempDir, { recursive: true }); + yield* fs.writeFileString(path.join(tempDir, "project-ref"), VALID_REF); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); + expect(ref).toBe(VALID_REF); + expect(api.requests).toEqual([]); + }).pipe(Effect.provide([layer, BunServices.layer])); + }); +}); + +describe("remoteTargetLayer in stack mode", () => { + const workdir = useTempWorkdir("remote-target-stack-"); + const FLAG_REF = "flagflagflagflagflag"; + const ENV_REF = "zyxwvutsrqponmlkjihg"; + const BRANCH_REF = "branchbranchbranchbr"; + const POOLER_URL = `postgres://postgres.${BRANCH_REF}:[YOUR-PASSWORD]@aws-0-us-east-1.pooler.supabase.com:6543/postgres`; + const LINKED_POOLER_URL = POOLER_URL.replace(BRANCH_REF, VALID_REF); + const RELINKED_POOLER_URL = POOLER_URL.replace(BRANCH_REF, ENV_REF); + + const linkWith = (activeBranch: { ref: string; name?: string; pooler_url?: string }) => + writeProjectLink(workdir.current, { + project: { ref: VALID_REF }, + active_branch: activeBranch, + fetchedAt: "2026-01-01T00:00:00.000Z", + }).pipe(Effect.provide(BunServices.layer)); + + const inStackMode = <A, E>( + effect: Effect.Effect<A, E, RemoteTarget>, + projectId?: string, + ): Effect.Effect<A, E> => { + const api = mockCommandPlatformApi({ response: { status: 200, body: [] } }); + const layer = remoteTargetLayer.pipe( + Layer.provide( + mockCommandSettings({ + workdir: workdir.current, + projectId: projectId === undefined ? Option.none() : Option.some(projectId), + }), + ), + Layer.provide(mockTty({ stdinIsTty: false })), + Layer.provide(mockOutput().layer), + Layer.provide(api.factoryLayer), + Layer.provide(BunServices.layer), + ); + return effect.pipe( + Effect.provide(layer.pipe(Layer.provideMerge(projectStateModeLayer("stack")))), + ); + }; + + const resolveWith = (flag: Option.Option<string>, projectId?: string) => + inStackMode( + Effect.gen(function* () { + const remoteTarget = yield* RemoteTarget; + return yield* remoteTarget.resolve(flag); + }), + projectId, + ); + + it.effect("resolves the flag before the env and the link", () => + Effect.gen(function* () { + yield* linkWith({ ref: VALID_REF }); + const target = yield* resolveWith(Option.some(FLAG_REF), ENV_REF); + expect(target).toMatchObject({ ref: FLAG_REF, source: "flag" }); + }), + ); + + it.effect("resolves the env before the link", () => + Effect.gen(function* () { + yield* linkWith({ ref: VALID_REF }); + const target = yield* resolveWith(Option.none(), ENV_REF); + expect(target).toMatchObject({ ref: ENV_REF, source: "env" }); + }), + ); + + it.effect("resolves the link when no flag or env names a project", () => + Effect.gen(function* () { + yield* linkWith({ ref: VALID_REF }); + const target = yield* resolveWith(Option.none()); + expect(target).toEqual({ ref: VALID_REF, source: "link" }); + }), + ); + + it.effect("reports a linked branch with its parent project and name", () => + Effect.gen(function* () { + yield* linkWith({ ref: BRANCH_REF, name: "feature-x" }); + const { target, parent } = yield* inStackMode( + Effect.gen(function* () { + const remoteTarget = yield* RemoteTarget; + return { + target: yield* remoteTarget.resolve(Option.none()), + parent: yield* remoteTarget.parent, + }; + }), + ); + expect(target).toEqual({ + ref: BRANCH_REF, + source: "link", + parentRef: VALID_REF, + branchName: "feature-x", + }); + expect(parent).toEqual({ kind: "resolved", ref: VALID_REF, source: "link" }); + }), + ); + + it.effect("carries the link's pooler URL to the linked ref only", () => + Effect.gen(function* () { + yield* linkWith({ ref: BRANCH_REF, name: "feature-x", pooler_url: POOLER_URL }); + const { linked, other } = yield* inStackMode( + Effect.gen(function* () { + const remoteTarget = yield* RemoteTarget; + return { + linked: yield* remoteTarget.load(Option.some(BRANCH_REF)), + other: yield* remoteTarget.load(Option.some(FLAG_REF)), + }; + }), + ); + expect(linked.poolerUrl).toBe(POOLER_URL); + expect(other).toEqual({ ref: FLAG_REF, source: "flag" }); + }), + ); + + for (const entryPoint of ["resolve", "load"] as const) { + it.effect(`${entryPoint} returns its first result after the link changes mid-command`, () => + Effect.gen(function* () { + yield* linkWith({ ref: VALID_REF, pooler_url: LINKED_POOLER_URL }); + const { first, second } = yield* inStackMode( + Effect.gen(function* () { + const remoteTarget = yield* RemoteTarget; + const first = yield* remoteTarget[entryPoint](Option.none()); + yield* linkWith({ ref: ENV_REF, pooler_url: RELINKED_POOLER_URL }); + const second = yield* remoteTarget[entryPoint](Option.none()); + return { first, second }; + }), + ); + expect(first).toEqual({ ref: VALID_REF, source: "link", poolerUrl: LINKED_POOLER_URL }); + expect(second).toEqual(first); + }), + ); + } +}); diff --git a/apps/cli/src/config/remote-target.layer.ts b/apps/cli/src/config/remote-target.layer.ts new file mode 100644 index 0000000000..91eebb26ac --- /dev/null +++ b/apps/cli/src/config/remote-target.layer.ts @@ -0,0 +1,277 @@ +import { Cache, Effect, FileSystem, Layer, Option, Path } from "effect"; + +import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; +import { block } from "../command-internal/command-permissions/model.ts"; +import { + type LinkSnapshot, + readLinkSnapshot, + readLinkSnapshotSoft, +} from "../command-internal/link-snapshot.ts"; +import { Output } from "../shared/output/output.service.ts"; +import { Tty } from "../shared/runtime/tty.service.ts"; +import { CommandSettings } from "./command-settings.service.ts"; +import { + InvalidProjectRefError, + ProjectRefNotLinkedError, + ProjectRefRequiredError, +} from "./project-ref.errors.ts"; +import { + INVALID_PROJECT_REF_MESSAGE, + type ParentRefResolution, + PROJECT_NOT_LINKED_MESSAGE, + PROJECT_REF_PATTERN, + RemoteTarget, + type RemoteTargetSource, + type ResolvedRemoteTarget, +} from "./remote-target.service.ts"; + +/** `RemoteTarget`'s interactive picker, reached only when no ref comes from a flag, env or the link. */ +export const projectRefPicker = block("project-ref", [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "project-ref", present: false }], + context: + "an interactive terminal (stdin and stdout are TTYs, output format not json), no SUPABASE_PROJECT_ID and no linked project in supabase/.link.json or supabase/.temp (`link` ignores the link)", + }, +]); + +const CACHE_CAPACITY = 8; + +function assertValid(ref: string): Effect.Effect<string, InvalidProjectRefError> { + if (PROJECT_REF_PATTERN.test(ref)) { + return Effect.succeed(ref); + } + return Effect.fail(new InvalidProjectRefError({ ref, message: INVALID_PROJECT_REF_MESSAGE })); +} + +const nonEmpty = (value: Option.Option<string>): Option.Option<string> => + Option.filter(value, (text) => text.length > 0); + +/** Attaches the link's metadata when `ref` is the linked ref, since it describes only that ref. */ +function targetFor( + ref: string, + source: RemoteTargetSource, + snapshot: LinkSnapshot, +): ResolvedRemoteTarget { + if (!Option.exists(snapshot.ref, (linkedRef) => linkedRef === ref)) return { ref, source }; + const parentRef = Option.filter( + Option.map(snapshot.parent, (parent) => parent.ref), + (parent) => parent !== ref, + ); + const poolerUrl = nonEmpty(snapshot.poolerUrl); + return { + ref, + source, + ...(Option.isSome(parentRef) + ? { + parentRef: parentRef.value, + ...(Option.isSome(snapshot.branchName) ? { branchName: snapshot.branchName.value } : {}), + } + : {}), + ...(Option.isSome(poolerUrl) ? { poolerUrl: poolerUrl.value } : {}), + }; +} + +export const remoteTargetLayer = Layer.effect( + RemoteTarget, + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const cliSettings = yield* CommandSettings; + const tty = yield* Tty; + const output = yield* Output; + const platformApi = yield* CommandPlatformApiFactory; + + const readSnapshot = readLinkSnapshot(fs, path, cliSettings.workdir); + const readSnapshotSoft = readLinkSnapshotSoft(fs, path, cliSettings.workdir); + const envRef = nonEmpty(cliSettings.projectId); + + const prompt = Effect.fn("RemoteTarget.prompt")(function* (title: string) { + const api = yield* platformApi.make.pipe( + Effect.mapError( + (cause) => + new ProjectRefNotLinkedError({ + message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: failed to retrieve projects: ${String( + cause, + )}`, + }), + ), + ); + const projects = yield* api.v1.listAllProjects().pipe( + Effect.mapError( + (cause) => + new ProjectRefNotLinkedError({ + message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: failed to retrieve projects: ${String( + cause, + )}`, + }), + ), + ); + const options = projects.map((project) => ({ + value: project.id, + label: project.id, + hint: `name: ${project.name}, org: ${project.organization_slug}, region: ${project.region}`, + })); + const chosen = yield* output.promptSelect(title, options).pipe( + Effect.mapError( + (cause) => + new ProjectRefNotLinkedError({ + message: `${PROJECT_NOT_LINKED_MESSAGE}\n Reason: ${cause.detail}`, + }), + ), + ); + // In text mode `output.info` writes to stderr; in json/stream-json modes it's a no-op. + yield* output.info(`Selected project: ${chosen}`); + return chosen; + }); + + const canPrompt = tty.stdinIsTty && output.interactive; + + const resolveSources = Effect.fn("RemoteTarget.resolve")(function* ( + flagKey: string, + withPicker: boolean, + ) { + yield* Effect.annotateCurrentSpan("remote_target.picker", withPicker); + const done = Effect.fnUntraced(function* (target: ResolvedRemoteTarget) { + yield* Effect.annotateCurrentSpan({ + "remote_target.source": target.source, + "project.ref": target.ref, + "remote_target.is_branch": target.parentRef !== undefined, + }); + return target; + }); + if (flagKey.length > 0) { + const ref = yield* assertValid(flagKey); + return yield* done(targetFor(ref, "flag", yield* readSnapshotSoft)); + } + if (Option.isSome(envRef)) { + const ref = yield* assertValid(envRef.value); + return yield* done(targetFor(ref, "env", yield* readSnapshotSoft)); + } + const snapshot = yield* readSnapshot; + if (Option.isSome(snapshot.ref)) { + const ref = yield* assertValid(snapshot.ref.value); + return yield* done(targetFor(ref, "link", snapshot)); + } + if (withPicker && canPrompt) { + const chosen = yield* prompt("Select a project:"); + const ref = yield* assertValid(chosen); + return yield* done({ ref, source: "flag" }); + } + return yield* new ProjectRefNotLinkedError({ message: PROJECT_NOT_LINKED_MESSAGE }); + }); + + const flagKeyOf = (flagValue: Option.Option<string>): string => + Option.getOrElse(nonEmpty(flagValue), () => ""); + + // One cache per entry point: a command reaching the same target twice reads the link once. + const resolveCache = yield* Cache.make({ + capacity: CACHE_CAPACITY, + lookup: (flagKey: string) => resolveSources(flagKey, true), + }); + const loadCache = yield* Cache.make({ + capacity: CACHE_CAPACITY, + lookup: (flagKey: string) => resolveSources(flagKey, false), + }); + + const resolve = (flagValue: Option.Option<string>) => + Cache.get(resolveCache, flagKeyOf(flagValue)); + const load = (flagValue: Option.Option<string>) => Cache.get(loadCache, flagKeyOf(flagValue)); + + const resolveForLink = Effect.fn("RemoteTarget.resolveForLink")(function* ( + flagValue: Option.Option<string>, + ) { + const flagRef = nonEmpty(flagValue); + if (Option.isSome(flagRef)) { + return { ref: yield* assertValid(flagRef.value), source: "flag" } as const; + } + if (Option.isSome(envRef)) { + return { ref: yield* assertValid(envRef.value), source: "env" } as const; + } + if (canPrompt) { + const chosen = yield* prompt("Select a project:"); + return { ref: yield* assertValid(chosen), source: "flag" } as const; + } + return yield* new ProjectRefRequiredError({ + message: `required flag(s) "project-ref" not set`, + }); + }); + + const resolveOptional = Effect.fn("RemoteTarget.resolveOptional")(function* ( + flagValue: Option.Option<string>, + ) { + const flagRef = nonEmpty(flagValue); + if (Option.isSome(flagRef)) { + return Option.some(targetFor(flagRef.value, "flag", yield* readSnapshotSoft)); + } + if (Option.isSome(envRef)) { + return Option.some(targetFor(envRef.value, "env", yield* readSnapshotSoft)); + } + const snapshot = yield* readSnapshotSoft; + return Option.map(snapshot.ref, (ref) => targetFor(ref, "link", snapshot)); + }); + + const parent = Effect.gen(function* () { + const snapshot = yield* readSnapshotSoft; + const linkedRef = snapshot.ref; + // The recorded parent alone is not proof a link completed, since `link` writes it on + // failure as well, so it counts only when the linked ref confirms the link. + const recordedParent = Option.isSome(linkedRef) + ? Option.map(snapshot.parent, (record) => record.ref) + : Option.none<string>(); + // An env value restating the linked ref carries no parent information. Only a valid value + // is dropped, so a malformed env equal to a malformed link still reads as invalid. + const env = + Option.isSome(envRef) && + Option.isSome(linkedRef) && + envRef.value === linkedRef.value && + PROJECT_REF_PATTERN.test(envRef.value) + ? Option.none<string>() + : envRef; + const candidates = [ + [env, "env"], + [recordedParent, "link"], + [linkedRef, "link"], + ] as const; + // The first present candidate decides: a malformed higher-priority one is `invalid` + // instead of falling through to a lower-priority project, which could run a mutation + // against the wrong project. + for (const [candidate, source] of candidates) { + if (Option.isSome(candidate)) { + return PROJECT_REF_PATTERN.test(candidate.value) + ? ({ kind: "resolved", ref: candidate.value, source } as const) + : ({ kind: "invalid" } as const); + } + } + return { kind: "absent" } as const satisfies ParentRefResolution; + }).pipe(Effect.withSpan("RemoteTarget.parent")); + + const resolveParentScoped = Effect.fn("RemoteTarget.resolveParentScoped")(function* ( + flagValue: Option.Option<string>, + ) { + if (Option.isSome(nonEmpty(flagValue))) { + yield* Effect.annotateCurrentSpan("remote_target.parent_scope", "flag"); + return yield* resolve(flagValue); + } + const linkedParent = yield* parent; + yield* Effect.annotateCurrentSpan("remote_target.linked_parent", linkedParent.kind); + if (linkedParent.kind === "resolved") { + yield* Effect.annotateCurrentSpan("remote_target.parent_scope", "linked_parent"); + return { ref: linkedParent.ref, source: linkedParent.source } as const; + } + yield* Effect.annotateCurrentSpan("remote_target.parent_scope", "resolver"); + return yield* resolve(Option.none()); + }); + + return RemoteTarget.of({ + resolve, + load, + resolveForLink, + resolveOptional, + prompt, + parent, + resolveParentScoped, + }); + }), +); diff --git a/apps/cli/src/config/project-ref.layer.unit.test.ts b/apps/cli/src/config/remote-target.layer.unit.test.ts similarity index 84% rename from apps/cli/src/config/project-ref.layer.unit.test.ts rename to apps/cli/src/config/remote-target.layer.unit.test.ts index 27e937141c..3a31b38fe1 100644 --- a/apps/cli/src/config/project-ref.layer.unit.test.ts +++ b/apps/cli/src/config/remote-target.layer.unit.test.ts @@ -8,8 +8,8 @@ import { CommandPlatformApi } from "../auth/command-platform-api.service.ts"; import { useTempWorkdir } from "../../tests/helpers/command-mocks.ts"; import { mockOutput, mockTty } from "../../tests/helpers/mocks.ts"; import { CommandSettings } from "./command-settings.service.ts"; -import { ProjectRefResolver } from "./project-ref.service.ts"; -import { projectRefLayer } from "./project-ref.layer.ts"; +import { RemoteTarget } from "./remote-target.service.ts"; +import { remoteTargetLayer } from "./remote-target.layer.ts"; const VALID_REF = "abcdefghijklmnopqrst"; const ANOTHER_REF = "qrstuvwxyzabcdefghij"; @@ -69,7 +69,7 @@ function makeLayer(opts: { format: opts.format ?? "text", promptSelectResponses: opts.promptSelectResponses, }); - const layer = projectRefLayer.pipe( + const layer = remoteTargetLayer.pipe( Layer.provide(mockCliSettings(opts)), Layer.provide(mockTty({ stdinIsTty: opts.stdinIsTty ?? false, stdoutIsTty: false })), Layer.provide(out.layer), @@ -93,13 +93,13 @@ const writeRefFile = Effect.fnUntraced(function* (workdir: string, content: stri yield* fs.writeFileString(path.join(tempDir, "project-ref"), content); }); -describe("projectRefLayer", () => { +describe("remoteTargetLayer", () => { it.effect("prefers --project-ref flag over env and file", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: ANOTHER_REF }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, ANOTHER_REF); - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.some(VALID_REF)); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.some(VALID_REF)); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -108,8 +108,8 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: VALID_REF }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, ANOTHER_REF); - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -118,8 +118,8 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, VALID_REF); - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -128,8 +128,8 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, ` ${VALID_REF}\n\n`); - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -146,8 +146,8 @@ describe("projectRefLayer", () => { promptSelectResponses: [ANOTHER_REF], }); return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; - const ref = yield* resolve(Option.none()); + const { resolve } = yield* RemoteTarget; + const { ref } = yield* resolve(Option.none()); expect(ref).toBe(ANOTHER_REF); const call = out.promptSelectCalls[0]; expect(call?.message).toBe("Select a project:"); @@ -176,7 +176,7 @@ describe("projectRefLayer", () => { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const refPath = path.join(tempRoot.current, "supabase", ".temp", "project-ref"); - const { resolve } = yield* ProjectRefResolver; + const { resolve } = yield* RemoteTarget; yield* resolve(Option.none()); expect(yield* fs.exists(refPath)).toBe(false); }).pipe(Effect.provide(layer)); @@ -185,7 +185,7 @@ describe("projectRefLayer", () => { it.effect("fails with ProjectRefNotLinkedError on non-TTY with no source", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; + const { resolve } = yield* RemoteTarget; const exit = yield* Effect.exit(resolve(Option.none())); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { @@ -199,7 +199,7 @@ describe("projectRefLayer", () => { it.effect("fails with InvalidProjectRefError when the resolved ref is malformed", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: "not-a-valid-ref" }); return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; + const { resolve } = yield* RemoteTarget; const exit = yield* Effect.exit(resolve(Option.none())); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { @@ -213,7 +213,7 @@ describe("projectRefLayer", () => { it.effect("rejects invalid ref from --project-ref flag", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { - const { resolve } = yield* ProjectRefResolver; + const { resolve } = yield* RemoteTarget; const exit = yield* Effect.exit(resolve(Option.some("BADREF"))); expect(Exit.isFailure(exit)).toBe(true); }).pipe(Effect.provide(layer)); @@ -223,7 +223,7 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, "BADREF"); - const { resolve } = yield* ProjectRefResolver; + const { resolve } = yield* RemoteTarget; const exit = yield* Effect.exit(resolve(Option.none())); expect(Exit.isFailure(exit)).toBe(true); }).pipe(Effect.provide(layer)); @@ -234,18 +234,18 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: ANOTHER_REF }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, ANOTHER_REF); - const { resolveOptional } = yield* ProjectRefResolver; + const { resolveOptional } = yield* RemoteTarget; const ref = yield* resolveOptional(Option.some(VALID_REF)); - expect(ref).toEqual(Option.some(VALID_REF)); + expect(Option.map(ref, (target) => target.ref)).toEqual(Option.some(VALID_REF)); }).pipe(Effect.provide(layer)); }); it.effect("falls back to projectId then the ref file", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: VALID_REF }); return Effect.gen(function* () { - const { resolveOptional } = yield* ProjectRefResolver; + const { resolveOptional } = yield* RemoteTarget; const ref = yield* resolveOptional(Option.none()); - expect(ref).toEqual(Option.some(VALID_REF)); + expect(Option.map(ref, (target) => target.ref)).toEqual(Option.some(VALID_REF)); }).pipe(Effect.provide(layer)); }); @@ -253,29 +253,29 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, VALID_REF); - const { resolveOptional } = yield* ProjectRefResolver; + const { resolveOptional } = yield* RemoteTarget; const ref = yield* resolveOptional(Option.none()); - expect(ref).toEqual(Option.some(VALID_REF)); + expect(Option.map(ref, (target) => target.ref)).toEqual(Option.some(VALID_REF)); }).pipe(Effect.provide(layer)); }); it.effect("returns None and never fails when nothing resolves", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { - const { resolveOptional } = yield* ProjectRefResolver; + const { resolveOptional } = yield* RemoteTarget; const ref = yield* resolveOptional(Option.none()); expect(Option.isNone(ref)).toBe(true); }).pipe(Effect.provide(layer)); }); }); - describe("loadProjectRef (Go flags.LoadProjectRef — non-prompting)", () => { + describe("load (Go flags.LoadProjectRef — non-prompting)", () => { it.effect("prefers flag, then projectId, then the ref file", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: ANOTHER_REF }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, ANOTHER_REF); - const { loadProjectRef } = yield* ProjectRefResolver; - expect(yield* loadProjectRef(Option.some(VALID_REF))).toBe(VALID_REF); + const { load } = yield* RemoteTarget; + expect((yield* load(Option.some(VALID_REF))).ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -283,8 +283,8 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, VALID_REF); - const { loadProjectRef } = yield* ProjectRefResolver; - expect(yield* loadProjectRef(Option.none())).toBe(VALID_REF); + const { load } = yield* RemoteTarget; + expect((yield* load(Option.none())).ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -298,8 +298,8 @@ describe("projectRefLayer", () => { promptSelectResponses: [VALID_REF], }); return Effect.gen(function* () { - const { loadProjectRef } = yield* ProjectRefResolver; - const exit = yield* Effect.exit(loadProjectRef(Option.none())); + const { load } = yield* RemoteTarget; + const exit = yield* Effect.exit(load(Option.none())); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { const errorJson = yield* encodeJson(exit.cause); @@ -313,8 +313,8 @@ describe("projectRefLayer", () => { it.effect("validates the resolved ref format", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: "not-a-valid-ref" }); return Effect.gen(function* () { - const { loadProjectRef } = yield* ProjectRefResolver; - const exit = yield* Effect.exit(loadProjectRef(Option.none())); + const { load } = yield* RemoteTarget; + const exit = yield* Effect.exit(load(Option.none())); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { expect(yield* encodeJson(exit.cause)).toContain("InvalidProjectRefError"); @@ -327,8 +327,8 @@ describe("projectRefLayer", () => { it.effect("prefers the --project-ref flag", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: ANOTHER_REF }); return Effect.gen(function* () { - const { resolveForLink } = yield* ProjectRefResolver; - const ref = yield* resolveForLink(Option.some(VALID_REF)); + const { resolveForLink } = yield* RemoteTarget; + const { ref } = yield* resolveForLink(Option.some(VALID_REF)); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -336,8 +336,8 @@ describe("projectRefLayer", () => { it.effect("uses SUPABASE_PROJECT_ID when the flag is unset", () => { const { layer } = makeLayer({ workdir: tempRoot.current, projectId: VALID_REF }); return Effect.gen(function* () { - const { resolveForLink } = yield* ProjectRefResolver; - const ref = yield* resolveForLink(Option.none()); + const { resolveForLink } = yield* RemoteTarget; + const { ref } = yield* resolveForLink(Option.none()); expect(ref).toBe(VALID_REF); }).pipe(Effect.provide(layer)); }); @@ -346,7 +346,7 @@ describe("projectRefLayer", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { yield* writeRefFile(tempRoot.current, VALID_REF); - const { resolveForLink } = yield* ProjectRefResolver; + const { resolveForLink } = yield* RemoteTarget; const exit = yield* Effect.exit(resolveForLink(Option.none())); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { @@ -368,8 +368,8 @@ describe("projectRefLayer", () => { promptSelectResponses: [VALID_REF], }); return Effect.gen(function* () { - const { resolveForLink } = yield* ProjectRefResolver; - const ref = yield* resolveForLink(Option.none()); + const { resolveForLink } = yield* RemoteTarget; + const { ref } = yield* resolveForLink(Option.none()); expect(ref).toBe(VALID_REF); expect(out.promptSelectCalls[0]?.message).toBe("Select a project:"); }).pipe(Effect.provide(layer)); @@ -378,7 +378,7 @@ describe("projectRefLayer", () => { it.effect("rejects an invalid --project-ref flag", () => { const { layer } = makeLayer({ workdir: tempRoot.current }); return Effect.gen(function* () { - const { resolveForLink } = yield* ProjectRefResolver; + const { resolveForLink } = yield* RemoteTarget; const exit = yield* Effect.exit(resolveForLink(Option.some("BADREF"))); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { @@ -388,7 +388,7 @@ describe("projectRefLayer", () => { }); }); - describe("promptProjectRef", () => { + describe("prompt", () => { it.effect("prompts with the given title, returns the choice, and echoes it", () => { const projects = [ { id: VALID_REF, name: "alpha", organization_slug: "acme", region: "us-east-1" }, @@ -401,8 +401,8 @@ describe("projectRefLayer", () => { promptSelectResponses: [ANOTHER_REF], }); return Effect.gen(function* () { - const { promptProjectRef } = yield* ProjectRefResolver; - const ref = yield* promptProjectRef("Which project do you want to delete?"); + const { prompt } = yield* RemoteTarget; + const ref = yield* prompt("Which project do you want to delete?"); expect(ref).toBe(ANOTHER_REF); expect(out.promptSelectCalls[0]?.message).toBe("Which project do you want to delete?"); const infos = out.messages.filter((m) => m.type === "info").map((m) => m.message); diff --git a/apps/cli/src/config/remote-target.service.ts b/apps/cli/src/config/remote-target.service.ts new file mode 100644 index 0000000000..70d09a3f07 --- /dev/null +++ b/apps/cli/src/config/remote-target.service.ts @@ -0,0 +1,113 @@ +import type { Effect, Option } from "effect"; +import { Context } from "effect"; + +import { BRANCH_PROJECT_REF_PATTERN } from "../command-internal/ref-patterns.ts"; +import type { LinkedProjectRefReadError } from "../command-internal/project-link.ts"; +import type { + InvalidProjectRefError, + ProjectRefNotLinkedError, + ProjectRefRequiredError, +} from "./project-ref.errors.ts"; + +/** Where a target's ref came from: the `--project-ref` flag, `SUPABASE_PROJECT_ID`, or the link. */ +export type RemoteTargetSource = "flag" | "env" | "link"; + +/** + * The hosted project a command acts on. `parentRef` and `branchName` are set when the ref is a + * linked branch, and `poolerUrl` when the link recorded an IPv4 pooler for the ref; all three + * describe the checkout's link, so they are absent for a flag or env ref that is not the linked + * one. + */ +export interface ResolvedRemoteTarget { + readonly ref: string; + readonly parentRef?: string; + readonly branchName?: string; + readonly poolerUrl?: string; + readonly source: RemoteTargetSource; +} + +/** The linked parent project, as seen from `SUPABASE_PROJECT_ID`, the link record, or the linked ref. */ +export type ParentRefResolution = + | { + readonly kind: "resolved"; + readonly ref: string; + readonly source: Exclude<RemoteTargetSource, "flag">; + } + | { readonly kind: "invalid" } + | { readonly kind: "absent" }; + +export type RemoteTargetError = + | ProjectRefNotLinkedError + | InvalidProjectRefError + | LinkedProjectRefReadError; + +interface RemoteTargetShape { + /** + * Resolves the target once per command: flag, then `SUPABASE_PROJECT_ID`, then the link, then + * (on a TTY) a project picker. Repeat calls with the same flag return the first result, so a + * `link` that lands mid-command cannot change a target already in use. A prompted ref counts + * as a `flag` source. + */ + readonly resolve: ( + flagValue: Option.Option<string>, + ) => Effect.Effect<ResolvedRemoteTarget, RemoteTargetError, never>; + /** + * {@link RemoteTargetShape.resolve} without the picker: fails with `ProjectRefNotLinkedError` + * when nothing resolves. Used by the `--linked` path of the `db` command family, so a run with + * a token but no link fails fast instead of opening a picker. + */ + readonly load: ( + flagValue: Option.Option<string>, + ) => Effect.Effect<ResolvedRemoteTarget, RemoteTargetError, never>; + /** + * Resolution used by `supabase link`, ignoring the existing link: flag, then + * `SUPABASE_PROJECT_ID`, then (on a TTY) the picker. Fails with `ProjectRefRequiredError` on a + * non-TTY when neither is set, matching cobra's `required flag(s) "project-ref" not set` wording. + */ + readonly resolveForLink: ( + flagValue: Option.Option<string>, + ) => Effect.Effect< + ResolvedRemoteTarget, + ProjectRefNotLinkedError | InvalidProjectRefError | ProjectRefRequiredError, + never + >; + /** + * Soft resolution (flag, env, link) with no picker and no failure; `None` when nothing + * resolves. The ref is not format-validated, so callers use it only as a display marker. + */ + readonly resolveOptional: ( + flagValue: Option.Option<string>, + ) => Effect.Effect<Option.Option<ResolvedRemoteTarget>, never, never>; + /** + * Lists all projects and prompts for one with the given title, writing "Selected project: <ref>" + * to stderr in text mode. Never reads the link. + */ + readonly prompt: (title: string) => Effect.Effect<string, ProjectRefNotLinkedError, never>; + /** + * The linked parent project rather than the branch's own ref. The first present candidate + * wins, in order: `SUPABASE_PROJECT_ID`, the recorded parent, the linked ref; a malformed + * candidate is `invalid` instead of falling through to a lower one. A recorded parent counts + * only when a linked ref confirms the link, and an env value restating the linked ref carries + * no parent information. + */ + readonly parent: Effect.Effect<ParentRefResolution, never, never>; + /** + * The target for the parent-scoped `branches` family, whose endpoints reject branch refs: an + * explicit flag wins, then the linked parent, then {@link RemoteTargetShape.resolve}. + */ + readonly resolveParentScoped: ( + flagValue: Option.Option<string>, + ) => Effect.Effect<ResolvedRemoteTarget, RemoteTargetError, never>; +} + +/** Resolves the hosted project a command acts on, once per command. */ +export class RemoteTarget extends Context.Service<RemoteTarget, RemoteTargetShape>()( + "supabase/cli/RemoteTarget", +) {} + +export const PROJECT_REF_PATTERN = BRANCH_PROJECT_REF_PATTERN; + +export const PROJECT_NOT_LINKED_MESSAGE = "Cannot find project ref. Have you run supabase link?"; + +export const INVALID_PROJECT_REF_MESSAGE = + "Invalid project ref format. Must be like `abcdefghijklmnopqrst`."; diff --git a/apps/cli/src/shared/cli/hidden-flag.unit.test.ts b/apps/cli/src/shared/cli/hidden-flag.unit.test.ts index 2c2e7c56c1..8779d23611 100644 --- a/apps/cli/src/shared/cli/hidden-flag.unit.test.ts +++ b/apps/cli/src/shared/cli/hidden-flag.unit.test.ts @@ -1,3 +1,4 @@ +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; import { Cause, Effect, Exit, FileSystem, Layer, Schema } from "effect"; import { BunServices } from "@effect/platform-bun"; import { CliOutput, Command, type HelpDoc } from "effect/cli"; @@ -60,6 +61,7 @@ const testRootLayer = ( Layer.succeed(CliArgs, { args }), mockOutput({ format: "text" }).layer, BunServices.layer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), mockRuntimeInfo(), mockAnalytics().layer, mockTelemetryRuntime(), diff --git a/apps/cli/src/shared/cli/run.ts b/apps/cli/src/shared/cli/run.ts index d9d952eb6b..3c8b17b127 100644 --- a/apps/cli/src/shared/cli/run.ts +++ b/apps/cli/src/shared/cli/run.ts @@ -24,6 +24,8 @@ import { CLI_VERSION } from "./version.ts"; import type { CliProjectHome } from "../config/cli-project-home.service.ts"; import type { CliSettings } from "../config/cli-settings.service.ts"; import type { ProjectLinkState } from "../config/project-link-state.service.ts"; +import { legacyProjectStateLayer } from "../../command-internal/legacy-project-state.layer.ts"; +import type { LegacyProjectState } from "../../command-internal/legacy-project-state.service.ts"; import { jsonCliOutputFormatter } from "../output/json-formatter.ts"; import { textCliOutputFormatter } from "../output/text-formatter.ts"; import { outputLayerFor } from "../output/output.layer.ts"; @@ -90,6 +92,7 @@ export type AllowedRunCliServices = | CommandRuntime | FileSystem.FileSystem | Crypto.Crypto + | LegacyProjectState | Path.Path | ProcessControl | ProjectLinkState @@ -610,6 +613,7 @@ function cliProgramFor< Layer.provideMerge(cliProjectHomeLayerFor(runtimeLayer)), Layer.provideMerge(cliProjectContextLayerFor(runtimeLayer)), Layer.provideMerge(projectLinkStateLayer), + Layer.provideMerge(legacyProjectStateLayer), Layer.provideMerge(runtimeLayer), Layer.provideMerge(fallbackCommandLayer), Layer.provideMerge(Layer.succeed(CliArgs, { args })), diff --git a/apps/cli/src/shared/compute/compute-paths.ts b/apps/cli/src/shared/compute/compute-paths.ts index f389e953ef..d42d636f6b 100644 --- a/apps/cli/src/shared/compute/compute-paths.ts +++ b/apps/cli/src/shared/compute/compute-paths.ts @@ -1,3 +1,4 @@ +import { LEGACY_PROJECT_STATE_DIRNAME } from "../../command-internal/legacy-project-state.paths.ts"; import { Effect, FileSystem, Option, Path } from "effect"; import { InvalidComputeSourceError } from "./compute.errors.ts"; @@ -16,7 +17,7 @@ const COMPUTE_DIR = "compute"; * may name one: `functions` and `migrations` belong to other parts of the CLI, * and `.temp` holds CLI state including the linked-project reference. */ -const RESERVED_SUPABASE_DIRS = ["functions", "migrations", ".temp"]; +const RESERVED_SUPABASE_DIRS = ["functions", "migrations", LEGACY_PROJECT_STATE_DIRNAME]; /** * Files directly under `supabase/` that the CLI owns. Refused separately from the diff --git a/apps/cli/src/shared/functions/deploy.ts b/apps/cli/src/shared/functions/deploy.ts index 393f3538cd..4c9d45e554 100644 --- a/apps/cli/src/shared/functions/deploy.ts +++ b/apps/cli/src/shared/functions/deploy.ts @@ -1,3 +1,4 @@ +import { scratchDir } from "../../command-internal/scratch-paths.ts"; import { brotliCompressSync, constants as zlibConstants } from "node:zlib"; import { URL } from "node:url"; import { @@ -1573,7 +1574,7 @@ const bundleFunctionWithDocker = Effect.fn("functions.deploy.bundleWithDocker")( const path = yield* Path.Path; yield* output.raw(`Bundling Function: ${styleEmphasis(config.slug)}\n`, "stderr"); - const outputRoot = path.resolve(functionsDir, "..", ".temp"); + const outputRoot = yield* scratchDir(path.resolve(functionsDir, "..", "..")); yield* fs .makeDirectory(outputRoot, { recursive: true }) .pipe(Effect.mapError(unknownHostError(outputRoot))); diff --git a/apps/cli/src/shared/functions/download.ts b/apps/cli/src/shared/functions/download.ts index a8699b36d2..c91cebd596 100644 --- a/apps/cli/src/shared/functions/download.ts +++ b/apps/cli/src/shared/functions/download.ts @@ -1,3 +1,5 @@ +import { scratchDir } from "../../command-internal/scratch-paths.ts"; +import { usesStackState } from "../../command-internal/stack-backend.ts"; import { BunPath } from "@effect/platform-bun"; import { operationDefinitions, SupabaseApiInputError, type ApiClient } from "@supabase/api/effect"; import { randomUUID } from "node:crypto"; @@ -965,26 +967,31 @@ const downloadWithDockerUnbundle = Effect.fn("functions.download.dockerUnbundle" const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const posixPath = yield* Effect.provide(Path.Path, BunPath.layerPosix); - const tempDir = path.join(dependencies.projectRoot, "supabase", ".temp"); - yield* fs.makeDirectory(tempDir, { recursive: true }).pipe( + const scratch = yield* scratchDir(dependencies.projectRoot); + yield* fs.makeDirectory(scratch, { recursive: true }).pipe( Effect.mapError( (error) => new FunctionEszipDownloadError({ - message: `failed to mkdir: ${hostErrorMessage(error, tempDir)}`, + message: `failed to mkdir: ${hostErrorMessage(error, scratch)}`, }), ), ); + // `$SUPABASE_HOME/tmp` is shared by every project, so stack state gives each run its own + // directory; legacy state keeps the fixed `supabase/.temp/output_<slug>.eszip`. + const ownsTempDir = yield* usesStackState; + const eszipPrefix = `.supabase-eszip-${slug}-`; + const tempDir = ownsTempDir + ? yield* fs.makeTempDirectory({ directory: scratch, prefix: eszipPrefix }).pipe( + Effect.mapError( + (error) => + new FunctionEszipDownloadError({ + message: `failed to mkdir: ${hostErrorMessage(error, path.join(scratch, eszipPrefix))}`, + }), + ), + ) + : scratch; const eszipFileName = `output_${slug}.eszip`; const eszipPath = path.join(tempDir, eszipFileName); - yield* fs.writeFile(eszipPath, eszip).pipe( - Effect.mapError( - (error) => - new FunctionEszipDownloadError({ - message: `failed to download file: ${hostErrorMessage(error, eszipPath)}`, - }), - ), - ); - // `Effect.ensuring` below wraps every step from here on so a failure // resolving the network/volume, spawning Docker, or a non-zero container // exit all still clean up the temp eszip file, not just the happy path. @@ -996,11 +1003,22 @@ const downloadWithDockerUnbundle = Effect.fn("functions.download.dockerUnbundle" const cleanupEszip = debugEnabled ? Effect.void : fs - .remove(eszipPath, { force: true }) + .remove(ownsTempDir ? tempDir : eszipPath, { force: true, recursive: ownsTempDir }) .pipe( Effect.catch((error) => output.raw(`${hostErrorMessage(error, eszipPath)}\n`, "stderr")), ); + yield* fs.writeFile(eszipPath, eszip).pipe( + Effect.mapError( + (error) => + new FunctionEszipDownloadError({ + message: `failed to download file: ${hostErrorMessage(error, eszipPath)}`, + }), + ), + // Only a run-owned directory is removed here; legacy leaves the fixed file in place. + Effect.tapError(() => (ownsTempDir ? cleanupEszip : Effect.void)), + ); + const { projectId, denoVersion, image, projectEnvValues } = edgeRuntimeImage; const functionsDir = path.resolve(dependencies.projectRoot, "supabase", "functions"); const hostEszipPath = path.resolve(eszipPath); diff --git a/apps/cli/src/shared/functions/functions.shared.ts b/apps/cli/src/shared/functions/functions.shared.ts index 1ef1612fee..7af2b67903 100644 --- a/apps/cli/src/shared/functions/functions.shared.ts +++ b/apps/cli/src/shared/functions/functions.shared.ts @@ -1,6 +1,14 @@ -import { Effect, FileSystem, Path } from "effect"; +import { legacyProjectStatePaths } from "../../command-internal/legacy-project-state.paths.ts"; +import { Effect, FileSystem, Option, Path } from "effect"; +import { usesStackState } from "../../command-internal/stack-backend.ts"; +import { readLockedPins } from "../../command-internal/stack-lock.ts"; import { dockerfileServiceImageRaw } from "../services/dockerfile-images.ts"; -import { imageTag, slimImageForCurrentPin } from "../services/slim-images.ts"; +import { + imageTag, + isSlimImageRef, + slimImageForCurrentPin, + slimImagesEnabled, +} from "../services/slim-images.ts"; const functionSlugPattern = /^[A-Za-z][A-Za-z0-9_-]*$/; @@ -28,26 +36,43 @@ export const DENO1_EDGE_RUNTIME_VERSION = "v1.68.4"; * Resolves the edge-runtime image for `tag`. The deno-1 pin always resolves * to the full docker.io image (no slim build exists for it); every other tag * is substituted verbatim into the (possibly slim-rewritten) default image, - * with no `v` prefix synthesized. + * with no `v` prefix synthesized. A slim image ref (the locked pin's own image) is used as is. */ export function edgeRuntimeImage(tag: string, slim: boolean): string { if (tag === DENO1_EDGE_RUNTIME_VERSION) { return `supabase/edge-runtime:${DENO1_EDGE_RUNTIME_VERSION}`; } + if (slim && isSlimImageRef(tag)) { + return tag; + } return slimImageForCurrentPin("edgeruntime", dockerfileServiceImageRaw("edgeruntime"), tag, slim); } /** - * Reads the `.temp/edge-runtime-version` pin file if present, falling back to - * the Dockerfile-embedded tag. Shared by every `functions` command that - * resolves a Docker edge-runtime image. + * The edge-runtime tag every `functions` command that resolves a Docker edge-runtime image uses. + * Stack state mode takes the tag from the locked Functions pin without loading config or checking + * the lock for staleness, since these commands never run the local stack, or the pin's own slim + * image ref when slim images are enabled; legacy mode reads `.temp/edge-runtime-version`. Either + * falls back to the Dockerfile-embedded tag. */ -export const resolveEdgeRuntimeVersionPin = Effect.fnUntraced(function* (supabaseDir: string) { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - return yield* fs.readFileString(path.join(supabaseDir, ".temp", "edge-runtime-version")).pipe( - Effect.map((version) => version.trim()), - Effect.orElseSucceed(() => ""), - Effect.map((version) => version || (imageTag(dockerfileServiceImageRaw("edgeruntime")) ?? "")), - ); -}); +export const resolveEdgeRuntimeVersionPin = Effect.fn("Functions.resolveEdgeRuntimeVersionPin")( + function* (workdir: string) { + const bundledTag = imageTag(dockerfileServiceImageRaw("edgeruntime")) ?? ""; + if (yield* usesStackState) { + const locked = yield* readLockedPins(workdir); + const slim = yield* slimImagesEnabled; + return Option.match(locked, { + onNone: () => bundledTag, + onSome: ({ functions }) => + slim ? functions.image : (imageTag(functions.upstreamImage) ?? bundledTag), + }); + } + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* fs.readFileString(legacyProjectStatePaths(path, workdir).edgeRuntimeVersion).pipe( + Effect.map((version) => version.trim()), + Effect.orElseSucceed(() => ""), + Effect.map((version) => version || bundledTag), + ); + }, +); diff --git a/apps/cli/src/shared/functions/functions.shared.unit.test.ts b/apps/cli/src/shared/functions/functions.shared.unit.test.ts index 6851ccc73e..dd36f1f16b 100644 --- a/apps/cli/src/shared/functions/functions.shared.unit.test.ts +++ b/apps/cli/src/shared/functions/functions.shared.unit.test.ts @@ -1,8 +1,12 @@ import { BunServices } from "@effect/platform-bun"; import { afterEach, describe, expect, it } from "@effect/vitest"; -import { Effect } from "effect"; +import { Effect, FileSystem, Layer, Path } from "effect"; import { vi } from "vitest"; +import { projectStateModeLayer } from "../../command-internal/project-state-mode.ts"; +import { mockCommandSettings } from "../../../tests/helpers/command-mocks.ts"; +import { seedLock } from "../../../tests/helpers/stack-lock.ts"; +import { runtimeInfoLayer } from "../runtime/runtime-info.layer.ts"; import { dockerfileServiceImageRaw } from "../services/dockerfile-images.ts"; import { expectedPinnedImage, @@ -48,6 +52,76 @@ describe("resolveEdgeRuntimeVersionPin", () => { const tag = yield* resolveEdgeRuntimeVersionPin("/no-such-supabase-dir"); expect(tag).toBe(currentEdgeRuntimeTag); expect(tag.includes("/")).toBe(false); - }).pipe(Effect.provide(BunServices.layer)), + }).pipe( + Effect.provide( + Layer.mergeAll( + BunServices.layer, + runtimeInfoLayer, + mockCommandSettings({ workdir: "/no-such-supabase-dir" }), + ), + ), + ), + ); + + it.effect("pins the locked slim image revision in stack mode only when slim images are on", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "edge-runtime-pin-" }); + yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(root, "supabase", "config.toml"), + 'project_id = "test-project"\n', + ); + const lockedImage = `ghcr.io/supabase/cli/edge-runtime:v1.99.0-r1@sha256:${"a".repeat(64)}`; + yield* seedLock(root, (services) => ({ + ...services, + functions: { + ...services.functions, + resolved: { + ...services.functions.resolved, + upstreamVersion: "v1.99.0", + revision: 1, + image: lockedImage, + upstreamImage: "supabase/edge-runtime:v1.99.0", + }, + }, + })).pipe( + Effect.provide( + Layer.mergeAll( + runtimeInfoLayer, + mockCommandSettings({ workdir: root, supabaseHome: root }), + ), + ), + ); + + vi.stubEnv("SUPABASE_USE_SLIM_IMAGES", ""); + const upstreamPin = yield* resolveEdgeRuntimeVersionPin(root); + expect(edgeRuntimeImage(upstreamPin, false)).toBe("supabase/edge-runtime:v1.99.0"); + + vi.stubEnv("SUPABASE_USE_SLIM_IMAGES", "true"); + const slimPin = yield* resolveEdgeRuntimeVersionPin(root); + expect(edgeRuntimeImage(slimPin, true)).toBe(lockedImage); + }).pipe( + Effect.scoped, + Effect.provide(Layer.mergeAll(BunServices.layer, projectStateModeLayer("stack"))), + ), + ); + + it.effect("fails with a typed error when the stack lock is unreadable", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "edge-runtime-pin-" }); + yield* fs.makeDirectory(path.join(root, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(root, "supabase", "supabase.lock"), "not a lock"); + + const error = yield* resolveEdgeRuntimeVersionPin(root).pipe(Effect.flip); + + expect(error).toMatchObject({ _tag: "LockResolutionError", reason: "invalid-lock" }); + }).pipe( + Effect.scoped, + Effect.provide(Layer.mergeAll(BunServices.layer, projectStateModeLayer("stack"))), + ), ); }); diff --git a/apps/cli/src/shared/functions/serve.ts b/apps/cli/src/shared/functions/serve.ts index cb5e00f3cb..3ded170a02 100644 --- a/apps/cli/src/shared/functions/serve.ts +++ b/apps/cli/src/shared/functions/serve.ts @@ -1,3 +1,4 @@ +import { legacyProjectStatePaths } from "../../command-internal/legacy-project-state.paths.ts"; import { bitbucketCloneDir } from "../../command-internal/bitbucket-pipeline.ts"; import { CliConfigSchema, @@ -1758,10 +1759,7 @@ export const startEdgeRuntimeContainer = Effect.fn("functions.startEdgeRuntimeCo // same tree keyed by container name, so these secret env artifacts don't // leak on host disk indefinitely after the container is torn down. const stagingDir = path.join( - input.projectRoot, - "supabase", - ".temp", - "start-secrets", + legacyProjectStatePaths(path, input.projectRoot).startSecretsDir, containerId, ); // A single directory-wide `rm` (not per-file cleanup closures) covers the @@ -2042,7 +2040,7 @@ const startEdgeRuntime = Effect.fn("functions.serve.startEdgeRuntime")(function* }); const localAuthArtifacts = yield* resolveLocalAuthArtifacts(resolved.auth, resolved.configPath); const edgeRuntimeVersionOverride = yield* resolveEdgeRuntimeVersionPin( - input.dependencies.supabaseDir, + input.dependencies.projectRoot, ); const edgeRuntimeVersion = yield* resolveEdgeRuntimeVersion( resolved.edgeRuntime.deno_version, diff --git a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt index d9dbd491c6..c7d23f7af4 100644 --- a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt +++ b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt @@ -239,14 +239,16 @@ EncryptionUnexpectedStatusError ExperimentalFeatureFlagError ExperimentalRequiredError ExperimentalStackDestroyError +ExperimentalStackInstallError ExperimentalStackListError ExperimentalStackLogsError -ExperimentalStackPrepareError +ExperimentalStackOutdatedError ExperimentalStackRestartError ExperimentalStackStartError ExperimentalStackStatusError ExperimentalStackStopError ExperimentalStackTargetError +ExperimentalStackUpdateError FeedbackBackendError FeedbackDeleteCancelledError FeedbackEmptyMessageError @@ -335,6 +337,7 @@ LinkProjectStatusError LinkProjectStatusNetworkError LinkRefArgConflictError LocalDbRunningError +LockResolutionError LoginCryptoError LoginDecryptError LoginFailedError @@ -419,6 +422,8 @@ PostgresConfigUpdateUnmarshalError ProfileLoadError ProfileSaveError ProjectConfigParseError +ProjectLinkIoError +ProjectLinkMalformedError ProjectNotLinkedError ProjectPausedError ProjectRefNotLinkedError @@ -519,6 +524,7 @@ SsoUpdateMetadataFileError SsoUpdateNetworkError SsoUpdateNotFoundError SsoUpdateUnexpectedStatusError +StackArtifactSelectionError StackBootstrapError StackCatalogSetupError StackConfigError diff --git a/apps/cli/src/shared/telemetry/identify-project-groups.ts b/apps/cli/src/shared/telemetry/identify-project-groups.ts new file mode 100644 index 0000000000..e33ae6ee5e --- /dev/null +++ b/apps/cli/src/shared/telemetry/identify-project-groups.ts @@ -0,0 +1,29 @@ +import { Effect } from "effect"; + +import type { Analytics } from "./analytics.service.ts"; +import { GroupOrganization, GroupProject } from "./event-catalog.ts"; + +interface IdentifiedProject { + readonly ref: string; + readonly name: string; + readonly organization_id: string; + readonly organization_slug: string; +} + +/** Publishes the organization and project group metadata for a linked project. */ +export const identifyProjectGroups = Effect.fnUntraced(function* ( + analytics: typeof Analytics.Service, + project: IdentifiedProject, +) { + if (project.organization_id.length > 0) { + yield* analytics.groupIdentify(GroupOrganization, project.organization_id, { + organization_slug: project.organization_slug, + }); + } + if (project.ref.length > 0) { + yield* analytics.groupIdentify(GroupProject, project.ref, { + name: project.name, + organization_slug: project.organization_slug, + }); + } +}); diff --git a/apps/cli/src/telemetry/analytics.layer.integration.test.ts b/apps/cli/src/telemetry/analytics.layer.integration.test.ts index 23acd05c42..7ae53e40f2 100644 --- a/apps/cli/src/telemetry/analytics.layer.integration.test.ts +++ b/apps/cli/src/telemetry/analytics.layer.integration.test.ts @@ -1,15 +1,18 @@ import { BunServices } from "@effect/platform-bun"; +import { gunzipSync } from "node:zlib"; import { describe, expect, it } from "@effect/vitest"; -import { ConfigProvider, Effect, Layer, Option } from "effect"; +import { ConfigProvider, Effect, Layer, Option, Schema } from "effect"; import { HttpClient, HttpClientResponse } from "effect/http"; import { useTempWorkdir } from "../../tests/helpers/command-mocks.ts"; import { mockRuntimeInfo, mockTty } from "../../tests/helpers/mocks.ts"; import { CliSettings } from "../shared/config/cli-settings.service.ts"; import { Analytics } from "../shared/telemetry/analytics.service.ts"; -import { analyticsLayer } from "./analytics.layer.ts"; +import { writeProjectLink } from "../command-internal/project-link.ts"; +import { analyticsLayer, analyticsLayerFor } from "./analytics.layer.ts"; const tempRoot = useTempWorkdir("supabase-analytics-destination-"); +const decodeJson = Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown)); describe("analytics destination", () => { it.live("uses ambient telemetry configuration despite project destination settings", () => { @@ -59,4 +62,75 @@ describe("analytics destination", () => { ), ); }); + + it.live("attributes events to the organization and project of a stack-mode link document", () => { + const events: unknown[] = []; + const recordingClient = HttpClient.make((request) => + Effect.gen(function* () { + if (request.body._tag === "Raw" && request.body.body instanceof Uint8Array) { + const bytes = + request.headers["content-encoding"] === "gzip" + ? gunzipSync(request.body.body) + : request.body.body; + const payload = yield* decodeJson(new TextDecoder().decode(bytes)).pipe(Effect.orDie); + const batch = + typeof payload === "object" ? Reflect.get(payload ?? {}, "batch") : undefined; + if (Array.isArray(batch)) events.push(...batch); + } + return HttpClientResponse.fromWeb(request, Response.json({ status: 1 })); + }), + ); + const settings = Layer.succeed(CliSettings, { + apiUrl: "https://api.supabase.com", + dashboardUrl: "https://supabase.com/dashboard", + projectHost: "supabase.co", + telemetryPosthogHost: "https://project-telemetry.invalid", + telemetryPosthogKey: Option.none(), + accessToken: Option.none(), + noKeyring: Option.none(), + supabaseHome: tempRoot.current, + debug: Option.none(), + telemetryDebug: Option.none(), + telemetryDisabled: Option.none(), + doNotTrack: Option.none(), + }); + const layer = analyticsLayerFor("stack").pipe( + Layer.provide(settings), + Layer.provide(mockRuntimeInfo({ homeDir: tempRoot.current, cwd: tempRoot.current })), + Layer.provide(mockTty({ stdoutIsTty: false })), + Layer.provide(BunServices.layer), + Layer.provide(Layer.succeed(HttpClient.HttpClient, recordingClient)), + ); + return Effect.gen(function* () { + yield* writeProjectLink(tempRoot.current, { + project: { + ref: "abcdefghijklmnopqrst", + organization_id: "org-1", + organization_slug: "acme", + }, + active_branch: { ref: "abcdefghijklmnopqrst" }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + yield* Analytics.use((analytics) => analytics.capture("stack_link_test")).pipe( + Effect.provide(layer), + ); + expect(events).toEqual([ + expect.objectContaining({ + event: "stack_link_test", + properties: expect.objectContaining({ + $groups: { organization: "org-1", project: "abcdefghijklmnopqrst" }, + }), + }), + ]); + }).pipe( + Effect.provide(BunServices.layer), + Effect.provideService( + ConfigProvider.ConfigProvider, + ConfigProvider.fromEnvRecord({ + SUPABASE_TELEMETRY_POSTHOG_HOST: "https://ambient-telemetry.invalid", + SUPABASE_TELEMETRY_POSTHOG_KEY: "ambient-key", + }), + ), + ); + }); }); diff --git a/apps/cli/src/telemetry/analytics.layer.ts b/apps/cli/src/telemetry/analytics.layer.ts index f72a1f7c0e..0bf2d2a0b0 100644 --- a/apps/cli/src/telemetry/analytics.layer.ts +++ b/apps/cli/src/telemetry/analytics.layer.ts @@ -1,4 +1,7 @@ +import { legacyProjectStatePaths } from "../command-internal/legacy-project-state.paths.ts"; import { Config, ConfigProvider, Effect, FileSystem, Layer, Option, Path, Schema } from "effect"; +import type { ProjectStateMode } from "../command-internal/project-state-mode.ts"; +import { readProjectLink } from "../command-internal/project-link.ts"; import { aiToolLayer } from "../shared/telemetry/ai-tool.layer.ts"; import { AiTool } from "../shared/telemetry/ai-tool.service.ts"; import { @@ -113,11 +116,30 @@ function makeLoadLinkedProject( fs: FileSystem.FileSystem, path: Path.Path, runtimeInfo: { readonly cwd: string }, + mode: ProjectStateMode, ): Effect.Effect<Option.Option<LinkedProjectCacheValue>> { return Effect.gen(function* () { const configuredWorkdir = yield* Config.option(Config.String("SUPABASE_WORKDIR")); const workdir = Option.getOrElse(configuredWorkdir, () => runtimeInfo.cwd); - const cachePath = path.join(workdir, "supabase", ".temp", "linked-project.json"); + if (mode === "stack") { + const link = yield* readProjectLink(workdir).pipe( + Effect.provideService(FileSystem.FileSystem, fs), + Effect.provideService(Path.Path, path), + Effect.option, + Effect.map(Option.flatten), + ); + return Option.flatMap(link, ({ project }) => + project.organization_slug === undefined + ? Option.none<LinkedProjectCacheValue>() + : Option.some({ + ref: project.ref, + name: project.name ?? "", + organization_id: project.organization_id ?? "", + organization_slug: project.organization_slug, + }), + ); + } + const cachePath = legacyProjectStatePaths(path, workdir).linkedProjectCache; const exists = yield* fs.exists(cachePath).pipe(Effect.orElseSucceed(() => false)); if (!exists) return Option.none<LinkedProjectCacheValue>(); @@ -136,104 +158,108 @@ function makeLoadLinkedProject( }).pipe(Effect.orElseSucceed(() => Option.none<LinkedProjectCacheValue>())); } -export const analyticsLayer = Layer.effect( - Analytics, - Effect.gen(function* () { - const runtime = yield* TelemetryRuntime; - const configProvider = yield* ConfigProvider.ConfigProvider; - const aiTool = yield* AiTool; - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const runtimeInfo = yield* RuntimeInfo; - const posthogConfig = yield* resolvePosthogConfig(configProvider); - - if (runtime.consent !== "granted" || Option.isNone(posthogConfig.key)) { - return Analytics.of({ - capture: () => Effect.void, - identify: () => Effect.void, - alias: () => Effect.void, - groupIdentify: () => Effect.void, - }); - } +/** `mode` picks where the linked project's group identity is read from. */ +export const analyticsLayerFor = (mode: ProjectStateMode) => + Layer.effect( + Analytics, + Effect.gen(function* () { + const runtime = yield* TelemetryRuntime; + const configProvider = yield* ConfigProvider.ConfigProvider; + const aiTool = yield* AiTool; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const runtimeInfo = yield* RuntimeInfo; + const posthogConfig = yield* resolvePosthogConfig(configProvider); - const client = yield* scopedPosthogClient(posthogConfig.key.value, posthogConfig.host); - - const loadLinkedProject = makeLoadLinkedProject(fs, path, runtimeInfo); - - const isAgent = Option.isSome(aiTool.name); - const envSignals = yield* collectEnvSignals; - - const baseProperties = stripUndefined({ - [PropPlatform]: "cli", - [PropSchemaVersion]: 1, - [PropDeviceId]: runtime.deviceId, - [PropSessionId]: runtime.sessionId, - [PropIsFirstRun]: runtime.isFirstRun, - [PropIsTty]: runtime.isTty, - [PropIsCi]: runtime.isCi, - [PropIsAgent]: isAgent, - [PropOs]: runtime.os, - [PropArch]: runtime.arch, - [PropCliVersion]: runtime.cliVersion, - [PropEnvSignals]: Option.isSome(envSignals) ? envSignals.value : undefined, - }); - - const capture = (event: string, properties: Record<string, unknown> = {}) => - Effect.gen(function* () { - const context = yield* CurrentAnalyticsContext; - const linkedProject = yield* loadLinkedProject; - const groups = resolveGroups(context, linkedProject); - - client.capture({ - event, - distinctId: context.distinct_id ?? runtime.identity.current() ?? runtime.deviceId, - ...(groups === undefined ? {} : { groups }), - properties: { - ...baseProperties, - ...contextProperties(context), - ...stripUndefined(properties), - }, + if (runtime.consent !== "granted" || Option.isNone(posthogConfig.key)) { + return Analytics.of({ + capture: () => Effect.void, + identify: () => Effect.void, + alias: () => Effect.void, + groupIdentify: () => Effect.void, }); + } + + const client = yield* scopedPosthogClient(posthogConfig.key.value, posthogConfig.host); + + const loadLinkedProject = makeLoadLinkedProject(fs, path, runtimeInfo, mode); + + const isAgent = Option.isSome(aiTool.name); + const envSignals = yield* collectEnvSignals; + + const baseProperties = stripUndefined({ + [PropPlatform]: "cli", + [PropSchemaVersion]: 1, + [PropDeviceId]: runtime.deviceId, + [PropSessionId]: runtime.sessionId, + [PropIsFirstRun]: runtime.isFirstRun, + [PropIsTty]: runtime.isTty, + [PropIsCi]: runtime.isCi, + [PropIsAgent]: isAgent, + [PropOs]: runtime.os, + [PropArch]: runtime.arch, + [PropCliVersion]: runtime.cliVersion, + [PropEnvSignals]: Option.isSome(envSignals) ? envSignals.value : undefined, }); - const identify = (distinctId: string, properties: Record<string, unknown> = {}) => - Effect.sync(() => { - client.identify({ - distinctId, - properties: stripUndefined({ - cli_version: runtime.cliVersion, - os: runtime.os, - arch: runtime.arch, - ...properties, - }), + const capture = (event: string, properties: Record<string, unknown> = {}) => + Effect.gen(function* () { + const context = yield* CurrentAnalyticsContext; + const linkedProject = yield* loadLinkedProject; + const groups = resolveGroups(context, linkedProject); + + client.capture({ + event, + distinctId: context.distinct_id ?? runtime.identity.current() ?? runtime.deviceId, + ...(groups === undefined ? {} : { groups }), + properties: { + ...baseProperties, + ...contextProperties(context), + ...stripUndefined(properties), + }, + }); }); - }); - const alias = (distinctId: string, aliasValue: string) => - Effect.sync(() => { - client.alias({ distinctId, alias: aliasValue }); - }); + const identify = (distinctId: string, properties: Record<string, unknown> = {}) => + Effect.sync(() => { + client.identify({ + distinctId, + properties: stripUndefined({ + cli_version: runtime.cliVersion, + os: runtime.os, + arch: runtime.arch, + ...properties, + }), + }); + }); - const groupIdentify = ( - groupType: string, - groupKey: string, - properties: Record<string, unknown> = {}, - ) => - Effect.gen(function* () { - const context = yield* CurrentAnalyticsContext; - client.groupIdentify({ - groupType, - groupKey, - distinctId: context.distinct_id ?? runtime.identity.current() ?? runtime.deviceId, - properties: stripUndefined(properties), + const alias = (distinctId: string, aliasValue: string) => + Effect.sync(() => { + client.alias({ distinctId, alias: aliasValue }); }); + + const groupIdentify = ( + groupType: string, + groupKey: string, + properties: Record<string, unknown> = {}, + ) => + Effect.gen(function* () { + const context = yield* CurrentAnalyticsContext; + client.groupIdentify({ + groupType, + groupKey, + distinctId: context.distinct_id ?? runtime.identity.current() ?? runtime.deviceId, + properties: stripUndefined(properties), + }); + }); + + return Analytics.of({ + capture, + identify, + alias, + groupIdentify, }); + }), + ).pipe(Layer.provide(telemetryRuntimeLayer), Layer.provide(aiToolLayer)); - return Analytics.of({ - capture, - identify, - alias, - groupIdentify, - }); - }), -).pipe(Layer.provide(telemetryRuntimeLayer), Layer.provide(aiToolLayer)); +export const analyticsLayer = analyticsLayerFor("legacy"); diff --git a/apps/cli/src/telemetry/linked-project-cache.integration.test.ts b/apps/cli/src/telemetry/linked-project-cache.integration.test.ts index 036a1ff54e..daa8067f2e 100644 --- a/apps/cli/src/telemetry/linked-project-cache.integration.test.ts +++ b/apps/cli/src/telemetry/linked-project-cache.integration.test.ts @@ -14,6 +14,7 @@ import { assertBlockDrift } from "../../tests/helpers/permission-drift.ts"; import { linkedProjectCache, linkedProjectCacheLayer } from "./linked-project-cache.layer.ts"; import { identityStitchLayer } from "../command-internal/identity-stitch.ts"; import { LinkedProjectCache } from "./linked-project-cache.service.ts"; +import { projectStateModeLayer } from "../command-internal/project-state-mode.ts"; const encodeJson = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); @@ -240,4 +241,63 @@ describe("linkedProjectCacheLayer", () => { }).pipe(Effect.provide(layer)); }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), ); + + it.live("keeps the stack-mode cache under SUPABASE_HOME and leaves supabase/.temp alone", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const workdir = yield* fs.makeTempDirectoryScoped({ prefix: "linked-cache-stack-" }); + const analytics = mockAnalytics(); + const api = mockCommandPlatformApi({ + response: { + status: 200, + body: { + ref: VALID_REF, + id: VALID_REF, + name: "proj", + organization_id: "org-1", + organization_slug: "acme", + region: "us-east-1", + created_at: "2026-01-01T00:00:00Z", + status: "ACTIVE_HEALTHY", + database: { + host: "db.example.com", + version: "15", + postgres_engine: "15", + release_channel: "ga", + }, + }, + }, + }); + const identityStitch = identityStitchLayer.pipe( + Layer.provide(analytics.layer), + Layer.provide( + mockTelemetryRuntime({ configDir: path.join(workdir, ".supabase"), consent: "granted" }), + ), + Layer.provide(BunServices.layer), + ); + const layer = linkedProjectCacheLayer.pipe( + Layer.provide(api.httpClientLayer), + Layer.provide(mockCommandSettings({ workdir, supabaseHome: path.join(workdir, "home") })), + Layer.provide(mockCommandCredentialsLayer), + Layer.provide(identityStitch), + Layer.provide(analytics.layer), + Layer.provide(BunServices.layer), + ); + yield* Effect.gen(function* () { + const cache = yield* LinkedProjectCache; + yield* cache.cache(VALID_REF, workdir); + yield* cache.cache(VALID_REF, workdir); + expect( + yield* fs.readFileString( + path.join(workdir, "home", "project-cache", `${VALID_REF}.json`), + ), + ).toBe( + `{"ref":"${VALID_REF}","name":"proj","organization_id":"org-1","organization_slug":"acme"}`, + ); + expect(api.requests).toHaveLength(1); + expect(yield* fs.exists(path.join(workdir, "supabase", ".temp"))).toBe(false); + }).pipe(Effect.provide(layer.pipe(Layer.provideMerge(projectStateModeLayer("stack"))))); + }).pipe(Effect.scoped, Effect.provide(BunServices.layer)), + ); }); diff --git a/apps/cli/src/telemetry/linked-project-cache.layer.ts b/apps/cli/src/telemetry/linked-project-cache.layer.ts index 8b15f21922..f35d2f5e9b 100644 --- a/apps/cli/src/telemetry/linked-project-cache.layer.ts +++ b/apps/cli/src/telemetry/linked-project-cache.layer.ts @@ -1,13 +1,17 @@ import { makeApiClient } from "@supabase/api/effect"; import { Effect, FileSystem, Layer, Option, Path, type Redacted, Schema } from "effect"; import * as HttpClient from "effect/http/HttpClient"; +import { publish } from "@supabase/stack/effect"; import { CommandCredentials } from "../auth/command-credentials.service.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; +import { PROJECT_REF_PATTERN } from "../config/remote-target.service.ts"; import { IdentityStitch } from "../command-internal/identity-stitch.ts"; import { Analytics } from "../shared/telemetry/analytics.service.ts"; -import { GroupOrganization, GroupProject } from "../shared/telemetry/event-catalog.ts"; -import { readProjectRefFile, tempPaths } from "../command-internal/temp-paths.ts"; +import { identifyProjectGroups } from "../shared/telemetry/identify-project-groups.ts"; +import { legacyProjectStatePaths } from "../command-internal/legacy-project-state.paths.ts"; +import { readLinkSnapshotSoft } from "../command-internal/link-snapshot.ts"; +import { currentProjectStateMode } from "../command-internal/project-state-mode.ts"; import { LinkedProjectCache } from "./linked-project-cache.service.ts"; import { block } from "../command-internal/command-permissions/model.ts"; @@ -17,7 +21,7 @@ export const linkedProjectCache = block("linked-project-cache", [ operationId: "v1-get-project", kind: "best-effort", context: - "an access token, no cached supabase/.temp/linked-project.json, and no supabase/.temp/project-ref naming a different project", + "an access token, no cached project metadata (supabase/.temp/linked-project.json, or $SUPABASE_HOME/project-cache/<ref>.json in stack mode), and no linked ref naming a different project", }, ]); @@ -33,16 +37,21 @@ const encodeLinkedProjectCache = Schema.encodeEffect( ); /** - * Writes `<workdir>/supabase/.temp/linked-project.json` after a `--project-ref` has been - * resolved. No write if the cache already exists (`supabase link` is authoritative), and any - * API/filesystem/parse error is swallowed. + * Writes the project's metadata after a `--project-ref` has been resolved: `<workdir>/supabase/.temp/linked-project.json` + * in legacy mode, or `$SUPABASE_HOME/project-cache/<ref>.json` in stack mode, so a stack-mode + * checkout never gains a `.temp`. No write if the cache already exists (`supabase link` is + * authoritative in legacy mode), and any API/filesystem/parse error is swallowed. * * Builds its own typed client per fill rather than consuming `CommandPlatformApi`, which * requires a stored token at construction; this layer skips quietly when there is none. * - * Also skips the write when `<workdir>/supabase/.temp/project-ref` exists, is non-empty, and - * names a different ref than `ref` — see the inline comment at the check itself for the failure - * mode this closes. This cache feeds `resolveLinkedParentRef`'s parent chain. + * The per-ref stack file is a "group-identify once per ref" marker: its presence skips the + * `v1-get-project` call and the `groupIdentify` on later runs, and nothing else reads it. It is + * published atomically so a concurrent run never sees a partial file. + * + * In legacy mode it also skips the write when the linked ref names a different project than + * `ref` — see the inline comment at the check itself for the failure mode this closes. That + * cache feeds `RemoteTarget.parent`. */ export const linkedProjectCacheLayer = Layer.effect( LinkedProjectCache, @@ -67,27 +76,29 @@ export const linkedProjectCacheLayer = Layer.effect( accessToken?: Option.Option<Redacted.Redacted<string>>, ) => Effect.gen(function* () { + const stackMode = (yield* currentProjectStateMode) === "stack"; + // The ref names the cache file, so only a well-formed one may reach the path. + if (stackMode && !PROJECT_REF_PATTERN.test(ref)) return; const resolvedWorkdir = workdir ?? cliSettings.workdir; - const cachePath = tempPaths(path, resolvedWorkdir).linkedProjectCache; + const cachePath = stackMode + ? path.join(cliSettings.supabaseHome, "project-cache", `${ref}.json`) + : legacyProjectStatePaths(path, resolvedWorkdir).linkedProjectCache; const exists = yield* fs.exists(cachePath).pipe(Effect.orElseSucceed(() => false)); yield* Effect.annotateCurrentSpan({ "cache.hit": exists }); if (exists) return; - // The cache must describe the linked workdir's own state, not whatever ref the calling - // command happens to have resolved: a mid-flight `link --project-ref B` failure still - // reaches this fill via `Effect.ensuring`, and if `getProject(B)` returns 200 (e.g. B - // is merely paused, not gone/forbidden), this would otherwise cache B as the linked - // project even though `project-ref` itself was never updated to B — `link`'s own - // mandatory write happens before this fill can ever fire, so a `project-ref` naming - // something else here means the workdir is still actually linked to that something - // else. Skip the write entirely when the file names a different ref. A file that's - // absent entirely falls through to the write below — the read side - // (`resolveLinkedParentRef`) already refuses to trust a cache with no `project-ref` - // file at all, so there is nothing to protect there yet. - const fileRef = yield* readProjectRefFile(fs, path, resolvedWorkdir).pipe( - Effect.orElseSucceed(() => Option.none<string>()), - ); - if (Option.isSome(fileRef) && fileRef.value !== ref) return; + // The legacy cache must describe the linked workdir's own state, not whatever ref the + // calling command happens to have resolved: a mid-flight `link --project-ref B` failure + // still reaches this fill via `Effect.ensuring`, and if `getProject(B)` returns 200 + // (e.g. B is merely paused, not gone/forbidden), this would otherwise cache B as the + // linked project even though the linked ref itself was never updated to B. Skip the + // write when the link names a different ref. An absent link falls through to the write + // below, since `RemoteTarget.parent` already refuses to trust a cache with no linked + // ref. The per-ref stack cache is keyed by `ref`, so it cannot describe another project. + if (!stackMode) { + const linkedRef = (yield* readLinkSnapshotSoft(fs, path, resolvedWorkdir)).ref; + if (Option.isSome(linkedRef) && linkedRef.value !== ref) return; + } // An explicit reconciled-profile token wins outright (`Some` → use, `None` → the // reconciled profile has no token, so skip rather than fall back to a stale profile's @@ -124,22 +135,12 @@ export const linkedProjectCacheLayer = Layer.effect( }; yield* fs.makeDirectory(path.dirname(cachePath), { recursive: true }); - yield* fs.writeFileString(cachePath, yield* encodeLinkedProjectCache(linked)); + yield* publish(fs, path, { + target: cachePath, + content: yield* encodeLinkedProjectCache(linked), + }); - // On the same cache miss, also publish the org/project group metadata via - // `groupIdentify` (same payload shape as the link handler) before the post-run - // `cli_command_executed` capture. Best-effort, wrapped in `Effect.ignore` below. - if (linked.organization_id.length > 0) { - yield* analytics.groupIdentify(GroupOrganization, linked.organization_id, { - organization_slug: linked.organization_slug, - }); - } - if (linked.ref.length > 0) { - yield* analytics.groupIdentify(GroupProject, linked.ref, { - name: linked.name, - organization_slug: linked.organization_slug, - }); - } + yield* identifyProjectGroups(analytics, linked); }).pipe(Effect.withSpan("LinkedProjectCache.cache"), Effect.ignore), }); }), diff --git a/apps/cli/src/telemetry/linked-project-cache.service.ts b/apps/cli/src/telemetry/linked-project-cache.service.ts index 71bae766e6..f5365ae632 100644 --- a/apps/cli/src/telemetry/linked-project-cache.service.ts +++ b/apps/cli/src/telemetry/linked-project-cache.service.ts @@ -3,8 +3,9 @@ import { Context } from "effect"; interface LinkedProjectCacheShape { /** - * Fire-and-forget: fetches the project metadata from the Management API and writes - * `<workdir>/supabase/.temp/linked-project.json` if no cache exists yet. + * Fire-and-forget: fetches the project metadata from the Management API and writes it if no + * cache exists yet: `<workdir>/supabase/.temp/linked-project.json` in legacy mode, + * `$SUPABASE_HOME/project-cache/<ref>.json` in stack mode. * * `workdir` overrides the directory the cache resolves against. Callers that have already * changed the working directory (e.g. `bootstrap`, whose target workdir can come from an diff --git a/apps/cli/tests/helpers/command-mocks.ts b/apps/cli/tests/helpers/command-mocks.ts index ed51b6b288..313187abba 100644 --- a/apps/cli/tests/helpers/command-mocks.ts +++ b/apps/cli/tests/helpers/command-mocks.ts @@ -1,3 +1,4 @@ +import { legacyProjectStateLayer } from "../../src/command-internal/legacy-project-state.layer.ts"; import { mkdtempSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -55,7 +56,7 @@ import { PGDATA_BASELINE_MARKER_NAME, PGDATA_PATH, } from "../../src/command-internal/db-bootstrap/pgdata-snapshot.ts"; -import { projectRefLayer } from "../../src/config/project-ref.layer.ts"; +import { remoteTargetLayer } from "../../src/config/remote-target.layer.ts"; import { LocalDockerEngine } from "../../src/command-internal/db-bootstrap/local-db-running.ts"; import { LinkedProjectCache } from "../../src/telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../src/telemetry/telemetry-state.service.ts"; @@ -1395,7 +1396,7 @@ export function mockDockerDaemonCliSpawner( } // Bundles the entire Layer.mergeAll(...) graph that every native-port integration test -// re-builds, including the easy-to-mis-wire `projectRefLayer.pipe(Layer.provide(...))` subgraph — +// re-builds, including the easy-to-mis-wire `remoteTargetLayer.pipe(Layer.provide(...))` subgraph — // `Layer.provide` does not share to siblings inside `Layer.mergeAll`, so centralizing the // subgraph here removes a recurring footgun. type GoOutputValue = "env" | "pretty" | "json" | "toml" | "yaml" | "table" | "csv"; @@ -1425,6 +1426,8 @@ export interface BuildTestRuntimeOpts { readonly goOutput?: Option.Option<GoOutputValue>; /** Raw argv seen by the handler (e.g. to exercise an explicit `--yes=false`). */ readonly cliArgs?: ReadonlyArray<string>; + /** Replaces the Bun `FileSystem` every service in the runtime reads and writes through. */ + readonly fileSystem?: Layer.Layer<FileSystem.FileSystem>; } export function buildTestRuntime(opts: BuildTestRuntimeOpts) { @@ -1437,6 +1440,10 @@ export function buildTestRuntime(opts: BuildTestRuntimeOpts) { const analytics = (opts.analytics ?? mockAnalytics()).layer; const goOutput = opts.goOutput ?? Option.none<GoOutputValue>(); const httpClient = opts.api.httpClientLayer; + const platform = + opts.fileSystem === undefined + ? BunServices.layer + : Layer.merge(BunServices.layer, opts.fileSystem); // When the caller doesn't expose an HttpClient layer, use a stub that fails // loudly if any code path tries to consume it. Always wiring HttpClient at @@ -1462,7 +1469,7 @@ export function buildTestRuntime(opts: BuildTestRuntimeOpts) { stdin, processControl, runtimeInfo, - projectRefLayer.pipe( + remoteTargetLayer.pipe( Layer.provide( Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(opts.api.layer)), @@ -1471,9 +1478,10 @@ export function buildTestRuntime(opts: BuildTestRuntimeOpts) { Layer.provide(opts.cliSettings), Layer.provide(tty), Layer.provide(opts.out.layer), - Layer.provide(BunServices.layer), + Layer.provide(platform), ), - BunServices.layer, + platform, + legacyProjectStateLayer.pipe(Layer.provide(platform)), Layer.succeed(OutputFlag, goOutput), Layer.succeed(CliArgs, { args: opts.cliArgs ?? [] }), linkedProjectCache, diff --git a/apps/cli/tests/helpers/compute.ts b/apps/cli/tests/helpers/compute.ts index 434e2f8424..e45be9412f 100644 --- a/apps/cli/tests/helpers/compute.ts +++ b/apps/cli/tests/helpers/compute.ts @@ -8,8 +8,7 @@ import * as HttpClientResponse from "effect/http/HttpClientResponse"; import { CommandPlatformApiFactory } from "../../src/auth/command-platform-api-factory.service.ts"; import { CommandPlatformApi } from "../../src/auth/command-platform-api.service.ts"; import { CommandSettings } from "../../src/config/command-settings.service.ts"; -import { projectRefLayer } from "../../src/config/project-ref.layer.ts"; -import { ProjectRefResolver } from "../../src/config/project-ref.service.ts"; +import { remoteTargetLayer } from "../../src/config/remote-target.layer.ts"; import { CliArgs } from "../../src/shared/cli/cli-args.service.ts"; import { OutputFlag, YesFlag } from "../../src/command-internal/global-flags.ts"; import { randomLayer } from "../../src/shared/runtime/random.layer.ts"; @@ -18,6 +17,7 @@ import { mockLinkedProjectCacheLayer } from "./command-mocks.ts"; import { TelemetryState } from "../../src/telemetry/telemetry-state.service.ts"; import { mockOutput, mockProcessControl, mockRuntimeInfo, mockTty } from "./mocks.ts"; import { realLinkedProjectCache } from "./permission-real-layers.ts"; +import { stubRemoteTarget } from "./remote-target.ts"; /** * Shared scaffolding for the `supabase compute` command integration tests. @@ -390,7 +390,7 @@ const testCliConfigLayer = (workdir: string, explicitWorkdir: boolean) => /** The resolver, stubbed: `--project-ref` wins, else the linked project. */ const testProjectRefLayer = (linked: boolean) => - Layer.succeed(ProjectRefResolver, { + stubRemoteTarget({ resolve: (flagValue: Option.Option<string>) => Option.isSome(flagValue) ? Effect.succeed(flagValue.value) @@ -405,9 +405,9 @@ const testProjectRefLayer = (linked: boolean) => Effect.fail(new ProjectRefNotLinkedError({ message: "Not available in compute tests" })), resolveOptional: () => Effect.succeed(linked ? Option.some(COMPUTE_PROJECT_REF) : Option.none()), - loadProjectRef: () => + load: () => Effect.fail(new ProjectRefNotLinkedError({ message: "Not available in compute tests" })), - promptProjectRef: () => + prompt: () => Effect.fail(new ProjectRefNotLinkedError({ message: "Not available in compute tests" })), }); @@ -450,7 +450,7 @@ export interface ComputeSetupOptions { */ readonly signal?: "SIGINT" | "SIGTERM" | "SIGHUP"; /** - * Runs the real `projectRefLayer` (an unlinked workdir reaches its picker) and the real + * Runs the real `remoteTargetLayer` (an unlinked workdir reaches its picker) and the real * linked-project cache, so their `GET /v1/projects` and `GET /v1/projects/{ref}` calls go * through `routes`. Linkage then comes from the workdir's `.temp/project-ref`, not `linked`. */ @@ -502,7 +502,7 @@ export function setupCompute(options: ComputeSetupOptions) { const cliSettings = testCliConfigLayer(options.workdir, options.explicitWorkdir ?? false); const projectRef = options.realLayers === true - ? projectRefLayer.pipe( + ? remoteTargetLayer.pipe( Layer.provide( Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(http.layer)), diff --git a/apps/cli/tests/helpers/legacy-state-guard.ts b/apps/cli/tests/helpers/legacy-state-guard.ts new file mode 100644 index 0000000000..f023936a5b --- /dev/null +++ b/apps/cli/tests/helpers/legacy-state-guard.ts @@ -0,0 +1,78 @@ +import { mkdirSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; + +import { BunServices } from "@effect/platform-bun"; +import { Effect, FileSystem, Layer } from "effect"; + +const LEGACY_STATE_SEGMENT = join("supabase", ".temp"); + +/** Wrong-on-purpose values for every `supabase/.temp` file a stack-mode command might still read. */ +const LEGACY_STATE_DECOYS: Readonly<Record<string, string>> = { + "project-ref": "decoyprojectrefabcdef", + "pooler-url": "postgresql://decoy:decoy@decoy.pooler.invalid:6543/postgres", + "linked-project.json": JSON.stringify({ ref: "decoyprojectrefabcdef", name: "decoy" }), + "postgres-version": "99.99.99.999", + "rest-version": "v99.99.99", + "gotrue-version": "v99.99.99", + "storage-api": "v99.99.99", + "storage-version": "v99.99.99", + "storage-migration": "decoy-migration", + "realtime-version": "v99.99.99", + "studio-version": "99.99.99", + "pgmeta-version": "v99.99.99", + "logflare-version": "99.99.99", + "pooler-version": "99.99.99", + "edge-runtime-version": "v99.99.99", + "cli-latest": "v99.99.99", +}; + +/** Writes the decoys into `<root>/supabase/.temp`, outside any recording filesystem. */ +export function seedLegacyStateDecoys(root: string): void { + const dir = join(root, LEGACY_STATE_SEGMENT); + mkdirSync(dir, { recursive: true }); + for (const [name, value] of Object.entries(LEGACY_STATE_DECOYS)) { + writeFileSync(join(dir, name), value); + } +} + +/** The recorded paths that fall under a `supabase/.temp` directory. */ +export function legacyStateAccesses(accessed: ReadonlyArray<string>): ReadonlyArray<string> { + return accessed.filter((entry) => entry.includes(LEGACY_STATE_SEGMENT)); +} + +/** + * A `FileSystem` that records every path argument it is handed, then delegates to the real one. + * Merge it through `buildTestRuntime({ fileSystem })` so every service in the runtime goes + * through it. It sees only the Effect `FileSystem`, so code that uses node:fs needs its own + * assertion that the decoys stay untouched. + */ +export function recordingFileSystem(): { + readonly layer: Layer.Layer<FileSystem.FileSystem>; + readonly accessed: Array<string>; +} { + const accessed: Array<string> = []; + const layer = Layer.effect( + FileSystem.FileSystem, + Effect.map( + FileSystem.FileSystem, + (real) => + new Proxy<FileSystem.FileSystem>(real, { + get(target, property, receiver) { + const value: unknown = Reflect.get(target, property, receiver); + if (typeof value !== "function") return value; + return (...args: ReadonlyArray<unknown>) => { + for (const arg of args) { + if (typeof arg === "string") accessed.push(arg); + else if (typeof arg === "object" && arg !== null) { + const directory: unknown = Reflect.get(arg, "directory"); + if (typeof directory === "string") accessed.push(directory); + } + } + return Reflect.apply(value, target, args); + }; + }, + }), + ), + ).pipe(Layer.provide(BunServices.layer)); + return { layer, accessed }; +} diff --git a/apps/cli/tests/helpers/mocks.ts b/apps/cli/tests/helpers/mocks.ts index e67e985305..072f1d594a 100644 --- a/apps/cli/tests/helpers/mocks.ts +++ b/apps/cli/tests/helpers/mocks.ts @@ -11,6 +11,7 @@ import { ProjectLinkState, type ProjectLinkStateValue, } from "../../src/shared/config/project-link-state.service.ts"; +import { legacyProjectStateLayer } from "../../src/command-internal/legacy-project-state.layer.ts"; import { CliProjectContext } from "../../src/shared/config/cli-project-context.service.ts"; import { NonInteractiveError } from "../../src/shared/output/errors.ts"; import { Output } from "../../src/shared/output/output.service.ts"; @@ -672,6 +673,7 @@ export function emptyEnv() { cliProjectContextLayer, cliProjectHomeLayer, projectLinkStateLayer, + legacyProjectStateLayer.pipe(Layer.provide(BunServices.layer)), analytics.layer, mockTelemetryRuntime(), mockTty(), diff --git a/apps/cli/tests/helpers/notebooks.ts b/apps/cli/tests/helpers/notebooks.ts index e4f460fe2a..07b90a76e8 100644 --- a/apps/cli/tests/helpers/notebooks.ts +++ b/apps/cli/tests/helpers/notebooks.ts @@ -9,8 +9,7 @@ import * as UrlParams from "effect/http/UrlParams"; import { CommandPlatformApiFactory } from "../../src/auth/command-platform-api-factory.service.ts"; import { CommandPlatformApi } from "../../src/auth/command-platform-api.service.ts"; import { ProjectRefNotLinkedError } from "../../src/config/project-ref.errors.ts"; -import { projectRefLayer } from "../../src/config/project-ref.layer.ts"; -import { ProjectRefResolver } from "../../src/config/project-ref.service.ts"; +import { remoteTargetLayer } from "../../src/config/remote-target.layer.ts"; import { OutputFlag } from "../../src/command-internal/global-flags.ts"; import { mockLinkedProjectCacheTracked, @@ -21,6 +20,7 @@ import { import { mockOutput, mockContextualAnalytics, mockProcessControl, mockTty } from "./mocks.ts"; import { realLinkedProjectCache } from "./permission-real-layers.ts"; import { commandRuntimeLayer } from "../../src/shared/runtime/command-runtime.layer.ts"; +import { stubRemoteTarget } from "./remote-target.ts"; /** * Shared scaffolding for the `supabase notebooks` command integration tests. @@ -244,12 +244,12 @@ function testProjectRefLayer(linked: boolean) { }), ), }); - return Layer.succeed(ProjectRefResolver, { + return stubRemoteTarget({ resolve, resolveForLink: resolve, resolveOptional: (flag) => Effect.succeed(optional(flag)), - loadProjectRef: resolve, - promptProjectRef: () => resolve(Option.none()), + load: resolve, + prompt: () => resolve(Option.none()), }); } @@ -267,7 +267,7 @@ export interface NotebooksSetupOptions { readonly command?: "pull" | "push"; readonly args?: ReadonlyArray<string>; /** - * Runs the real `projectRefLayer` (an unlinked workdir reaches its picker) and the real + * Runs the real `remoteTargetLayer` (an unlinked workdir reaches its picker) and the real * linked-project cache, so their `GET /v1/projects` and `GET /v1/projects/{ref}` calls go * through `routes`. Linkage then comes from the workdir's `.temp/project-ref`, not `linked`. */ @@ -292,7 +292,7 @@ export function setupNotebooks(options: NotebooksSetupOptions) { : mockCommandSettings({ workdir: options.workdir }); const projectRef = options.realLayers === true - ? projectRefLayer.pipe( + ? remoteTargetLayer.pipe( Layer.provide( Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(http.layer)), diff --git a/apps/cli/tests/helpers/permission-real-layers.ts b/apps/cli/tests/helpers/permission-real-layers.ts index 279b8edd09..fa4476e8c4 100644 --- a/apps/cli/tests/helpers/permission-real-layers.ts +++ b/apps/cli/tests/helpers/permission-real-layers.ts @@ -61,7 +61,10 @@ export function realDbConfigResolver(opts: { readonly workdir: string; readonly dbConnection: Layer.Layer<DbConnection>; readonly env: Record<string, string | undefined>; + /** Replaces the platform services the resolver reads files through. */ + readonly services?: typeof BunServices.layer; }): Layer.Layer<DbConfigResolver> { + const services = opts.services ?? BunServices.layer; const deps = Layer.mergeAll( mockCommandSettings({ workdir: opts.workdir, @@ -83,9 +86,9 @@ export function realDbConfigResolver(opts: { identityStitchLayer.pipe( Layer.provide(mockAnalytics().layer), Layer.provide(mockTelemetryRuntime()), - Layer.provide(BunServices.layer), + Layer.provide(services), ), - BunServices.layer, + services, ConfigProvider.layer(ConfigProvider.fromEnvRecord(opts.env, { preserveEmptyStrings: true })), ); return dbConfigLayer.pipe(Layer.provide(deps)); diff --git a/apps/cli/tests/helpers/pins.ts b/apps/cli/tests/helpers/pins.ts new file mode 100644 index 0000000000..a4905e0a3f --- /dev/null +++ b/apps/cli/tests/helpers/pins.ts @@ -0,0 +1,21 @@ +import { + databaseArtifactPin, + orioledbPostgresVersion, + orioledbVersions, +} from "@supabase/stack/internal/artifacts"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; +import { Effect } from "effect"; + +/** The compiled default pin of every service kind. */ +export const selection = defaultArtifactSelection(); + +/** The PostgreSQL 15 database pin, the non-default stock line. */ +export const database15 = Effect.runSync(databaseArtifactPin("15")); + +/** The `[db] orioledb_version` value of the catalog's OrioleDB build. */ +export const [orioledbConfigVersion = ""] = orioledbVersions(); + +/** The OrioleDB database pin. */ +export const orioledb = Effect.runSync( + databaseArtifactPin(orioledbPostgresVersion(orioledbConfigVersion)), +); diff --git a/apps/cli/tests/helpers/release-archive.ts b/apps/cli/tests/helpers/release-archive.ts new file mode 100644 index 0000000000..eefbba0753 --- /dev/null +++ b/apps/cli/tests/helpers/release-archive.ts @@ -0,0 +1,25 @@ +import { zstdCompressSync } from "node:zlib"; + +const encoder = new TextEncoder(); + +/** A `.tar.zst` holding an empty executable file at each of `paths`. */ +export const releaseArchive = (paths: ReadonlyArray<string>): Uint8Array => { + const bytes = new Uint8Array((paths.length + 2) * 512); + for (const [index, name] of paths.entries()) { + const header = bytes.subarray(index * 512, (index + 1) * 512); + header.set(encoder.encode(name), 0); + header.set(encoder.encode("0000755\0"), 100); + header.set(encoder.encode("0000000\0"), 108); + header.set(encoder.encode("0000000\0"), 116); + header.set(encoder.encode("00000000000\0"), 124); + header[156] = 48; + header.set(encoder.encode("ustar\0"), 257); + header.fill(32, 148, 156); + const checksum = header + .reduce((sum, value) => sum + value, 0) + .toString(8) + .padStart(6, "0"); + header.set(encoder.encode(`${checksum}\0 `), 148); + } + return zstdCompressSync(bytes); +}; diff --git a/apps/cli/tests/helpers/release-fixture.ts b/apps/cli/tests/helpers/release-fixture.ts new file mode 100644 index 0000000000..fe89bedeea --- /dev/null +++ b/apps/cli/tests/helpers/release-fixture.ts @@ -0,0 +1,120 @@ +import { createHash } from "node:crypto"; +// oxlint-disable-next-line effecttsgo/node-builtin-import -- the fixture owns a local native listener. +import { createServer } from "node:http"; +import { Effect } from "effect"; + +const TARGETS = ["darwin-arm64", "linux-amd64", "linux-arm64"] as const; + +interface Served { + readonly body: string | Uint8Array; + readonly headers?: Record<string, string>; +} + +/** A slim-services release host and registry on one loopback port. */ +export interface ReleaseFixture { + readonly origin: string; + readonly routes: Map<string, Served>; + /** HTTP status to answer a path with instead of its route. */ + readonly failing: Map<string, number>; + /** The path of every request received, in arrival order. */ + readonly requests: Array<string>; +} + +const sha256 = (text: string | Uint8Array) => createHash("sha256").update(text).digest("hex"); + +/** Serves `/releases/…`, `/token` and `/v2/…` from OS-assigned loopback port until the scope closes. */ +export const startReleaseFixture = Effect.acquireRelease( + Effect.callback<{ close: () => Promise<void>; fixture: ReleaseFixture }>((resume) => { + const routes = new Map<string, Served>([ + ["/token", { body: JSON.stringify({ token: "anon" }) }], + ]); + const failing = new Map<string, number>(); + const requests: Array<string> = []; + const server = createServer((request, response) => { + const path = new URL(request.url ?? "/", "http://fixture").pathname; + requests.push(path); + const served = routes.get(path); + const status = failing.get(path); + if (status !== undefined) response.statusCode = status; + else if (path.startsWith("/v2/") && request.headers.authorization !== "Bearer anon") + response.statusCode = 401; + else if (served === undefined) response.statusCode = 404; + else + for (const [name, value] of Object.entries(served.headers ?? {})) + response.setHeader(name, value); + response.end(served?.body ?? ""); + }); + server.listen(0, "127.0.0.1", () => { + const address = server.address(); + if (address === null || typeof address === "string") return; + resume( + Effect.succeed({ + close: () => + new Promise<void>((done) => { + server.close(() => done()); + server.closeAllConnections(); + }), + fixture: { origin: `http://127.0.0.1:${address.port}`, routes, failing, requests }, + }), + ); + }); + }), + ({ close }) => Effect.promise(close), +).pipe(Effect.map(({ fixture }) => fixture)); + +/** The env that points the stack's release resolver at `fixture`. */ +export const releaseHostEnv = (fixture: ReleaseFixture) => ({ + SUPABASE_INTERNAL_SLIM_RELEASES_URL: `${fixture.origin}/releases/`, + SUPABASE_INTERNAL_SLIM_REGISTRY_URL: fixture.origin, +}); + +/** Publishes `<producer>-<upstream>-r<revision>` the way slim-services lays it out; returns its image digest. */ +export const publishRelease = ( + fixture: ReleaseFixture, + release: { + producer: string; + upstream: string; + revision: number; + /** Serves manifests that no longer match their `SHA256SUMS` entries. */ + tamperedManifest?: boolean; + /** The `.tar.zst` served for every target; without it the sums cover placeholder text and no archive route exists. */ + archive?: Uint8Array; + /** Publishes the `oci-provenance.json` that mirrored producers (vector, mailpit, imgproxy) resolve their upstream image from. */ + provenanceSource?: string; + }, +) => { + const version = `${release.upstream}-r${release.revision}`; + const tag = `${release.producer}-${version}`; + const sums: Array<string> = []; + for (const target of TARGETS) { + const stem = `${tag}-${target}`; + const image = `docker.io/upstream/${release.producer}:${release.upstream}@sha256:${"a".repeat(64)}`; + const manifest = JSON.stringify({ + service: release.producer, + version, + target, + ...(target === "darwin-arm64" ? { upstream_image: image } : { source_image: image }), + }); + sums.push(`${sha256(release.archive ?? `archive ${stem}`)} ${stem}.tar.zst`); + if (release.archive !== undefined) + fixture.routes.set(`/releases/${tag}/${stem}.tar.zst`, { body: release.archive }); + sums.push(`${sha256(manifest)} ${stem}.manifest.json`); + fixture.routes.set(`/releases/${tag}/${stem}.manifest.json`, { + body: release.tamperedManifest === true ? `${manifest} ` : manifest, + }); + } + if (release.provenanceSource !== undefined) { + const name = `${release.producer}-${release.upstream}.oci-provenance.json`; + const provenance = JSON.stringify({ source: release.provenanceSource }); + sums.push(`${sha256(provenance)} ${name}`); + fixture.routes.set(`/releases/${tag}/${name}`, { body: provenance }); + } + fixture.routes.set(`/releases/${tag}/SHA256SUMS`, { body: `${sums.join("\n")}\n` }); + const imageManifest = JSON.stringify({ schemaVersion: 2, tag }); + const digest = `sha256:${sha256(imageManifest)}`; + fixture.routes.set(`/v2/supabase/cli/${release.producer}/manifests/${version}`, { + body: imageManifest, + headers: { "docker-content-digest": digest }, + }); + return { digest }; +}; diff --git a/apps/cli/tests/helpers/remote-target.ts b/apps/cli/tests/helpers/remote-target.ts new file mode 100644 index 0000000000..0ff1ff47cc --- /dev/null +++ b/apps/cli/tests/helpers/remote-target.ts @@ -0,0 +1,73 @@ +import { Effect, Layer, Option } from "effect"; + +import type { + InvalidProjectRefError, + ProjectRefNotLinkedError, + ProjectRefRequiredError, +} from "../../src/config/project-ref.errors.ts"; +import { + type ParentRefResolution, + RemoteTarget, + type RemoteTargetError, + type ResolvedRemoteTarget, +} from "../../src/config/remote-target.service.ts"; + +type RefEffect = Effect.Effect<string, RemoteTargetError>; + +interface RemoteTargetStubs { + /** The ref the stub resolves; every other entry point defaults to it. */ + readonly resolve: (flagValue: Option.Option<string>) => RefEffect; + readonly load?: (flagValue: Option.Option<string>) => RefEffect; + readonly resolveForLink?: ( + flagValue: Option.Option<string>, + ) => Effect.Effect< + string, + ProjectRefNotLinkedError | InvalidProjectRefError | ProjectRefRequiredError + >; + readonly resolveOptional?: ( + flagValue: Option.Option<string>, + ) => Effect.Effect<Option.Option<string>>; + readonly prompt?: () => Effect.Effect<string, ProjectRefNotLinkedError>; +} + +const targetFor = + (flagValue: Option.Option<string>) => + (ref: string): ResolvedRemoteTarget => ({ + ref, + source: Option.isSome(flagValue) && flagValue.value.length > 0 ? "flag" : "link", + }); +const noParent: ParentRefResolution = { kind: "absent" }; + +/** A `RemoteTarget` whose entry points answer with the refs the stubs return. */ +export function stubRemoteTarget(stubs: RemoteTargetStubs): Layer.Layer<RemoteTarget> { + const { resolve } = stubs; + const load = stubs.load ?? resolve; + return Layer.succeed(RemoteTarget, { + resolve: (flagValue) => Effect.map(resolve(flagValue), targetFor(flagValue)), + load: (flagValue) => Effect.map(load(flagValue), targetFor(flagValue)), + resolveForLink: (flagValue) => + Effect.map( + stubs.resolveForLink?.(flagValue) ?? + resolve(flagValue).pipe( + Effect.catch((error) => + error._tag === "ProjectRefNotLinkedError" || error._tag === "InvalidProjectRefError" + ? Effect.fail(error) + : Effect.die(error), + ), + ), + targetFor(flagValue), + ), + resolveOptional: (flagValue) => + Effect.map( + stubs.resolveOptional?.(flagValue) ?? + resolve(flagValue).pipe( + Effect.map(Option.some), + Effect.orElseSucceed(() => Option.none<string>()), + ), + Option.map(targetFor(flagValue)), + ), + prompt: stubs.prompt ?? (() => Effect.orDie(resolve(Option.none()))), + parent: Effect.succeed(noParent), + resolveParentScoped: (flagValue) => Effect.map(resolve(flagValue), targetFor(flagValue)), + }); +} diff --git a/apps/cli/tests/helpers/stack-cleanup.integration.test.ts b/apps/cli/tests/helpers/stack-cleanup.integration.test.ts index 17e5d42cb0..57430d8b08 100644 --- a/apps/cli/tests/helpers/stack-cleanup.integration.test.ts +++ b/apps/cli/tests/helpers/stack-cleanup.integration.test.ts @@ -3,6 +3,7 @@ import { describe, expect, it } from "@effect/vitest"; import { Cause, Effect, Exit, FileSystem, Layer, Path } from "effect"; import { StackApi, stackApiLayer } from "../../src/command-internal/stack-api.ts"; import { destroyTestStacks } from "./stack-cleanup.ts"; +import { selection } from "./pins.ts"; const liveStackApi = stackApiLayer.pipe(Layer.provide(BunServices.layer)); @@ -29,7 +30,12 @@ describe("test stack cleanup", () => { Effect.succeed(undefined), () => Effect.gen(function* () { - yield* stack.services.create({ service: "mail", config: {}, endpoints: {} }); + yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + endpoints: {}, + }); const before = yield* api.discover({ stateRoot }); expect(before).toHaveLength(1); expect(before[0]?.host).toBeDefined(); diff --git a/apps/cli/tests/helpers/stack-lock.ts b/apps/cli/tests/helpers/stack-lock.ts new file mode 100644 index 0000000000..abe0509575 --- /dev/null +++ b/apps/cli/tests/helpers/stack-lock.ts @@ -0,0 +1,65 @@ +import { BunCrypto, BunServices } from "@effect/platform-bun"; +import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; +import { readLock, writeLock, type Lock } from "@supabase/stack/effect"; +import { seedSavedStack, type SeedSavedStackOptions } from "@supabase/stack/testing"; +import { Effect, Layer, Option, Path } from "effect"; + +import { loadStackConfig } from "../../src/command-internal/stack-config.ts"; +import { lockedSelection } from "../../src/command-internal/stack-lock.ts"; +import { Output } from "../../src/shared/output/output.service.ts"; +import { mockOutput } from "./mocks.ts"; + +/** A published pin the CLI does not bundle: the same release a hundred revisions later. */ +export const unbundled = (pin: ArtifactPin): ArtifactPin => { + const revision = pin.revision + 100; + return { + ...pin, + revision, + image: pin.image.replace( + `:${pin.upstreamVersion}-r${pin.revision}@`, + `:${pin.upstreamVersion}-r${revision}@`, + ), + }; +}; + +/** A pin of another upstream build of the same service, with its image tagged to match. */ +export const rebuilt = (pin: ArtifactPin, upstreamVersion: string): ArtifactPin => ({ + ...pin, + upstreamVersion, + image: pin.image.replace( + `:${pin.upstreamVersion}-r${pin.revision}@`, + `:${upstreamVersion}-r${pin.revision}@`, + ), +}); + +/** + * Writes the default `supabase/supabase.lock` of a project, as a first `supabase start` does; + * `edit` then rewrites its services, for example to pin a release the CLI does not bundle. + */ +export const seedLock = ( + projectRoot: string, + edit: (services: Lock["services"]) => Lock["services"] = (services) => services, +) => + Effect.gen(function* () { + const output = yield* Output; + yield* loadStackConfig(projectRoot, { selection: lockedSelection({ frozen: false, output }) }); + const lock = yield* readLock(projectRoot).pipe(Effect.map(Option.getOrThrow)); + yield* writeLock(projectRoot, { ...lock, services: edit(lock.services) }); + }).pipe(Effect.provide(mockOutput().layer), Effect.asVoid, Effect.orDie); + +/** + * Saves a stack of `projectRoot` under `supabaseHome` as a start leaves it and returns its id; see + * `seedSavedStack` for the data it can leave. + */ +export const seedProjectStack = ( + options: Omit<SeedSavedStackOptions, "stateRoot"> & { readonly supabaseHome: string }, +) => + Effect.gen(function* () { + const path = yield* Path.Path; + const { supabaseHome, ...seed } = options; + const { stackId } = yield* seedSavedStack({ + ...seed, + stateRoot: path.join(supabaseHome, "stacks"), + }); + return stackId; + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, BunCrypto.layer)), Effect.orDie); diff --git a/apps/cli/tests/helpers/storage.ts b/apps/cli/tests/helpers/storage.ts index c8be85648a..e5a81fc817 100644 --- a/apps/cli/tests/helpers/storage.ts +++ b/apps/cli/tests/helpers/storage.ts @@ -19,7 +19,6 @@ import { CommandPlatformApi } from "../../src/auth/command-platform-api.service. import { CommandPlatformApiFactory } from "../../src/auth/command-platform-api-factory.service.ts"; import { ProjectRefNotLinkedError } from "../../src/config/project-ref.errors.ts"; import { unusedGateway } from "./unused-stack.ts"; -import { ProjectRefResolver } from "../../src/config/project-ref.service.ts"; import { generateGoJwt } from "../../src/command-internal/go-jwt.ts"; import { YesFlag } from "../../src/command-internal/global-flags.ts"; import { StackApi } from "../../src/command-internal/stack-api.ts"; @@ -38,6 +37,8 @@ import { mockTelemetryStateTracked, } from "./command-mocks.ts"; import { projectResponse, realLinkedProjectCache } from "./permission-real-layers.ts"; +import { selection } from "./pins.ts"; +import { stubRemoteTarget } from "./remote-target.ts"; interface StorageRoute { readonly method: string; @@ -122,8 +123,8 @@ const stackCredentials = { } satisfies StackCredentials; const databaseCreation: Extract<ServiceCreation, { service: "database" }> = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(STORAGE_TEST_JWT_SECRET), jwtExpiry: 3600, @@ -132,6 +133,7 @@ const databaseCreation: Extract<ServiceCreation, { service: "database" }> = { }; const storageCreation: Extract<ServiceCreation, { service: "storage" }> = { service: "storage", + pin: selection.storage, config: { jwtSecret: STORAGE_TEST_JWT_SECRET, serviceRoleKey: stackCredentials.serviceRoleKey, @@ -390,7 +392,7 @@ export function setupStorage(workdir: string, opts: SetupStorageOptions) { new ProjectRefNotLinkedError({ message: "Cannot find project ref. Have you run supabase link?", }); - const projectRefLayer = Layer.succeed(ProjectRefResolver, { + const remoteTargetLayer = stubRemoteTarget({ resolve: () => opts.linkedFails === true ? Effect.fail(notLinked()) : Effect.succeed(projectRefRef), resolveForLink: () => @@ -398,13 +400,13 @@ export function setupStorage(workdir: string, opts: SetupStorageOptions) { resolveOptional: () => Effect.succeed(Option.some(projectRefRef)), // An explicit `--project-ref` flag takes precedence and short-circuits before // `linkedFails`, so a test can resolve a ref even for an "unlinked" workdir. - loadProjectRef: (flagValue: Option.Option<string>) => + load: (flagValue: Option.Option<string>) => Option.isSome(flagValue) && flagValue.value.length > 0 ? Effect.succeed(flagValue.value) : opts.linkedFails === true ? Effect.fail(notLinked()) : Effect.succeed(projectRefRef), - promptProjectRef: () => Effect.succeed(projectRefRef), + prompt: () => Effect.succeed(projectRefRef), }); const defaultApiKeys = [ @@ -451,7 +453,7 @@ export function setupStorage(workdir: string, opts: SetupStorageOptions) { : realLinkedProjectCache({ httpClient: recorded.httpClientLayer, cliSettings }), cliSettings, BunServices.layer, - projectRefLayer, + remoteTargetLayer, recorded === undefined ? Layer.succeed(CommandPlatformApiFactory, { make: CommandPlatformApi.pipe(Effect.provide(managementApi.layer)), diff --git a/apps/docs/public/cli/config.schema.json b/apps/docs/public/cli/config.schema.json index 0047ef3cf9..526ce97a0c 100644 --- a/apps/docs/public/cli/config.schema.json +++ b/apps/docs/public/cli/config.schema.json @@ -2514,6 +2514,176 @@ }, "additionalProperties": false }, + "local": { + "type": "object", + "properties": { + "services": { + "type": "object", + "properties": { + "default": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "database": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "auth": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "rest": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "realtime": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "storage": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "imgproxy": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "functions": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "studio": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "pgmeta": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "mail": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "analytics": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + }, + "pooler": { + "type": "object", + "properties": { + "version": { + "type": "string", + "description": "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + "examples": ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"] + } + }, + "required": ["version"], + "additionalProperties": false + } + }, + "additionalProperties": false, + "description": "Per-service version specifiers for the local stack, keyed by service." + } + }, + "additionalProperties": false, + "description": "Local-stack-only settings; hosted commands never read this section." + }, "remotes": { "anyOf": [ { diff --git a/apps/docs/public/cli/supabase-lock/v1.schema.json b/apps/docs/public/cli/supabase-lock/v1.schema.json new file mode 100644 index 0000000000..705b48f7d5 --- /dev/null +++ b/apps/docs/public/cli/supabase-lock/v1.schema.json @@ -0,0 +1,267 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://supabase.com/docs/cli/supabase-lock/v1.schema.json", + "title": "supabase.lock", + "description": "The service versions that supabase start pins in supabase/supabase.lock.", + "type": "object", + "properties": { + "$schema": { + "type": "string", + "enum": [ + "https://supabase.com/docs/cli/supabase-lock/v1.schema.json" + ] + }, + "lockfileVersion": { + "type": "number", + "enum": [ + 1 + ] + }, + "linked": { + "type": "object", + "properties": { + "project": { + "type": "string" + }, + "branch": { + "type": "object", + "properties": { + "ref": { + "type": "string" + }, + "isDefault": { + "type": "boolean" + } + }, + "required": [ + "ref", + "isDefault" + ], + "additionalProperties": true + }, + "observedAt": { + "type": "string" + } + }, + "required": [ + "project", + "branch", + "observedAt" + ], + "additionalProperties": true + }, + "services": { + "type": "object", + "properties": { + "database": { + "$ref": "#/$defs/DatabaseLock" + }, + "rest": { + "$ref": "#/$defs/ServiceLock" + }, + "auth": { + "$ref": "#/$defs/ServiceLock" + }, + "realtime": { + "$ref": "#/$defs/ServiceLock" + }, + "storage": { + "$ref": "#/$defs/ServiceLock" + }, + "imgproxy": { + "$ref": "#/$defs/ServiceLock" + }, + "functions": { + "$ref": "#/$defs/ServiceLock" + }, + "studio": { + "$ref": "#/$defs/ServiceLock" + }, + "pgmeta": { + "$ref": "#/$defs/ServiceLock" + }, + "mail": { + "$ref": "#/$defs/ServiceLock" + }, + "analytics": { + "$ref": "#/$defs/ServiceLock" + }, + "pooler": { + "$ref": "#/$defs/ServiceLock" + } + }, + "required": [ + "database", + "rest", + "auth", + "realtime", + "storage", + "imgproxy", + "functions", + "studio", + "pgmeta", + "mail", + "analytics", + "pooler" + ], + "additionalProperties": true + } + }, + "required": [ + "$schema", + "lockfileVersion", + "services" + ], + "additionalProperties": true, + "$defs": { + "LinkedVersion": { + "type": "object", + "properties": { + "raw": { + "type": "string" + }, + "version": { + "type": "string" + } + }, + "required": [ + "raw", + "version" + ], + "additionalProperties": true + }, + "NativePin": { + "type": "object", + "properties": { + "archive": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + }, + "manifest": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + } + }, + "required": [ + "archive", + "manifest" + ], + "additionalProperties": true + }, + "ArtifactPin": { + "type": "object", + "properties": { + "upstreamVersion": { + "type": "string" + }, + "revision": { + "type": "integer" + }, + "image": { + "type": "string" + }, + "upstreamImage": { + "type": "string" + }, + "natives": { + "type": "object", + "properties": { + "darwin-arm64": { + "$ref": "#/$defs/NativePin" + }, + "linux-amd64": { + "$ref": "#/$defs/NativePin" + }, + "linux-arm64": { + "$ref": "#/$defs/NativePin" + } + }, + "required": [ + "darwin-arm64", + "linux-amd64", + "linux-arm64" + ], + "additionalProperties": true + } + }, + "required": [ + "upstreamVersion", + "revision", + "image", + "upstreamImage", + "natives" + ], + "additionalProperties": true + }, + "DatabaseLock": { + "type": "object", + "properties": { + "specifier": { + "type": "string" + }, + "linked": { + "$ref": "#/$defs/LinkedVersion" + }, + "resolution": { + "type": "string", + "enum": [ + "resolved", + "unlinked", + "unobserved", + "unpublished" + ] + }, + "resolved": { + "$ref": "#/$defs/ArtifactPin" + }, + "majorVersion": { + "type": "integer" + }, + "line": { + "type": "string", + "enum": [ + "stock", + "orioledb" + ] + } + }, + "required": [ + "specifier", + "resolution", + "resolved", + "majorVersion", + "line" + ], + "additionalProperties": true + }, + "ServiceLock": { + "type": "object", + "properties": { + "specifier": { + "type": "string" + }, + "linked": { + "$ref": "#/$defs/LinkedVersion" + }, + "resolution": { + "type": "string", + "enum": [ + "resolved", + "unlinked", + "unobserved", + "unpublished" + ] + }, + "resolved": { + "$ref": "#/$defs/ArtifactPin" + } + }, + "required": [ + "specifier", + "resolution", + "resolved" + ], + "additionalProperties": true + } + } +} diff --git a/docs/adr/0017-simplified-managed-stack-architecture.md b/docs/adr/0017-simplified-managed-stack-architecture.md index 7c874b657c..faa2113681 100644 --- a/docs/adr/0017-simplified-managed-stack-architecture.md +++ b/docs/adr/0017-simplified-managed-stack-architecture.md @@ -162,6 +162,12 @@ environment values take precedence. Smoke-only environment files are excluded from packaged profiles. The artifact release is the boundary for service startup defaults; the stack package owns runtime selection and lifecycle around the selected artifact. +Every service creation carries an exact artifact pin (upstream version and +revision) taken from the selection the CLI builds from the project lock file or +the bundled defaults, instead of following the compiled default at launch. The +detached owner never resolves versions itself. One-shot jobs (initialization, +command runner jobs, and the Postgres client tools) use the same selection. See +[ADR-0026](0026-slim-artifact-mirrors.md) for how the lock is resolved. Read-only discovery never acquires ownership. Owner metadata points to a Unix domain socket on POSIX or a named pipe on Windows, and the ownership lock is the diff --git a/docs/adr/0026-slim-artifact-mirrors.md b/docs/adr/0026-slim-artifact-mirrors.md index bed767bf6b..06d70fd65d 100644 --- a/docs/adr/0026-slim-artifact-mirrors.md +++ b/docs/adr/0026-slim-artifact-mirrors.md @@ -2,6 +2,9 @@ **Status**: accepted **Date**: 2026-09-29 +**Amended**: 2026-10-15 — stack mode resolves versions into a project lock file; see "The lock +file is the stack-mode version table". Catalog statements below apply to legacy mode and to the +`bundled` pins the lock can select. ## Problem Statement @@ -34,11 +37,11 @@ semver library — ordering is the service's upstream version key, then revision **Legacy tags** (without `-rN`) are frozen: nothing parses, audits, rewrites, or unpublishes them, so CLIs pinned to a legacy tag keep working. -### The catalog is the single version table +### The catalog is the version table for legacy mode -The CLI's stack catalog (`packages/stack/src/Artifacts.ts`) is the single version table for -every consumer of a slim-capable service: the new stack, legacy `supabase start`, and legacy -slim mode. Each slim-capable service pins exactly one committed slim-services release per +The CLI's stack catalog (`packages/stack/src/Artifacts.ts`) is the version table for legacy +`supabase start` and legacy slim mode, and supplies the `bundled` pins that stack mode can +select (see the next section). Each slim-capable service pins exactly one committed slim-services release per release line it carries (`ArtifactPin`, keyed by upstream version; only postgres carries more than one line, its default plus additional Postgres 15 and OrioleDB pins). Everything else is derived from that pin — never the other way around. @@ -54,6 +57,32 @@ share `library/kong`'s or `supabase/postgres`'s Dependabot `ignore` entry — se The one-shot job images (migra, pg_prove, pgadmin-schema-diff) also have no slim build, but are upstream-only: they are the images Dependabot still bumps. +### The lock file is the stack-mode version table + +In stack mode, the project lock file `supabase/supabase.lock` (committed JSON, written by the +CLI) is the version table. For every service it records the effective specifier, the resolution +outcome, and the exact pin: `<U>-r<N>`, the sha256 of each target's native archive and manifest, +and the GHCR image digest. The CLI builds the stack's artifact selection from the lock, never from +the compiled default. + +Users request versions in `config.toml` under `[local.services]`: +`<svc> = { version = "linked" | "bundled" | "<U>" | "<U>-rN" }`. `default` applies to services +without an entry, and the implicit default is `linked`. + +- `linked` follows the versions of the linked hosted project. When the project is not linked, a + version cannot be observed, or the hosted build is not published, the service falls back to its + bundled pin and the lock records the reason as the resolution outcome. +- `bundled` selects the pin compiled into the CLI's catalog. +- `<U>` selects the highest published revision of that upstream version; `<U>-rN` selects exactly + that revision. + +GitHub releases of slim-services are authoritative at resolution time. The CLI probes the +`<producer>-<U>-r<N>` release assets (`SHA256SUMS`; a 404 is the only "not published" signal) and +reads the GHCR digest, then writes both into the lock. The lock pins the canonical GHCR image +reference and rejects any other registry. S3 and ECR remain byte mirrors of those releases: +downloads may come from any mirror, derived from the canonical reference at pull time, and every +download is verified against the hashes and digests in the lock. + ### Content pins, not runtime checksums Every `ArtifactPin` pins its release by content: @@ -68,7 +97,8 @@ Every `ArtifactPin` pins its release by content: field must equal `R`) and hash-checks the archive against its pin while it streams. There is no runtime checksum authority: **GitHub Releases, the S3 mirror, GHCR, and ECR Public are byte mirrors only** for both images and native archives. None of them is consulted for the expected -hash — that comes only from the pin already committed to the catalog. +hash at download time — that comes only from the pin already committed to the catalog (legacy +mode) or to the lock file (stack mode). ### Invocations @@ -106,6 +136,11 @@ verification with no fallback. ### Hotfix and upgrade pickup +The catalog PRs below keep the catalog, and so legacy mode and the `bundled` pins, current. In +stack mode a hotfix revision reaches a project only when its lock is rewritten by `stack update` +(or by `link`, for `linked` services); `start` re-resolves only services whose request changed. A +CLI upgrade does not move any service already in the lock, including `bundled` ones. + The last step of slim-services' `publish-release` sends a `repository_dispatch` (`slim-release-published`) to the CLI repo with `{service, upstream_version, revision, release_version}`. The payload arrives with whatever authority holds the dispatch token, so @@ -226,7 +261,8 @@ or considered stale and backfilled — never rewritten in place. - Hotfixing packaging never touches previously published bytes; it always publishes a new revision. - The CLI verifies every artifact it downloads against a pin already committed to its own - catalog — no destination is trusted to supply the expected hash at runtime. + catalog or to the project lock file — no destination is trusted to supply the expected hash at + download time. - A same-version republish is no longer possible; every fix is a new, allocatable revision. - A revision's ECR/S3 copy can briefly lag GitHub after publish; the sync's S3-staleness check and the mirror backfill are what keep them converging. @@ -234,6 +270,9 @@ or considered stale and backfilled — never rewritten in place. ### Tradeoffs of moving updates onto the catalog +These apply to legacy mode and to `bundled` pins. A stack-mode project with a lock picks up +updates only when the lock is rewritten, as described under "Hotfix and upgrade pickup". + - Upstream and security fixes for slim-capable images now reach the CLI only once slim-services publishes a release. Direct Dependabot discovery is gone for them; Dependabot's `docker` ecosystem `ignore` list (`.github/dependabot.yml`) now excludes every slim-capable diff --git a/packages/config/docs/cli-config-loading.md b/packages/config/docs/cli-config-loading.md index 31f55c6aef..a442547964 100644 --- a/packages/config/docs/cli-config-loading.md +++ b/packages/config/docs/cli-config-loading.md @@ -47,7 +47,7 @@ The `Cli*` prefix names the local checkout side — what the CLI reads, writes, itself on disk. A bare `Project*` name is reserved for the hosted Supabase project side. (Deliberate exceptions live in apps/cli: services that describe the hosted project itself or the CLI's link to it — `ProjectLinkRemote`, `ProjectLinkState` (both in `shared/config`), and -`ProjectRefResolver` (in `apps/cli/src/config`) — keep the bare `Project*` root under this same +`RemoteTarget` (in `apps/cli/src/config`) — keep the bare `Project*` root under this same rule.) ## The `/io` facade diff --git a/packages/config/src/base.ts b/packages/config/src/base.ts index a13a079fd8..461930b8ef 100644 --- a/packages/config/src/base.ts +++ b/packages/config/src/base.ts @@ -7,6 +7,7 @@ import { edge_runtime } from "./edge_runtime.ts"; import { experimental } from "./experimental.ts"; import { functions } from "./functions.ts"; import { inbucket } from "./inbucket.ts"; +import { local } from "./local.ts"; import { toPublishedJsonSchemaDocument } from "./json-schema-document.ts"; import { realtime } from "./realtime.ts"; import { storage } from "./storage.ts"; @@ -41,6 +42,7 @@ const baseCliConfigFields = { studio, compute, experimental, + local: Schema.optionalKey(local), }; const remoteCliConfigBlock = Schema.Struct({ diff --git a/packages/config/src/entrypoint-purity.unit.test.ts b/packages/config/src/entrypoint-purity.unit.test.ts index c0f68f0c96..fe6bf007f1 100644 --- a/packages/config/src/entrypoint-purity.unit.test.ts +++ b/packages/config/src/entrypoint-purity.unit.test.ts @@ -314,6 +314,7 @@ const expectedPureGraphFiles = [ "storage.ts", "studio.ts", "compute.ts", + "local.ts", ] .map((relativePath) => join(srcDir, relativePath)) .sort(); diff --git a/packages/config/src/local.ts b/packages/config/src/local.ts new file mode 100644 index 0000000000..dd790fea30 --- /dev/null +++ b/packages/config/src/local.ts @@ -0,0 +1,75 @@ +import { Schema } from "effect"; + +const tags = ["local"]; + +const upstreamVersion = /^[A-Za-z0-9][A-Za-z0-9._-]*$/u; +const malformedRevision = /-r(?:0\d+)?$/u; +const orioledbMarker = /-orioledb(?:-|$)/u; + +function versionIssue(key: string, version: string): string | undefined { + if (version === "linked" || version === "bundled") { + return undefined; + } + if (key === "default") { + return `local.services.default.version "${version}" must be "linked" or "bundled"`; + } + if (version === "default" || !upstreamVersion.test(version) || malformedRevision.test(version)) { + return `local.services.${key}.version "${version}" must be "linked", "bundled", or an upstream version optionally followed by -r<N> (N a non-negative integer without leading zeros)`; + } + if (orioledbMarker.test(version) && key !== "database") { + return `local.services.${key}.version "${version}" uses the -orioledb marker, which is only valid for the database service`; + } + return undefined; +} + +const serviceEntry = Schema.Struct({ + version: Schema.String.annotate({ + description: + "Version specifier: `linked` (the linked project's version), `bundled` (the version shipped with this CLI), or an upstream version optionally followed by `-r<N>`. A `-orioledb` version is only valid for `database`.", + examples: ["bundled", "v2.197.0", "v1.30.2-r1", "17.6.1.200-orioledb"], + tags, + }), +}); + +const entry = Schema.optionalKey(serviceEntry); + +const services = Schema.Struct({ + default: entry, + database: entry, + auth: entry, + rest: entry, + realtime: entry, + storage: entry, + imgproxy: entry, + functions: entry, + studio: entry, + pgmeta: entry, + mail: entry, + analytics: entry, + pooler: entry, +}) + .annotate({ + description: "Per-service version specifiers for the local stack, keyed by service.", + tags, + }) + .check( + Schema.makeFilter( + (value: { readonly [key: string]: { readonly version: string } | undefined }) => { + for (const [key, entry] of Object.entries(value)) { + const issue = entry === undefined ? undefined : versionIssue(key, entry.version); + if (issue !== undefined) { + return { path: [key, "version"], issue }; + } + } + return undefined; + }, + ), + ); + +/** `[local]` — settings for the local stack only; hosted commands never read it. */ +export const local = Schema.Struct({ + services: Schema.optionalKey(services), +}).annotate({ + description: "Local-stack-only settings; hosted commands never read this section.", + tags, +}); diff --git a/packages/config/src/local.unit.test.ts b/packages/config/src/local.unit.test.ts new file mode 100644 index 0000000000..9cbe883c47 --- /dev/null +++ b/packages/config/src/local.unit.test.ts @@ -0,0 +1,66 @@ +import { Schema } from "effect"; +import { parse } from "smol-toml"; +import { describe, expect, test } from "vitest"; +import { CliConfigSchema } from "./base.ts"; +import { fromConfigDocument } from "./project-config/project-config.ts"; + +const decode = (toml: string) => + Schema.decodeUnknownSync(CliConfigSchema)(parse(toml), { onExcessProperty: "error" }); + +const services = (entries: string) => `[local.services]\n${entries}\n`; + +describe("[local.services]", () => { + test("decodes version specifiers and keeps them out of the hosted projection", () => { + const document = decode( + services( + [ + 'default = { version = "bundled" }', + 'auth = { version = "v2.197.0" }', + 'storage = { version = "v1.30.2-r1" }', + 'realtime = { version = "v2.34.47-r0" }', + 'studio = { version = "2025.06.30-sha-6f5982d" }', + 'database = { version = "17.6.1.200-orioledb-r2" }', + 'rest = { version = "linked" }', + ].join("\n"), + ), + ); + + expect(document.local?.services?.database).toEqual({ version: "17.6.1.200-orioledb-r2" }); + expect(document.local?.services?.realtime).toEqual({ version: "v2.34.47-r0" }); + expect(document.local?.services?.studio).toEqual({ version: "2025.06.30-sha-6f5982d" }); + expect(Object.keys(fromConfigDocument(document))).not.toContain("local"); + }); + + test("rejects a service key outside the closed set", () => { + expect(() => decode(services('kong = { version = "bundled" }'))).toThrow(/kong/u); + }); + + test("rejects a [local] key outside services", () => { + expect(() => decode("[local]\nports = {}\n")).toThrow(/ports/u); + }); + + test("is not accepted in a remote block", () => { + expect(() => + decode( + '[remotes.prod]\nproject_id = "abc"\n[remotes.prod.local.services]\nauth = { version = "v1" }\n', + ), + ).toThrow(/local/u); + }); + + test.each([ + ["auth", "v2.197.0-r", /local\.services\.auth\.version/u], + ["auth", "v2.197.0-r01", /local\.services\.auth\.version/u], + ["auth", "v2 197", /local\.services\.auth\.version/u], + ["auth", "v2.197.0+build", /local\.services\.auth\.version/u], + ["auth", "-v1", /local\.services\.auth\.version/u], + ["auth", "default", /local\.services\.auth\.version "default"/u], + ["auth", "17.6.1.200-orioledb", /orioledb.*only valid for the database/u], + [ + "default", + "v2.197.0", + /local\.services\.default\.version "v2.197.0" must be "linked" or "bundled"/u, + ], + ])("rejects %s version %j", (key, version, message) => { + expect(() => decode(services(`${key} = { version = "${version}" }`))).toThrow(message); + }); +}); diff --git a/packages/config/src/project-config/hosted-sections.ts b/packages/config/src/project-config/hosted-sections.ts index d8e924bcb6..769c0fe41f 100644 --- a/packages/config/src/project-config/hosted-sections.ts +++ b/packages/config/src/project-config/hosted-sections.ts @@ -23,7 +23,7 @@ export type HostedSectionKey = (typeof HOSTED_SECTION_KEYS)[number]; * `enabled`/`port`, the `db.migrations`/`db.seed` subtrees, every config-side `realtime.*` field, * `db.orioledb_version` (plus its deprecated `experimental.orioledb_version` alias — a document * that hasn't gone through the loader's deprecation normalization can still carry it), and - * local-only `experimental.*` engine/backend selection. + * local-only `experimental.*` engine/backend selection, and the whole top-level `local` section. * * `db.major_version` and `db.pooler`'s other three fields (`pool_mode`, `default_pool_size`, * `max_client_conn`) are real hosted facts and excluded from this list, so `config @@ -55,6 +55,7 @@ export const DOCUMENT_ONLY_LOCAL_PATHS = [ ["experimental", "s3_region"], ["experimental", "pgdelta"], ["experimental", "inspect"], + ["local"], ] as const satisfies ReadonlyArray<ReadonlyArray<string>>; /** One member tuple of {@link DOCUMENT_ONLY_LOCAL_PATHS}. */ diff --git a/packages/config/src/project-config/project-config.unit.test.ts b/packages/config/src/project-config/project-config.unit.test.ts index e7cf72ab42..d92aee1427 100644 --- a/packages/config/src/project-config/project-config.unit.test.ts +++ b/packages/config/src/project-config/project-config.unit.test.ts @@ -596,6 +596,7 @@ describe("fromConfigDocument — CLI-only field exclusion (CLI-2316)", () => { inspect: { rules: [{ name: "r1" }] }, webhooks: { enabled: true }, }, + local: { services: { auth: { version: "bundled" } } }, }); const projected = fromConfigDocument(document); diff --git a/packages/stack/ARCHITECTURE.md b/packages/stack/ARCHITECTURE.md index 74bc88286d..26afbad85c 100644 --- a/packages/stack/ARCHITECTURE.md +++ b/packages/stack/ARCHITECTURE.md @@ -155,10 +155,9 @@ This sketch retains the existing `services.create` shape while making creation i ```ts const shadow = await stack.services.create({ service: "database", - config: { - version: "17", - endpoints: { sql: { port: "auto" } }, - }, + pin: selection.database, + config: {}, + endpoints: { sql: { port: "auto" } }, }); try { @@ -325,7 +324,9 @@ The backend provides mechanical operations: launch, observe exit/logs, stop and A successful launch returns an owned runtime handle with readiness observation and cleanup. One shared readiness program belongs to each runtime launch and has a bounded outcome. Readiness timeout or initialization failure leaves the process running with unhealthy status and an actionable error. It does not automatically stop or restart the process. Stop remains available, and traffic/dependent launches do not treat unhealthy as ready. Initialization that requires a running process belongs to that runtime session, not the start transition. For PostgreSQL, healthy means required catalog initialization and credential reconciliation have completed—not merely that TCP accepts a connection. Stopping the running instance closes that session, settling its health probes and initialization helpers as ordinary resource cleanup. There are not two competing lifecycle operations. -The database keeps only the initialization information needed to reuse its data on a normal stop/start. Data is reusable only within one release line: the PostgreSQL major for stock builds (`17`), so same-major minor upgrades keep their data, or the exact build for OrioleDB (`17.11.0.002-orioledb`), since a newer OrioleDB build may refuse older data. The readiness marker records the full artifact version, from which the line is derived. A first start records the requested line before initdb runs (a native line marker in the instance root, or the Docker storage marker), so a first start interrupted before readiness resumes only on that line. `PG_VERSION` records only the major, so data with neither a readiness marker nor a recorded line counts as the stock line and an OrioleDB request refuses it. Mark initialization complete only after it succeeds. If an ordinary stop interrupts application initialization, the recipe must either support a safe retry or report that initialization is incomplete; do not add a generic persisted operation journal or recovery workflow. +The database keeps only the initialization information needed to reuse its data on a normal stop/start. Data is reusable only by a slim-services revision of the build that initialized it: its identity is `{line, major, build}`, where the build is the upstream version. A first start records the identity in `.supabase-database-identity.json` in the instance root before initdb runs, so a first start interrupted before readiness resumes only on that build; the readiness marker records only runtime and profile. `Database.prepare`, the gated launch, the Docker storage preparation and snapshot save and restore all check the recorded identity, and data with none is refused. Snapshot keys include the identity. Mark initialization complete only after it succeeds. If an ordinary stop interrupts application initialization, the recipe must either support a safe retry or report that initialization is incomplete; do not add a generic persisted operation journal or recovery workflow. + +Services that migrate schemas (Auth, Storage, Realtime, Analytics, the pooler) can never run an older upstream than the newest one that already migrated the database. The instance root records that version per service in `schema-writers.json`; a missing record is empty and the format is a plain object keyed by service. Before a service launch or a finite initialization command migrates, it holds a per-database-and-service SQLite lease and, under a database-scoped critical section, raises the record to the selected version with an atomic, fsynced rename; a selection older than the record fails with `SchemaWriterDowngrade`, whose advice offers to keep the data by pinning the recorded version before it offers to recreate the stack. Versions order by their leading dotted number, which covers `v2.196.0`, `2026.10.05-sha-...` and `17.6.1.054`; a suffix is ignored and an unparseable version is oldest. The composition check reads the record of the reused database before saving a changed pin, for the writers the requested composition binds to it through `databaseUrl`, but only the launch and command guard hold the lease. `findDataConflicts` judges the saved composition by the same binding and the same initialized-data rule as a start, and a container runtime's recorded identity that a start's storage preparation checks, without the lease. Services without schemas accept any version change, and a service whose `databaseUrl` is not bound to a composition database, such as a standalone service on an external database, has no local record and never touches one. The initialization RPCs carry the database instance id, which the owner validates, instead of deriving it from the URL. Unexpected process exit is observed by the living host. The execution records the exit result, cleans the exact owned session under its execution lock, and reports the exit once cleanup is confirmed, or a cleanup failure while resources remain. Nothing launches a replacement while resources are held. Old-handle events cannot affect a replacement. This adds no cleanup scanner: the host already owns the handle and its exit observation. @@ -396,7 +397,7 @@ import { postgres } from "@supabase/stack"; // Connection values are plain data, rendered for the stack runtime. const { databaseUrl } = await database.credentials({ from: "runtime" }); const result = await stack.commands.run( - postgres.pgDump({ major: 17 }), + postgres.pgDump({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--schema-only", "--no-owner"], stdout: (bytes) => destination.write(bytes), @@ -407,16 +408,16 @@ const result = await stack.commands.run( // result: { jobId, exitCode } -await stack.commands.run({ type: "auth.initialize", databaseUrl }); -await stack.commands.run({ type: "storage.initialize", databaseUrl, filePath }); -await stack.commands.run({ type: "realtime.initialize", databaseUrl }); +await stack.commands.run({ type: "auth.initialize", pin, databaseId, databaseUrl }); +await stack.commands.run({ type: "storage.initialize", pin, databaseId, databaseUrl, filePath }); +await stack.commands.run({ type: "realtime.initialize", pin, databaseId, databaseUrl }); ``` -`postgres.pgDump({ major })` describes the selected client package and how to launch its native and container forms. The caller chooses the version; 17 is illustrative. The stack's runtime chooses the execution form. The invocation supplies ordinary arguments, environment and byte streams. Initialization invocations select Auth, Storage, or Realtime, provide the database URL, and provide Storage's file path. They require stack credentials already established by database or composition setup; credential lookup is read-only and fails when no stack identity has been established. They resolve the same service recipe artifact, environment, mounts and working directory as normal service launch while leaving no service instance behind. The CLI awaits selected initialization commands concurrently before applying its catalog overlay. +`postgres.pgDump({ pin })` describes the selected client package and how to launch its native and container forms. The caller supplies the artifact pin. The stack's runtime chooses the execution form. The invocation supplies ordinary arguments, environment and byte streams. Initialization invocations select Auth, Storage, or Realtime, provide the database instance id and URL, and provide Storage's file path; the owner validates the id, and the schema-writers guard runs before the migration starts. They require stack credentials already established by database or composition setup; credential lookup is read-only and fails when no stack identity has been established. They resolve the same service recipe artifact, environment, mounts and working directory as normal service launch while leaving no service instance behind. The CLI awaits selected initialization commands concurrently before applying its catalog overlay. `databaseUrl` is a plain string. The same invocation can target the primary database, a shadow database or an external database without changing the command definition or the runner. The proxy handles any wake-up caused by connecting to a sleeping managed service. A URL obtained for container execution is already reachable from that container; endpoint address selection remains outside the generic command runner. The orchestrator uses the same endpoint renderer when supplying connection values to service instances. It handles the host gateway and the listener binding needed to reach it, not just hostname substitution, and returns an error if the selected networking configuration cannot reach the endpoint. Caller-supplied external URLs remain unchanged. -Arguments are passed as an argument vector, not interpreted as shell text. Stdin accepts an optional asynchronous byte stream; stdout/stderr callbacks receive bytes and may return Promises, which the runner awaits for backpressure. All examples use byte sinks whose writes await capacity. `psql` uses the same operation with `postgres.psql({ major })`, optionally supplying stdin. Callers supply stdout/stderr sinks explicitly. Neither facade collects unbounded output into a return value. +Arguments are passed as an argument vector, not interpreted as shell text. Stdin accepts an optional asynchronous byte stream; stdout/stderr callbacks receive bytes and may return Promises, which the runner awaits for backpressure. All examples use byte sinks whose writes await capacity. `psql` uses the same operation with `postgres.psql({ pin })`, optionally supplying stdin. Callers supply stdout/stderr sinks explicitly. Neither facade collects unbounded output into a return value. The operation settles after process exit, output delivery and owned-resource cleanup. PostgreSQL client commands return a nonzero process exit in `exitCode` for CLI-specific handling; initialization commands reject with a typed error on nonzero exit. Preparation, transport, sink and cleanup failures also reject with typed execution errors. Cancellation stops and cleans up the attached job and reports cancellation. The StackHost assigns `jobId` and owns the resources. No public job registry, job healthcheck, persistent command service or separate launch/attach/wait sequence is needed for these use cases. @@ -814,6 +815,7 @@ Keep the contract narrow: - Both operations hold the instance's storage reservation in the lifecycle reducer and leave it stopped. While the reservation is held, a start keeps the instance stopped until it is released and another storage operation is rejected at once. A destroy waits for the storage operation to settle, then takes the reservation; once a destroy is pending, new storage operations are refused, so it cannot be starved. No new lifecycle states are necessary; the observable current operation identifies snapshot work. An armed wake route is not a substitute for explicit stop. - Native snapshots copy or clone the host data; container snapshots copy database data through a managed volume and helper. Docker data normally lives in a managed volume, while existing host data can be retained through the host-backed fallback; Podman always uses host-backed data. The host storage marker detects a missing or mismatched Docker volume; deleting that volume loses its database data. - Restore transfers compatible database contents, not the source instance's identity, public port claims or composition membership. The target retains its own data location and configuration, with database-specific credentials reconciled before readiness. Cache snapshots survive destruction of the source instance because their managed storage is separate; instance snapshots restore only into their own instance. +- A snapshot carries the instance's schema-writers record (see below). Restore holds every service's schema-writer lease, so no writer is admitted or still migrating, and makes three publications: raise the target's record to the newer version of each service and fsync it, publish the data, then set the record to the snapshot's. A process killed after the first leaves the older writers refused, so a restore can never make a downgrade look safe; a failed restore leaves the record raised, which a reset clears. These are physical database snapshots for the cache use case. A `pg_dump` invocation remains an ordinary client command for logical exports. CLI code owns cache keys, migrations and the decision to fall back to rebuilding a baseline; managed storage owns publication and retention. The snapshot API does not acquire CLI cache policy. @@ -821,7 +823,7 @@ These are physical database snapshots for the cache use case. A `pg_dump` invoca `DatabaseInstance.resetData` removes the selected database instance's owned data and initialization metadata while retaining its registration, configuration, composition bindings, and public port assignments. It is database-specific, alongside snapshot save and restore; other service types do not expose a reset operation. -The caller must stop the database with wake disabled first. Reset holds the same storage reservation as snapshots and leaves the database stopped. Deletion uses the database's ownership checks and runtime-specific filesystem handling. The next normal start initializes a fresh PostgreSQL cluster using the retained configuration. Reset does not apply project migrations or seeds, stop other services, or resume the composition; those decisions belong to the CLI. It adds no lifecycle state or persisted recovery phase. +The caller must stop the database with wake disabled first. Reset holds the same storage reservation as snapshots and leaves the database stopped. Deletion uses the database's ownership checks and runtime-specific filesystem handling. Reset holds every service's schema-writer lease and removes the data before the schema-writers record, so a crash in between leaves the record guarding an empty database. The next normal start initializes a fresh PostgreSQL cluster using the retained configuration. Reset does not apply project migrations or seeds, stop other services, or resume the composition; those decisions belong to the CLI. It adds no lifecycle state or persisted recovery phase. ## 9. What changes for the CLI diff --git a/packages/stack/README.md b/packages/stack/README.md index fe3ed3cfee..21036cde44 100644 --- a/packages/stack/README.md +++ b/packages/stack/README.md @@ -11,6 +11,10 @@ The Promise entrypoint accepts plain configuration: ```ts import { create } from "@supabase/stack"; import { postgres } from "@supabase/stack/commands"; +import { defaultArtifactSelection } from "@supabase/stack/defaults"; + +// Every service names the exact artifact pin it runs; the defaults are the compiled catalog pins. +const selection = defaultArtifactSelection(); const stack = await create({ projectRoot: process.cwd(), @@ -21,8 +25,8 @@ const stack = await create({ const database = await stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: "local-password", jwtSecret: "local-development-secret-at-least-32-characters", jwtExpiry: 3600, @@ -35,7 +39,7 @@ await database.ready(); // initialization and health completed const { databaseUrl } = await database.credentials({ from: "runtime" }); if (databaseUrl === undefined) throw new Error("Database endpoint missing"); -await stack.commands.run(postgres.psql({ major: 17 }), { +await stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--command", "SELECT 1"], stdout: (bytes) => { process.stdout.write(bytes); @@ -48,14 +52,19 @@ await stack.commands.run(postgres.psql({ major: 17 }), { // Runs recipe initialization without registering a service instance. const { authDatabaseUrl } = await database.credentials({ from: "runtime" }); if (authDatabaseUrl === undefined) throw new Error("Auth database URL missing"); -await stack.commands.run({ type: "auth.initialize", databaseUrl: authDatabaseUrl }); +await stack.commands.run({ + type: "auth.initialize", + pin: selection.auth, + databaseId: database.id, + databaseUrl: authDatabaseUrl, +}); await database.stop(); await database.saveSnapshot("baseline"); await stack.close(); // disconnects this client ``` -`start` and `ready` are separate operations. `restart({ config })` replaces recipe configuration while retaining the instance identity and endpoint intentions. A health failure leaves a launched process running and observable; it does not prevent `stop`. Database snapshots require a stopped instance with wake disabled. `saveSnapshot(key)` publishes complete data to managed backend storage and replaces the previous entry for that key; `restoreSnapshot(key)` returns `false` on a miss and `true` after restoring a compatible entry. By default snapshots live in the shared cache, whose retention may evict older keys, and survive destruction of the source stack. `{ scope: "instance" }` keeps a snapshot with the instance instead: it is never evicted, restores only into that instance, survives `resetData`, and is removed when the instance is destroyed. Other service handles have no snapshot methods. +`start` and `ready` are separate operations. `restart({ config })` replaces recipe configuration while retaining the instance identity and endpoint intentions. A health failure leaves a launched process running and observable; it does not prevent `stop`. Database snapshots require a stopped instance with wake disabled. `saveSnapshot(key)` publishes complete data to managed backend storage and replaces the previous entry for that key; `restoreSnapshot(key)` returns `false` on a miss and `true` after restoring a compatible entry. By default snapshots live in the shared cache, whose retention may evict older keys, and survive destruction of the source stack. `{ scope: "instance" }` keeps a snapshot with the instance instead: it is never evicted, restores only into that instance, survives `resetData`, and is removed when the instance is destroyed. A snapshot also carries the database's schema-writers record, and a restore never lowers it before the data is in place. Other service handles have no snapshot methods. Creating a service records its definition. Configured public ports are bound during startup and retained across normal stop/start and owner reopening. An occupied saved port reports a conflict instead of moving. Automatic ports avoid every port reserved in the per-user port registry (see Public ports); a fixed port another stack holds is a conflict that names that stack. Omitted public endpoints are not exposed. @@ -63,7 +72,7 @@ Native public listeners bind to loopback. Container-runtime (Docker and Podman) On macOS, Podman runs containers in a `podman machine` VM. A full stack needs about 8 GiB of VM memory (`podman machine set --memory 8192`); the 2 GiB default runs out of memory and the kernel kills services. -Functions use a package-provided, self-contained Edge Runtime main service unless the configuration supplies `bootstrap` source; only an explicit `bootstrap` is saved with the service definition. Database versions belong in `config.version`; other recipes accept an optional top-level artifact `version`. +Functions use a package-provided, self-contained Edge Runtime main service unless the configuration supplies `bootstrap` source; only an explicit `bootstrap` is saved with the service definition. Every creation carries a required top-level `pin` naming its exact artifact (upstream version, revision, content hashes, and image digest); the owner never chooses a version. On Linux, native Functions project files must be outside `/tmp`: Edge Runtime uses a private filesystem at that path. Docker mounts project files at a separate runtime path. @@ -71,6 +80,8 @@ On Linux, native Functions project files must be outside `/tmp`: Edge Runtime us `find({ stateRoot, projectRoot, name })` derives the stack ID with the same identity rules as `create` and reads only that stack; `find({ stateRoot, id })` reads a known ID. It returns the saved definition with the live owner's endpoint, if any, or nothing when no such stack is saved. Unlike `discover`, an unreadable state document fails the call, naming the stack's directory, instead of being skipped. The Effect entrypoint's `StackId` schema validates an ID before lookup. +`findDataConflicts({ stateRoot, projectRoot, requested })` reads the saved stacks of `projectRoot`, without any lease or owner probe, and lists what a start at the `requested` pins (a partial artifact selection) would refuse; it creates nothing, so a missing `stateRoot` has no conflicts. A `database-build` conflict is database data built from another upstream than `requested.database`: initialized data, judged by the rules a start applies for the stack's runtime, or the identity a Docker or Podman first start recorded before it was interrupted; its `recorded` is `undefined` for data with no recorded identity, and a newer slim-services revision of the same build is no conflict. A `schema-downgrade` conflict is a schema-owning service the saved composition binds to that database through its `databaseUrl`, requested older than the newest version recorded for it. Each conflict carries its `stackId`, the database `instanceId` that holds the data, and the `message` a start reports. Uninitialized data with no recorded identity never conflicts with a build. The messages offer to keep the data by pinning the recorded build in `[local.services.database] version`, also setting `[db] major_version` when the major differs and removing any `[db] orioledb_version` when an OrioleDB build is requested, before they offer to recreate the stack. `@supabase/stack/testing` exports `seedSavedStack` to save such a stack and its data records without starting anything. + Pass `startOwner: true` to `open` when live status and other owner-backed operations are needed; this starts only the detached owner and does not start services. The owner writes its output to `<stateRoot>/<id>/owner.log`, which each owner start truncates; owner start and connection failures name that file. A client drives only an owner of its own release; other operations fail and ask you to stop or destroy the stack, which work across releases. ### Lifetimes @@ -139,8 +150,8 @@ Registering a service does not add it to the application composition. For a fres const services = await stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: "local-password", jwtSecret: "local-development-secret-at-least-32-characters", jwtExpiry: 3600, @@ -149,6 +160,7 @@ const services = await stack.composition.supabase([ }, { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }, @@ -160,7 +172,7 @@ The factory accepts one instance of each selected recipe, binds its configured p Inputs that the composition binds or the owner fills from the stack credentials, such as REST's `databaseUrl` or a JWT secret, are optional in creation configuration. Starting or restarting an instance without a required input that was neither bound nor provided fails before any lifecycle change with a `ServiceError` whose `operation` is `"input"` and whose message names the input; a running instance keeps running. -To recompose a stopped stack, pass the IDs to keep in `reuseIds`. `composition.plan(services)` compares requested creations with every saved instance of the same kinds without contacting the owner or changing state. Each entry reports its instance `id`, `service`, whether it is a composition `member`, and a `change`: `unchanged`, `changed` with the differing config `paths`, or `incompatible` with the `paths` of an endpoint, artifact version, or PostgreSQL major-version change that the saved instance cannot adopt. Inputs that the composition or the stack credentials supply are ignored, an automatic API port is compared as the shared fixed port the composition would assign, and a PostgreSQL major alias matches its pinned version. Recomposing with `reuseIds` replaces a `changed` instance's configuration while keeping its identity, data, and ports. +To recompose a stopped stack, pass the IDs to keep in `reuseIds`. `composition.plan(services)` compares requested creations with every saved instance of the same kinds without contacting the owner or changing state. Each entry reports its instance `id`, `service`, whether it is a composition `member`, and a `change`: `unchanged`, `changed` with the differing config `paths`, or `incompatible` with the `paths` of an endpoint or PostgreSQL release-line change that the saved instance cannot adopt. Inputs that the composition or the stack credentials supply are ignored, an automatic API port is compared as the shared fixed port the composition would assign; a pin change on any other service is a `changed` configuration. Recomposing with `reuseIds` replaces a `changed` instance's configuration while keeping its identity, data, and ports. Add `retainMembers: true` to rebind only the requested services, which must not expose an API endpoint, and leave the other saved members and their dependencies in place. The database accepts a changed pin on initialized data only as a new slim-services revision of the build that created it. Auth, Storage, Realtime, Analytics and the pooler refuse a pin whose upstream version is older than the newest one that already migrated the database, which the database instance records in `schema-writers.json`; the error names the stack to destroy. Other services accept any pin change. The whole-stack E2E suite covers the default lazy lifecycle and reopen, all-eager startup, idle and wake, and parallel stack isolation for native, Docker, and Podman runtimes, split into a `lifecycle` and an `idle-parallel` file per runtime. The Podman files need `SUPABASE_STACK_TEST_ENGINE=podman`; CI runs them, and the container integration suite, on the Linux runner image's rootless Podman. macOS `podman machine` is not covered by CI. Run one runtime with a filter such as `pnpm --filter @supabase/stack test:e2e:run src/whole-stack.native` (matches both native files) or target one file, for example `src/whole-stack.docker.idle-parallel.e2e.test.ts`. @@ -169,6 +181,7 @@ For multiple instances or custom dependencies, configure members and bindings ex ```ts const rest = await stack.services.create({ service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -220,6 +233,8 @@ The stack is a session stack composed with `composition.supabase` and `{ eager: Every test stack for the current OS user shares one state root and the package's artifact cache root under the OS temp directory, so their ports are coordinated and Docker uses one data volume (Podman uses host directories); each gets a unique temporary project root and name. Pass `stateRoot`, `cacheRoot`, or `projectRoot` to override them. The runtime comes from `runtime`, then `SUPABASE_STACK_TEST_RUNTIME`, then the platform default: native where the catalog publishes native artifacts and Docker elsewhere. A startup failure names the state of each registered service and the owner log. +Services run the compiled default pins. With a caller-owned `projectRoot`, the stack reads `supabase/supabase.lock` once at creation and runs the pins it records, so a test runs the versions the project locked; it never resolves versions or checks the lock for staleness. A lock that is malformed or from a newer format fails creation with `LockInvalidError`, `LockUnsupportedVersionError`, or `LockIoError`, never a fallback to the defaults. An explicit `pin` on a service still wins. + `checkpoint(name)` stops the composition, saves the database data as an instance-scoped snapshot, and starts the composition again. `reset(name)` stops it, clears the database data, restores that snapshot, and waits until every service is ready. Checkpoints are never evicted by other stacks' snapshots and are removed when the test stack is destroyed. Effect tests use the scoped `makeTestStack`, which destroys the stack when its scope closes: diff --git a/packages/stack/scripts/prepare-test-artifacts.ts b/packages/stack/scripts/prepare-test-artifacts.ts index fd182c98f4..c98bb5c19c 100644 --- a/packages/stack/scripts/prepare-test-artifacts.ts +++ b/packages/stack/scripts/prepare-test-artifacts.ts @@ -1,6 +1,6 @@ import { NodeHttpClient, NodeRuntime, NodeServices } from "@effect/platform-node"; import { Console, Effect, Layer } from "effect"; -import { artifactServiceKinds, catalogPins, prepareNativeArtifact } from "../src/Artifacts.ts"; +import { catalogPins, prepareNativeArtifact } from "../src/Artifacts.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; /** @@ -8,9 +8,9 @@ import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; * With `--images`, prints every catalog container image instead so CI can pull them. */ const prepareNative = Effect.forEach( - [...artifactServiceKinds(), "vector" as const], - (service) => - prepareNativeArtifact({ service }, testArtifactCacheRoot).pipe( + catalogPins().filter(({ isDefault }) => isDefault), + ({ service, pin }) => + prepareNativeArtifact({ service, pin }, testArtifactCacheRoot).pipe( Effect.tap((artifact) => Console.log(`${service} ${artifact.version}`)), ), { concurrency: "unbounded", discard: true }, diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 8bcd3a8546..fe8a57c70a 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -1,6 +1,7 @@ -import { Data, Effect, Path, Record } from "effect"; +import { Data, Effect, Path, Record, Schema } from "effect"; import type { StackFailureKind } from "./FailureKind.ts"; import { makeArtifactStore } from "./preparation/ArtifactStore.ts"; +import { SLIM_IMAGE_GHCR_REGISTRY, slimReleasesUrl } from "./preparation/SlimEndpoints.ts"; import { makeSlimServicesSource, type SlimServicesArtifact, @@ -37,20 +38,24 @@ export class ArtifactError extends Data.TaggedError("ArtifactError")<{ }> {} /** Lowercase hexadecimal SHA-256 digest. */ -type Sha256 = string; +const Sha256 = Schema.String.check(Schema.isPattern(/^[0-9a-f]{64}$/u)); /** Content digests of one native target's published archive and manifest. */ -export interface NativePin { - readonly archive: Sha256; - readonly manifest: Sha256; -} +export const NativePin = Schema.Struct({ archive: Sha256, manifest: Sha256 }).annotate({ + identifier: "NativePin", +}); +export interface NativePin extends Schema.Schema.Type<typeof NativePin> {} -/** One published slim-services revision of an upstream version, pinned by content. */ -export interface ArtifactPin { - readonly upstreamVersion: string; - readonly revision: number; +/** + * One published slim-services revision of an upstream version, pinned by content. A pin stands + * alone and need not be in the compiled catalog; a service kind's executable and runtime paths + * come from its artifact definition. + */ +export const ArtifactPin = Schema.Struct({ + upstreamVersion: Schema.String, + revision: Schema.Int, /** `ghcr.io/supabase/cli/<service>:<release version>@sha256:<digest>`. */ - readonly image: string; + image: Schema.String, /** * The exact upstream image this release was built or mirrored from, as slim-services recorded * it: the release manifest's `upstream_image` for a derived service, or the release's @@ -58,9 +63,17 @@ export interface ArtifactPin { * the Dockerfile's `FROM` form — no leading `docker.io/`, no digest — so legacy non-slim mode * can use it as the upstream tag directly. */ - readonly upstreamImage: string; - readonly natives: Readonly<Record<NativeTarget, NativePin>>; -} + upstreamImage: Schema.String, + natives: Schema.Struct({ + "darwin-arm64": NativePin, + "linux-amd64": NativePin, + "linux-arm64": NativePin, + }), +}).annotate({ identifier: "ArtifactPin" }); +export interface ArtifactPin extends Schema.Schema.Type<typeof ArtifactPin> {} + +/** The pin each stack service runs, chosen once per command and written into every creation. */ +export type ArtifactSelection = Readonly<Record<ServiceKind, ArtifactPin>>; /** The published release version, `<upstream>-r<revision>`. */ const releaseVersion = (pin: ArtifactPin): string => `${pin.upstreamVersion}-r${pin.revision}`; @@ -83,12 +96,14 @@ export interface PreparedNativeArtifact { readonly executable: string; /** The generation's digest lock file; a native workload spawned from `root` pins this too. */ readonly lockPath: string; + /** Whether this call downloaded the generation or found it already installed. */ + readonly outcome: "cached" | "downloaded"; } interface ArtifactDefinition { readonly sourceService: string; - readonly defaultVersion: string; - /** Pins keyed by upstream version. */ + readonly defaultPin: ArtifactPin; + /** Compiled pins keyed by upstream version. */ readonly pins: Readonly<Record<string, ArtifactPin>>; readonly requiredRuntimePaths: ReadonlyArray<string>; readonly executablePath: string; @@ -102,14 +117,12 @@ const definition = ( additionalPins: Readonly<Record<string, ArtifactPin>> = {}, ): ArtifactDefinition => ({ sourceService, - defaultVersion: pin.upstreamVersion, + defaultPin: pin, pins: { [pin.upstreamVersion]: pin, ...additionalPins }, requiredRuntimePaths, executablePath, }); -const SLIM_IMAGE_GHCR_REGISTRY = "ghcr.io/supabase/cli/"; - const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { database: definition( "postgres", @@ -561,8 +574,6 @@ const platformText = (platform: { readonly os: string; readonly arch: string }): const errorMessage = (cause: unknown): string => cause instanceof Error ? cause.message : typeof cause === "string" ? cause : String(cause); -const SLIM_NATIVE_GITHUB_RELEASES = "https://github.com/supabase/slim-services/releases/download"; - /** * Public S3 copy of the release assets for hosts that block GitHub release downloads, such as * agent sandboxes that allow `*.amazonaws.com`. @@ -582,11 +593,12 @@ const artifactFor = ( service: ArtifactKind, resolved: ArtifactResolution, target: NativeTarget, + releasesUrl: string, ): SlimServicesArtifact => { const sourceService = definitions[service].sourceService; const releaseTag = `${sourceService}-${resolved.releaseVersion}`; const assetName = `${releaseTag}-${target}`; - const githubRelease = `${SLIM_NATIVE_GITHUB_RELEASES}/${releaseTag}`; + const githubRelease = `${releasesUrl}/${releaseTag}`; const supabaseS3 = `${SLIM_NATIVE_SUPABASE_S3_MIRROR}/${sourceService}/${resolved.releaseVersion}`; const pin = resolved.natives[target]; return { @@ -614,9 +626,10 @@ const artifactFor = ( }; }; +/** Binds an explicit pin to its service kind's executable and runtime paths. */ export const resolveArtifact = Effect.fn("Artifacts.resolveArtifact")(function* (request: { readonly service: ArtifactKind; - readonly version?: string; + readonly pin: ArtifactPin; }) { if (!Object.hasOwn(definitions, request.service)) return yield* new ArtifactError({ @@ -624,18 +637,10 @@ export const resolveArtifact = Effect.fn("Artifacts.resolveArtifact")(function* kind: "configuration", }); const selected = definitions[request.service]; - const version = request.version ?? selected.defaultVersion; - const pin = Object.entries(selected.pins).find(([candidate]) => candidate === version)?.[1]; - if (pin === undefined) - return yield* new ArtifactError({ - message: `Unsupported ${request.service} artifact version: ${version}`, - service: request.service, - version, - kind: "configuration", - }); + const { pin } = request; return { service: request.service, - version, + version: pin.upstreamVersion, releaseVersion: releaseVersion(pin), image: pin.image, natives: pin.natives, @@ -644,6 +649,22 @@ export const resolveArtifact = Effect.fn("Artifacts.resolveArtifact")(function* }; }); +/** The compiled default pin of every stack service; the database runs the stock default line. */ +export const defaultArtifactSelection = (): ArtifactSelection => ({ + database: definitions.database.defaultPin, + rest: definitions.rest.defaultPin, + auth: definitions.auth.defaultPin, + realtime: definitions.realtime.defaultPin, + storage: definitions.storage.defaultPin, + imgproxy: definitions.imgproxy.defaultPin, + functions: definitions.functions.defaultPin, + studio: definitions.studio.defaultPin, + pgmeta: definitions.pgmeta.defaultPin, + mail: definitions.mail.defaultPin, + analytics: definitions.analytics.defaultPin, + pooler: definitions.pooler.defaultPin, +}); + const ORIOLEDB_SUFFIX = "-orioledb"; /** PostgreSQL major of a database artifact version or major alias. */ @@ -672,11 +693,34 @@ export const orioledbConfigVersion = (version: string): string => export const orioledbVersions = (): ReadonlyArray<string> => Object.keys(definitions.database.pins).filter(isOrioledbVersion).map(orioledbConfigVersion); -/** Resolves a PostgreSQL major alias against the pinned stock database artifacts. */ -export const postgresVersion = (version: string): string => - Object.keys(definitions.database.pins).find( - (candidate) => !isOrioledbVersion(candidate) && postgresMajor(candidate) === version, - ) ?? version; +/** + * The compiled database pin for a PostgreSQL major alias (the pinned stock build of that major) or + * an exact pinned upstream version such as an OrioleDB build. + */ +export const databaseArtifactPin = (version: string): Effect.Effect<ArtifactPin, ArtifactError> => { + const pins = Object.values(definitions.database.pins); + const pin = + pins.find((candidate) => candidate.upstreamVersion === version) ?? + pins.find( + (candidate) => + !isOrioledbVersion(candidate.upstreamVersion) && + postgresMajor(candidate.upstreamVersion) === version, + ); + return pin === undefined + ? Effect.fail( + new ArtifactError({ + message: `Unsupported database artifact version: ${version}`, + service: "database", + version, + kind: "configuration", + }), + ) + : Effect.succeed(pin); +}; + +/** The slim-services producer that publishes releases of an artifact. */ +export const artifactSourceService = (service: ArtifactKind): string => + definitions[service].sourceService; /** Service kinds the stack runs, in artifact catalog order; legacy-only artifacts are omitted. */ export const artifactServiceKinds = (): ReadonlyArray<ServiceKind> => @@ -684,7 +728,7 @@ export const artifactServiceKinds = (): ReadonlyArray<ServiceKind> => /** * Every catalog pin in catalog order, including additional upstream lines. `isDefault` marks the - * pin `resolveArtifact` picks when no version is requested (postgres's 17.x line today); every + * pin the default selection runs (postgres's 17.x line today); every * other pin (postgres's 15.x and OrioleDB lines) carries `isDefault: false`. */ export const catalogPins = (): ReadonlyArray<{ @@ -694,12 +738,12 @@ export const catalogPins = (): ReadonlyArray<{ readonly isDefault: boolean; }> => Record.keys(definitions).flatMap((service) => { - const { sourceService, defaultVersion, pins } = definitions[service]; + const { sourceService, defaultPin, pins } = definitions[service]; return Object.values(pins).map((pin) => ({ service, sourceService, pin, - isDefault: pin.upstreamVersion === defaultVersion, + isDefault: pin.upstreamVersion === defaultPin.upstreamVersion, })); }); @@ -708,7 +752,7 @@ const artifactKey = (artifact: SlimServicesArtifact): string => /** Builds the native store request shared by ahead-of-time preparation and launch-time use. */ const nativeStoreRequest = Effect.fn("Artifacts.nativeStoreRequest")(function* ( - request: { readonly service: ArtifactKind; readonly version?: string }, + request: { readonly service: ArtifactKind; readonly pin: ArtifactPin }, platform: { readonly os: string; readonly arch: string }, ) { const resolved = yield* resolveArtifact(request); @@ -721,7 +765,8 @@ const nativeStoreRequest = Effect.fn("Artifacts.nativeStoreRequest")(function* ( platform: platformText(platform), kind: "platform-unsupported", }); - const sourceArtifact = artifactFor(request.service, resolved, target); + const releasesUrl = yield* slimReleasesUrl; + const sourceArtifact = artifactFor(request.service, resolved, target, releasesUrl); const key = artifactKey(sourceArtifact); const source = makeSlimServicesSource((candidate) => candidate.key === key ? sourceArtifact : undefined, @@ -739,7 +784,11 @@ const nativeStoreRequest = Effect.fn("Artifacts.nativeStoreRequest")(function* ( const toNativeArtifact = ( resolved: ArtifactResolution, - prepared: { readonly path: string; readonly lockPath: string }, + prepared: { + readonly path: string; + readonly lockPath: string; + readonly outcome: "cached" | "downloaded"; + }, path: Path.Path, ): PreparedNativeArtifact => ({ service: resolved.service, @@ -747,11 +796,12 @@ const toNativeArtifact = ( root: prepared.path, executable: path.join(prepared.path, resolved.executablePath), lockPath: prepared.lockPath, + outcome: prepared.outcome, }); /** Ahead-of-time: downloads and publishes the generation, but pins nothing. */ export const prepareNativeArtifact = Effect.fn("Artifacts.prepareNativeArtifact")(function* ( - request: { readonly service: ArtifactKind; readonly version?: string }, + request: { readonly service: ArtifactKind; readonly pin: ArtifactPin }, cacheRoot: string, platform: { readonly os: string; readonly arch: string } = { os: process.platform, @@ -784,7 +834,7 @@ export const prepareNativeArtifact = Effect.fn("Artifacts.prepareNativeArtifact" * its paths. Every consumer must use the returned paths only inside this scope. */ export const useNativeArtifact = Effect.fn("Artifacts.useNativeArtifact")(function* ( - request: { readonly service: ServiceKind; readonly version?: string }, + request: { readonly service: ServiceKind; readonly pin: ArtifactPin }, cacheRoot: string, platform: { readonly os: string; readonly arch: string } = { os: process.platform, diff --git a/packages/stack/src/Commands.integration.test.ts b/packages/stack/src/Commands.integration.test.ts index d7b4d0bb14..566f82d770 100644 --- a/packages/stack/src/Commands.integration.test.ts +++ b/packages/stack/src/Commands.integration.test.ts @@ -27,6 +27,7 @@ import { bindTcp, serveTcp } from "./Proxy.ts"; import { makeDockerDatabaseRoot } from "../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../tests/engine-target.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { database15, selection } from "../tests/pins.ts"; const cacheRoot = testArtifactCacheRoot; @@ -99,7 +100,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { const service = yield* makeStandaloneService(database.definition, { id: "database", config: { - version: "17", + pin: selection.database, databasePassword: Redacted.make("tool-password"), jwtSecret: Redacted.make("tool-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -135,7 +136,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { Ref.update(stderr, (text) => text + new TextDecoder().decode(bytes)), }; const sql = yield* runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: ["-X", "-v", "ON_ERROR_STOP=1"], env, stdin: Stream.make( @@ -149,7 +150,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(yield* Ref.get(stdout)).toContain("streamed-row"); yield* Ref.set(stdout, ""); const dump = yield* runPostgres(runner, { - postgres: postgres.pgDump({ major: 17 }), + postgres: postgres.pgDump({ pin: selection.database }), args: ["--data-only", "--table=tool_story"], env, ...output, @@ -159,7 +160,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(yield* Ref.get(stdout)).toContain("COPY public.tool_story"); expect(yield* Ref.get(stdout)).toContain("streamed-row"); const failed = yield* runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: ["-X", "-v", "ON_ERROR_STOP=1", "-c", "SELECT missing_column FROM tool_story"], env, ...output, @@ -168,7 +169,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(yield* Ref.get(stderr)).toContain("missing_column"); const rejected = yield* Effect.flip( runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: [], env, pgProve: { mounts: [] }, @@ -178,7 +179,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(rejected.message).toContain("pgProve options require the pg_prove command"); yield* Ref.set(stderr, ""); const early = yield* runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: ["-X", "-v", "ON_ERROR_STOP=1"], env, stdin: Stream.make( @@ -192,7 +193,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(yield* Ref.get(stderr)).toContain("missing_column"); const attached = yield* Deferred.make<void>(); const job = yield* runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: ["-X", "-q", "-t", "-A"], env: { ...env, PGAPPNAME: "attached-tool-story" }, stdin: Stream.make(new TextEncoder().encode("SELECT 'attached-ready';\n")).pipe( @@ -208,7 +209,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { yield* Fiber.interrupt(job); yield* Ref.set(stdout, ""); const remaining = yield* runPostgres(runner, { - postgres: postgres.psql({ major: 17 }), + postgres: postgres.psql({ pin: selection.database }), args: [ "-X", "-t", @@ -229,6 +230,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { } for (const major of [15, 17] as const) { + const pin = major === 15 ? database15 : selection.database; for (const runtime of ["native", testEngine] as const) { it.live(`${runtime} runs bundled pg_prove for PostgreSQL ${major}`, () => Effect.scoped( @@ -250,7 +252,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { const service = yield* makeStandaloneService(database.definition, { id: "database", config: { - version: String(major), + pin, databasePassword: Redacted.make(`pgprove-${runtime}-${major}-password`), jwtSecret: Redacted.make("tool-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -298,7 +300,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { Ref.update(stderr, (text) => text + new TextDecoder().decode(bytes)), }; const extension = yield* runPostgres(runner, { - postgres: postgres.psql({ major }), + postgres: postgres.psql({ pin }), args: ["-X", "-v", "ON_ERROR_STOP=1", "-c", "CREATE EXTENSION IF NOT EXISTS pgtap"], env, ...output, @@ -306,7 +308,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { expect(extension.exitCode).toBe(0); const pgProve = (file: string) => runPostgres(runner, { - postgres: postgres.pgProve({ major }), + postgres: postgres.pgProve({ pin }), args: ["--ext", ".sql", file, "--verbose"], env, pgProve: { @@ -347,7 +349,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { ); const cancelOutput = yield* Ref.make(""); const running = yield* runPostgres(runner, { - postgres: postgres.pgProve({ major }), + postgres: postgres.pgProve({ pin }), args: ["--ext", ".sql", "cancel-ready.sql", "cancel.sql", "--verbose"], env: { ...env, PGAPPNAME: applicationName }, pgProve: { @@ -371,7 +373,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { const idle = Effect.gen(function* () { yield* Ref.set(idleOutput, ""); const probe = yield* runPostgres(runner, { - postgres: postgres.psql({ major }), + postgres: postgres.psql({ pin }), args: [ "-X", "-t", @@ -397,7 +399,7 @@ describe("finite PostgreSQL commands", { timeout: 180_000 }, () => { const remaining = Effect.gen(function* () { yield* Ref.set(remainingOutput, ""); const probe = yield* runPostgres(runner, { - postgres: postgres.psql({ major }), + postgres: postgres.psql({ pin }), args: [ "-X", "-t", diff --git a/packages/stack/src/Commands.ts b/packages/stack/src/Commands.ts index ad45aef91e..d44bc25aa8 100644 --- a/packages/stack/src/Commands.ts +++ b/packages/stack/src/Commands.ts @@ -1,4 +1,5 @@ import { Schema } from "effect"; +import { ArtifactPin } from "./Artifacts.ts"; import { rejectExcessKeys } from "./internal/reject-excess-keys.ts"; const PgProveMount = Schema.Struct({ @@ -15,21 +16,25 @@ export interface PgProveOptions extends Schema.Schema.Type<typeof PgProveOptions export const PostgresCommand = Schema.Struct({ command: Schema.Literals(["pg_dump", "pg_dumpall", "pg_prove", "psql"]), - major: Schema.Literals([15, 17]), + pin: ArtifactPin, }); export interface PostgresCommand extends Schema.Schema.Type<typeof PostgresCommand> {} const AuthInitializationCommand = rejectExcessKeys( Schema.Struct({ type: Schema.Literal("auth.initialize"), - version: Schema.optional(Schema.String), + pin: ArtifactPin, + /** The database instance whose schemas this initialization writes. */ + databaseId: Schema.String, databaseUrl: Schema.String, }), ); const StorageInitializationCommand = rejectExcessKeys( Schema.Struct({ type: Schema.Literal("storage.initialize"), - version: Schema.optional(Schema.String), + pin: ArtifactPin, + /** The database instance whose schemas this initialization writes. */ + databaseId: Schema.String, databaseUrl: Schema.String, filePath: Schema.String, }), @@ -37,7 +42,9 @@ const StorageInitializationCommand = rejectExcessKeys( const RealtimeInitializationCommand = rejectExcessKeys( Schema.Struct({ type: Schema.Literal("realtime.initialize"), - version: Schema.optional(Schema.String), + pin: ArtifactPin, + /** The database instance whose schemas this initialization writes. */ + databaseId: Schema.String, databaseUrl: Schema.String, }), ); @@ -74,7 +81,7 @@ export type CommandInvocation = Schema.Schema.Type<typeof CommandInvocation>; export interface ResolvedCommand { readonly service: "auth" | "storage" | "realtime"; - readonly version: string; + readonly pin: ArtifactPin; readonly image: string; readonly nativeExecutable: string; readonly containerEntrypoint: string; @@ -91,21 +98,21 @@ export interface ResolvedCommand { /** Describes PostgreSQL clients without managing a database service. */ export const postgres = { - pgDump: ({ major }: { readonly major: 15 | 17 }): PostgresCommand => ({ + pgDump: ({ pin }: { readonly pin: ArtifactPin }): PostgresCommand => ({ command: "pg_dump", - major, + pin, }), - pgDumpAll: ({ major }: { readonly major: 15 | 17 }): PostgresCommand => ({ + pgDumpAll: ({ pin }: { readonly pin: ArtifactPin }): PostgresCommand => ({ command: "pg_dumpall", - major, + pin, }), - pgProve: ({ major }: { readonly major: 15 | 17 }): PostgresCommand => ({ + pgProve: ({ pin }: { readonly pin: ArtifactPin }): PostgresCommand => ({ command: "pg_prove", - major, + pin, }), - psql: ({ major }: { readonly major: 15 | 17 }): PostgresCommand => ({ + psql: ({ pin }: { readonly pin: ArtifactPin }): PostgresCommand => ({ command: "psql", - major, + pin, }), }; diff --git a/packages/stack/src/Defaults.ts b/packages/stack/src/Defaults.ts index 5df2d49d9b..fde219822e 100644 --- a/packages/stack/src/Defaults.ts +++ b/packages/stack/src/Defaults.ts @@ -39,3 +39,6 @@ export const DEFAULT_LOCAL_SERVICE_SECRET_KEY_BASE = "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG"; /** Key used to encrypt local Pooler tenant secrets. */ export const DEFAULT_POOLER_VAULT_ENCRYPTION_KEY = "12345678901234567890123456789032"; + +/** The compiled default artifact pin of every service kind. */ +export { defaultArtifactSelection } from "./Artifacts.ts"; diff --git a/packages/stack/src/FailureKind.ts b/packages/stack/src/FailureKind.ts index 744b6ac17c..a868bf7cd0 100644 --- a/packages/stack/src/FailureKind.ts +++ b/packages/stack/src/FailureKind.ts @@ -190,6 +190,9 @@ const classify = ( ); case "SupabaseCompositionError": return within(value.cause === undefined ? "configuration" : undefined); + case "DatabaseIdentityMismatch": + case "SchemaWriterDowngrade": + return "state"; case "InvalidStackIdentityError": case "InvalidProjectRootError": case "SchemaError": diff --git a/packages/stack/src/HostProcess.integration.test.ts b/packages/stack/src/HostProcess.integration.test.ts index 1f618349a2..b3015b994d 100644 --- a/packages/stack/src/HostProcess.integration.test.ts +++ b/packages/stack/src/HostProcess.integration.test.ts @@ -40,6 +40,7 @@ import type { ServiceCreationInput } from "./services/Catalog.ts"; import { watchLeaseRelease } from "../tests/owner.ts"; import { watchEntry } from "../tests/watch-entry.ts"; import * as StackNamespace from "./StackNamespace.ts"; +import { selection } from "../tests/pins.ts"; class ProcessTestError extends Data.TaggedError("ProcessTestError")<{ readonly message: string }> {} @@ -71,8 +72,8 @@ const savedStack = ( id: "database-1", creation: { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(databasePassword), jwtSecret: Redacted.make("argv-secrecy-jwt-secret-with-32-characters"), jwtExpiry: 3600, @@ -594,8 +595,8 @@ it.live( const requestedCreations: ReadonlyArray<ServiceCreationInput> = [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(marker), jwtSecret: Redacted.make("argv-secrecy-jwt-secret-with-32-characters"), jwtExpiry: 3600, diff --git a/packages/stack/src/Lock.integration.test.ts b/packages/stack/src/Lock.integration.test.ts new file mode 100644 index 0000000000..a2b679efe0 --- /dev/null +++ b/packages/stack/src/Lock.integration.test.ts @@ -0,0 +1,261 @@ +import { NodeServices } from "@effect/platform-node"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Option, Path, Schema } from "effect"; +import { database15, selection } from "../tests/pins.ts"; +import { + Lock, + LOCK_SCHEMA_URL, + lockSelection, + LockInvalidError, + LockUnsupportedVersionError, + readLock, + writeLock, +} from "./Lock.ts"; + +const bundled = (pin: Lock["services"]["rest"]["resolved"]) => + ({ specifier: "bundled", resolution: "resolved", resolved: pin }) as const; + +const lockOf = (pins: typeof selection, majorVersion = 17): Lock => ({ + $schema: LOCK_SCHEMA_URL, + lockfileVersion: 1, + services: { + pooler: bundled(pins.pooler), + analytics: bundled(pins.analytics), + mail: bundled(pins.mail), + pgmeta: bundled(pins.pgmeta), + studio: bundled(pins.studio), + functions: bundled(pins.functions), + imgproxy: bundled(pins.imgproxy), + storage: bundled(pins.storage), + realtime: bundled(pins.realtime), + auth: bundled(pins.auth), + rest: bundled(pins.rest), + database: { ...bundled(pins.database), majorVersion, line: "stock" }, + }, +}); + +const json = Schema.fromJsonString(Schema.Unknown); +const parse = Schema.decodeUnknownSync(json); +const stringify = Schema.encodeSync(json); + +const keysSortedEverywhere = (value: unknown): boolean => { + if (typeof value !== "object" || value === null) return true; + const keys = Object.keys(value); + return keys.toSorted().join() === keys.join() && Object.values(value).every(keysSortedEverywhere); +}; + +const inProject = <A, E>( + body: ( + projectRoot: string, + lockFile: string, + ) => Effect.Effect<A, E, FileSystem.FileSystem | Path.Path>, +) => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const projectRoot = yield* fs.makeTempDirectoryScoped({ prefix: "supabase-stack-lock-" }); + return yield* body(projectRoot, `${projectRoot}/supabase/supabase.lock`); + }), + ).pipe(Effect.provide(NodeServices.layer)); + +describe("lock", () => { + it.live("a project without a lock reads as none", () => + inProject((projectRoot) => + Effect.gen(function* () { + expect(yield* readLock(projectRoot)).toEqual(Option.none()); + }), + ), + ); + + it.live("writing, reading and writing again produces identical sorted bytes", () => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const lock = lockOf(selection); + + yield* writeLock(projectRoot, lock); + const first = yield* fs.readFileString(lockFile); + const read = yield* readLock(projectRoot); + yield* writeLock(projectRoot, Option.getOrThrow(read)); + const second = yield* fs.readFileString(lockFile); + + expect(Option.getOrThrow(read)).toEqual(lock); + expect(second).toBe(first); + expect(first.endsWith("}\n")).toBe(true); + expect(keysSortedEverywhere(parse(first))).toBe(true); + }), + ), + ); + + it.live.skipIf(process.platform === "win32")( + "a written lock is readable by everyone, as a committed file", + () => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + + yield* writeLock(projectRoot, lockOf(selection)); + + expect((yield* fs.stat(lockFile)).mode & 0o777).toBe(0o644 & ~process.umask()); + }), + ), + ); + + it.live("a lock with a newer lockfileVersion fails with an upgrade error", () => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.makeDirectory(`${projectRoot}/supabase`, { recursive: true }); + yield* fs.writeFileString(lockFile, stringify({ lockfileVersion: 2, shape: "new" })); + + const error = yield* Effect.flip(readLock(projectRoot)); + + expect(error).toBeInstanceOf(LockUnsupportedVersionError); + expect(error.message).toContain("upgrade the CLI"); + }), + ), + ); + + it.live("a lock of the supported version that does not decode is invalid, never absent", () => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.makeDirectory(`${projectRoot}/supabase`, { recursive: true }); + const { rest: _rest, ...services } = lockOf(selection).services; + yield* fs.writeFileString(lockFile, stringify({ ...lockOf(selection), services })); + const missingService = yield* Effect.flip(readLock(projectRoot)); + + yield* fs.writeFileString(lockFile, '{"lockfileVersion":1,'); + const truncated = yield* Effect.flip(readLock(projectRoot)); + + expect(missingService).toBeInstanceOf(LockInvalidError); + expect(truncated).toBeInstanceOf(LockInvalidError); + }), + ), + ); + + describe("pin images", () => { + const digest = "a".repeat(64); + const restPin = (image: string, upstreamVersion = "v14.1", revision = 3) => ({ + ...selection.rest, + upstreamVersion, + revision, + image, + }); + + const readWithRest = (pin: typeof selection.rest) => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.makeDirectory(`${projectRoot}/supabase`, { recursive: true }); + const lock = lockOf(selection); + yield* fs.writeFileString( + lockFile, + stringify({ ...lock, services: { ...lock.services, rest: bundled(pin) } }), + ); + return yield* readLock(projectRoot); + }), + ); + + it.live.each([ + { name: "a revision above zero", repository: "ghcr.io/supabase/cli/postgrest", revision: 3 }, + { name: "revision zero", repository: "ghcr.io/supabase/cli/postgrest", revision: 0 }, + ])("accepts the image of $name", ({ repository, revision }) => + Effect.gen(function* () { + const read = yield* readWithRest( + restPin(`${repository}:v14.1-r${revision}@sha256:${digest}`, "v14.1", revision), + ); + + expect(Option.getOrThrow(read).services.rest.resolved.revision).toBe(revision); + }), + ); + + it.live("rejects an image of a registry other than the canonical one", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + readWithRest( + restPin(`registry.example.com:5000/mirror/postgrest:v14.1-r3@sha256:${digest}`), + ), + ); + + expect(error).toBeInstanceOf(LockInvalidError); + expect(error.message).toContain("rest image"); + expect(error.message).toContain("must be ghcr.io/supabase/cli/postgrest:v14.1-r3@sha256:"); + }), + ); + + it.live("rejects an image without a digest", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + readWithRest(restPin("ghcr.io/supabase/cli/postgrest:v14.1-r3")), + ); + + expect(error).toBeInstanceOf(LockInvalidError); + expect(error.message).toContain("rest image"); + expect(error.message).toContain("@sha256:<64 hex digits>"); + }), + ); + + it.live("rejects an image tagged for another build than the pin", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + readWithRest(restPin(`ghcr.io/supabase/cli/postgrest:v14.1-r2@sha256:${digest}`)), + ); + + expect(error).toBeInstanceOf(LockInvalidError); + expect(error.message).toContain("rest image"); + expect(error.message).toContain("tagged v14.1-r2 but the pin is v14.1-r3"); + }), + ); + + it.live("rejects an image of another service's producer", () => + Effect.gen(function* () { + const error = yield* Effect.flip( + readWithRest(restPin(`ghcr.io/supabase/cli/auth:v14.1-r3@sha256:${digest}`)), + ); + + expect(error).toBeInstanceOf(LockInvalidError); + expect(error.message).toContain("is not a postgrest image"); + }), + ); + + it.live("rejects a database whose image names another build than its pin", () => + inProject((projectRoot, lockFile) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.makeDirectory(`${projectRoot}/supabase`, { recursive: true }); + const lock = lockOf(selection); + const stale = { ...selection.database, revision: selection.database.revision + 1 }; + yield* fs.writeFileString( + lockFile, + stringify({ + ...lock, + services: { + ...lock.services, + database: { ...lock.services.database, resolved: stale }, + }, + }), + ); + + const error = yield* Effect.flip(readLock(projectRoot)); + + expect(error).toBeInstanceOf(LockInvalidError); + expect(error.message).toContain("database image"); + }), + ), + ); + }); + + it.live("the selection of a lock is its resolved pins, including overrides", () => + inProject((projectRoot) => + Effect.gen(function* () { + const lock = lockOf({ ...selection, database: database15 }, 15); + yield* writeLock(projectRoot, lock); + + const read = Option.getOrThrow(yield* readLock(projectRoot)); + + expect(lockSelection(read)).toEqual({ ...selection, database: database15 }); + }), + ), + ); +}); diff --git a/packages/stack/src/Lock.ts b/packages/stack/src/Lock.ts new file mode 100644 index 0000000000..f4d1c31a15 --- /dev/null +++ b/packages/stack/src/Lock.ts @@ -0,0 +1,227 @@ +import { Effect, FileSystem, Option, Path, Predicate, Record, Schema } from "effect"; +import { + ArtifactPin, + artifactSourceService, + type ArtifactSelection, + type ServiceKind, +} from "./Artifacts.ts"; +import { SLIM_IMAGE_GHCR_REGISTRY } from "./preparation/SlimEndpoints.ts"; +import { publish } from "./StackNamespace.ts"; + +/** The `$schema` URL of lock format v1; it never changes once published. */ +export const LOCK_SCHEMA_URL = "https://supabase.com/docs/cli/supabase-lock/v1.schema.json"; + +const LOCK_VERSION = 1; + +/** `supabase.lock` inside the project's `supabase` directory. */ +const lockPath = (path: Path.Path, projectRoot: string): string => + path.join(projectRoot, "supabase", "supabase.lock"); + +/** What the hosted project reported for a service, kept even when a specifier overrides it. */ +const LinkedVersion = Schema.Struct({ raw: Schema.String, version: Schema.String }).annotate({ + identifier: "LinkedVersion", +}); + +/** + * How the specifier became `resolved`: as requested, or as the bundled pin because a `linked` + * request had no project, no observable version, or no published release. + */ +const Resolution = Schema.Literals(["resolved", "unlinked", "unobserved", "unpublished"]); + +/** `<producer>:<tag>@sha256:<digest>`, the part of a canonical image after the registry prefix. */ +const IMAGE_REFERENCE = /^([^/:@]+):([^/:@]+)@sha256:[0-9a-f]{64}$/u; + +/** + * The image of a pin must be the digest-pinned GHCR release `<upstreamVersion>-r<revision>` of the + * service's producer; download mirrors are derived from it when pulling, never recorded. + */ +const resolvedPin = (kind: ServiceKind) => + ArtifactPin.check( + Schema.makeFilter((pin) => { + const producer = artifactSourceService(kind); + const tag = `${pin.upstreamVersion}-r${pin.revision}`; + const canonical = `${SLIM_IMAGE_GHCR_REGISTRY}${producer}:${tag}@sha256:`; + const match = pin.image.startsWith(SLIM_IMAGE_GHCR_REGISTRY) + ? IMAGE_REFERENCE.exec(pin.image.slice(SLIM_IMAGE_GHCR_REGISTRY.length)) + : null; + if (match === null) return `${kind} image "${pin.image}" must be ${canonical}<64 hex digits>`; + if (match[1] !== producer) + return `${kind} image "${pin.image}" is not a ${producer} image, expected ${canonical}<digest>`; + return match[2] === tag + ? undefined + : `${kind} image "${pin.image}" is tagged ${match[2]} but the pin is ${tag}`; + }), + ); + +const serviceFields = (kind: ServiceKind) => ({ + /** The effective version request. */ + specifier: Schema.String, + linked: Schema.optionalKey(LinkedVersion), + resolution: Resolution, + resolved: resolvedPin(kind), +}); + +const serviceLock = (kind: ServiceKind) => + Schema.Struct(serviceFields(kind)).annotate({ identifier: "ServiceLock" }); + +const DatabaseLock = Schema.Struct({ + ...serviceFields("database"), + majorVersion: Schema.Int, + line: Schema.Literals(["stock", "orioledb"]), +}).annotate({ identifier: "DatabaseLock" }); + +/** The remote the `linked` observations belong to: a project plus one of its branches. */ +const LinkedRemote = Schema.Struct({ + project: Schema.String, + branch: Schema.Struct({ ref: Schema.String, isDefault: Schema.Boolean }), + observedAt: Schema.String, +}); + +/** The `supabase/supabase.lock` v1 document: the pin every stack service runs. */ +export const Lock = Schema.Struct({ + $schema: Schema.Literal(LOCK_SCHEMA_URL), + lockfileVersion: Schema.Literal(LOCK_VERSION), + linked: Schema.optionalKey(LinkedRemote), + services: Schema.Struct({ + database: DatabaseLock, + rest: serviceLock("rest"), + auth: serviceLock("auth"), + realtime: serviceLock("realtime"), + storage: serviceLock("storage"), + imgproxy: serviceLock("imgproxy"), + functions: serviceLock("functions"), + studio: serviceLock("studio"), + pgmeta: serviceLock("pgmeta"), + mail: serviceLock("mail"), + analytics: serviceLock("analytics"), + pooler: serviceLock("pooler"), + }), +}); +export interface Lock extends Schema.Schema.Type<typeof Lock> {} + +/** The lock declares a `lockfileVersion` newer than this build reads. */ +export class LockUnsupportedVersionError extends Schema.TaggedError<LockUnsupportedVersionError>()( + "LockUnsupportedVersionError", + { + path: Schema.String, + found: Schema.Int, + supported: Schema.Int, + message: Schema.String, + }, +) {} + +/** The lock, or a lock being written, is not a valid document of the supported version. */ +export class LockInvalidError extends Schema.TaggedError<LockInvalidError>()("LockInvalidError", { + path: Schema.String, + message: Schema.String, +}) {} + +/** Reading or writing the lock failed for a reason other than absence. */ +export class LockIoError extends Schema.TaggedError<LockIoError>()("LockIoError", { + path: Schema.String, + message: Schema.String, + cause: Schema.optionalKey(Schema.Defect()), +}) {} + +const decodeJson = Schema.decodeUnknownEffect(Schema.fromJsonString(Schema.Unknown)); +const decodeVersion = Schema.decodeUnknownOption( + Schema.Struct({ lockfileVersion: Schema.Int.check(Schema.isGreaterThan(LOCK_VERSION)) }), +); +const decodeLock = Schema.decodeUnknownEffect(Lock); + +const sortedKeysReplacer = (_key: string, value: unknown): unknown => + Predicate.isObject(value) && !Array.isArray(value) + ? Object.fromEntries(Object.entries(value).toSorted(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))) + : value; + +const encodeLock = Schema.encodeEffect( + Schema.fromJsonString(Lock, { space: 2, replacer: sortedKeysReplacer }), +); + +/** Reads `supabase/supabase.lock`, `None` when absent; takes no lease because writes are atomic renames. */ +export const readLock = Effect.fn("Lock.read")(function* (projectRoot: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const file = lockPath(path, projectRoot); + const content = yield* fs.readFileString(file).pipe( + Effect.map(Option.some), + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.succeed(Option.none<string>()) + : Effect.fail( + new LockIoError({ + path: file, + message: `failed to read ${file}: ${error.message}`, + cause: error, + }), + ), + ), + ); + if (Option.isNone(content)) { + yield* Effect.annotateCurrentSpan({ "lock.found": false }); + return Option.none<Lock>(); + } + const document = yield* decodeJson(content.value).pipe( + Effect.mapError( + (cause) => + new LockInvalidError({ + path: file, + message: `${file} is not valid JSON: ${cause.message}`, + }), + ), + ); + const newer = decodeVersion(document); + if (Option.isSome(newer)) { + return yield* new LockUnsupportedVersionError({ + path: file, + found: newer.value.lockfileVersion, + supported: LOCK_VERSION, + message: `${file} has lockfileVersion ${newer.value.lockfileVersion}, but this CLI reads version ${LOCK_VERSION}; upgrade the CLI.`, + }); + } + const lock = yield* decodeLock(document).pipe( + Effect.mapError( + (cause) => + new LockInvalidError({ + path: file, + message: `${file} is not a valid lock: ${cause.message}`, + }), + ), + ); + yield* Effect.annotateCurrentSpan({ "lock.found": true }); + return Option.some(lock); +}); + +/** Atomically writes `supabase/supabase.lock` as sorted-key JSON, so equal locks are equal bytes. */ +export const writeLock = Effect.fn("Lock.write")(function* (projectRoot: string, lock: Lock) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const file = lockPath(path, projectRoot); + const json = yield* encodeLock(lock).pipe( + Effect.mapError( + (cause) => + new LockInvalidError({ + path: file, + message: `refusing to write an invalid lock: ${cause.message}`, + }), + ), + ); + const content = `${json}\n`; + yield* Effect.gen(function* () { + yield* fs.makeDirectory(path.dirname(file), { recursive: true }); + yield* publish(fs, path, { target: file, content, mode: 0o644 }); + }).pipe( + Effect.mapError( + (cause) => + new LockIoError({ + path: file, + message: `failed to write ${file}: ${cause.message}`, + cause, + }), + ), + ); +}); + +/** The pin each service runs under this lock. */ +export const lockSelection = (lock: Lock): ArtifactSelection => + Record.map(lock.services, (service) => service.resolved); diff --git a/packages/stack/src/Owner.analytics.integration.test.ts b/packages/stack/src/Owner.analytics.integration.test.ts index 2ede6efb82..a926590c78 100644 --- a/packages/stack/src/Owner.analytics.integration.test.ts +++ b/packages/stack/src/Owner.analytics.integration.test.ts @@ -26,6 +26,7 @@ import type { ServiceCreationInput } from "./services/Catalog.ts"; import { makeDockerDatabaseRoot } from "../tests/docker-fixture.ts"; import { ownerFor } from "../tests/owner-rpc.ts"; import { engineTarget, testEngine } from "../tests/engine-target.ts"; +import { selection } from "../tests/pins.ts"; const cacheRoot = `${tmpdir()}/supabase-stack-artifacts`; @@ -69,8 +70,8 @@ const eventsTable = (internalDatabaseUrl: string, source: string) => const database: ServiceCreationInput = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-logs-password"), jwtSecret: Redacted.make("owner-logs-jwt-secret-with-at-least-32-chars"), jwtExpiry: 3600, @@ -79,6 +80,7 @@ const database: ServiceCreationInput = { }; const analytics: ServiceCreationInput = { service: "analytics", + pin: selection.analytics, config: { backend: "postgres", apiKey: "owner-logs-key" }, endpoints: { http: { port: "auto" } }, }; @@ -181,7 +183,12 @@ it.live( "owner-logs", [ database, - { service: "rest", config: {}, endpoints: { http: { port: "auto" } } }, + { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: "auto" } }, + }, analytics, ], 3_000, diff --git a/packages/stack/src/Owner.integration.test.ts b/packages/stack/src/Owner.integration.test.ts index 3fcf85660e..fffc45953d 100644 --- a/packages/stack/src/Owner.integration.test.ts +++ b/packages/stack/src/Owner.integration.test.ts @@ -17,6 +17,7 @@ import { ownerFor } from "../tests/owner-rpc.ts"; import { engineTarget, testEngine } from "../tests/engine-target.ts"; import { makeDockerDatabaseRoot } from "../tests/docker-fixture.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; const stateFor = (root: string) => Effect.gen(function* () { @@ -130,8 +131,8 @@ it.effect("credential lookup leaves a fresh stack untouched for custom database const jwtSecret = "custom-owner-credentials-jwt-secret-long-enough"; yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-credentials-database-password"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -162,8 +163,8 @@ it.live("forwards and rotates saved identity across composed services in one own const servicesFor = (jwtSecret?: string): ReadonlyArray<ServiceCreationInput> => [ { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("identity-rotation-password"), jwtExpiry: 3600, ...(jwtSecret === undefined ? {} : { jwtSecret: Redacted.make(jwtSecret) }), @@ -172,31 +173,37 @@ it.live("forwards and rotates saved identity across composed services in one own }, { service: "rest" as const, + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" as const } }, }, { service: "auth" as const, + pin: selection.auth, config: {}, endpoints: { http: { port: "auto" as const } }, }, { service: "storage" as const, + pin: selection.storage, config: { filePath: `${root}/uploads` }, endpoints: { http: { port: "auto" as const } }, }, { service: "realtime" as const, + pin: selection.realtime, config: {}, endpoints: { http: { port: "auto" as const } }, }, { service: "functions" as const, + pin: selection.functions, config: { functionsRoot: `${root}/functions`, bootstrap: "export {};" }, endpoints: { http: { port: "auto" as const } }, }, { service: "studio" as const, + pin: selection.studio, config: {}, endpoints: { http: { port: "auto" as const } }, }, @@ -265,6 +272,7 @@ it.live("forwards and rotates saved identity across composed services in one own const standaloneRest = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: {}, endpoints: {}, }); @@ -274,6 +282,7 @@ it.live("forwards and rotates saved identity across composed services in one own }); const standaloneAuth = yield* owner.rpc.createService({ service: "auth", + pin: selection.auth, config: {}, endpoints: {}, }); @@ -283,6 +292,7 @@ it.live("forwards and rotates saved identity across composed services in one own }); const standaloneStorage = yield* owner.rpc.createService({ service: "storage", + pin: selection.storage, config: { filePath: `${root}/standalone-uploads` }, endpoints: {}, }); @@ -294,6 +304,7 @@ it.live("forwards and rotates saved identity across composed services in one own }); const standaloneFunctions = yield* owner.rpc.createService({ service: "functions", + pin: selection.functions, config: { functionsRoot: `${root}/standalone-functions`, bootstrap: "export {};", @@ -419,6 +430,7 @@ it.effect("publishes service removal and composition pruning together", () => yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const service = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" }, pop3: { port: "auto" } }, }); @@ -457,8 +469,8 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const database = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-password"), jwtSecret: Redacted.make("owner-integration-jwt-secret-long-enough"), jwtExpiry: 3600, @@ -467,13 +479,14 @@ it.live( }); const rest = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }); const shadow = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-password"), jwtSecret: Redacted.make("owner-integration-jwt-secret-long-enough"), jwtExpiry: 3600, @@ -575,6 +588,7 @@ it.effect("isolates owner graphs built in one scope", () => yield* Effect.addFinalizer(() => secondOwner.namespace.destroy.pipe(Effect.ignore)); const created = yield* firstOwner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" }, pop3: { port: "auto" } }, }); @@ -610,8 +624,8 @@ it.live( services: [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-factory-password"), jwtSecret: Redacted.make("owner-factory-jwt-secret-long-enough"), jwtExpiry: 3600, @@ -620,11 +634,13 @@ it.live( }, { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }, { service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" } }, }, @@ -674,8 +690,8 @@ it.effect("validates Supabase composition recipes before creating instances", () yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const database = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-errors-password"), jwtSecret: Redacted.make("owner-errors-jwt"), jwtExpiry: 3600, @@ -692,11 +708,13 @@ it.effect("validates Supabase composition recipes before creating instances", () services: [ { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: 41_001 } }, }, { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: 41_002 } }, }, @@ -723,6 +741,7 @@ it.effect("validates Supabase composition recipes before creating instances", () const databaseWithoutSql = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: database.config, endpoints: {}, }); @@ -734,6 +753,7 @@ it.effect("validates Supabase composition recipes before creating instances", () { ...database, endpoints: {} }, { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }, @@ -757,7 +777,8 @@ it.effect("lets a retry choose other credentials after the first database creati yield* state.save(stack); const database = (password: string) => ({ service: "database" as const, - config: { version: "17", databasePassword: Redacted.make(password), jwtExpiry: 3600 }, + pin: selection.database, + config: { databasePassword: Redacted.make(password), jwtExpiry: 3600 }, endpoints: {}, }); const badDataRoot = `${root}/data-file`; @@ -789,7 +810,14 @@ it.effect( const owner = yield* ownerFor({ saved: stack, state, root: `${root}/data`, cacheRoot }); yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const definitions = yield* owner.rpc.supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: { http: { port: "auto" } } }], + services: [ + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], }); const mail = definitions.find((entry) => entry.creation.service === "mail"); if (mail === undefined) return yield* Effect.die("mail missing from the composition"); @@ -836,7 +864,14 @@ it.live("destroying one service deletes its port reservation", () => yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); // Composing reserves every member's public port up front without starting any of them. const definitions = yield* owner.rpc.supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: { http: { port: "auto" } } }], + services: [ + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], }); const mail = definitions.find((entry) => entry.creation.service === "mail"); if (mail === undefined) return yield* Effect.die("mail missing from the composition"); @@ -907,6 +942,7 @@ it.live("refuses to destroy one service after its data directory became a symlin yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const service = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" }, pop3: { port: "auto" } }, }); @@ -976,7 +1012,8 @@ it.effect("refuses to generate credentials for a stack whose saved instances con yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); yield* owner.rpc.createService({ service: "database", - config: { version: "17", jwtExpiry: 3600 }, + pin: selection.database, + config: { jwtExpiry: 3600 }, endpoints: {}, }); const withCredentials = yield* state.read(stack.id); @@ -986,7 +1023,7 @@ it.effect("refuses to generate credentials for a stack whose saved instances con yield* state.save(withoutCredentials); const refused = yield* owner.rpc - .createService({ service: "auth", config: {} }) + .createService({ service: "auth", pin: selection.auth, config: {} }) .pipe(Effect.flip); expect(refused.message).toBe( @@ -1013,6 +1050,7 @@ it.live("blocks a credential change while a standalone credential consumer runs" yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const standalone = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://authenticator@127.0.0.1:1/postgres" }, endpoints: {}, }); @@ -1021,7 +1059,7 @@ it.live("blocks a credential change while a standalone credential consumer runs" const refused = yield* owner.rpc .supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: {} }], + services: [{ service: "mail", pin: selection.mail, config: {}, endpoints: {} }], keys: { publishableKey: "publishable-rotated", secretKey: "secret-rotated", @@ -1060,19 +1098,21 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const rest = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://authenticator@127.0.0.1:1/postgres" }, endpoints: {}, }); const explicitKeys = '[{"kty":"oct","k":"explicit"}]'; const auth = yield* owner.rpc.createService({ service: "auth", + pin: selection.auth, config: { gotrueJwtKeys: explicitKeys }, endpoints: {}, }); const before = yield* owner.getStackCredentials; yield* owner.rpc.supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: {} }], + services: [{ service: "mail", pin: selection.mail, config: {}, endpoints: {} }], keys: { gotrueJwtKeys: "[]", publicSigningKeys: '[{"kty":"EC","kid":"rotated"}]', @@ -1109,6 +1149,7 @@ it.live("rejects a missing required input before starting or stopping the servic yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const unbound = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: {}, endpoints: {}, }); @@ -1120,12 +1161,16 @@ it.live("rejects a missing required input before starting or stopping the servic const provided = yield* owner.rpc.createService({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://authenticator@127.0.0.1:1/postgres" }, endpoints: {}, }); yield* owner.rpc.startService({ id: provided.id }); const restartFailure = yield* owner.rpc - .restartService({ id: provided.id, config: { service: "rest", config: { maxRows: 5 } } }) + .restartService({ + id: provided.id, + config: { service: "rest", config: { maxRows: 5 } }, + }) .pipe(Effect.flip); expect(restartFailure.message).toContain("rest requires input databaseUrl"); @@ -1164,8 +1209,8 @@ it.live( const created = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-no-registration-password"), jwtSecret: Redacted.make("owner-no-registration-jwt-secret-at-least-32-characters"), jwtExpiry: 3600, @@ -1236,8 +1281,8 @@ describe.runIf(testEngine === "docker")("Docker volume storage", () => { const created = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-no-data-root-password"), jwtSecret: Redacted.make("owner-no-data-root-jwt-secret-at-least-32-characters"), jwtExpiry: 3600, diff --git a/packages/stack/src/Owner.logs.integration.test.ts b/packages/stack/src/Owner.logs.integration.test.ts index 839290c235..31de225ccf 100644 --- a/packages/stack/src/Owner.logs.integration.test.ts +++ b/packages/stack/src/Owner.logs.integration.test.ts @@ -23,6 +23,7 @@ import { StackError, streamStackLogs } from "./effect.ts"; import * as LogStore from "./host/LogStore.ts"; import * as StackNamespace from "./StackNamespace.ts"; import type { SavedStack } from "./StackNamespace.ts"; +import { selection } from "../tests/pins.ts"; const cacheRoot = `${tmpdir()}/supabase-stack-artifacts`; const services = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); @@ -113,6 +114,7 @@ describe("owner persisted logs", () => { const owner = yield* openOwner("owner-logs-native-", "native"); const mail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -153,6 +155,7 @@ describe("owner persisted logs", () => { ); const mail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -162,12 +165,12 @@ describe("owner persisted logs", () => { yield* owner.rpc.stopService({ id: mail.id }); yield* Scope.close(ownerScope, Exit.void); - const selection = { stateRoot: owner.stateRoot, stackId: owner.stack.id }; - const records = yield* streamStackLogs(selection).pipe( + const logSelection = { stateRoot: owner.stateRoot, stackId: owner.stack.id }; + const records = yield* streamStackLogs(logSelection).pipe( Stream.filter(({ instanceId }) => instanceId === mail.id), Stream.runCollect, ); - const invalid = yield* streamStackLogs({ ...selection, since: "soon" }).pipe( + const invalid = yield* streamStackLogs({ ...logSelection, since: "soon" }).pipe( Stream.runDrain, Effect.flip, ); @@ -190,6 +193,7 @@ describe("owner persisted logs", () => { ); const mail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -239,6 +243,7 @@ describe("owner persisted logs", () => { ); const rest = yield* first.rpc.createService({ service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -301,6 +306,7 @@ describe("owner persisted logs", () => { const owner = yield* openOwner("owner-logs-resume-", "native"); const mail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -339,8 +345,8 @@ describe("owner persisted logs", () => { const owner = yield* openOwner("owner-logs-reset-", "native"); const database = yield* owner.rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("owner-logs-reset-password"), jwtExpiry: 3600, }, @@ -375,6 +381,7 @@ describe("owner persisted logs", () => { const owner = yield* openOwner("owner-logs-docker-", testEngine); const mail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); diff --git a/packages/stack/src/Owner.native-socket-sweep.integration.test.ts b/packages/stack/src/Owner.native-socket-sweep.integration.test.ts index 67ba27fa56..5dd757a539 100644 --- a/packages/stack/src/Owner.native-socket-sweep.integration.test.ts +++ b/packages/stack/src/Owner.native-socket-sweep.integration.test.ts @@ -19,11 +19,12 @@ import { NativeRuntimeRootBase, nativeRuntimeRootPath } from "./runtime/postgres import { ServiceCreation } from "./services/Catalog.ts"; import * as StackNamespace from "./StackNamespace.ts"; import { reclaimStack } from "./Sweep.ts"; +import { selection } from "../tests/pins.ts"; const databaseCreation = Schema.decodeEffect(ServiceCreation)({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("sweep-password"), jwtSecret: Redacted.make("sweep-integration-jwt-secret-long-enough"), jwtExpiry: 3600, diff --git a/packages/stack/src/Owner.ts b/packages/stack/src/Owner.ts index 354c8505ed..4467d43b89 100644 --- a/packages/stack/src/Owner.ts +++ b/packages/stack/src/Owner.ts @@ -2,6 +2,7 @@ import { Context, Crypto, Effect, + Exit, Fiber, FileSystem, Layer, @@ -18,6 +19,13 @@ import { HttpClient } from "effect/http"; import { ChildProcessSpawner } from "effect/process"; import type { Rpc, RpcGroup } from "effect/rpc"; import { failureMessage } from "./internal/failure-message.ts"; +import { + boundDatabaseId, + checkSchemaWriter, + holdSchemaWriter, + isSchemaWriter, + type SchemaWriterService, +} from "./internal/schema-writers.ts"; import * as Network from "./Network.ts"; import type { NetworkEndpoint, NetworkNamespace } from "./Network.ts"; import * as Orchestrator from "./Orchestrator.ts"; @@ -25,8 +33,11 @@ import type { CompositionConfig } from "./Orchestrator.ts"; import { makeService, ServiceError, + type RuntimeSession, type ServiceInstance, type ServiceInstanceContext, + type ServiceLaunchContext, + type ServiceLaunchError, } from "./Service.ts"; import { ProxyError } from "./Proxy.ts"; import { @@ -60,6 +71,7 @@ import { serviceSchemas, type CatalogRecipe, type ServiceCreationInput, + type ServiceRestartInput, } from "./services/Catalog.ts"; import type { CatalogError } from "./services/Recipe.ts"; import * as Container from "./runtime/Container.ts"; @@ -145,6 +157,10 @@ export interface Interface { StackCredentials, StackNamespace.NamespaceError | CredentialError >; + /** Fails unless `id` is a registered database instance, so a command cannot name another data directory. */ + readonly requireDatabase: ( + id: string, + ) => Effect.Effect<void, Orchestrator.OrchestratorError | ServiceError>; readonly namespace: { /** Stops every instance after in-flight definition changes settle, then removes containers. */ readonly stop: Effect.Effect<void, NamespaceError>; @@ -472,6 +488,54 @@ const makeOwner = Effect.fn("Owner.make")(function* ( ), ); + /** The composition database `instanceId` takes its `databaseUrl` from; `undefined` when the URL is not bound to one. */ + const boundDatabase = (instanceId: string) => + Effect.gen(function* () { + const { dependencies } = yield* orchestrator.composition; + const entries = yield* orchestrator.instances; + return boundDatabaseId( + dependencies, + (id) => entries.find((entry) => entry.id === id)?.service, + instanceId, + ); + }); + + const schemaWriterSelection = ( + databaseId: string, + service: SchemaWriterService, + version: string, + ) => ({ stackId, instanceRoot: path.join(options.root, databaseId), service, version }); + + /** + * Runs `launch` with the schema-writer lease of the local database the instance's `databaseUrl` + * is bound to held until the service first reports healthy, which is when its migrations have + * finished. A failed or interrupted check keeps the lease until the launch scope closes after + * the process is cleaned up, and a later successful probe releases it. A service that is not + * bound to a local database has no record and launches directly. + */ + const launchAsSchemaWriter = Effect.fnUntraced(function* ( + context: ServiceLaunchContext<ServiceCreation>, + launch: Effect.Effect<RuntimeSession, ServiceError | ServiceLaunchError>, + ) { + const { service, pin } = context.config; + const databaseId = isSchemaWriter(service) ? yield* boundDatabase(context.id) : undefined; + if (!isSchemaWriter(service) || databaseId === undefined) return yield* launch; + const lease = yield* Scope.fork(context.scope, "sequential"); + yield* holdSchemaWriter( + fs, + path, + schemaWriterSelection(databaseId, service, pin.upstreamVersion), + ).pipe(Scope.provide(lease), Effect.mapError(serviceError("launch"))); + const session = yield* launch; + return { + ...session, + health: session.health.pipe(Effect.tap(() => Scope.close(lease, Exit.void))), + ...(session.probe === undefined + ? {} + : { probe: session.probe.pipe(Effect.tap(() => Scope.close(lease, Exit.void))) }), + } satisfies RuntimeSession; + }); + const register = Effect.fn("Owner.register")(function* (id: string, recipe: CatalogRecipe) { if (Option.isSome(yield* orchestrator.get(id).pipe(Effect.option))) return yield* new Orchestrator.OrchestratorError({ @@ -494,7 +558,7 @@ const makeOwner = Effect.fn("Owner.make")(function* ( ), Effect.andThen(persistCreation({ id, creation }, context.config)), Effect.mapError(serviceError("state")), - Effect.andThen(recipe.definition.launch(context)), + Effect.andThen(launchAsSchemaWriter(context, recipe.definition.launch(context))), ), }, { @@ -656,6 +720,38 @@ const makeOwner = Effect.fn("Owner.make")(function* ( const clearUnusedCredentials = updateState(withoutUnusedCredentials).pipe( Effect.catch((cause) => Effect.logWarning("Unused stack credentials were not cleared", cause)), ); + /** + * Rejects a selection older than what already migrated the reused database that the requested + * schema writers bind to, before anything changes. + */ + const checkSchemaWriters = Effect.fn("Owner.checkSchemaWriters")( + function* ( + creations: ReadonlyArray<ServiceCreation>, + reuseIds: ReadonlyArray<string> | undefined, + ) { + if (!creations.some(({ service }) => service === "database")) return; + const reused = yield* Effect.forEach(reuseIds ?? [], (id) => + orchestrator.get(id).pipe(Effect.option), + ); + const databaseId = reused + .flatMap((entry) => (Option.isSome(entry) ? [entry.value] : [])) + .find(({ service }) => service === "database")?.id; + if (databaseId === undefined) return; + yield* Effect.forEach( + creations, + ({ service, pin }) => + isSchemaWriter(service) + ? checkSchemaWriter( + fs, + path, + schemaWriterSelection(databaseId, service, pin.upstreamVersion), + ) + : Effect.void, + { discard: true }, + ); + }, + Effect.mapError(serviceError("compose")), + ); const compose = Effect.fn("Owner.compose")( function* ( inputs: ReadonlyArray<ServiceCreationInput>, @@ -663,6 +759,7 @@ const makeOwner = Effect.fn("Owner.make")(function* ( ) { yield* resolveStackCredentials(yield* credentialOverrides(inputs), compositionOptions.keys); const creations = yield* Effect.forEach(inputs, resolveCredentials); + yield* checkSchemaWriters(creations, compositionOptions.reuseIds); return yield* makeSupabaseComposition<ComposeError>( { currentComposition: orchestrator.composition, @@ -721,16 +818,16 @@ const makeOwner = Effect.fn("Owner.make")(function* ( const restart = Effect.fn("Owner.restart")(function* ( id: string, - input: ServiceCreationInput | undefined, + input: ServiceRestartInput | undefined, ) { if (input === undefined) return yield* orchestrator.restart(id); const previous = yield* Ref.get((yield* orchestrator.get(id)).creation); - const next = yield* resolveCredentials(input); - if (next.service !== previous.service) + if (input.service !== previous.service) return yield* new ServiceError({ operation: "restart", message: "Service kind cannot change", }); + const next = yield* resolveCredentials({ ...input, pin: previous.pin }); return yield* orchestrator.restart(id, next); }); @@ -858,8 +955,8 @@ const makeOwner = Effect.fn("Owner.make")(function* ( ).pipe(rpcError("restoreSnapshot")), resetData: ({ id }) => databaseData(id, ({ resetDatabaseData }) => resetDatabaseData).pipe(rpcError("resetData")), - supabaseComposition: ({ services, reuseIds, keys, eager }) => - definitionChange(compose(services, { reuseIds, keys, eager })).pipe( + supabaseComposition: ({ services, reuseIds, keys, eager, retainMembers }) => + definitionChange(compose(services, { reuseIds, keys, eager, retainMembers })).pipe( rpcError("supabaseComposition"), ), configureComposition: (configuration) => @@ -900,6 +997,18 @@ const makeOwner = Effect.fn("Owner.make")(function* ( }, ).pipe(Effect.withSpan("Owner.claimEndpoints")), getStackCredentials, + requireDatabase: (id) => + orchestrator.get(id).pipe( + Effect.filterOrFail( + (entry) => entry.service === "database", + () => + new ServiceError({ + operation: "initialize", + message: `Instance ${id} is not a database`, + }), + ), + Effect.asVoid, + ), namespace: { // Services stop in reverse dependency order with their listeners open, so a dependent's // graceful stop can still reach its prerequisites through the proxy. diff --git a/packages/stack/src/Ports.integration.test.ts b/packages/stack/src/Ports.integration.test.ts index 9541a9c75b..4d075d7226 100644 --- a/packages/stack/src/Ports.integration.test.ts +++ b/packages/stack/src/Ports.integration.test.ts @@ -31,6 +31,7 @@ import { CONTAINER_ENV_DIRNAME } from "./namespace/Paths.ts"; import * as PortReservations from "./namespace/PortReservations.ts"; import * as StackNamespace from "./StackNamespace.ts"; import { ownerFor } from "../tests/owner-rpc.ts"; +import { selection } from "../tests/pins.ts"; /** * Scenarios the public `effect.ts` facade cannot reach deterministically: forcing an @@ -319,7 +320,14 @@ it.live( // mail), with no member ever started: a real reservation with no native artifact needed, // so this runs on every platform, Windows included. const definitions = yield* owner.rpc.supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: { http: { port: "auto" } } }], + services: [ + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], }); const mail = definitions.find((entry) => entry.creation.service === "mail"); if (mail === undefined) return yield* Effect.die("mail missing from the composition"); @@ -408,7 +416,14 @@ it.live("a destroy that cannot remove the registration keeps every reservation", }).pipe(Effect.ignore), ); const definitions = yield* owner.rpc.supabaseComposition({ - services: [{ service: "mail", config: {}, endpoints: { http: { port: "auto" } } }], + services: [ + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], }); const mail = definitions.find((entry) => entry.creation.service === "mail"); if (mail === undefined) return yield* Effect.die("mail missing from the composition"); diff --git a/packages/stack/src/PromiseClient.ts b/packages/stack/src/PromiseClient.ts index 739cf44d89..dd509fcf1a 100644 --- a/packages/stack/src/PromiseClient.ts +++ b/packages/stack/src/PromiseClient.ts @@ -13,7 +13,10 @@ import { import type * as StackEffect from "./effect.ts"; import { failureMessage } from "./internal/failure-message.ts"; import { StackError } from "./Rpc.ts"; -import { ServiceCreationInput as CreationSchema } from "./services/Catalog.ts"; +import { + ServiceCreationInput as CreationSchema, + ServiceRestartInput as RestartSchema, +} from "./services/Catalog.ts"; import type { InitializationCommand, PgProveOptions, PostgresCommand } from "./Commands.ts"; /** Optional cancellation ends the caller's wait, or its attached command job. */ @@ -296,6 +299,14 @@ const decodeCreation = (operation: string, creation: unknown) => (cause) => new StackError({ operation, message: cause.message, kind: "configuration" }), ), ); +const restartJson = Schema.toCodecJson(RestartSchema); +const decodeRestart = (candidate: unknown) => + Schema.decodeUnknownEffect(restartJson)(candidate).pipe( + Effect.mapError( + (cause) => + new StackError({ operation: "restart", message: cause.message, kind: "configuration" }), + ), + ); const decodeCreations = (operation: string, creations: ReadonlyArray<unknown>) => Effect.forEach(creations, (creation) => decodeCreation(operation, creation)); const sinkError = (cause: unknown) => @@ -314,7 +325,9 @@ export const stackAdapter = (client: Client) => { function instance<K extends Kind>(service: StackEffect.ServiceInstances[K]): ServiceInstances[K]; function instance(service: { readonly service: Kind; - readonly restart: (input?: StackEffect.ServiceCreationInput) => Effect.Effect<void, StackError>; + readonly restart: ( + input?: Pick<StackEffect.ServiceCreationInput, "config">, + ) => Effect.Effect<void, StackError>; }): unknown { return { ...promised(service), @@ -322,13 +335,10 @@ export const stackAdapter = (client: Client) => { client.run( input === undefined ? service.restart() - : decodeCreation( - "restart", - "service" in input ? input : { service: service.service, config: input.config }, - ).pipe( - Effect.flatMap((creation) => - creation.service === service.service - ? service.restart(creation) + : decodeRestart({ service: service.service, ...input }).pipe( + Effect.flatMap((candidate) => + candidate.service === service.service + ? service.restart(candidate) : Effect.fail( new StackError({ operation: "restart", diff --git a/packages/stack/src/Rpc.ts b/packages/stack/src/Rpc.ts index 2829997151..3c6f9e2f56 100644 --- a/packages/stack/src/Rpc.ts +++ b/packages/stack/src/Rpc.ts @@ -1,6 +1,6 @@ import { Cause, Exit, Option, Predicate, Schema } from "effect"; import { Rpc, RpcGroup } from "effect/rpc"; -import { ServiceCreation, ServiceCreationInput } from "./services/Catalog.ts"; +import { ServiceCreation, ServiceCreationInput, ServiceRestartInput } from "./services/Catalog.ts"; import { snapshotScopes } from "./services/DatabaseSnapshot.ts"; import { causeMessage, CompositionConfig, OrchestratorError } from "./Orchestrator.ts"; import { CommandInvocation } from "./Commands.ts"; @@ -180,7 +180,7 @@ export const OwnerRpc = RpcGroup.make( Rpc.make("readyService", { payload: Instance, error: StackError }), Rpc.make("stopService", { payload: Instance, error: StackError }), Rpc.make("restartService", { - payload: { ...Instance, config: Schema.optionalKey(ServiceCreationInput) }, + payload: { ...Instance, config: Schema.optionalKey(ServiceRestartInput) }, error: StackError, }), Rpc.make("destroyService", { payload: Instance, error: StackError }), @@ -225,6 +225,7 @@ export const OwnerRpc = RpcGroup.make( reuseIds: Schema.optionalKey(Schema.Array(Schema.String)), keys: Schema.optionalKey(StackKeysInput), eager: Schema.optionalKey(Schema.Boolean), + retainMembers: Schema.optionalKey(Schema.Boolean), }, success: Schema.Array(Definition), error: StackError, diff --git a/packages/stack/src/StackHost.container-shutdown.integration.test.ts b/packages/stack/src/StackHost.container-shutdown.integration.test.ts index 0a5a02e7cf..61dc222ece 100644 --- a/packages/stack/src/StackHost.container-shutdown.integration.test.ts +++ b/packages/stack/src/StackHost.container-shutdown.integration.test.ts @@ -32,6 +32,7 @@ import { engineTarget, testEngine } from "../tests/engine-target.ts"; import { holdReleaseFifo } from "../tests/release-fifo.ts"; import { shutdownOwner, watchLeaseRelease } from "../tests/owner.ts"; import { watchEntry } from "../tests/watch-entry.ts"; +import { selection } from "../tests/pins.ts"; const shortRegistrationPollFixture = fileURLToPath( new URL("../tests/short-registration-poll-fixture.ts", import.meta.url), @@ -179,8 +180,8 @@ it.live.skipIf(process.platform === "win32")( Effect.flatMap((rpc) => rpc.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(`stack-shutdown-password-${suffix}`), jwtSecret: Redacted.make( `stack-shutdown-jwt-secret-${suffix}-at-least-thirty-two-characters`, @@ -292,6 +293,7 @@ it.live.skipIf(process.platform === "win32")( const clientB3 = yield* ownerClient(accessB3); const mailA3 = yield* clientA3.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -299,6 +301,7 @@ it.live.skipIf(process.platform === "win32")( yield* clientA3.readyService({ id: mailA3.id }); const mailB3 = yield* clientB3.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -386,8 +389,8 @@ it.live.skipIf(process.platform === "win32")( const client = yield* ownerClient(access); const database = yield* client.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("registration-lost-password"), jwtSecret: Redacted.make("registration-lost-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -535,8 +538,8 @@ const losesRegistrationWhileStopRuns = (trigger: StopTrigger) => const client = yield* ownerClient(access); const database = yield* client.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("registration-lost-gated-password"), jwtSecret: Redacted.make( "registration-lost-gated-jwt-secret-at-least-thirty-two-characters", @@ -637,6 +640,7 @@ it.live.skipIf(process.platform === "win32")( // reaction to losing its mounted storage. const mail = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -693,6 +697,7 @@ it.live.skipIf(process.platform === "win32")( const reclaimedClient = yield* ownerClient(reclaimedAccess); const reclaimedMail = yield* reclaimedClient.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port } }, }); @@ -749,8 +754,8 @@ it.live.skipIf(process.platform === "win32")( const password = Redacted.make("pinned-postgres-password"); const database = yield* client.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: password, jwtSecret: Redacted.make("pinned-postgres-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, diff --git a/packages/stack/src/StackHost.integration.test.ts b/packages/stack/src/StackHost.integration.test.ts index 55bd6c79fa..6c32a34b60 100644 --- a/packages/stack/src/StackHost.integration.test.ts +++ b/packages/stack/src/StackHost.integration.test.ts @@ -38,6 +38,7 @@ import { shutdownOwner, watchLeaseRelease } from "../tests/owner.ts"; import { postgres } from "./Commands.ts"; import * as CommandRunner from "./host/CommandRunner.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; const shortRegistrationPollFixture = fileURLToPath( new URL("../tests/short-registration-poll-fixture.ts", import.meta.url), @@ -248,11 +249,13 @@ it.live("preserves composition outcomes over RPC", () => const port = yield* occupiedPort; const lazy = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); const blocked = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port } }, }); @@ -552,6 +555,7 @@ it.live( expect(yield* client.startComposition()).toEqual([]); const mail = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" }, pop3: { port: "auto" } }, }); @@ -563,7 +567,7 @@ it.live( attachmentId: toolAttachment, command: { type: "postgres", - command: postgres.psql({ major: 17 }), + command: postgres.psql({ pin: selection.database }), args: ["--version"], env: {}, stdin: true, @@ -582,8 +586,8 @@ it.live( expect((yield* client.status({ id: mail.id })).lifecycle).toBe("running"); const database = yield* client.createService({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("host-test-password"), jwtSecret: Redacted.make("host-test-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -616,7 +620,7 @@ it.live( attachmentId: "draining-stdin", command: { type: "postgres", - command: postgres.psql({ major: 17 }), + command: postgres.psql({ pin: selection.database }), args: [ "-X", "-t", @@ -772,6 +776,7 @@ it.live("withdraws a command waiting for its prerequisite", () => const client = yield* ownerClient(runtime.access); const mail = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -849,6 +854,7 @@ it.live("finishes a service creation after its caller disconnects", () => const creation = yield* Effect.forkScoped( client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }), @@ -912,6 +918,7 @@ it.live("persists a composition change after its caller disconnects", () => const client = yield* ownerClient(runtime.access); const mail = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -923,7 +930,12 @@ it.live("persists a composition change after its caller disconnects", () => yield* Fiber.interrupt(configure); yield* Deferred.await(interrupted); yield* Deferred.succeed(allow, undefined); - yield* client.createService({ service: "mail", config: {}, endpoints: {} }); + yield* client.createService({ + service: "mail", + pin: selection.mail, + config: {}, + endpoints: {}, + }); expect((yield* state.read(saved.id))?.composition).toEqual({ members, dependencies: [] }); yield* shutdownOwner(runtime.access, true); @@ -979,8 +991,8 @@ const abandonedComposition = (prefix: string, destroy: boolean) => services: [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("abandoned-composition-password"), jwtSecret: Redacted.make("abandoned-composition-jwt-secret-at-least-32-chars"), jwtExpiry: 3600, diff --git a/packages/stack/src/StackHost.ts b/packages/stack/src/StackHost.ts index 22ced23f74..12dec1b1e3 100644 --- a/packages/stack/src/StackHost.ts +++ b/packages/stack/src/StackHost.ts @@ -54,6 +54,7 @@ import { engineUnreachable, makeHostGateway, resolveEngineTarget } from "./runti import * as StackNamespace from "./StackNamespace.ts"; import { makeScopedPorts } from "./Ports.ts"; import { applyEndpointReplan, reportedEndpointChanges } from "./composition/EndpointReplan.ts"; +import { initializedDatabaseIds } from "./internal/database-reuse.ts"; import type { ServiceCreationInput } from "./services/Catalog.ts"; import { sweepOrphans } from "./Sweep.ts"; import { makeCommandAttachments } from "./host/CommandAttachments.ts"; @@ -224,7 +225,8 @@ export const makeRuntime = Effect.fn("StackHost.makeRuntime")( stdout: input.stdout, stderr: input.stderr, }) - : owner.getStackCredentials.pipe( + : owner.requireDatabase(input.command.databaseId).pipe( + Effect.andThen(owner.getStackCredentials), Effect.mapError( (cause) => new CommandRunner.CommandError({ message: cause.message, cause }), ), @@ -499,6 +501,12 @@ export const runStackHost = Effect.fn("StackHost.run")( replanRequest.ports, registered, replanRequest.requested, + yield* initializedDatabaseIds( + fs, + path, + path.join(options.stateRoot, registered.id, "data"), + registered.instances, + ), ).pipe(Effect.mapError((cause) => hostError("startup", cause))); const saved = replan?.saved ?? registered; const control = yield* bindControl(); diff --git a/packages/stack/src/StackNamespace.ts b/packages/stack/src/StackNamespace.ts index f7511a0deb..9d94d9450f 100644 --- a/packages/stack/src/StackNamespace.ts +++ b/packages/stack/src/StackNamespace.ts @@ -6,6 +6,7 @@ import { restrictDirectoryToOwner } from "./runtime/postgres-user.ts"; export { NamespaceError } from "./namespace/Capabilities.ts"; export { LeaseHeldError, LeaseHolder } from "./namespace/Lease.ts"; +export { holdLock, isLockBusy } from "./namespace/Lock.ts"; export { resolveStackDataRoot, stackDataRootLabel, stackLogsRoot } from "./namespace/Paths.ts"; export { publish } from "./namespace/Publication.ts"; export { diff --git a/packages/stack/src/composition/EndpointReplan.ts b/packages/stack/src/composition/EndpointReplan.ts index 4ed7098402..d2e8a1a3b1 100644 --- a/packages/stack/src/composition/EndpointReplan.ts +++ b/packages/stack/src/composition/EndpointReplan.ts @@ -33,8 +33,9 @@ export const applyEndpointReplan = Effect.fn("EndpointReplan.apply")(function* ( ports: Ports, saved: StackNamespace.SavedStack, requested: ReadonlyArray<ServiceCreationInput>, + initializedDatabases: ReadonlySet<string>, ) { - const plan = planEndpointReplan(saved, requested); + const plan = planEndpointReplan(saved, requested, initializedDatabases); if (plan === undefined) return undefined; const changed = yield* Effect.forEach(plan.changes, (change) => diff --git a/packages/stack/src/composition/Supabase.native.integration.test.ts b/packages/stack/src/composition/Supabase.native.integration.test.ts index 87a92d163b..a8716333ca 100644 --- a/packages/stack/src/composition/Supabase.native.integration.test.ts +++ b/packages/stack/src/composition/Supabase.native.integration.test.ts @@ -24,6 +24,7 @@ import { type SupabaseCompositionOperations, } from "./Supabase.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; const cacheRoot = testArtifactCacheRoot; // Below the stack's native and auto port ranges and every OS ephemeral range, so neither another @@ -80,7 +81,7 @@ it.live("reports the failed binding and cleanup reason with the retained instanc configure: unused, }; const error = yield* makeSupabaseComposition(operations, [ - { service: "mail", config: {}, endpoints: {} }, + { service: "mail", pin: selection.mail, config: {}, endpoints: {} }, ]).pipe(Effect.flip); expect(error.message).toContain("mail retained-mail: port is occupied"); expect(error.message).toContain("retained-mail: cannot remove owned data"); @@ -101,6 +102,7 @@ it.live("removes a managed SMTP binding when Mail is excluded", () => Effect.gen(function* () { const auth = { service: "auth" as const, + pin: selection.auth, config: { jwtSecret: "exclude-mail-jwt-secret-with-32-chars", jwtExpiry: 3600, @@ -110,7 +112,12 @@ it.live("removes a managed SMTP binding when Mail is excluded", () => const members = yield* owner.rpc.supabaseComposition({ services: [ auth, - { service: "mail", config: {}, endpoints: { smtp: { port: "auto" } } }, + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { smtp: { port: "auto" } }, + }, ], }); const instance = members.find(({ creation }) => creation.service === "auth"); @@ -145,8 +152,8 @@ it.live("cleans created registrations when composition is interrupted", () => const destroyed = yield* Ref.make<ReadonlyArray<string>>([]); const database = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("interrupt-password"), jwtSecret: Redacted.make("interrupt-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -208,8 +215,8 @@ it.live( services: [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -218,6 +225,7 @@ it.live( }, { service: "auth", + pin: selection.auth, config: { jwtSecret, jwtExpiry: 3600, @@ -226,6 +234,7 @@ it.live( }, { service: "storage", + pin: selection.storage, config: { filePath: storageRoot, jwtSecret, @@ -346,8 +355,8 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const databaseCreation = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("reuse-database-password"), jwtSecret: Redacted.make("reuse-database-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -371,6 +380,7 @@ it.live( databaseCreation, { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, }, @@ -397,6 +407,7 @@ it.live( rest.creation, { service: "auth", + pin: selection.auth, config: { jwtSecret: Redacted.value(databaseCreation.config.jwtSecret), jwtExpiry: 3600, @@ -423,6 +434,68 @@ it.live( { timeout: 180_000 }, ); +it.live( + "rebinds one stopped member alone and keeps the other members and their dependencies", + () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-native-retain-" }); + const stack = initial("catalog-native-retain"); + const state = yield* stateFor(`${root}/state`); + yield* state.save(stack); + const owner = yield* ownerFor({ + saved: stack, + state, + root: `${root}/data`, + cacheRoot, + }); + yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); + const databaseCreation = { + service: "database" as const, + pin: selection.database, + config: { + databasePassword: Redacted.make("retain-database-password"), + jwtSecret: Redacted.make("retain-database-jwt-secret-with-32-chars"), + jwtExpiry: 3600, + }, + endpoints: { sql: { port: "auto" as const } }, + }; + const created = yield* owner.rpc.supabaseComposition({ + services: [ + databaseCreation, + { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], + }); + const database = created.find((entry) => entry.creation.service === "database"); + if (database === undefined) return yield* Effect.die("Database member missing"); + const saved = yield* state.read(stack.id); + const before = saved?.composition; + expect(before?.members).toHaveLength(2); + expect(before?.dependencies.length).toBeGreaterThan(0); + + yield* owner.rpc.supabaseComposition({ + services: [databaseCreation], + reuseIds: [database.id], + retainMembers: true, + }); + expect((yield* state.read(stack.id))?.composition).toEqual(before); + + yield* owner.rpc.supabaseComposition({ + services: [databaseCreation], + reuseIds: [database.id], + }); + expect((yield* state.read(stack.id))?.composition.members).toHaveLength(1); + }), + ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + { timeout: 180_000 }, +); + it.live( "reuses an excluded Studio instance and its fixed port", () => @@ -442,8 +515,8 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const database = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("studio-reuse-password"), jwtSecret: Redacted.make("studio-reuse-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -452,16 +525,19 @@ it.live( }; const rest = { service: "rest" as const, + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" as const } }, }; const pgmeta = { service: "pgmeta" as const, + pin: selection.pgmeta, config: {}, endpoints: { http: { port: "auto" as const } }, }; const studio = { service: "studio" as const, + pin: selection.studio, config: { analyticsApiKey: "studio-reuse-analytics-key", jwtSecret: "studio-reuse-jwt-secret-with-32-chars", @@ -527,8 +603,8 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const databaseCreation = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("reuse-running-password"), jwtSecret: Redacted.make("reuse-running-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -571,6 +647,7 @@ it.live( yield* Effect.addFinalizer(() => owner.namespace.destroy.pipe(Effect.ignore)); const standaloneMail = yield* owner.rpc.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -582,8 +659,8 @@ it.live( if (mailPort === undefined) return yield* Effect.die("Missing mail http endpoint"); const database = { service: "database" as const, + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("reuse-cleanup-password"), jwtSecret: Redacted.make("reuse-cleanup-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -594,7 +671,12 @@ it.live( .supabaseComposition({ services: [ database, - { service: "mail", config: {}, endpoints: { http: { port: mailPort } } }, + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: mailPort } }, + }, ], }) .pipe(Effect.flip); diff --git a/packages/stack/src/composition/Supabase.ts b/packages/stack/src/composition/Supabase.ts index 9754ff1644..b707855045 100644 --- a/packages/stack/src/composition/Supabase.ts +++ b/packages/stack/src/composition/Supabase.ts @@ -1,5 +1,5 @@ import { Data, Effect, Exit, Redacted, Schema } from "effect"; -import { postgresVersion } from "../Artifacts.ts"; +import { postgresLine } from "../Artifacts.ts"; import { causeMessage, type CompositionConfig, isStoppedAndWakeDisabled } from "../Orchestrator.ts"; import type { Observation } from "../Rpc.ts"; import { ServiceCreation, type ServiceCreationInput } from "../services/Catalog.ts"; @@ -204,6 +204,8 @@ export interface SupabaseCompositionOptions { /** Reuses stopped instances; inputs declare desired bindings, not previous resolved creations. */ readonly reuseIds?: ReadonlyArray<string>; readonly keys?: StackKeysInput; + /** Keeps unnamed saved members and dependencies; supports rebinding services without an API endpoint. */ + readonly retainMembers?: boolean; /** * Starts every member with the composition. By default the database and members without an * endpoint start eagerly, and other members start on their first connection. @@ -216,7 +218,7 @@ export type CreationChange = | { readonly change: "unchanged" } | { readonly change: "changed"; readonly paths: ReadonlyArray<string> } | { - /** The instance cannot adopt the request: its endpoints, artifact or data version differ. */ + /** The instance cannot adopt the request: its endpoints or database line differ. */ readonly change: "incompatible"; readonly paths: ReadonlyArray<string>; /** @@ -281,8 +283,11 @@ const comparable = ( const config: Record<string, unknown> = Object.fromEntries( Object.entries(creation.config).filter(([key]) => compared.has(key) || !managed.has(key)), ); - if (creation.service === "database") config.version = postgresVersion(creation.config.version); - return { version: creation.version, endpoints: withSharedApiPort(creation, sharedPort), config }; + return { + pin: creation.pin, + endpoints: withSharedApiPort(creation, sharedPort), + config, + }; }; /** Compares a requested creation with a saved one, ignoring inputs the package supplies. */ @@ -291,6 +296,7 @@ const compareCreation = ( requested: ServiceCreationInput, sharedPort: number | undefined, memberKinds: ReadonlySet<ServiceCreation["service"]>, + initializedData: boolean, ): CreationChange => { const overridable = overridableInputs[requested.service] ?? []; // Without its supplying member, the composition keeps the project's own value of an input. @@ -303,13 +309,22 @@ const compareCreation = ( .map(([input]) => input), ]); const requestedComparable = comparable(requested, sharedPort, compared); - const paths = differences(comparable(saved, undefined, compared), requestedComparable, ""); + const paths = [ + ...new Set( + differences(comparable(saved, undefined, compared), requestedComparable, "").map((path) => + path.startsWith("pin.") ? "pin" : path, + ), + ), + ]; const isEndpointPath = (path: string) => path === "endpoints" || path.startsWith("endpoints."); + // The planner flags a line or major change on initialized data; Database.prepare enforces the + // exact build there, so any other pin change is planned as a restart. + const databaseLineChanged = + initializedData && + saved.service === "database" && + postgresLine(saved.pin.upstreamVersion) !== postgresLine(requested.pin.upstreamVersion); const incompatible = paths.filter( - (path) => - path === "version" || - isEndpointPath(path) || - (saved.service === "database" && path === "config.version"), + (path) => isEndpointPath(path) || (path === "pin" && databaseLineChanged), ); const savedNames = endpointNames(saved); const requestedNames = endpointNames({ @@ -337,6 +352,16 @@ export interface PlanOptions { readonly requestKind?: "partial" | "complete"; } +/** Planning options plus what the caller read from the instances' data. */ +export interface PlanInputs extends PlanOptions { + /** + * The database instances whose data is initialized. A database line or major change is + * incompatible only for these, since uninitialized data adopts any build. Omitted, every + * database counts as initialized. + */ + readonly initializedDatabases?: ReadonlySet<string>; +} + /** * The fixed shared API port the request agrees on. A `partial` request falls back to a saved * sibling's port for a member kind it leaves out; a `complete` request never does, so a request @@ -365,7 +390,7 @@ const sharedApiPortFor = ( export const planSupabaseComposition = ( saved: Pick<SavedStack, "instances" | "composition">, requested: ReadonlyArray<ServiceCreationInput>, - options: PlanOptions = {}, + options: PlanInputs = {}, ): ReadonlyArray<PlannedInstance> => { const members = new Set(saved.composition.members.map(({ id }) => id)); const memberKinds = new Set( @@ -381,7 +406,13 @@ export const planSupabaseComposition = ( id, service: creation.service, member: members.has(id), - ...compareCreation(creation, request, sharedPort, memberKinds), + ...compareCreation( + creation, + request, + sharedPort, + memberKinds, + options.initializedDatabases?.has(id) ?? true, + ), }, ]; }); @@ -452,8 +483,8 @@ const endpointRewrite = ( /** * Plans the endpoint port changes a stopped stack's start can apply instead of failing. * Returns `undefined` when a member is incompatible for a reason besides a pure endpoint port - * reassignment (an artifact or database version change, or an added or removed endpoint), so the - * caller leaves that case to the existing incompatible-change rejection. + * reassignment (an added or removed endpoint), so the caller leaves that case to the existing + * incompatible-change rejection. * * A requested kind with no saved member and no compatible stopped instance also has its sole * endpoint-only incompatible instance rewritten, so the start reuses it instead of creating a @@ -462,10 +493,11 @@ const endpointRewrite = ( export const planEndpointReplan = ( saved: Pick<SavedStack, "instances" | "composition">, requested: ReadonlyArray<ServiceCreationInput>, + initializedDatabases?: ReadonlySet<string>, ): EndpointReplan | undefined => { // The start re-plan always replaces the whole composition, so an excluded sibling's stale // saved port must not anchor the shared port this request would otherwise move to automatic. - const planOptions: PlanOptions = { requestKind: "complete" }; + const planOptions: PlanInputs = { requestKind: "complete", initializedDatabases }; const planned = planSupabaseComposition(saved, requested, planOptions); const sharedPort = sharedApiPortFor(saved, requested, planOptions); const savedMemberIds = new Set(saved.composition.members.map(({ id }) => id)); @@ -780,6 +812,14 @@ export const makeSupabaseComposition = Effect.fn("Supabase.compose")( return entry; }), ); + const requestedIds = new Set(configured.map(({ id }) => id)); + const retained = options.retainMembers === true; + const retainedMembers = retained + ? current.members.filter(({ id }) => !requestedIds.has(id)) + : []; + const retainedDependencies = retained + ? current.dependencies.filter(({ from, to }) => !dependencyMap.has(`${from}->${to}`)) + : []; const members = configured.map(({ id, creation }) => { const lazy = options.eager !== true && @@ -790,8 +830,8 @@ export const makeSupabaseComposition = Effect.fn("Supabase.compose")( : { id, activation: "eager" as const }; }); yield* operations.configure({ - members, - dependencies: [...dependencyMap.values()], + members: [...members, ...retainedMembers], + dependencies: [...dependencyMap.values(), ...retainedDependencies], }); return configured; }); diff --git a/packages/stack/src/composition/Supabase.unit.test.ts b/packages/stack/src/composition/Supabase.unit.test.ts index b64b6b59b8..22b8ee1807 100644 --- a/packages/stack/src/composition/Supabase.unit.test.ts +++ b/packages/stack/src/composition/Supabase.unit.test.ts @@ -10,6 +10,7 @@ import { planEndpointReplan, type SupabaseCompositionOperations, } from "./Supabase.ts"; +import { selection } from "../../tests/pins.ts"; /** A registered instance with no runtime behavior beyond an immediate healthy start and stop. */ const makeInstance = ( @@ -78,8 +79,8 @@ it.live( const inputs: ReadonlyArray<ServiceCreation> = [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("studio-idle-password"), jwtSecret: Redacted.make("studio-idle-jwt-secret-with-32-chars"), jwtExpiry: 3600, @@ -88,16 +89,28 @@ it.live( }, { service: "auth", + pin: selection.auth, config: { jwtSecret: "studio-idle-jwt-secret-with-32-chars", jwtExpiry: 3600 }, endpoints: { http: { port: "auto" } }, }, { service: "functions", + pin: selection.functions, config: { functionsRoot: "/project/supabase/functions" }, endpoints: { http: { port: "auto" } }, }, - { service: "studio", config: {}, endpoints: { http: { port: "auto" } } }, - { service: "pgmeta", config: {}, endpoints: { http: { port: "auto" } } }, + { + service: "studio", + pin: selection.studio, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + { + service: "pgmeta", + pin: selection.pgmeta, + config: {}, + endpoints: { http: { port: "auto" } }, + }, ]; const baseById = new Map(inputs.map((creation) => [idFor(creation.service), creation])); const captured = yield* Ref.make<Orchestrator.CompositionConfig | undefined>(undefined); @@ -204,11 +217,21 @@ it("does not let an excluded sibling's stale fixed port mask a shared endpoint's instances: [ { id: "rest-1", - creation: { service: "rest", config: {}, endpoints: { http: { port: fixedPort } } }, + creation: { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: fixedPort } }, + }, }, { id: "auth-1", - creation: { service: "auth", config: {}, endpoints: { http: { port: fixedPort } } }, + creation: { + service: "auth", + pin: selection.auth, + config: {}, + endpoints: { http: { port: fixedPort } }, + }, }, ], composition: { @@ -221,7 +244,7 @@ it("does not let an excluded sibling's stale fixed port mask a shared endpoint's }; // The config dropped [api] port and the start excludes auth, so only REST is requested. const requested: ReadonlyArray<ServiceCreation> = [ - { service: "rest", config: {}, endpoints: { http: { port: "auto" } } }, + { service: "rest", pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } } }, ]; const plan = planEndpointReplan(saved, requested); expect(plan?.changes).toEqual([ @@ -240,14 +263,19 @@ it("releases the shared listener's automatic claim when another sharing service instances: [ { id: "auth-1", - creation: { service: "auth", config: {}, endpoints: { http: { port: "auto" } } }, + creation: { + service: "auth", + pin: selection.auth, + config: {}, + endpoints: { http: { port: "auto" } }, + }, }, ], composition: { members: [{ id: "auth-1", activation: "eager" }], dependencies: [] }, }; // Auth is excluded from this start and REST takes the config's fixed [api] port. const requested: ReadonlyArray<ServiceCreation> = [ - { service: "rest", config: {}, endpoints: { http: { port: 8_911 } } }, + { service: "rest", pin: selection.rest, config: {}, endpoints: { http: { port: 8_911 } } }, ]; expect(planEndpointReplan(saved, requested)?.releasesSharedApi).toBe(true); expect(planEndpointReplan(saved, [])).toBeUndefined(); diff --git a/packages/stack/src/credentials.integration.test.ts b/packages/stack/src/credentials.integration.test.ts index 2f704f035a..2f07e42fa7 100644 --- a/packages/stack/src/credentials.integration.test.ts +++ b/packages/stack/src/credentials.integration.test.ts @@ -3,6 +3,7 @@ import { expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer, Redacted } from "effect"; import { create, open } from "./effect.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; const layer = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); @@ -21,8 +22,8 @@ it.live("persists effective credentials across service creation, reopen, and res yield* Effect.addFinalizer(() => stack.destroy.pipe(Effect.orDie)); const database = yield* stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", jwtSecret: Redacted.make("credentials-jwt-override"), rootKey: Redacted.make("b".repeat(64)), jwtExpiry: 3600, @@ -61,13 +62,14 @@ it.live("resolves composition credentials before creating services", () => const created = yield* stack.composition.supabase([ { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: "auto" } }, }, { service: "database", + pin: selection.database, config: { - version: "17", jwtSecret: Redacted.make("composition-jwt-override"), rootKey: Redacted.make("c".repeat(64)), jwtExpiry: 3600, diff --git a/packages/stack/src/data-conflicts.integration.test.ts b/packages/stack/src/data-conflicts.integration.test.ts new file mode 100644 index 0000000000..15fe9e9ad3 --- /dev/null +++ b/packages/stack/src/data-conflicts.integration.test.ts @@ -0,0 +1,216 @@ +import { NodeServices } from "@effect/platform-node"; +import { expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Path } from "effect"; +import { findDataConflicts } from "./effect.ts"; +import { seedSavedStack, type SeedSavedStackOptions } from "./testing.ts"; +import { selection } from "../tests/pins.ts"; + +const built = selection.database.upstreamVersion; +const newerBuild = "17.99.0.001"; + +const pinned = <Pin extends { readonly upstreamVersion: string }>( + pin: Pin, + upstreamVersion: string, +) => ({ ...pin, upstreamVersion }); + +const project = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-data-conflicts-" }); + const projectRoot = path.join(root, "project"); + yield* fs.makeDirectory(projectRoot); + const stateRoot = path.join(root, "state"); + return { + root, + projectRoot, + stateRoot, + seed: (options: Partial<SeedSavedStackOptions> = {}) => + seedSavedStack({ projectRoot, stateRoot, ...options }), + conflicts: (requested: Parameters<typeof findDataConflicts>[0]["requested"]) => + findDataConflicts({ stateRoot, projectRoot, requested }), + }; +}); + +const run = <A, E, R>(effect: Effect.Effect<A, E, R>) => + effect.pipe(Effect.scoped, Effect.provide(NodeServices.layer)); + +it.effect( + "reports initialized data built by another upstream, but not a newer revision of its own", + () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + const { stackId, databaseId } = yield* seed({ built }); + + const [conflict] = yield* conflicts({ database: pinned(selection.database, newerBuild) }); + + expect(conflict).toMatchObject({ + kind: "database-build", + stackId, + service: "database", + instanceId: databaseId, + recorded: built, + requested: newerBuild, + }); + expect(conflict?.message).toContain("was requested; keep this data by pinning"); + expect( + yield* conflicts({ + database: { ...selection.database, revision: selection.database.revision + 1 }, + }), + ).toEqual([]); + }), + ), +); + +it.effect("reports initialized data that has no recorded identity", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + const { stackId } = yield* seed({ built, withoutIdentity: true }); + + const [conflict] = yield* conflicts({ database: selection.database }); + + expect(conflict).toMatchObject({ + kind: "database-build", + recorded: undefined, + requested: built, + }); + expect(conflict?.message).toBe( + `PostgreSQL data has no recorded identity, but stock PostgreSQL build ${built} was requested; run \`supabase stack destroy --stack-id ${stackId}\` to recreate the stack — this permanently deletes its local database data`, + ); + }), + ), +); + +it.effect("accepts any database build while the data was never initialized", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + yield* seed(); + + expect(yield* conflicts({ database: pinned(selection.database, newerBuild) })).toEqual([]); + }), + ), +); + +it.effect("reports the identity a Docker first start recorded before it was interrupted", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + const { stackId, databaseId } = yield* seed({ built, interrupted: true, runtime: "docker" }); + + const [conflict] = yield* conflicts({ database: pinned(selection.database, newerBuild) }); + + expect(conflict).toMatchObject({ + kind: "database-build", + stackId, + instanceId: databaseId, + recorded: built, + requested: newerBuild, + }); + expect(yield* conflicts({ database: selection.database })).toEqual([]); + }), + ), +); + +it.effect("stays quiet about a Docker data directory that has no recorded identity", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + yield* seed({ built, interrupted: true, runtime: "docker", withoutIdentity: true }); + + expect(yield* conflicts({ database: pinned(selection.database, newerBuild) })).toEqual([]); + }), + ), +); + +it.effect( + "reports a composed schema-owning service older than the newest one that migrated the database", + () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + const { stackId, databaseId } = yield* seed({ + composed: ["auth", "storage"], + migrated: { auth: "v2.197.0" }, + }); + + const found = yield* conflicts({ + auth: pinned(selection.auth, "v2.196.1"), + storage: pinned(selection.storage, "v0.0.1"), + rest: pinned(selection.rest, "v0.0.1"), + }); + + expect(found).toHaveLength(1); + expect(found[0]).toMatchObject({ + kind: "schema-downgrade", + stackId, + service: "auth", + instanceId: databaseId, + recorded: "v2.197.0", + requested: "v2.196.1", + }); + expect(found[0]?.message).toContain( + 'keep this data by pinning `[local.services.auth] version = "v2.197.0"`', + ); + expect(yield* conflicts({ auth: pinned(selection.auth, "v2.197.0") })).toEqual([]); + expect(yield* conflicts({ auth: pinned(selection.auth, "v2.198.0") })).toEqual([]); + }), + ), +); + +it.effect("keeps reporting a recorded downgrade before the database data initialized", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + yield* seed({ composed: ["auth"], migrated: { auth: "v2.197.0" } }); + + expect(yield* conflicts({ auth: pinned(selection.auth, "v2.196.1") })).toHaveLength(1); + }), + ), +); + +it.effect("stays quiet about a schema writer the saved composition no longer runs", () => + run( + Effect.gen(function* () { + const { seed, conflicts } = yield* project; + yield* seed({ composed: ["auth"], migrated: { auth: "v2.197.0", storage: "v1.50.0" } }); + + expect( + yield* conflicts({ + auth: pinned(selection.auth, "v2.197.0"), + storage: pinned(selection.storage, "v0.0.1"), + }), + ).toEqual([]); + }), + ), +); + +it.effect("reports only the stacks of the project it is asked about", () => + run( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { root, stateRoot, seed, conflicts } = yield* project; + const other = `${root}/other`; + yield* fs.makeDirectory(other); + yield* seedSavedStack({ projectRoot: other, stateRoot, built }); + const { stackId } = yield* seed({ built }); + + const found = yield* conflicts({ database: pinned(selection.database, newerBuild) }); + + expect(found.map((conflict) => conflict.stackId)).toEqual([stackId]); + }), + ), +); + +it.effect("reports nothing, and creates nothing, when there is no saved state", () => + run( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { stateRoot, conflicts } = yield* project; + + expect(yield* conflicts({ database: pinned(selection.database, newerBuild) })).toEqual([]); + expect(yield* fs.exists(stateRoot)).toBe(false); + }), + ), +); diff --git a/packages/stack/src/defaults.integration.test.ts b/packages/stack/src/defaults.integration.test.ts index 1f0b3ab928..f2226674d6 100644 --- a/packages/stack/src/defaults.integration.test.ts +++ b/packages/stack/src/defaults.integration.test.ts @@ -5,18 +5,17 @@ import { DEFAULT_LOCAL_JWT_SECRET, DEFAULT_POSTGRES_ROOT_KEY } from "./Defaults. import { create as createEffect, StackError } from "./effect.ts"; import { create as createPromise } from "./index.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; const artifactCacheRoot = testArtifactCacheRoot; const effectLayer = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); const effectDatabaseConfig = { - version: "17", databasePassword: Redacted.make("defaults-test-password"), jwtExpiry: 3600, } as const; const promiseDatabaseConfig = { - version: "17", databasePassword: "defaults-test-password", jwtExpiry: 3600, } as const; @@ -43,6 +42,7 @@ describe("database configuration defaults", { timeout: 180_000 }, () => { Effect.gen(function* () { const database = yield* current.services.create({ service: "database", + pin: selection.database, config: effectDatabaseConfig, }); const initial = yield* database.status; @@ -73,7 +73,7 @@ describe("database configuration defaults", { timeout: 180_000 }, () => { expect(conflict.message).toContain("conflicts with the saved stack value"); yield* database.destroy; const composed = yield* current.composition.supabase([ - { service: "database", config: effectDatabaseConfig }, + { service: "database", pin: selection.database, config: effectDatabaseConfig }, ]); const composedStatus = yield* composed[0]!.status; if (composedStatus.config.service !== "database") @@ -119,6 +119,7 @@ describe("database configuration defaults", { timeout: 180_000 }, () => { const database = yield* promise(() => current.services.create({ service: "database", + pin: selection.database, config: promiseDatabaseConfig, }), ); @@ -157,7 +158,7 @@ describe("database configuration defaults", { timeout: 180_000 }, () => { yield* promise(() => database.destroy()); const composed = yield* promise(() => current.composition.supabase([ - { service: "database", config: promiseDatabaseConfig }, + { service: "database", pin: selection.database, config: promiseDatabaseConfig }, ]), ); const composedStatus = yield* promise(() => composed[0]!.status()); diff --git a/packages/stack/src/effect.integration.test.ts b/packages/stack/src/effect.integration.test.ts index b11ceb00e6..7847586b06 100644 --- a/packages/stack/src/effect.integration.test.ts +++ b/packages/stack/src/effect.integration.test.ts @@ -39,6 +39,7 @@ import { engineStub } from "../tests/docker-fixture.ts"; import { destroyTestStack } from "../tests/stack-cleanup.ts"; import { deriveStackId, resolveStackIdentity } from "./identity/Identity.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { database15, selection } from "../tests/pins.ts"; const layer = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); // Below the stack's native and auto port ranges and every OS ephemeral range, so neither another @@ -73,8 +74,8 @@ it.live("registers and discovers saved definitions without inventing live observ const database = stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("test"), jwtSecret: Redacted.make("test"), jwtExpiry: 3600, @@ -82,6 +83,7 @@ it.live("registers and discovers saved definitions without inventing live observ }); const rest = stack.services.create({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgres://external" }, }); expectTypeOf<Effect.Success<typeof database>>().toEqualTypeOf<DatabaseInstance>(); @@ -137,7 +139,11 @@ it.live("starts the owner on opt-in reopen without starting saved services", () const stack = yield* create(options); yield* Effect.ensuring( Effect.gen(function* () { - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + }); yield* stack.stop; const offline = yield* open({ ...options, id: stack.id }); @@ -171,8 +177,8 @@ it.live( yield* Effect.addFinalizer(() => stack.stop.pipe(Effect.ignore)); const instance = yield* stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("destroy-test-password"), jwtSecret: Redacted.make("destroy-test-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -237,8 +243,8 @@ it.live( yield* Effect.addFinalizer(() => destroyTestStack(stack)); const database = yield* stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("idle-tcp-password"), jwtSecret: Redacted.make("idle-tcp-jwt-secret-at-least-thirty-two-characters"), jwtExpiry: 3600, @@ -288,8 +294,8 @@ const resetDataStory = (runtime: "native" | ContainerEngine) => Effect.gen(function* () { const target = yield* current.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("reset-shared-password"), jwtSecret: Redacted.make("reset-shared-jwt-secret-long-enough-32chars"), jwtExpiry: 3600, @@ -298,8 +304,8 @@ const resetDataStory = (runtime: "native" | ContainerEngine) => }); const sibling = yield* current.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("reset-shared-password"), jwtSecret: Redacted.make("reset-shared-jwt-secret-long-enough-32chars"), jwtExpiry: 3600, @@ -338,7 +344,11 @@ const resetDataStory = (runtime: "native" | ContainerEngine) => if (runtime === "native") { const existingServices = yield* current.services.list; const initialized = yield* current.commands.run( - initialization.realtime({ databaseUrl: targetUrl }), + initialization.realtime({ + pin: selection.realtime, + databaseId: target.id, + databaseUrl: targetUrl, + }), ); expect(initialized.exitCode).toBe(0); expect((yield* current.services.list).map((service) => service.id)).toEqual( @@ -354,7 +364,7 @@ const resetDataStory = (runtime: "native" | ContainerEngine) => const stdout: Array<string> = []; const stderr: Array<string> = []; return client.commands - .run(postgres.psql({ major: 17 }), { + .run(postgres.psql({ pin: selection.database }), { args: ["--dbname", url, "-Atc", sql], stdout: (bytes) => Effect.sync(() => stdout.push(new TextDecoder().decode(bytes))), stderr: (bytes) => Effect.sync(() => stderr.push(new TextDecoder().decode(bytes))), @@ -469,7 +479,11 @@ it.live("stops an instance without starting an owner when none is live", () => const stack = yield* create(options); yield* Effect.ensuring( Effect.gen(function* () { - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + }); yield* stack.stop; expect((yield* discover(options))[0]?.host).toBeUndefined(); @@ -610,7 +624,9 @@ it.live("reclaims a registered stack's leftovers when stopping without an owner" identity, lifetime: "session", runtime: "native", - instances: [{ id: "leftover", creation: { service: "mail", config: {} } }], + instances: [ + { id: "leftover", creation: { service: "mail", pin: selection.mail, config: {} } }, + ], composition: { members: [], dependencies: [] }, }); const leftover = `${options.stateRoot}/${id}/data/leftover`; @@ -853,7 +869,7 @@ it.live("reports a stopped owner as unavailable to attach-only calls", () => runtime: "native", } satisfies Parameters<typeof create>[0]; const stack = yield* create(options); - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ service: "mail", pin: selection.mail, config: {} }); yield* stack.stop; const unavailable = yield* Effect.flip(mail.status); @@ -900,7 +916,7 @@ it.live("sends no call to a process that took a dead owner's port", () => runtime: "native", } satisfies Parameters<typeof create>[0]; const stack = yield* create(options); - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ service: "mail", pin: selection.mail, config: {} }); yield* mail.status; const state = yield* StackNamespace.Service.pipe( Effect.provide(StackNamespace.layer({ root: options.stateRoot })), @@ -931,7 +947,7 @@ it.live("sends no call to a process that took the port of a replaced owner", () runtime: "native", } satisfies Parameters<typeof create>[0]; const stack = yield* create(options); - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ service: "mail", pin: selection.mail, config: {} }); yield* mail.status; const state = yield* StackNamespace.Service.pipe( Effect.provide(StackNamespace.layer({ root: options.stateRoot })), @@ -976,7 +992,7 @@ it.live( Effect.gen(function* () { yield* mail.status; yield* mail.followStatus.pipe(Stream.take(1), Stream.runDrain); - const version = yield* stack.commands.run(postgres.psql({ major: 17 }), { + const version = yield* stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--version"], stdout: () => Effect.void, stderr: () => Effect.void, @@ -985,7 +1001,11 @@ it.live( }); yield* Effect.ensuring( Effect.gen(function* () { - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + }); yield* useHandle(mail); const settled = handleFinalizers(); @@ -1055,7 +1075,9 @@ it.live("replaces a dead session stack that holds the requested identity", () => identity, lifetime: "session", runtime: "native", - instances: [{ id: "abandoned", creation: { service: "mail", config: {} } }], + instances: [ + { id: "abandoned", creation: { service: "mail", pin: selection.mail, config: {} } }, + ], composition: { members: [], dependencies: [] }, }); @@ -1084,14 +1106,18 @@ it.live( } satisfies Parameters<typeof create>[0]; const stack = yield* create(options); const version = (stack: Effect.Success<ReturnType<typeof open>>) => - stack.commands.run(postgres.psql({ major: 17 }), { + stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--version"], stdout: () => Effect.void, stderr: () => Effect.void, }); yield* Effect.ensuring( Effect.gen(function* () { - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + }); expect((yield* version(stack)).exitCode).toBe(0); const first = (yield* discover(options))[0]?.host; @@ -1116,6 +1142,38 @@ it.live( { timeout: 120_000 }, ); +it.live( + "keeps the saved pin on a configuration-only restart that starts a stopped owner", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-restart-pin-" }); + const options = { + projectRoot: root, + stateRoot: `${root}/state`, + cacheRoot: testArtifactCacheRoot, + runtime: "native", + } satisfies Parameters<typeof create>[0]; + const stack = yield* create(options); + yield* Effect.ensuring( + Effect.gen(function* () { + const savedPin = { ...selection.mail, upstreamImage: "axllent/mailpit:saved-pin" }; + const mail = yield* stack.services.create({ service: "mail", pin: savedPin, config: {} }); + yield* stack.stop; + expect(Option.getOrUndefined(yield* find(options))?.host).toBeUndefined(); + + yield* mail.restart({ config: {} }); + + const restarted = yield* mail.status; + expect(restarted.config.pin).toEqual(savedPin); + expect(restarted.lifecycle).toBe("running"); + }), + destroyTestStack(stack), + ); + }).pipe(Effect.scoped, Effect.provide(layer)), + { timeout: 120_000 }, +); + it.live("finds one saved stack by identity or id and fails on unreadable state", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -1162,8 +1220,8 @@ it.live("plans requested creations against the saved composition and honours eag Effect.gen(function* () { const database = { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("plan-database-password"), jwtSecret: Redacted.make("plan-database-jwt-secret-at-least-32-chars"), jwtExpiry: 3600, @@ -1172,11 +1230,13 @@ it.live("plans requested creations against the saved composition and honours eag } as const; const rest = { service: "rest", + pin: selection.rest, config: { maxRows: 100 }, endpoints: { http: { port: FIXED_API_PORT } }, } as const; const auth = { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: "auto" } }, } as const; @@ -1195,7 +1255,7 @@ it.live("plans requested creations against the saved composition and honours eag database, { ...rest, config: { maxRows: 500 } }, auth, - { service: "mail", config: {} }, + { service: "mail", pin: selection.mail, config: {} }, ]), ).toEqual([ { id: databaseId, service: "database", member: true, change: "unchanged" }, @@ -1208,6 +1268,29 @@ it.live("plans requested creations against the saved composition and honours eag }, { id: authId, service: "auth", member: true, change: "unchanged" }, ]); + // Any pin field counts; only a database release-line change is incompatible. + const sameLineDatabasePin = { + ...selection.database, + revision: selection.database.revision + 1, + }; + const upstreamImageOnlyPin = { ...selection.auth, upstreamImage: "supabase/gotrue:other" }; + expect( + yield* stack.composition.plan([ + { ...database, pin: sameLineDatabasePin }, + { ...rest, pin: { ...selection.rest, image: `${selection.rest.image}-other` } }, + { ...auth, pin: upstreamImageOnlyPin }, + ]), + ).toEqual([ + { + id: databaseId, + service: "database", + member: true, + change: "changed", + paths: ["pin"], + }, + { id: restId, service: "rest", member: true, change: "changed", paths: ["pin"] }, + { id: authId, service: "auth", member: true, change: "changed", paths: ["pin"] }, + ]); const client = yield* Effect.promise(() => PromiseApi.open({ id: stack.id, stateRoot: `${root}/state`, cacheRoot: `${root}/cache` }), ); @@ -1219,9 +1302,27 @@ it.live("plans requested creations against the saved composition and honours eag expect(promisePlan).toEqual([ { id: restId, service: "rest", member: true, change: "unchanged" }, ]); + if (databaseId === undefined) return yield* Effect.die("Composition is missing a database"); + expect( + yield* stack.composition.plan([ + { ...database, pin: database15 }, + { ...rest, endpoints: { http: { port: FIXED_API_PORT + 1 } } }, + ]), + ).toEqual([ + { id: databaseId, service: "database", member: true, change: "changed", paths: ["pin"] }, + { + id: restId, + service: "rest", + member: true, + change: "incompatible", + paths: ["endpoints.http.port"], + endpointsOnly: true, + }, + ]); + yield* markDatabaseInitialized(`${root}/state`, stack.id, databaseId); expect( yield* stack.composition.plan([ - { ...database, config: { ...database.config, version: "15" } }, + { ...database, pin: database15 }, { ...rest, endpoints: { http: { port: FIXED_API_PORT + 1 } } }, ]), ).toEqual([ @@ -1230,7 +1331,7 @@ it.live("plans requested creations against the saved composition and honours eag service: "database", member: true, change: "incompatible", - paths: ["config.version"], + paths: ["pin"], endpointsOnly: false, }, { @@ -1271,21 +1372,27 @@ it.live("plans a Studio public API URL the project sets but not the one the stac Effect.gen(function* () { const studio = { service: "studio", + pin: selection.studio, config: {}, endpoints: { http: { port: "auto" } }, } as const; const members = yield* stack.composition.supabase([ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("plan-database-password"), jwtSecret: Redacted.make("plan-database-jwt-secret-at-least-32-chars"), jwtExpiry: 3600, }, endpoints: { sql: { port: "auto" } }, }, - { service: "rest", config: {}, endpoints: { http: { port: FIXED_API_PORT } } }, + { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: FIXED_API_PORT } }, + }, studio, ]); const studioId = members.find(({ service }) => service === "studio")?.id; @@ -1326,11 +1433,13 @@ it.live("plans a project's own URL for an input whose supplying member is absent const moved = "postgresql://postgres@moved.example.test:5432/postgres"; const rest = { service: "rest", + pin: selection.rest, config: { databaseUrl: external }, endpoints: { http: { port: "auto" } }, } as const; const functions = { service: "functions", + pin: selection.functions, config: { functionsRoot: `${root}/functions`, databaseUrl: external }, endpoints: { http: { port: "auto" } }, } as const; @@ -1372,8 +1481,8 @@ it.live("plans a project's own URL for an input whose supplying member is absent const replanDatabase = (tag: string) => ({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make(`${tag}-password`), jwtSecret: Redacted.make(`${tag}-jwt-secret-at-least-32-characters-long`), jwtExpiry: 3600, @@ -1381,6 +1490,18 @@ const replanDatabase = (tag: string) => endpoints: { sql: { port: "auto" } }, }) as const; +/** Writes the readiness marker a completed database bootstrap leaves, which marks its data initialized. */ +const markDatabaseInitialized = (stateRoot: string, stackId: string, databaseId: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const instanceRoot = `${stateRoot}/${stackId}/data/${databaseId}`; + yield* fs.makeDirectory(instanceRoot, { recursive: true }); + yield* fs.writeFileString( + `${instanceRoot}/.supabase-database-ready.json`, + '{"runtime":"native","profile":"supabase"}', + ); + }); + const restPort = (status: { readonly endpoints: ReadonlyArray<{ name: string; port: number }> }) => status.endpoints.find(({ name }) => name === "http")?.port; @@ -1422,6 +1543,7 @@ it.live( const database = replanDatabase("replan-auto"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1483,6 +1605,7 @@ it.live( const database = replanDatabase("replan-fixed"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, } as const; @@ -1557,6 +1680,7 @@ it.live( const database = replanDatabase("replan-conflict"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: "auto" } }, } as const; @@ -1641,6 +1765,7 @@ it.live( const database = replanDatabase("replan-collide"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1675,7 +1800,7 @@ it.live( ); it.live( - "leaves a changed Postgres major version incompatible without re-planning any endpoint", + "leaves a changed Postgres major version incompatible on initialized data without re-planning any endpoint", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -1688,6 +1813,7 @@ it.live( const database = replanDatabase("replan-version"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1696,8 +1822,9 @@ it.live( const restId = members.find(({ service }) => service === "rest")?.id; if (databaseId === undefined || restId === undefined) return yield* Effect.die("Composition is missing a member"); + yield* markDatabaseInitialized(stateRoot, stack.id, databaseId); - const olderVersion = { ...database, config: { ...database.config, version: "15" } }; + const olderVersion = { ...database, pin: database15 }; const requestedAuto = { ...rest, endpoints: { http: { port: "auto" } } } as const; const reopened = yield* restartWithReplan(stack, stateRoot, cacheRoot, [ olderVersion, @@ -1710,7 +1837,7 @@ it.live( service: "database", member: true, change: "incompatible", - paths: ["config.version"], + paths: ["pin"], endpointsOnly: false, }, { @@ -1728,6 +1855,56 @@ it.live( }).pipe(Effect.scoped, Effect.provide(layer)), ); +it.live( + "plans a changed Postgres major version as a restart while the database data is uninitialized", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-api-uninit-version-" }); + const stateRoot = `${root}/state`; + const cacheRoot = `${root}/cache`; + const stack = yield* create({ projectRoot: root, stateRoot, cacheRoot, runtime: "native" }); + yield* Effect.ensuring( + Effect.gen(function* () { + const database = replanDatabase("replan-uninit-version"); + const rest = { + service: "rest", + pin: selection.rest, + config: {}, + endpoints: { http: { port: FIXED_API_PORT } }, + } as const; + const members = yield* stack.composition.supabase([database, rest], { eager: true }); + const databaseId = members.find(({ service }) => service === "database")?.id; + const restId = members.find(({ service }) => service === "rest")?.id; + if (databaseId === undefined || restId === undefined) + return yield* Effect.die("Composition is missing a member"); + + const olderVersion = { ...database, pin: database15 }; + const requestedAuto = { ...rest, endpoints: { http: { port: "auto" } } } as const; + const reopened = yield* restartWithReplan(stack, stateRoot, cacheRoot, [ + olderVersion, + requestedAuto, + ]); + const changes = yield* reopened.startupEndpointChanges; + expect( + changes.map(({ service, endpoint, from }) => ({ service, endpoint, from })), + ).toEqual([{ service: "rest", endpoint: "http", from: FIXED_API_PORT }]); + expect(yield* reopened.composition.plan([olderVersion, requestedAuto])).toEqual([ + { + id: databaseId, + service: "database", + member: true, + change: "changed", + paths: ["pin"], + }, + { id: restId, service: "rest", member: true, change: "unchanged" }, + ]); + }), + destroyTestStack(stack), + ); + }).pipe(Effect.scoped, Effect.provide(layer)), +); + it.live( "does not let an excluded sibling's stale fixed port mask a shared endpoint moving to automatic", () => @@ -1742,11 +1919,13 @@ it.live( const database = replanDatabase("replan-excluded"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; const auth = { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1799,11 +1978,13 @@ it.live( const database = replanDatabase("replan-readd"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; const auth = { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1885,11 +2066,13 @@ it.live( const database = replanDatabase("replan-swapped"); const auth = { service: "auth", + pin: selection.auth, config: {}, endpoints: { http: { port: "auto" } }, } as const; const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1929,6 +2112,7 @@ it.live("attaches to a live owner without re-planning or leaving a startup paylo const database = replanDatabase("replan-attach"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; @@ -1968,6 +2152,7 @@ it.live( const database = replanDatabase("replan-concurrent"); const rest = { service: "rest", + pin: selection.rest, config: {}, endpoints: { http: { port: FIXED_API_PORT } }, } as const; diff --git a/packages/stack/src/effect.ts b/packages/stack/src/effect.ts index 39915fdfaf..a8159dcd47 100644 --- a/packages/stack/src/effect.ts +++ b/packages/stack/src/effect.ts @@ -42,6 +42,9 @@ import { } from "./composition/Supabase.ts"; import { deriveStackId, resolveStackIdentity } from "./identity/Identity.ts"; import { failureKind, isStackFailureKind, type StackFailureKind } from "./FailureKind.ts"; +import type { ArtifactSelection } from "./Artifacts.ts"; +import { collectDataConflicts } from "./internal/data-conflicts.ts"; +import { initializedDatabaseIds } from "./internal/database-reuse.ts"; import { failureMessage } from "./internal/failure-message.ts"; import * as StackNamespace from "./StackNamespace.ts"; import { engineUnreachable, resolveEngineTarget } from "./runtime/Container.ts"; @@ -59,6 +62,7 @@ import { gatewayLog } from "./host/GatewayLog.ts"; import type { LogPosition, LogRecord, StackLogRecord } from "./host/LogRecord.ts"; import { ServiceCreationInput as ServiceCreationInputSchema, + ServiceRestartInput as ServiceRestartInputSchema, type ServiceCreation, type ServiceCreationInput as CatalogServiceCreationInput, } from "./services/Catalog.ts"; @@ -76,7 +80,24 @@ export { resolveNativePostgresUser } from "./runtime/postgres-user.ts"; export { apiRoute, sharesApiEndpoint } from "./host/Endpoints.ts"; export { gatewayLog }; export { StackError } from "./Rpc.ts"; +/** Atomically writes a file so readers only ever see the prior or the complete new content. */ +export { NamespaceError, publish } from "./StackNamespace.ts"; +/** Holds an exclusive cross-process lock on a file for the enclosing scope, waiting up to a budget. */ +export { holdLock } from "./StackNamespace.ts"; +/** Whether a `holdLock` failure means another holder kept the lock for the whole budget. */ +export { isLockBusy } from "./StackNamespace.ts"; +export { + Lock, + LOCK_SCHEMA_URL, + LockInvalidError, + LockIoError, + LockUnsupportedVersionError, + lockSelection, + readLock, + writeLock, +} from "./Lock.ts"; export type { StackFailureKind } from "./FailureKind.ts"; +export type { DataConflict } from "./internal/data-conflicts.ts"; export type { ServiceCreation } from "./services/Catalog.ts"; /** A service creation as `services.create` accepts it, before stack credentials fill its inputs. */ export type ServiceCreationInput = CatalogServiceCreationInput; @@ -167,9 +188,10 @@ const failure = (operation: string, cause: unknown, kind?: StackFailureKind): St }; type Kind = ServiceCreation["service"]; -type ServiceCreationRestartInput<K extends Kind> = - | Pick<Extract<ServiceCreationInput, { service: K }>, "config"> - | Extract<ServiceCreationInput, { service: K }>; +type ServiceCreationRestartInput<K extends Kind> = Omit< + Extract<ServiceCreationInput, { service: K }>, + "service" | "pin" +>; /** An individually identified service controlled through the owner. */ export interface ServiceInstance<K extends Kind = Kind> { readonly id: string; @@ -282,7 +304,8 @@ export interface Stack { * inputs the composition supplies, without changing state or contacting the owner. A * `complete` request (the caller's whole desired composition) never falls back to a saved * sibling's port for a shared endpoint kind the request leaves out; a `partial` request - * (comparing only the kinds it names) does. + * (comparing only the kinds it names) does. A database line or major change is `incompatible` + * only while its data is initialized. */ readonly plan: ( services: ReadonlyArray<ServiceCreationInput>, @@ -369,6 +392,7 @@ const makeHandle = Effect.fn("Stack.makeHandle")(function* ( ); const crypto = yield* Crypto.Crypto; const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; const handleScope = yield* Scope.Scope; const session = saved.lifetime === "session"; const launchOptions = { ...locations, stackId: saved.id, lifeline: session }; @@ -609,9 +633,7 @@ const makeHandle = Effect.fn("Stack.makeHandle")(function* ( restart: (input) => input === undefined ? call("restart", (rpc) => rpc.restartService({ id })) - : Schema.decodeUnknownEffect(ServiceCreationInputSchema)( - "service" in input ? input : { service, config: input.config }, - ).pipe( + : Schema.decodeUnknownEffect(ServiceRestartInputSchema)({ service, ...input }).pipe( Effect.mapError((cause) => failure("restart", cause)), Effect.flatMap((creation) => call("restart", (rpc) => rpc.restartService({ id, config: creation })), @@ -846,19 +868,27 @@ const makeHandle = Effect.fn("Stack.makeHandle")(function* ( ...(options?.reuseIds === undefined ? {} : { reuseIds: options.reuseIds }), ...(options?.keys === undefined ? {} : { keys: options.keys }), ...(options?.eager === undefined ? {} : { eager: options.eager }), + ...(options?.retainMembers === undefined + ? {} + : { retainMembers: options.retainMembers }), }), ).pipe(Effect.map((definitions) => definitions.map(instance))), - plan: (services: ReadonlyArray<ServiceCreationInput>, options?: PlanOptions) => - Effect.forEach(services, (service) => + plan: Effect.fn("Stack.plan")(function* ( + services: ReadonlyArray<ServiceCreationInput>, + options?: PlanOptions, + ) { + const requested = yield* Effect.forEach(services, (service) => Schema.decodeEffect(ServiceCreationInputSchema)(service), - ).pipe( - Effect.mapError((cause) => failure("plan", cause)), - Effect.flatMap((requested) => - savedDefinition.pipe( - Effect.map((current) => planSupabaseComposition(current, requested, options)), - ), - ), - ), + ).pipe(Effect.mapError((cause) => failure("plan", cause))); + const current = yield* savedDefinition; + const initializedDatabases = yield* initializedDatabaseIds( + fs, + path, + path.join(locations.stateRoot, saved.id, "data"), + current.instances, + ).pipe(Effect.mapError((cause) => failure("plan", cause))); + return planSupabaseComposition(current, requested, { ...options, initializedDatabases }); + }), configure: (config: Orchestrator.CompositionConfig) => call("configureComposition", (rpc) => rpc.configureComposition(config)), describe: savedDefinition.pipe(Effect.map((current) => current.composition)), @@ -993,6 +1023,37 @@ export const discover = Effect.fn("Stack.discover")( Effect.mapError((cause) => failure("discover", cause)), ); +/** + * Lists the saved data of the stacks of `projectRoot` that a start at the `requested` pins would + * refuse. It reads their files without taking a lease or probing an owner, and creates nothing. + */ +export const findDataConflicts = Effect.fn("Stack.findDataConflicts")( + function* ( + options: Pick<StackLocations, "stateRoot"> & { + readonly projectRoot: string; + readonly requested: Partial<ArtifactSelection>; + }, + ) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + if (!(yield* fs.exists(options.stateRoot))) return []; + const projectRoot = yield* fs.realPath(options.projectRoot); + const stacks = (yield* (yield* stateFor(options.stateRoot)).list).filter( + ({ identity }) => identity.projectRoot === projectRoot, + ); + const conflicts = yield* Effect.forEach(stacks, (saved) => + collectDataConflicts(fs, path, { + saved, + dataRoot: path.join(options.stateRoot, saved.id, "data"), + requested: options.requested, + }), + ); + yield* Effect.annotateCurrentSpan({ "data_conflicts.stack_count": stacks.length }); + return conflicts.flat(); + }, + Effect.mapError((cause) => failure("findDataConflicts", cause)), +); + /** Selects a saved stack by id, or by the identity a project root and stack name derive. */ export type FindOptions = Pick<StackLocations, "stateRoot"> & ({ readonly id: string } | { readonly projectRoot: string; readonly name?: string }); diff --git a/packages/stack/src/host/CommandAttachments.integration.test.ts b/packages/stack/src/host/CommandAttachments.integration.test.ts index 7dd4a49536..b2cafc6011 100644 --- a/packages/stack/src/host/CommandAttachments.integration.test.ts +++ b/packages/stack/src/host/CommandAttachments.integration.test.ts @@ -3,6 +3,7 @@ import { Deferred, Effect, Fiber, Ref, Stream } from "effect"; import { StackError } from "../Rpc.ts"; import { postgres } from "../Commands.ts"; import { makeCommandAttachments } from "./CommandAttachments.ts"; +import { selection } from "../../tests/pins.ts"; it.live("stopAll waits for an admitted command to register and interrupt its runner", () => Effect.scoped( @@ -32,7 +33,7 @@ it.live("stopAll waits for an admitted command to register and interrupt its run attachmentId: "admitted", command: { type: "postgres", - command: postgres.psql({ major: 17 }), + command: postgres.psql({ pin: selection.database }), args: [], env: {}, stdin: false, diff --git a/packages/stack/src/host/CommandRunner.initialization.integration.test.ts b/packages/stack/src/host/CommandRunner.initialization.integration.test.ts index 840ca45fcd..78ee7ea3c4 100644 --- a/packages/stack/src/host/CommandRunner.initialization.integration.test.ts +++ b/packages/stack/src/host/CommandRunner.initialization.integration.test.ts @@ -2,7 +2,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { expect, it } from "@effect/vitest"; import { Context, Deferred, Effect, Fiber, FileSystem, Layer, Path, Ref } from "effect"; import { TestClock } from "effect/testing"; -import { catalogPins, resolveArtifact } from "../Artifacts.ts"; +import { resolveArtifact, type ArtifactPin } from "../Artifacts.ts"; import { initialization } from "../Commands.ts"; import { makeArtifactStore, @@ -12,6 +12,7 @@ import { import { PreparationError } from "../preparation/Errors.ts"; import type { StackCredentials } from "../StackNamespace.ts"; import * as CommandRunner from "./CommandRunner.ts"; +import { selection } from "../../tests/pins.ts"; const target = process.platform === "darwin" && process.arch === "arm64" @@ -22,18 +23,14 @@ const target = ? "linux-arm64" : undefined; -// The real catalog's default auth pin, not hardcoded — `resolveArtifact({ service: "auth", -// version })` below would otherwise fail once a catalog bump moves past a literal. -const authVersion = catalogPins().find((entry) => entry.sourceService === "auth" && entry.isDefault) - ?.pin.upstreamVersion; -if (authVersion === undefined) { - throw new Error("no default auth catalog pin found"); -} - -const prepareAuthArtifact = Effect.fn(function* (cacheRoot: string, script: string) { +const prepareAuthArtifact = Effect.fn(function* ( + cacheRoot: string, + script: string, + pin: ArtifactPin = selection.auth, +) { if (target === undefined) return yield* Effect.fail("Unsupported test platform"); const fs = yield* FileSystem.FileSystem; - const { releaseVersion, natives } = yield* resolveArtifact({ service: "auth" }); + const { releaseVersion, natives } = yield* resolveArtifact({ service: "auth", pin }); const request: ArtifactRequest = { key: `slim-services/auth/${releaseVersion}/${target}`, requiredRuntimePaths: ["bin/auth"], @@ -64,7 +61,12 @@ const prepareAuthArtifact = Effect.fn(function* (cacheRoot: string, script: stri yield* store.prepare(request); }); +const databaseId = "database"; + const makeRunner = Effect.fn(function* (root: string, cacheRoot: string) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* fs.makeDirectory(path.join(root, databaseId), { recursive: true }); const layer = CommandRunner.layer({ stackId: "initialization-command-test", root, @@ -108,7 +110,8 @@ it.live.skipIf(target === undefined || process.platform === "win32")( const error = yield* runner .run({ command: initialization.auth({ - version: authVersion, + pin: selection.auth, + databaseId, databaseUrl: "postgresql://invalid:invalid@127.0.0.1:1/postgres", }), credentials, @@ -130,6 +133,105 @@ it.live.skipIf(target === undefined || process.platform === "win32")( ), ); +it.live.skipIf(target === undefined || process.platform === "win32")( + "runs the selected pin of an initialization command, including one outside the catalog", + () => + Effect.scoped( + Effect.gen(function* () { + if (target === undefined) return; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "initialization-command-pin-" }); + const cacheRoot = path.join(root, "cache"); + const pin: ArtifactPin = { + ...selection.auth, + upstreamVersion: "v0.0.1-selected", + revision: 7, + natives: { + ...selection.auth.natives, + [target]: { archive: "a".repeat(64), manifest: "b".repeat(64) }, + }, + }; + yield* prepareAuthArtifact( + cacheRoot, + `#!${process.execPath}\nconsole.log("selected pin ran");\n`, + pin, + ); + const runner = yield* makeRunner(root, cacheRoot); + const output: string[] = []; + const result = yield* runner.run({ + command: initialization.auth({ + pin, + databaseId, + databaseUrl: "postgresql://invalid:invalid@127.0.0.1:1/postgres", + }), + credentials, + stdout: (bytes) => Effect.sync(() => output.push(new TextDecoder().decode(bytes))), + stderr: () => Effect.void, + }); + + expect(result.exitCode).toBe(0); + expect(output.join("")).toContain("selected pin ran"); + }).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + ), +); + +it.live.skipIf(target === undefined || process.platform === "win32")( + "refuses an initialization command older than the one that last migrated the database", + () => + Effect.scoped( + Effect.gen(function* () { + if (target === undefined) return; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "initialization-command-guard-" }); + const cacheRoot = path.join(root, "cache"); + const pinAt = (upstreamVersion: string, digest: string): ArtifactPin => ({ + ...selection.auth, + upstreamVersion, + revision: 1, + natives: { + ...selection.auth.natives, + [target]: { archive: digest.repeat(64), manifest: digest.repeat(64) }, + }, + }); + const newer = pinAt("v2.300.0", "c"); + const older = pinAt("v2.299.0", "d"); + yield* prepareAuthArtifact( + cacheRoot, + `#!${process.execPath}\nconsole.log("ran newer");\n`, + newer, + ); + yield* prepareAuthArtifact( + cacheRoot, + `#!${process.execPath}\nconsole.log("ran older");\n`, + older, + ); + const runner = yield* makeRunner(root, cacheRoot); + const output: string[] = []; + const run = (pin: ArtifactPin) => + runner.run({ + command: initialization.auth({ + pin, + databaseId, + databaseUrl: "postgresql://invalid:invalid@127.0.0.1:1/postgres", + }), + credentials, + stdout: (bytes) => Effect.sync(() => output.push(new TextDecoder().decode(bytes))), + stderr: () => Effect.void, + }); + + expect((yield* run(newer)).exitCode).toBe(0); + const refused = yield* run(older).pipe(Effect.flip); + expect(refused.message).toContain("auth v2.299.0 is older than v2.300.0"); + expect(refused.message).toContain("supabase stack destroy --stack-id"); + expect((yield* run(newer)).exitCode).toBe(0); + + expect(output.join("")).toBe("ran newer\nran newer\n"); + }).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + ), +); + it.live.skipIf(target === undefined || process.platform === "win32")( "interrupts a running initialization command and completes its cleanup", () => @@ -151,7 +253,8 @@ it.live.skipIf(target === undefined || process.platform === "win32")( const command = yield* runner .run({ command: initialization.auth({ - version: authVersion, + pin: selection.auth, + databaseId, databaseUrl: "postgresql://invalid:invalid@127.0.0.1:1/postgres", }), credentials, @@ -209,7 +312,8 @@ it.effect.skipIf(target === undefined || process.platform === "win32")( const command = yield* runner .run({ command: initialization.auth({ - version: authVersion, + pin: selection.auth, + databaseId, databaseUrl: "postgresql://invalid:invalid@127.0.0.1:1/postgres", }), credentials, diff --git a/packages/stack/src/host/CommandRunner.native-cleanup.integration.test.ts b/packages/stack/src/host/CommandRunner.native-cleanup.integration.test.ts index 57237ccd5e..69742ce820 100644 --- a/packages/stack/src/host/CommandRunner.native-cleanup.integration.test.ts +++ b/packages/stack/src/host/CommandRunner.native-cleanup.integration.test.ts @@ -12,6 +12,7 @@ import { } from "../preparation/ArtifactStore.ts"; import { PreparationError } from "../preparation/Errors.ts"; import * as CommandRunner from "./CommandRunner.ts"; +import { selection } from "../../tests/pins.ts"; const target = process.platform === "darwin" && process.arch === "arm64" @@ -34,6 +35,7 @@ const prepareDatabaseArtifact = Effect.fn(function* (cacheRoot: string) { const fs = yield* FileSystem.FileSystem; const { releaseVersion, natives, requiredRuntimePaths, executablePath } = yield* resolveArtifact({ service: "database", + pin: selection.database, }); const request: ArtifactRequest = { key: `slim-services/postgres/${releaseVersion}/${target}`, @@ -123,7 +125,7 @@ it.live.skipIf(target === undefined || process.platform === "win32")( .run({ command: { type: "postgres", - command: postgres.psql({ major: 17 }), + command: postgres.psql({ pin: selection.database }), args: ["--version"], env: {}, stdin: false, diff --git a/packages/stack/src/host/CommandRunner.ts b/packages/stack/src/host/CommandRunner.ts index ee2f9f4e41..99cdd36f06 100644 --- a/packages/stack/src/host/CommandRunner.ts +++ b/packages/stack/src/host/CommandRunner.ts @@ -15,13 +15,9 @@ import { import { rmdir } from "node:fs/promises"; import { ChildProcessSpawner } from "effect/process"; import { HttpClient } from "effect/http"; -import { - slimImageMirrors, - postgresVersion, - resolveArtifact, - useNativeArtifact, -} from "../Artifacts.ts"; +import { slimImageMirrors, resolveArtifact, useNativeArtifact } from "../Artifacts.ts"; import { isSafeId } from "../identity/SafeId.ts"; +import { holdSchemaWriter } from "../internal/schema-writers.ts"; import * as Environment from "../namespace/Environment.ts"; import { borrow } from "../namespace/Paths.ts"; import { makeContainerRuntime, type EngineTarget, type HostGateway } from "../runtime/Container.ts"; @@ -157,14 +153,28 @@ const makeCommandRunner = (options: { const directory = yield* fs .makeTempDirectoryScoped({ directory: jobsRoot, prefix: `${jobId}-` }) .pipe(Effect.mapError(failure)); - const initialization = + const invocation = "credentials" in input - ? yield* resolveInitializationCommand(input.command, { - runtime: options.runtime, - credentials: input.credentials, - }).pipe(Effect.mapError(failure)) - : undefined; - const postgresCommand = "stdin" in input ? input.command : undefined; + ? { + initialization: yield* resolveInitializationCommand(input.command, { + runtime: options.runtime, + credentials: input.credentials, + }).pipe(Effect.mapError(failure)), + postgresCommand: undefined, + } + : { initialization: undefined, postgresCommand: input.command }; + const { initialization, postgresCommand } = invocation; + if (initialization !== undefined && "credentials" in input) { + const databaseId = input.command.databaseId; + if (!isSafeId(databaseId)) return yield* failure("Invalid database instance id"); + // Held for the whole job, so the migration finishes before another writer of the service starts. + yield* holdSchemaWriter(fs, path, { + stackId: options.stackId, + instanceRoot: path.join(options.root, databaseId), + service: initialization.service, + version: initialization.pin.upstreamVersion, + }).pipe(Effect.mapError(failure)); + } const inputStream = "stdin" in input ? (input.stdin ?? Stream.empty) : Stream.empty; if ( postgresCommand !== undefined && @@ -177,15 +187,17 @@ const makeCommandRunner = (options: { ...(postgresCommand?.pgProve?.mounts ?? []), ]) yield* borrowCallerPath(mount.source); - const version = - initialization?.version ?? postgresVersion(String(postgresCommand?.command.major)); + const pin = + invocation.initialization === undefined + ? invocation.postgresCommand.command.pin + : invocation.initialization.pin; const process = yield* Effect.gen(function* () { if (container === undefined) { // Pins the generation for the life of this job's scope, before the spawn below: the // pin is registered ahead of the process, so it releases only after the process's // own cleanup finalizer (added later) has run. const artifact = yield* useNativeArtifact( - { service: initialization?.service ?? "database", version }, + { service: initialization?.service ?? "database", pin }, options.cacheRoot, ).pipe( Effect.provideService(FileSystem.FileSystem, fs), @@ -236,8 +248,7 @@ const makeCommandRunner = (options: { }; } const image = - initialization?.image ?? - (yield* resolveArtifact({ service: "database", version })).image; + initialization?.image ?? (yield* resolveArtifact({ service: "database", pin })).image; yield* container.prepare(image); const child = yield* container .launchCommand({ diff --git a/packages/stack/src/host/LogForwarder.integration.test.ts b/packages/stack/src/host/LogForwarder.integration.test.ts index 4df2a23003..7d356a5d9f 100644 --- a/packages/stack/src/host/LogForwarder.integration.test.ts +++ b/packages/stack/src/host/LogForwarder.integration.test.ts @@ -32,6 +32,7 @@ import * as GatewayLog from "./GatewayLog.ts"; import type { LogRecord } from "./LogRecord.ts"; import * as LogForwarder from "./LogForwarder.ts"; import * as LogStore from "./LogStore.ts"; +import { selection } from "../../tests/pins.ts"; const { flushWindowMillis, pollMillis, postTimeoutMillis, retryMillis, stuckMillis } = LogForwarder; @@ -149,7 +150,11 @@ const analyticsTarget = (port: Effect.Effect<number>) => port: current, })), ), - creation: Effect.succeed({ service: "analytics" as const, config: { apiKey: "test-key" } }), + creation: Effect.succeed({ + service: "analytics" as const, + pin: selection.analytics, + config: { apiKey: "test-key" }, + }), serving: SubscriptionRef.changes(observation), } satisfies LogForwarder.ForwardedInstance, }; diff --git a/packages/stack/src/host/LogflareStorage.integration.test.ts b/packages/stack/src/host/LogflareStorage.integration.test.ts index d18a2584ef..3ac06f363c 100644 --- a/packages/stack/src/host/LogflareStorage.integration.test.ts +++ b/packages/stack/src/host/LogflareStorage.integration.test.ts @@ -4,12 +4,13 @@ import { describe, expect, it } from "@effect/vitest"; import { Context, Effect, Exit, FileSystem, Layer, Redacted, Ref, Scope } from "effect"; import { connect, createServer, type Socket } from "node:net"; // oxlint-disable-line effecttsgo/node-builtin-import -- real socket fixture. import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; -import { makeDatabase, type DatabaseConfig } from "../services/Database.ts"; +import { makeDatabase, type PinnedDatabaseConfig } from "../services/Database.ts"; import * as LogflareStorage from "./LogflareStorage.ts"; -const config: DatabaseConfig = { - version: "17", +const config: PinnedDatabaseConfig = { + pin: selection.database, databasePassword: Redacted.make("logflare-storage-password"), jwtSecret: Redacted.make("logflare-storage-jwt-secret-with-32-chars"), jwtExpiry: 3600, diff --git a/packages/stack/src/index.ts b/packages/stack/src/index.ts index 5b7774723e..a506ae0ef7 100644 --- a/packages/stack/src/index.ts +++ b/packages/stack/src/index.ts @@ -15,6 +15,7 @@ export type { export type { StackCredentials, StackKeysInput } from "./StackNamespace.ts"; export { initialization, postgres } from "./Commands.ts"; export { StackError } from "./Rpc.ts"; +export { LockInvalidError, LockIoError, LockUnsupportedVersionError } from "./Lock.ts"; export type { CompositionConfig } from "./Orchestrator.ts"; export type { Observation } from "./Rpc.ts"; export type { diff --git a/packages/stack/src/internal/artifacts.ts b/packages/stack/src/internal/artifacts.ts index 95c4f32023..5907e3bae5 100644 --- a/packages/stack/src/internal/artifacts.ts +++ b/packages/stack/src/internal/artifacts.ts @@ -3,18 +3,28 @@ export { ArtifactError, artifactServiceKinds, catalogPins, + databaseArtifactPin, defaultRuntime, isOrioledbVersion, orioledbConfigVersion, orioledbPostgresVersion, orioledbVersions, postgresMajor, - postgresVersion, prepareNativeArtifact, resolveArtifact, useNativeArtifact, type ArtifactKind, type ArtifactPin, + type ArtifactSelection, type NativePin, type ServiceKind, } from "../Artifacts.ts"; +export { + ReleaseIntegrity, + ReleaseTransient, + ReleaseUnpublished, + resolvePin, + type ReleaseError, + type ResolvePinRequest, +} from "../preparation/ReleaseResolver.ts"; +export { prepareArtifactSelection, type PreparedRelease } from "../preparation/PrepareSelection.ts"; diff --git a/packages/stack/src/internal/data-conflicts.ts b/packages/stack/src/internal/data-conflicts.ts new file mode 100644 index 0000000000..20ca1e7bc9 --- /dev/null +++ b/packages/stack/src/internal/data-conflicts.ts @@ -0,0 +1,119 @@ +import { Effect } from "effect"; +import type { FileSystem, Path } from "effect"; +import type { ArtifactSelection } from "../Artifacts.ts"; +import type { SavedStack } from "../StackNamespace.ts"; +import { + DatabaseDataState, + databaseIdentityMismatch, + readDatabaseDataState, + readDatabaseIdentity, +} from "./database-reuse.ts"; +import { + boundDatabaseId, + readSchemaWriters, + schemaWriterDowngrade, + schemaWriterServices, + type SchemaWriterService, +} from "./schema-writers.ts"; + +/** + * Saved data in database instance `instanceId` that a start at the requested pins would refuse, + * with the failure a start reports (`message`). `recorded` is `undefined` when the data has no + * identity. + */ +export type DataConflict = + | { + readonly kind: "database-build"; + readonly stackId: string; + readonly service: "database"; + readonly instanceId: string; + readonly recorded: string | undefined; + readonly requested: string; + readonly message: string; + } + | { + readonly kind: "schema-downgrade"; + readonly stackId: string; + readonly service: SchemaWriterService; + readonly instanceId: string; + readonly recorded: string; + readonly requested: string; + readonly message: string; + }; + +/** + * The conflicts `requested` would meet in a saved stack's data under `dataRoot`, judged by the + * rules a start applies: initialized data per its runtime, the identity a container runtime + * recorded before an interrupted first start, and the schema writers the composition binds to each + * database. + */ +export const collectDataConflicts = Effect.fn("DataConflicts.find")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + options: { + readonly saved: Pick<SavedStack, "id" | "runtime" | "instances" | "composition">; + readonly dataRoot: string; + readonly requested: Partial<ArtifactSelection>; + }, +) { + const { saved, dataRoot, requested } = options; + const serviceOf = (id: string) => + saved.instances.find((instance) => instance.id === id)?.creation.service; + const conflicts: Array<DataConflict> = []; + for (const { id: databaseId } of saved.instances.filter( + ({ creation }) => creation.service === "database", + )) { + const instanceRoot = path.join(dataRoot, databaseId); + const databasePin = requested.database; + if (databasePin !== undefined) { + const state = yield* readDatabaseDataState(fs, path, instanceRoot, saved.runtime); + const initialized = DatabaseDataState.$is("Initialized")(state); + // A container volume is invisible to the host; a start's prepare judges the identity recorded + // before initdb whenever the volume holds PG_VERSION. + const maybeInVolume = DatabaseDataState.$is("Fresh")(state) && saved.runtime !== "native"; + const recorded = + initialized || maybeInVolume + ? yield* readDatabaseIdentity(fs, path, instanceRoot) + : undefined; + const mismatch = + initialized || recorded !== undefined + ? databaseIdentityMismatch(saved.id, recorded, databasePin.upstreamVersion) + : undefined; + if (mismatch !== undefined) + conflicts.push({ + kind: "database-build", + stackId: saved.id, + service: "database", + instanceId: databaseId, + recorded: mismatch.recorded?.build, + requested: mismatch.requested.build, + message: mismatch.message, + }); + } + const writers = yield* readSchemaWriters(fs, path, instanceRoot); + for (const service of schemaWriterServices) { + const pin = requested[service]; + const bound = saved.instances.some( + ({ id, creation }) => + creation.service === service && + boundDatabaseId(saved.composition.dependencies, serviceOf, id) === databaseId, + ); + const downgrade = + pin === undefined || !bound + ? undefined + : schemaWriterDowngrade(saved.id, writers, service, pin.upstreamVersion); + if (downgrade !== undefined) + conflicts.push({ + kind: "schema-downgrade", + stackId: saved.id, + service, + instanceId: databaseId, + recorded: downgrade.recorded, + requested: downgrade.selected, + message: downgrade.message, + }); + } + } + yield* Effect.annotateCurrentSpan({ "data_conflicts.count": conflicts.length }); + return conflicts; +}); diff --git a/packages/stack/src/internal/database-reuse.ts b/packages/stack/src/internal/database-reuse.ts index 365d01cb0d..e5590eeef9 100644 --- a/packages/stack/src/internal/database-reuse.ts +++ b/packages/stack/src/internal/database-reuse.ts @@ -1,33 +1,224 @@ -import { isOrioledbVersion, postgresLine, postgresMajor } from "../Artifacts.ts"; +import { Data, Effect, Schema } from "effect"; +import type { FileSystem, Path } from "effect"; +import { isOrioledbVersion, postgresMajor } from "../Artifacts.ts"; /** Recovery for local database data a stack cannot reuse, worded like the CLI's saved-stack advice. */ export const recreateStackAdvice = (stackId: string): string => `run \`supabase stack destroy --stack-id ${stackId}\` to recreate the stack — this permanently deletes its local database data`; +/** Recovery that keeps data created at `version` by pinning it again, before the destructive recreate. */ +export const keepOrRecreateAdvice = (stackId: string, service: string, version: string): string => + `keep this data by pinning \`[local.services.${service}] version = "${version}"\` in supabase/config.toml, or ${recreateStackAdvice(stackId)}`; + +/** The runtimes a database runs under. */ +export const DatabaseRuntime = Schema.Literals(["native", "docker", "podman"]); +export type DatabaseRuntime = typeof DatabaseRuntime.Type; + /** - * Why existing PostgreSQL data cannot serve the requested artifact version, or `undefined` when it - * can. `PG_VERSION` proves only the major, so an unknown `line` counts as the stock line. + * What initialized PostgreSQL data was created by: its release line, major and upstream build. Only + * a slim-services revision of the same build can reuse it. */ -export const unusableDatabaseData = ( - data: { - readonly major: string; - readonly line: string | undefined; - readonly initialized: boolean; +export const DatabaseIdentity = Schema.Struct({ + line: Schema.Literals(["stock", "orioledb"]), + major: Schema.String, + build: Schema.String, +}); +export interface DatabaseIdentity extends Schema.Schema.Type<typeof DatabaseIdentity> {} + +/** The identity a database artifact version initializes. */ +export const databaseIdentity = (version: string): DatabaseIdentity => ({ + line: isOrioledbVersion(version) ? "orioledb" : "stock", + major: postgresMajor(version), + build: version, +}); + +/** Whether two identities describe the same line, major and build. */ +export const sameDatabaseIdentity = (a: DatabaseIdentity, b: DatabaseIdentity): boolean => + a.line === b.line && a.major === b.major && a.build === b.build; + +const describeIdentity = (identity: DatabaseIdentity): string => + `${identity.line === "orioledb" ? "OrioleDB" : "stock PostgreSQL"} build ${identity.build}`; + +/** Existing database data was created by a different build, line or major than the one requested. */ +export class DatabaseIdentityMismatch extends Schema.TaggedError<DatabaseIdentityMismatch>()( + "DatabaseIdentityMismatch", + { + message: Schema.String, + recorded: Schema.optionalKey(DatabaseIdentity), + requested: DatabaseIdentity, }, +) {} + +/** + * How to keep data `recorded` created: pin its build, which selects its line, plus the `[db]` edits + * that keep the pin valid whichever setting selected the requested build. + */ +const keepDatabaseAdvice = ( + stackId: string, + recorded: DatabaseIdentity, + requested: DatabaseIdentity, +): string => { + const edits = [ + ...(recorded.major === requested.major + ? [] + : [`setting \`[db] major_version = ${recorded.major}\``]), + ...(requested.line === "orioledb" ? ["removing any `[db] orioledb_version`"] : []), + ]; + if (edits.length === 0) return keepOrRecreateAdvice(stackId, "database", recorded.build); + const clauses = new Intl.ListFormat("en", { type: "conjunction" }).format([ + `pinning \`[local.services.database] version = "${recorded.build}"\``, + ...edits, + ]); + return `keep this data by ${clauses} in supabase/config.toml, or ${recreateStackAdvice(stackId)}`; +}; + +/** + * The failure to reuse existing database data for `version`, or `undefined` when it can serve it. + * Data with no recorded identity cannot be verified, so it is refused. + */ +export const databaseIdentityMismatch = ( + stackId: string, + recorded: DatabaseIdentity | undefined, version: string, -): string | undefined => { - const subject = data.initialized - ? "Initialized PostgreSQL data" - : "PostgreSQL data from an unfinished first start"; - const major = postgresMajor(version); - if (data.major !== major) - return `${subject} is major ${data.major}, but major ${major} was requested`; - const line = postgresLine(version); - if (data.line === undefined) - return isOrioledbVersion(version) - ? "Unmarked PostgreSQL data cannot be verified as OrioleDB data" - : undefined; - return data.line === line - ? undefined - : `${subject} belongs to release line ${data.line}, but ${line} was requested`; +): DatabaseIdentityMismatch | undefined => { + const requested = databaseIdentity(version); + if (recorded !== undefined && sameDatabaseIdentity(recorded, requested)) return undefined; + const subject = + recorded === undefined + ? "PostgreSQL data has no recorded identity" + : `PostgreSQL data was initialized by ${describeIdentity(recorded)}`; + const advice = + recorded === undefined + ? recreateStackAdvice(stackId) + : keepDatabaseAdvice(stackId, recorded, requested); + return new DatabaseIdentityMismatch({ + message: `${subject}, but ${describeIdentity(requested)} was requested; ${advice}`, + ...(recorded === undefined ? {} : { recorded }), + requested, + }); }; + +/** Written once a database's bootstrap completed; its presence marks initialized data. */ +export const databaseReadyMarkerFile = ".supabase-database-ready.json"; + +/** The content of {@link databaseReadyMarkerFile}. */ +export const DatabaseReadyMarker = Schema.Struct({ + runtime: DatabaseRuntime, + profile: Schema.Literal("supabase"), +}); +export interface DatabaseReadyMarker extends Schema.Schema.Type<typeof DatabaseReadyMarker> {} + +const identityFile = ".supabase-database-identity.json"; +const IdentityJson = Schema.fromJsonString(DatabaseIdentity); + +/** The identity recorded for an instance's data, or `undefined` when none was recorded. */ +export const readDatabaseIdentity = Effect.fn("Database.readIdentity")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, +) { + const file = path.join(instanceRoot, identityFile); + if (!(yield* fs.exists(file))) return undefined; + return yield* fs.readFileString(file).pipe(Effect.flatMap(Schema.decodeEffect(IdentityJson))); +}); + +/** Forgets an instance's data identity together with its data. */ +export const removeDatabaseIdentity = ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, +) => fs.remove(path.join(instanceRoot, identityFile), { force: true }); + +/** Records an instance's data identity; callers write it before initdb so an interrupted first start keeps it. */ +export const recordDatabaseIdentity = Effect.fn("Database.recordIdentity")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, + identity: DatabaseIdentity, +) { + const encoded = yield* Schema.encodeEffect(IdentityJson)(identity); + const stage = yield* fs.makeTempDirectoryScoped({ + directory: instanceRoot, + prefix: ".identity-", + }); + yield* fs.writeFileString(path.join(stage, "marker"), encoded, { mode: 0o600 }); + yield* fs.rename(path.join(stage, "marker"), path.join(instanceRoot, identityFile)); +}, Effect.scoped); + +/** What a start at one runtime finds under an instance root. */ +export type DatabaseDataState = Data.TaggedEnum<{ + Fresh: {}; + Initialized: {}; + RuntimeMismatch: { readonly recorded: DatabaseRuntime }; +}>; +export const DatabaseDataState = Data.taggedEnum<DatabaseDataState>(); + +/** + * How a start at `runtime` treats what it found: a readiness marker means initialized data of the + * runtime that wrote it, and a bare `PG_VERSION` (an interrupted first start) counts only natively, + * since a Docker volume is invisible to the host and its storage is checked at launch. + */ +export const databaseDataState = ( + runtime: DatabaseRuntime, + found: { readonly marker: DatabaseReadyMarker | undefined; readonly pgVersion: boolean }, +): DatabaseDataState => { + if (found.marker !== undefined) + return found.marker.runtime === runtime + ? DatabaseDataState.Initialized() + : DatabaseDataState.RuntimeMismatch({ recorded: found.marker.runtime }); + return runtime === "native" && found.pgVersion + ? DatabaseDataState.Initialized() + : DatabaseDataState.Fresh(); +}; + +/** The {@link databaseDataState} of the data under `instanceRoot`. */ +export const readDatabaseDataState = Effect.fn("Database.dataState")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, + runtime: DatabaseRuntime, +) { + const markerPath = path.join(instanceRoot, databaseReadyMarkerFile); + const marker = (yield* fs.exists(markerPath)) + ? yield* fs + .readFileString(markerPath) + .pipe(Effect.flatMap(Schema.decodeEffect(Schema.fromJsonString(DatabaseReadyMarker)))) + : undefined; + const pgVersion = + marker === undefined && (yield* fs.exists(path.join(instanceRoot, "data", "PG_VERSION"))); + return databaseDataState(runtime, { marker, pgVersion }); +}); + +/** + * Whether `instanceRoot` holds initialized database data: a readiness marker, or a PostgreSQL data + * directory an interrupted first start left behind. A Docker volume is invisible to the host, so + * only its marker counts here; the launch guard checks its storage. + */ +const databaseDataInitialized = Effect.fn("Database.dataInitialized")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, +) { + const initialized = + (yield* fs.exists(path.join(instanceRoot, databaseReadyMarkerFile))) || + (yield* fs.exists(path.join(instanceRoot, "data", "PG_VERSION"))); + yield* Effect.annotateCurrentSpan({ "database.data.initialized": initialized }); + return initialized; +}); + +/** The ids of the saved database instances under `dataRoot` whose data is initialized. */ +export const initializedDatabaseIds = Effect.fn("Database.initializedIds")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + dataRoot: string, + instances: ReadonlyArray<{ + readonly id: string; + readonly creation: { readonly service: string }; + }>, +) { + const databases = instances.filter(({ creation }) => creation.service === "database"); + const initialized = yield* Effect.filter(databases, ({ id }) => + databaseDataInitialized(fs, path, path.join(dataRoot, id)), + ); + return new Set(initialized.map(({ id }) => id)); +}); diff --git a/packages/stack/src/internal/database-reuse.unit.test.ts b/packages/stack/src/internal/database-reuse.unit.test.ts index 8dc80b74f9..ee83aa23ab 100644 --- a/packages/stack/src/internal/database-reuse.unit.test.ts +++ b/packages/stack/src/internal/database-reuse.unit.test.ts @@ -1,66 +1,123 @@ -import { describe, expect, it } from "@effect/vitest"; -import { unusableDatabaseData } from "./database-reuse.ts"; +import { expect, it } from "@effect/vitest"; +import { + databaseDataState, + databaseIdentity, + databaseIdentityMismatch, + type DatabaseReadyMarker, +} from "./database-reuse.ts"; -const stock = "17.11.0.002"; -const orioledb = "17.11.0.002-orioledb"; +const STOCK_17 = "17.6.1.054"; +const STOCK_17_NEWER = "17.6.1.060"; +const STOCK_15 = "15.8.1.085"; +const ORIOLEDB_17 = "17.11.0.002-orioledb"; +const ORIOLEDB_17_NEWER = "17.11.0.004-orioledb"; -describe("unusableDatabaseData", () => { - it.each([ - { - name: "initialized data from another major", - data: { major: "15", line: "15", initialized: true }, - version: stock, - reason: "Initialized PostgreSQL data is major 15, but major 17 was requested", - }, - { - name: "unfinished data from another major", - data: { major: "15", line: undefined, initialized: false }, - version: stock, - reason: - "PostgreSQL data from an unfinished first start is major 15, but major 17 was requested", - }, - { - name: "unrecorded data for OrioleDB", - data: { major: "17", line: undefined, initialized: false }, - version: orioledb, - reason: "Unmarked PostgreSQL data cannot be verified as OrioleDB data", - }, - { - name: "stock-recorded data for OrioleDB", - data: { major: "17", line: "17", initialized: false }, - version: orioledb, - reason: - "PostgreSQL data from an unfinished first start belongs to release line 17, but 17.11.0.002-orioledb was requested", - }, - { - name: "OrioleDB-recorded data for stock", - data: { major: "17", line: "17.11.0.002-orioledb", initialized: false }, - version: stock, - reason: - "PostgreSQL data from an unfinished first start belongs to release line 17.11.0.002-orioledb, but 17 was requested", - }, - { - name: "data from another OrioleDB build", - data: { major: "17", line: "17.9.0.028-orioledb", initialized: true }, - version: orioledb, - reason: - "Initialized PostgreSQL data belongs to release line 17.9.0.028-orioledb, but 17.11.0.002-orioledb was requested", - }, - ])("refuses $name", ({ data, version, reason }) => { - expect(unusableDatabaseData(data, version)).toBe(reason); - }); +const destroy = + "run `supabase stack destroy --stack-id stack-a` to recreate the stack — this permanently deletes its local database data"; - it.each([ - { name: "unrecorded data for stock", line: undefined, version: stock }, - { name: "stock-recorded data for stock", line: "17", version: stock }, - { - name: "OrioleDB-recorded data for OrioleDB", - line: "17.11.0.002-orioledb", - version: orioledb, - }, - ])("reuses $name", ({ line, version }) => { - expect( - unusableDatabaseData({ major: "17", line, initialized: false }, version), - ).toBeUndefined(); - }); +const messageFor = (recorded: string | undefined, requested: string) => + databaseIdentityMismatch( + "stack-a", + recorded === undefined ? undefined : databaseIdentity(recorded), + requested, + )?.message; + +it("offers to keep data by pinning its build when the major and line are unchanged", () => { + expect(messageFor(STOCK_17, STOCK_17_NEWER)).toBe( + `PostgreSQL data was initialized by stock PostgreSQL build ${STOCK_17}, but stock PostgreSQL build ${STOCK_17_NEWER} was requested; keep this data by pinning \`[local.services.database] version = "${STOCK_17}"\` in supabase/config.toml, or ${destroy}`, + ); +}); + +it("offers to restore the recorded major beside the pin when the major differs", () => { + expect(messageFor(STOCK_15, STOCK_17)).toBe( + `PostgreSQL data was initialized by stock PostgreSQL build ${STOCK_15}, but stock PostgreSQL build ${STOCK_17} was requested; keep this data by pinning \`[local.services.database] version = "${STOCK_15}"\` and setting \`[db] major_version = 15\` in supabase/config.toml, or ${destroy}`, + ); +}); + +it("offers to remove any orioledb_version when stock data is asked for an OrioleDB build of the same major", () => { + expect(messageFor(STOCK_17, ORIOLEDB_17)).toBe( + `PostgreSQL data was initialized by stock PostgreSQL build ${STOCK_17}, but OrioleDB build ${ORIOLEDB_17} was requested; keep this data by pinning \`[local.services.database] version = "${STOCK_17}"\` and removing any \`[db] orioledb_version\` in supabase/config.toml, or ${destroy}`, + ); +}); + +it("offers only the pin when OrioleDB data is asked for a stock build of the same major", () => { + expect(messageFor(ORIOLEDB_17, STOCK_17)).toBe( + `PostgreSQL data was initialized by OrioleDB build ${ORIOLEDB_17}, but stock PostgreSQL build ${STOCK_17} was requested; keep this data by pinning \`[local.services.database] version = "${ORIOLEDB_17}"\` in supabase/config.toml, or ${destroy}`, + ); +}); + +it("offers to remove any orioledb_version when OrioleDB data is asked for another OrioleDB build", () => { + expect(messageFor(ORIOLEDB_17, ORIOLEDB_17_NEWER)).toBe( + `PostgreSQL data was initialized by OrioleDB build ${ORIOLEDB_17}, but OrioleDB build ${ORIOLEDB_17_NEWER} was requested; keep this data by pinning \`[local.services.database] version = "${ORIOLEDB_17}"\` and removing any \`[db] orioledb_version\` in supabase/config.toml, or ${destroy}`, + ); +}); + +it("offers each edit that applies when both the major and the line differ", () => { + expect(messageFor(STOCK_15, ORIOLEDB_17)).toContain( + `keep this data by pinning \`[local.services.database] version = "${STOCK_15}"\`, setting \`[db] major_version = 15\`, and removing any \`[db] orioledb_version\` in supabase/config.toml`, + ); + expect(messageFor(ORIOLEDB_17, STOCK_15)).toContain( + `keep this data by pinning \`[local.services.database] version = "${ORIOLEDB_17}"\` and setting \`[db] major_version = 17\` in supabase/config.toml`, + ); +}); + +it("offers only the recreate for data with no recorded identity", () => { + expect(messageFor(undefined, STOCK_17_NEWER)).toBe( + `PostgreSQL data has no recorded identity, but stock PostgreSQL build ${STOCK_17_NEWER} was requested; ${destroy}`, + ); +}); + +const marker = (runtime: DatabaseReadyMarker["runtime"]): DatabaseReadyMarker => ({ + runtime, + profile: "supabase", }); + +it.each([ + { + found: "a marker of the same runtime", + runtime: "native", + observed: { marker: marker("native"), pgVersion: false }, + expected: "Initialized", + }, + { + found: "a marker beside PG_VERSION", + runtime: "docker", + observed: { marker: marker("docker"), pgVersion: true }, + expected: "Initialized", + }, + { + found: "a bare PG_VERSION", + runtime: "native", + observed: { marker: undefined, pgVersion: true }, + expected: "Initialized", + }, + { + found: "a bare PG_VERSION", + runtime: "docker", + observed: { marker: undefined, pgVersion: true }, + expected: "Fresh", + }, + { + found: "a bare PG_VERSION", + runtime: "podman", + observed: { marker: undefined, pgVersion: true }, + expected: "Fresh", + }, + { + found: "nothing", + runtime: "native", + observed: { marker: undefined, pgVersion: false }, + expected: "Fresh", + }, + { + found: "a marker of another runtime", + runtime: "native", + observed: { marker: marker("docker"), pgVersion: true }, + expected: "RuntimeMismatch", + }, +] as const)( + "treats $found at the $runtime runtime as $expected", + ({ runtime, observed, expected }) => { + expect(databaseDataState(runtime, observed)._tag).toBe(expected); + }, +); diff --git a/packages/stack/src/internal/schema-writers.integration.test.ts b/packages/stack/src/internal/schema-writers.integration.test.ts new file mode 100644 index 0000000000..3a11a9fea5 --- /dev/null +++ b/packages/stack/src/internal/schema-writers.integration.test.ts @@ -0,0 +1,45 @@ +import { NodeServices } from "@effect/platform-node"; +import { expect, it } from "@effect/vitest"; +import { Effect, Exit, Fiber, FileSystem, Path, Scope } from "effect"; +import { TestClock } from "effect/testing"; +import { holdSchemaWriter, type SchemaWriterService } from "./schema-writers.ts"; + +const databaseRoot = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + return yield* fs.makeTempDirectoryScoped({ prefix: "schema-writers-" }); +}); + +const selecting = (instanceRoot: string, service: SchemaWriterService, version: string) => ({ + stackId: "schema-writers-test", + instanceRoot, + service, + version, +}); + +it.effect("holds one writer per service and database until its migration ends", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const instanceRoot = yield* databaseRoot; + const first = yield* Scope.make(); + yield* holdSchemaWriter(fs, path, selecting(instanceRoot, "auth", "v2.197.0")).pipe( + Scope.provide(first), + ); + + yield* holdSchemaWriter(fs, path, selecting(instanceRoot, "storage", "v1.0.0")).pipe( + Effect.scoped, + ); + const waiting = yield* holdSchemaWriter( + fs, + path, + selecting(instanceRoot, "auth", "v2.197.0"), + ).pipe(Effect.scoped, Effect.flip, Effect.forkChild); + yield* TestClock.adjust("61 seconds"); + expect((yield* Fiber.join(waiting)).message).toContain("locked by another operation"); + + yield* Scope.close(first, Exit.void); + yield* holdSchemaWriter(fs, path, selecting(instanceRoot, "auth", "v2.197.0")).pipe( + Effect.scoped, + ); + }).pipe(Effect.scoped, Effect.provide(NodeServices.layer)), +); diff --git a/packages/stack/src/internal/schema-writers.ts b/packages/stack/src/internal/schema-writers.ts new file mode 100644 index 0000000000..1f1025345f --- /dev/null +++ b/packages/stack/src/internal/schema-writers.ts @@ -0,0 +1,307 @@ +import { Effect, Schema } from "effect"; +import type { FileSystem, Path } from "effect"; +import type { CompositionConfig } from "../Orchestrator.ts"; +import { namespaceError, type NamespaceError } from "../namespace/Capabilities.ts"; +import { holdLock } from "../namespace/Lock.ts"; +import { publish } from "../namespace/Publication.ts"; +import { keepOrRecreateAdvice } from "./database-reuse.ts"; + +/** The services whose migrations write schemas into the database. */ +export const schemaWriterServices = ["auth", "storage", "realtime", "analytics", "pooler"] as const; +export const SchemaWriterService = Schema.Literals(schemaWriterServices); +export type SchemaWriterService = typeof SchemaWriterService.Type; + +/** Whether `service` owns schemas in the database, so it can never run an older upstream than the record. */ +export const isSchemaWriter = Schema.is(SchemaWriterService); + +/** + * The database instance `instanceId` takes its `databaseUrl` from, or `undefined` when that URL is + * not bound to a local database. `serviceOf` names an instance's service kind. + */ +export const boundDatabaseId = ( + dependencies: CompositionConfig["dependencies"], + serviceOf: (id: string) => string | undefined, + instanceId: string, +): string | undefined => + dependencies.find( + ({ from, to, bindings }) => + to === instanceId && + serviceOf(from) === "database" && + bindings?.some(({ input }) => input === "databaseUrl") === true, + )?.from; + +/** The newest upstream version of each schema-owning service that migrated a database. */ +export const SchemaWriters = Schema.Struct({ + auth: Schema.optionalKey(Schema.String), + storage: Schema.optionalKey(Schema.String), + realtime: Schema.optionalKey(Schema.String), + analytics: Schema.optionalKey(Schema.String), + pooler: Schema.optionalKey(Schema.String), +}); +export interface SchemaWriters extends Schema.Schema.Type<typeof SchemaWriters> {} + +const numericCore = /^v?(\d+(?:\.\d+)*)/u; +const components = (version: string): ReadonlyArray<number> => + numericCore + .exec(version)?.[1] + ?.split(".") + .map((part) => Number.parseInt(part, 10)) ?? []; + +/** + * Orders upstream versions by their leading dotted number, such as `v2.196.0`, `2026.10.05-sha-…` + * or `1.30`; a suffix is ignored and a version with no leading number is older than any other. + */ +export const compareUpstreamVersions = (a: string, b: string): number => { + const left = components(a); + const right = components(b); + for (let index = 0; index < Math.max(left.length, right.length); index += 1) { + const difference = (left[index] ?? 0) - (right[index] ?? 0); + if (difference !== 0) return difference < 0 ? -1 : 1; + } + return left.length === 0 && right.length > 0 ? -1 : left.length > 0 && right.length === 0 ? 1 : 0; +}; + +/** A schema-owning service was selected at an older upstream than one that already migrated the database. */ +export class SchemaWriterDowngrade extends Schema.TaggedError<SchemaWriterDowngrade>()( + "SchemaWriterDowngrade", + { + message: Schema.String, + service: SchemaWriterService, + selected: Schema.String, + recorded: Schema.String, + }, +) {} + +/** The failure to run `service` at `selected` against `writers`, or `undefined` when it may migrate. */ +export const schemaWriterDowngrade = ( + stackId: string, + writers: SchemaWriters, + service: SchemaWriterService, + selected: string, +): SchemaWriterDowngrade | undefined => { + const recorded = writers[service]; + if (recorded === undefined || compareUpstreamVersions(selected, recorded) >= 0) return undefined; + return new SchemaWriterDowngrade({ + message: `${service} ${selected} is older than ${recorded}, which already migrated this database; downgrading does not restore the old schema, so ${keepOrRecreateAdvice(stackId, service, recorded)}`, + service, + selected, + recorded, + }); +}; + +/** The record's file name at a database instance root. */ +export const schemaWritersFile = "schema-writers.json"; +const WritersJson = Schema.fromJsonString(SchemaWriters); +const recordError = (cause: unknown) => namespaceError("schema-writers", cause); + +/** The record at `instanceRoot`; a missing record means no writer has migrated yet. */ +export const readSchemaWriters = Effect.fn("SchemaWriters.read")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, +) { + const file = path.join(instanceRoot, schemaWritersFile); + return yield* fs.readFileString(file).pipe( + Effect.flatMap(Schema.decodeEffect(WritersJson)), + Effect.catchIf( + (error) => "reason" in error && error.reason._tag === "NotFound", + (): Effect.Effect<SchemaWriters> => Effect.succeed({}), + ), + ); +}, Effect.mapError(recordError)); + +/** Forgets the record together with the data it describes. */ +export const removeSchemaWriters = ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, +) => fs.remove(path.join(instanceRoot, schemaWritersFile), { force: true }); + +const writeSchemaWriters = ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, + writers: SchemaWriters, +) => + Schema.encodeEffect(WritersJson)(writers).pipe( + Effect.mapError(recordError), + Effect.flatMap((content) => + publish(fs, path, { + target: path.join(instanceRoot, schemaWritersFile), + content, + }), + ), + ); + +const recordLockFile = "schema-writers.lock"; +const leaseFile = (service: SchemaWriterService) => `schema-writer-${service}.lock`; + +/** Lock files that outlive a data clear, since another operation may hold them. */ +export const schemaWriterLockFiles: ReadonlyArray<string> = [ + recordLockFile, + ...schemaWriterServices.map(leaseFile), +]; + +const criticalSection = <A, E, R>( + path: Path.Path, + instanceRoot: string, + effect: Effect.Effect<A, E, R>, +): Effect.Effect<A, E | NamespaceError, R> => + Effect.scoped( + holdLock( + path.join(instanceRoot, recordLockFile), + "5 seconds", + "The schema-writers record", + ).pipe(Effect.andThen(effect)), + ); + +const leaseBudget = "60 seconds"; + +/** + * Holds every service's lease for the enclosing scope, so no writer is admitted or still + * migrating while the caller clears or replaces the data and its record. Leases are taken in + * service order, and a writer holds only one, so waiting here cannot form a cycle. + */ +export const holdSchemaWriterLeases = Effect.fn("SchemaWriters.holdAll")(function* ( + path: Path.Path, + instanceRoot: string, +) { + for (const service of schemaWriterServices) + yield* holdLock( + path.join(instanceRoot, leaseFile(service)), + leaseBudget, + `The ${service} schema writer`, + ); +}); + +/** Runs `effect` with every schema-writer lease held; see {@link holdSchemaWriterLeases}. */ +export const withSchemaWriterLeases = <A, E, R>( + path: Path.Path, + instanceRoot: string, + effect: Effect.Effect<A, E, R>, +): Effect.Effect<A, E | NamespaceError, R> => + Effect.scoped(holdSchemaWriterLeases(path, instanceRoot).pipe(Effect.andThen(effect))); + +interface SchemaWriterSelection { + readonly stackId: string; + readonly instanceRoot: string; + readonly service: SchemaWriterService; + readonly version: string; +} + +/** Fails when the record already holds a newer `service` than `version`, without touching it. */ +export const checkSchemaWriter = Effect.fn("SchemaWriters.check")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + selection: SchemaWriterSelection, +) { + const writers = yield* readSchemaWriters(fs, path, selection.instanceRoot); + const downgrade = schemaWriterDowngrade( + selection.stackId, + writers, + selection.service, + selection.version, + ); + yield* Effect.annotateCurrentSpan({ + "schema_writers.service": selection.service, + "schema_writers.outcome": downgrade === undefined ? "allowed" : "downgrade", + }); + if (downgrade !== undefined) return yield* downgrade; +}); + +/** + * Holds `service`'s lease for the enclosing scope, then raises the record to `version` and fsyncs + * it; fails with {@link SchemaWriterDowngrade} when the record is newer. Callers release the lease + * once the migration completes, and a crashed holder's lease dies with its process. + */ +export const holdSchemaWriter = Effect.fn("SchemaWriters.hold")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + selection: SchemaWriterSelection, +) { + yield* holdLock( + path.join(selection.instanceRoot, leaseFile(selection.service)), + leaseBudget, + `The ${selection.service} schema writer`, + ); + const outcome = yield* criticalSection( + path, + selection.instanceRoot, + Effect.gen(function* () { + const writers = yield* readSchemaWriters(fs, path, selection.instanceRoot); + const downgrade = schemaWriterDowngrade( + selection.stackId, + writers, + selection.service, + selection.version, + ); + if (downgrade !== undefined) return yield* downgrade; + const recorded = writers[selection.service]; + if (recorded !== undefined && compareUpstreamVersions(selection.version, recorded) === 0) + return "unchanged" as const; + yield* writeSchemaWriters(fs, path, selection.instanceRoot, { + ...writers, + [selection.service]: selection.version, + }); + return "raised" as const; + }), + ).pipe( + Effect.tapError((error) => + Effect.annotateCurrentSpan({ + "schema_writers.outcome": error._tag === "SchemaWriterDowngrade" ? "downgrade" : "error", + }), + ), + ); + yield* Effect.annotateCurrentSpan({ + "schema_writers.service": selection.service, + "schema_writers.outcome": outcome, + }); +}); + +const newest = (current: string | undefined, other: string | undefined) => + current === undefined || (other !== undefined && compareUpstreamVersions(other, current) > 0) + ? other + : current; + +/** + * Restore step one: raises every service in the record to at least `snapshot`'s version and + * fsyncs it, so the record stays at least as new as the data about to be published. + */ +export const raiseSchemaWritersTo = Effect.fn("SchemaWriters.raiseTo")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, + snapshot: SchemaWriters, +) { + yield* criticalSection( + path, + instanceRoot, + Effect.gen(function* () { + const current = yield* readSchemaWriters(fs, path, instanceRoot); + const raised: { -readonly [K in SchemaWriterService]?: string } = {}; + for (const service of schemaWriterServices) { + const version = newest(current[service], snapshot[service]); + if (version !== undefined) raised[service] = version; + } + yield* writeSchemaWriters(fs, path, instanceRoot, raised); + }), + ); +}); + +/** Restore step three: sets the record to the snapshot's, now that its data is published. */ +export const setSchemaWriters = Effect.fn("SchemaWriters.set")(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + instanceRoot: string, + snapshot: SchemaWriters, +) { + yield* criticalSection(path, instanceRoot, writeSchemaWriters(fs, path, instanceRoot, snapshot)); +}); + +/** Encodes a record for a snapshot entry. */ +export const encodeSchemaWriters = (writers: SchemaWriters) => + Schema.encodeEffect(WritersJson)(writers).pipe(Effect.mapError(recordError)); + +/** Decodes the record stored in a snapshot entry. */ +export const decodeSchemaWriters = (text: string) => + Schema.decodeEffect(WritersJson)(text).pipe(Effect.mapError(recordError)); diff --git a/packages/stack/src/internal/schema-writers.unit.test.ts b/packages/stack/src/internal/schema-writers.unit.test.ts new file mode 100644 index 0000000000..92db6fc811 --- /dev/null +++ b/packages/stack/src/internal/schema-writers.unit.test.ts @@ -0,0 +1,61 @@ +import { expect, it } from "@effect/vitest"; +import { + boundDatabaseId, + compareUpstreamVersions, + schemaWriterDowngrade, +} from "./schema-writers.ts"; + +it("orders each catalog version format by its leading dotted number", () => { + const ascending = [ + ["v2.196.0", "v2.197.0"], + ["v2.9.0", "v2.10.0"], + ["2026.10.05-sha-aaaaaaa", "2026.10.12-sha-bbbbbbb"], + ["17.6.1.054", "17.11.0.004"], + ["0.58.0", "v0.100.0"], + ["v1.30", "v1.30.1"], + ["unversioned", "v0.0.1"], + ] as const; + for (const [older, newer] of ascending) { + expect(compareUpstreamVersions(older, newer), `${older} < ${newer}`).toBe(-1); + expect(compareUpstreamVersions(newer, older), `${newer} > ${older}`).toBe(1); + } +}); + +it("treats versions that differ only by suffix or a v prefix as the same", () => { + expect(compareUpstreamVersions("2026.10.12-sha-aaaaaaa", "2026.10.12-sha-bbbbbbb")).toBe(0); + expect(compareUpstreamVersions("v2.197.0", "2.197.0")).toBe(0); + expect(compareUpstreamVersions("v1.30", "v1.30.0")).toBe(0); + expect(compareUpstreamVersions("unversioned", "other")).toBe(0); +}); + +it("offers to keep the data by pinning the recorded version before recreating the stack", () => { + expect(schemaWriterDowngrade("stack-a", { auth: "v2.197.0" }, "auth", "v2.196.1")?.message).toBe( + 'auth v2.196.1 is older than v2.197.0, which already migrated this database; downgrading does not restore the old schema, so keep this data by pinning `[local.services.auth] version = "v2.197.0"` in supabase/config.toml, or run `supabase stack destroy --stack-id stack-a` to recreate the stack — this permanently deletes its local database data', + ); +}); + +const dependencies = [ + { from: "db", to: "auth", bindings: [{ output: "authDatabaseUrl", input: "databaseUrl" }] }, + { from: "db", to: "studio", bindings: [{ output: "databaseUrl", input: "databaseUrl" }] }, + { from: "mail", to: "auth", bindings: [{ output: "smtpUrl", input: "smtpUrl" }] }, + { from: "db", to: "functions" }, +]; +const services: Record<string, string> = { + db: "database", + mail: "mail", + auth: "auth", + studio: "studio", + functions: "functions", +}; +const serviceOf = (id: string) => services[id]; + +it("resolves the database an instance takes its databaseUrl from", () => { + expect(boundDatabaseId(dependencies, serviceOf, "auth")).toBe("db"); + expect(boundDatabaseId(dependencies, serviceOf, "studio")).toBe("db"); +}); + +it("leaves an instance unbound when no database feeds its databaseUrl", () => { + expect(boundDatabaseId(dependencies, serviceOf, "functions")).toBeUndefined(); + expect(boundDatabaseId(dependencies, serviceOf, "storage")).toBeUndefined(); + expect(boundDatabaseId([], serviceOf, "auth")).toBeUndefined(); +}); diff --git a/packages/stack/src/lifetime.integration.test.ts b/packages/stack/src/lifetime.integration.test.ts index 2c95cc6fa5..d08ac58a10 100644 --- a/packages/stack/src/lifetime.integration.test.ts +++ b/packages/stack/src/lifetime.integration.test.ts @@ -14,6 +14,7 @@ import { } from "../tests/owner.ts"; import { watchEntry } from "../tests/watch-entry.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; class LifetimeTestError extends Data.TaggedError("LifetimeTestError")<{ readonly message: string; @@ -231,6 +232,7 @@ it.live.skipIf(process.platform === "win32")( const client = yield* ownerClient(access); const mail = yield* client.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); @@ -282,6 +284,7 @@ it.live.skipIf(process.platform === "win32")( const reclaimedClient = yield* ownerClient(reclaimedAccess); const reclaimedMail = yield* reclaimedClient.createService({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port } }, }); diff --git a/packages/stack/src/namespace/Lock.ts b/packages/stack/src/namespace/Lock.ts new file mode 100644 index 0000000000..db4a11d52e --- /dev/null +++ b/packages/stack/src/namespace/Lock.ts @@ -0,0 +1,49 @@ +import { Effect, Exit, Schedule, Schema, Scope } from "effect"; +import type { Duration } from "effect"; +import { namespaceError, type NamespaceError } from "./Capabilities.ts"; +import { acquireLock, isBusy, takeLock } from "./drivers/Sqlite.ts"; + +class LockBusy extends Schema.TaggedError<LockBusy>()("Namespace.LockBusy", { + message: Schema.String, +}) {} + +/** Whether `error` is a {@link holdLock} failure because another holder kept the lock for the whole budget. */ +export const isLockBusy = (error: NamespaceError): boolean => Schema.is(LockBusy)(error.cause); + +/** + * Takes the exclusive lock at `file` for the enclosing scope, waiting up to `budget` for another + * holder. Each attempt owns a child scope, so a failed open or lock never leaks the connection. + */ +export const holdLock = ( + file: string, + budget: Duration.Input, + subject: string, +): Effect.Effect<void, NamespaceError, Scope.Scope> => + Effect.gen(function* () { + const held = yield* Effect.scope; + yield* Effect.uninterruptible( + Effect.gen(function* () { + const attempt = yield* Scope.fork(held, "sequential"); + yield* acquireLock(file, "create").pipe( + Scope.provide(attempt), + Effect.flatMap(takeLock), + Effect.onExit((exit) => + Exit.isFailure(exit) ? Scope.close(attempt, exit) : Effect.void, + ), + ); + }), + ).pipe( + Effect.retry({ + schedule: Schedule.spaced("50 millis").pipe(Schedule.upTo({ duration: budget })), + while: isBusy, + }), + Effect.catchIf(isBusy, () => + Effect.fail( + namespaceError( + "lock", + new LockBusy({ message: `${subject} is locked by another operation; retry shortly` }), + ), + ), + ), + ); + }); diff --git a/packages/stack/src/namespace/Namespace.e2e.test.ts b/packages/stack/src/namespace/Namespace.e2e.test.ts index 9f3e7e20c4..35359f90ae 100644 --- a/packages/stack/src/namespace/Namespace.e2e.test.ts +++ b/packages/stack/src/namespace/Namespace.e2e.test.ts @@ -26,6 +26,7 @@ import { create, open } from "../effect.ts"; import * as StackNamespace from "../StackNamespace.ts"; import { captureOwnerPid, watchLeaseRelease } from "../../tests/owner.ts"; import { testEngine } from "../../tests/test-engine.ts"; +import { selection } from "../../tests/pins.ts"; class PublishBarrierError extends Data.TaggedError("PublishBarrierError")<{ readonly message: string; @@ -358,7 +359,11 @@ it.live.skipIf(process.platform === "win32" || process.platform === "darwin")( startOwner: true, }); yield* Effect.addFinalizer(() => stack.destroy.pipe(Effect.ignore)); - const mail = yield* stack.services.create({ service: "mail", config: {} }); + const mail = yield* stack.services.create({ + service: "mail", + pin: selection.mail, + config: {}, + }); yield* mail.start; yield* mail.ready; diff --git a/packages/stack/src/namespace/Namespace.integration.test.ts b/packages/stack/src/namespace/Namespace.integration.test.ts index 69c267ab77..04a22f6fd3 100644 --- a/packages/stack/src/namespace/Namespace.integration.test.ts +++ b/packages/stack/src/namespace/Namespace.integration.test.ts @@ -24,6 +24,7 @@ import { destroyTestStack } from "../../tests/stack-cleanup.ts"; import { errorCode } from "../internal/sharing-violation.ts"; import * as StackNamespace from "../StackNamespace.ts"; import * as Paths from "./Paths.ts"; +import { selection } from "../../tests/pins.ts"; const makeTestState = (root: string) => Layer.build(StackNamespace.layer({ root })).pipe( @@ -68,7 +69,7 @@ describe("durable registry", () => { ...current, instances: [ ...current.instances, - { id, creation: { service: "mail", config: {} } }, + { id, creation: { service: "mail", pin: selection.mail, config: {} } }, ], }); }), @@ -284,7 +285,7 @@ describe("durable registry", () => { }), ).pipe(Effect.map((context) => Context.get(context, StackNamespace.Service))); yield* store.save(initial); - const mail = { service: "mail", config: {} }; + const mail = { service: "mail", pin: selection.mail, config: {} }; const invalid = { "bad-composition": { composition: { members: "none", dependencies: [] } }, "bad-creation": { instances: [{ id: "one", creation: { service: "unknown" } }] }, @@ -501,6 +502,7 @@ describe("registry lock", () => { const result = yield* Fiber.join(waiter); expect(result.operation).toBe("lock"); expect(result.message).toBe("Stack registry is locked by another operation; retry shortly"); + expect(StackNamespace.isLockBusy(result)).toBe(true); expect(yield* second.read(initial.id)).toBeUndefined(); yield* Fiber.interrupt(owner); yield* second.withLock(second.save(initial)); diff --git a/packages/stack/src/namespace/Publication.ts b/packages/stack/src/namespace/Publication.ts index 6b3c2558ec..2fd584cf71 100644 --- a/packages/stack/src/namespace/Publication.ts +++ b/packages/stack/src/namespace/Publication.ts @@ -6,6 +6,8 @@ import * as Drivers from "./drivers/FileSystem.ts"; export interface PublishOptions { readonly target: string; readonly content: string; + /** File mode of the published document; private (`0o600`) unless set. */ + readonly mode?: number; readonly platform?: NodeJS.Platform; } @@ -34,7 +36,7 @@ export const publish = Effect.fn("Namespace.publish")(function* ( Effect.succeed(staging), () => Effect.gen(function* () { - yield* fs.writeFileString(staging, options.content, { mode: 0o600 }); + yield* fs.writeFileString(staging, options.content, { mode: options.mode ?? 0o600 }); yield* Drivers.fsyncFile(fs, staging); yield* retry(fs.rename(staging, options.target)); yield* Drivers.fsyncDirectory(fs, directory); diff --git a/packages/stack/src/namespace/Registry.ts b/packages/stack/src/namespace/Registry.ts index 38aa0b5768..616ec0b7b2 100644 --- a/packages/stack/src/namespace/Registry.ts +++ b/packages/stack/src/namespace/Registry.ts @@ -1,4 +1,4 @@ -import { Clock, Effect, Exit, FileSystem, Option, Path, Schedule, Schema, Scope } from "effect"; +import { Clock, Effect, FileSystem, Option, Path, Schema, Scope } from "effect"; import { CompositionConfig } from "../Orchestrator.ts"; import { ServiceCreation } from "../services/Catalog.ts"; import { SafeId } from "../identity/SafeId.ts"; @@ -8,7 +8,7 @@ import { OWNER_FILE, OWNER_LOG_FILE } from "./Lease.ts"; import { stackLogsRoot } from "./Paths.ts"; import * as Publication from "./Publication.ts"; import { removeEmptyDirectory } from "./drivers/FileSystem.ts"; -import { acquireLock, isBusy, takeLock } from "./drivers/Sqlite.ts"; +import { holdLock } from "./Lock.ts"; /** How long a destroyed stack's empty directories stay, so a same-id restart finds them. */ const DESTROYED_RETENTION_MILLIS = 10 * 60 * 1000; @@ -205,28 +205,6 @@ export const make = ( const modified = Option.getOrUndefined((yield* fs.stat(directory)).mtime)?.getTime(); return modified !== undefined && now - modified >= DESTROYED_RETENTION_MILLIS; }); - /** - * One attempt: forks `attemptScope` under `guardScope` before opening anything, so there is - * always a scope to close if the open or the `BEGIN IMMEDIATE` below fails or defects. Runs - * uninterruptibly as a single unit; the retry waiting on busy, and `effect` once this - * succeeds, both stay interruptible around it. - */ - const acquireAttempt = ( - guardScope: Scope.Scope, - ): Effect.Effect<Scope.Closeable, NamespaceError> => - Effect.uninterruptible( - Effect.gen(function* () { - const attemptScope = yield* Scope.fork(guardScope, "sequential"); - yield* acquireLock(lock, "create").pipe( - Scope.provide(attemptScope), - Effect.flatMap(takeLock), - Effect.onExit((exit) => - Exit.isFailure(exit) ? Scope.close(attemptScope, exit) : Effect.void, - ), - ); - return attemptScope; - }), - ); /** * Holds the lock in its own standalone scope, so `effect`'s ambient scope (for example a port * listener it binds) stays whatever the caller already had, not one this lock closes early. @@ -236,19 +214,8 @@ export const make = ( Effect.acquireUseRelease( Scope.make(), (guardScope) => - acquireAttempt(guardScope).pipe( - Effect.retry({ - schedule: Schedule.spaced("50 millis").pipe( - Schedule.upTo({ duration: "5 seconds" }), - ), - while: isBusy, - }), - Effect.catchIf(isBusy, () => - namespaceError( - "lock", - "Stack registry is locked by another operation; retry shortly", - ), - ), + holdLock(lock, "5 seconds", "Stack registry").pipe( + Scope.provide(guardScope), Effect.andThen(effect), ), (guardScope, exit) => Scope.close(guardScope, exit), diff --git a/packages/stack/src/port-reservation.e2e.test.ts b/packages/stack/src/port-reservation.e2e.test.ts index 1d55cd9476..0b8fea5836 100644 --- a/packages/stack/src/port-reservation.e2e.test.ts +++ b/packages/stack/src/port-reservation.e2e.test.ts @@ -9,6 +9,7 @@ import { captureOwnerPid } from "../tests/owner.ts"; import { destroyTestStack } from "../tests/stack-cleanup.ts"; import { testEngine } from "../tests/test-engine.ts"; import type { ContainerEngine } from "./runtime/Container.ts"; +import { selection } from "../tests/pins.ts"; /** * The per-user port reservation registry: a stopped stack keeps its automatic ports across state @@ -21,7 +22,12 @@ const run = <A, E, R>(effect: Effect.Effect<A, E, R>) => Effect.scoped(effect).pipe(Effect.provide(servicesLayer)); const mailOn = (port: number | "auto") => - ({ service: "mail" as const, config: {}, endpoints: { http: { port } } }) as const; + ({ + service: "mail" as const, + pin: selection.mail, + config: {}, + endpoints: { http: { port } }, + }) as const; const portOf = (observation: Observation): number => { const endpoint = observation.endpoints.find((candidate) => candidate.name === "http"); diff --git a/packages/stack/src/preparation/PrepareSelection.ts b/packages/stack/src/preparation/PrepareSelection.ts new file mode 100644 index 0000000000..126036bf5d --- /dev/null +++ b/packages/stack/src/preparation/PrepareSelection.ts @@ -0,0 +1,109 @@ +import { Crypto, Effect, FileSystem, Path } from "effect"; +import type { HttpClient } from "effect/http"; +import { ChildProcessSpawner } from "effect/process"; +import { + artifactServiceKinds, + type ArtifactError, + prepareNativeArtifact, + slimImageMirrors, + type ArtifactPin, + type ArtifactSelection, + type ServiceKind, +} from "../Artifacts.ts"; +import { stackError } from "../Rpc.ts"; +import { + makeContainerRuntime, + resolveEngineTarget, + type ContainerEngine, + type ContainerError, +} from "../runtime/Container.ts"; + +/** One selected service's artifact and whether this call had to fetch it. */ +export interface PreparedRelease { + readonly service: ServiceKind; + /** The published release, `<upstream>-r<revision>`. */ + readonly release: string; + readonly outcome: "downloaded" | "cached"; +} + +const PREPARE_CONCURRENCY = 4; + +type Preparer = ( + service: ServiceKind, + pin: ArtifactPin, +) => Effect.Effect< + PreparedRelease["outcome"], + ArtifactError | ContainerError, + | FileSystem.FileSystem + | Path.Path + | Crypto.Crypto + | HttpClient.HttpClient + | ChildProcessSpawner.ChildProcessSpawner +>; + +const releaseOf = (pin: ArtifactPin): string => `${pin.upstreamVersion}-r${pin.revision}`; + +const imagePreparer = Effect.fn("Artifacts.imagePreparer")(function* ( + engine: ContainerEngine, + cacheRoot: string, +) { + const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; + const target = yield* resolveEngineTarget(spawner, engine); + const runtime = yield* makeContainerRuntime({ + target, + root: cacheRoot, + imageMirrors: slimImageMirrors, + }); + return (_service: ServiceKind, pin: ArtifactPin) => + runtime + .ensureImage(pin.image) + .pipe( + Effect.map(({ pulled }): PreparedRelease["outcome"] => (pulled ? "downloaded" : "cached")), + ); +}); + +/** + * Fetches every artifact of `selection` that `runtime` runs, without creating or starting a stack: + * native archives land in the artifact store under `cacheRoot` with their pinned digests verified, + * and container images are pulled by digest through the selected engine. Artifacts already present + * are left alone. + */ +export const prepareArtifactSelection = Effect.fn("Artifacts.prepareSelection")( + function* ( + selection: ArtifactSelection, + options: { + readonly runtime: "native" | ContainerEngine; + readonly cacheRoot: string; + readonly platform?: { readonly os: string; readonly arch: string }; + }, + ) { + const prepareOne: Preparer = + options.runtime === "native" + ? (service: ServiceKind, pin: ArtifactPin) => + prepareNativeArtifact({ service, pin }, options.cacheRoot, options.platform).pipe( + Effect.map((artifact) => artifact.outcome), + ) + : yield* imagePreparer(options.runtime, options.cacheRoot); + const releases = yield* Effect.forEach( + artifactServiceKinds(), + (service) => + prepareOne(service, selection[service]).pipe( + Effect.map((outcome): PreparedRelease => ({ + service, + release: releaseOf(selection[service]), + outcome, + })), + ), + { concurrency: PREPARE_CONCURRENCY }, + ); + yield* Effect.annotateCurrentSpan({ + "artifact.runtime": options.runtime, + "artifact.downloaded_count": releases.filter(({ outcome }) => outcome === "downloaded") + .length, + "artifact.cached_count": releases.filter(({ outcome }) => outcome === "cached").length, + }); + return releases; + }, + Effect.scoped, + Effect.mapError((cause) => stackError("install", cause)), +); diff --git a/packages/stack/src/preparation/ReleaseResolver.integration.test.ts b/packages/stack/src/preparation/ReleaseResolver.integration.test.ts new file mode 100644 index 0000000000..d587c76d3c --- /dev/null +++ b/packages/stack/src/preparation/ReleaseResolver.integration.test.ts @@ -0,0 +1,462 @@ +import { NodeHttpClient, NodeServices } from "@effect/platform-node"; +import { describe, expect, it } from "@effect/vitest"; +import { ConfigProvider, Crypto, Effect, Layer, Schema } from "effect"; +// oxlint-disable-next-line effecttsgo/node-builtin-import -- the fixture owns a local native listener. +import { createServer, type Server } from "node:http"; +// oxlint-disable-next-line effecttsgo/node-builtin-import -- the fixture owns a local native listener. +import { createServer as createTcpServer, type Server as NetServer } from "node:net"; +import type { ArtifactKind } from "../Artifacts.ts"; +import { digestHex } from "./Integrity.ts"; +import { resolvePin } from "./ReleaseResolver.ts"; + +const toJson = Schema.encodeSync(Schema.fromJsonString(Schema.Json)); + +const TARGETS = ["darwin-arm64", "linux-amd64", "linux-arm64"] as const; + +interface Served { + readonly body: string; + readonly headers?: Record<string, string>; +} + +interface Request { + readonly path: string; + readonly authorization: string | undefined; + readonly accept: string | undefined; +} + +interface Fixture { + readonly origin: string; + readonly routes: Map<string, Served>; + /** Paths answered with a bare status instead of their route. */ + readonly failing: Map<string, number>; + readonly requests: Array<Request>; +} + +/** A release host and registry on one OS-assigned loopback port: `/releases/…`, `/token`, `/v2/…`. */ +const startFixture = Effect.acquireRelease( + Effect.callback<{ server: Server; fixture: Fixture }>((resume) => { + const routes = new Map<string, Served>([["/token", { body: toJson({ token: "anon" }) }]]); + const failing = new Map<string, number>(); + const requests: Array<Request> = []; + const server = createServer((request, response) => { + const path = new URL(request.url ?? "/", "http://fixture").pathname; + requests.push({ + path, + authorization: request.headers.authorization, + accept: request.headers.accept, + }); + const status = failing.get(path); + const served = routes.get(path); + if (status !== undefined) response.statusCode = status; + else if (path.startsWith("/v2/") && request.headers.authorization !== "Bearer anon") + response.statusCode = 401; + else if (served === undefined) response.statusCode = 404; + else + for (const [name, value] of Object.entries(served.headers ?? {})) + response.setHeader(name, value); + response.end(status === undefined ? (served?.body ?? "") : ""); + }); + server.listen(0, "127.0.0.1", () => { + const address = server.address(); + if (address === null || typeof address === "string") return; + resume( + Effect.succeed({ + server, + fixture: { origin: `http://127.0.0.1:${address.port}`, routes, failing, requests }, + }), + ); + }); + }), + ({ server }) => + Effect.callback<void>((resume) => { + server.close(() => resume(Effect.void)); + server.closeAllConnections(); + }), +).pipe(Effect.map(({ fixture }) => fixture)); + +/** A loopback TCP listener that accepts every connection and resets it before any HTTP reply. */ +const startResettingListener = Effect.acquireRelease( + Effect.callback<{ origin: string; server: NetServer }>((resume) => { + const server = createTcpServer((socket) => socket.resetAndDestroy()); + server.listen(0, "127.0.0.1", () => { + const address = server.address(); + if (address === null || typeof address === "string") return; + resume( + Effect.succeed({ + origin: `http://127.0.0.1:${address.port}`, + server, + }), + ); + }); + }), + ({ server }) => Effect.callback<void>((resume) => void server.close(() => resume(Effect.void))), +).pipe(Effect.map(({ origin }) => origin)); + +const sha256Of = Effect.fn(function* (text: string) { + const crypto = yield* Crypto.Crypto; + return digestHex(yield* crypto.digest("SHA-256", new TextEncoder().encode(text))); +}); + +interface Publication { + readonly producer: string; + readonly upstream: string; + readonly revision: number; + /** Serves different manifest bytes than the ones SHA256SUMS lists. */ + readonly tamperedManifest?: boolean; + readonly omitSumsEntryFor?: string; + /** Lists a digest that is not 64 lowercase hexadecimal characters. */ + readonly malformedDigest?: boolean; + /** The manifest asset of this stem is listed in SHA256SUMS but answers 404. */ + readonly missingManifestAsset?: string; + readonly omitProvenanceFromSums?: boolean; + /** Serves different provenance bytes than the ones SHA256SUMS lists. */ + readonly tamperedProvenance?: boolean; + readonly omitImage?: boolean; + /** Registry `Docker-Content-Digest` that differs from the manifest body. */ + readonly imageDigestHeader?: string; + readonly provenanceSource?: string; +} + +/** Publishes one release to the fixture the way slim-services lays it out, and returns its content. */ +const publish = Effect.fn(function* (fixture: Fixture, release: Publication) { + const releaseVersion = `${release.upstream}-r${release.revision}`; + const tag = `${release.producer}-${releaseVersion}`; + const sums: Array<string> = []; + const natives: Record<string, { archive: string; manifest: string }> = {}; + for (const target of TARGETS) { + const stem = `${tag}-${target}`; + const image = `docker.io/upstream/${release.producer}:${release.upstream}@sha256:${"a".repeat(64)}`; + const manifest = toJson({ + service: release.producer, + version: releaseVersion, + target, + ...(target === "darwin-arm64" ? { upstream_image: image } : { source_image: image }), + }); + const archive = `archive ${stem}`; + natives[target] = { archive: yield* sha256Of(archive), manifest: yield* sha256Of(manifest) }; + if (release.omitSumsEntryFor !== stem) { + sums.push(`${natives[target].archive} ${stem}.tar.zst`); + sums.push(`${natives[target].manifest} ${stem}.manifest.json`); + } + if (release.missingManifestAsset !== stem) + fixture.routes.set(`/releases/${tag}/${stem}.manifest.json`, { + body: release.tamperedManifest === true ? `${manifest} ` : manifest, + }); + } + if (release.provenanceSource !== undefined) { + const provenanceName = `${release.producer}-${release.upstream}.oci-provenance.json`; + const provenance = toJson({ source: release.provenanceSource }); + if (release.omitProvenanceFromSums !== true) + sums.push(`${yield* sha256Of(provenance)} ${provenanceName}`); + fixture.routes.set(`/releases/${tag}/${provenanceName}`, { + body: release.tamperedProvenance === true ? `${provenance} ` : provenance, + }); + } + if (release.malformedDigest === true) + sums.push(`${"g".repeat(64)} ${release.producer}-bad.tar.zst`); + fixture.routes.set(`/releases/${tag}/SHA256SUMS`, { body: `${sums.join("\n")}\n` }); + const imageManifest = toJson({ schemaVersion: 2, tag }); + const digest = `sha256:${yield* sha256Of(imageManifest)}`; + if (release.omitImage !== true) + fixture.routes.set(`/v2/supabase/cli/${release.producer}/manifests/${releaseVersion}`, { + body: imageManifest, + headers: { "docker-content-digest": release.imageDigestHeader ?? digest }, + }); + return { natives, digest }; +}); + +/** Runs `effect` with the fixture standing in for GitHub releases and GHCR, through the env override. */ +const against = <A, E, R>(fixture: Fixture, effect: Effect.Effect<A, E, R>) => + effect.pipe( + Effect.provide( + Layer.mergeAll( + NodeServices.layer, + NodeHttpClient.layerNodeHttp, + ConfigProvider.layer( + ConfigProvider.fromEnv({ + env: { + SUPABASE_INTERNAL_SLIM_RELEASES_URL: `${fixture.origin}/releases/`, + SUPABASE_INTERNAL_SLIM_REGISTRY_URL: fixture.origin, + }, + }), + ), + ), + ), + ); + +const sumsRequests = (fixture: Fixture): ReadonlyArray<string> => + fixture.requests.map(({ path }) => path).filter((path) => path.endsWith("/SHA256SUMS")); + +describe("release resolver", () => { + it.live("pins the highest published revision with the content of that release", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + yield* publish(fixture, { producer: "postgrest", upstream: "v99.0.0", revision: 0 }); + yield* publish(fixture, { producer: "postgrest", upstream: "v99.0.0", revision: 1 }); + const latest = yield* publish(fixture, { + producer: "postgrest", + upstream: "v99.0.0", + revision: 2, + }); + + const pin = yield* against(fixture, resolvePin({ service: "rest", upstream: "v99.0.0" })); + + expect(pin).toEqual({ + upstreamVersion: "v99.0.0", + revision: 2, + image: `ghcr.io/supabase/cli/postgrest:v99.0.0-r2@${latest.digest}`, + upstreamImage: "upstream/postgrest:v99.0.0", + natives: latest.natives, + }); + expect(sumsRequests(fixture)).toEqual( + [0, 1, 2, 3].map((revision) => `/releases/postgrest-v99.0.0-r${revision}/SHA256SUMS`), + ); + const manifestRequest = fixture.requests.find(({ path }) => path.includes("/manifests/")); + expect(manifestRequest?.authorization).toBe("Bearer anon"); + expect(manifestRequest?.accept).toContain("application/vnd.oci.image.index.v1+json"); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live("pins exactly the requested revision, not the highest", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + const first = yield* publish(fixture, { + producer: "auth", + upstream: "v2.1.0", + revision: 0, + }); + yield* publish(fixture, { producer: "auth", upstream: "v2.1.0", revision: 1 }); + + const pin = yield* against( + fixture, + resolvePin({ service: "auth", upstream: "v2.1.0", revision: 0 }), + ); + + expect(pin.revision).toBe(0); + expect(pin.natives).toEqual(first.natives); + expect(pin.image).toBe(`ghcr.io/supabase/cli/auth:v2.1.0-r0@${first.digest}`); + expect(sumsRequests(fixture)).toEqual(["/releases/auth-v2.1.0-r0/SHA256SUMS"]); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live("reads a mirrored service's upstream image from its provenance file", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + yield* publish(fixture, { + producer: "mailpit", + upstream: "v1.31.0", + revision: 0, + provenanceSource: "docker.io/axllent/mailpit:v1.31.0@sha256:" + "b".repeat(64), + }); + + const pin = yield* against(fixture, resolvePin({ service: "mail", upstream: "v1.31.0" })); + + expect(pin.upstreamImage).toBe("axllent/mailpit:v1.31.0"); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live("reports a release that is not published as unpublished", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + yield* publish(fixture, { producer: "storage", upstream: "v1.0.0", revision: 0 }); + + const none = yield* against( + fixture, + resolvePin({ service: "storage", upstream: "v9.9.9" }).pipe(Effect.flip), + ); + const missingRevision = yield* against( + fixture, + resolvePin({ service: "storage", upstream: "v1.0.0", revision: 3 }).pipe(Effect.flip), + ); + + expect(none).toMatchObject({ + _tag: "ReleaseUnpublished", + service: "storage", + upstream: "v9.9.9", + url: `${fixture.origin}/releases/storage-v9.9.9-r0/SHA256SUMS`, + }); + expect(missingRevision).toMatchObject({ + _tag: "ReleaseUnpublished", + upstream: "v1.0.0", + revision: 3, + url: `${fixture.origin}/releases/storage-v1.0.0-r3/SHA256SUMS`, + }); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live.each([ + { status: 500, path: "/releases/realtime-v2.0.0-r0/SHA256SUMS", revision: undefined }, + { status: 429, path: "/releases/realtime-v2.0.0-r0/SHA256SUMS", revision: undefined }, + { status: 403, path: "/releases/realtime-v2.0.0-r0/SHA256SUMS", revision: undefined }, + // Registry outage after a healthy release is transient as well. + { status: 503, path: "/token", revision: 0 }, + ])("treats HTTP $status from $path as transient, never as unpublished", (row) => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + yield* publish(fixture, { producer: "realtime", upstream: "v2.0.0", revision: 0 }); + fixture.failing.set(row.path, row.status); + + const error = yield* against( + fixture, + resolvePin({ + service: "realtime", + upstream: "v2.0.0", + ...(row.revision === undefined ? {} : { revision: row.revision }), + }).pipe(Effect.flip), + ); + + expect(error).toMatchObject({ + _tag: "ReleaseTransient", + status: row.status, + url: expect.stringContaining(row.path), + }); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live("reports a connection that is reset as transient", () => + Effect.scoped( + Effect.gen(function* () { + const resetting = yield* startResettingListener; + + const error = yield* against( + { origin: resetting, routes: new Map(), failing: new Map(), requests: [] }, + resolvePin({ service: "realtime", upstream: "v2.0.0" }).pipe(Effect.flip), + ); + + expect(error).toMatchObject({ + _tag: "ReleaseTransient", + url: expect.stringContaining(resetting), + }); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + it.live.each([ + { name: "r0 and r1 published, r2 absent", published: [0, 1], expected: 1 }, + { name: "a gap at r1 ends the sequence at r0", published: [0, 2], expected: 0 }, + ])("pins the last revision before the first 404 when $name", (row) => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + for (const revision of row.published) + yield* publish(fixture, { producer: "pgmeta", upstream: "v1.0.0", revision }); + + const pin = yield* against(fixture, resolvePin({ service: "pgmeta", upstream: "v1.0.0" })); + + expect(pin.revision).toBe(row.expected); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + + const mirrored = { + provenanceSource: `docker.io/axllent/mailpit:v1.0.0@sha256:${"b".repeat(64)}`, + }; + + it.live.each<{ + readonly name: string; + readonly service: ArtifactKind; + readonly producer: string; + readonly release: Partial<Publication>; + readonly message: RegExp; + readonly digests?: "expected-hash" | "registry-header"; + }>([ + { + name: "manifest hash mismatch", + service: "auth", + producer: "auth", + release: { tamperedManifest: true }, + message: /does not match SHA256SUMS/u, + digests: "expected-hash", + }, + { + name: "malformed SHA256SUMS digest", + service: "auth", + producer: "auth", + release: { malformedDigest: true }, + message: /SHA256SUMS contains a malformed line/u, + }, + { + name: "target missing from SHA256SUMS", + service: "pgmeta", + producer: "pgmeta", + release: { omitSumsEntryFor: "pgmeta-v1.0.0-r0-linux-arm64" }, + message: /no entry for pgmeta-v1.0.0-r0-linux-arm64.tar.zst/u, + }, + { + name: "manifest asset listed in SHA256SUMS but absent from the release", + service: "storage", + producer: "storage", + release: { missingManifestAsset: "storage-v1.0.0-r0-linux-amd64" }, + message: /no storage-v1.0.0-r0-linux-amd64.manifest.json/u, + }, + { + name: "registry digest header differs from the manifest body", + service: "studio", + producer: "studio", + release: { imageDigestHeader: `sha256:${"c".repeat(64)}` }, + message: /Registry digest does not match/u, + digests: "registry-header", + }, + { + name: "slim image absent from the registry", + service: "analytics", + producer: "analytics", + release: { omitImage: true }, + message: /has no image supabase\/cli\/analytics:v1.0.0-r0/u, + }, + { + name: "provenance file hash mismatch", + service: "mail", + producer: "mailpit", + release: { ...mirrored, tamperedProvenance: true }, + message: /oci-provenance does not match SHA256SUMS/u, + digests: "expected-hash", + }, + { + name: "provenance file missing from SHA256SUMS", + service: "mail", + producer: "mailpit", + release: { ...mirrored, omitProvenanceFromSums: true }, + message: /no entry for mailpit-v1.0.0.oci-provenance.json/u, + }, + ])("rejects a release with $name", (row) => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* startFixture; + yield* publish(fixture, { + producer: row.producer, + upstream: "v1.0.0", + revision: 0, + ...row.release, + }); + + const error = yield* against( + fixture, + resolvePin({ service: row.service, upstream: "v1.0.0", revision: 0 }).pipe(Effect.flip), + ); + + expect(error).toMatchObject({ + _tag: "ReleaseIntegrity", + service: row.service, + upstream: "v1.0.0", + revision: 0, + message: expect.stringMatching(row.message), + }); + if (row.digests === "expected-hash") + expect(error).toMatchObject({ expected: expect.stringMatching(/^[0-9a-f]{64}$/u) }); + if (row.digests === "registry-header") + expect(error).toMatchObject({ expected: `sha256:${"c".repeat(64)}` }); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); +}); diff --git a/packages/stack/src/preparation/ReleaseResolver.ts b/packages/stack/src/preparation/ReleaseResolver.ts new file mode 100644 index 0000000000..bd0fe87592 --- /dev/null +++ b/packages/stack/src/preparation/ReleaseResolver.ts @@ -0,0 +1,462 @@ +import { Crypto, Effect, Option, Record, Schema } from "effect"; +import { Headers, HttpClient, HttpClientRequest } from "effect/http"; +import { + artifactSourceService, + type ArtifactKind, + type ArtifactPin, + type NativePin, +} from "../Artifacts.ts"; +import { digestHex } from "./Integrity.ts"; +import { SLIM_IMAGE_GHCR_REGISTRY, slimRegistryUrl, slimReleasesUrl } from "./SlimEndpoints.ts"; +import { + MIRROR_PRODUCERS, + UPSTREAM_IMAGE_PATTERN, + normalizeUpstreamImage, + parseSha256Sums, + type SlimNativeTarget, +} from "./SlimRelease.ts"; + +type NativeTarget = SlimNativeTarget; + +const releaseFields = { + service: Schema.String, + upstream: Schema.String, + revision: Schema.optionalKey(Schema.Int), + url: Schema.String, + message: Schema.String, +}; + +/** The requested release, or every revision of the upstream version, is not published. */ +export class ReleaseUnpublished extends Schema.TaggedError<ReleaseUnpublished>()( + "ReleaseUnpublished", + releaseFields, +) {} + +/** A release host failed in a way a later attempt may resolve: transport, rate limit, or 5xx. */ +export class ReleaseTransient extends Schema.TaggedError<ReleaseTransient>()("ReleaseTransient", { + ...releaseFields, + status: Schema.optionalKey(Schema.Int), + cause: Schema.optionalKey(Schema.Defect()), +}) {} + +/** A published release is malformed or its bytes disagree with its own checksums or digest. */ +export class ReleaseIntegrity extends Schema.TaggedError<ReleaseIntegrity>()("ReleaseIntegrity", { + ...releaseFields, + expected: Schema.optionalKey(Schema.String), + actual: Schema.optionalKey(Schema.String), + cause: Schema.optionalKey(Schema.Defect()), +}) {} + +export type ReleaseError = ReleaseUnpublished | ReleaseTransient | ReleaseIntegrity; + +export interface ResolvePinRequest { + readonly service: ArtifactKind; + /** Upstream version exactly as the release tag spells it. */ + readonly upstream: string; + /** Exact revision to pin; the highest published one when omitted. */ + readonly revision?: number; +} + +/** Revisions are allocated `0, 1, 2, …`; a host that never answers 404 must not loop forever. */ +const MAX_PROBED_REVISIONS = 1000; + +const REQUEST_TIMEOUT = "30 seconds"; + +const UPSTREAM_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]*$/u; +const MANIFEST_ACCEPT = [ + "application/vnd.oci.image.index.v1+json", + "application/vnd.oci.image.manifest.v1+json", + "application/vnd.docker.distribution.manifest.list.v2+json", + "application/vnd.docker.distribution.manifest.v2+json", +].join(", "); + +const nativeManifestSchema = Schema.Struct({ + service: Schema.String, + version: Schema.String, + target: Schema.String, + upstream_image: Schema.optionalKey(Schema.String), + source_image: Schema.optionalKey(Schema.String), +}); + +const provenanceSchema = Schema.Struct({ source: Schema.String }); +const registryTokenSchema = Schema.Struct({ token: Schema.String }); + +interface Subject { + readonly service: string; + readonly upstream: string; + readonly revision?: number; +} + +const at = (subject: Subject, revision: number): Subject => ({ ...subject, revision }); + +const fieldsOf = (subject: Subject, url: string, message: string) => ({ + service: subject.service, + upstream: subject.upstream, + ...(subject.revision === undefined ? {} : { revision: subject.revision }), + url, + message, +}); + +const integrity = ( + subject: Subject, + url: string, + message: string, + details: { + readonly expected?: string; + readonly actual?: string; + readonly cause?: unknown; + } = {}, +) => new ReleaseIntegrity({ ...fieldsOf(subject, url, message), ...details }); + +interface Fetched { + readonly status: number; + readonly bytes: Uint8Array; + readonly digestHeader: string | undefined; +} + +const decodeText = (bytes: Uint8Array): string => new TextDecoder().decode(bytes); + +/** Fetches one URL; every transport failure or timeout is `transient`, status classification is the caller's. */ +const fetchResource = Effect.fnUntraced(function* ( + subject: Subject, + url: string, + headers: Record<string, string> = {}, +) { + const client = HttpClient.followRedirects(yield* HttpClient.HttpClient); + const fetched = yield* Effect.gen(function* () { + const response = yield* client.execute( + HttpClientRequest.get(url).pipe(HttpClientRequest.setHeaders(headers)), + ); + const buffer = yield* response.arrayBuffer; + return { + status: response.status, + bytes: new Uint8Array(buffer), + digestHeader: Option.getOrUndefined(Headers.get(response.headers, "docker-content-digest")), + } satisfies Fetched; + }).pipe( + Effect.timeout(REQUEST_TIMEOUT), + Effect.mapError( + (cause) => + new ReleaseTransient({ + ...fieldsOf(subject, url, `Unable to reach ${url}`), + cause, + }), + ), + ); + return fetched; +}); + +const isOk = (status: number): boolean => status >= 200 && status < 300; + +const unexpectedStatus = (subject: Subject, url: string, status: number) => + new ReleaseTransient({ + ...fieldsOf(subject, url, `HTTP ${status} from ${url}`), + status, + }); + +const sha256Hex = Effect.fnUntraced(function* (subject: Subject, url: string, bytes: Uint8Array) { + const crypto = yield* Crypto.Crypto; + return digestHex( + yield* crypto + .digest("SHA-256", bytes) + .pipe( + Effect.mapError((cause) => + integrity(subject, url, `Unable to compute SHA-256: ${String(cause)}`), + ), + ), + ); +}); + +const decodeJson = <S extends Schema.Decoder<unknown>>( + schema: S, + subject: Subject, + url: string, + what: string, + bytes: Uint8Array, +) => + Schema.decodeEffect(Schema.fromJsonString(schema))(decodeText(bytes)).pipe( + Effect.mapError((cause) => integrity(subject, url, `${what} is malformed`, { cause })), + ); + +/** A release's `SHA256SUMS`, keyed by asset basename. */ +const parseSums = (subject: Subject, url: string, text: string) => { + const parsed = parseSha256Sums(text); + return parsed.kind === "sums" + ? Effect.succeed(parsed.sums) + : Effect.fail( + integrity(subject, url, parsed.message, { + ...(parsed.expected === undefined ? {} : { expected: parsed.expected }), + ...(parsed.actual === undefined ? {} : { actual: parsed.actual }), + }), + ); +}; + +interface Published { + readonly revision: number; + readonly url: string; + readonly sums: string; +} + +const probeRelease = Effect.fnUntraced(function* ( + subject: Subject, + releasesUrl: string, + producer: string, + revision: number, +) { + const url = `${releasesUrl}/${producer}-${subject.upstream}-r${revision}/SHA256SUMS`; + const probed = at(subject, revision); + const fetched = yield* fetchResource(probed, url); + if (fetched.status === 404) return undefined; + if (!isOk(fetched.status)) return yield* unexpectedStatus(probed, url, fetched.status); + return { revision, url, sums: decodeText(fetched.bytes) } satisfies Published; +}); + +/** The `SHA256SUMS` of the revision to pin, and how many releases were probed to find it. */ +const findRevision = Effect.fn("ReleaseResolver.findRevision")(function* ( + subject: Subject, + releasesUrl: string, + producer: string, +) { + if (subject.revision !== undefined) { + const found = yield* probeRelease(subject, releasesUrl, producer, subject.revision); + yield* Effect.annotateCurrentSpan({ "release.probes": 1 }); + if (found === undefined) + return yield* new ReleaseUnpublished( + fieldsOf( + subject, + `${releasesUrl}/${producer}-${subject.upstream}-r${subject.revision}/SHA256SUMS`, + `${producer}-${subject.upstream}-r${subject.revision} is not a published release`, + ), + ); + return found; + } + // The first 404 ends the sequence: revisions are contiguous (ADR 0026, "Immutable revisions"). + let latest: Published | undefined; + let probes = 0; + let ended = false; + while (!ended && probes < MAX_PROBED_REVISIONS) { + const found = yield* probeRelease(subject, releasesUrl, producer, probes); + probes += 1; + if (found === undefined) ended = true; + else latest = found; + } + yield* Effect.annotateCurrentSpan({ + "release.probes": probes, + ...(latest === undefined ? {} : { "release.revision": latest.revision }), + }); + if (latest === undefined) + return yield* new ReleaseUnpublished( + fieldsOf( + subject, + `${releasesUrl}/${producer}-${subject.upstream}-r0/SHA256SUMS`, + `${producer} has no published release of ${subject.upstream}`, + ), + ); + if (!ended) + return yield* integrity( + subject, + latest.url, + `${producer}-${subject.upstream} lists ${MAX_PROBED_REVISIONS} revisions without ending`, + ); + return latest; +}); + +const validUpstreamImage = (subject: Subject, url: string, image: string) => + UPSTREAM_IMAGE_PATTERN.test(image) + ? Effect.succeed(image) + : Effect.fail(integrity(subject, url, `Release names an invalid upstream image: ${image}`)); + +/** Verifies one target's manifest against its release checksum and returns its pin. */ +const verifyTarget = Effect.fnUntraced(function* ( + subject: Subject, + sumsUrl: string, + releaseBase: string, + releaseVersion: string, + producer: string, + sums: ReadonlyMap<string, string>, + target: NativeTarget, +) { + const stem = `${producer}-${releaseVersion}-${target}`; + const archive = sums.get(`${stem}.tar.zst`); + const manifest = sums.get(`${stem}.manifest.json`); + if (archive === undefined || manifest === undefined) + return yield* integrity( + subject, + sumsUrl, + `SHA256SUMS has no entry for ${archive === undefined ? `${stem}.tar.zst` : `${stem}.manifest.json`}`, + ); + const url = `${releaseBase}/${stem}.manifest.json`; + const fetched = yield* fetchResource(subject, url); + if (fetched.status === 404) + return yield* integrity(subject, url, `Published release has no ${stem}.manifest.json`); + if (!isOk(fetched.status)) return yield* unexpectedStatus(subject, url, fetched.status); + const actual = yield* sha256Hex(subject, url, fetched.bytes); + if (actual !== manifest) + return yield* integrity(subject, url, "Manifest does not match SHA256SUMS", { + expected: manifest, + actual, + }); + const decoded = yield* decodeJson(nativeManifestSchema, subject, url, "Manifest", fetched.bytes); + if ( + decoded.service !== producer || + decoded.version !== releaseVersion || + decoded.target !== target + ) + return yield* integrity(subject, url, "Manifest describes a different release or target"); + return { + pin: { archive, manifest } satisfies NativePin, + upstreamImage: decoded.upstream_image ?? decoded.source_image, + }; +}); + +const resolveUpstreamImage = Effect.fn("ReleaseResolver.resolveUpstreamImage")(function* ( + subject: Subject, + releaseBase: string, + producer: string, + sums: ReadonlyMap<string, string>, + manifestImages: ReadonlyArray<string | undefined>, +) { + if (MIRROR_PRODUCERS.has(producer)) { + const name = `${producer}-${subject.upstream}.oci-provenance.json`; + const url = `${releaseBase}/${name}`; + yield* Effect.annotateCurrentSpan({ "release.upstream_image_source": "provenance" }); + const expected = sums.get(name); + if (expected === undefined) + return yield* integrity(subject, url, `SHA256SUMS has no entry for ${name}`); + const fetched = yield* fetchResource(subject, url); + if (fetched.status === 404) + return yield* integrity(subject, url, "Published release has no oci-provenance file"); + if (!isOk(fetched.status)) return yield* unexpectedStatus(subject, url, fetched.status); + const actual = yield* sha256Hex(subject, url, fetched.bytes); + if (actual !== expected) + return yield* integrity(subject, url, "oci-provenance does not match SHA256SUMS", { + expected, + actual, + }); + const { source } = yield* decodeJson( + provenanceSchema, + subject, + url, + "oci-provenance", + fetched.bytes, + ); + return yield* validUpstreamImage(subject, url, normalizeUpstreamImage(source)); + } + yield* Effect.annotateCurrentSpan({ "release.upstream_image_source": "manifests" }); + const images = new Set( + manifestImages.map((image) => + image === undefined ? undefined : normalizeUpstreamImage(image), + ), + ); + const [only] = images; + if (images.size !== 1 || only === undefined) + return yield* integrity( + subject, + releaseBase, + "Target manifests do not agree on one upstream image", + ); + return yield* validUpstreamImage(subject, releaseBase, only); +}); + +/** The registry's digest of the slim image tag: the manifest body's sha256, checked against the registry's header. */ +const imageDigest = Effect.fn("ReleaseResolver.imageDigest")(function* ( + subject: Subject, + registryUrl: string, + producer: string, + tag: string, +) { + const repository = `supabase/cli/${producer}`; + const tokenUrl = `${registryUrl}/token?scope=repository:${repository}:pull`; + const tokenResponse = yield* fetchResource(subject, tokenUrl); + if (!isOk(tokenResponse.status)) + return yield* unexpectedStatus(subject, tokenUrl, tokenResponse.status); + const { token } = yield* decodeJson( + registryTokenSchema, + subject, + tokenUrl, + "Registry token", + tokenResponse.bytes, + ); + const url = `${registryUrl}/v2/${repository}/manifests/${tag}`; + const fetched = yield* fetchResource(subject, url, { + Authorization: `Bearer ${token}`, + Accept: MANIFEST_ACCEPT, + }); + if (fetched.status === 404) + return yield* integrity(subject, url, `Published release has no image ${repository}:${tag}`); + if (!isOk(fetched.status)) return yield* unexpectedStatus(subject, url, fetched.status); + const actual = `sha256:${yield* sha256Hex(subject, url, fetched.bytes)}`; + if (fetched.digestHeader !== undefined && fetched.digestHeader !== actual) + return yield* integrity(subject, url, "Registry digest does not match the manifest body", { + expected: fetched.digestHeader, + actual, + }); + return actual; +}); + +/** + * Resolves the content pin of a published slim-services release: the highest revision of + * `upstream`, or exactly `revision`. The pin carries the same fields the compiled catalog does. + */ +export const resolvePin = Effect.fn("ReleaseResolver.resolvePin")( + function* (request: ResolvePinRequest) { + const subject: Subject = { service: request.service, upstream: request.upstream }; + const producer = artifactSourceService(request.service); + yield* Effect.annotateCurrentSpan({ + "release.service": request.service, + "release.upstream": request.upstream, + }); + const revisionValid = + request.revision === undefined || + (Number.isSafeInteger(request.revision) && request.revision >= 0); + if (!UPSTREAM_PATTERN.test(request.upstream) || !revisionValid) + return yield* new ReleaseUnpublished( + fieldsOf( + subject, + "", + `${request.upstream}${request.revision === undefined ? "" : `-r${request.revision}`} cannot name a release`, + ), + ); + const releasesUrl = yield* slimReleasesUrl; + const registryUrl = yield* slimRegistryUrl; + const found = yield* findRevision( + request.revision === undefined ? subject : at(subject, request.revision), + releasesUrl, + producer, + ); + const resolved = at(subject, found.revision); + const releaseVersion = `${request.upstream}-r${found.revision}`; + const releaseBase = `${releasesUrl}/${producer}-${releaseVersion}`; + const sums = yield* parseSums(resolved, found.url, found.sums); + const verify = (target: NativeTarget) => + verifyTarget(resolved, found.url, releaseBase, releaseVersion, producer, sums, target); + const targets = yield* Effect.all( + { + "darwin-arm64": verify("darwin-arm64"), + "linux-amd64": verify("linux-amd64"), + "linux-arm64": verify("linux-arm64"), + } satisfies Record<NativeTarget, unknown>, + { concurrency: "unbounded" }, + ).pipe(Effect.withSpan("ReleaseResolver.verifyNatives")); + const upstreamImage = yield* resolveUpstreamImage( + resolved, + releaseBase, + producer, + sums, + Object.values(targets).map((target) => target.upstreamImage), + ); + const digest = yield* imageDigest(resolved, registryUrl, producer, releaseVersion); + yield* Effect.annotateCurrentSpan({ + "release.revision": found.revision, + "release.outcome": "resolved", + }); + return { + upstreamVersion: request.upstream, + revision: found.revision, + image: `${SLIM_IMAGE_GHCR_REGISTRY}${producer}:${releaseVersion}@${digest}`, + upstreamImage, + natives: Record.map(targets, (target) => target.pin), + } satisfies ArtifactPin; + }, + Effect.tapError((error) => Effect.annotateCurrentSpan({ "release.outcome": error._tag })), +); diff --git a/packages/stack/src/preparation/SlimEndpoints.ts b/packages/stack/src/preparation/SlimEndpoints.ts new file mode 100644 index 0000000000..4ffeec0e41 --- /dev/null +++ b/packages/stack/src/preparation/SlimEndpoints.ts @@ -0,0 +1,22 @@ +import { Config } from "effect"; + +/** Slim image repositories, `ghcr.io/supabase/cli/<service>`; catalog pins always name this host. */ +export const SLIM_IMAGE_GHCR_REGISTRY = "ghcr.io/supabase/cli/"; + +const trimTrailingSlashes = (url: string): string => url.replace(/\/+$/u, ""); + +/** + * Base URL of the slim-services release downloads, `<base>/<tag>/<asset>`. Overridable through + * `SUPABASE_INTERNAL_SLIM_RELEASES_URL`, which a spawned stack owner inherits with the rest of the + * process environment. + */ +export const slimReleasesUrl = Config.NonEmptyString("SUPABASE_INTERNAL_SLIM_RELEASES_URL").pipe( + Config.withDefault("https://github.com/supabase/slim-services/releases/download"), + Config.map(trimTrailingSlashes), +); + +/** Origin of the registry API serving the slim images, overridable through `SUPABASE_INTERNAL_SLIM_REGISTRY_URL`. */ +export const slimRegistryUrl = Config.NonEmptyString("SUPABASE_INTERNAL_SLIM_REGISTRY_URL").pipe( + Config.withDefault("https://ghcr.io"), + Config.map(trimTrailingSlashes), +); diff --git a/packages/stack/src/preparation/SlimRelease.ts b/packages/stack/src/preparation/SlimRelease.ts new file mode 100644 index 0000000000..9746c278d2 --- /dev/null +++ b/packages/stack/src/preparation/SlimRelease.ts @@ -0,0 +1,66 @@ +/** + * Slim-services release conventions shared by the stack's release resolver and the catalog sync + * scripts under `.github/scripts`. Dependency-free so both can import it. + */ + +/** Native targets every slim release publishes, in catalog serialization order. */ +export const NATIVE_TARGETS = ["darwin-arm64", "linux-amd64", "linux-arm64"] as const; + +export type SlimNativeTarget = (typeof NATIVE_TARGETS)[number]; + +/** Producers that mirror an unmodified upstream image: its provenance file names it, not a manifest. */ +export const MIRROR_PRODUCERS: ReadonlySet<string> = new Set(["vector", "mailpit", "imgproxy"]); + +/** + * A normalized upstream image: lowercase `[a-z0-9]` path segments optionally separated by + * `.`/`_`/`-`, a `:`, and a tag in Docker's grammar. Anchored, so the value can be embedded in + * generated source. + */ +export const UPSTREAM_IMAGE_PATTERN = + /^[a-z0-9]+(?:[._-][a-z0-9]+)*(?:\/[a-z0-9]+(?:[._-][a-z0-9]+)*)*:[A-Za-z0-9_][A-Za-z0-9._-]{0,127}$/u; + +/** Strips a `docker.io/` prefix and any `@sha256:…` digest. */ +export const normalizeUpstreamImage = (image: string): string => { + const withoutDigest = image.split("@")[0] ?? image; + return withoutDigest.startsWith("docker.io/") + ? withoutDigest.slice("docker.io/".length) + : withoutDigest; +}; + +const SUMS_LINE = /^([0-9a-f]{64})[ \t]+\*?(.+)$/iu; + +export type ParsedSums = + | { readonly kind: "sums"; readonly sums: ReadonlyMap<string, string> } + | { + readonly kind: "malformed"; + readonly message: string; + readonly expected?: string; + readonly actual?: string; + }; + +/** + * Parses a release `SHA256SUMS` into lowercase digests keyed by asset basename. Rejects a line + * that is not `<sha256> <name>` and a name listed with two different digests. + */ +export const parseSha256Sums = (text: string): ParsedSums => { + const sums = new Map<string, string>(); + for (const line of text.split(/\r?\n/u)) { + const trimmed = line.trim(); + if (trimmed.length === 0) continue; + const match = SUMS_LINE.exec(trimmed); + if (match === null) + return { kind: "malformed", message: "SHA256SUMS contains a malformed line" }; + const name = (match[2] ?? "").split("/").at(-1) ?? ""; + const digest = (match[1] ?? "").toLowerCase(); + const known = sums.get(name); + if (known !== undefined && known !== digest) + return { + kind: "malformed", + message: `SHA256SUMS lists ${name} with two digests`, + expected: known, + actual: digest, + }; + sums.set(name, digest); + } + return { kind: "sums", sums }; +}; diff --git a/packages/stack/src/preparation/slim-services.integration.test.ts b/packages/stack/src/preparation/slim-services.integration.test.ts index 3000f3cb80..26f28c3464 100644 --- a/packages/stack/src/preparation/slim-services.integration.test.ts +++ b/packages/stack/src/preparation/slim-services.integration.test.ts @@ -2,6 +2,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; import { Cause, + ConfigProvider, Crypto, Deferred, Duration, @@ -18,6 +19,7 @@ import { createServer, type Server } from "node:http"; import { zstdCompress } from "node:zlib"; import { FetchHttpClient } from "effect/http"; import { catalogPins, prepareNativeArtifact, resolveArtifact } from "../Artifacts.ts"; +import { selection } from "../../tests/pins.ts"; import { makeArtifactStore, type ArtifactRequest, type ArtifactSource } from "./ArtifactStore.ts"; import { digestHex } from "./Integrity.ts"; import { @@ -792,7 +794,10 @@ describe("native artifact catalog", () => { () => Effect.scoped( Effect.gen(function* () { - const { version, releaseVersion } = yield* resolveArtifact({ service: "rest" }); + const { version, releaseVersion } = yield* resolveArtifact({ + service: "rest", + pin: selection.rest, + }); expect(releaseVersion).toMatch(/-r(0|[1-9][0-9]*)$/u); expect(releaseVersion.startsWith(`${version}-r`)).toBe(true); const asset = `postgrest-${releaseVersion}-linux-amd64`; @@ -807,7 +812,9 @@ describe("native artifact catalog", () => { const cacheRoot = yield* fs.makeTempDirectoryScoped({ prefix: "slim-services-catalog-" }); const failed = yield* withFetch( fetcher, - prepareNativeArtifact({ service: "rest" }, cacheRoot, linux).pipe(Effect.exit), + prepareNativeArtifact({ service: "rest", pin: selection.rest }, cacheRoot, linux).pipe( + Effect.exit, + ), ); const github = `https://github.com/supabase/slim-services/releases/download/postgrest-${releaseVersion}`; const bucket = `https://${s3}/postgrest/${releaseVersion}`; @@ -826,12 +833,44 @@ describe("native artifact catalog", () => { ), ); + it.live("downloads from the base named by SUPABASE_INTERNAL_SLIM_RELEASES_URL", () => + Effect.scoped( + Effect.gen(function* () { + const { releaseVersion } = yield* resolveArtifact({ service: "rest", pin: selection.rest }); + const asset = `postgrest-${releaseVersion}-linux-amd64`; + const requested: string[] = []; + const fs = yield* FileSystem.FileSystem; + const cacheRoot = yield* fs.makeTempDirectoryScoped({ prefix: "slim-services-override-" }); + + yield* withFetch( + serving({}, requested), + prepareNativeArtifact({ service: "rest", pin: selection.rest }, cacheRoot, linux).pipe( + Effect.exit, + Effect.provide( + ConfigProvider.layer( + ConfigProvider.fromEnv({ + env: { + SUPABASE_INTERNAL_SLIM_RELEASES_URL: "https://releases.fixture.test/base/", + }, + }), + ), + ), + ), + ); + + expect(requested[0]).toBe( + `https://releases.fixture.test/base/postgrest-${releaseVersion}/${asset}.manifest.json`, + ); + }).pipe(Effect.provide(NodeServices.layer)), + ), + ); + it.live( "keys the native cache by content: a fixture seeded under the real pinned digest is used, and an unpinned digest is never requested", () => Effect.scoped( Effect.gen(function* () { - const resolved = yield* resolveArtifact({ service: "rest" }); + const resolved = yield* resolveArtifact({ service: "rest", pin: selection.rest }); const fs = yield* FileSystem.FileSystem; const cacheRoot = yield* fs.makeTempDirectoryScoped({ prefix: "slim-services-cache-" }); const requested: string[] = []; @@ -853,7 +892,7 @@ describe("native artifact catalog", () => { ); const prepared = yield* withFetch( serving({}, requested), - prepareNativeArtifact({ service: "rest" }, cacheRoot, linux), + prepareNativeArtifact({ service: "rest", pin: selection.rest }, cacheRoot, linux), ); expect(prepared.root).toBe(seeded.path); expect(yield* fs.readFileString(prepared.executable)).toBe("pinned"); diff --git a/packages/stack/src/promise-api.integration.test.ts b/packages/stack/src/promise-api.integration.test.ts index 79fb1b0042..7233f96c7b 100644 --- a/packages/stack/src/promise-api.integration.test.ts +++ b/packages/stack/src/promise-api.integration.test.ts @@ -2,7 +2,13 @@ import { NodeServices, NodeSocketServer } from "@effect/platform-node"; import { expect, it } from "@effect/vitest"; import { Effect, Exit, FileSystem, Redacted, Schema } from "effect"; import * as NetAddress from "effect/net/NetAddress"; -import { discover, StackError, type LogRecord, type Observation } from "./index.ts"; +import { + discover, + LockUnsupportedVersionError, + StackError, + type LogRecord, + type Observation, +} from "./index.ts"; import { createTestStack } from "./testing.ts"; const databaseSecret = (observation: Observation) => @@ -78,3 +84,23 @@ it.live( }).pipe(Effect.scoped, Effect.provide(NodeServices.layer)), { timeout: 120_000 }, ); + +it.live("rejects with the typed lock error when the project lock is from a newer CLI", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const stateRoot = yield* fs.makeTempDirectoryScoped({ prefix: "stack-promise-lock-" }); + const projectRoot = yield* fs.makeTempDirectoryScoped({ prefix: "stack-promise-lock-root-" }); + yield* fs.makeDirectory(`${projectRoot}/supabase`); + yield* fs.writeFileString(`${projectRoot}/supabase/supabase.lock`, '{"lockfileVersion":2}'); + + const failure = yield* Effect.promise(() => + createTestStack({ runtime: "native", stateRoot, projectRoot }).then( + () => undefined, + (error: unknown) => error, + ), + ); + + expect(failure).toBeInstanceOf(LockUnsupportedVersionError); + expect(yield* Effect.promise(() => discover({ stateRoot }))).toEqual([]); + }).pipe(Effect.scoped, Effect.provide(NodeServices.layer)), +); diff --git a/packages/stack/src/promise-api.unit.test.ts b/packages/stack/src/promise-api.unit.test.ts index 3f0cf7c683..588ac1b7f4 100644 --- a/packages/stack/src/promise-api.unit.test.ts +++ b/packages/stack/src/promise-api.unit.test.ts @@ -1,9 +1,11 @@ import type { Effect } from "effect"; import { describe, expectTypeOf, it } from "vitest"; +import type { ArtifactPin } from "./Artifacts.ts"; import type { InitializationCommand } from "./Commands.ts"; import type * as StackEffect from "./effect.ts"; import type * as PromiseApi from "./index.ts"; import { createTestStack, makeTestStack } from "./testing.ts"; +import { selection } from "../tests/pins.ts"; type Kind = StackEffect.ServiceCreationInput["service"]; type CallOptions = PromiseApi.CallOptions; @@ -65,11 +67,12 @@ describe("Promise API derived from the Effect API", () => { it("accepts plain secrets wherever the Effect API takes Redacted configuration", () => { expectTypeOf<{ service: "database"; - config: { version: "17"; jwtExpiry: 3600; databasePassword: string; jwtSecret: string }; + config: { jwtExpiry: 3600; databasePassword: string; jwtSecret: string }; + pin: ArtifactPin; endpoints: {}; }>().toExtend<PromiseApi.ServiceCreationInput>(); expectTypeOf<{ - config: { version: string; jwtExpiry: number; databasePassword: string }; + config: { jwtExpiry: number; databasePassword: string }; }>().toExtend<Parameters<PromiseApi.DatabaseInstance["restart"]>[0]>(); }); @@ -86,11 +89,12 @@ describe("Promise API derived from the Effect API", () => { const createDatabase = (stack: PromiseApi.Stack) => stack.services.create({ service: "database", - config: { version: "17", jwtExpiry: 3600 }, + pin: selection.database, + config: { jwtExpiry: 3600 }, endpoints: {}, }); const createRest = (stack: PromiseApi.Stack) => - stack.services.create({ service: "rest", config: {}, endpoints: {} }); + stack.services.create({ service: "rest", pin: selection.rest, config: {}, endpoints: {} }); type Rest = Awaited<ReturnType<typeof createRest>>; expectTypeOf< Awaited<ReturnType<typeof createDatabase>> @@ -105,9 +109,9 @@ describe("Promise API derived from the Effect API", () => { it("types test stack services by the selected kinds", () => { const selectDefault = () => createTestStack().then((test) => test.services); const selectMany = () => - createTestStack({ services: ["database", { service: "rest", config: {} }] }).then( - (test) => test.services, - ); + createTestStack({ + services: ["database", { service: "rest", pin: selection.rest, config: {} }], + }).then((test) => test.services); expectTypeOf<keyof Awaited<ReturnType<typeof selectDefault>>>().toEqualTypeOf<"database">(); expectTypeOf<Awaited<ReturnType<typeof selectMany>>>().toEqualTypeOf<{ readonly database: PromiseApi.DatabaseInstance; diff --git a/packages/stack/src/public.e2e.test.ts b/packages/stack/src/public.e2e.test.ts index 3393f6eb5a..553c4db161 100644 --- a/packages/stack/src/public.e2e.test.ts +++ b/packages/stack/src/public.e2e.test.ts @@ -9,6 +9,7 @@ import * as PromiseStack from "./index.ts"; import { assertOwnerExited, captureOwnerPid } from "../tests/owner.ts"; import { destroyTestStack } from "../tests/stack-cleanup.ts"; import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; for (const runtime of ["node", "bun"] as const) { it.live( @@ -111,7 +112,7 @@ it.live( expect((yield* mail.credentials()).url).toBe(credentials.url); const stdoutChunks: Array<Uint8Array> = []; - const tool = yield* stack.commands.run(postgres.psql({ major: 17 }), { + const tool = yield* stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--version"], stdout: (bytes) => Effect.sync(() => { @@ -125,8 +126,8 @@ it.live( ); const database = yield* stack.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("public-pgprove-password"), jwtSecret: Redacted.make("public-pgprove-jwt-secret"), jwtExpiry: 3600, @@ -146,14 +147,14 @@ it.live( path.join(pgProveRoot, "nested.sql"), "SELECT plan(1);\nSELECT pass('public pgProve');\nSELECT * FROM finish();\n", ); - const extension = yield* stack.commands.run(postgres.psql({ major: 17 }), { + const extension = yield* stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "-c", "CREATE EXTENSION IF NOT EXISTS pgtap"], stdout: () => Effect.void, stderr: () => Effect.void, }); expect(extension.exitCode).toBe(0); const pgProveOutput: Array<Uint8Array> = []; - const pgProve = yield* stack.commands.run(postgres.pgProve({ major: 17 }), { + const pgProve = yield* stack.commands.run(postgres.pgProve({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--ext", ".sql", "main.sql", "--verbose"], pgProve: { mounts: [{ source: pgProveRoot, target: "/tests" }], @@ -182,8 +183,8 @@ it.live( const database = yield* Effect.tryPromise(() => client.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: "public-pgprove-password", jwtSecret: "public-pgprove-jwt-secret", jwtExpiry: 3600, @@ -195,7 +196,6 @@ it.live( yield* Effect.tryPromise(() => database.restart({ config: { - version: "17", databasePassword: "public-pgprove-password", jwtSecret: "public-pgprove-jwt-secret", jwtExpiry: 3600, @@ -207,7 +207,7 @@ it.live( const databaseUrl = urls.databaseUrl; if (databaseUrl === undefined) return yield* Effect.die("Database URL missing"); const promiseExtension = yield* Effect.tryPromise(() => - client.commands.run(postgres.psql({ major: 17 }), { + client.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "-c", "CREATE EXTENSION IF NOT EXISTS pgtap"], stdout: () => {}, stderr: () => {}, @@ -217,7 +217,7 @@ it.live( const promiseProveOutput: Array<Uint8Array> = []; const promiseProveError: Array<Uint8Array> = []; const promiseProve = yield* Effect.tryPromise(() => - client.commands.run(postgres.pgProve({ major: 17 }), { + client.commands.run(postgres.pgProve({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--ext", ".sql", "main.sql", "--verbose"], pgProve: { mounts: [{ source: pgProveRoot, target: "/tests" }], @@ -241,7 +241,7 @@ it.live( ).toContain("public pgProve"); const sqlOutput: Array<Uint8Array> = []; const sql = yield* Effect.tryPromise(() => - client.commands.run(postgres.psql({ major: 17 }), { + client.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "-At"], stdin: Stream.toAsyncIterable( Stream.make(new TextEncoder().encode("SELECT 42;\n")), @@ -266,7 +266,7 @@ it.live( return yield* Effect.die("Snapshot source URL missing"); const sourceUrl = sourceCredentials.databaseUrl; const seed = yield* Effect.tryPromise(() => - client.commands.run(postgres.psql({ major: 17 }), { + client.commands.run(postgres.psql({ pin: selection.database }), { args: [ "--dbname", sourceUrl, @@ -285,8 +285,8 @@ it.live( const restored = yield* Effect.tryPromise(() => client.services.create({ service: "database", + pin: selection.database, config: { - version: "17", databasePassword: "public-pgprove-password", jwtSecret: "public-pgprove-jwt-secret", jwtExpiry: 3600, @@ -307,7 +307,7 @@ it.live( const restoredUrl = restoredCredentials.databaseUrl; const restoredOutput: Array<Uint8Array> = []; const restoredQuery = yield* Effect.tryPromise(() => - client.commands.run(postgres.psql({ major: 17 }), { + client.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", restoredUrl, "-Atc", "SELECT value FROM snapshot_rows"], stdout: (bytes) => { restoredOutput.push(bytes); @@ -327,7 +327,12 @@ it.live( ); const selected = yield* Effect.tryPromise(() => client.composition.supabase([ - { service: "mail", config: {}, endpoints: { http: { port: "auto" } } }, + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, ]), ); const selectedMail = selected[0]; @@ -342,7 +347,14 @@ it.live( ); const reused = yield* Effect.tryPromise(() => client.composition.supabase( - [{ service: "mail", config: {}, endpoints: { http: { port: "auto" } } }], + [ + { + service: "mail", + pin: selection.mail, + config: {}, + endpoints: { http: { port: "auto" } }, + }, + ], { reuseIds: [selectedMail.id] }, ), ); @@ -391,6 +403,7 @@ it.live( const mail = yield* Effect.tryPromise(() => stack.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" }, smtp: { port: "auto" }, pop3: { port: "auto" } }, }), diff --git a/packages/stack/src/runtime/Container.integration.test.ts b/packages/stack/src/runtime/Container.integration.test.ts index 8051607c28..3cd82ba7b4 100644 --- a/packages/stack/src/runtime/Container.integration.test.ts +++ b/packages/stack/src/runtime/Container.integration.test.ts @@ -40,6 +40,7 @@ import * as Owner from "../Owner.ts"; import * as StackNamespace from "../StackNamespace.ts"; import { makeDockerDatabaseStorage } from "../storage/DockerDatabaseStorage.ts"; import { reconcileContainerPassword } from "../services/Database.ts"; +import { selection } from "../../tests/pins.ts"; const image = await Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -80,13 +81,14 @@ describe("container process adapter", { timeout: 120_000 }, () => { const pinnedImage = `${image}@${digest}`; const pullAttempted = yield* Ref.make(false); const spawner = makePullFailureSpawner(delegate, pullAttempted); - yield* makeContainerRuntime({ + const ensured = yield* makeContainerRuntime({ target: containerTarget, root: ".", }).pipe( - Effect.flatMap((runtime) => runtime.prepare(pinnedImage)), + Effect.flatMap((runtime) => runtime.ensureImage(pinnedImage)), Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), ); + expect(ensured).toEqual({ image: pinnedImage, pulled: false }); expect(yield* Ref.get(pullAttempted)).toBe(false); }).pipe(Effect.provide(NodeServices.layer)), ); @@ -1710,7 +1712,7 @@ describe("container process adapter", { timeout: 120_000 }, () => { container, spawner, }); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); // Password reconcile: the consumer F2 found spawning the engine unpinned. yield* reconcileContainerPassword( diff --git a/packages/stack/src/runtime/Container.ts b/packages/stack/src/runtime/Container.ts index db315e9fea..7e64d2ae34 100644 --- a/packages/stack/src/runtime/Container.ts +++ b/packages/stack/src/runtime/Container.ts @@ -85,6 +85,10 @@ export class ContainerLaunchError extends Data.TaggedError("ContainerLaunchError export interface ContainerRuntime { readonly prepare: (image: string) => Effect.Effect<void, ContainerError>; readonly prepareImage: (image: string) => Effect.Effect<string, ContainerError>; + /** Prepares an image and reports whether it had to be pulled. */ + readonly ensureImage: ( + image: string, + ) => Effect.Effect<{ readonly image: string; readonly pulled: boolean }, ContainerError>; readonly launch: ( spec: ContainerSpec, ) => Effect.Effect<ContainerProcess, ContainerError | ContainerLaunchError, Scope.Scope>; @@ -571,13 +575,14 @@ export const makeContainerRuntime = (options: { image: string, mirrors: ReadonlyArray<string>, primaryError: ContainerError, - ): Effect.Effect<string, ContainerError> => { + ): Effect.Effect<{ readonly image: string; readonly pulled: boolean }, ContainerError> => { const [mirror, ...rest] = mirrors; if (mirror === undefined) return Effect.fail(primaryError); return Effect.gen(function* () { - if (!(yield* present(mirror))) yield* pull(mirror); + const pulled = !(yield* present(mirror)); + if (pulled) yield* pull(mirror); yield* Ref.update(mirrored, (map) => new Map(map).set(image, mirror)); - return mirror; + return { image: mirror, pulled }; }).pipe( Effect.tap(() => Effect.logInfo(`Pulled image from mirror ${mirror}`)), Effect.tapError((cause) => Effect.logWarning(`Image mirror ${mirror} failed`, cause)), @@ -585,7 +590,7 @@ export const makeContainerRuntime = (options: { ); }; - const prepareImage = Effect.fn("Container.prepareImage")(function* (image: string) { + const ensureImage = Effect.fn("Container.ensureImage")(function* (image: string) { // A mirror chosen earlier may have been pruned since; launches follow the primary again. const usePrimary = Ref.update(mirrored, (map) => { if (!map.has(image)) return map; @@ -598,15 +603,15 @@ export const makeContainerRuntime = (options: { const attempt = Effect.gen(function* () { if (yield* present(image)) { yield* usePrimary; - return image; + return { image, pulled: false }; } return yield* pull(image).pipe( - Effect.as(image), + Effect.as({ image, pulled: true }), Effect.tap(() => usePrimary), Effect.catch((primaryError) => fromMirror(image, mirrors, primaryError)), ); }); - return yield* attempt.pipe( + const prepared = yield* attempt.pipe( Effect.tapError((error) => !retryablePull(error) ? Effect.void @@ -616,7 +621,11 @@ export const makeContainerRuntime = (options: { ), Effect.retry({ schedule: pullBackoff, times: PULL_MAX_RETRIES, while: retryablePull }), ); + yield* Effect.annotateCurrentSpan({ "container.image_pulled": prepared.pulled }); + return prepared; }); + const prepareImage = (image: string) => + ensureImage(image).pipe(Effect.map((prepared) => prepared.image)); const prepare = Effect.fn("Container.prepare")((image: string) => prepareImage(image).pipe(Effect.asVoid), ); @@ -1160,6 +1169,7 @@ export const makeContainerRuntime = (options: { return { prepare, prepareImage, + ensureImage, launch, launchCommand: (spec) => launch(spec, true, true), }; diff --git a/packages/stack/src/schema-writers.integration.test.ts b/packages/stack/src/schema-writers.integration.test.ts new file mode 100644 index 0000000000..b508745311 --- /dev/null +++ b/packages/stack/src/schema-writers.integration.test.ts @@ -0,0 +1,386 @@ +import { NodeHttpClient, NodeServices } from "@effect/platform-node"; +import { expect, it } from "@effect/vitest"; +import { + Data, + Deferred, + Effect, + Fiber, + FileSystem, + Layer, + Path, + Redacted, + Ref, + Stream, +} from "effect"; +import { ChildProcess, ChildProcessSpawner } from "effect/process"; +import { fileURLToPath } from "node:url"; +import { create, open, type ServiceCreationInput } from "./effect.ts"; +import { initialization } from "./Commands.ts"; +import { databaseIdentity, recordDatabaseIdentity } from "./internal/database-reuse.ts"; +import { readSchemaWriters } from "./internal/schema-writers.ts"; +import { makeDatabaseSnapshots } from "./services/DatabaseSnapshot.ts"; +import { testArtifactCacheRoot } from "../tests/artifact-cache.ts"; +import { selection } from "../tests/pins.ts"; +import { destroyTestStack } from "../tests/stack-cleanup.ts"; + +const layer = Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp); +const jwtSecret = "schema-writers-jwt-secret-with-at-least-32-chars"; +const database = { + service: "database", + pin: selection.database, + config: { + databasePassword: Redacted.make("schema-writers-password"), + jwtSecret: Redacted.make(jwtSecret), + jwtExpiry: 3600, + }, + endpoints: { sql: { port: "auto" } }, +} as const satisfies ServiceCreationInput; +const auth = { + service: "auth", + pin: selection.auth, + config: { jwtSecret, jwtExpiry: 3600 }, + endpoints: { http: { port: "auto" } }, +} as const satisfies ServiceCreationInput; +const olderAuth = { ...auth, pin: { ...selection.auth, upstreamVersion: "v0.0.1" } } as const; +const migrated = { auth: selection.auth.upstreamVersion }; +const unreachableDatabaseUrl = "postgresql://invalid:invalid@127.0.0.1:1/postgres"; + +const barrierFixture = fileURLToPath( + new URL("../tests/snapshot-restore-barrier-fixture.ts", import.meta.url), +); + +class BarrierError extends Data.TaggedError("BarrierError")<{ readonly message: string }> {} + +it.live( + "refuses an older auth once a newer one migrated the database, until a reset forgets it", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-schema-writers-" }); + const stateRoot = path.join(root, "state"); + const stack = yield* create({ + projectRoot: root, + stateRoot, + cacheRoot: testArtifactCacheRoot, + runtime: "native", + }); + yield* Effect.acquireUseRelease( + Effect.succeed(stack), + (current) => + Effect.gen(function* () { + const members = yield* current.composition.supabase([database, auth], { eager: true }); + const databaseId = members.find(({ service }) => service === "database")?.id; + const authId = members.find(({ service }) => service === "auth")?.id; + if (databaseId === undefined || authId === undefined) + return yield* Effect.die("Composition is missing a member"); + const instanceRoot = path.join(stateRoot, current.id, "data", databaseId); + const recorded = readSchemaWriters(fs, path, instanceRoot); + const downgrade = { reuseIds: [databaseId, authId], eager: true }; + const recreate = `supabase stack destroy --stack-id ${current.id}`; + + yield* current.composition.start; + expect(yield* recorded).toEqual(migrated); + yield* current.composition.stop; + + const composed = yield* Effect.flip( + current.composition.supabase([database, olderAuth], downgrade), + ); + expect(composed.message).toContain("auth v0.0.1 is older than"); + expect(composed.message).toContain(recreate); + const oneShot = yield* Effect.flip( + current.commands.run( + initialization.auth({ + pin: olderAuth.pin, + databaseId, + databaseUrl: unreachableDatabaseUrl, + }), + ), + ); + expect(oneShot.message).toContain("auth v0.0.1 is older than"); + expect(oneShot.message).toContain(recreate); + expect(yield* recorded).toEqual(migrated); + + const target = yield* current.services.get(databaseId); + if (target.service !== "database") + return yield* Effect.die("The database member is not a database"); + yield* target.saveSnapshot("auth-migrated", { scope: "instance" }); + yield* target.resetData; + expect(yield* recorded).toEqual({}); + + yield* current.composition.supabase([database, olderAuth], downgrade); + expect(yield* target.restoreSnapshot("auth-migrated", { scope: "instance" })).toBe( + true, + ); + expect(yield* recorded).toEqual(migrated); + const restored = yield* Effect.flip( + current.composition.supabase([database, olderAuth], downgrade), + ); + expect(restored.message).toContain("auth v0.0.1 is older than"); + + const newer = '{"auth":"v999.0.0"}'; + yield* current.composition.supabase([database, auth], downgrade); + yield* fs.writeFileString(path.join(instanceRoot, "schema-writers.json"), newer); + const started = yield* Effect.flip(current.composition.start); + const launch = started.outcomes?.find(({ id }) => id === authId); + expect(launch?.succeeded).toBe(false); + expect(launch?.error).toContain( + `auth ${selection.auth.upstreamVersion} is older than v999.0.0`, + ); + expect(launch?.error).toContain(recreate); + expect(yield* recorded).toEqual({ auth: "v999.0.0" }); + }), + destroyTestStack, + ); + }).pipe(Effect.scoped, Effect.provide(layer)), + { timeout: 10 * 60_000 }, +); + +it.live( + "leaves the composition database's record alone for a standalone schema writer on an external database", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ + prefix: "stack-schema-writers-standalone-", + }); + const stateRoot = path.join(root, "state"); + const stack = yield* create({ + projectRoot: root, + stateRoot, + cacheRoot: testArtifactCacheRoot, + runtime: "native", + }); + yield* Effect.acquireUseRelease( + Effect.succeed(stack), + (current) => + Effect.gen(function* () { + const members = yield* current.composition.supabase([database, auth]); + const databaseId = members.find(({ service }) => service === "database")?.id; + if (databaseId === undefined) return yield* Effect.die("Composition has no database"); + const instanceRoot = path.join(stateRoot, current.id, "data", databaseId); + const recorded = readSchemaWriters(fs, path, instanceRoot); + const record = (version: string) => + fs + .makeDirectory(instanceRoot, { recursive: true }) + .pipe( + Effect.andThen( + fs.writeFileString( + path.join(instanceRoot, "schema-writers.json"), + JSON.stringify({ auth: version }), + ), + ), + ); + const startStandalone = Effect.gen(function* () { + const standalone = yield* current.services.create({ + ...auth, + config: { ...auth.config, databaseUrl: unreachableDatabaseUrl }, + endpoints: {}, + }); + return yield* Effect.flip(standalone.start); + }); + + yield* record("v999.0.0"); + const newerRecord = yield* startStandalone; + expect(newerRecord.message).toContain("startup exited"); + expect(newerRecord.message).not.toContain("older than"); + expect(yield* recorded).toEqual({ auth: "v999.0.0" }); + + yield* record("v1.0.0"); + yield* startStandalone; + expect(yield* recorded).toEqual({ auth: "v1.0.0" }); + }), + destroyTestStack, + ); + }).pipe(Effect.scoped, Effect.provide(layer)), + { timeout: 5 * 60_000 }, +); + +it.live( + "keeps every service's version when schema-owning initializations run together", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ + prefix: "stack-schema-writers-concurrent-", + }); + const stateRoot = path.join(root, "state"); + const stack = yield* create({ + projectRoot: root, + stateRoot, + cacheRoot: testArtifactCacheRoot, + runtime: "native", + }); + yield* Effect.acquireUseRelease( + Effect.succeed(stack), + (current) => + Effect.gen(function* () { + const members = yield* current.composition.supabase([database]); + const databaseId = members.find(({ service }) => service === "database")?.id; + if (databaseId === undefined) return yield* Effect.die("Composition has no database"); + const target = { databaseId, databaseUrl: unreachableDatabaseUrl }; + + yield* Effect.forEach( + [ + initialization.auth({ pin: selection.auth, ...target }), + initialization.storage({ + pin: selection.storage, + filePath: path.join(root, "storage"), + ...target, + }), + initialization.realtime({ pin: selection.realtime, ...target }), + ], + (command) => Effect.exit(current.commands.run(command)), + { concurrency: "unbounded", discard: true }, + ); + + expect( + yield* readSchemaWriters( + fs, + path, + path.join(stateRoot, current.id, "data", databaseId), + ), + ).toEqual({ + auth: selection.auth.upstreamVersion, + storage: selection.storage.upstreamVersion, + realtime: selection.realtime.upstreamVersion, + }); + }), + destroyTestStack, + ); + }).pipe(Effect.scoped, Effect.provide(layer)), + { timeout: 5 * 60_000 }, +); + +for (const window of ["before-data", "before-record"] as const) { + it.live( + `keeps refusing an older auth when the restoring process is killed ${window}`, + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-schema-writers-kill-" }); + const stateRoot = path.join(root, "state"); + const snapshotCache = path.join(root, "snapshots"); + const stack = yield* create({ + projectRoot: root, + stateRoot, + cacheRoot: testArtifactCacheRoot, + runtime: "native", + }); + yield* Effect.ensuring( + Effect.gen(function* () { + const members = yield* stack.composition.supabase([database, auth]); + const databaseId = members.find(({ service }) => service === "database")?.id; + const authId = members.find(({ service }) => service === "auth")?.id; + if (databaseId === undefined || authId === undefined) + return yield* Effect.die("Composition is missing a member"); + const instanceRoot = path.join(stateRoot, stack.id, "data", databaseId); + yield* stack.stop; + + const source = path.join(root, "source"); + yield* fs.makeDirectory(path.join(source, "data"), { recursive: true }); + yield* fs.writeFileString( + path.join(source, "data", "PG_VERSION"), + `${selection.database.upstreamVersion.split(".")[0]}\n`, + ); + yield* recordDatabaseIdentity( + fs, + path, + source, + databaseIdentity(selection.database.upstreamVersion), + ); + yield* fs.writeFileString( + path.join(source, ".supabase-database-ready.json"), + '{"runtime":"native","profile":"supabase"}', + ); + yield* fs.writeFileString( + path.join(source, "schema-writers.json"), + `{"auth":"${selection.auth.upstreamVersion}"}`, + ); + const snapshots = yield* makeDatabaseSnapshots({ + instanceRoot: source, + cacheRoot: snapshotCache, + runtime: "native", + pin: selection.database, + }); + yield* snapshots.saveSnapshot("auth-migrated"); + + yield* Effect.scoped( + Effect.gen(function* () { + const restoring = yield* spawner.spawn( + ChildProcess.make( + process.execPath, + [barrierFixture, instanceRoot, snapshotCache, "auth-migrated", window], + { stdin: "ignore", stdout: "pipe", stderr: "pipe" }, + ), + ); + const reached = yield* Deferred.make<void>(); + const stderr = yield* Ref.make(""); + yield* restoring.stderr.pipe( + Stream.decodeText, + Stream.runForEach((chunk) => Ref.update(stderr, (text) => text + chunk)), + Effect.forkScoped, + ); + const output = yield* restoring.stdout.pipe( + Stream.decodeText, + Stream.splitLines, + Stream.tap((line) => + line === "barrier" ? Deferred.succeed(reached, undefined) : Effect.void, + ), + Stream.runDrain, + Effect.forkScoped, + ); + const failed = (reason: string) => + Ref.get(stderr).pipe( + Effect.flatMap((text) => + Effect.fail(new BarrierError({ message: `${reason}: ${text}` })), + ), + ); + // The fixture announces the boundary itself, right before the real rename, so + // the kill lands between the restore steps on every run. + yield* Deferred.await(reached).pipe( + Effect.raceFirst( + restoring.exitCode.pipe( + Effect.flatMap((code) => + failed(`Restore exited before the barrier (${code})`), + ), + ), + ), + Effect.timeoutOrElse({ + duration: "30 seconds", + orElse: () => failed("Restore did not reach the barrier"), + }), + ); + yield* Effect.sync(() => process.kill(Number(restoring.pid), "SIGKILL")); + yield* restoring.exitCode.pipe(Effect.ignore); + yield* Fiber.join(output); + }), + ); + + expect(yield* fs.exists(path.join(instanceRoot, "data", "PG_VERSION"))).toBe( + window === "before-record", + ); + const reopened = yield* open({ + id: stack.id, + stateRoot, + cacheRoot: testArtifactCacheRoot, + }); + const reuse = { reuseIds: [databaseId, authId] }; + const refused = yield* Effect.flip( + reopened.composition.supabase([database, olderAuth], reuse), + ); + expect(refused.message).toContain("auth v0.0.1 is older than"); + expect(refused.message).toContain(`supabase stack destroy --stack-id ${stack.id}`); + + yield* reopened.composition.supabase([database, auth], reuse); + }), + destroyTestStack(stack), + ); + }).pipe(Effect.scoped, Effect.provide(layer)), + { timeout: 3 * 60_000 }, + ); +} diff --git a/packages/stack/src/services/Analytics.unit.test.ts b/packages/stack/src/services/Analytics.unit.test.ts index c267474ab9..2379d7cc08 100644 --- a/packages/stack/src/services/Analytics.unit.test.ts +++ b/packages/stack/src/services/Analytics.unit.test.ts @@ -1,9 +1,11 @@ import { expect, it } from "@effect/vitest"; import { Effect } from "effect"; import { makeSpec, type Creation } from "./Analytics.ts"; +import { selection } from "../../tests/pins.ts"; const creation = (databaseUrl: string): Creation => ({ service: "analytics", + pin: selection.analytics, config: { databaseUrl, apiKey: "analytics-key" }, }); diff --git a/packages/stack/src/services/AnalyticsImgproxy.integration.test.ts b/packages/stack/src/services/AnalyticsImgproxy.integration.test.ts index 4383826f7b..d872ef2ce2 100644 --- a/packages/stack/src/services/AnalyticsImgproxy.integration.test.ts +++ b/packages/stack/src/services/AnalyticsImgproxy.integration.test.ts @@ -8,6 +8,7 @@ import { makeDockerTcpRelay } from "../../tests/docker-relay.ts"; import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-analytics", @@ -42,8 +43,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -63,6 +64,7 @@ describe("service catalog", () => { const analyticsRecipe = yield* makeServiceRecipe( { service: "analytics", + pin: selection.analytics, config: { databaseUrl, backend: "postgres", apiKey: "catalog-analytics" }, }, dockerOptions(root), @@ -84,7 +86,7 @@ describe("service catalog", () => { const imageRoot = `${callerRoot}/images`; yield* fs.makeDirectory(imageRoot, { recursive: true }); const imgproxyRecipe = yield* makeServiceRecipe( - { service: "imgproxy", config: { filePath: imageRoot } }, + { service: "imgproxy", pin: selection.imgproxy, config: { filePath: imageRoot } }, dockerOptions(root), ); const imgproxy = yield* makeStandaloneService(imgproxyRecipe.definition, { diff --git a/packages/stack/src/services/AuthSettings.unit.test.ts b/packages/stack/src/services/AuthSettings.unit.test.ts index 82ab1f8cc8..66caf0f533 100644 --- a/packages/stack/src/services/AuthSettings.unit.test.ts +++ b/packages/stack/src/services/AuthSettings.unit.test.ts @@ -2,11 +2,13 @@ import { describe, expect, it } from "vitest"; import { Effect, Schema } from "effect"; import { Creation, makeSpec } from "./Auth.ts"; import { Settings, settingsEnvironment } from "./AuthSettings.ts"; +import { selection } from "../../tests/pins.ts"; describe("Auth settings environment", () => { it("derives public Auth defaults from configured external URLs", () => { const creation = Schema.decodeSync(Creation)({ service: "auth", + pin: selection.auth, config: { databaseUrl: "postgresql://localhost/postgres", apiExternalUrl: "https://api.example", @@ -26,6 +28,7 @@ describe("Auth settings environment", () => { const run = (config: Record<string, unknown>) => { const creation = Schema.decodeSync(Creation)({ service: "auth", + pin: selection.auth, config: { databaseUrl: "postgresql://localhost/postgres", ...config }, }); return Effect.runSync(makeSpec().env(creation, new Map(), false)); diff --git a/packages/stack/src/services/AuthStorage.integration.test.ts b/packages/stack/src/services/AuthStorage.integration.test.ts index 7f09e4676c..611564657d 100644 --- a/packages/stack/src/services/AuthStorage.integration.test.ts +++ b/packages/stack/src/services/AuthStorage.integration.test.ts @@ -10,6 +10,7 @@ import { makeDockerHttpRelay, makeDockerTcpRelay } from "../../tests/docker-rela import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-auth-storage", @@ -44,8 +45,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -64,6 +65,7 @@ describe("service catalog", () => { const authRecipe = yield* makeServiceRecipe( { service: "auth", + pin: selection.auth, config: { databaseUrl, jwtSecret: secret, jwtExpiry: 3600 }, }, dockerOptions(root), @@ -106,7 +108,7 @@ describe("service catalog", () => { const storageRoot = `${callerRoot}/storage`; yield* fs.makeDirectory(storageRoot, { recursive: true }); const imgproxyRecipe = yield* makeServiceRecipe( - { service: "imgproxy", config: { filePath: storageRoot } }, + { service: "imgproxy", pin: selection.imgproxy, config: { filePath: storageRoot } }, dockerOptions(root), ); const imgproxy = yield* makeStandaloneService(imgproxyRecipe.definition, { @@ -119,6 +121,7 @@ describe("service catalog", () => { const storageRecipe = yield* makeServiceRecipe( { service: "storage", + pin: selection.storage, config: { databaseUrl, filePath: storageRoot, @@ -213,8 +216,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -231,7 +234,11 @@ describe("service catalog", () => { const databaseRelay = yield* makeDockerTcpRelay(databaseRecipe.endpoint("sql")); const databaseUrl = `postgresql://supabase_admin:postgres@${databaseRelay.host}:${databaseRelay.port}/postgres`; const authRecipe = yield* makeServiceRecipe( - { service: "auth", config: { databaseUrl, jwtSecret: secret, jwtExpiry: 3600 } }, + { + service: "auth", + pin: selection.auth, + config: { databaseUrl, jwtSecret: secret, jwtExpiry: 3600 }, + }, dockerOptions(root), ); const auth = yield* makeStandaloneService(authRecipe.definition, { diff --git a/packages/stack/src/services/Catalog.integration.test.ts b/packages/stack/src/services/Catalog.integration.test.ts index 6d9e575f74..580be245eb 100644 --- a/packages/stack/src/services/Catalog.integration.test.ts +++ b/packages/stack/src/services/Catalog.integration.test.ts @@ -1,6 +1,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Exit, FileSystem, Layer, Redacted } from "effect"; +import { selection } from "../../tests/pins.ts"; import { makeServiceRecipe } from "./Catalog.ts"; const options = (root: string) => ({ @@ -13,7 +14,7 @@ const options = (root: string) => ({ }); describe("service catalog", () => { - it.live("validates endpoint names at the service creation boundary", () => + it.live("validates endpoint names and the artifact pin at the service creation boundary", () => Effect.scoped( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -21,18 +22,17 @@ describe("service catalog", () => { const result = yield* makeServiceRecipe( { service: "rest", + pin: selection.rest, config: { databaseUrl: "postgres://db" }, endpoints: { smtp: { port: "auto" } }, }, options(root), ).pipe(Effect.exit); expect(Exit.isFailure(result)).toBe(true); - const misplacedDatabaseVersion = yield* makeServiceRecipe( + const unpinnedDatabase = yield* makeServiceRecipe( { service: "database", - version: "17", config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make("catalog-database-secret-with-at-least-32-chars"), jwtExpiry: 3600, @@ -40,7 +40,7 @@ describe("service catalog", () => { }, options(root), ).pipe(Effect.exit); - expect(Exit.isFailure(misplacedDatabaseVersion)).toBe(true); + expect(Exit.isFailure(unpinnedDatabase)).toBe(true); }), ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); @@ -54,6 +54,7 @@ describe("service catalog", () => { makeServiceRecipe( { service: "storage", + pin: selection.storage, config: { databaseUrl: "postgres://db" }, }, options(root), diff --git a/packages/stack/src/services/Catalog.ts b/packages/stack/src/services/Catalog.ts index ab80f98ced..fdd8393767 100644 --- a/packages/stack/src/services/Catalog.ts +++ b/packages/stack/src/services/Catalog.ts @@ -1,4 +1,4 @@ -import { Crypto, Effect, FileSystem, Path, Ref, Schema } from "effect"; +import { Crypto, Effect, FileSystem, Path, Ref, Schema, Tuple } from "effect"; import { ChildProcessSpawner } from "effect/process"; import { HttpClient } from "effect/http"; import { makeContainerRuntime } from "../runtime/Container.ts"; @@ -8,6 +8,7 @@ import { DatabaseConfig, DatabaseCreation, DatabaseEndpoints, + pinnedDatabaseConfig, type DatabaseComponent, } from "./Database.ts"; import * as Analytics from "./Analytics.ts"; @@ -138,6 +139,11 @@ export const ServiceCreationInput = Schema.Union([ Pooler.Creation, ]); export type ServiceCreationInput = Schema.Schema.Type<typeof ServiceCreationInput>; +/** A restart candidate; a restart keeps the instance's saved pin. */ +export const ServiceRestartInput = ServiceCreationInput.mapMembers( + Tuple.map(Schema.fieldsAssign({ pin: Schema.optionalKey(Schema.Never) })), +); +export type ServiceRestartInput = Schema.Schema.Type<typeof ServiceRestartInput>; export const serviceSchemas = { database: DatabaseConfig, rest: Rest.Config, @@ -213,21 +219,24 @@ const databaseRecipe = ( definition: { prepare: (candidate) => candidate.service === "database" - ? (component.definition.prepare?.(candidate.config) ?? Effect.void) + ? (component.definition.prepare?.(pinnedDatabaseConfig(candidate)) ?? Effect.void) : Effect.fail(serviceError("prepare", "Service kind cannot change during restart")), launch: (context) => context.config.service === "database" - ? component.definition.launch({ ...context, config: context.config.config }) + ? component.definition.launch({ ...context, config: pinnedDatabaseConfig(context.config) }) : Effect.fail(serviceError("launch", "Service kind cannot change during restart")), removeData: (context) => context.config.service === "database" - ? component.definition.removeData({ ...context, config: context.config.config }) + ? component.definition.removeData({ + ...context, + config: pinnedDatabaseConfig(context.config), + }) : Effect.fail(serviceError("destroy", "Service kind cannot change during restart")), }, resetDatabaseData: (context) => context.config.service === "database" ? component - .resetData({ ...context, config: context.config.config }) + .resetData({ ...context, config: pinnedDatabaseConfig(context.config) }) .pipe( Effect.mapError( (cause) => new CatalogError({ operation: "reset", message: cause.message, cause }), @@ -239,7 +248,7 @@ const databaseRecipe = ( saveDatabaseSnapshot: (context, key, scope) => context.config.service === "database" ? component - .saveSnapshot({ ...context, config: context.config.config }, key, scope) + .saveSnapshot({ ...context, config: pinnedDatabaseConfig(context.config) }, key, scope) .pipe( Effect.mapError( (cause) => @@ -252,7 +261,7 @@ const databaseRecipe = ( restoreDatabaseSnapshot: (context, key, scope) => context.config.service === "database" ? component - .restoreSnapshot({ ...context, config: context.config.config }, key, scope) + .restoreSnapshot({ ...context, config: pinnedDatabaseConfig(context.config) }, key, scope) .pipe( Effect.mapError( (cause) => @@ -300,12 +309,6 @@ export const makeServiceRecipe = Effect.fn("Catalog.makeServiceRecipe")( Effect.gen(function* () { const endpointError = validateEndpointNames(input); if (endpointError !== undefined) return yield* endpointError; - if (isRecord(input) && input.service === "database" && input.version !== undefined) - return yield* new CatalogError({ - operation: "config", - message: "Database artifact version belongs in config.version", - service: "database", - }); const creation = yield* Schema.decodeUnknownEffect(ServiceCreation)(input).pipe( Effect.mapError( (cause) => diff --git a/packages/stack/src/services/Database.integration.test.ts b/packages/stack/src/services/Database.integration.test.ts index 5c44b5871d..4bea5eebe2 100644 --- a/packages/stack/src/services/Database.integration.test.ts +++ b/packages/stack/src/services/Database.integration.test.ts @@ -2,10 +2,12 @@ import { PgClient } from "@effect/sql-pg"; import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; import { + Clock, ConfigProvider, Context, Deferred, Effect, + Exit, Fiber, FileSystem, Layer, @@ -15,20 +17,26 @@ import { Redacted, Ref, Schema, + Scope, Stream, } from "effect"; -import { postgresVersion } from "../Artifacts.ts"; +import { HttpClient } from "effect/http"; import { createHash } from "node:crypto"; +import { createServer } from "node:http"; // oxlint-disable-line effecttsgo/node-builtin-import -- raw release fixture server. import { DEFAULT_POSTGRES_ROOT_KEY } from "../Defaults.ts"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; -import { makeDatabase, type BackendEndpoint, type DatabaseConfig } from "./Database.ts"; +import { databaseIdentity, recordDatabaseIdentity } from "../internal/database-reuse.ts"; +import { holdSchemaWriter, readSchemaWriters } from "../internal/schema-writers.ts"; +import { makeDatabase, type BackendEndpoint, type PinnedDatabaseConfig } from "./Database.ts"; import { makeDockerDatabaseRoot, runEngine } from "../../tests/docker-fixture.ts"; import { observeContainerStop, cleanStopEvents } from "../../tests/engine-events.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { database15, orioledb, selection } from "../../tests/pins.ts"; +import { makeManualClock } from "../../tests/manual-clock.ts"; -const config: DatabaseConfig = { - version: "17", +const config: PinnedDatabaseConfig = { + pin: selection.database, databasePassword: Redacted.make("supabase-test-password"), jwtSecret: Redacted.make("supabase-test-jwt-secret"), jwtExpiry: 3600, @@ -62,9 +70,9 @@ const query = <Row extends object = object>( describe("database component", { timeout: 180_000 }, () => { for (const target of [ - { runtime: "native", version: "17" }, - { runtime: testEngine, version: "15" }, - { runtime: testEngine, version: "17" }, + { runtime: "native", version: "17", pin: selection.database }, + { runtime: testEngine, version: "15", pin: database15 }, + { runtime: testEngine, version: "17", pin: selection.database }, ] as const) it.live( `preserves default encryption keys and Vault secrets after ${target.runtime} PostgreSQL ${target.version} recreation`, @@ -85,9 +93,9 @@ describe("database component", { timeout: 180_000 }, () => { runtime: target.runtime, ...(target.runtime !== "native" ? { engineTarget } : {}), }); - const defaults: DatabaseConfig = { + const defaults: PinnedDatabaseConfig = { healthTimeoutMs: 120_000, - version: target.version, + pin: target.pin, databasePassword: config.databasePassword, jwtSecret: config.jwtSecret, jwtExpiry: config.jwtExpiry, @@ -170,7 +178,7 @@ describe("database component", { timeout: 180_000 }, () => { ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); - it.live("refuses unmarked or stock data for OrioleDB before preparing an artifact", () => + it.live("refuses initialized data with no or another identity before preparing an artifact", () => Effect.scoped( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -186,26 +194,100 @@ describe("database component", { timeout: 180_000 }, () => { const prepare = recipe.definition.prepare; if (prepare === undefined) return yield* Effect.die("database recipe has no prepare"); const instanceRoot = path.join(root, "database"); - const orioledb: DatabaseConfig = { ...config, version: "17.11.0.002-orioledb" }; + const orioledbConfig: PinnedDatabaseConfig = { ...config, pin: orioledb }; const recreate = "run `supabase stack destroy --stack-id database-line-test` to recreate the stack — this permanently deletes its local database data"; yield* fs.makeDirectory(path.join(instanceRoot, "data"), { recursive: true }); yield* fs.writeFileString(path.join(instanceRoot, "data", "PG_VERSION"), "17\n"); - expect((yield* Effect.flip(prepare(orioledb))).message).toContain( - `Unmarked PostgreSQL data cannot be verified as OrioleDB data; ${recreate}`, + expect((yield* Effect.flip(prepare(orioledbConfig))).message).toContain( + `PostgreSQL data has no recorded identity, but OrioleDB build ${orioledb.upstreamVersion} was requested; ${recreate}`, + ); + + yield* recordDatabaseIdentity( + fs, + path, + instanceRoot, + databaseIdentity(selection.database.upstreamVersion), + ); + yield* fs.writeFileString( + path.join(instanceRoot, ".supabase-database-ready.json"), + '{"runtime":"native","profile":"supabase"}', + ); + expect((yield* Effect.flip(prepare(orioledbConfig))).message).toContain( + `PostgreSQL data was initialized by stock PostgreSQL build ${selection.database.upstreamVersion}, but OrioleDB build ${orioledb.upstreamVersion} was requested; keep this data by pinning \`[local.services.database] version = "${selection.database.upstreamVersion}"\` and removing any \`[db] orioledb_version\` in supabase/config.toml, or ${recreate}`, + ); + expect( + (yield* Effect.flip( + prepare({ + ...config, + pin: { ...selection.database, upstreamVersion: "17.99.0.001" }, + }), + )).message, + ).toContain( + `stock PostgreSQL build 17.99.0.001 was requested; keep this data by pinning \`[local.services.database] version = "${selection.database.upstreamVersion}"\``, ); yield* fs.writeFileString( path.join(instanceRoot, ".supabase-database-ready.json"), - '{"version":"17.11.0.002","runtime":"native","profile":"supabase"}', + '{"runtime":"docker","profile":"supabase"}', ); - expect((yield* Effect.flip(prepare(orioledb))).message).toContain( - `Initialized database data is 17.11.0.002 on the native runtime, but 17.11.0.002-orioledb on the native runtime was requested; ${recreate}`, + expect((yield* Effect.flip(prepare(config))).message).toContain( + `Initialized database data is for the docker runtime, but the native runtime was requested; ${recreate}`, ); }), ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); + it.live("keeps a reset from forgetting the record of a writer that is still migrating", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-database-reset-" }); + const recipe = yield* makeDatabase({ + stackId: "database-reset-test", + instanceId: "database", + root, + cacheRoot: artifactCacheRoot, + runtime: "native", + }); + const instanceRoot = path.join(root, "database"); + const scope = yield* Scope.make(); + const reset = recipe.resetData({ id: "database", config, scope }); + yield* fs.makeDirectory(path.join(instanceRoot, "data"), { recursive: true }); + yield* fs.writeFileString(path.join(instanceRoot, "data", "PG_VERSION"), "17\n"); + + const migrating = yield* Scope.make(); + yield* holdSchemaWriter(fs, path, { + stackId: "database-reset-test", + instanceRoot, + service: "auth", + version: "v2.198.0", + }).pipe(Scope.provide(migrating)); + + const manual = yield* makeManualClock; + const blocked = yield* reset.pipe( + Effect.provideService(Clock.Clock, manual.clock), + Effect.flip, + Effect.forkChild, + ); + yield* manual.sleeping(50); + yield* manual.advance(61_000); + expect((yield* Fiber.join(blocked)).message).toContain("locked by another operation"); + expect(yield* readSchemaWriters(fs, path, instanceRoot)).toEqual({ auth: "v2.198.0" }); + expect(yield* fs.exists(path.join(instanceRoot, "data", "PG_VERSION"))).toBe(true); + + yield* Scope.close(migrating, Exit.void); + yield* reset; + expect(yield* readSchemaWriters(fs, path, instanceRoot)).toEqual({}); + expect(yield* fs.exists(path.join(instanceRoot, "data"))).toBe(false); + for (const name of ["schema-writers.lock", "schema-writer-auth.lock"]) + expect(yield* fs.exists(path.join(instanceRoot, name)), name).toBe(true); + }).pipe( + Effect.scoped, + Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp)), + ), + ); + it.live("resumes an interrupted stock native first start and refuses it for OrioleDB", () => Effect.scoped( Effect.gen(function* () { @@ -231,17 +313,17 @@ describe("database component", { timeout: 180_000 }, () => { // A restart can pass prepare before the first start writes readiness; launch checks again. const prepared = yield* makeStandaloneService( { ...recipe.definition, prepare: () => Effect.void }, - { id: "database", config: { ...config, version: "17.11.0.002-orioledb" } }, + { id: "database", config: { ...config, pin: orioledb } }, ); expect( (yield* Effect.flip(prepared.start.pipe(Effect.andThen(prepared.ready)))).message, - ).toContain("17.11.0.002-orioledb on the native runtime was requested"); + ).toContain(`OrioleDB build ${orioledb.upstreamVersion} was requested`); expect(yield* fs.readFileString(path.join(root, "database", "data", "PG_VERSION"))).toBe( "17\n", ); - // The line is recorded before initdb writes any data, so an interrupted first start has it. + // The identity is recorded before initdb writes any data, so an interrupted first start has it. const recorded = yield* fs.stat( - path.join(root, "database", ".supabase-database-line.json"), + path.join(root, "database", ".supabase-database-identity.json"), ); const initialized = yield* fs.stat(path.join(root, "database", "data", "PG_VERSION")); expect(Option.getOrThrow(recorded.mtime).getTime()).toBeLessThanOrEqual( @@ -250,9 +332,9 @@ describe("database component", { timeout: 180_000 }, () => { // Initialized data without its readiness marker is what an interrupted first start leaves. yield* fs.remove(path.join(root, "database", ".supabase-database-ready.json")); - expect( - (yield* Effect.flip(prepare({ ...config, version: "17.11.0.002-orioledb" }))).message, - ).toContain("belongs to release line 17, but 17.11.0.002-orioledb was requested"); + expect((yield* Effect.flip(prepare({ ...config, pin: orioledb }))).message).toContain( + `initialized by stock PostgreSQL build ${selection.database.upstreamVersion}, but OrioleDB build ${orioledb.upstreamVersion} was requested`, + ); yield* service.start; yield* service.ready; expect(yield* fs.exists(path.join(root, "database", ".supabase-database-ready.json"))).toBe( @@ -263,6 +345,128 @@ describe("database component", { timeout: 180_000 }, () => { ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); + it.live( + "restarts initialized data on a new release revision of its build and refuses another build", + () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const published = selection.database; + const releaseVersion = (revision: number) => `${published.upstreamVersion}-r${revision}`; + const nextRevision = published.revision + 1; + const upstreamReleases = "https://github.com/supabase/slim-services/releases/download"; + const assetPath = (revision: number, target: string, extension: string) => + `/postgres-${releaseVersion(revision)}/postgres-${releaseVersion(revision)}-${target}.${extension}`; + // The next revision is the published archive under a manifest that names the new revision. + const targets = ["darwin-arm64", "linux-amd64", "linux-arm64"] as const; + const manifestJson = Schema.fromJsonString(Schema.Record(Schema.String, Schema.Unknown)); + const client = HttpClient.followRedirects(yield* HttpClient.HttpClient); + const manifests = new Map<string, string>(); + const manifestPins = new Map<string, string>(); + for (const target of targets) { + const upstream = yield* client.get( + `${upstreamReleases}${assetPath(published.revision, target, "manifest.json")}`, + ); + const manifest = yield* Schema.decodeEffect(manifestJson)(yield* upstream.text); + const text = yield* Schema.encodeEffect(manifestJson)({ + ...manifest, + version: releaseVersion(nextRevision), + }); + manifests.set(assetPath(nextRevision, target, "manifest.json"), text); + manifestPins.set(target, createHash("sha256").update(text).digest("hex")); + } + const releaseServer = createServer((request, response) => { + const pathname = request.url ?? ""; + const manifest = manifests.get(pathname); + if (manifest !== undefined) return response.end(manifest); + response.writeHead(302, { + location: `${upstreamReleases}${pathname.replaceAll(releaseVersion(nextRevision), releaseVersion(published.revision))}`, + }); + return response.end(); + }); + const releasesPort = yield* Effect.acquireRelease( + Effect.callback<number>((resume) => { + releaseServer.listen(0, "127.0.0.1", () => { + const address = releaseServer.address(); + resume( + Effect.succeed( + typeof address === "object" && address !== null ? address.port : 0, + ), + ); + }); + }), + () => + Effect.callback<void>((resume) => { + releaseServer.close(() => resume(Effect.void)); + }), + ); + const pinFor = (target: (typeof targets)[number]) => ({ + ...published.natives[target], + manifest: manifestPins.get(target) ?? "", + }); + const nextPin = { + ...published, + revision: nextRevision, + natives: { + "darwin-arm64": pinFor("darwin-arm64"), + "linux-amd64": pinFor("linux-amd64"), + "linux-arm64": pinFor("linux-arm64"), + }, + }; + + const root = yield* fs.makeTempDirectoryScoped({ prefix: "stack-database-revision-" }); + const recipe = yield* makeDatabase({ + stackId: "database-revision-test", + instanceId: "database", + root, + cacheRoot: artifactCacheRoot, + runtime: "native", + }); + const withReleases = Effect.provide( + ConfigProvider.layerAdd( + ConfigProvider.fromEnvRecord({ + SUPABASE_INTERNAL_SLIM_RELEASES_URL: `http://127.0.0.1:${releasesPort}`, + }), + { asPrimary: true }, + ), + ); + yield* Effect.gen(function* () { + const service = yield* makeStandaloneService(recipe.definition, { + id: "database", + config: { ...config, healthTimeoutMs: 120_000 }, + }); + yield* service.start; + yield* service.ready; + yield* query( + yield* recipe.endpoint, + config.databasePassword, + "CREATE TABLE revision_rows AS SELECT 'kept' AS value", + ); + yield* service.stop; + + const restartConfig = { ...config, healthTimeoutMs: 120_000 }; + const otherBuild = { ...published, upstreamVersion: "17.99.0.001" }; + expect( + (yield* service.restart({ ...restartConfig, pin: otherBuild }).pipe(Effect.flip)) + .message, + ).toContain("stock PostgreSQL build 17.99.0.001 was requested"); + + yield* service.restart({ ...restartConfig, pin: nextPin }); + yield* service.ready; + expect( + yield* query( + yield* recipe.endpoint, + config.databasePassword, + "SELECT value FROM revision_rows", + ), + ).toEqual([{ value: "kept" }]); + yield* service.destroy; + }).pipe(withReleases); + }), + ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), + { timeout: 300_000 }, + ); + it.live("requires passwords from non-superusers on the native socket", () => Effect.scoped( Effect.gen(function* () { @@ -468,7 +672,10 @@ describe("database component", { timeout: 180_000 }, () => { ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); - for (const version of ["15", "17"]) + for (const [version, pin] of [ + ["15", database15], + ["17", selection.database], + ] as const) it.live(`runs native PostgreSQL ${version} cron jobs in supautils-guarded workers`, () => Effect.scoped( Effect.gen(function* () { @@ -481,7 +688,7 @@ describe("database component", { timeout: 180_000 }, () => { cacheRoot: artifactCacheRoot, runtime: "native", }); - const databaseConfig = { ...config, version }; + const databaseConfig = { ...config, pin }; const service = yield* makeStandaloneService(database.definition, { id: "database:cron", config: databaseConfig, @@ -654,7 +861,10 @@ describe("database component", { timeout: 180_000 }, () => { cacheRoot, runtime: "native", }); - const loggedConfig: DatabaseConfig = { ...config, settings: { log_statement: "all" } }; + const loggedConfig: PinnedDatabaseConfig = { + ...config, + settings: { log_statement: "all" }, + }; const service = yield* makeStandaloneService(first.definition, { id: "database:first", config: loggedConfig, @@ -718,18 +928,18 @@ describe("database component", { timeout: 180_000 }, () => { .restart({ ...config, settings: { listen_addresses: "*" } }) .pipe(Effect.flip); expect((yield* service.get).lifecycle).toBe("running"); - yield* service.restart({ ...config, version: "unsupported" }).pipe(Effect.flip); - expect((yield* service.get).lifecycle).toBe("running"); - const mismatch = yield* service.restart({ ...config, version: "15" }).pipe(Effect.flip); + const mismatch = yield* service.restart({ ...config, pin: database15 }).pipe(Effect.flip); expect(mismatch.message).toContain( - `Initialized database data is ${postgresVersion("17")} on the native runtime, but ${postgresVersion("15")} on the native runtime was requested`, + `initialized by stock PostgreSQL build ${selection.database.upstreamVersion}, but stock PostgreSQL build ${database15.upstreamVersion} was requested`, ); expect((yield* service.get).lifecycle).toBe("running"); yield* service.stop; yield* fs.remove(path.join(root, "first", ".supabase-database-ready.json")); - const incomplete = yield* service.restart({ ...config, version: "15" }).pipe(Effect.flip); + const incomplete = yield* service + .restart({ ...config, pin: database15 }) + .pipe(Effect.flip); expect(incomplete.message).toContain( - "PostgreSQL data from an unfinished first start is major 17, but major 15 was requested", + `initialized by stock PostgreSQL build ${selection.database.upstreamVersion}, but stock PostgreSQL build ${database15.upstreamVersion} was requested`, ); const reopened = yield* makeDatabase({ @@ -802,11 +1012,14 @@ describe("database component", { timeout: 180_000 }, () => { ).pipe(Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp))), ); - for (const version of ["15", "17"]) + for (const [version, pin] of [ + ["15", database15], + ["17", selection.database], + ] as const) it.live(`PostgreSQL ${version} persists container data and reconciles credentials`, () => Effect.scoped( Effect.gen(function* () { - const databaseConfig: DatabaseConfig = { ...config, version }; + const databaseConfig: PinnedDatabaseConfig = { ...config, pin }; const root = yield* makeDockerDatabaseRoot( "stack-database-container-", "stack-integration-container", diff --git a/packages/stack/src/services/Database.ts b/packages/stack/src/services/Database.ts index 31a591e43e..677d746c63 100644 --- a/packages/stack/src/services/Database.ts +++ b/packages/stack/src/services/Database.ts @@ -24,10 +24,9 @@ import { HttpClient } from "effect/http"; import { slimImageMirrors, prepareNativeArtifact, - postgresLine, - postgresVersion, resolveArtifact, useNativeArtifact, + type ArtifactPin, type PreparedNativeArtifact, } from "../Artifacts.ts"; import { @@ -90,10 +89,25 @@ import { type DockerDatabaseStorage, type DockerDatabaseStorageError, } from "../storage/DockerDatabaseStorage.ts"; -import { recreateStackAdvice, unusableDatabaseData } from "../internal/database-reuse.ts"; +import { + DatabaseDataState, + DatabaseReadyMarker, + databaseIdentity, + databaseIdentityMismatch, + databaseReadyMarkerFile, + readDatabaseDataState, + readDatabaseIdentity, + recordDatabaseIdentity, + recreateStackAdvice, +} from "../internal/database-reuse.ts"; +import { + removeSchemaWriters, + schemaWriterLockFiles, + schemaWritersFile, + withSchemaWriterLeases, +} from "../internal/schema-writers.ts"; export const DatabaseConfig = Schema.Struct({ - version: Schema.String, databasePassword: Schema.Redacted(Schema.String), jwtSecret: Schema.Redacted(Schema.String), jwtExpiry: Schema.Finite, @@ -106,21 +120,22 @@ export const DatabaseConfig = Schema.Struct({ ), }); -const DatabaseReadyMarker = Schema.Struct({ - version: Schema.String, - runtime: Schema.Literals(["native", "docker", "podman"]), - profile: Schema.Literal("supabase"), -}); - -/** Release line a native first start initializes, written before initdb so an interrupted start can resume. */ -const DatabaseLineMarker = Schema.Struct({ line: Schema.String }); -const lineMarkerFile = ".supabase-database-line.json"; - // Health reconciles role passwords as supabase_admin, including after a configured password change. const NATIVE_HBA_RULES = "local all supabase_admin trust\nlocal all all scram-sha-256\n"; export interface DatabaseConfig extends Schema.Schema.Type<typeof DatabaseConfig> {} +/** A database's configuration with the artifact pin its creation selected. */ +export interface PinnedDatabaseConfig extends DatabaseConfig { + readonly pin: ArtifactPin; +} + +/** Joins a database creation's pin and configuration into what the component runs. */ +export const pinnedDatabaseConfig = (creation: { + readonly pin: ArtifactPin; + readonly config: DatabaseConfig; +}): PinnedDatabaseConfig => ({ ...creation.config, pin: creation.pin }); + export const DatabaseEndpoints = Schema.Struct({ sql: Schema.optionalKey(EndpointIntent) }); export interface DatabaseEndpoints extends Schema.Schema.Type<typeof DatabaseEndpoints> {} export const DatabaseCreation = serviceCreation("database", DatabaseConfig, DatabaseEndpoints); @@ -155,17 +170,17 @@ export interface DatabaseOptions { } export interface DatabaseComponent { - readonly definition: ServiceDefinition<DatabaseConfig>; + readonly definition: ServiceDefinition<PinnedDatabaseConfig>; readonly resetData: ( - context: ServiceInstanceContext<DatabaseConfig>, + context: ServiceInstanceContext<PinnedDatabaseConfig>, ) => Effect.Effect<void, ServiceError>; readonly saveSnapshot: ( - context: ServiceInstanceContext<DatabaseConfig>, + context: ServiceInstanceContext<PinnedDatabaseConfig>, key: string, scope: SnapshotScope, ) => Effect.Effect<void, ServiceError>; readonly restoreSnapshot: ( - context: ServiceInstanceContext<DatabaseConfig>, + context: ServiceInstanceContext<PinnedDatabaseConfig>, key: string, scope: SnapshotScope, ) => Effect.Effect<boolean, ServiceError>; @@ -343,7 +358,6 @@ const health = Effect.fn("Database.health")(function* ( const readyMarker = yield* Schema.encodeEffect( Schema.fromJsonString(DatabaseReadyMarker), )({ - version: context.version, runtime: context.runtime, profile: "supabase", }).pipe(Effect.mapError((cause) => errorFor("bootstrap", cause))); @@ -355,7 +369,7 @@ const health = Effect.fn("Database.health")(function* ( if (context.markInitialized !== undefined) yield* context.markInitialized; yield* context.fs.rename( `${stage}/marker`, - `${context.instanceRoot}/.supabase-database-ready.json`, + `${context.instanceRoot}/${databaseReadyMarkerFile}`, ); }), ), @@ -605,12 +619,12 @@ export const makeDatabase = ( ...(options.helpers === undefined ? {} : { helpers: options.helpers }), }).pipe(Effect.mapError((cause) => databaseError("storage", cause))); - const snapshots = (version: string) => + const snapshots = (pin: ArtifactPin) => makeDatabaseSnapshots({ instanceRoot, cacheRoot: options.cacheRoot, runtime: options.runtime, - version, + pin, }).pipe( Effect.provideService(FileSystem.FileSystem, fs), Effect.provideService(Path.Path, path), @@ -620,12 +634,9 @@ export const makeDatabase = ( ); const dataCommand = Effect.fn("Database.containerFiles")( - function* (version: string, args: ReadonlyArray<string>) { + function* (pin: ArtifactPin, args: ReadonlyArray<string>) { if (container === undefined) return ""; - const artifact = yield* resolveArtifact({ - service: "database", - version: postgresVersion(version), - }); + const artifact = yield* resolveArtifact({ service: "database", pin }); return yield* Effect.scoped( Effect.gen(function* () { const child = yield* container.launchCommand({ @@ -640,7 +651,7 @@ export const makeDatabase = ( mounts: [ storage === undefined ? { source: instanceRoot, target: containerInstancePath, readOnly: false } - : yield* storage.mount(version).pipe( + : yield* storage.mount(pin).pipe( Effect.map((mount) => ({ ...mount, target: "/var/lib/postgresql/data" })), Effect.mapError((cause) => errorFor("data", cause)), ), @@ -687,9 +698,9 @@ export const makeDatabase = ( ); const prepareArtifact = Effect.fn("Database.prepareArtifact")(( - input: DatabaseConfig, + input: PinnedDatabaseConfig, ): Effect.Effect<void, ServiceError> => { - const version = postgresVersion(input.version); + const { pin } = input; if (!Number.isInteger(input.jwtExpiry) || input.jwtExpiry <= 0) return Effect.fail( new ServiceError({ @@ -716,7 +727,7 @@ export const makeDatabase = ( if (options.runtime === "native") // Ahead-of-time warm-up only: downloads and publishes the generation but pins nothing. // `launch` resolves (or prepares) and pins its own copy right before it spawns. - return prepareNativeArtifact({ service: "database", version }, options.cacheRoot).pipe( + return prepareNativeArtifact({ service: "database", pin }, options.cacheRoot).pipe( Effect.provideService(FileSystem.FileSystem, fs), Effect.provideService(Path.Path, path), Effect.provideService(Crypto.Crypto, crypto), @@ -730,81 +741,53 @@ export const makeDatabase = ( return Effect.fail( new ServiceError({ operation: "prepare", message: "Container runtime is unavailable" }), ); - return resolveArtifact({ service: "database", version }).pipe( + return resolveArtifact({ service: "database", pin }).pipe( Effect.flatMap((artifact) => selectedContainer.prepare(artifact.image)), Effect.mapError((cause) => errorFor("prepare", cause)), ); }); - /** Records the release line before initdb runs; initialized data keeps the line it has. */ - const recordLine = Effect.fn("Database.recordLine")( - function* (dataPath: string, version: string) { - if (yield* fs.exists(path.join(dataPath, "PG_VERSION"))) return; - const encoded = yield* Schema.encodeEffect(Schema.fromJsonString(DatabaseLineMarker))({ - line: postgresLine(version), - }); - const stage = yield* fs.makeTempDirectoryScoped({ - directory: instanceRoot, - prefix: ".line-", - }); - yield* fs.writeFileString(path.join(stage, "marker"), encoded, { mode: 0o600 }); - yield* fs.rename(path.join(stage, "marker"), path.join(instanceRoot, lineMarkerFile)); - }, - Effect.scoped, - Effect.mapError((cause) => errorFor("launch", cause)), - ); - /** - * Why the instance's data cannot serve `requested`, or `undefined` when it can. Prepare runs - * outside the lifecycle gate, so the gated launch checks again before touching the data. + * Fails when the instance's initialized data cannot serve `requested`. Prepare runs outside the + * lifecycle gate, so the gated launch checks again before touching the data. A Docker volume is + * invisible to the host, so Docker data without a readiness marker is checked by its storage. */ - const reuseFailure = Effect.fn("Database.reuseFailure")(function* (requested: string) { - const recreate = recreateStackAdvice(String(options.stackId)); - const markerPath = path.join(instanceRoot, ".supabase-database-ready.json"); - if (yield* fs.exists(markerPath)) { - const marker = yield* fs - .readFileString(markerPath) - .pipe(Effect.flatMap(Schema.decodeEffect(Schema.fromJsonString(DatabaseReadyMarker)))); - return postgresLine(marker.version) === postgresLine(requested) && - marker.runtime === options.runtime - ? undefined - : `Initialized database data is ${marker.version} on the ${marker.runtime} runtime, but ${requested} on the ${options.runtime} runtime was requested; ${recreate}`; + const checkReuse = Effect.fn("Database.checkReuse")(function* (requested: string) { + const state = yield* readDatabaseDataState(fs, path, instanceRoot, options.runtime); + if (DatabaseDataState.$is("RuntimeMismatch")(state)) { + yield* Effect.annotateCurrentSpan({ "database.reuse.outcome": "runtime_mismatch" }); + return yield* errorFor( + "prepare", + `Initialized database data is for the ${state.recorded} runtime, but the ${options.runtime} runtime was requested; ${recreateStackAdvice(String(options.stackId))}`, + ); } - const versionPath = path.join(instanceRoot, "data", "PG_VERSION"); - if (options.runtime !== "native" || !(yield* fs.exists(versionPath))) return undefined; - const linePath = path.join(instanceRoot, lineMarkerFile); - const recorded = (yield* fs.exists(linePath)) - ? (yield* fs - .readFileString(linePath) - .pipe(Effect.flatMap(Schema.decodeEffect(Schema.fromJsonString(DatabaseLineMarker))))) - .line - : undefined; - const reason = unusableDatabaseData( - { - major: (yield* fs.readFileString(versionPath)).trim(), - line: recorded, - initialized: false, - }, - requested, - ); - return reason === undefined ? undefined : `${reason}; ${recreate}`; + if (DatabaseDataState.$is("Fresh")(state)) { + yield* Effect.annotateCurrentSpan({ "database.reuse.outcome": "fresh" }); + return; + } + const recorded = yield* readDatabaseIdentity(fs, path, instanceRoot); + const mismatch = databaseIdentityMismatch(String(options.stackId), recorded, requested); + yield* Effect.annotateCurrentSpan({ + "database.reuse.outcome": mismatch === undefined ? "reusable" : "mismatch", + "database.reuse.recorded_build": recorded?.build ?? "none", + "database.reuse.requested_build": requested, + }); + if (mismatch !== undefined) return yield* errorFor("prepare", mismatch); }); const prepare = Effect.fn("Database.prepare")( - function* (input: DatabaseConfig) { - const reason = yield* reuseFailure(postgresVersion(input.version)); - if (reason !== undefined) return yield* errorFor("prepare", reason); + function* (input: PinnedDatabaseConfig) { + yield* checkReuse(input.pin.upstreamVersion); yield* prepareArtifact(input); }, Effect.mapError((cause) => errorFor("prepare", cause)), ); const openDatabaseContainer = Effect.fn("Database.openContainer")(function* ( - context: ServiceInstanceContext<DatabaseConfig>, + context: ServiceInstanceContext<PinnedDatabaseConfig>, ) { const config = { ...context.config, - version: postgresVersion(context.config.version), rootKey: context.config.rootKey ?? Redacted.make(DEFAULT_POSTGRES_ROOT_KEY), }; const dataPath = path.join(instanceRoot, "data"); @@ -819,7 +802,7 @@ export const makeDatabase = ( }), ) : yield* storage - .mount(config.version) + .mount(config.pin) .pipe(Effect.mapError((cause) => errorFor("launch", cause))); const rootKeyPath = path.join(instanceRoot, "pgsodium_root.key"); yield* fs @@ -833,13 +816,13 @@ export const makeDatabase = ( return yield* errorFor("launch", "Container runtime is unavailable"); const image = yield* resolveArtifact({ service: "database", - version: config.version, + pin: config.pin, }).pipe(Effect.mapError((cause) => errorFor("launch", cause))); if ( storage === undefined || (yield* storage.needsDataChown.pipe(Effect.mapError((cause) => errorFor("launch", cause)))) ) { - yield* dataCommand(config.version, ["chown", "100:101", "/var/lib/postgresql/data"]); + yield* dataCommand(config.pin, ["chown", "100:101", "/var/lib/postgresql/data"]); } const launched = selectedContainer .launch({ @@ -889,7 +872,7 @@ export const makeDatabase = ( const launch = Effect.fn("Database.launch")( ( - context: ServiceLaunchContext<DatabaseConfig>, + context: ServiceLaunchContext<PinnedDatabaseConfig>, ): Effect.Effect<RuntimeSession, ServiceError | ServiceLaunchError> => Effect.gen(function* () { const publish = yield* (yield* launchOutputPublisher(logs, context.launchId)).part; @@ -914,16 +897,14 @@ export const makeDatabase = ( ); const config = { ...context.config, - version: postgresVersion(context.config.version), rootKey: context.config.rootKey ?? Redacted.make(DEFAULT_POSTGRES_ROOT_KEY), }; - const reuse = yield* reuseFailure(config.version).pipe( + yield* checkReuse(config.pin.upstreamVersion).pipe( Effect.mapError((cause) => errorFor("launch", cause)), ); - if (reuse !== undefined) return yield* errorFor("launch", reuse); if (storage !== undefined) yield* storage - .prepare(config.version) + .prepare(config.pin) .pipe(Effect.mapError((cause) => errorFor("launch", cause))); const settings = postgresArguments(config); if (options.runtime === "native") { @@ -943,7 +924,16 @@ export const makeDatabase = ( yield* fs .makeDirectory(dataPath, { recursive: true, mode: 0o700 }) .pipe(Effect.mapError((cause) => errorFor("launch", cause))); - yield* recordLine(dataPath, config.version); + const initializing = !(yield* fs + .exists(path.join(dataPath, "PG_VERSION")) + .pipe(Effect.mapError((cause) => errorFor("launch", cause)))); + if (initializing) + yield* recordDatabaseIdentity( + fs, + path, + instanceRoot, + databaseIdentity(config.pin.upstreamVersion), + ).pipe(Effect.mapError((cause) => errorFor("launch", cause))); const rootKeyPath = path.join(nativeRoot, "pgsodium_root.key"); yield* fs .writeFileString(rootKeyPath, Redacted.value(config.rootKey), { mode: 0o600 }) @@ -1002,7 +992,7 @@ export const makeDatabase = ( // process's cleanup before releasing the pin. const launchScope = yield* Scope.fork(context.scope, "sequential"); const artifact = yield* useNativeArtifact( - { service: "database", version: config.version }, + { service: "database", pin: config.pin }, options.cacheRoot, ).pipe( Scope.provide(launchScope), @@ -1054,13 +1044,13 @@ export const makeDatabase = ( const setup = health(selectedEndpoint, config, Effect.void, { fs, instanceRoot, - version: config.version, + version: config.pin.upstreamVersion, runtime: options.runtime, markInitialized: storage === undefined ? undefined : storage - .markInitialized(config.version) + .markInitialized(config.pin) .pipe(Effect.mapError((cause) => errorFor("health", cause))), }); return { @@ -1095,13 +1085,13 @@ export const makeDatabase = ( { fs, instanceRoot, - version: config.version, + version: config.pin.upstreamVersion, runtime: options.runtime, markInitialized: storage === undefined ? undefined : storage - .markInitialized(config.version) + .markInitialized(config.pin) .pipe(Effect.mapError((cause) => errorFor("health", cause))), }, ); @@ -1126,7 +1116,7 @@ export const makeDatabase = ( ) : Effect.void; - const definition: ServiceDefinition<DatabaseConfig> = { + const definition: ServiceDefinition<PinnedDatabaseConfig> = { prepare, launch, removeData: (context) => @@ -1140,7 +1130,7 @@ export const makeDatabase = ( storage === undefined ? Effect.void : storage - .destroyData(postgresVersion(context.config.version)) + .destroyData(context.config.pin) .pipe(Effect.mapError((cause) => errorFor("destroy", cause))), ), ), @@ -1148,34 +1138,46 @@ export const makeDatabase = ( ), }; const resetData = Effect.fn("Database.resetData")(( - context: ServiceInstanceContext<DatabaseConfig>, + context: ServiceInstanceContext<PinnedDatabaseConfig>, ) => { const clear: Effect.Effect<void, ServiceError | DockerDatabaseStorageError> = storage === undefined ? destroyOwnedRoot(fs, path, instanceRoot, options.root, Effect.void, errorFor, [ instanceSnapshotsDirectory, - ]) - : storage.removeData(postgresVersion(context.config.version)); - return clear.pipe(Effect.mapError((cause) => errorFor("reset", cause))); + schemaWritersFile, + ...schemaWriterLockFiles, + ]).pipe( + // Forgotten last, so a crash leaves the record at least as new as the data. + Effect.andThen( + removeSchemaWriters(fs, path, instanceRoot).pipe( + Effect.mapError((cause) => errorFor("reset", cause)), + ), + ), + ) + : storage.removeData(context.config.pin); + // An admitted writer's raised record must not be forgotten while it still migrates. + return withSchemaWriterLeases(path, instanceRoot, clear).pipe( + Effect.mapError((cause) => errorFor("reset", cause)), + ); }); return { definition, resetData, saveSnapshot: (context, key, scope) => storage === undefined - ? Effect.flatMap(snapshots(context.config.version), (store) => + ? Effect.flatMap(snapshots(context.config.pin), (store) => store.saveSnapshot(key, scope), ).pipe(Effect.mapError((cause) => errorFor("snapshot", cause))) : storage - .saveSnapshot(postgresVersion(context.config.version), key, scope) + .saveSnapshot(context.config.pin, key, scope) .pipe(Effect.mapError((cause) => errorFor("snapshot", cause))), restoreSnapshot: (context, key, scope) => storage === undefined - ? Effect.flatMap(snapshots(context.config.version), (store) => + ? Effect.flatMap(snapshots(context.config.pin), (store) => store.restoreSnapshot(key, scope), ).pipe(Effect.mapError((cause) => errorFor("snapshot", cause))) : storage - .restoreSnapshot(postgresVersion(context.config.version), key, scope) + .restoreSnapshot(context.config.pin, key, scope) .pipe(Effect.mapError((cause) => errorFor("snapshot", cause))), endpoint: Ref.get(endpoint).pipe( Effect.flatMap((value) => diff --git a/packages/stack/src/services/DatabaseSnapshot.integration.test.ts b/packages/stack/src/services/DatabaseSnapshot.integration.test.ts index 187ad46266..a04ea1f8c1 100644 --- a/packages/stack/src/services/DatabaseSnapshot.integration.test.ts +++ b/packages/stack/src/services/DatabaseSnapshot.integration.test.ts @@ -1,18 +1,40 @@ import { NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; -import { Crypto, Data, Effect, Exit, FileSystem, Path, Ref, Schema, Scope, Stream } from "effect"; +import { + Clock, + Crypto, + Data, + Effect, + Exit, + Fiber, + FileSystem, + Path, + Ref, + Schema, + Scope, + Stream, +} from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; -import { postgresVersion, resolveArtifact } from "../Artifacts.ts"; +import { resolveArtifact } from "../Artifacts.ts"; +import { selection } from "../../tests/pins.ts"; +import { makeManualClock } from "../../tests/manual-clock.ts"; import { makeContainerRuntime, resolveEngineTarget } from "../runtime/Container.ts"; import { makeDockerDatabaseStorage } from "../storage/DockerDatabaseStorage.ts"; import { makeDockerHelperRegistry } from "../storage/DockerHelperRegistry.ts"; import { shellQuote } from "../storage/DockerSnapshotBackend.ts"; import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { testEngine } from "../../tests/test-engine.ts"; +import { + databaseIdentity, + readDatabaseIdentity, + recordDatabaseIdentity, +} from "../internal/database-reuse.ts"; +import { holdSchemaWriter, readSchemaWriters } from "../internal/schema-writers.ts"; import { makeDatabaseSnapshots, type SnapshotScope } from "./DatabaseSnapshot.ts"; // Derived from the real catalog (not hardcoded), so a Postgres pin bump never makes this go stale. -const version = postgresVersion("17"); +const pin = selection.database; +const version = pin.upstreamVersion; class ShellError extends Data.TaggedError("ShellError")<{ readonly message: string }> {} @@ -68,6 +90,7 @@ interface Overrides { interface SnapshotError { readonly operation: string; + readonly message: string; } interface Instance { @@ -96,11 +119,11 @@ interface Engine { readonly instance: (name: string, overrides?: Overrides) => Effect.Effect<Instance>; } -const readyMarker = (runtime: string) => - `{"version":"${version}","runtime":"${runtime}","profile":"supabase"}`; +const readyMarker = (runtime: string) => `{"runtime":"${runtime}","profile":"supabase"}`; const native = Effect.fnUntraced(function* () { const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; const services = yield* Effect.context<Services>(); const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; const root = yield* fs.makeTempDirectoryScoped({ prefix: "database-snapshot-contract-" }); @@ -126,7 +149,7 @@ const native = Effect.fnUntraced(function* () { instanceRoot, cacheRoot, runtime: "native", - version, + pin, }).pipe( Effect.provideService(FileSystem.FileSystem, overrides.fs ?? fs), Effect.provideService( @@ -148,6 +171,7 @@ const native = Effect.fnUntraced(function* () { yield* fs.writeFileString(`${data}/PG_VERSION`, "17\n"); yield* fs.writeFileString(`${data}/fixture`, value); yield* fs.writeFileString(`${data}/nested/fixture`, value); + yield* recordDatabaseIdentity(fs, path, instanceRoot, databaseIdentity(version)); yield* fs.writeFileString( `${instanceRoot}/.supabase-database-ready.json`, readyMarker("native"), @@ -183,7 +207,7 @@ const docker = Effect.fnUntraced(function* () { const stackId = "snapshot-contract"; const root = yield* makeDockerDatabaseRoot("database-snapshot-contract-", stackId); const cacheRoot = path.resolve(root, "../../../cache"); - const image = (yield* resolveArtifact({ service: "database", version }).pipe(Effect.orDie)).image; + const image = (yield* resolveArtifact({ service: "database", pin }).pipe(Effect.orDie)).image; const target = yield* resolveEngineTarget(spawner, "docker"); const container = yield* makeContainerRuntime({ target, @@ -261,8 +285,8 @@ const docker = Effect.fnUntraced(function* () { spawner: overrides.partialCopy === true ? partialCopy : spawner, helpers, }).pipe(Scope.provide(scope)); - yield* storage.mount(version); - if (overrides.fresh !== true) yield* storage.prepare(version); + yield* storage.mount(pin); + if (overrides.fresh !== true) yield* storage.prepare(pin); const marker = yield* fs .readFileString(`${instanceRoot}/.supabase-database-storage.json`) .pipe(Effect.flatMap(Schema.decodeEffect(StorageMarker))); @@ -275,23 +299,23 @@ const docker = Effect.fnUntraced(function* () { entries: `${cache}/entries`, stages: `${cache}/stages`, restoreStages: `${cache}/stages`, - saveSnapshot: (key, scope) => storage.saveSnapshot(version, key, scope), - restoreSnapshot: (key, scope) => storage.restoreSnapshot(version, key, scope), + saveSnapshot: (key, scope) => storage.saveSnapshot(pin, key, scope), + restoreSnapshot: (key, scope) => storage.restoreSnapshot(pin, key, scope), seed: (value) => Effect.gen(function* () { - yield* storage.prepare(version); + yield* storage.prepare(pin); yield* shell( `set -eu; /usr/bin/busybox mkdir -p ${shellQuote(`${data}/nested`)}; printf 17 > ${shellQuote(`${data}/PG_VERSION`)}; printf '%s' ${shellQuote(value)} > ${shellQuote(`${data}/fixture`)}; printf '%s' ${shellQuote(value)} > ${shellQuote(`${data}/nested/fixture`)}`, ); - yield* storage.markInitialized(version); + yield* storage.markInitialized(pin); yield* fs.writeFileString( `${instanceRoot}/.supabase-database-ready.json`, readyMarker("docker"), ); }).pipe(Effect.orDie), - clear: storage.removeData(version).pipe(Effect.orDie), + clear: storage.removeData(pin).pipe(Effect.orDie), destroy: storage - .destroyData(version) + .destroyData(pin) .pipe(Effect.andThen(fs.remove(instanceRoot, { recursive: true })), Effect.orDie), }; return instance; @@ -307,7 +331,6 @@ const engines = [ ]; const Descriptor = Schema.fromJsonString(Schema.Struct({ logicalKey: Schema.String })); -const ReadyVersion = Schema.fromJsonString(Schema.Struct({ version: Schema.String })); const setup = <E>(make: () => Effect.Effect<Engine, E, Services>) => Effect.gen(function* () { @@ -363,11 +386,9 @@ for (const { name, make } of engines) expect(yield* target.restoreSnapshot("key")).toBe(true); expect(yield* fixture(target)).toBe("saved\nsaved"); - expect( - yield* Schema.decodeEffect(ReadyVersion)( - yield* fs.readFileString(`${target.root}/.supabase-database-ready.json`), - ), - ).toEqual({ version }); + expect(yield* readDatabaseIdentity(fs, yield* Path.Path, target.root)).toEqual( + databaseIdentity(version), + ); }), ), ); @@ -387,6 +408,76 @@ for (const { name, make } of engines) ), ); + it.live("restores the schema-writers record that was saved with the snapshot", () => + live( + Effect.gen(function* () { + const { fs, engine } = yield* setup(make); + const path = yield* Path.Path; + const source = yield* engine.instance("source"); + yield* source.seed("saved"); + yield* fs.writeFileString( + `${source.root}/schema-writers.json`, + '{"auth":"v2.197.0","storage":"v1.30.0"}', + ); + yield* source.saveSnapshot("key"); + yield* fs.writeFileString( + `${source.root}/schema-writers.json`, + '{"auth":"v2.400.0","storage":"v1.99.0"}', + ); + + const target = yield* engine.instance("target"); + yield* fs.writeFileString( + `${target.root}/schema-writers.json`, + '{"auth":"v2.300.0","realtime":"v2.9.0"}', + ); + expect(yield* target.restoreSnapshot("key")).toBe(true); + + expect(yield* readSchemaWriters(fs, path, target.root)).toEqual({ + auth: "v2.197.0", + storage: "v1.30.0", + }); + }), + ), + ); + + it.live("waits for an admitted schema writer before replacing the record", () => + live( + Effect.gen(function* () { + const { fs, engine, contents } = yield* setup(make); + const path = yield* Path.Path; + const source = yield* engine.instance("source"); + yield* source.seed("saved"); + yield* fs.writeFileString(`${source.root}/schema-writers.json`, '{"auth":"v2.197.0"}'); + yield* source.saveSnapshot("key"); + const target = yield* engine.instance("target"); + yield* fs.writeFileString(`${target.root}/schema-writers.json`, '{"auth":"v2.200.0"}'); + + const migrating = yield* Scope.make(); + yield* holdSchemaWriter(fs, path, { + stackId: "snapshot-contract", + instanceRoot: target.root, + service: "storage", + version: "v1.31.0", + }).pipe(Scope.provide(migrating)); + const admitted = { auth: "v2.200.0", storage: "v1.31.0" }; + + const manual = yield* makeManualClock; + const blocked = yield* target + .restoreSnapshot("key") + .pipe(Effect.provideService(Clock.Clock, manual.clock), Effect.flip, Effect.forkChild); + yield* manual.sleeping(50); + yield* manual.advance(61_000); + expect((yield* Fiber.join(blocked)).message).toContain("locked by another operation"); + expect(yield* readSchemaWriters(fs, path, target.root)).toEqual(admitted); + expect(yield* contents(target.data)).toBe(""); + + yield* Scope.close(migrating, Exit.void); + expect(yield* target.restoreSnapshot("key")).toBe(true); + expect(yield* readSchemaWriters(fs, path, target.root)).toEqual({ auth: "v2.197.0" }); + }), + ), + ); + it.live("replaces an existing key with the newer generation", () => live( Effect.gen(function* () { diff --git a/packages/stack/src/services/DatabaseSnapshot.ts b/packages/stack/src/services/DatabaseSnapshot.ts index ab318908fd..79106f2ad7 100644 --- a/packages/stack/src/services/DatabaseSnapshot.ts +++ b/packages/stack/src/services/DatabaseSnapshot.ts @@ -15,8 +15,24 @@ import { } from "effect"; import type { PlatformError } from "effect/PlatformError"; import { ChildProcessSpawner } from "effect/process"; -import { postgresVersion } from "../Artifacts.ts"; +import type { ArtifactPin } from "../Artifacts.ts"; +import { + DatabaseIdentity, + databaseIdentity, + readDatabaseIdentity, + recordDatabaseIdentity, + sameDatabaseIdentity, +} from "../internal/database-reuse.ts"; import { failureMessage } from "../internal/failure-message.ts"; +import { + decodeSchemaWriters, + encodeSchemaWriters, + holdSchemaWriterLeases, + raiseSchemaWritersTo, + readSchemaWriters, + schemaWritersFile, + setSchemaWriters, +} from "../internal/schema-writers.ts"; import { lstatPath } from "../namespace/drivers/FileSystem.ts"; import { copyDirectory, type DirectoryCopyError } from "../storage/DirectoryCopy.ts"; import type { DatabaseRuntime } from "./Database.ts"; @@ -46,7 +62,7 @@ export const decodeSnapshotStop = Schema.decodeUnknownEffect(SnapshotStop); * instead of failing. `Rename` creates the destination's parent and refuses to replace * anything but an empty directory; an optional rename runs only when its source exists and * its destination does not. `Recover` moves each `retired-<digest>-<token>` stage back to - * its entry when that entry is missing. + * its entry when that entry is missing. `Read` ends the program, completing it with the file's text. */ export type SnapshotStep = Data.TaggedEnum<{ Ensure: { readonly directory: string }; @@ -58,6 +74,7 @@ export type SnapshotStep = Data.TaggedEnum<{ Copy: { readonly from: string; readonly to: string }; Adopt: { readonly directory: string }; Write: { readonly file: string; readonly text: string }; + Read: { readonly file: string }; Rename: { readonly from: string; readonly to: string; readonly optional: boolean }; Remove: { readonly path: string }; Touch: { readonly path: string }; @@ -65,9 +82,9 @@ export type SnapshotStep = Data.TaggedEnum<{ }>; export const SnapshotStep = Data.taggedEnum<SnapshotStep>(); -/** Result of a snapshot program; a stopped `ExpectText` carries the file's actual text. */ +/** Result of a snapshot program; a completed `Read` or a stopped `ExpectText` carries a file's text. */ export type SnapshotRun = Data.TaggedEnum<{ - Completed: {}; + Completed: { readonly text: string }; Stopped: { readonly outcome: SnapshotStop; readonly text: string }; }>; export const SnapshotRun = Data.taggedEnum<SnapshotRun>(); @@ -104,7 +121,7 @@ const format = "supabase-database-snapshot-v1" as const; const runtimes = Schema.Literals(["native", "docker", "podman"]); const SnapshotIdentity = Schema.Struct({ format: Schema.Literal(format), - version: Schema.String, + identity: DatabaseIdentity, runtime: runtimes, platform: Schema.String, arch: Schema.String, @@ -113,7 +130,6 @@ const SnapshotIdentity = Schema.Struct({ }); const SnapshotDescriptor = Schema.Struct({ ...SnapshotIdentity.fields, keyDigest: Schema.String }); const ReadyMarker = Schema.Struct({ - version: Schema.String, runtime: runtimes, profile: Schema.Literal("supabase"), }); @@ -161,17 +177,17 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio const crypto = yield* Crypto.Crypto; const { backends, runtime, version } = options; const { Adopt, Clear, Copy, Ensure, Expect, ExpectEmpty, ExpectText } = SnapshotStep; - const { Prune, Recover, Remove, Rename, Touch, Write } = SnapshotStep; + const { Prune, Read, Recover, Remove, Rename, Touch, Write } = SnapshotStep; const major = version.split(".")[0] ?? version; const markerPath = path.join(options.instanceRoot, ".supabase-database-ready.json"); - const mapError = <A, E>(operation: string, effect: Effect.Effect<A, E>) => + const mapError = <A, E, R>(operation: string, effect: Effect.Effect<A, E, R>) => effect.pipe(Effect.mapError((cause) => errorFor(operation, cause))); const describe = Effect.fnUntraced( function* (logicalKey: string) { const identity = { format, - version, + identity: databaseIdentity(version), runtime, platform: process.platform, arch: process.arch, @@ -232,9 +248,18 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio .readFileString(markerPath) .pipe(Effect.flatMap(Schema.decodeEffect(Schema.fromJsonString(ReadyMarker)))), ); - if (ready.version !== version || ready.runtime !== runtime) + const recorded = yield* mapError("ready", readDatabaseIdentity(fs, path, options.instanceRoot)); + if ( + ready.runtime !== runtime || + recorded === undefined || + !sameDatabaseIdentity(recorded, databaseIdentity(version)) + ) return yield* errorFor("ready", "Database readiness marker does not match the instance"); const { keyDigest, descriptor } = yield* describe(logicalKey); + const writers = yield* mapError( + "schema-writers", + readSchemaWriters(fs, path, options.instanceRoot).pipe(Effect.flatMap(encodeSchemaWriters)), + ); yield* locked( backend, Effect.gen(function* () { @@ -259,6 +284,7 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio Ensure({ directory: stage }), Copy({ from: backend.data, to: backend.join(stage, "data") }), Write({ file: backend.join(stage, "descriptor.json"), text: descriptor }), + Write({ file: backend.join(stage, schemaWritersFile), text: writers }), Rename({ from: target, to: retired, optional: true }), Rename({ from: stage, to: target, optional: false }), Remove({ path: retired }), @@ -281,11 +307,19 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio ); }); + // Recorded after the data is published, because the target's emptiness is proven only inside the + // restore program; a crash in between leaves the identity of an earlier interrupted start, if any. + const recordIdentity = recordDatabaseIdentity( + fs, + path, + options.instanceRoot, + databaseIdentity(version), + ).pipe(Effect.mapError((cause) => errorFor("ready", cause))); + const publishReadyMarker = Effect.fnUntraced( function* (id: string) { const staged = `${markerPath}.${id}`; const marker = yield* Schema.encodeEffect(Schema.fromJsonString(ReadyMarker))({ - version, runtime, profile: "supabase", }); @@ -318,7 +352,7 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio ]; return yield* Effect.uninterruptibleMask((restore) => Effect.gen(function* () { - const result = yield* restore( + const checked = yield* restore( backend.run([ Ensure({ directory: backend.entries }), ...reclaimStages(backend), @@ -339,21 +373,22 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio present: false, otherwise: "running", }), - Copy({ from: backend.join(entry, "data"), to: stage }), - Adopt({ directory: stage }), - Touch({ path: entry }), - Write({ file: published, text: id }), - Rename({ from: stage, to: backend.data, optional: false }), + Expect({ + path: backend.join(entry, schemaWritersFile), + present: true, + otherwise: "miss", + }), + Read({ file: backend.join(entry, schemaWritersFile) }), ]), ); - if (result._tag === "Stopped") { - switch (result.outcome) { + if (checked._tag === "Stopped") { + switch (checked.outcome) { case "miss": return false; case "descriptor": // A well-formed descriptor for another identity is a miss; anything else is corrupt. return yield* Schema.decodeEffect(Schema.fromJsonString(SnapshotDescriptor))( - result.text, + checked.text, ).pipe( Effect.as(false), Effect.mapError((cause) => errorFor("descriptor", cause)), @@ -369,6 +404,37 @@ export const makeSnapshotStore = Effect.fn("DatabaseSnapshot.makeStore")(functio return yield* errorFor("validate", "Snapshot contains a running database"); } } + const writers = yield* mapError("schema-writers", decodeSchemaWriters(checked.text)); + // The record is raised and fsynced before the data is published, and set to the + // snapshot's afterwards, so a crash between steps leaves it at least as new as the data. + // The leases keep a writer admitted meanwhile from having its raise overwritten below. + yield* Effect.scoped( + Effect.gen(function* () { + yield* restore( + mapError("schema-writers", holdSchemaWriterLeases(path, options.instanceRoot)), + ); + yield* restore( + mapError( + "schema-writers", + raiseSchemaWritersTo(fs, path, options.instanceRoot, writers), + ), + ); + yield* restore( + backend.run([ + Copy({ from: backend.join(entry, "data"), to: stage }), + Adopt({ directory: stage }), + Touch({ path: entry }), + Write({ file: published, text: id }), + Rename({ from: stage, to: backend.data, optional: false }), + ]), + ); + yield* mapError( + "schema-writers", + setSchemaWriters(fs, path, options.instanceRoot, writers), + ); + }), + ); + yield* recordIdentity; yield* publishReadyMarker(id); return true; }), @@ -452,6 +518,10 @@ const makeNativeSnapshotBackend = Effect.fnUntraced(function* (options: { Adopt: () => Effect.succeedNone, Write: ({ file, text }) => fs.writeFileString(file, text, { mode: 0o600 }).pipe(Effect.as(proceed)), + Read: ({ file }) => + fs + .readFileString(file) + .pipe(Effect.map((text) => Option.some(SnapshotRun.Completed({ text })))), Rename: ({ from, to, optional }) => Effect.gen(function* () { const destination = yield* exists(to); @@ -506,7 +576,7 @@ const makeNativeSnapshotBackend = Effect.fnUntraced(function* (options: { ); if (Option.isSome(stopped)) return stopped.value; } - return SnapshotRun.Completed(); + return SnapshotRun.Completed({ text: "" }); }), }; return backend; @@ -517,7 +587,7 @@ export const makeDatabaseSnapshots = Effect.fn("DatabaseSnapshot.make")(function readonly instanceRoot: string; readonly cacheRoot: string; readonly runtime: DatabaseRuntime; - readonly version: string; + readonly pin: ArtifactPin; }) { const path = yield* Path.Path; const cache = path.join(options.cacheRoot, "stack-database-snapshots"); @@ -539,6 +609,6 @@ export const makeDatabaseSnapshots = Effect.fn("DatabaseSnapshot.make")(function }, instanceRoot: options.instanceRoot, runtime: options.runtime, - version: postgresVersion(options.version), + version: options.pin.upstreamVersion, }); }); diff --git a/packages/stack/src/services/Functions.integration.test.ts b/packages/stack/src/services/Functions.integration.test.ts index 5a82801666..4686a62337 100644 --- a/packages/stack/src/services/Functions.integration.test.ts +++ b/packages/stack/src/services/Functions.integration.test.ts @@ -25,6 +25,7 @@ import { makeProcessRecipe } from "./ProcessRecipe.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-functions", @@ -76,6 +77,7 @@ describe("service catalog", () => { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, databaseUrl: "postgres://functions-db", @@ -163,6 +165,7 @@ describe("service catalog", () => { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, databaseUrl: "postgres://functions-db" }, }, { @@ -254,6 +257,7 @@ describe("service catalog", () => { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, verifyJwt: false, @@ -326,6 +330,7 @@ describe("service catalog", () => { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, verifyJwt: false, @@ -400,6 +405,7 @@ describe("service catalog", () => { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, verifyJwt: false, @@ -497,6 +503,7 @@ for (const runtime of ["native", testEngine] as const) { const recipe = yield* makeServiceRecipe( { service: "functions", + pin: selection.functions, config: { functionsRoot, filesRoot, @@ -598,6 +605,7 @@ it.effect("passes POSIX project paths to a docker Functions container from a Win const container: ContainerRuntime = { prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launchCommand: () => Effect.die("Functions has no startup commands"), launch: (spec) => Ref.set(launched, spec).pipe( @@ -609,6 +617,7 @@ it.effect("passes POSIX project paths to a docker Functions container from a Win const filesRoot = "C:\\Users\\dev\\project"; const creation: Functions.Creation = { service: "functions", + pin: selection.functions, config: { functionsRoot: `${filesRoot}\\supabase\\functions`, filesRoot, diff --git a/packages/stack/src/services/Initialization.ts b/packages/stack/src/services/Initialization.ts index 591a1c7f05..a0513d60f4 100644 --- a/packages/stack/src/services/Initialization.ts +++ b/packages/stack/src/services/Initialization.ts @@ -22,7 +22,7 @@ const resolve = <C extends RecipeCreation<"auth" | "storage" | "realtime", unkno runtime: CatalogOptions["runtime"], ): Effect.Effect<ResolvedCommand, ServiceError> => Effect.gen(function* () { - const artifact = yield* resolveArtifact({ service, version: creation.version }); + const artifact = yield* resolveArtifact({ service, pin: creation.pin }); const startup = yield* resolveStartupCommand( creation, spec, @@ -32,7 +32,7 @@ const resolve = <C extends RecipeCreation<"auth" | "storage" | "realtime", unkno ); return { service, - version: artifact.version, + pin: creation.pin, image: artifact.image, nativeExecutable: startup.executable, containerEntrypoint: command.containerEntrypoint, @@ -66,7 +66,7 @@ export const resolveInitializationCommand = ( "auth", { service: "auth", - ...(input.version === undefined ? {} : { version: input.version }), + pin: input.pin, config: { databaseUrl: input.databaseUrl, jwtSecret: options.credentials.jwtSecret, @@ -82,7 +82,7 @@ export const resolveInitializationCommand = ( "storage", { service: "storage", - ...(input.version === undefined ? {} : { version: input.version }), + pin: input.pin, config: { databaseUrl: input.databaseUrl, filePath: input.filePath, @@ -101,7 +101,7 @@ export const resolveInitializationCommand = ( "realtime", { service: "realtime", - ...(input.version === undefined ? {} : { version: input.version }), + pin: input.pin, config: { databaseUrl: input.databaseUrl, jwtSecret: options.credentials.jwtSecret, diff --git a/packages/stack/src/services/Mail.integration.test.ts b/packages/stack/src/services/Mail.integration.test.ts index 468453275f..84a0ed4bde 100644 --- a/packages/stack/src/services/Mail.integration.test.ts +++ b/packages/stack/src/services/Mail.integration.test.ts @@ -6,6 +6,7 @@ import { makeStandaloneService } from "../../tests/standalone-service.ts"; import { makeServiceRecipe } from "./Catalog.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-mail", @@ -66,7 +67,7 @@ describe("service catalog", () => { const client = yield* HttpClient.HttpClient; const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-" }); const recipe = yield* makeServiceRecipe( - { service: "mail", config: {} }, + { service: "mail", pin: selection.mail, config: {} }, dockerOptions(root), ); const instance = yield* makeStandaloneService(recipe.definition, { @@ -94,7 +95,10 @@ describe("service catalog", () => { const path = yield* Path.Path; const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-isolation-" }); const recipeFor = (instanceId: string) => - makeServiceRecipe({ service: "mail", config: {} }, { ...options(root), instanceId }); + makeServiceRecipe( + { service: "mail", pin: selection.mail, config: {} }, + { ...options(root), instanceId }, + ); const first = yield* recipeFor("mail-a"); const second = yield* recipeFor("mail-b"); const firstInstance = yield* makeStandaloneService(first.definition, { @@ -125,7 +129,10 @@ describe("service catalog", () => { Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-persistence-" }); - const recipe = yield* makeServiceRecipe({ service: "mail", config: {} }, options(root)); + const recipe = yield* makeServiceRecipe( + { service: "mail", pin: selection.mail, config: {} }, + options(root), + ); const instance = yield* makeStandaloneService(recipe.definition, { id: "mail", config: recipe.creation, @@ -157,7 +164,7 @@ describe("service catalog", () => { prefix: "catalog-mail-persistence-docker-", }); const recipe = yield* makeServiceRecipe( - { service: "mail", config: {} }, + { service: "mail", pin: selection.mail, config: {} }, dockerOptions(root), ); const instance = yield* makeStandaloneService(recipe.definition, { @@ -189,7 +196,10 @@ describe("service catalog", () => { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-destroy-" }); - const recipe = yield* makeServiceRecipe({ service: "mail", config: {} }, options(root)); + const recipe = yield* makeServiceRecipe( + { service: "mail", pin: selection.mail, config: {} }, + options(root), + ); const instance = yield* makeStandaloneService(recipe.definition, { id: "mail", config: recipe.creation, @@ -211,7 +221,7 @@ describe("service catalog", () => { const path = yield* Path.Path; const root = yield* fs.makeTempDirectoryScoped({ prefix: "catalog-mail-traversal-" }); const recipe = yield* makeServiceRecipe( - { service: "mail", config: {} }, + { service: "mail", pin: selection.mail, config: {} }, { ...options(root), instanceId: "../escaped" }, ); const instance = yield* makeStandaloneService(recipe.definition, { diff --git a/packages/stack/src/services/Pooler.integration.test.ts b/packages/stack/src/services/Pooler.integration.test.ts index 63248ad61f..343b9249be 100644 --- a/packages/stack/src/services/Pooler.integration.test.ts +++ b/packages/stack/src/services/Pooler.integration.test.ts @@ -9,6 +9,7 @@ import { makeDockerTcpRelay } from "../../tests/docker-relay.ts"; import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-pooler", @@ -37,8 +38,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -64,6 +65,7 @@ describe("service catalog", () => { const poolerRecipe = yield* makeServiceRecipe( { service: "pooler", + pin: selection.pooler, config: { databaseUrl: runtime === "native" ? nativeDatabaseUrl : dockerDatabaseUrl, jwtSecret: secret, diff --git a/packages/stack/src/services/ProcessRecipe.integration.test.ts b/packages/stack/src/services/ProcessRecipe.integration.test.ts index 93c38327e5..694d40496f 100644 --- a/packages/stack/src/services/ProcessRecipe.integration.test.ts +++ b/packages/stack/src/services/ProcessRecipe.integration.test.ts @@ -51,6 +51,7 @@ import { import type { CatalogOptions, RecipeCreation } from "./Recipe.ts"; import * as Realtime from "./Realtime.ts"; import * as Pooler from "./Pooler.ts"; +import { selection } from "../../tests/pins.ts"; type TestCreation = RecipeCreation<"rest", Record<string, never>> & { readonly service: "rest"; @@ -68,6 +69,7 @@ if (postgrestVersion === undefined) { const creation: TestCreation = { service: "rest", + pin: selection.rest, version: postgrestVersion, config: {}, }; @@ -150,6 +152,7 @@ describe("ProcessRecipe launch cleanup", () => { const container: ContainerRuntime = { prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launchCommand: () => Effect.gen(function* () { const launch = yield* Ref.updateAndGet(startupLaunches, (value) => value + 1); @@ -437,6 +440,7 @@ const startupContainer = (tool: { }): ContainerRuntime => ({ prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launch: () => Effect.die("the main process must not launch after a failed startup"), launchCommand: () => Effect.succeed({ @@ -456,6 +460,7 @@ const startupContainer = (tool: { const realtimeService = Effect.fn(function* (container: ContainerRuntime) { const creation: Realtime.Creation = { service: "realtime", + pin: selection.realtime, config: { databaseUrl: "postgresql://postgres:postgres@host.docker.internal:54322/postgres" }, }; const recipe = yield* makeProcessRecipe( @@ -503,7 +508,10 @@ const nativeFixtureArtifact = Effect.fn(function* ( : undefined; if (target === undefined) return yield* Effect.fail(`Unsupported test platform: ${platformName}`); - const { releaseVersion, natives } = yield* resolveArtifact({ service: artifact.service }); + const { releaseVersion, natives } = yield* resolveArtifact({ + service: artifact.service, + pin: selection[artifact.service], + }); const request: ArtifactRequest = { key: `slim-services/${artifact.name}/${releaseVersion}/${target}`, requiredRuntimePaths: Object.keys(artifact.files), @@ -954,6 +962,7 @@ describe("process recipe startup", () => { yield* Effect.addFinalizer(() => Scope.close(testScope, Exit.void)); const creation: Pooler.Creation = { service: "pooler", + pin: selection.pooler, config: { databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", jwtSecret: "pooler-collision-test-secret-with-more-than-32-characters", @@ -1081,6 +1090,7 @@ describe("process recipe startup", () => { const startupLaunches = yield* Ref.make<ReadonlyArray<string>>([]); const creation: Pooler.Creation = { service: "pooler", + pin: selection.pooler, config: { databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", jwtSecret: "pooler-collision-test-secret-with-more-than-32-characters", @@ -1219,6 +1229,7 @@ describe("process recipe startup", () => { }; const creation: Pooler.Creation = { service: "pooler", + pin: selection.pooler, config: { databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", jwtSecret: "pooler-collision-test-secret-with-more-than-32-characters", @@ -1287,6 +1298,7 @@ describe("process recipe startup", () => { const startupLaunches = yield* Ref.make<ReadonlyArray<string>>([]); const creation: Pooler.Creation = { service: "pooler", + pin: selection.pooler, config: { databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", jwtSecret: "pooler-collision-test-secret-with-more-than-32-characters", @@ -1397,6 +1409,7 @@ describe("process recipe startup", () => { }; const creation: Pooler.Creation = { service: "pooler", + pin: selection.pooler, config: { databaseUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", jwtSecret: "pooler-collision-test-secret-with-more-than-32-characters", diff --git a/packages/stack/src/services/ProcessRecipe.ts b/packages/stack/src/services/ProcessRecipe.ts index e94e1cdd02..792fa0718f 100644 --- a/packages/stack/src/services/ProcessRecipe.ts +++ b/packages/stack/src/services/ProcessRecipe.ts @@ -434,13 +434,13 @@ export const makeProcessRecipe = <C extends RecipeCreation<ServiceKind, unknown> if (spec.prepare !== undefined) yield* spec.prepare(candidate); const resolved = yield* resolveArtifact({ service: candidate.service, - version: candidate.version, + pin: candidate.pin, }).pipe(Effect.mapError((cause) => serviceError("prepare", cause))); if (options.runtime === "native") { // Ahead-of-time warm-up only: downloads and publishes the generation but pins nothing. // `launch` resolves (or prepares) and pins its own copy right before it spawns. yield* prepareNativeArtifact( - { service: candidate.service, version: candidate.version }, + { service: candidate.service, pin: candidate.pin }, options.cacheRoot, options.platform, ).pipe( @@ -473,7 +473,7 @@ export const makeProcessRecipe = <C extends RecipeCreation<ServiceKind, unknown> // `context.scope`), so closing it always runs their cleanup before releasing the pin. const launchScope = yield* Scope.fork(context.scope, "sequential"); const artifact = yield* useNativeArtifact( - { service: context.config.service, version: context.config.version }, + { service: context.config.service, pin: context.config.pin }, options.cacheRoot, options.platform, ).pipe( @@ -778,7 +778,7 @@ export const makeProcessRecipe = <C extends RecipeCreation<ServiceKind, unknown> return yield* serviceError("launch", "Container runtime unavailable"); const resolved = yield* resolveArtifact({ service: context.config.service, - version: context.config.version, + pin: context.config.pin, }).pipe(Effect.mapError((cause) => serviceError("launch", cause))); const containerDesired = startupEndpointsFor(context.config, spec, { container: true }); for (const process of spec.startupCommands) { diff --git a/packages/stack/src/services/Realtime.integration.test.ts b/packages/stack/src/services/Realtime.integration.test.ts index b0112e12cd..03c0b72c80 100644 --- a/packages/stack/src/services/Realtime.integration.test.ts +++ b/packages/stack/src/services/Realtime.integration.test.ts @@ -8,6 +8,7 @@ import { makeDockerHttpRelay, makeDockerTcpRelay } from "../../tests/docker-rela import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; +import { selection } from "../../tests/pins.ts"; const options = (root: string) => ({ stackId: "catalog-realtime", @@ -36,8 +37,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -55,7 +56,11 @@ describe("service catalog", () => { const databaseUrl = `postgresql://supabase_admin:postgres@${databaseRelay.host}:${databaseRelay.port}/postgres`; const realtimeRecipe = yield* makeServiceRecipe( - { service: "realtime", config: { databaseUrl, jwtSecret: secret } }, + { + service: "realtime", + pin: selection.realtime, + config: { databaseUrl, jwtSecret: secret }, + }, dockerOptions(root), ); const realtime = yield* makeStandaloneService(realtimeRecipe.definition, { @@ -73,7 +78,7 @@ describe("service catalog", () => { expect(realtimeResponse.status).toBe(200); const pgmetaRecipe = yield* makeServiceRecipe( - { service: "pgmeta", config: { databaseUrl } }, + { service: "pgmeta", pin: selection.pgmeta, config: { databaseUrl } }, dockerOptions(root), ); const pgmeta = yield* makeStandaloneService(pgmetaRecipe.definition, { @@ -95,6 +100,7 @@ describe("service catalog", () => { const studioRecipe = yield* makeServiceRecipe( { service: "studio", + pin: selection.studio, config: { databaseUrl, pgmetaUrl: `http://${pgmetaRelay.host}:${pgmetaRelay.port}`, diff --git a/packages/stack/src/services/Realtime.unit.test.ts b/packages/stack/src/services/Realtime.unit.test.ts index bd2edb5d9a..719d907aa8 100644 --- a/packages/stack/src/services/Realtime.unit.test.ts +++ b/packages/stack/src/services/Realtime.unit.test.ts @@ -2,11 +2,13 @@ import { expect, it } from "@effect/vitest"; import { Effect } from "effect"; import type { ServiceEndpoint } from "./Recipe.ts"; import { makeSpec, type Creation } from "./Realtime.ts"; +import { selection } from "../../tests/pins.ts"; const realtimeEnv = (ipVersion: Creation["config"]["ipVersion"]) => makeSpec().env( { service: "realtime", + pin: selection.realtime, config: { databaseUrl: "postgresql://supabase_admin:postgres@host.docker.internal:54322/postgres", ...(ipVersion === undefined ? {} : { ipVersion }), diff --git a/packages/stack/src/services/Recipe.ts b/packages/stack/src/services/Recipe.ts index 3545f64ee6..88e4b96ef3 100644 --- a/packages/stack/src/services/Recipe.ts +++ b/packages/stack/src/services/Recipe.ts @@ -1,6 +1,6 @@ import { Data, Schema } from "effect"; import type { Effect, PubSub, Ref, Scope } from "effect"; -import type { ServiceKind } from "../Artifacts.ts"; +import { ArtifactPin, type ServiceKind } from "../Artifacts.ts"; import type { PortError } from "../Ports.ts"; import type { LaunchOutput } from "../runtime/Session.ts"; import type { ServiceDefinition } from "../Service.ts"; @@ -37,7 +37,7 @@ export const serviceCreation = < ) => Schema.Struct({ service: Schema.Literal(service), - version: Schema.optionalKey(Schema.String), + pin: ArtifactPin, config, endpoints: Schema.optionalKey(endpoints), }); @@ -76,7 +76,7 @@ export interface CatalogRecipe<C> { export interface RecipeCreation<K extends ServiceKind, C> { readonly service: K; - readonly version?: string; + readonly pin: ArtifactPin; readonly config: C; readonly endpoints?: unknown; } diff --git a/packages/stack/src/services/Rest.integration.test.ts b/packages/stack/src/services/Rest.integration.test.ts index baa98e8a88..1a1156c960 100644 --- a/packages/stack/src/services/Rest.integration.test.ts +++ b/packages/stack/src/services/Rest.integration.test.ts @@ -13,6 +13,7 @@ import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { httpHost } from "../../tests/helpers/endpoint.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; const makeTestState = (root: string) => Layer.build(StackNamespace.layer({ root })).pipe( @@ -69,8 +70,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -130,6 +131,7 @@ describe("service catalog", () => { const restRecipe = yield* makeServiceRecipe( { service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://supabase_admin:postgres@" + @@ -212,8 +214,8 @@ describe("service catalog", () => { const databaseRecipe = yield* makeServiceRecipe( { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -266,6 +268,7 @@ END $$;`); const restRecipe = yield* makeServiceRecipe( { service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://supabase_admin:postgres@127.0.0.1/postgres?host=" + diff --git a/packages/stack/src/services/ServiceIdentity.unit.test.ts b/packages/stack/src/services/ServiceIdentity.unit.test.ts index be63b159ae..5bb4575414 100644 --- a/packages/stack/src/services/ServiceIdentity.unit.test.ts +++ b/packages/stack/src/services/ServiceIdentity.unit.test.ts @@ -5,10 +5,12 @@ import type { ServiceEndpoint } from "./Recipe.ts"; import * as Rest from "./Rest.ts"; import * as Storage from "./Storage.ts"; import * as Studio from "./Studio.ts"; +import { selection } from "../../tests/pins.ts"; it("forwards PostgREST JWKS through its consumed JWT secret setting", () => { const creation = Schema.decodeSync(Rest.Creation)({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://localhost/postgres", jwtSecret: "active-hmac-secret", @@ -24,6 +26,7 @@ it("forwards PostgREST JWKS through its consumed JWT secret setting", () => { it("forwards a PostgREST HMAC secret when JWKS is omitted", () => { const creation = Schema.decodeSync(Rest.Creation)({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://localhost/postgres", jwtSecret: "active-hmac-secret" }, }); const env = Effect.runSync(Rest.makeSpec().env(creation, new Map(), false)); @@ -36,6 +39,7 @@ it.effect("probes PostgREST readiness on its admin server port", () => const spec = Rest.makeSpec(); const creation = yield* Schema.decodeEffect(Rest.Creation)({ service: "rest", + pin: selection.rest, config: { databaseUrl: "postgresql://localhost/postgres", jwtSecret: "active-hmac-secret" }, }); const endpoints: ReadonlyMap<string, ServiceEndpoint> = new Map([ @@ -54,6 +58,7 @@ it.effect("probes PostgREST readiness on its admin server port", () => it("forwards Realtime JWKS while retaining its default JWT secret", () => { const creation = Schema.decodeSync(Realtime.Creation)({ service: "realtime", + pin: selection.realtime, config: { databaseUrl: "postgresql://localhost/postgres", jwks: "active-jwks" }, }); const env = Effect.runSync(Realtime.makeSpec().env(creation, new Map(), false)); @@ -66,6 +71,7 @@ it.effect("forwards Storage JWKS and active anon/service-role keys", () => Effect.gen(function* () { const creation = yield* Schema.decodeEffect(Storage.Creation)({ service: "storage", + pin: selection.storage, config: { databaseUrl: "postgresql://localhost/postgres", filePath: "/tmp/storage", @@ -85,6 +91,7 @@ it.effect("forwards Storage JWKS and active anon/service-role keys", () => it("forwards active client keys to Studio without requiring a JWT secret", () => { const creation = Schema.decodeSync(Studio.Creation)({ service: "studio", + pin: selection.studio, config: { anonKey: "active-anon", serviceRoleKey: "active-service-role", diff --git a/packages/stack/src/services/StorageGateway.integration.test.ts b/packages/stack/src/services/StorageGateway.integration.test.ts index 2edbfa8472..6b7a744cc1 100644 --- a/packages/stack/src/services/StorageGateway.integration.test.ts +++ b/packages/stack/src/services/StorageGateway.integration.test.ts @@ -13,6 +13,7 @@ import { makeDockerDatabaseRoot } from "../../tests/docker-fixture.ts"; import { ownerFor } from "../../tests/owner-rpc.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { testArtifactCacheRoot } from "../../tests/artifact-cache.ts"; +import { selection } from "../../tests/pins.ts"; const cacheRoot = testArtifactCacheRoot; const jwtSecret = "storage-gateway-secret-with-at-least-32-chars"; @@ -76,8 +77,8 @@ const serveStorage = Effect.fnUntraced(function* (runtime: SavedStack["runtime"] services: [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(jwtSecret), jwtExpiry: 3600, @@ -86,6 +87,7 @@ const serveStorage = Effect.fnUntraced(function* (runtime: SavedStack["runtime"] }, { service: "storage", + pin: selection.storage, config: { filePath: storageRoot, jwtSecret, s3ProtocolEnabled: true }, endpoints: { http: { port: "auto" } }, }, diff --git a/packages/stack/src/services/Studio.integration.test.ts b/packages/stack/src/services/Studio.integration.test.ts index bf8ffe63fc..e86f1fd392 100644 --- a/packages/stack/src/services/Studio.integration.test.ts +++ b/packages/stack/src/services/Studio.integration.test.ts @@ -2,6 +2,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { describe, expect, it } from "@effect/vitest"; import { Effect, FileSystem, Layer } from "effect"; import { makeStandaloneService } from "../../tests/standalone-service.ts"; +import { selection } from "../../tests/pins.ts"; import { makeServiceRecipe } from "./Catalog.ts"; const options = (root: string) => ({ @@ -26,7 +27,11 @@ describe("studio recipe", () => { const siblingInstance = `${root}/functions-instance`; yield* fs.makeDirectory(siblingInstance, { recursive: true }); const recipe = yield* makeServiceRecipe( - { service: "studio", config: { functionsRoot: siblingInstance } }, + { + service: "studio", + pin: selection.studio, + config: { functionsRoot: siblingInstance }, + }, options(root), ); const studio = yield* makeStandaloneService(recipe.definition, { diff --git a/packages/stack/src/services/Studio.unit.test.ts b/packages/stack/src/services/Studio.unit.test.ts index e471f95578..3df34ffb7f 100644 --- a/packages/stack/src/services/Studio.unit.test.ts +++ b/packages/stack/src/services/Studio.unit.test.ts @@ -1,8 +1,13 @@ import { expect, it } from "@effect/vitest"; import { Effect } from "effect"; import { makeSpec, type Creation } from "./Studio.ts"; +import { selection } from "../../tests/pins.ts"; -const creation = (config: Creation["config"]): Creation => ({ service: "studio", config }); +const creation = (config: Creation["config"]): Creation => ({ + service: "studio", + pin: selection.studio, + config, +}); it.effect("points Studio's pg-meta connection at the bound database as the postgres role", () => Effect.gen(function* () { diff --git a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts index b1dbc2ec90..436f181c09 100644 --- a/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts +++ b/packages/stack/src/storage/DockerDatabaseStorage.integration.test.ts @@ -22,26 +22,26 @@ import { } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import * as TestClock from "effect/testing/TestClock"; -import { postgresVersion, resolveArtifact } from "../Artifacts.ts"; +import { resolveArtifact, type ArtifactPin } from "../Artifacts.ts"; import { makeContainerRuntime, removeStackContainers, type EngineTarget, } from "../runtime/Container.ts"; +import { databaseIdentity, recordDatabaseIdentity } from "../internal/database-reuse.ts"; import { makeDatabaseSnapshots } from "../services/DatabaseSnapshot.ts"; import { makeDockerDatabaseStorage } from "./DockerDatabaseStorage.ts"; import { makeDockerHelperRegistry } from "./DockerHelperRegistry.ts"; import { engineTarget, testEngine } from "../../tests/engine-target.ts"; import { removeTestRunVolumes } from "../../tests/docker-volume-run.ts"; +import { database15, orioledb, selection } from "../../tests/pins.ts"; import type { DockerHelperRegistry } from "./DockerHelperRegistry.ts"; // An unpinned target: these tests exercise storage marker/volume logic, not endpoint pinning. const dockerTarget: EngineTarget = { engine: "docker", argv: [], daemonId: "test-daemon-id" }; const containerTarget: EngineTarget = { ...engineTarget, argv: [] }; -const postgresImage = (version: string) => - resolveArtifact({ service: "database", version: postgresVersion(version) }).pipe( - Effect.map(({ image }) => image), - ); +const postgresImage = (pin: ArtifactPin) => + resolveArtifact({ service: "database", pin }).pipe(Effect.map(({ image }) => image)); const Marker = Schema.Struct({ backend: Schema.Literals(["docker", "host"]), volume: Schema.optionalKey(Schema.String), @@ -204,7 +204,7 @@ const ownerDeathScenario = (mode: "instance" | "shared" | "stall-before-start") const filter = `label=com.supabase.stack=${stackId}`; // Pull before spawning so a cold image cache does not consume the owner's readiness timeout. const container = yield* makeContainerRuntime({ target: dockerTarget, root }); - yield* container.prepareImage(yield* postgresImage("17")); + yield* container.prepareImage(yield* postgresImage(selection.database)); const child = yield* spawner.spawn( ChildProcess.make( process.execPath, @@ -397,12 +397,13 @@ describe.runIf(testEngine === "docker")( container: { prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launch: () => Effect.die("unused"), launchCommand: () => Effect.die("unused"), }, }); - yield* storage.prepare("17"); - const operation = yield* storage.removeData("17").pipe(Effect.forkScoped); + yield* storage.prepare(selection.database); + const operation = yield* storage.removeData(selection.database).pipe(Effect.forkScoped); yield* Deferred.await(started); yield* Effect.sync(() => operation.interruptUnsafe()); yield* Deferred.succeed(release, undefined); @@ -468,12 +469,15 @@ describe.runIf(testEngine === "docker")( container: { prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launch: () => Effect.die("unused"), launchCommand: () => Effect.die("unused"), }, }); - yield* storage.prepare("17"); - const operation = yield* storage.removeData("17").pipe(Effect.flip, Effect.forkScoped); + yield* storage.prepare(selection.database); + const operation = yield* storage + .removeData(selection.database) + .pipe(Effect.flip, Effect.forkScoped); yield* Deferred.await(waiting); yield* TestClock.adjust("30 seconds"); const failure = yield* Fiber.join(operation); @@ -560,12 +564,15 @@ describe.runIf(testEngine === "docker")( container: { prepare: () => Effect.void, prepareImage: (image) => Effect.succeed(image), + ensureImage: (image) => Effect.succeed({ image, pulled: false }), launch: () => Effect.die("unused"), launchCommand: () => Effect.die("unused"), }, }).pipe(Scope.provide(owner)); - yield* storage.prepare("17"); - const operation = yield* storage.removeData("17").pipe(Effect.flip, Effect.forkScoped); + yield* storage.prepare(selection.database); + const operation = yield* storage + .removeData(selection.database) + .pipe(Effect.flip, Effect.forkScoped); yield* Deferred.await(creating); yield* TestClock.adjust("30 seconds"); const failure = yield* Fiber.join(operation); @@ -622,8 +629,8 @@ describe.runIf(testEngine === "docker")( spawner, }); - yield* storage.prepare("17"); - yield* storage.removeData("17"); + yield* storage.prepare(selection.database); + yield* storage.removeData(selection.database); const create = engine.commands.find((args) => args[0] === "create"); const shellIndex = create?.indexOf("/bin/sh") ?? -1; @@ -641,7 +648,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("15"); + const helperImage = yield* postgresImage(database15); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-pg15-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -677,8 +684,8 @@ describe.runIf(testEngine === "docker")( yield* Scope.addFinalizer( ownerScope, Effect.gen(function* () { - yield* source.destroyData("15").pipe(Effect.ignore); - yield* target.destroyData("15").pipe(Effect.ignore); + yield* source.destroyData(database15).pipe(Effect.ignore); + yield* target.destroyData(database15).pipe(Effect.ignore); const marker = yield* fs .readFileString(path.join(sourceRoot, ".supabase-database-storage.json")) .pipe(Effect.option); @@ -695,7 +702,7 @@ describe.runIf(testEngine === "docker")( ), ); - yield* source.prepare("15"); + yield* source.prepare(database15); const sourceMarker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(sourceRoot, ".supabase-database-storage.json")), ); @@ -711,15 +718,15 @@ describe.runIf(testEngine === "docker")( "-c", `set -eu; printf 15 > ${quote(`/store/${sourceMarker.namespace}/data/PG_VERSION`)}; printf pg15-source > ${quote(`/store/${sourceMarker.namespace}/data/fixture`)}`, ]); - yield* source.markInitialized("15"); + yield* source.markInitialized(database15); yield* fs.writeFileString( path.join(sourceRoot, ".supabase-database-ready.json"), - '{"version":"15","runtime":"docker","profile":"supabase"}', + `{"runtime":"docker","profile":"supabase"}`, ); - yield* source.saveSnapshot("15", "restore-me"); + yield* source.saveSnapshot(database15, "restore-me"); - yield* target.prepare("15"); - expect(yield* target.restoreSnapshot("15", "restore-me")).toBe(true); + yield* target.prepare(database15); + expect(yield* target.restoreSnapshot(database15, "restore-me")).toBe(true); const targetMarker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(targetRoot, ".supabase-database-storage.json")), ); @@ -734,8 +741,8 @@ describe.runIf(testEngine === "docker")( "-c", 'test "$(cat /store/PG_VERSION)" = 15 && test "$(cat /store/fixture)" = pg15-source', ]); - yield* target.destroyData("15"); - yield* source.destroyData("15"); + yield* target.destroyData(database15); + yield* source.destroyData(database15); yield* engine(["volume", "rm", sourceMarker.volume]); }), ).pipe(Effect.provide(NodeServices.layer)), @@ -747,7 +754,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -790,8 +797,8 @@ describe.runIf(testEngine === "docker")( yield* Scope.addFinalizer( ownerScope, Effect.gen(function* () { - yield* source.destroyData("17").pipe(Effect.ignore); - yield* target.destroyData("17").pipe(Effect.ignore); + yield* source.destroyData(selection.database).pipe(Effect.ignore); + yield* target.destroyData(selection.database).pipe(Effect.ignore); const marker = yield* fs .readFileString(path.join(sourceRoot, ".supabase-database-storage.json")) .pipe(Effect.option); @@ -807,7 +814,7 @@ describe.runIf(testEngine === "docker")( Effect.ignore, ), ); - yield* source.prepare("17"); + yield* source.prepare(selection.database); const sourceMarker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(sourceRoot, ".supabase-database-storage.json")), ); @@ -823,12 +830,12 @@ describe.runIf(testEngine === "docker")( "-c", `set -eu; printf 17 > ${quote(`/store/${sourceMarker.namespace}/data/PG_VERSION`)}; printf source > ${quote(`/store/${sourceMarker.namespace}/data/fixture`)}; /usr/bin/busybox setfattr -n user.storage-smoke -v preserved ${quote(`/store/${sourceMarker.namespace}/data/fixture`)}`, ]); - yield* source.markInitialized("17"); + yield* source.markInitialized(selection.database); yield* fs.writeFileString( path.join(sourceRoot, ".supabase-database-ready.json"), - '{"version":"17","runtime":"docker","profile":"supabase"}', + `{"runtime":"docker","profile":"supabase"}`, ); - yield* source.saveSnapshot("17", "roundtrip"); + yield* source.saveSnapshot(selection.database, "roundtrip"); const copiedRoot = path.join(root, "copied-state", "stack", "data", "source"); const copiedStorageRoot = path.join(root, "copied-state", "stack", "data"); @@ -845,11 +852,11 @@ describe.runIf(testEngine === "docker")( copiedStorageRoot, copiedCacheRoot, ); - const mountError = yield* copied.mount("17").pipe(Effect.flip); + const mountError = yield* copied.mount(selection.database).pipe(Effect.flip); expect(mountError.message).toContain("another state directory"); - const removeError = yield* copied.removeData("17").pipe(Effect.flip); + const removeError = yield* copied.removeData(selection.database).pipe(Effect.flip); expect(removeError.message).toContain("another state directory"); - const destroyError = yield* copied.destroyData("17").pipe(Effect.flip); + const destroyError = yield* copied.destroyData(selection.database).pipe(Effect.flip); expect(destroyError.message).toContain("another state directory"); yield* engine([ "run", @@ -862,8 +869,8 @@ describe.runIf(testEngine === "docker")( `test "$(cat /store/${sourceMarker.namespace}/data/fixture)" = source`, ]); - yield* target.prepare("17"); - expect(yield* target.restoreSnapshot("17", "roundtrip")).toBe(true); + yield* target.prepare(selection.database); + expect(yield* target.restoreSnapshot(selection.database, "roundtrip")).toBe(true); const targetMarker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(targetRoot, ".supabase-database-storage.json")), ); @@ -879,7 +886,7 @@ describe.runIf(testEngine === "docker")( `set -eu; test "$(cat /store/fixture)" = source; /usr/bin/busybox setfattr -x user.storage-smoke /store/fixture`, ]); - yield* target.removeData("unsupported"); + yield* target.removeData(selection.database); const resetMarker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(targetRoot, ".supabase-database-storage.json")), ); @@ -896,16 +903,16 @@ describe.runIf(testEngine === "docker")( "-c", 'descriptor=$(/usr/bin/busybox find /store -name descriptor.json -print -quit); printf corrupt > "$descriptor"', ]); - const corrupt = yield* target.restoreSnapshot("17", "roundtrip").pipe( + const corrupt = yield* target.restoreSnapshot(selection.database, "roundtrip").pipe( Effect.matchEffect({ onFailure: () => Effect.succeed(true), onSuccess: () => Effect.succeed(false), }), ); expect(corrupt).toBe(true); - yield* source.saveSnapshot("17", "roundtrip"); + yield* source.saveSnapshot(selection.database, "roundtrip"); - yield* target.restoreSnapshot("17", "roundtrip"); + yield* target.restoreSnapshot(selection.database, "roundtrip"); yield* engine([ "run", "--rm", @@ -916,13 +923,13 @@ describe.runIf(testEngine === "docker")( "-c", `rm -rf ${quote(`/store/${targetMarker.namespace}/data`)}`, ]); - const missing = yield* target.prepare("17").pipe(Effect.exit); + const missing = yield* target.prepare(selection.database).pipe(Effect.exit); expect(missing).toSatisfy((exit) => Exit.isFailure(exit)); - yield* target.removeData("17"); - yield* source.saveSnapshot("17", "retention-first"); - yield* source.saveSnapshot("17", "retention-second"); - yield* source.saveSnapshot("17", "retention-third"); + yield* target.removeData(selection.database); + yield* source.saveSnapshot(selection.database, "retention-first"); + yield* source.saveSnapshot(selection.database, "retention-second"); + yield* source.saveSnapshot(selection.database, "retention-third"); yield* engine([ "run", "--rm", @@ -933,11 +940,11 @@ describe.runIf(testEngine === "docker")( "-c", `test "$(/usr/bin/busybox find ${quote(`/store/${sourceMarker.cacheNamespace}/entries`)} -mindepth 1 -maxdepth 1 -type d | /usr/bin/busybox wc -l)" -eq 3`, ]); - expect(yield* target.restoreSnapshot("17", "roundtrip")).toBe(false); - yield* source.saveSnapshot("17", "roundtrip"); - yield* source.destroyData("17"); - expect(yield* target.restoreSnapshot("17", "roundtrip")).toBe(true); - yield* target.destroyData("17"); + expect(yield* target.restoreSnapshot(selection.database, "roundtrip")).toBe(false); + yield* source.saveSnapshot(selection.database, "roundtrip"); + yield* source.destroyData(selection.database); + expect(yield* target.restoreSnapshot(selection.database, "roundtrip")).toBe(true); + yield* target.destroyData(selection.database); yield* engine(["volume", "rm", sourceMarker.volume]); }), ).pipe(Effect.provide(NodeServices.layer)), @@ -978,7 +985,7 @@ describe.runIf(testEngine === "docker")( spawner, }); yield* storage - .prepare("17") + .prepare(selection.database) .pipe( Effect.provide( ConfigProvider.layer( @@ -1026,8 +1033,8 @@ describe.runIf(testEngine === "docker")( container: undefined, spawner, }); - yield* storage.removeData("17"); - yield* storage.destroyData("17"); + yield* storage.removeData(selection.database); + yield* storage.destroyData(selection.database); expect(yield* fs.exists(path.join(instanceRoot, ".supabase-database-storage.json"))).toBe( false, ); @@ -1050,7 +1057,7 @@ describe.runIf(testEngine === "docker")( const cacheRoot = path.join(root, "cache"); yield* fs.makeDirectory(instanceRoot, { recursive: true }); const markerPath = path.join(instanceRoot, ".supabase-database-storage.json"); - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const storage = yield* makeDockerDatabaseStorage({ runtime: "docker", target: dockerTarget, @@ -1071,8 +1078,10 @@ describe.runIf(testEngine === "docker")( // Removes the real volume the seeding step below creates, rather than leaving it for // the global test-run teardown, since a container that mounted it may not yet be // reaped by the time that teardown runs. - yield* Effect.addFinalizer(() => storage.destroyData("17").pipe(Effect.ignore)); - yield* storage.prepare("17"); + yield* Effect.addFinalizer(() => + storage.destroyData(selection.database).pipe(Effect.ignore), + ); + yield* storage.prepare(selection.database); const before = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(markerPath), ); @@ -1173,7 +1182,7 @@ describe.runIf(testEngine === "docker")( (yield* fs.readDirectory(instanceRoot)).filter((entry) => entry.endsWith(".tmp")), ).toHaveLength(1); - yield* storage.markInitialized("17"); + yield* storage.markInitialized(selection.database); const afterRetry = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(markerPath), ); @@ -1189,7 +1198,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-unmarked-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -1236,11 +1245,11 @@ describe.runIf(testEngine === "docker")( }); const markerPath = path.join(instanceRoot, ".supabase-database-storage.json"); - const failure = yield* storage.prepare("17").pipe(Effect.flip); + const failure = yield* storage.prepare(selection.database).pipe(Effect.flip); expect(failure.message).toContain("without a storage marker"); expect(yield* fs.exists(markerPath)).toBe(false); - yield* storage.destroyData("17"); + yield* storage.destroyData(selection.database); expect(yield* fs.exists(dataRoot)).toBe(false); expect(yield* fs.exists(checkpointsRoot)).toBe(false); expect(yield* fs.exists(markerPath)).toBe(false); @@ -1279,7 +1288,7 @@ describe.runIf(testEngine === "docker")( spawner: yield* ChildProcessSpawner.ChildProcessSpawner, }); - const failure = yield* storage.prepare("17").pipe(Effect.flip); + const failure = yield* storage.prepare(selection.database).pipe(Effect.flip); expect(failure.message).toContain("without a storage marker"); expect(yield* fs.exists(path.join(instanceRoot, ".supabase-database-storage.json"))).toBe( false, @@ -1328,7 +1337,7 @@ describe.runIf(testEngine === "docker")( spawner, helpers: helperRegistry, }); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(instanceRoot, ".supabase-database-storage.json")), ); @@ -1346,8 +1355,8 @@ describe.runIf(testEngine === "docker")( if (helper === undefined) return yield* new DockerTestError({ message: "Owned helper was not discoverable" }); yield* engine(["rm", "-f", helper]); - yield* storage.prepare("17"); - yield* storage.destroyData("17"); + yield* storage.prepare(selection.database); + yield* storage.destroyData(selection.database); yield* Scope.close(helperScope, Exit.void); yield* engine(["volume", "rm", volume]); }), @@ -1419,8 +1428,8 @@ describe.runIf(testEngine === "docker")( const secondStorage = yield* makeStorage(secondHelpers).pipe( Effect.provideService(Scope.Scope, secondScope), ); - yield* firstStorage.prepare("17"); - yield* secondStorage.prepare("17"); + yield* firstStorage.prepare(selection.database); + yield* secondStorage.prepare(selection.database); const runningHelpers = (yield* engine([ "ps", "--filter", @@ -1444,8 +1453,8 @@ describe.runIf(testEngine === "docker")( .filter((name) => name.length > 0); expect(remainingHelpers).toHaveLength(1); expect(runningHelpers).toContain(remainingHelpers[0]); - yield* secondStorage.prepare("17"); - yield* secondStorage.destroyData("17"); + yield* secondStorage.prepare(selection.database); + yield* secondStorage.destroyData(selection.database); yield* Scope.close(secondScope, Exit.void); }), ).pipe(Effect.provide(NodeServices.layer)), @@ -1457,7 +1466,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-missing-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -1489,7 +1498,7 @@ describe.runIf(testEngine === "docker")( yield* Effect.scoped( Effect.gen(function* () { const storage = yield* makeStorage(); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString( path.join(instanceRoot, ".supabase-database-storage.json"), @@ -1498,7 +1507,7 @@ describe.runIf(testEngine === "docker")( volume = marker.volume; if (volume === undefined) return yield* new DockerTestError({ message: "Docker storage volume missing" }); - const mount = yield* storage.mount("17"); + const mount = yield* storage.mount(selection.database); if (mount.type !== "volume" || mount.volumeSubpath === undefined) return yield* new DockerTestError({ message: "Docker volume backend unavailable" }); yield* engine([ @@ -1511,7 +1520,7 @@ describe.runIf(testEngine === "docker")( "-c", "printf 17 > /data/PG_VERSION", ]); - yield* storage.markInitialized("17"); + yield* storage.markInitialized(selection.database); }), ); if (volume === undefined) @@ -1521,15 +1530,15 @@ describe.runIf(testEngine === "docker")( yield* Effect.scoped( Effect.gen(function* () { const storage = yield* makeStorage(); - const prepare = yield* storage.prepare("17").pipe(Effect.exit); + const prepare = yield* storage.prepare(selection.database).pipe(Effect.exit); expect(prepare).toSatisfy((exit) => Exit.isFailure(exit)); - const removeError = yield* storage.removeData("17").pipe(Effect.flip); + const removeError = yield* storage.removeData(selection.database).pipe(Effect.flip); expect(removeError.message).toMatch(/no such volume/iu); expect( yield* engine(["volume", "inspect", removedVolume]).pipe(Effect.exit), ).toSatisfy((exit) => Exit.isFailure(exit)); - expect(yield* storage.destroyData("17").pipe(Effect.exit)).toSatisfy((exit) => - Exit.isSuccess(exit), + expect(yield* storage.destroyData(selection.database).pipe(Effect.exit)).toSatisfy( + (exit) => Exit.isSuccess(exit), ); }), ); @@ -1545,7 +1554,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-no-marker-" }); const storageRoot = path.join(root, "state", "stack", "data"); const cacheRoot = path.join(root, "cache"); @@ -1574,7 +1583,7 @@ describe.runIf(testEngine === "docker")( container, spawner, }); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); const markerPath = path.join(instanceRoot, ".supabase-database-storage.json"); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(markerPath), @@ -1582,7 +1591,7 @@ describe.runIf(testEngine === "docker")( if (marker.backend !== "docker" || marker.volume === undefined) return yield* new DockerTestError({ message: "Docker test selected host fallback" }); const volume = marker.volume; - const mount = yield* storage.mount("17"); + const mount = yield* storage.mount(selection.database); if (mount.type !== "volume" || mount.volumeSubpath === undefined) return yield* new DockerTestError({ message: "Docker volume backend unavailable" }); yield* engine([ @@ -1595,13 +1604,13 @@ describe.runIf(testEngine === "docker")( "-c", "printf 17 > /data/PG_VERSION", ]); - yield* storage.markInitialized("17"); + yield* storage.markInitialized(selection.database); // The database sleeps (the registered instance stays stopped); its storage object keeps // running, so this is the only change: the registration's on-disk marker is now gone. yield* fs.remove(markerPath); - yield* storage.destroyData("17"); + yield* storage.destroyData(selection.database); const remaining = yield* engine([ "run", @@ -1627,7 +1636,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); // `root` is deleted entirely below; nest it under the scope's own temp dir so the test // framework's own cleanup of that outer, still-present directory does not fail. const tempDir = yield* fs.makeTempDirectoryScoped({ @@ -1661,7 +1670,7 @@ describe.runIf(testEngine === "docker")( container, spawner, }); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); const markerPath = path.join(instanceRoot, ".supabase-database-storage.json"); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(markerPath), @@ -1669,7 +1678,7 @@ describe.runIf(testEngine === "docker")( if (marker.backend !== "docker" || marker.volume === undefined) return yield* new DockerTestError({ message: "Docker test selected host fallback" }); const volume = marker.volume; - const mount = yield* storage.mount("17"); + const mount = yield* storage.mount(selection.database); if (mount.type !== "volume" || mount.volumeSubpath === undefined) return yield* new DockerTestError({ message: "Docker volume backend unavailable" }); yield* engine([ @@ -1682,13 +1691,13 @@ describe.runIf(testEngine === "docker")( "-c", "printf 17 > /data/PG_VERSION", ]); - yield* storage.markInitialized("17"); + yield* storage.markInitialized(selection.database); // The database sleeps; its storage object keeps running (the resolved identity is // already in memory), then the whole state root disappears from under the live owner. yield* fs.remove(root, { recursive: true, force: true }); - yield* storage.destroyData("17"); + yield* storage.destroyData(selection.database); const remaining = yield* engine([ "run", @@ -1712,7 +1721,7 @@ describe.runIf(testEngine === "docker")( const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-reopen-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -1763,8 +1772,8 @@ describe.runIf(testEngine === "docker")( yield* Effect.scoped( Effect.gen(function* () { const source = yield* makeStorage("source", sourceRoot); - yield* source.prepare("17"); - const mount = yield* source.mount("17"); + yield* source.prepare(selection.database); + const mount = yield* source.mount(selection.database); if (mount.type !== "volume" || mount.volumeSubpath === undefined) return yield* new DockerTestError({ message: "Docker volume backend unavailable" }); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( @@ -1783,12 +1792,12 @@ describe.runIf(testEngine === "docker")( "-c", "printf 17 > /data/PG_VERSION; printf reopened > /data/fixture", ]); - yield* source.markInitialized("17"); + yield* source.markInitialized(selection.database); yield* fs.writeFileString( path.join(sourceRoot, ".supabase-database-ready.json"), - '{"version":"17","runtime":"docker","profile":"supabase"}', + `{"runtime":"docker","profile":"supabase"}`, ); - yield* source.saveSnapshot("17", "reopened"); + yield* source.saveSnapshot(selection.database, "reopened"); }), ); yield* Effect.scoped( @@ -1800,19 +1809,23 @@ describe.runIf(testEngine === "docker")( sourceRoot, changedCacheRoot, ); - yield* changedCache.prepare("17"); - yield* changedCache.removeData("17"); - expect(yield* changedCache.restoreSnapshot("17", "reopened")).toBe(false); + yield* changedCache.prepare(selection.database); + yield* changedCache.removeData(selection.database); + expect(yield* changedCache.restoreSnapshot(selection.database, "reopened")).toBe( + false, + ); const source = yield* makeStorage("source", sourceRoot); - yield* source.prepare("17"); - yield* source.removeData("17"); - expect(yield* source.restoreSnapshot("17", "reopened")).toBe(true); - yield* source.saveSnapshot("17", "reopened-again"); - yield* source.destroyData("17"); + yield* source.prepare(selection.database); + yield* source.removeData(selection.database); + expect(yield* source.restoreSnapshot(selection.database, "reopened")).toBe(true); + yield* source.saveSnapshot(selection.database, "reopened-again"); + yield* source.destroyData(selection.database); const target = yield* makeStorage("target", targetRoot); - yield* target.prepare("17"); - expect(yield* target.restoreSnapshot("17", "reopened-again")).toBe(true); - const targetMount = yield* target.mount("17"); + yield* target.prepare(selection.database); + expect(yield* target.restoreSnapshot(selection.database, "reopened-again")).toBe( + true, + ); + const targetMount = yield* target.mount(selection.database); if (targetMount.type !== "volume" || targetMount.volumeSubpath === undefined) return yield* new DockerTestError({ message: "Docker target volume unavailable" }); yield* engine([ @@ -1825,92 +1838,89 @@ describe.runIf(testEngine === "docker")( "-c", 'test "$(cat /data/fixture)" = reopened', ]); - yield* target.destroyData("17"); + yield* target.destroyData(selection.database); }), ); }), ).pipe(Effect.provide(NodeServices.layer)), ); - it.live( - "resumes unfinished volume data only on the release line its first start recorded", - () => - Effect.scoped( - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); - const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-line-" }); - const storageRoot = path.join(root, "state", "stack", "data"); - const cacheRoot = path.join(root, "cache"); - const instanceRoot = path.join(storageRoot, "line"); - yield* fs.makeDirectory(instanceRoot, { recursive: true }); - yield* fs.makeDirectory(cacheRoot, { recursive: true }); - const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; - const storage = yield* makeDockerDatabaseStorage({ - runtime: "docker", - target: dockerTarget, - stackId: `storage-line-${yield* crypto.randomUUIDv4}`, - instanceId: "line", - instanceRoot, - root: storageRoot, - cacheRoot, - fs, - path, - crypto, - container: yield* makeContainerRuntime({ target: dockerTarget, root }), - spawner, - }); - let volume: string | undefined; - yield* Effect.addFinalizer(() => - Effect.gen(function* () { - yield* storage.destroyData("17").pipe(Effect.ignore); - if (volume !== undefined) - yield* engine(["volume", "rm", volume]).pipe(Effect.ignore); - }).pipe(Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner)), - ); - yield* storage.prepare("17"); - const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( - yield* fs.readFileString(path.join(instanceRoot, ".supabase-database-storage.json")), - ); - volume = marker.volume; - if (marker.backend !== "docker" || volume === undefined) - return yield* new DockerTestError({ message: "Docker test selected host fallback" }); - // Stands in for initdb output that an interrupted first start leaves without readiness. - const writePgVersion = engine([ - "run", - "--rm", - "--mount", - `type=volume,src=${volume},dst=/store`, - helperImage, - "/bin/sh", - "-c", - `printf 17 > ${quote(`/store/${marker.namespace}/data/PG_VERSION`)}`, - ]); - yield* writePgVersion; + it.live("resumes unfinished volume data only on the identity its first start recorded", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const crypto = yield* Crypto.Crypto; + const helperImage = yield* postgresImage(selection.database); + const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-line-" }); + const storageRoot = path.join(root, "state", "stack", "data"); + const cacheRoot = path.join(root, "cache"); + const instanceRoot = path.join(storageRoot, "line"); + yield* fs.makeDirectory(instanceRoot, { recursive: true }); + yield* fs.makeDirectory(cacheRoot, { recursive: true }); + const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; + const storage = yield* makeDockerDatabaseStorage({ + runtime: "docker", + target: dockerTarget, + stackId: `storage-line-${yield* crypto.randomUUIDv4}`, + instanceId: "line", + instanceRoot, + root: storageRoot, + cacheRoot, + fs, + path, + crypto, + container: yield* makeContainerRuntime({ target: dockerTarget, root }), + spawner, + }); + let volume: string | undefined; + yield* Effect.addFinalizer(() => + Effect.gen(function* () { + yield* storage.destroyData(selection.database).pipe(Effect.ignore); + if (volume !== undefined) yield* engine(["volume", "rm", volume]).pipe(Effect.ignore); + }).pipe(Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner)), + ); + yield* storage.prepare(selection.database); + const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( + yield* fs.readFileString(path.join(instanceRoot, ".supabase-database-storage.json")), + ); + volume = marker.volume; + if (marker.backend !== "docker" || volume === undefined) + return yield* new DockerTestError({ message: "Docker test selected host fallback" }); + // Stands in for initdb output that an interrupted first start leaves without readiness. + const writePgVersion = engine([ + "run", + "--rm", + "--mount", + `type=volume,src=${volume},dst=/store`, + helperImage, + "/bin/sh", + "-c", + `printf 17 > ${quote(`/store/${marker.namespace}/data/PG_VERSION`)}`, + ]); + yield* writePgVersion; - const oriole = "17.11.0.002-orioledb"; - expect((yield* storage.prepare(oriole).pipe(Effect.flip)).message).toContain( - "PostgreSQL data from an unfinished first start belongs to release line 17, but 17.11.0.002-orioledb was requested", - ); - yield* storage.prepare("17"); + const oriole = orioledb; + expect((yield* storage.prepare(oriole).pipe(Effect.flip)).message).toContain( + `initialized by stock PostgreSQL build ${selection.database.upstreamVersion}, but OrioleDB build ${oriole.upstreamVersion} was requested`, + ); + yield* storage.prepare(selection.database); - yield* storage.removeData("17"); - yield* storage.prepare(oriole); - yield* writePgVersion; - expect((yield* storage.prepare("17").pipe(Effect.flip)).message).toContain( - "PostgreSQL data from an unfinished first start belongs to release line 17.11.0.002-orioledb, but 17 was requested", - ); - yield* storage.prepare(oriole); - }), - ).pipe(Effect.provide(NodeServices.layer)), + yield* storage.removeData(selection.database); + yield* storage.prepare(oriole); + yield* writePgVersion; + expect((yield* storage.prepare(selection.database).pipe(Effect.flip)).message).toContain( + `initialized by OrioleDB build ${oriole.upstreamVersion}, but stock PostgreSQL build ${selection.database.upstreamVersion} was requested`, + ); + yield* storage.prepare(oriole); + }), + ).pipe(Effect.provide(NodeServices.layer)), ); }, ); describe("Host database storage", { timeout: 120_000 }, () => { - it.live("resumes unfinished host data only on the release line its first start recorded", () => + it.live("resumes unfinished host data only on the identity its first start recorded", () => Effect.scoped( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -1925,8 +1935,7 @@ describe("Host database storage", { timeout: 120_000 }, () => { const writePgVersion = fs.writeFileString(path.join(data, "PG_VERSION"), "17\n"); yield* fs.makeDirectory(data, { recursive: true }); yield* fs.makeDirectory(cacheRoot, { recursive: true }); - // A host marker written before release lines were recorded; it also keeps the bind-mount - // backend on volume-capable engines. + // A host marker keeps the bind-mount backend on volume-capable engines. yield* fs.writeFileString( path.join(instanceRoot, ".supabase-database-storage.json"), yield* Schema.encodeEffect(Schema.fromJsonString(Marker))({ @@ -1955,28 +1964,30 @@ describe("Host database storage", { timeout: 120_000 }, () => { }); yield* Effect.addFinalizer(() => storage - .destroyData("17") + .destroyData(selection.database) .pipe( Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), Effect.ignore, ), ); - const oriole = "17.11.0.002-orioledb"; + const oriole = orioledb; expect((yield* storage.prepare(oriole).pipe(Effect.flip)).message).toContain( - "Unmarked PostgreSQL data cannot be verified as OrioleDB data; run `supabase stack destroy --stack-id storage-host-line` to recreate the stack — this permanently deletes its local database data", + "PostgreSQL data has no recorded identity, but OrioleDB build", + ); + expect((yield* storage.prepare(selection.database).pipe(Effect.flip)).message).toContain( + "PostgreSQL data has no recorded identity, but stock PostgreSQL build", ); - yield* storage.prepare("17"); - expect(yield* fs.readFileString(path.join(data, "PG_VERSION"))).toBe("17\n"); - // A fresh first start records its line, so its unfinished data resumes only on that line. + // A fresh first start records its identity, so its unfinished data resumes only on it. yield* fs.remove(data, { recursive: true }); yield* storage.prepare(oriole); yield* writePgVersion; - expect((yield* storage.prepare("17").pipe(Effect.flip)).message).toContain( - "PostgreSQL data from an unfinished first start belongs to release line 17.11.0.002-orioledb, but 17 was requested", + expect((yield* storage.prepare(selection.database).pipe(Effect.flip)).message).toContain( + `initialized by OrioleDB build ${oriole.upstreamVersion}, but stock PostgreSQL build ${selection.database.upstreamVersion} was requested`, ); yield* storage.prepare(oriole); + expect(yield* fs.readFileString(path.join(data, "PG_VERSION"))).toBe("17\n"); }), ).pipe(Effect.provide(NodeServices.layer)), ); @@ -1987,7 +1998,7 @@ describe("Host database storage", { timeout: 120_000 }, () => { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; const crypto = yield* Crypto.Crypto; - const helperImage = yield* postgresImage("17"); + const helperImage = yield* postgresImage(selection.database); const root = yield* fs.makeTempDirectoryScoped({ prefix: "docker-storage-host-" }); const stateRoot = path.join(root, "state"); const storageRoot = path.join(stateRoot, "stack", "data"); @@ -2011,9 +2022,15 @@ describe("Host database storage", { timeout: 120_000 }, () => { ); yield* fs.writeFileString(path.join(dataRoot, "PG_VERSION"), "17\n"); yield* fs.writeFileString(path.join(dataRoot, "fixture"), "adopted"); + yield* recordDatabaseIdentity( + fs, + path, + instanceRoot, + databaseIdentity(selection.database.upstreamVersion), + ); yield* fs.writeFileString( path.join(instanceRoot, ".supabase-database-ready.json"), - `{"version":"17","runtime":"${testEngine}","profile":"supabase"}`, + `{"runtime":"${testEngine}","profile":"supabase"}`, ); const container = yield* makeContainerRuntime({ target: containerTarget, @@ -2047,12 +2064,12 @@ describe("Host database storage", { timeout: 120_000 }, () => { const ownerScope = yield* Scope.Scope; yield* Scope.addFinalizer( ownerScope, - storage.destroyData("17").pipe( + storage.destroyData(selection.database).pipe( Effect.provideService(ChildProcessSpawner.ChildProcessSpawner, spawner), Effect.catchCause((cause) => Effect.die(cause)), ), ); - yield* storage.prepare("17"); + yield* storage.prepare(selection.database); const marker = yield* Schema.decodeEffect(Schema.fromJsonString(Marker))( yield* fs.readFileString(path.join(instanceRoot, ".supabase-database-storage.json")), ); @@ -2069,28 +2086,32 @@ describe("Host database storage", { timeout: 120_000 }, () => { "stat -c '%u:%g' /instance/data/PG_VERSION", ]); if (process.platform === "linux") expect(expectedOwnership).toBe("100:101"); - yield* storage.saveSnapshot("17", "adopted"); - yield* storage.saveSnapshot("17", "checkpoint", "instance"); + yield* storage.saveSnapshot(selection.database, "adopted"); + yield* storage.saveSnapshot(selection.database, "checkpoint", "instance"); const nativeRoot = path.join(root, "native"); yield* fs.makeDirectory(path.join(nativeRoot, "data"), { recursive: true }); yield* fs.writeFileString(path.join(nativeRoot, "data", "PG_VERSION"), "17\n"); + yield* recordDatabaseIdentity( + fs, + path, + nativeRoot, + databaseIdentity(selection.database.upstreamVersion), + ); yield* fs.writeFileString( path.join(nativeRoot, ".supabase-database-ready.json"), - // `makeDatabaseSnapshots` below resolves `version: "17"` through the real catalog - // (`postgresVersion`), so the ready marker must carry that same resolved version. - `{"version":"${postgresVersion("17")}","runtime":"native","profile":"supabase"}`, + `{"runtime":"native","profile":"supabase"}`, ); const nativeSnapshots = yield* makeDatabaseSnapshots({ instanceRoot: nativeRoot, cacheRoot, runtime: "native", - version: "17", + pin: selection.database, }); yield* nativeSnapshots.saveSnapshot("native"); yield* fs.remove(path.join(nativeRoot, "data"), { recursive: true }); expect(yield* nativeSnapshots.restoreSnapshot("native")).toBe(true); - yield* storage.removeData("17"); - expect(yield* storage.restoreSnapshot("17", "adopted")).toBe(true); + yield* storage.removeData(selection.database); + expect(yield* storage.restoreSnapshot(selection.database, "adopted")).toBe(true); expect( yield* engine([ "run", @@ -2103,10 +2124,12 @@ describe("Host database storage", { timeout: 120_000 }, () => { "stat -c '%u:%g' /instance/data/PG_VERSION", ]), ).toBe(expectedOwnership); - yield* storage.removeData("17"); - expect(yield* storage.restoreSnapshot("17", "checkpoint", "instance")).toBe(true); - yield* storage.removeData("17"); - yield* storage.destroyData("unsupported"); + yield* storage.removeData(selection.database); + expect(yield* storage.restoreSnapshot(selection.database, "checkpoint", "instance")).toBe( + true, + ); + yield* storage.removeData(selection.database); + yield* storage.destroyData(selection.database); expect(yield* fs.exists(path.join(instanceRoot, ".supabase-database-storage.json"))).toBe( true, ); diff --git a/packages/stack/src/storage/DockerDatabaseStorage.ts b/packages/stack/src/storage/DockerDatabaseStorage.ts index 4b3d737e54..c5a6c145fe 100644 --- a/packages/stack/src/storage/DockerDatabaseStorage.ts +++ b/packages/stack/src/storage/DockerDatabaseStorage.ts @@ -18,9 +18,17 @@ import type { ChildProcessHandle, ChildProcessSpawner as ChildProcessSpawnerService, } from "effect/process/ChildProcessSpawner"; -import { postgresLine, postgresVersion, resolveArtifact } from "../Artifacts.ts"; -import { recreateStackAdvice, unusableDatabaseData } from "../internal/database-reuse.ts"; +import { resolveArtifact, type ArtifactPin } from "../Artifacts.ts"; +import { + databaseIdentity, + databaseIdentityMismatch, + databaseReadyMarkerFile, + readDatabaseIdentity, + recordDatabaseIdentity, + removeDatabaseIdentity, +} from "../internal/database-reuse.ts"; import { failureMessage } from "../internal/failure-message.ts"; +import { removeSchemaWriters } from "../internal/schema-writers.ts"; import { testRunLabelArgs as readTestRunLabelArgs } from "../internal/test-run-label.ts"; import * as Publication from "../namespace/Publication.ts"; import type { ContainerRuntime, EngineTarget } from "../runtime/Container.ts"; @@ -42,8 +50,6 @@ const Marker = Schema.Struct({ cacheNamespace: Schema.String, daemonId: Schema.optionalKey(Schema.String), initialized: Schema.Boolean, - /** Release line recorded before initdb; markers written before this field have none. */ - line: Schema.optionalKey(Schema.String), }); type Marker = Schema.Schema.Type<typeof Marker>; const DaemonIdentity = Schema.Struct({ @@ -80,21 +86,21 @@ export interface DatabaseStorageMount { } export interface DockerDatabaseStorage { - readonly prepare: (version: string) => Effect.Effect<void, DockerDatabaseStorageError>; + readonly prepare: (pin: ArtifactPin) => Effect.Effect<void, DockerDatabaseStorageError>; readonly needsDataChown: Effect.Effect<boolean, DockerDatabaseStorageError>; readonly mount: ( - version: string, + pin: ArtifactPin, ) => Effect.Effect<DatabaseStorageMount, DockerDatabaseStorageError>; - readonly markInitialized: (version: string) => Effect.Effect<void, DockerDatabaseStorageError>; - readonly removeData: (version: string) => Effect.Effect<void, DockerDatabaseStorageError>; - readonly destroyData: (version: string) => Effect.Effect<void, DockerDatabaseStorageError>; + readonly markInitialized: (pin: ArtifactPin) => Effect.Effect<void, DockerDatabaseStorageError>; + readonly removeData: (pin: ArtifactPin) => Effect.Effect<void, DockerDatabaseStorageError>; + readonly destroyData: (pin: ArtifactPin) => Effect.Effect<void, DockerDatabaseStorageError>; readonly saveSnapshot: ( - version: string, + pin: ArtifactPin, key: string, scope?: SnapshotScope, ) => Effect.Effect<void, DockerDatabaseStorageError>; readonly restoreSnapshot: ( - version: string, + pin: ArtifactPin, key: string, scope?: SnapshotScope, ) => Effect.Effect<boolean, DockerDatabaseStorageError>; @@ -711,14 +717,9 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") Effect.ignore, ), ); - const imageForVersion = (version: string, fallback: boolean) => - resolveArtifact({ service: "database", version: postgresVersion(version) }).pipe( + const imageForPin = (pin: ArtifactPin) => + resolveArtifact({ service: "database", pin }).pipe( Effect.map(({ image }) => image), - Effect.catchTag("ArtifactError", (cause) => - fallback - ? resolveArtifact({ service: "database" }).pipe(Effect.map(({ image }) => image)) - : Effect.fail(cause), - ), Effect.mapError((cause) => errorFor("helper", cause)), ); const acquireHelper = Effect.fn("DockerDatabaseStorage.acquireHelper")( @@ -828,15 +829,14 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") const runHelper = Effect.fn("DockerDatabaseStorage.helper")(( command: string, mounts: ReadonlyArray<DatabaseStorageMount>, - version: string, - fallback = false, + pin: ArtifactPin, ): Effect.Effect<string, DockerDatabaseStorageError> => { const cleanupAfterFailure = Effect.gen(function* () { if ((yield* Ref.get(helperId)) === undefined) return; yield* Ref.set(helperCleanupPending, true); yield* removeHelper(); }); - return imageForVersion(version, fallback).pipe( + return imageForPin(pin).pipe( Effect.flatMap((image) => { const sharedKey = volumeHelperKey(mounts, image); if (sharedKey !== undefined && options.helpers !== undefined) { @@ -911,18 +911,21 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") if (cached !== undefined) return { marker: Option.some(cached), validated: true as const }; return { marker: yield* getMarkerIfPresent, validated: false as const }; }); - const readyMarkerPath = options.path.join( - options.instanceRoot, - ".supabase-database-ready.json", - ); + const readyMarkerPath = options.path.join(options.instanceRoot, databaseReadyMarkerFile); // The host owns this file; a helper's bind-mount view can still list it after the host // removes it, which makes the helper's unlink fail. const removeReadyMarker = options.fs .remove(readyMarkerPath, { force: true }) .pipe(Effect.mapError((cause) => errorFor("reset", cause))); + // Forgotten after the data, so a crash leaves the record at least as new as the data. + const removeWriters = removeSchemaWriters( + options.fs, + options.path, + options.instanceRoot, + ).pipe(Effect.mapError((cause) => errorFor("reset", cause))); /** Unmarked data cannot start, so removal is its only in-product recovery. */ const removeUnmarkedData = ( - version: string, + pin: ArtifactPin, { checkpoints }: { readonly checkpoints: boolean }, ) => Effect.gen(function* () { @@ -935,36 +938,32 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") yield* runHelper( `set -eu; rm -rf /instance/data /instance/.supabase-restore${removeCheckpoints ? ` ${shellQuote(`/instance/${instanceSnapshotsDirectory}`)}` : ""}`, [{ source: options.instanceRoot, target: "/instance", readOnly: false }], - version, - true, + pin, ); yield* removeReadyMarker; + yield* removeDatabaseIdentity(options.fs, options.path, options.instanceRoot).pipe( + Effect.mapError((cause) => errorFor("reset", cause)), + ); + yield* removeWriters; }); const writeMarker = (marker: Marker) => encodeMarker(marker).pipe(Effect.flatMap((encoded) => publishMarker(encoded))); - const checkInitialized = Effect.fnUntraced(function* ( - pgVersion: string, - version: string, - marker: Marker, - ) { - // The database checks a present readiness marker's line before preparing storage. - const verified = yield* options.fs.exists(readyMarkerPath); - const reason = unusableDatabaseData( - { - major: pgVersion.trim(), - line: verified ? postgresLine(version) : marker.line, - initialized: marker.initialized, - }, - version, + const checkInitialized = Effect.fnUntraced(function* (pin: ArtifactPin) { + const mismatch = databaseIdentityMismatch( + options.stackId, + yield* readDatabaseIdentity(options.fs, options.path, options.instanceRoot), + pin.upstreamVersion, ); - if (reason !== undefined) - return yield* errorFor("prepare", `${reason}; ${recreateStackAdvice(options.stackId)}`); + if (mismatch !== undefined) return yield* errorFor("prepare", mismatch); }); - const recordLine = (marker: Marker, version: string) => - marker.line === postgresLine(version) - ? Effect.void - : writeMarker({ ...marker, line: postgresLine(version) }); - const setup = Effect.fn("DockerDatabaseStorage.prepare")((version: string) => + const recordIdentity = (pin: ArtifactPin) => + recordDatabaseIdentity( + options.fs, + options.path, + options.instanceRoot, + databaseIdentity(pin.upstreamVersion), + ); + const setup = Effect.fn("DockerDatabaseStorage.prepare")((pin: ArtifactPin) => Effect.gen(function* () { yield* selected; const marker = yield* getMarker; @@ -973,55 +972,53 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") if (marker.initialized) { if (!(yield* options.fs.exists(data))) return yield* errorFor("prepare", "Initialized database data is missing"); - const major = yield* runHelper( + yield* runHelper( "set -eu; if [ ! -f /instance/data/PG_VERSION ]; then echo 'Initialized PostgreSQL data is missing PG_VERSION' >&2; exit 1; fi; cat /instance/data/PG_VERSION", [ { source: options.instanceRoot, target: "/instance", readOnly: false }, { source: options.cacheRoot, target: "/cache", readOnly: false }, ], - version, + pin, ); - yield* checkInitialized(major, version, marker); + yield* checkInitialized(pin); } else if (yield* options.fs.exists(data)) { // An interrupted first start can leave initdb output behind without readiness. const major = yield* runHelper( "set -eu; if [ -f /instance/data/PG_VERSION ]; then cat /instance/data/PG_VERSION; fi", [{ source: options.instanceRoot, target: "/instance", readOnly: false }], - version, - true, + pin, ); - if (major.trim() !== "") yield* checkInitialized(major, version, marker); - else yield* recordLine(marker, version); + if (major.trim() !== "") yield* checkInitialized(pin); + else yield* recordIdentity(pin); } else { yield* options.fs.makeDirectory(data, { recursive: true, mode: 0o700 }); - yield* recordLine(marker, version); + yield* recordIdentity(pin); } return; } const store = `/store/${marker.namespace}`; const cache = `/store/${marker.cacheNamespace}`; if (marker.initialized) { - const major = yield* runHelper( + yield* runHelper( `set -eu; if [ ! -f ${shellQuote(`${store}/data/PG_VERSION`)} ]; then echo 'Initialized PostgreSQL data is missing PG_VERSION' >&2; exit 1; fi; cat ${shellQuote(`${store}/data/PG_VERSION`)}`, [{ source: marker.volume ?? "", target: "/store", readOnly: false, type: "volume" }], - version, + pin, ); - yield* checkInitialized(major, version, marker); + yield* checkInitialized(pin); } else { const major = yield* runHelper( `set -eu; if [ -f ${shellQuote(`${store}/data/PG_VERSION`)} ]; then cat ${shellQuote(`${store}/data/PG_VERSION`)}; fi; mkdir -p ${shellQuote(`${store}/data`)} ${shellQuote(`${cache}/entries`)} ${shellQuote(`${cache}/stages`)}; chown -R 100:101 ${shellQuote(`${store}/data`)}`, [{ source: marker.volume ?? "", target: "/store", readOnly: false, type: "volume" }], - version, - true, + pin, ); - if (major.trim() !== "") yield* checkInitialized(major, version, marker); - else yield* recordLine(marker, version); + if (major.trim() !== "") yield* checkInitialized(pin); + else yield* recordIdentity(pin); } }).pipe(Effect.mapError((cause) => errorFor("prepare", cause))), ); const majorVersion = (version: string) => version.split(".")[0] ?? version; - const mount = (_version: string) => + const mount = (_pin: ArtifactPin) => selected.pipe( Effect.flatMap(() => getMarker), Effect.map((marker) => @@ -1046,50 +1043,52 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") Effect.mapError((cause) => errorFor("marker", cause)), ); const markInitialized = Effect.fn("DockerDatabaseStorage.markInitialized")( - (version: string) => + (pin: ArtifactPin) => Effect.gen(function* () { const marker = yield* getMarker; if (marker.backend === "docker") { const versionFile = `/store/${marker.namespace}/data/PG_VERSION`; yield* runHelper( - `set -eu; if [ ! -f ${shellQuote(versionFile)} ]; then echo 'Database readiness requires PG_VERSION' >&2; exit 1; fi; actual=$(cat ${shellQuote(versionFile)}); if [ "$actual" != ${shellQuote(majorVersion(version))} ]; then echo 'Database PostgreSQL major does not match requested version' >&2; exit 1; fi`, + `set -eu; if [ ! -f ${shellQuote(versionFile)} ]; then echo 'Database readiness requires PG_VERSION' >&2; exit 1; fi; actual=$(cat ${shellQuote(versionFile)}); if [ "$actual" != ${shellQuote(majorVersion(pin.upstreamVersion))} ]; then echo 'Database PostgreSQL major does not match requested version' >&2; exit 1; fi`, snapshotPaths(marker).mounts, - version, + pin, ); } yield* writeMarker({ ...marker, initialized: true }); }).pipe(Effect.mapError((cause) => errorFor("ready", cause))), ); - const removeData = Effect.fn("DockerDatabaseStorage.removeData")((version: string) => + const removeData = Effect.fn("DockerDatabaseStorage.removeData")((pin: ArtifactPin) => Effect.gen(function* () { const markerOption = yield* getMarkerForRemoval; if (Option.isNone(markerOption)) - return yield* removeUnmarkedData(version, { checkpoints: false }); + return yield* removeUnmarkedData(pin, { checkpoints: false }); const marker = markerOption.value; if (marker.backend === "host") { yield* runHelper( `set -eu; rm -rf /instance/data /instance/.supabase-restore; mkdir -p /instance/data; chown 100:101 /instance/data`, snapshotPaths(marker).mounts, - version, - true, + pin, ); } else { yield* runHelper( `set -eu; rm -rf ${shellQuote(`/store/${marker.namespace}/data`)}; mkdir -p ${shellQuote(`/store/${marker.namespace}/data`)}; chown 100:101 ${shellQuote(`/store/${marker.namespace}/data`)}`, [{ source: marker.volume ?? "", target: "/store", readOnly: false, type: "volume" }], - version, - true, + pin, ); } yield* removeReadyMarker; + yield* removeDatabaseIdentity(options.fs, options.path, options.instanceRoot).pipe( + Effect.mapError((cause) => errorFor("reset", cause)), + ); + yield* removeWriters; yield* writeMarker({ ...marker, initialized: false }); }).pipe(Effect.mapError((cause) => errorFor("reset", cause))), ); - const destroyData = Effect.fn("DockerDatabaseStorage.destroyData")((version: string) => + const destroyData = Effect.fn("DockerDatabaseStorage.destroyData")((pin: ArtifactPin) => Effect.gen(function* () { const { marker: markerOption, validated } = yield* getMarkerForDestroy; if (Option.isNone(markerOption)) { - yield* removeUnmarkedData(version, { checkpoints: true }); + yield* removeUnmarkedData(pin, { checkpoints: true }); return yield* removeHelper(); } const marker = markerOption.value; @@ -1097,8 +1096,7 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") yield* runHelper( `set -eu; rm -rf /instance/data /instance/.supabase-restore ${shellQuote(`/instance/${instanceSnapshotsDirectory}`)}`, snapshotPaths(marker).mounts, - version, - true, + pin, ); yield* removeReadyMarker; yield* removeHelper(); @@ -1118,8 +1116,7 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") yield* runHelper( `set -eu; rm -rf ${shellQuote(`/store/${marker.namespace}`)}`, [{ source: marker.volume, target: "/store", readOnly: false, type: "volume" }], - version, - true, + pin, ); yield* removeHelper(); } @@ -1147,13 +1144,13 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") { source: options.cacheRoot, target: "/cache", readOnly: false as const }, ], }; - const snapshots = (store: Marker, version: string) => { + const snapshots = (store: Marker, pin: ArtifactPin) => { const paths = snapshotPaths(store); const host = store.backend === "host"; const instanceSnapshotRoot = host ? `/instance/${instanceSnapshotsDirectory}` : `/store/${store.namespace}/snapshots`; - const exec = (script: string) => runHelper(script, paths.mounts, version); + const exec = (script: string) => runHelper(script, paths.mounts, pin); return makeSnapshotStore({ backends: { cache: makeDockerSnapshotBackend({ @@ -1194,7 +1191,7 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") }, instanceRoot: options.instanceRoot, runtime: options.runtime, - version, + version: pin.upstreamVersion, }).pipe( Effect.provideService(FileSystem.FileSystem, options.fs), Effect.provideService(Path.Path, options.path), @@ -1202,22 +1199,21 @@ export const makeDockerDatabaseStorage = Effect.fn("DockerDatabaseStorage.make") ); }; const saveSnapshot = Effect.fn("DockerDatabaseStorage.saveSnapshot")( - (version: string, key: string, scope: SnapshotScope = "cache") => + (pin: ArtifactPin, key: string, scope: SnapshotScope = "cache") => Effect.gen(function* () { yield* selected; const store = yield* getMarker; if (!store.initialized) return yield* errorFor("snapshot", "Database is not initialized"); - yield* (yield* snapshots(store, version)).saveSnapshot(key, scope); + yield* (yield* snapshots(store, pin)).saveSnapshot(key, scope); }).pipe(Effect.mapError((cause) => errorFor("snapshot", cause))), ); const restoreSnapshot = Effect.fn("DockerDatabaseStorage.restoreSnapshot")( - (version: string, key: string, scope: SnapshotScope = "cache") => + (pin: ArtifactPin, key: string, scope: SnapshotScope = "cache") => Effect.gen(function* () { yield* selected; const store = yield* getMarker; - if (!(yield* (yield* snapshots(store, version)).restoreSnapshot(key, scope))) - return false; + if (!(yield* (yield* snapshots(store, pin)).restoreSnapshot(key, scope))) return false; yield* writeMarker({ ...store, initialized: true }); return true; }).pipe(Effect.mapError((cause) => errorFor("snapshot", cause))), diff --git a/packages/stack/src/storage/DockerSnapshotBackend.ts b/packages/stack/src/storage/DockerSnapshotBackend.ts index 07addab4f3..34a80c6aa3 100644 --- a/packages/stack/src/storage/DockerSnapshotBackend.ts +++ b/packages/stack/src/storage/DockerSnapshotBackend.ts @@ -37,6 +37,7 @@ const stepScript = (step: SnapshotStep, adoptOwner: string | undefined): string Adopt: ({ directory }) => adoptOwner === undefined ? ":" : `${busybox} chown -R ${adoptOwner} ${shellQuote(directory)}`, Write: ({ file, text }) => `printf '%s' ${shellQuote(text)} > ${shellQuote(file)}`, + Read: ({ file }) => `echo ${doneMarker}; ${busybox} cat ${shellQuote(file)}; exit 0`, Rename: ({ from, to, optional }) => optional ? `if [ -e ${shellQuote(from)} ] && [ ! -e ${shellQuote(to)} ]; then ${busybox} mv ${shellQuote(from)} ${shellQuote(to)}; fi` @@ -74,7 +75,7 @@ const helperError = (message: string, cause: unknown) => { const parseRun = (output: string) => Effect.gen(function* () { const [first = "", ...rest] = output.split("\n"); - if (first === doneMarker) return SnapshotRun.Completed(); + if (first === doneMarker) return SnapshotRun.Completed({ text: rest.join("\n") }); if (!first.startsWith(stopMarker)) return yield* new DatabaseSnapshotError({ operation: "helper", diff --git a/packages/stack/src/testing.integration.test.ts b/packages/stack/src/testing.integration.test.ts index fee077b4b0..66e1cef661 100644 --- a/packages/stack/src/testing.integration.test.ts +++ b/packages/stack/src/testing.integration.test.ts @@ -14,16 +14,20 @@ import { Stream, } from "effect"; import * as NetAddress from "effect/net/NetAddress"; -import { discover, type Stack } from "./effect.ts"; +import { discover, type DatabaseInstance, type Stack } from "./effect.ts"; import { makeTestStack } from "./testing.ts"; import { postgres } from "./Commands.ts"; +import { StackError } from "./Rpc.ts"; +import { LockInvalidError, writeLock } from "./Lock.ts"; +import { lockWithDatabase } from "../tests/locks.ts"; +import { database15, selection } from "../tests/pins.ts"; const sql = (stack: Stack, databaseUrl: string, command: string) => Effect.gen(function* () { const decoder = new TextDecoder(); let stdout = ""; let stderr = ""; - const result = yield* stack.commands.run(postgres.psql({ major: 17 }), { + const result = yield* stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--set", "ON_ERROR_STOP=1", "-tA", "--command", command], stdout: (bytes) => Effect.sync(() => (stdout += decoder.decode(bytes))), stderr: (bytes) => Effect.sync(() => (stderr += decoder.decode(bytes))), @@ -233,7 +237,7 @@ it.live( const port = occupied.address.port; const startup = yield* makeTestStack({ - services: [{ service: "mail", endpoints: { http: { port } } }], + services: [{ service: "mail", pin: selection.mail, endpoints: { http: { port } } }], runtime: "native", stateRoot, }).pipe(Effect.scoped, Effect.exit); @@ -243,7 +247,8 @@ it.live( ? Cause.findErrorOption(startup.cause) : Option.none(); if (Option.isNone(failure)) return yield* Effect.die("Expected a typed startup failure"); - expect(failure.value.operation).toBe("test-startup"); + expect(failure.value).toBeInstanceOf(StackError); + expect(failure.value).toMatchObject({ operation: "test-startup" }); // macOS and Windows refuse the port while claiming it; Linux reports the failed bind. expect(failure.value.message).toMatch(new RegExp(`\\b${port}\\b.*\\bin use\\b`)); expect(failure.value.message).toContain("Services: none"); @@ -291,7 +296,8 @@ it.live("rejects an unknown SUPABASE_STACK_TEST_RUNTIME before creating a stack" Effect.flip, ); - expect(failure.operation).toBe("test-stack"); + expect(failure).toBeInstanceOf(StackError); + expect(failure).toMatchObject({ operation: "test-stack" }); expect(failure.message).toContain("SUPABASE_STACK_TEST_RUNTIME"); expect(yield* discover({ stateRoot })).toEqual([]); }).pipe( @@ -299,3 +305,58 @@ it.live("rejects an unknown SUPABASE_STACK_TEST_RUNTIME before creating a stack" Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp)), ), ); + +const databaseVersion = (test: { + readonly stack: Stack; + readonly services: { readonly database: DatabaseInstance }; +}) => + Effect.gen(function* () { + const { databaseUrl } = yield* test.services.database.credentials(); + if (databaseUrl === undefined) return yield* Effect.die("Database URL missing"); + return yield* sql(test.stack, databaseUrl, "SHOW server_version"); + }); + +it.live( + "runs the versions locked in a caller-owned project root and the bundled ones without a lock", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const stateRoot = yield* fs.makeTempDirectoryScoped({ prefix: "stack-testing-lock-" }); + const unlocked = yield* fs.makeTempDirectoryScoped({ prefix: "stack-testing-unlocked-" }); + const locked = yield* fs.makeTempDirectoryScoped({ prefix: "stack-testing-locked-" }); + yield* writeLock(locked, lockWithDatabase(database15, 15)); + + const bundled = yield* makeTestStack({ runtime: "native", stateRoot, projectRoot: unlocked }); + expect(yield* databaseVersion(bundled)).toMatch(/^17\./); + const pinned = yield* makeTestStack({ runtime: "native", stateRoot, projectRoot: locked }); + expect(yield* databaseVersion(pinned)).toMatch(/^15\./); + }).pipe( + Effect.scoped, + Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp)), + ), + { timeout: 240_000 }, +); + +it.live( + "fails with the invalid-lock error instead of running bundled when the lock is malformed", + () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const stateRoot = yield* fs.makeTempDirectoryScoped({ prefix: "stack-testing-bad-lock-" }); + const projectRoot = yield* fs.makeTempDirectoryScoped({ prefix: "stack-testing-bad-root-" }); + yield* fs.makeDirectory(path.join(projectRoot, "supabase")); + yield* fs.writeFileString(path.join(projectRoot, "supabase", "supabase.lock"), "{"); + + const failure = yield* makeTestStack({ runtime: "native", stateRoot, projectRoot }).pipe( + Effect.scoped, + Effect.flip, + ); + + expect(failure).toBeInstanceOf(LockInvalidError); + expect(yield* discover({ stateRoot })).toEqual([]); + }).pipe( + Effect.scoped, + Effect.provide(Layer.merge(NodeServices.layer, NodeHttpClient.layerNodeHttp)), + ), +); diff --git a/packages/stack/src/testing.ts b/packages/stack/src/testing.ts index c36d785e40..268faf82f6 100644 --- a/packages/stack/src/testing.ts +++ b/packages/stack/src/testing.ts @@ -1,9 +1,29 @@ -import { Config, Crypto, Effect, FileSystem, Option, Path, Ref, Schema, Scope } from "effect"; +import { + Config, + Context, + Crypto, + Effect, + FileSystem, + Layer, + Option, + Path, + Redacted, + Ref, + Schema, + Scope, +} from "effect"; // oxlint-disable-next-line effecttsgo/node-builtin-import -- FileSystem exposes no OS temp or home directory path for the shared roots. import { homedir, tmpdir, userInfo } from "node:os"; import type { PlatformError } from "effect/PlatformError"; -import { defaultRuntime } from "./Artifacts.ts"; +import { defaultArtifactSelection, defaultRuntime, type ArtifactPin } from "./Artifacts.ts"; import * as StackEffect from "./effect.ts"; +import { + LockInvalidError, + LockIoError, + LockUnsupportedVersionError, + lockSelection, + readLock, +} from "./Lock.ts"; import { acquire, stackAdapter, @@ -15,10 +35,24 @@ import { type Stack, } from "./PromiseClient.ts"; import { StackError, stackError } from "./Rpc.ts"; +import { deriveStackId, resolveStackIdentity } from "./identity/Identity.ts"; +import { + DatabaseReadyMarker, + databaseIdentity, + databaseReadyMarkerFile, + recordDatabaseIdentity, +} from "./internal/database-reuse.ts"; +import { + setSchemaWriters, + type SchemaWriters, + type SchemaWriterService, +} from "./internal/schema-writers.ts"; import { allowedEndpointNames, + ServiceCreation, ServiceCreationInput as CreationSchema, } from "./services/Catalog.ts"; +import * as StackNamespace from "./StackNamespace.ts"; /** Exposed for test fakes that need to reproduce the package's own saved-stack comparison. */ export { planSupabaseComposition } from "./composition/Supabase.ts"; @@ -26,12 +60,14 @@ export { planSupabaseComposition } from "./composition/Supabase.ts"; type Kind = StackEffect.ServiceCreationInput["service"]; type Creation<K extends Kind> = Extract<StackEffect.ServiceCreationInput, { readonly service: K }>; -/** A service kind to compose, optionally with config and endpoint overrides. */ +/** A service kind to compose, optionally with a pin, config and endpoint overrides. */ export type TestService<K extends Kind = Kind> = K extends Kind ? | K | { readonly service: K; + /** Defaults to the project lock's pin of the kind, then the compiled default. */ + readonly pin?: ArtifactPin; readonly config?: Partial<Creation<K>["config"]>; readonly endpoints?: Creation<K>["endpoints"]; } @@ -77,7 +113,9 @@ export interface TestStackOptions<S> { readonly cacheRoot?: string; /** * A caller-owned project root; by default a temporary one is removed on disposal. Default - * Storage and Functions directories live in the temporary root, never in a caller's root. + * Storage and Functions directories live in the temporary root, never in a caller's root. Its + * `supabase/supabase.lock` pins the versions the stack runs; without one the bundled versions + * run, and an unreadable lock fails the stack. */ readonly projectRoot?: string; } @@ -124,6 +162,14 @@ const cacheHome = Config.option(Config.NonEmptyString("XDG_CACHE_HOME")); const testFailure = (operation: string) => (cause: unknown) => stackError(operation, cause); +const isLockError = Schema.is( + Schema.Union([LockInvalidError, LockUnsupportedVersionError, LockIoError]), +); + +/** Lock failures stay typed so callers can tell a bad lock from a startup failure. */ +const creationFailure = (cause: unknown) => + isLockError(cause) ? cause : testFailure("test-stack")(cause); + /** Local-development configuration for each kind, with every endpoint on an automatic port. */ const localCreation = Effect.fnUntraced(function* ( kind: Kind, @@ -138,7 +184,7 @@ const localCreation = Effect.fnUntraced(function* ( }); const config = kind === "database" - ? { version: "17", jwtExpiry: 3600 } + ? { jwtExpiry: 3600 } : kind === "storage" || kind === "imgproxy" ? { filePath: yield* directory("storage") } : kind === "functions" @@ -159,189 +205,195 @@ function byKind( return Object.fromEntries(members.map((member) => [member.service, member])); } -const make = Effect.fn("TestStack.make")( - function* <Required extends Kind, Optional extends Kind = never>( - options: TestStackOptions<ReadonlyArray<TestService | PlainTestService>>, - decode: (creation: unknown) => Effect.Effect<StackEffect.ServiceCreationInput, StackError>, - ) { - const fs = yield* FileSystem.FileSystem; - const path = yield* Path.Path; - const crypto = yield* Crypto.Crypto; - const runtime = - options.runtime ?? Option.getOrUndefined(yield* runtimeOverride) ?? defaultRuntime(); - // Shared roots are created owner-only, so each OS user needs its own. - const user = process.getuid?.() ?? userInfo().username; - const stateRoot = options.stateRoot ?? path.join(tmpdir(), `supabase-stack-tests-${user}`); - const cacheRoot = - options.cacheRoot ?? - path.join( - Option.getOrElse(yield* cacheHome, () => path.join(homedir(), ".cache")), - "supabase-stack-artifacts", - ); - // Native Edge Runtime on Linux cannot read project files below /tmp. - const temporaryRoot = fs.makeTempDirectoryScoped({ - prefix: "supabase-test-stack-", - ...(runtime === "native" && process.platform === "linux" ? { directory: homedir() } : {}), - }); - const projectRoot = options.projectRoot ?? (yield* temporaryRoot); - const dataRoot = - options.projectRoot === undefined - ? Effect.succeed(projectRoot) - : yield* Effect.cached(temporaryRoot); - const services: ReadonlyArray<TestService | PlainTestService> = options.services ?? [ - "database", - ]; - const creations = yield* Effect.forEach(services, (service) => - Effect.gen(function* () { - const kind = typeof service === "string" ? service : service.service; - const local = yield* localCreation(kind, dataRoot); - return yield* decode({ - service: kind, - config: { ...local.config, ...(typeof service === "string" ? {} : service.config) }, - endpoints: { - ...local.endpoints, - ...(typeof service === "string" ? {} : service.endpoints), - }, - }); - }), - ); - const stack = yield* Effect.acquireRelease( - StackEffect.create({ - projectRoot, - stateRoot, - cacheRoot, - runtime, - name: `test-${yield* crypto.randomUUIDv4}`, - lifetime: "session", - }), - (created) => created.destroy.pipe(Effect.orDie), +const make = Effect.fn("TestStack.make")(function* < + Required extends Kind, + Optional extends Kind = never, +>( + options: TestStackOptions<ReadonlyArray<TestService | PlainTestService>>, + decode: (creation: unknown) => Effect.Effect<StackEffect.ServiceCreationInput, StackError>, +) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const crypto = yield* Crypto.Crypto; + const runtime = + options.runtime ?? Option.getOrUndefined(yield* runtimeOverride) ?? defaultRuntime(); + // Shared roots are created owner-only, so each OS user needs its own. + const user = process.getuid?.() ?? userInfo().username; + const stateRoot = options.stateRoot ?? path.join(tmpdir(), `supabase-stack-tests-${user}`); + const cacheRoot = + options.cacheRoot ?? + path.join( + Option.getOrElse(yield* cacheHome, () => path.join(homedir(), ".cache")), + "supabase-stack-artifacts", ); - const diagnose = (operation: string) => (cause: StackError) => - stack.services.list.pipe( - Effect.flatMap((registered) => - Effect.forEach(registered, (member) => - member.status.pipe( - Effect.map( - (status) => - `${member.service}=${status.lifecycle}` + - `${status.health === undefined ? "" : `/${status.health}`}` + - `${status.error === undefined ? "" : ` (${status.error.message})`}`, - ), - Effect.orElseSucceed(() => `${member.service}=unobservable`), + // Native Edge Runtime on Linux cannot read project files below /tmp. + const temporaryRoot = fs.makeTempDirectoryScoped({ + prefix: "supabase-test-stack-", + ...(runtime === "native" && process.platform === "linux" ? { directory: homedir() } : {}), + }); + const projectRoot = options.projectRoot ?? (yield* temporaryRoot); + const dataRoot = + options.projectRoot === undefined + ? Effect.succeed(projectRoot) + : yield* Effect.cached(temporaryRoot); + const services: ReadonlyArray<TestService | PlainTestService> = options.services ?? ["database"]; + const selection = + options.projectRoot === undefined + ? defaultArtifactSelection() + : Option.match(yield* readLock(options.projectRoot), { + onNone: defaultArtifactSelection, + onSome: lockSelection, + }); + const creations = yield* Effect.forEach(services, (service) => + Effect.gen(function* () { + const kind = typeof service === "string" ? service : service.service; + const local = yield* localCreation(kind, dataRoot); + return yield* decode({ + service: kind, + pin: (typeof service === "string" ? undefined : service.pin) ?? selection[kind], + config: { ...local.config, ...(typeof service === "string" ? {} : service.config) }, + endpoints: { + ...local.endpoints, + ...(typeof service === "string" ? {} : service.endpoints), + }, + }); + }), + ); + const stack = yield* Effect.acquireRelease( + StackEffect.create({ + projectRoot, + stateRoot, + cacheRoot, + runtime, + name: `test-${yield* crypto.randomUUIDv4}`, + lifetime: "session", + }), + (created) => created.destroy.pipe(Effect.orDie), + ); + const diagnose = (operation: string) => (cause: StackError) => + stack.services.list.pipe( + Effect.flatMap((registered) => + Effect.forEach(registered, (member) => + member.status.pipe( + Effect.map( + (status) => + `${member.service}=${status.lifecycle}` + + `${status.health === undefined ? "" : `/${status.health}`}` + + `${status.error === undefined ? "" : ` (${status.error.message})`}`, ), + Effect.orElseSucceed(() => `${member.service}=unobservable`), ), ), - Effect.orElseSucceed((): ReadonlyArray<string> => []), - Effect.flatMap((statuses) => - Effect.fail( - new StackError({ - ...cause, - operation, - message: `${cause.message}\nServices: ${statuses.length === 0 ? "none" : statuses.join(", ")}\nOwner log: ${path.join(stateRoot, stack.id, "owner.log")}`, - }), - ), - ), - ); - const members = yield* stack.composition - .supabase(creations, { eager: true }) - .pipe(Effect.catch(diagnose("test-startup"))); - const start = (operation: string) => - stack.composition.start.pipe( - Effect.andThen( - Effect.forEach(members, (member) => member.ready, { - concurrency: "unbounded", - discard: true, + ), + Effect.orElseSucceed((): ReadonlyArray<string> => []), + Effect.flatMap((statuses) => + Effect.fail( + new StackError({ + ...cause, + operation, + message: `${cause.message}\nServices: ${statuses.length === 0 ? "none" : statuses.join(", ")}\nOwner log: ${path.join(stateRoot, stack.id, "owner.log")}`, }), ), - Effect.catch(diagnose(operation)), - ); - yield* start("test-startup"); - const database = members.find( - (member): member is StackEffect.DatabaseInstance => member.service === "database", + ), + ); + const members = yield* stack.composition + .supabase(creations, { eager: true }) + .pipe(Effect.catch(diagnose("test-startup"))); + const start = (operation: string) => + stack.composition.start.pipe( + Effect.andThen( + Effect.forEach(members, (member) => member.ready, { + concurrency: "unbounded", + discard: true, + }), + ), + Effect.catch(diagnose(operation)), ); - const checkpoints = yield* Ref.make<ReadonlySet<string>>(new Set()); - const withContext = (suffix: string) => - Effect.mapError( - (failure: StackError) => - new StackError({ ...failure, message: `${failure.message}; ${suffix}` }), - ); - /** Stops the composition around `work` and brings it back whether or not `work` succeeds. */ - const whileStopped = ( - operation: string, - work: (database: StackEffect.DatabaseInstance) => Effect.Effect<void, StackError>, - ) => - database === undefined - ? Effect.fail( - new StackError({ operation, message: "Test stack checkpoints require a database" }), - ) - : // The owner completes a stop its caller abandons, so the restart must follow its reply. - Effect.uninterruptible(stack.composition.stop).pipe( - Effect.andThen(work(database)), - Effect.matchEffect({ - onSuccess: () => start(operation), - onFailure: (failure) => - start(operation).pipe( - Effect.matchEffect({ - onSuccess: () => Effect.fail(failure), - onFailure: (restart) => - Effect.fail(failure).pipe( - withContext(`restarting the composition also failed: ${restart.message}`), + yield* start("test-startup"); + const database = members.find( + (member): member is StackEffect.DatabaseInstance => member.service === "database", + ); + const checkpoints = yield* Ref.make<ReadonlySet<string>>(new Set()); + const withContext = (suffix: string) => + Effect.mapError( + (failure: StackError) => + new StackError({ ...failure, message: `${failure.message}; ${suffix}` }), + ); + /** Stops the composition around `work` and brings it back whether or not `work` succeeds. */ + const whileStopped = ( + operation: string, + work: (database: StackEffect.DatabaseInstance) => Effect.Effect<void, StackError>, + ) => + database === undefined + ? Effect.fail( + new StackError({ operation, message: "Test stack checkpoints require a database" }), + ) + : // The owner completes a stop its caller abandons, so the restart must follow its reply. + Effect.uninterruptible(stack.composition.stop).pipe( + Effect.andThen(work(database)), + Effect.matchEffect({ + onSuccess: () => start(operation), + onFailure: (failure) => + start(operation).pipe( + Effect.matchEffect({ + onSuccess: () => Effect.fail(failure), + onFailure: (restart) => + Effect.fail(failure).pipe( + withContext(`restarting the composition also failed: ${restart.message}`), + ), + }), + ), + }), + // Signal aborts and test timeouts interrupt; the shared stack must not stay stopped. + Effect.onInterrupt(() => start(operation)), + ); + const testStack: EffectTestStack<Required, Optional> = { + stack, + services: byKind<Required, Optional>(members), + projectRoot, + checkpoint: (name) => + whileStopped("checkpoint", (current) => + current + .saveSnapshot(name, { scope: "instance" }) + .pipe(Effect.andThen(Ref.update(checkpoints, (names) => new Set([...names, name])))), + ), + reset: (name) => + Ref.get(checkpoints).pipe( + Effect.flatMap((names) => + names.has(name) + ? whileStopped("reset", (current) => + current.resetData.pipe( + withContext("the database data may have been partially reset"), + Effect.andThen( + current.restoreSnapshot(name, { scope: "instance" }).pipe( + Effect.flatMap((restored) => + restored + ? Effect.void + : Effect.fail( + new StackError({ + operation: "reset", + message: `Checkpoint ${name} is missing`, + }), + ), ), - }), - ), - }), - // Signal aborts and test timeouts interrupt; the shared stack must not stay stopped. - Effect.onInterrupt(() => start(operation)), - ); - const testStack: EffectTestStack<Required, Optional> = { - stack, - services: byKind<Required, Optional>(members), - projectRoot, - checkpoint: (name) => - whileStopped("checkpoint", (current) => - current - .saveSnapshot(name, { scope: "instance" }) - .pipe(Effect.andThen(Ref.update(checkpoints, (names) => new Set([...names, name])))), - ), - reset: (name) => - Ref.get(checkpoints).pipe( - Effect.flatMap((names) => - names.has(name) - ? whileStopped("reset", (current) => - current.resetData.pipe( - withContext("the database data may have been partially reset"), - Effect.andThen( - current.restoreSnapshot(name, { scope: "instance" }).pipe( - Effect.flatMap((restored) => - restored - ? Effect.void - : Effect.fail( - new StackError({ - operation: "reset", - message: `Checkpoint ${name} is missing`, - }), - ), - ), - withContext( - `the database data was reset without restoring checkpoint ${name}`, - ), + withContext( + `the database data was reset without restoring checkpoint ${name}`, ), ), ), - ) - : Effect.fail( - new StackError({ - operation: "reset", - message: `No checkpoint named ${name}; the database data was not reset`, - }), ), - ), + ) + : Effect.fail( + new StackError({ + operation: "reset", + message: `No checkpoint named ${name}; the database data was not reset`, + }), + ), ), - }; - return testStack; - }, - Effect.mapError(testFailure("test-stack")), -); + ), + }; + return testStack; +}, Effect.mapError(creationFailure)); const decodeCreation = (creation: unknown) => Schema.decodeUnknownEffect(CreationSchema)(creation).pipe(Effect.mapError(testFailure("config"))); @@ -375,3 +427,103 @@ export const createTestStack = < [Symbol.asyncDispose]: client.close, }; }); + +/** What {@link seedSavedStack} saves for a project. */ +export interface SeedSavedStackOptions { + readonly projectRoot: string; + readonly stateRoot: string; + /** Defaults to the native runtime. */ + readonly runtime?: StackEffect.CreateOptions["runtime"]; + /** Marks the database initialized by this upstream build; without it the data was never initialized. */ + readonly built?: string; + /** Leaves `built` as an interrupted first start does: a data directory, but no readiness marker. */ + readonly interrupted?: boolean; + /** Leaves `built`'s data without its recorded build identity. */ + readonly withoutIdentity?: boolean; + /** Schema-owning services the saved composition binds to the database. */ + readonly composed?: ReadonlyArray<SchemaWriterService>; + /** The newest upstream version each service migrated the database with. */ + readonly migrated?: SchemaWriters; +} + +/** + * Saves a stack of `projectRoot` without starting anything and writes the database data a start + * left behind, with the package's own record writers. + */ +export const seedSavedStack = Effect.fn("TestStack.seedSavedStack")( + function* (options: SeedSavedStackOptions) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const crypto = yield* Crypto.Crypto; + const runtime = options.runtime ?? "native"; + const pins = defaultArtifactSelection(); + const identity = yield* resolveStackIdentity({ projectRoot: options.projectRoot }); + const stackId = yield* deriveStackId(identity); + const databaseId = yield* crypto.randomUUIDv4; + const database: ServiceCreation = { + service: "database", + pin: pins.database, + config: { + databasePassword: Redacted.make("seeded-stack-password"), + jwtSecret: Redacted.make("seeded-stack-jwt-secret-with-at-least-32-chars"), + jwtExpiry: 3600, + }, + endpoints: { sql: { port: "auto" } }, + }; + const writers = yield* Effect.forEach(options.composed ?? [], (service) => + Effect.all({ + id: crypto.randomUUIDv4, + creation: Schema.decodeUnknownEffect(ServiceCreation)({ + service, + pin: pins[service], + config: service === "storage" ? { filePath: "storage" } : {}, + endpoints: {}, + }), + }), + ); + const state = Context.get( + yield* Layer.build(StackNamespace.layer({ root: options.stateRoot })), + StackNamespace.Service, + ); + yield* state.save({ + id: stackId, + identity, + runtime, + lifetime: "detached", + instances: [{ id: databaseId, creation: database }, ...writers], + composition: { + members: [databaseId, ...writers.map(({ id }) => id)].map((id) => ({ + id, + activation: "eager", + })), + dependencies: writers.map(({ id }) => ({ + from: databaseId, + to: id, + bindings: [{ output: "databaseUrl", input: "databaseUrl" }], + })), + }, + }); + const instanceRoot = path.join(options.stateRoot, stackId, "data", databaseId); + yield* fs.makeDirectory(instanceRoot, { recursive: true }); + if (options.built !== undefined) { + if (options.interrupted === true) { + yield* fs.makeDirectory(path.join(instanceRoot, "data")); + yield* fs.writeFileString(path.join(instanceRoot, "data", "PG_VERSION"), "17\n"); + } else + yield* fs.writeFileString( + path.join(instanceRoot, databaseReadyMarkerFile), + yield* Schema.encodeEffect(Schema.fromJsonString(DatabaseReadyMarker))({ + runtime, + profile: "supabase", + }), + ); + if (options.withoutIdentity !== true) + yield* recordDatabaseIdentity(fs, path, instanceRoot, databaseIdentity(options.built)); + } + if (options.migrated !== undefined) + yield* setSchemaWriters(fs, path, instanceRoot, options.migrated); + return { stackId, databaseId }; + }, + Effect.scoped, + Effect.mapError(testFailure("seed-saved-stack")), +); diff --git a/packages/stack/tests/docker-fixture.integration.test.ts b/packages/stack/tests/docker-fixture.integration.test.ts index 64afb8e10c..ebf32760ed 100644 --- a/packages/stack/tests/docker-fixture.integration.test.ts +++ b/packages/stack/tests/docker-fixture.integration.test.ts @@ -7,6 +7,7 @@ import { makeStandaloneService } from "./standalone-service.ts"; import { makeDatabase, type BackendEndpoint } from "../src/services/Database.ts"; import { makeDockerDatabaseRoot } from "./docker-fixture.ts"; import { engineTarget, testEngine } from "./engine-target.ts"; +import { selection } from "./pins.ts"; const marker = Schema.Struct({ backend: Schema.Literal("docker"), volume: Schema.String }); const password = Redacted.make("supabase-test-password"); @@ -84,7 +85,7 @@ describe.runIf(testEngine === "docker")( makeStandaloneService(first.definition, { id: "docker-fixture-first", config: { - version: "17", + pin: selection.database, databasePassword: password, jwtSecret: Redacted.make("fixture-secret-with-at-least-32-chars"), jwtExpiry: 3600, @@ -93,7 +94,7 @@ describe.runIf(testEngine === "docker")( makeStandaloneService(second.definition, { id: "docker-fixture-second", config: { - version: "17", + pin: selection.database, databasePassword: password, jwtSecret: Redacted.make("fixture-secret-with-at-least-32-chars"), jwtExpiry: 3600, diff --git a/packages/stack/tests/docker-storage-publish-loop-fixture.ts b/packages/stack/tests/docker-storage-publish-loop-fixture.ts index 27a1e73ca1..4e25945a35 100644 --- a/packages/stack/tests/docker-storage-publish-loop-fixture.ts +++ b/packages/stack/tests/docker-storage-publish-loop-fixture.ts @@ -3,6 +3,7 @@ import { Console, Crypto, Effect, FileSystem, Layer, Path } from "effect"; import { ChildProcessSpawner } from "effect/process"; import { makeContainerRuntime } from "../src/runtime/Container.ts"; import { makeDockerDatabaseStorage } from "../src/storage/DockerDatabaseStorage.ts"; +import { selection } from "./pins.ts"; const [instanceRoot, root, cacheRoot] = process.argv.slice(2); if (instanceRoot === undefined || root === undefined || cacheRoot === undefined) @@ -54,7 +55,7 @@ const program = Effect.scoped( container, spawner, }); - yield* storage.markInitialized("17"); + yield* storage.markInitialized(selection.database); }), ).pipe(Effect.provide(barrierFs.pipe(Layer.provideMerge(NodeServices.layer)))); diff --git a/packages/stack/tests/helper-owner-fixture.ts b/packages/stack/tests/helper-owner-fixture.ts index 71f51e28d8..a1f0e51497 100644 --- a/packages/stack/tests/helper-owner-fixture.ts +++ b/packages/stack/tests/helper-owner-fixture.ts @@ -4,6 +4,7 @@ import { ChildProcess, ChildProcessSpawner } from "effect/process"; import { makeContainerRuntime } from "../src/runtime/Container.ts"; import { makeDockerDatabaseStorage } from "../src/storage/DockerDatabaseStorage.ts"; import { makeDockerHelperRegistry } from "../src/storage/DockerHelperRegistry.ts"; +import { selection } from "./pins.ts"; const stackId = process.argv[2]; const root = process.argv[3]; @@ -67,8 +68,8 @@ const run = Effect.gen(function* () { spawner, helpers, }); - yield* storage.prepare("17"); - if (!shared) yield* storage.removeData("17"); + yield* storage.prepare(selection.database); + if (!shared) yield* storage.removeData(selection.database); yield* Effect.sync(() => process.stdout.write("HELPER_READY\n")); return yield* Effect.never; }); diff --git a/packages/stack/tests/locks.ts b/packages/stack/tests/locks.ts new file mode 100644 index 0000000000..cba7e67f4e --- /dev/null +++ b/packages/stack/tests/locks.ts @@ -0,0 +1,24 @@ +import { Record } from "effect"; +import type { ArtifactPin } from "../src/Artifacts.ts"; +import { LOCK_SCHEMA_URL, type Lock } from "../src/Lock.ts"; +import { selection } from "./pins.ts"; + +/** A full lock of the bundled pins, with the database pinned to `database`. */ +export const lockWithDatabase = (database: ArtifactPin, majorVersion: number): Lock => ({ + $schema: LOCK_SCHEMA_URL, + lockfileVersion: 1, + services: { + ...Record.map(selection, (resolved) => ({ + specifier: "bundled", + resolution: "resolved" as const, + resolved, + })), + database: { + specifier: "bundled", + resolution: "resolved", + resolved: database, + majorVersion, + line: "stock", + }, + }, +}); diff --git a/packages/stack/tests/namespace-lock-fixture.ts b/packages/stack/tests/namespace-lock-fixture.ts index d449cfc796..5af07509ac 100644 --- a/packages/stack/tests/namespace-lock-fixture.ts +++ b/packages/stack/tests/namespace-lock-fixture.ts @@ -3,6 +3,7 @@ import { Console, Effect } from "effect"; import { ChildProcess, ChildProcessSpawner } from "effect/process"; import { fileURLToPath } from "node:url"; import * as StackNamespace from "../src/StackNamespace.ts"; +import { selection } from "./pins.ts"; const [mode, root, id] = process.argv.slice(2); if (root === undefined) throw new Error("Registry root missing"); @@ -27,7 +28,10 @@ const program = Effect.scoped( return yield* Effect.die("Missing writer state"); yield* state.save({ ...saved, - instances: [...saved.instances, { id, creation: { service: "mail", config: {} } }], + instances: [ + ...saved.instances, + { id, creation: { service: "mail", pin: selection.mail, config: {} } }, + ], }); return; } diff --git a/packages/stack/tests/owner-exit-client.ts b/packages/stack/tests/owner-exit-client.ts index d542af630f..0ea57197ba 100644 --- a/packages/stack/tests/owner-exit-client.ts +++ b/packages/stack/tests/owner-exit-client.ts @@ -1,6 +1,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { Data, Effect, Layer, Predicate } from "effect"; import * as Stack from "../src/index.ts"; +import { selection } from "./pins.ts"; class FixtureError extends Data.TaggedError("OwnerExitFixtureError")<{ readonly message: string; @@ -51,6 +52,7 @@ const program = Effect.gen(function* () { const mail = yield* promise(() => client.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }), diff --git a/packages/stack/tests/pins.ts b/packages/stack/tests/pins.ts new file mode 100644 index 0000000000..49ba5c58ec --- /dev/null +++ b/packages/stack/tests/pins.ts @@ -0,0 +1,11 @@ +import { Effect } from "effect"; +import { databaseArtifactPin, defaultArtifactSelection } from "../src/Artifacts.ts"; + +/** The compiled default pin of every service kind. */ +export const selection = defaultArtifactSelection(); + +/** The PostgreSQL 15 database pin, the non-default stock line. */ +export const database15 = Effect.runSync(databaseArtifactPin("15")); + +/** The OrioleDB database pin. */ +export const orioledb = Effect.runSync(databaseArtifactPin("17.11.0.002-orioledb")); diff --git a/packages/stack/tests/public-client-fixture.ts b/packages/stack/tests/public-client-fixture.ts index 4f9bdaa20e..edaf343c60 100644 --- a/packages/stack/tests/public-client-fixture.ts +++ b/packages/stack/tests/public-client-fixture.ts @@ -1,5 +1,6 @@ import { create } from "../src/index.ts"; import { inspect } from "node:util"; +import { selection } from "./pins.ts"; const [root, cacheRoot] = process.argv.slice(2); if (root === undefined || cacheRoot === undefined) throw new Error("Client fixture roots missing"); const stack = await create({ @@ -15,6 +16,7 @@ let readLogs: Promise<void> | undefined; try { const mail = await stack.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); diff --git a/packages/stack/tests/session-client-fixture.ts b/packages/stack/tests/session-client-fixture.ts index aa7b2020c8..6cdb8392bf 100644 --- a/packages/stack/tests/session-client-fixture.ts +++ b/packages/stack/tests/session-client-fixture.ts @@ -1,6 +1,7 @@ import { NodeHttpClient, NodeServices } from "@effect/platform-node"; import { Effect, Layer, Schema } from "effect"; import { create } from "../src/effect.ts"; +import { selection } from "./pins.ts"; const [root, cacheRoot] = process.argv.slice(2); if (root === undefined || cacheRoot === undefined) throw new Error("Session fixture roots missing"); @@ -17,6 +18,7 @@ const program = Effect.scoped( }); const mail = yield* stack.services.create({ service: "mail", + pin: selection.mail, config: {}, endpoints: { http: { port: "auto" } }, }); diff --git a/packages/stack/tests/snapshot-restore-barrier-fixture.ts b/packages/stack/tests/snapshot-restore-barrier-fixture.ts new file mode 100644 index 0000000000..c58c4c4ab6 --- /dev/null +++ b/packages/stack/tests/snapshot-restore-barrier-fixture.ts @@ -0,0 +1,64 @@ +import { NodeServices } from "@effect/platform-node"; +import { Console, Effect, FileSystem, Layer, Path, Ref } from "effect"; +import { schemaWritersFile } from "../src/internal/schema-writers.ts"; +import { makeDatabaseSnapshots } from "../src/services/DatabaseSnapshot.ts"; +import { selection } from "./pins.ts"; + +const [instanceRoot, cacheRoot, key, window] = process.argv.slice(2); +if ( + instanceRoot === undefined || + cacheRoot === undefined || + key === undefined || + (window !== "before-data" && window !== "before-record") +) + throw new Error("instanceRoot, cacheRoot, key or a before-data/before-record window missing"); + +/** + * Restores a snapshot with the production store. It announces `barrier` on stdout the instant it + * is about to cross the restore step named by `window` (publishing the data, or setting the record + * to the snapshot's), then hangs so a caller can kill this process at that boundary on every run. + */ +const barrierFs = Layer.effect( + FileSystem.FileSystem, + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const recordPublications = yield* Ref.make(0); + const data = path.join(instanceRoot, "data"); + const record = path.join(instanceRoot, schemaWritersFile); + const atBarrier = (to: string) => + window === "before-data" + ? Effect.succeed(to === data) + : to === record + ? Ref.updateAndGet(recordPublications, (count) => count + 1).pipe( + // The first publication raises the record, the second sets it to the snapshot's. + Effect.map((count) => count === 2), + ) + : Effect.succeed(false); + return { + ...fs, + rename: (from: string, to: string) => + atBarrier(to).pipe( + Effect.flatMap((reached) => + reached + ? Console.log("barrier").pipe(Effect.andThen(Effect.never)) + : fs.rename(from, to), + ), + ), + }; + }), +); + +const program = Effect.scoped( + Effect.gen(function* () { + const snapshots = yield* makeDatabaseSnapshots({ + instanceRoot, + cacheRoot, + runtime: "native", + pin: selection.database, + }); + yield* snapshots.restoreSnapshot(key); + }), +).pipe(Effect.provide(barrierFs.pipe(Layer.provideMerge(NodeServices.layer)))); + +await Effect.runPromise(program); diff --git a/packages/stack/tests/test-stack-client.ts b/packages/stack/tests/test-stack-client.ts index 01e76a5b46..a3b4c18e8f 100644 --- a/packages/stack/tests/test-stack-client.ts +++ b/packages/stack/tests/test-stack-client.ts @@ -1,6 +1,7 @@ /* oxlint-disable effecttsgo/async-function, effecttsgo/global-fetch -- This fixture exercises the Promise testing API as a non-Effect consumer does. */ import { discover, postgres, type Stack } from "../src/index.ts"; import { createTestStack } from "../src/testing.ts"; +import { selection } from "./pins.ts"; const check = (condition: boolean, message: string) => { if (!condition) throw new Error(message); @@ -8,7 +9,7 @@ const check = (condition: boolean, message: string) => { const sql = async (stack: Stack, databaseUrl: string, command: string) => { const errors: Array<string> = []; - const result = await stack.commands.run(postgres.psql({ major: 17 }), { + const result = await stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--dbname", databaseUrl, "--set", "ON_ERROR_STOP=1", "--command", command], stdout: () => {}, stderr: (bytes) => { diff --git a/packages/stack/tests/whole-stack/fixture.ts b/packages/stack/tests/whole-stack/fixture.ts index 430afe7014..7738d31d2d 100644 --- a/packages/stack/tests/whole-stack/fixture.ts +++ b/packages/stack/tests/whole-stack/fixture.ts @@ -23,6 +23,7 @@ import { cleanupDockerRoot } from "../docker-cleanup.ts"; import { destroyTestStack } from "../stack-cleanup.ts"; import { testEngine } from "../test-engine.ts"; import { testArtifactCacheRoot } from "../artifact-cache.ts"; +import { selection } from "../pins.ts"; type AnyService = Effect.Success<Stack["services"]["list"]>[number]; @@ -159,8 +160,8 @@ export const wholeStack = Effect.fn("WholeStack.fixture")((runtime: Runtime) => [ { service: "database", + pin: selection.database, config: { - version: "17", databasePassword: Redacted.make("postgres"), jwtSecret: Redacted.make(secret), jwtExpiry: 3600, @@ -169,21 +170,25 @@ export const wholeStack = Effect.fn("WholeStack.fixture")((runtime: Runtime) => }, { service: "rest", + pin: selection.rest, config: { jwtSecret: secret }, endpoints: { http: endpoint("auto") }, }, { service: "auth", + pin: selection.auth, config: { jwtSecret: secret }, endpoints: { http: endpoint("auto") }, }, { service: "realtime", + pin: selection.realtime, config: { jwtSecret: secret }, endpoints: { http: endpoint("auto"), rpc: endpoint("auto") }, }, { service: "storage", + pin: selection.storage, config: { filePath: storageRoot, jwtSecret: secret, @@ -192,32 +197,43 @@ export const wholeStack = Effect.fn("WholeStack.fixture")((runtime: Runtime) => }, { service: "imgproxy", + pin: selection.imgproxy, config: { filePath: storageRoot }, endpoints: { http: endpoint("auto") }, }, { service: "functions", + pin: selection.functions, config: { functionsRoot, verifyJwt: true, jwtSecret: secret }, endpoints: { http: endpoint("auto") }, }, - { service: "studio", config: { jwtSecret: secret }, endpoints: { http: endpoint("auto") } }, + { + service: "studio", + pin: selection.studio, + config: { jwtSecret: secret }, + endpoints: { http: endpoint("auto") }, + }, { service: "pgmeta", + pin: selection.pgmeta, config: {}, endpoints: { http: endpoint("auto") }, }, { service: "mail", + pin: selection.mail, config: {}, endpoints: { http: endpoint("auto"), smtp: endpoint("auto"), pop3: endpoint("auto") }, }, { service: "analytics", + pin: selection.analytics, config: { backend: "postgres", apiKey: secret }, endpoints: { http: endpoint("auto") }, }, { service: "pooler", + pin: selection.pooler, config: { jwtSecret: secret, tenant: "whole", @@ -357,7 +373,7 @@ export const sql = Effect.fn("WholeStack.sql")((fixture: WholeStack, statement: if (databaseUrl === undefined) return yield* Effect.die("Database URL missing"); const output: Array<Uint8Array> = []; const errors: Array<Uint8Array> = []; - const result = yield* fixture.stack.commands.run(postgres.psql({ major: 17 }), { + const result = yield* fixture.stack.commands.run(postgres.psql({ pin: selection.database }), { args: ["--set", "ON_ERROR_STOP=1", "--dbname", databaseUrl, "-At"], stdin: Stream.make(new TextEncoder().encode(`${statement}\n`)), stdout: (bytes) => Effect.sync(() => output.push(bytes)), diff --git a/turbo.json b/turbo.json index ab03c3987f..21923aea38 100644 --- a/turbo.json +++ b/turbo.json @@ -95,12 +95,14 @@ "!content/docs/commands/**", "!public/cli/config.schema.json", "!public/cli/project-config.schema.json", + "!public/cli/supabase-lock/v1.schema.json", "$TURBO_ROOT$/mise.lock" ], "outputs": [ "content/docs/commands/**", "public/cli/config.schema.json", - "public/cli/project-config.schema.json" + "public/cli/project-config.schema.json", + "public/cli/supabase-lock/v1.schema.json" ] }, "@supabase/docs#build": { @@ -111,6 +113,7 @@ "!content/docs/commands/**", "!public/cli/config.schema.json", "!public/cli/project-config.schema.json", + "!public/cli/supabase-lock/v1.schema.json", "$TURBO_ROOT$/mise.lock" ], "outputs": [".next/**"] From 9271bfabf419ad6f2a8d630317723ae7d246e4be Mon Sep 17 00:00:00 2001 From: Colum Ferry <cferry09@gmail.com> Date: Tue, 20 Oct 2026 10:37:31 +0000 Subject: [PATCH 02/24] test(cli): match Management API requests by URL origin (#7260) ## Summary The `link` and `bootstrap` integration tests picked out Management API requests with `url.startsWith("https://api.supabase.com")`, which CodeQL reports as incomplete URL substring sanitization (code scanning alerts #59 and #60). They now parse each request URL and compare its origin. The alerts also fail the CodeQL check on unrelated PRs whenever a diff shifts those lines. --- .../bootstrap/bootstrap.workdir-cache.integration.test.ts | 5 +++-- apps/cli/src/commands/link/link.integration.test.ts | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts b/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts index 1d47fcf599..04b3cb7bb0 100644 --- a/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts +++ b/apps/cli/src/commands/bootstrap/bootstrap.workdir-cache.integration.test.ts @@ -320,8 +320,9 @@ describe("bootstrap linked-project cache location", () => { // requests are mapped. expect( api.requests - .filter(({ url }) => url.startsWith("https://api.supabase.com")) - .map(({ method, url }) => `${method} ${url.slice("https://api.supabase.com".length)}`), + .map(({ method, url }) => ({ method, url: new URL(url) })) + .filter(({ url }) => url.origin === "https://api.supabase.com") + .map(({ method, url }) => `${method} ${url.pathname}${url.search}`), ).toEqual([ "GET /v1/organizations", "POST /v1/projects", diff --git a/apps/cli/src/commands/link/link.integration.test.ts b/apps/cli/src/commands/link/link.integration.test.ts index 922a43a4a3..4e3e4abdf3 100644 --- a/apps/cli/src/commands/link/link.integration.test.ts +++ b/apps/cli/src/commands/link/link.integration.test.ts @@ -1585,8 +1585,9 @@ describe("link permissions", () => { const managementCalls = (api: { requests: ReadonlyArray<{ method: string; url: string }> }) => api.requests - .filter(({ url }) => url.startsWith(API)) - .map(({ method, url }) => `${method} ${url.slice(API.length)}`); + .map(({ method, url }) => ({ method, url: new URL(url) })) + .filter(({ url }) => url.origin === API) + .map(({ method, url }) => `${method} ${url.pathname}${url.search}`); it.effect("links by --project-ref, reading the project, its keys, storage and the pooler", () => { const { layer, api, workdir } = setupRecording(); From 617be96e84543f5ae64d439467856ca803fc6063 Mon Sep 17 00:00:00 2001 From: Colum Ferry <cferry09@gmail.com> Date: Tue, 20 Oct 2026 10:58:58 +0000 Subject: [PATCH 03/24] feat(cli): warn before the v3 switch to a local default target (#7186) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary v3 (CLI-2695, landing on `next`) makes the local stack the default target for every command that can run against either the local stack or a hosted project, with one `--remote` flag to reach a project. Today nothing tells you that's coming. This adds a warning on develop, without changing any default: ``` Warning: supabase db push will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. ``` - Printed on stderr, once, when one of these commands runs with no target selector and goes to the linked project: `db push`, `db pull`, `db dump`, `migration fetch`, `migration list`, `migration repair`, `inspect db *` (including the deprecated aliases, named as typed), `inspect report`, `gen types`, `storage ls/cp/mv/rm`. Storage says "the local Storage API". - Any of `--linked`, `--local`, `--db-url`, `--project-ref` (or `gen types --project-id`) silences it, in any form including `--linked=false`. `SUPABASE_PROJECT_ID` doesn't, since it won't select the project in v3. - `gen types` warns only once a project ref has resolved, so an unlinked run still prints just the existing "Must specify one of" error. - stdout, exit codes and the target itself are unchanged in every output format. The opt-in `target` config key and `SUPABASE_TARGET` this PR previously added are gone; the v3 design drops them in favour of `--remote`. ## Linked issue Part of CLI-2695 (the non-breaking half for the current major). ## Checklist - [x] The PR title follows Conventional Commits. - [x] Tests added or updated for the change. - [x] From the repository root, `pnpm check:all` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- .../target-default-warning.ts | 33 +++++ .../target-default-warning.unit.test.ts | 139 ++++++++++++++++++ apps/cli/src/commands/db/dump/SIDE_EFFECTS.md | 8 + apps/cli/src/commands/db/dump/dump.handler.ts | 9 ++ .../commands/db/dump/dump.integration.test.ts | 36 +++++ apps/cli/src/commands/db/pull/SIDE_EFFECTS.md | 10 ++ apps/cli/src/commands/db/pull/pull.handler.ts | 14 +- .../commands/db/pull/pull.integration.test.ts | 45 ++++++ apps/cli/src/commands/db/push/SIDE_EFFECTS.md | 8 + apps/cli/src/commands/db/push/push.handler.ts | 5 + .../commands/db/push/push.integration.test.ts | 66 +++++++++ .../src/commands/gen/types/SIDE_EFFECTS.md | 8 + .../src/commands/gen/types/types.handler.ts | 6 + .../gen/types/types.integration.test.ts | 45 ++++++ .../src/commands/inspect/db/SIDE_EFFECTS.md | 10 ++ .../db/inspect-deprecated.integration.test.ts | 32 +++- .../db/inspect-query.integration.test.ts | 82 +++++++++++ .../src/commands/inspect/db/inspect-query.ts | 8 +- .../db/inspect-specs.integration.test.ts | 4 + .../commands/inspect/report/SIDE_EFFECTS.md | 8 + .../commands/inspect/report/report.handler.ts | 5 + .../inspect/report/report.integration.test.ts | 69 ++++++++- .../commands/migration/fetch/SIDE_EFFECTS.md | 8 + .../commands/migration/fetch/fetch.handler.ts | 7 +- .../migration/fetch/fetch.integration.test.ts | 53 ++++++- .../commands/migration/list/SIDE_EFFECTS.md | 8 + .../commands/migration/list/list.handler.ts | 5 + .../migration/list/list.integration.test.ts | 53 ++++++- .../commands/migration/repair/SIDE_EFFECTS.md | 8 + .../migration/repair/repair.handler.ts | 5 + .../repair/repair.integration.test.ts | 54 ++++++- .../src/commands/storage/cp/SIDE_EFFECTS.md | 8 + .../cli/src/commands/storage/cp/cp.command.ts | 2 + .../src/commands/storage/ls/SIDE_EFFECTS.md | 8 + .../cli/src/commands/storage/ls/ls.command.ts | 2 + .../src/commands/storage/mv/SIDE_EFFECTS.md | 8 + .../cli/src/commands/storage/mv/mv.command.ts | 2 + .../src/commands/storage/rm/SIDE_EFFECTS.md | 8 + .../cli/src/commands/storage/rm/rm.command.ts | 2 + .../cli/src/commands/storage/storage.flags.ts | 15 +- ...target-default-warning.integration.test.ts | 119 +++++++++++++++ .../tests/helpers/target-default-warning.ts | 81 ++++++++++ 42 files changed, 1093 insertions(+), 13 deletions(-) create mode 100644 apps/cli/src/command-internal/target-default-warning.ts create mode 100644 apps/cli/src/command-internal/target-default-warning.unit.test.ts create mode 100644 apps/cli/src/commands/storage/storage.target-default-warning.integration.test.ts create mode 100644 apps/cli/tests/helpers/target-default-warning.ts diff --git a/apps/cli/src/command-internal/target-default-warning.ts b/apps/cli/src/command-internal/target-default-warning.ts new file mode 100644 index 0000000000..671c137dea --- /dev/null +++ b/apps/cli/src/command-internal/target-default-warning.ts @@ -0,0 +1,33 @@ +import { Effect, FileSystem, Option, Path } from "effect"; + +import { CommandSettings } from "../config/command-settings.service.ts"; +import { Output } from "../shared/output/output.service.ts"; +import { yellow } from "./colors.ts"; +import { readLinkSnapshotSoft } from "./link-snapshot.ts"; + +/** Warns on stderr that a run with no target selector but a configured linked project will default to the local target in the next major. */ +export const warnLinkedTargetDefault = Effect.fnUntraced(function* (input: { + readonly command: string; + readonly targetSelected: boolean; + readonly linkedRefResolved?: boolean; + readonly localTarget?: string; +}) { + if (input.targetSelected) return; + if (input.linkedRefResolved !== true) { + const cliSettings = yield* CommandSettings; + if (Option.isNone(Option.filter(cliSettings.projectId, (ref) => ref.length > 0))) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const link = yield* readLinkSnapshotSoft(fs, path, cliSettings.workdir); + if (Option.isNone(link.ref)) return; + } + } + const output = yield* Output; + const localTarget = input.localTarget ?? "the local database"; + yield* output.raw( + `${yellow( + `Warning: supabase ${input.command} will target ${localTarget} by default in the next major version. Pass --linked to keep targeting the linked project.`, + )}\n`, + "stderr", + ); +}); diff --git a/apps/cli/src/command-internal/target-default-warning.unit.test.ts b/apps/cli/src/command-internal/target-default-warning.unit.test.ts new file mode 100644 index 0000000000..d1f27d95ed --- /dev/null +++ b/apps/cli/src/command-internal/target-default-warning.unit.test.ts @@ -0,0 +1,139 @@ +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; + +import { stripAnsi } from "../../tests/helpers/ansi.ts"; +import { VALID_REF, mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; +import { mockOutput } from "../../tests/helpers/mocks.ts"; +import { targetDefaultWarning } from "../../tests/helpers/target-default-warning.ts"; +import { writeProjectLink } from "./project-link.ts"; +import { projectStateModeLayer } from "./project-state-mode.ts"; +import { warnLinkedTargetDefault } from "./target-default-warning.ts"; + +interface Scenario { + readonly projectId?: Option.Option<string>; + readonly refFile?: string; + readonly refPathIsDirectory?: boolean; + readonly stackLinkRef?: string; +} + +const run = (input: Parameters<typeof warnLinkedTargetDefault>[0], scenario: Scenario = {}) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const workdir = yield* fs.makeTempDirectoryScoped({ prefix: "target-warning-" }); + const refPath = path.join(workdir, "supabase", ".temp", "project-ref"); + if (scenario.refFile !== undefined) { + yield* fs.makeDirectory(path.dirname(refPath), { recursive: true }); + yield* fs.writeFileString(refPath, scenario.refFile); + } + if (scenario.refPathIsDirectory === true) { + yield* fs.makeDirectory(refPath, { recursive: true }); + } + if (scenario.stackLinkRef !== undefined) { + yield* writeProjectLink(workdir, { + project: { ref: scenario.stackLinkRef }, + active_branch: { ref: scenario.stackLinkRef }, + fetchedAt: "2026-01-01T00:00:00.000Z", + }); + } + const out = mockOutput(); + yield* warnLinkedTargetDefault(input).pipe( + Effect.provide( + Layer.mergeAll( + out.layer, + mockCommandSettings({ workdir, projectId: scenario.projectId ?? Option.none() }), + BunServices.layer, + projectStateModeLayer(scenario.stackLinkRef === undefined ? "legacy" : "stack"), + ), + ), + ); + return { stdout: out.stdoutText, stderr: stripAnsi(out.stderrText) }; + }).pipe(Effect.provide(BunServices.layer)); + +const BARE = { command: "db push", targetSelected: false } as const; +const WARNING = `${targetDefaultWarning("db push")}\n`; + +describe("warnLinkedTargetDefault", () => { + it.effect("warns once on stderr when SUPABASE_PROJECT_ID configures the linked project", () => + Effect.gen(function* () { + const result = yield* run(BARE, { projectId: Option.some(VALID_REF) }); + expect(result).toEqual({ stdout: "", stderr: WARNING }); + }), + ); + + it.effect("warns when only the link file configures the linked project", () => + Effect.gen(function* () { + const result = yield* run(BARE, { refFile: `${VALID_REF}\n` }); + expect(result).toEqual({ stdout: "", stderr: WARNING }); + }), + ); + + it.effect("warns when a stack-mode link record configures the linked project", () => + Effect.gen(function* () { + const result = yield* run(BARE, { stackLinkRef: VALID_REF }); + expect(result).toEqual({ stdout: "", stderr: WARNING }); + }), + ); + + it.effect("stays silent when neither SUPABASE_PROJECT_ID nor the link file is configured", () => + Effect.gen(function* () { + const result = yield* run(BARE); + expect(result).toEqual({ stdout: "", stderr: "" }); + }), + ); + + it.effect("stays silent when the link file is blank", () => + Effect.gen(function* () { + const result = yield* run(BARE, { refFile: " \n" }); + expect(result.stderr).toBe(""); + }), + ); + + it.effect("stays silent when the link file cannot be read", () => + Effect.gen(function* () { + const result = yield* run(BARE, { refPathIsDirectory: true }); + expect(result.stderr).toBe(""); + }), + ); + + it.effect("stays silent when a target selector was passed, even with a linked project", () => + Effect.gen(function* () { + const result = yield* run( + { command: "db push", targetSelected: true }, + { projectId: Option.some(VALID_REF), refFile: VALID_REF }, + ); + expect(result.stderr).toBe(""); + }), + ); + + it.effect("warns when the caller already resolved the ref and nothing is configured", () => + Effect.gen(function* () { + const result = yield* run({ ...BARE, linkedRefResolved: true }); + expect(result).toEqual({ stdout: "", stderr: WARNING }); + }), + ); + + it.effect("stays silent for a resolved ref when a target selector was passed", () => + Effect.gen(function* () { + const result = yield* run({ + command: "db push", + targetSelected: true, + linkedRefResolved: true, + }); + expect(result.stderr).toBe(""); + }), + ); + + it.effect("names the local target the caller supplies", () => + Effect.gen(function* () { + const result = yield* run( + { command: "storage ls", targetSelected: false, localTarget: "the local Storage API" }, + { projectId: Option.some(VALID_REF) }, + ); + expect(result.stderr).toBe( + `${targetDefaultWarning("storage ls", "the local Storage API")}\n`, + ); + }), + ); +}); diff --git a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md index 2e43311884..0684a7892c 100644 --- a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md @@ -82,6 +82,14 @@ shell inherits the suppressing variables and is missed. > password) is printed **in cleartext** to stdout. Operators piping `--dry-run` > output to logs or CI artifacts should treat that output as a secret. +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** after flag validation and before the connection is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase db dump will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Notes / Divergences - `--data-only` XOR `--role-only`; `--keep-comments` XOR `--data-only`; diff --git a/apps/cli/src/commands/db/dump/dump.handler.ts b/apps/cli/src/commands/db/dump/dump.handler.ts index 007d1a1dc9..8a5d426551 100644 --- a/apps/cli/src/commands/db/dump/dump.handler.ts +++ b/apps/cli/src/commands/db/dump/dump.handler.ts @@ -7,6 +7,7 @@ import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.serv import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import type { DbConnType } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { loadProjectEnv, readDbToml, @@ -145,6 +146,14 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { } } + yield* warnLinkedTargetDefault({ + command: "db dump", + targetSelected: + Option.isSome(flags.dbUrl) || + Option.isSome(flags.linked) || + Option.isSome(flags.local) || + Option.isSome(flags.projectRef), + }); // 3. Resolve the connection: db-url > local > linked, defaulting to linked when // neither is set (unlike other db subcommands, which default to local). const useLocal = Option.isNone(flags.dbUrl) && Option.isSome(flags.local); diff --git a/apps/cli/src/commands/db/dump/dump.integration.test.ts b/apps/cli/src/commands/db/dump/dump.integration.test.ts index d144558b58..1dec38c4e7 100644 --- a/apps/cli/src/commands/db/dump/dump.integration.test.ts +++ b/apps/cli/src/commands/db/dump/dump.integration.test.ts @@ -28,6 +28,10 @@ vi.mock("node:net", (importOriginal) => const reachableDirectHost = withReachableDirectHost(fixtureNet); import { mockOutput, mockTty, processEnvLayer } from "../../../../tests/helpers/mocks.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { unusedGateway } from "../../../../tests/helpers/unused-stack.ts"; import { withReachableDirectHost } from "../../../../tests/helpers/refuse-invalid-db-hosts.ts"; import { @@ -1492,3 +1496,35 @@ describe("db dump permissions through the real resolver and cache", () => { }).pipe(Effect.scoped, Effect.provide(BunServices.layer)); }); }); + +describe("db dump target default warning", () => { + const tmp = useTempWorkdir("supabase-db-dump-warning-"); + const selectors = [ + { label: "--linked", over: { linked: Option.some(true) } }, + { label: "--linked=false", over: { linked: Option.some(false) } }, + { label: "--local", over: { local: Option.some(true) } }, + { label: "--db-url", over: { dbUrl: Option.some("postgresql://u:p@h/db") } }, + { label: "--project-ref", over: { projectRef: Option.some("flagflagflagflagflag") } }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const { layer, out } = setup({ conn: REMOTE_CONN, isLocal: false, workdir: tmp.current }); + return dbDump(flags({ dryRun: true, ...selector?.over })).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "db dump", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); diff --git a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md index da619cd34a..b1373ac65b 100644 --- a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md @@ -180,6 +180,16 @@ Progress strings still go to stderr; stdout carries a single structured envelope `{ declarative, schemaWritten, remoteHistoryUpdated, engine }` and suppresses the `Finished supabase db pull.` line. +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase db pull will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + +`db remote commit`, which reuses the pull implementation, and top-level `pull` never print it. + ## Notes - `--declarative` / deprecated `--use-pg-delta` are mutually exclusive with diff --git a/apps/cli/src/commands/db/pull/pull.handler.ts b/apps/cli/src/commands/db/pull/pull.handler.ts index 5c972d065a..ee25d4bafc 100644 --- a/apps/cli/src/commands/db/pull/pull.handler.ts +++ b/apps/cli/src/commands/db/pull/pull.handler.ts @@ -1,8 +1,9 @@ -import { Effect } from "effect"; +import { Effect, Option } from "effect"; import { aqua } from "../../../command-internal/colors.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { runDbPull, type DbPullInvoke } from "../../../command-internal/db-pull-run.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import type { DbPullFlags } from "./pull.command.ts"; /** @@ -12,6 +13,17 @@ import type { DbPullFlags } from "./pull.command.ts"; */ export const dbPull = Effect.fn("db.pull")(function* (flags: DbPullFlags, invoke?: DbPullInvoke) { const output = yield* Output; + // `db remote commit` reuses this handler and keeps its own linked default. + if (invoke === undefined) { + yield* warnLinkedTargetDefault({ + command: "db pull", + targetSelected: + Option.isSome(flags.dbUrl) || + Option.isSome(flags.linked) || + Option.isSome(flags.local) || + Option.isSome(flags.projectRef), + }); + } const outcome = yield* runDbPull(flags, invoke); yield* Effect.annotateCurrentSpan({ diff --git a/apps/cli/src/commands/db/pull/pull.integration.test.ts b/apps/cli/src/commands/db/pull/pull.integration.test.ts index 24f322600f..ff2be7539e 100644 --- a/apps/cli/src/commands/db/pull/pull.integration.test.ts +++ b/apps/cli/src/commands/db/pull/pull.integration.test.ts @@ -28,6 +28,10 @@ vi.mock("node:net", (importOriginal) => const reachableDirectHost = withReachableDirectHost(fixtureNet); import { stripAnsi } from "../../../../tests/helpers/ansi.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { withReachableDirectHost } from "../../../../tests/helpers/refuse-invalid-db-hosts.ts"; import { assertBlockDrift, @@ -2511,6 +2515,46 @@ describe("db pull", () => { }); }); +describe("db pull target default warning", () => { + const selectors = [ + { label: "--linked", over: { linked: Option.some(true) } }, + { label: "--linked=false", over: { linked: Option.some(false) } }, + { label: "--local", over: { local: Option.some(true) } }, + { label: "--db-url", over: { dbUrl: Option.some("postgresql://u:p@h/db") } }, + { label: "--project-ref", over: { projectRef: Option.some("flagflagflagflagflag") } }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const s = setup(tmp.current, { + migrations: ["20240101000000"], + remoteVersions: ["20240101000000"], + edgeStdout: "", + projectId: Option.none(), + }); + return dbPull(flags(selector?.over)).pipe( + Effect.exit, + Effect.provide(s.layer), + Effect.map(() => ({ + stdout: streamText(s.out, "stdout"), + stderr: streamText(s.out, "stderr"), + })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "db pull", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); + describe("db remote commit", () => { it.effect("writes a remote_commit migration in-process and skips the pull PostRun line", () => { const s = setup(tmp.current, { @@ -2548,6 +2592,7 @@ describe("db remote commit", () => { ); expect(streamText(s.out, "stdout")).not.toContain("Finished supabase db pull."); expect(streamText(s.out, "stdout")).not.toContain("is deprecated"); + expect(streamText(s.out, "stderr")).not.toContain("by default in the next major version"); expect(s.engineCalls).toHaveLength(1); expect(s.engineCalls[0]?.operation).toBe("diff"); }).pipe(Effect.provide(s.layer)); diff --git a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md index d959a3bfcc..444a46c2ef 100644 --- a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md @@ -91,6 +91,14 @@ stdout is payload-only. A single `result` object is emitted: } ``` +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase db push will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Notes - **Targets**: `--db-url`, `--linked` (default), and `--local` are mutually diff --git a/apps/cli/src/commands/db/push/push.handler.ts b/apps/cli/src/commands/db/push/push.handler.ts index 25455ce20c..66207e74c9 100644 --- a/apps/cli/src/commands/db/push/push.handler.ts +++ b/apps/cli/src/commands/db/push/push.handler.ts @@ -14,6 +14,7 @@ import { } from "../../../command-internal/db-config.toml-read.ts"; import { dbPushCore } from "../../../command-internal/db-push-core.ts"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import type { DbPushFlags } from "./push.command.ts"; @@ -47,6 +48,10 @@ export const dbPush = Effect.fn("db.push")(function* (flags: DbPushFlags) { const body = Effect.gen(function* () { const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command: "db push", + targetSelected: target.connType !== undefined || Option.isSome(flags.projectRef), + }); // Mutually-exclusive db-url/linked/local group, keyed off the // explicitly-set flags, not the `--linked` default value. if (target.setFlags.length > 1) { diff --git a/apps/cli/src/commands/db/push/push.integration.test.ts b/apps/cli/src/commands/db/push/push.integration.test.ts index 94234b5439..8401cbb6d6 100644 --- a/apps/cli/src/commands/db/push/push.integration.test.ts +++ b/apps/cli/src/commands/db/push/push.integration.test.ts @@ -14,6 +14,12 @@ vi.mock("node:net", (importOriginal) => import { mockOutput, mockStdin, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { stripAnsi } from "../../../../tests/helpers/ansi.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, + targetDefaultWarning, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, provideEnv, @@ -1391,3 +1397,63 @@ describe("db push", () => { }); }); }); + +describe("db push target default warning", () => { + const tmp = useTempWorkdir("supabase-db-push-warning-"); + const selectors = [ + { label: "--linked", args: ["--linked"], over: { linked: true, local: false } }, + { label: "--linked=false", args: ["--linked=false"], over: { linked: false, local: false } }, + { label: "--local", args: ["--local"], over: { linked: false, local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x"), linked: false, local: false }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { projectRef: Option.some(VALID_REF), linked: false, local: false }, + }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const { layer, out } = setup(tmp.current, { + toml: 'project_id = "test"\n', + args: ["db", "push", ...(selector?.args ?? [])], + isLocal: false, + noProjectId: true, + }); + return dbPush({ ...DEFAULT_FLAGS, ...selector?.over }).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "db push", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); + + it.effect( + "warns on a bare run when only SUPABASE_PROJECT_ID configures the linked project", + () => { + const { layer, out } = setup(tmp.current, { + toml: 'project_id = "test"\n', + args: ["db", "push"], + isLocal: false, + }); + return Effect.gen(function* () { + yield* dbPush(DEFAULT_FLAGS).pipe(Effect.provide(layer)); + expect(stripAnsi(out.stderrText)).toContain(targetDefaultWarning("db push")); + }); + }, + ); +}); diff --git a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md index e97cb639ae..f7a3c8fcce 100644 --- a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md @@ -143,6 +143,14 @@ Not applicable. Not applicable. +### Target default warning + +A run that passes none of `--local`, `--linked` (in any form, including `--linked=false`), `--db-url` or `--project-id` writes one line to **stderr** in every `--output-format`, once the linked project ref has resolved from `SUPABASE_PROJECT_ID`, the link file or the picker. A run that cannot resolve a ref prints only the existing "Must specify one of" error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase gen types will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Notes - TypeScript generated in-process is unformatted. stderr suggests `npx oxfmt <generated-file.ts>`. diff --git a/apps/cli/src/commands/gen/types/types.handler.ts b/apps/cli/src/commands/gen/types/types.handler.ts index 8ceecbcd1c..f4aebf1422 100644 --- a/apps/cli/src/commands/gen/types/types.handler.ts +++ b/apps/cli/src/commands/gen/types/types.handler.ts @@ -41,6 +41,7 @@ import { } from "../../../command-internal/workdir-project.ts"; import { shouldSearchAncestors } from "../../../command-internal/workdir-search.ts"; import { validateWorkdirIsDirectory } from "../../../command-internal/workdir-validation.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { LinkedProjectCache } from "../../../telemetry/linked-project-cache.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { @@ -734,6 +735,11 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) ), ), ); + yield* warnLinkedTargetDefault({ + command: "gen types", + targetSelected: occurrences.has("linked") || occurrences.has("local"), + linkedRefResolved: true, + }); const loaded = schemas.length > 0 ? null : yield* loadConfig(resolvedTarget.ref); yield* runProjectTypes( resolvedTarget, diff --git a/apps/cli/src/commands/gen/types/types.integration.test.ts b/apps/cli/src/commands/gen/types/types.integration.test.ts index ac46209254..78671fba60 100644 --- a/apps/cli/src/commands/gen/types/types.integration.test.ts +++ b/apps/cli/src/commands/gen/types/types.integration.test.ts @@ -33,6 +33,7 @@ import { CommandPlatformApiFactory } from "../../../auth/command-platform-api-fa import { CommandPlatformApi } from "../../../auth/command-platform-api.service.ts"; import { mockOutput, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { assertTargetDefaultWarning } from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, primaryPoolerConfigs, @@ -3349,3 +3350,47 @@ describe("gen types permissions through the real layers", () => { ); } }); + +describe("gen types target default warning", () => { + const selectors = [ + { label: "--linked", args: ["--linked"], over: { linked: true } }, + { label: "--linked=false", args: ["--linked=false"], over: {} }, + { label: "--local", args: ["--local"], over: { local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x") }, + }, + { + label: "--project-id", + args: ["--project-id", VALID_REF], + over: { projectId: Option.some(VALID_REF) }, + }, + ] as const; + + it.effect("warns on a bare run and stays silent with every selector", () => + assertTargetDefaultWarning({ + command: "gen types", + sameTarget: selectors[0], + selectors, + run: (selector) => + Effect.gen(function* () { + const { layer, out } = yield* setup({ + projectId: Option.some(VALID_REF), + projectTypes: "export type Database = {};", + args: ["gen", "types", ...(selector?.args ?? [])], + }); + yield* genTypes(defaultFlags(selector?.over)).pipe(Effect.provide(layer), Effect.exit); + return { stdout: out.stdoutText, stderr: out.stderrText }; + }).pipe(Effect.provide(BunServices.layer)), + }), + ); + + it.effect("stays silent when an unlinked run fails before targeting a project", () => + Effect.gen(function* () { + const { layer, out } = yield* setup(); + yield* genTypes(defaultFlags()).pipe(Effect.provide(layer), Effect.exit); + expect(out.stderrText).not.toContain("by default in the next major version"); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md index 13828c0ed9..9c83fc2cd5 100644 --- a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md @@ -116,6 +116,16 @@ Emit one extra stderr line before the table: `total-index-size`, `total-table-sizes`, and `table-index-sizes` are removed in v3 and emit `Command "<name>" is deprecated and will be removed in v3, use "<target>" instead.` +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase inspect db <subcommand> will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + +`<subcommand>` is the subcommand as typed, so a deprecated alias names itself. + ## Notes - The Management API stack is built lazily, only on the `--linked` path, so `--local` diff --git a/apps/cli/src/commands/inspect/db/inspect-deprecated.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-deprecated.integration.test.ts index d3d08c3319..c437a974b4 100644 --- a/apps/cli/src/commands/inspect/db/inspect-deprecated.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-deprecated.integration.test.ts @@ -1,8 +1,15 @@ +import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Layer, Option } from "effect"; +import { stripAnsi } from "../../../../tests/helpers/ansi.ts"; import { mockOutput } from "../../../../tests/helpers/mocks.ts"; -import { mockTelemetryStateTracked } from "../../../../tests/helpers/command-mocks.ts"; +import { targetDefaultWarning } from "../../../../tests/helpers/target-default-warning.ts"; +import { + mockCommandSettings, + mockTelemetryStateTracked, + useTempWorkdir, +} from "../../../../tests/helpers/command-mocks.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -34,13 +41,18 @@ const LOCAL_CONN: PgConnInput = { database: "postgres", }; -function setup() { +function setup(opts: { workdir?: string; noProjectId?: boolean } = {}) { const out = mockOutput({ format: "text" }); const telemetry = mockTelemetryStateTracked(); let querySql: string | undefined; const layer = Layer.mergeAll( out.layer, telemetry.layer, + mockCommandSettings({ + workdir: opts.workdir ?? "/work/project", + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }), + BunServices.layer, Layer.succeed(DnsResolverFlag, "native"), Layer.succeed(CliArgs, { args: [] }), Layer.succeed(DbConfigResolver, { @@ -172,6 +184,8 @@ const cases: ReadonlyArray<AliasCase> = [ ]; describe("inspect db deprecated aliases", () => { + const tempRoot = useTempWorkdir("supabase-inspect-deprecated-"); + it("covers all 12 deprecated aliases", () => { expect(cases).toHaveLength(12); }); @@ -185,9 +199,23 @@ describe("inspect db deprecated aliases", () => { ? `Command "${testCase.alias}" is deprecated and will be removed in v3, use "${testCase.target}" instead.\n` : `Command "${testCase.alias}" is deprecated, use "${testCase.target}" instead.\n`; expect(ctx.out.stderrText).toContain(expected); + expect(stripAnsi(ctx.out.stderrText)).toContain( + targetDefaultWarning(`inspect db ${testCase.alias}`), + ); expect(ctx.out.stdoutText).not.toContain("is deprecated"); expect(ctx.querySql).toBe(testCase.routedSpec.sql); }).pipe(Effect.provide(ctx.layer)); }); } + + it.effect("prints only the deprecation notice when no linked project is configured", () => { + const ctx = setup({ workdir: tempRoot.current, noProjectId: true }); + return Effect.gen(function* () { + yield* inspectDbCacheHit(flags); + expect(ctx.out.stderrText).toBe( + 'Command "cache-hit" is deprecated, use "db-stats" instead.\nConnecting to local database...\n', + ); + expect(ctx.querySql).toBe(dbStatsSpec.sql); + }).pipe(Effect.provide(ctx.layer)); + }); }); diff --git a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts index a4224c6c72..bfbd69ffb3 100644 --- a/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-query.integration.test.ts @@ -15,10 +15,15 @@ import { commandTreeFor } from "../../../../tests/helpers/command-flags.ts"; import { VALID_REF, VALID_TOKEN, + mockCommandSettings, mockTelemetryStateTracked, useTempWorkdir, } from "../../../../tests/helpers/command-mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, provideEnv, @@ -32,6 +37,7 @@ import { DnsResolverFlag } from "../../../command-internal/global-flags.ts"; import { renderGlamourTable } from "../../../output/glamour-table.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConfigLoadError } from "../../../command-internal/db-config.errors.ts"; +import { PROJECT_NOT_LINKED_MESSAGE } from "../../../config/remote-target.service.ts"; import type { DbConfigFlags, ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; import { DbConnectError, DbExecError } from "../../../command-internal/db-connection.errors.ts"; import { DbConnection, type PgConnInput } from "../../../command-internal/db-connection.service.ts"; @@ -165,6 +171,8 @@ interface SetupOpts { dnsResolver?: "native" | "https"; /** Raw CLI args slice; drives explicitly-passed flag detection. */ cliArgs?: ReadonlyArray<string>; + workdir?: string; + noProjectId?: boolean; /** Runs the real `DbConfigResolver` against the temp `workdir`. */ realLayers?: { readonly workdir: string }; } @@ -195,6 +203,11 @@ function setup(opts: SetupOpts = {}) { telemetry.layer, Layer.succeed(DnsResolverFlag, opts.dnsResolver ?? "native"), Layer.succeed(CliArgs, { args: opts.cliArgs ?? [] }), + mockCommandSettings({ + workdir: opts.realLayers?.workdir ?? opts.workdir ?? "/work/project", + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }), + BunServices.layer, ); return { layer, out, resolver, connection, telemetry }; } @@ -570,6 +583,30 @@ describe("inspect db through the real database-config resolver", () => { }, ); + it.effect( + "a bare run in an unlinked workdir fails as not linked without the default warning", + () => { + const { layer, out } = setup({ + rows: [DB_STATS_ROW], + cliArgs: ["inspect", "db", "db-stats"], + realLayers: { workdir: tempRoot.current }, + noProjectId: true, + }); + const requests: Array<StubbedRequest> = []; + return Effect.gen(function* () { + yield* stubManagementApi(requests, () => undefined); + const error = yield* inspectDbDbStats(flags()).pipe(Effect.flip); + expect(error.message).toBe(PROJECT_NOT_LINKED_MESSAGE); + expect(out.stderrText).not.toContain("by default in the next major version"); + expect(requests).toEqual([]); + }).pipe( + Effect.scoped, + Effect.provide(Layer.mergeAll(layer, BunServices.layer)), + provideEnv({ SUPABASE_ACCESS_TOKEN: VALID_TOKEN }), + ); + }, + ); + it.effect("makes no Management API request for --local", () => { const { layer, connection } = setup({ rows: [DB_STATS_ROW], @@ -621,3 +658,48 @@ describe("inspect db through the real database-config resolver", () => { ); }); }); + +describe("inspect db target default warning", () => { + const tempRoot = useTempWorkdir("supabase-inspect-db-warning-"); + const selectors = [ + { label: "--linked", args: ["--linked"], over: { linked: true } }, + { label: "--linked=false", args: ["--linked=false"], over: {} }, + { label: "--local", args: ["--local"], over: { local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x") }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { linked: true, projectRef: Option.some(VALID_REF) }, + }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tempRoot.current, linked); + const { layer, out } = setup({ + rows: [DB_STATS_ROW], + cliArgs: selector?.args ?? [], + workdir: tempRoot.current, + noProjectId: true, + }); + return inspectDbDbStats(flags(selector?.over)).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "inspect db db-stats", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); diff --git a/apps/cli/src/commands/inspect/db/inspect-query.ts b/apps/cli/src/commands/inspect/db/inspect-query.ts index 6e87add435..62ba818da3 100644 --- a/apps/cli/src/commands/inspect/db/inspect-query.ts +++ b/apps/cli/src/commands/inspect/db/inspect-query.ts @@ -13,6 +13,7 @@ import { DbConfigResolver } from "../../../command-internal/db-config.service.ts import type { ResolvedDbConfig } from "../../../command-internal/db-config.types.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; /** @@ -140,6 +141,7 @@ export const runInspectQuery = Effect.fn("Inspect.runQuery")(function* ( spec: InspectQuerySpec, flags: InspectConnectionFlags, dnsResolver: "native" | "https", + command = `inspect db ${spec.name}`, ) { const output = yield* Output; const resolver = yield* DbConfigResolver; @@ -149,6 +151,10 @@ export const runInspectQuery = Effect.fn("Inspect.runQuery")(function* ( // Mutual exclusivity is keyed off raw argv (which flags were explicitly passed), not the // parsed boolean value — `--local=false` counts as explicitly set. const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command, + targetSelected: target.connType !== undefined || Option.isSome(flags.projectRef), + }); if (target.setFlags.length > 1) { return yield* new InspectMutuallyExclusiveFlagsError({ message: `if any flags in the group [db-url linked local] are set none of the others can be; [${target.setFlags.join(" ")}] were all set`, @@ -232,7 +238,7 @@ export function makeInspectDbHandler( const output = yield* Output; yield* output.raw(deprecation, "stderr"); } - yield* runInspectQuery(spec, flags, dnsResolver); + yield* runInspectQuery(spec, flags, dnsResolver, traceName.replaceAll(".", " ")); }).pipe(Effect.ensuring(telemetryState.flush)); }); } diff --git a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts index 6081127282..deac9c841b 100644 --- a/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts +++ b/apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts @@ -1,6 +1,8 @@ +import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Layer, Option } from "effect"; +import { mockCommandSettings } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput } from "../../../../tests/helpers/mocks.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; @@ -36,6 +38,8 @@ function setup(rows: ReadonlyArray<Record<string, unknown>>) { let queryParams: ReadonlyArray<unknown> | undefined; const layer = Layer.mergeAll( out.layer, + mockCommandSettings({ workdir: "/work/project" }), + BunServices.layer, Layer.succeed(CliArgs, { args: [] }), Layer.succeed(DbConfigResolver, { resolve: (_flags: DbConfigFlags) => diff --git a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md index a914fded01..5054652f91 100644 --- a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md @@ -121,6 +121,14 @@ instead a structured result is emitted: { "outputDir": "<abs path>", "files": [{ "name": "locks", "path": "..." }, ...], "rules": [{ "name": "...", "status": "...", "matches": "..." }, ...] } ``` +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase inspect report will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Notes - **`--project-ref`** (TS-only, no Go equivalent on any user-facing command) diff --git a/apps/cli/src/commands/inspect/report/report.handler.ts b/apps/cli/src/commands/inspect/report/report.handler.ts index 16e4957977..e18af2b2a1 100644 --- a/apps/cli/src/commands/inspect/report/report.handler.ts +++ b/apps/cli/src/commands/inspect/report/report.handler.ts @@ -10,6 +10,7 @@ import { renderGlamourTable } from "../../../output/glamour-table.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { InspectMutuallyExclusiveFlagsError } from "../db/inspect-query.ts"; import type { InspectReportFlags } from "./report.command.ts"; @@ -69,6 +70,10 @@ const runInspectReport = Effect.fnUntraced(function* ( // Mutual exclusivity is keyed off raw argv, not the parsed boolean value: `--local=false` // was explicitly passed, so value-based detection would miss it and default to linked. const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command: "inspect report", + targetSelected: target.connType !== undefined || Option.isSome(flags.projectRef), + }); if (target.setFlags.length > 1) { return yield* new InspectMutuallyExclusiveFlagsError({ message: `if any flags in the group [db-url linked local] are set none of the others can be; [${target.setFlags.join(" ")}] were all set`, diff --git a/apps/cli/src/commands/inspect/report/report.integration.test.ts b/apps/cli/src/commands/inspect/report/report.integration.test.ts index 65b293d0c8..0f0b34d817 100644 --- a/apps/cli/src/commands/inspect/report/report.integration.test.ts +++ b/apps/cli/src/commands/inspect/report/report.integration.test.ts @@ -31,6 +31,10 @@ import { withEnvVar, } from "../../../../tests/helpers/command-mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, realDbConfigResolver, @@ -151,6 +155,7 @@ interface SetupOpts { stdoutIsTty?: boolean; cwd?: string; workdir?: string; + noProjectId?: boolean; /** Raw CLI args slice, used to detect which flags were explicitly passed. */ cliArgs?: ReadonlyArray<string>; /** Runs the real `DbConfigResolver` against `workdir`. */ @@ -170,6 +175,11 @@ function setupReport(opts: SetupOpts = {}) { copyFails: opts.copyFails, }); const telemetry = mockTelemetryStateTracked(); + const settings = (workdir: string) => + mockCommandSettings({ + workdir, + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }); const layer = Layer.mergeAll( out.layer, opts.realLayers !== true || opts.workdir === undefined @@ -184,8 +194,8 @@ function setupReport(opts: SetupOpts = {}) { Layer.succeed(DnsResolverFlag, "native"), Layer.succeed(CliArgs, { args: opts.cliArgs ?? [] }), opts.workdir === undefined - ? tempDirLayer("supabase-report-workdir-", (workdir) => mockCommandSettings({ workdir })) - : mockCommandSettings({ workdir: opts.workdir }), + ? tempDirLayer("supabase-report-workdir-", settings) + : settings(opts.workdir), opts.cwd === undefined ? tempDirLayer("supabase-report-cwd-", (cwd) => mockRuntimeInfo({ cwd })) : mockRuntimeInfo({ cwd: opts.cwd }), @@ -549,10 +559,14 @@ describe("inspect report", () => { ); it.effect("emits a structured result and writes CSVs but no table in json mode", () => { - const { layer, out } = setupReport({ format: "json", csvs: DEFAULT_RULE_CSVS }); + const { layer, out } = setupReport({ + format: "json", + csvs: DEFAULT_RULE_CSVS, + cliArgs: ["--linked"], + }); return Effect.gen(function* () { const base = yield* tempDir("supabase-report-out-"); - yield* inspectReport(flags({ outputDir: base })); + yield* inspectReport(flags({ outputDir: base, linked: true })); expect(out.stdoutText).toBe(""); expect(out.messages).toContainEqual( expect.objectContaining({ type: "success", message: "inspect report" }), @@ -764,3 +778,50 @@ describe("inspect report through the real database-config resolver", () => { }).pipe(Effect.scoped, Effect.provide(layer)); }); }); + +describe("inspect report target default warning", () => { + const tempRoot = useTempWorkdir("supabase-inspect-report-warning-"); + const selectors = [ + { label: "--linked", args: ["--linked"], over: { linked: true } }, + { label: "--linked=false", args: ["--linked=false"], over: {} }, + { label: "--local", args: ["--local"], over: { local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgres://x"], + over: { dbUrl: Option.some("postgres://x") }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { linked: true, projectRef: Option.some(VALID_REF) }, + }, + ] as const; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + Effect.gen(function* () { + const base = yield* tempDir("supabase-report-out-"); + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tempRoot.current, linked); + const { layer, out } = setupReport({ + csvs: DEFAULT_RULE_CSVS, + cliArgs: selector?.args ?? [], + workdir: tempRoot.current, + noProjectId: true, + }); + return inspectReport(flags({ outputDir: base, ...selector?.over })).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + yield* assertTargetDefaultWarning({ + command: "inspect report", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }); + }).pipe(Effect.provide(BunServices.layer)), + ); +}); diff --git a/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md index 7b9d25165c..93d389918d 100644 --- a/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md @@ -54,6 +54,14 @@ Emits `output.success("Migration history fetched", { files: [<absolute path>] }) Same structured `files` result delivered as an NDJSON `result` event. +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase migration fetch will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Prompts - When the migrations directory is non-empty, prompts diff --git a/apps/cli/src/commands/migration/fetch/fetch.handler.ts b/apps/cli/src/commands/migration/fetch/fetch.handler.ts index cea9ee3147..37c3df686f 100644 --- a/apps/cli/src/commands/migration/fetch/fetch.handler.ts +++ b/apps/cli/src/commands/migration/fetch/fetch.handler.ts @@ -1,7 +1,8 @@ -import { Effect } from "effect"; +import { Effect, Option } from "effect"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; import { runMigrationFetch } from "../../../command-internal/migration-fetch-run.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { CliArgs } from "../../../shared/cli/cli-args.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; @@ -12,6 +13,10 @@ export const migrationFetch = Effect.fn("migration.fetch")(function* (flags: Mig const telemetryState = yield* TelemetryState; const cliArgs = yield* CliArgs; const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command: "migration fetch", + targetSelected: target.connType !== undefined || Option.isSome(flags.projectRef), + }); const outcome = yield* runMigrationFetch({ flags, target, assumeYes: undefined }).pipe( Effect.ensuring(telemetryState.flush), ); diff --git a/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts b/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts index ebec1025c3..5b8feb2bda 100644 --- a/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts +++ b/apps/cli/src/commands/migration/fetch/fetch.integration.test.ts @@ -20,6 +20,10 @@ import { } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput, mockStdin, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, provideEnv, @@ -62,6 +66,7 @@ interface SetupOpts { readonly confirm?: boolean; readonly rows?: ReadonlyArray<MigrationRow>; readonly resolveFails?: boolean; + readonly noProjectId?: boolean; /** Raw argv seen by `resolveDbTargetFlags` (e.g. to exercise a flag conflict). */ readonly cliArgs?: ReadonlyArray<string>; } @@ -145,7 +150,10 @@ function setup(workdir: string, opts: SetupOpts = {}) { : resolver, connection, projectRef, - mockCommandSettings({ workdir }), + mockCommandSettings({ + workdir, + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }), Layer.succeed(DnsResolverFlag, "native"), Layer.succeed(YesFlag, opts.yes ?? false), Layer.succeed(CliArgs, { args: opts.cliArgs ?? [] }), @@ -551,3 +559,46 @@ describe("migration fetch", () => { ); }); }); + +describe("migration fetch target default warning", () => { + const selectors = [ + { label: "--linked", args: ["--linked"], over: {} }, + { label: "--linked=false", args: ["--linked=false"], over: { linked: false } }, + { label: "--local", args: ["--local"], over: { linked: false, local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x") }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { projectRef: Option.some(VALID_REF) }, + }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const { layer, out } = setup(tmp.current, { + cliArgs: selector?.args ?? [], + rows: [], + noProjectId: true, + }); + return migrationFetch(flags(selector?.over)).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "migration fetch", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); diff --git a/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md index 29cff072d6..619f33cf95 100644 --- a/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md @@ -54,6 +54,14 @@ Emits `output.success("Migrations listed", { migrations: [{ local, remote, time Same structured `migrations` result delivered as an NDJSON `result` event. +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase migration list will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Notes - `--linked` (default true) lists migrations from the linked project via direct DB connection. diff --git a/apps/cli/src/commands/migration/list/list.handler.ts b/apps/cli/src/commands/migration/list/list.handler.ts index 9822c122cd..756d140664 100644 --- a/apps/cli/src/commands/migration/list/list.handler.ts +++ b/apps/cli/src/commands/migration/list/list.handler.ts @@ -9,6 +9,7 @@ import { renderGlamourTable } from "../../../output/glamour-table.ts"; import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection } from "../../../command-internal/db-connection.service.ts"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { listRemoteMigrations, loadLocalVersions, @@ -117,5 +118,9 @@ export const migrationList = Effect.fn("migration.list")(function* (flags: Migra const telemetryState = yield* TelemetryState; const cliArgs = yield* CliArgs; const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command: "migration list", + targetSelected: target.connType !== undefined || Option.isSome(flags.projectRef), + }); yield* runList(flags, target).pipe(Effect.ensuring(telemetryState.flush)); }); diff --git a/apps/cli/src/commands/migration/list/list.integration.test.ts b/apps/cli/src/commands/migration/list/list.integration.test.ts index f421827dd2..8c5e5f3d02 100644 --- a/apps/cli/src/commands/migration/list/list.integration.test.ts +++ b/apps/cli/src/commands/migration/list/list.integration.test.ts @@ -21,6 +21,10 @@ import { } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, provideEnv, @@ -54,6 +58,7 @@ interface SetupOpts { readonly isLocal?: boolean; readonly remote?: ReadonlyArray<string>; readonly remoteError?: DbExecError; + readonly noProjectId?: boolean; /** Runs the real `DbConfigResolver` and linked-project cache against `workdir`. */ readonly realLayers?: boolean; } @@ -133,7 +138,10 @@ function setup(workdir: string, opts: SetupOpts = {}) { : resolver, connection, projectRef, - mockCommandSettings({ workdir }), + mockCommandSettings({ + workdir, + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }), Layer.succeed(DnsResolverFlag, "native"), Layer.succeed(CliArgs, { args: opts.args ?? [] }), BunServices.layer, @@ -429,3 +437,46 @@ describe("migration list", () => { ); }); }); + +describe("migration list target default warning", () => { + const selectors = [ + { label: "--linked", args: ["--linked"], over: {} }, + { label: "--linked=false", args: ["--linked=false"], over: { linked: false } }, + { label: "--local", args: ["--local"], over: { linked: false, local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x") }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { projectRef: Option.some(VALID_REF) }, + }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const { layer, out } = setup(tmp.current, { + args: selector?.args ?? [], + remote: ["20240101000000"], + noProjectId: true, + }); + return migrationList(flags(selector?.over)).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "migration list", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); diff --git a/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md index 7f64e7834d..7ba716718e 100644 --- a/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md @@ -64,6 +64,14 @@ Emits `output.success("Migration history repaired", { versions, status, repairAl Same structured result delivered as an NDJSON `result` event. +### Target default warning + +A run that passes none of `--local`, `--linked`, `--db-url` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** before the target is resolved, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local database by default. + +``` +Warning: supabase migration repair will target the local database by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Prompts - With no version arguments (repair-all), prompts `Do you want to repair the entire diff --git a/apps/cli/src/commands/migration/repair/repair.handler.ts b/apps/cli/src/commands/migration/repair/repair.handler.ts index 7ae9e0cd12..f08bd5e26f 100644 --- a/apps/cli/src/commands/migration/repair/repair.handler.ts +++ b/apps/cli/src/commands/migration/repair/repair.handler.ts @@ -15,6 +15,7 @@ import { loadProjectEnv } from "../../../command-internal/db-config.toml-read.ts import { DbConfigResolver } from "../../../command-internal/db-config.service.ts"; import { DbConnection, type DbSession } from "../../../command-internal/db-connection.service.ts"; import { resolveDbTargetFlags } from "../../../command-internal/db-target-flags.ts"; +import { warnLinkedTargetDefault } from "../../../command-internal/target-default-warning.ts"; import { DELETE_MIGRATION_VERSION, type MigrationFile, @@ -245,5 +246,9 @@ export const migrationRepair = Effect.fn("migration.repair")(function* ( const telemetryState = yield* TelemetryState; const cliArgs = yield* CliArgs; const target = resolveDbTargetFlags(cliArgs.args); + yield* warnLinkedTargetDefault({ + command: "migration repair", + targetSelected: target.connType !== undefined || Option.isSome(input.projectRef), + }); yield* runRepair(input, target).pipe(Effect.ensuring(telemetryState.flush)); }); diff --git a/apps/cli/src/commands/migration/repair/repair.integration.test.ts b/apps/cli/src/commands/migration/repair/repair.integration.test.ts index 4ce69a2da5..f63bccd905 100644 --- a/apps/cli/src/commands/migration/repair/repair.integration.test.ts +++ b/apps/cli/src/commands/migration/repair/repair.integration.test.ts @@ -23,6 +23,10 @@ import { } from "../../../../tests/helpers/command-mocks.ts"; import { mockOutput, mockStdin, mockTty } from "../../../../tests/helpers/mocks.ts"; import { assertPermissionDrift } from "../../../../tests/helpers/permission-drift.ts"; +import { + assertTargetDefaultWarning, + setLinkedRefFile, +} from "../../../../tests/helpers/target-default-warning.ts"; import { asRecordedRequests, provideEnv, @@ -62,6 +66,7 @@ interface SetupOpts { readonly args?: ReadonlyArray<string>; readonly failSql?: string; readonly failResolve?: boolean; + readonly noProjectId?: boolean; } function setup(workdir: string, opts: SetupOpts = {}) { @@ -156,7 +161,10 @@ function setup(workdir: string, opts: SetupOpts = {}) { : resolver, connection, projectRef, - mockCommandSettings({ workdir }), + mockCommandSettings({ + workdir, + ...(opts.noProjectId === true ? { projectId: Option.none() } : {}), + }), Layer.succeed(DnsResolverFlag, "native"), Layer.succeed(YesFlag, opts.yes ?? false), Layer.succeed(CliArgs, { args: opts.args ?? [] }), @@ -592,3 +600,47 @@ describe("migration repair", () => { ); }); }); + +describe("migration repair target default warning", () => { + const selectors = [ + { label: "--linked", args: ["--linked"], over: {} }, + { label: "--linked=false", args: ["--linked=false"], over: { linked: false } }, + { label: "--local", args: ["--local"], over: { linked: false, local: true } }, + { + label: "--db-url", + args: ["--db-url", "postgresql://x"], + over: { dbUrl: Option.some("postgresql://x") }, + }, + { + label: "--project-ref", + args: ["--project-ref", VALID_REF], + over: { projectRef: Option.some(VALID_REF) }, + }, + ] as const; + + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tmp.current, linked); + const { layer, out } = setup(tmp.current, { + args: selector?.args ?? [], + noProjectId: true, + }); + return migrationRepair( + input({ versions: ["20240101000000"], status: "reverted", ...selector?.over }), + ).pipe( + Effect.provide(layer), + Effect.map(() => ({ stdout: out.stdoutText, stderr: out.stderrText })), + ); + }; + + it.effect( + "warns on a bare run in a linked workdir and stays silent with every selector or when unlinked", + () => + assertTargetDefaultWarning({ + command: "migration repair", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + }), + ); +}); diff --git a/apps/cli/src/commands/storage/cp/SIDE_EFFECTS.md b/apps/cli/src/commands/storage/cp/SIDE_EFFECTS.md index 03aa55baac..9c035cdad9 100644 --- a/apps/cli/src/commands/storage/cp/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/storage/cp/SIDE_EFFECTS.md @@ -91,6 +91,14 @@ backend — same roles as `storage ls`. {"type":"result","data":{"uploaded":[…],"downloaded":[…]}} ``` +### Target default warning + +A run that passes none of `--local`, `--linked` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** after the experimental gate and the `--linked`/`--local` exclusivity check, before any credential lookup or request, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local Storage API by default. + +``` +Warning: supabase storage cp will target the local Storage API by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Telemetry Events Fired | Event | When | Notable properties | diff --git a/apps/cli/src/commands/storage/cp/cp.command.ts b/apps/cli/src/commands/storage/cp/cp.command.ts index 1f1e245ce5..6ae3ed3fc6 100644 --- a/apps/cli/src/commands/storage/cp/cp.command.ts +++ b/apps/cli/src/commands/storage/cp/cp.command.ts @@ -17,6 +17,7 @@ import { StorageLocalFlagDef, StorageProjectRefFlagDef, assertStorageTargetsExclusive, + warnStorageTargetDefault, } from "../storage.flags.ts"; import { storageCp } from "./cp.handler.ts"; @@ -101,6 +102,7 @@ export const storageCpCommand = Command.make("cp", config).pipe( yield* requireExperimental; const cliArgs = yield* CliArgs; yield* assertStorageTargetsExclusive(cliArgs.args); + yield* warnStorageTargetDefault("storage cp", cliArgs.args, flags.projectRef); const telemetryFlags = { recursive: flags.recursive, cacheControl: flags.cacheControl, diff --git a/apps/cli/src/commands/storage/ls/SIDE_EFFECTS.md b/apps/cli/src/commands/storage/ls/SIDE_EFFECTS.md index 1f2e8b73ec..d84cbce60d 100644 --- a/apps/cli/src/commands/storage/ls/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/storage/ls/SIDE_EFFECTS.md @@ -85,6 +85,14 @@ Pagination prints `Loading page: <N>` to **stderr**. {"type":"result","data":{"paths":["bucket/","bucket/folder/file.png"]}} ``` +### Target default warning + +A run that passes none of `--local`, `--linked` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** after the experimental gate and the `--linked`/`--local` exclusivity check, before any credential lookup or request, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local Storage API by default. + +``` +Warning: supabase storage ls will target the local Storage API by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Telemetry Events Fired | Event | When | Notable properties | diff --git a/apps/cli/src/commands/storage/ls/ls.command.ts b/apps/cli/src/commands/storage/ls/ls.command.ts index ceeec4688f..2346566f62 100644 --- a/apps/cli/src/commands/storage/ls/ls.command.ts +++ b/apps/cli/src/commands/storage/ls/ls.command.ts @@ -14,6 +14,7 @@ import { StorageLocalFlagDef, StorageProjectRefFlagDef, assertStorageTargetsExclusive, + warnStorageTargetDefault, } from "../storage.flags.ts"; import { storageLs } from "./ls.handler.ts"; @@ -50,6 +51,7 @@ export const storageLsCommand = Command.make("ls", config).pipe( yield* requireExperimental; const cliArgs = yield* CliArgs; yield* assertStorageTargetsExclusive(cliArgs.args); + yield* warnStorageTargetDefault("storage ls", cliArgs.args, flags.projectRef); const telemetryFlags = { recursive: flags.recursive, linked: flags.linked, diff --git a/apps/cli/src/commands/storage/mv/SIDE_EFFECTS.md b/apps/cli/src/commands/storage/mv/SIDE_EFFECTS.md index 606a75d753..8192c76b42 100644 --- a/apps/cli/src/commands/storage/mv/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/storage/mv/SIDE_EFFECTS.md @@ -86,6 +86,14 @@ backend — same roles as `storage ls`. {"type":"result","data":{"message":"Successfully moved"}} ``` +### Target default warning + +A run that passes none of `--local`, `--linked` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** after the experimental gate and the `--linked`/`--local` exclusivity check, before any credential lookup or request, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local Storage API by default. + +``` +Warning: supabase storage mv will target the local Storage API by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Telemetry Events Fired | Event | When | Notable properties | diff --git a/apps/cli/src/commands/storage/mv/mv.command.ts b/apps/cli/src/commands/storage/mv/mv.command.ts index 3cbe023e5a..cbd8170832 100644 --- a/apps/cli/src/commands/storage/mv/mv.command.ts +++ b/apps/cli/src/commands/storage/mv/mv.command.ts @@ -14,6 +14,7 @@ import { StorageLocalFlagDef, StorageProjectRefFlagDef, assertStorageTargetsExclusive, + warnStorageTargetDefault, } from "../storage.flags.ts"; import { storageMv } from "./mv.handler.ts"; @@ -48,6 +49,7 @@ export const storageMvCommand = Command.make("mv", config).pipe( yield* requireExperimental; const cliArgs = yield* CliArgs; yield* assertStorageTargetsExclusive(cliArgs.args); + yield* warnStorageTargetDefault("storage mv", cliArgs.args, flags.projectRef); const telemetryFlags = { recursive: flags.recursive, linked: flags.linked, diff --git a/apps/cli/src/commands/storage/rm/SIDE_EFFECTS.md b/apps/cli/src/commands/storage/rm/SIDE_EFFECTS.md index 3df8fb7e3f..619edb111a 100644 --- a/apps/cli/src/commands/storage/rm/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/storage/rm/SIDE_EFFECTS.md @@ -99,6 +99,14 @@ Requires `--yes`/`SUPABASE_YES` on the same terms as `json`. {"type":"result","data":{"deleted":["abstract.pdf"],"buckets_deleted":["private"]}} ``` +### Target default warning + +A run that passes none of `--local`, `--linked` or `--project-ref` (in any form, including `--linked=false`) writes one line to **stderr** after the experimental gate and the `--linked`/`--local` exclusivity check, before any credential lookup or request, in every `--output-format`, when a linked project ref is configured (from `SUPABASE_PROJECT_ID` or the link file). An unlinked run prints only the existing not-linked error. stdout is unchanged. The next major version targets the local Storage API by default. + +``` +Warning: supabase storage rm will target the local Storage API by default in the next major version. Pass --linked to keep targeting the linked project. +``` + ## Telemetry Events Fired | Event | When | Notable properties | diff --git a/apps/cli/src/commands/storage/rm/rm.command.ts b/apps/cli/src/commands/storage/rm/rm.command.ts index e518565512..5fdca3a40c 100644 --- a/apps/cli/src/commands/storage/rm/rm.command.ts +++ b/apps/cli/src/commands/storage/rm/rm.command.ts @@ -14,6 +14,7 @@ import { StorageLocalFlagDef, StorageProjectRefFlagDef, assertStorageTargetsExclusive, + warnStorageTargetDefault, } from "../storage.flags.ts"; import { storageRm } from "./rm.handler.ts"; @@ -52,6 +53,7 @@ export const storageRmCommand = Command.make("rm", config).pipe( yield* requireExperimental; const cliArgs = yield* CliArgs; yield* assertStorageTargetsExclusive(cliArgs.args); + yield* warnStorageTargetDefault("storage rm", cliArgs.args, flags.projectRef); const telemetryFlags = { recursive: flags.recursive, linked: flags.linked, diff --git a/apps/cli/src/commands/storage/storage.flags.ts b/apps/cli/src/commands/storage/storage.flags.ts index 2407c59d07..58a91b2b7d 100644 --- a/apps/cli/src/commands/storage/storage.flags.ts +++ b/apps/cli/src/commands/storage/storage.flags.ts @@ -1,9 +1,10 @@ -import { Effect } from "effect"; +import { Effect, Option } from "effect"; import { Flag } from "effect/cli"; import { compose, withCondition } from "../../command-internal/command-permissions/model.ts"; import { changedLinkedLocalFlags } from "../../command-internal/db-target-flags.ts"; import { storageCredentials } from "../../command-internal/storage-credentials.ts"; +import { warnLinkedTargetDefault } from "../../command-internal/target-default-warning.ts"; import { linkedProjectCache } from "../../telemetry/linked-project-cache.layer.ts"; import { StorageMutuallyExclusiveFlagsError } from "./storage.errors.ts"; @@ -52,3 +53,15 @@ export const assertStorageTargetsExclusive = Effect.fnUntraced(function* ( }); } }); + +/** Warns when a storage leaf's run carries none of `--linked`, `--local`, or `--project-ref`. */ +export const warnStorageTargetDefault = ( + command: string, + args: ReadonlyArray<string>, + projectRef: Option.Option<string>, +) => + warnLinkedTargetDefault({ + command, + targetSelected: changedLinkedLocalFlags(args).length > 0 || Option.isSome(projectRef), + localTarget: "the local Storage API", + }); diff --git a/apps/cli/src/commands/storage/storage.target-default-warning.integration.test.ts b/apps/cli/src/commands/storage/storage.target-default-warning.integration.test.ts new file mode 100644 index 0000000000..f74e0f2ffe --- /dev/null +++ b/apps/cli/src/commands/storage/storage.target-default-warning.integration.test.ts @@ -0,0 +1,119 @@ +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, Layer } from "effect"; +import { CliOutput, Command } from "effect/cli"; + +import { CliArgs } from "../../shared/cli/cli-args.service.ts"; +import { textCliOutputFormatter } from "../../shared/output/text-formatter.ts"; +import { GLOBAL_FLAGS } from "../../command-internal/global-flags.ts"; +import { + mockAnalytics, + mockOutput, + mockProcessControl, + mockTelemetryRuntime, + mockTty, +} from "../../../tests/helpers/mocks.ts"; +import { + VALID_REF, + isolatedHomeLayer, + useTempWorkdir, +} from "../../../tests/helpers/command-mocks.ts"; +import { stripAnsi } from "../../../tests/helpers/ansi.ts"; +import { + assertTargetDefaultWarning, + describeFailure, + setLinkedRefFile, + targetDefaultWarning, +} from "../../../tests/helpers/target-default-warning.ts"; +import { storageCommand } from "./storage.command.ts"; + +const tempRoot = useTempWorkdir("supabase-storage-warning-int-"); + +const testRoot = Command.make("supabase").pipe( + Command.withSubcommands([storageCommand]), + Command.withGlobalFlags(GLOBAL_FLAGS), +); + +function setup(args: ReadonlyArray<string>, env: Readonly<Record<string, string>> = {}) { + const out = mockOutput({ format: "text" }); + const layer = Layer.mergeAll( + BunServices.layer, + CliOutput.layer(textCliOutputFormatter()), + out.layer, + Layer.succeed(CliArgs, { args }), + isolatedHomeLayer(tempRoot.current, { + SUPABASE_NO_KEYRING: "1", + SUPABASE_WORKDIR: tempRoot.current, + ...env, + }), + mockProcessControl().layer, + mockTty({ stdinIsTty: false, stdoutIsTty: false }), + mockAnalytics().layer, + mockTelemetryRuntime({ configDir: `${tempRoot.current}/.supabase` }), + ); + return { layer, out }; +} + +const selectors = [ + { label: "--linked", args: ["--linked"] }, + { label: "--linked=false", args: ["--linked=false"] }, + { label: "--local", args: ["--local"] }, + { label: "--project-ref", args: ["--project-ref", VALID_REF] }, +] as const; + +describe("storage target default warning", () => { + const leaves: ReadonlyArray<{ readonly name: string; readonly args: ReadonlyArray<string> }> = [ + { name: "ls", args: ["storage", "ls", "ss:///bucket"] }, + { name: "cp", args: ["storage", "cp", "ss:///bucket/a", "ss:///bucket/b"] }, + { name: "mv", args: ["storage", "mv", "ss:///bucket/a", "ss:///bucket/b"] }, + { name: "rm", args: ["storage", "rm", "ss:///bucket/a"] }, + ]; + + for (const { name, args } of leaves) { + const runWith = (linked: boolean) => (selector: (typeof selectors)[number] | undefined) => { + setLinkedRefFile(tempRoot.current, linked); + const argv = [...args, "--experimental", ...(selector?.args ?? [])]; + const { layer, out } = setup(argv); + return Command.runWith(testRoot, { version: "0.0.0-test" })(argv).pipe( + Effect.exit, + Effect.provide(layer), + Effect.map((exit) => ({ + stdout: out.stdoutText, + stderr: out.stderrText, + failure: describeFailure(exit), + })), + ); + }; + + it.effect( + `${name} warns on a bare run in a linked workdir and stays silent with every selector or when unlinked`, + () => + assertTargetDefaultWarning({ + command: `storage ${name}`, + localTarget: "the local Storage API", + sameTarget: selectors[0], + selectors, + run: runWith(true), + runUnlinked: runWith(false), + unlinkedFailure: "Cannot find project ref. Have you run supabase link?", + }), + ); + } + + it.effect( + "warns on a bare run when only SUPABASE_PROJECT_ID configures the linked project", + () => { + const argv = ["storage", "ls", "ss:///bucket", "--experimental"]; + const { layer, out } = setup(argv, { SUPABASE_PROJECT_ID: VALID_REF }); + return Command.runWith(testRoot, { version: "0.0.0-test" })(argv).pipe( + Effect.exit, + Effect.provide(layer), + Effect.map(() => { + expect(stripAnsi(out.stderrText)).toBe( + `${targetDefaultWarning("storage ls", "the local Storage API")}\n`, + ); + }), + ); + }, + ); +}); diff --git a/apps/cli/tests/helpers/target-default-warning.ts b/apps/cli/tests/helpers/target-default-warning.ts new file mode 100644 index 0000000000..54f052ce78 --- /dev/null +++ b/apps/cli/tests/helpers/target-default-warning.ts @@ -0,0 +1,81 @@ +import { mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; + +import { expect } from "@effect/vitest"; +import { Cause, Effect, Exit } from "effect"; + +import { stripAnsi } from "./ansi.ts"; +import { VALID_REF } from "./command-mocks.ts"; + +export interface TargetRunOutput { + readonly stdout: string; + readonly stderr: string; + readonly failure?: string; +} + +export const targetDefaultWarning = (command: string, localTarget = "the local database") => + `Warning: supabase ${command} will target ${localTarget} by default in the next major version. Pass --linked to keep targeting the linked project.`; + +/** The failure message a run ended with, or `undefined` when it succeeded. */ +export const describeFailure = (exit: Exit.Exit<unknown, unknown>): string | undefined => { + if (Exit.isSuccess(exit)) return undefined; + const error = exit.cause.reasons.find(Cause.isFailReason)?.error; + return error instanceof Error ? error.message : Cause.pretty(exit.cause); +}; + +/** Writes (`linked`) or removes the `supabase link` ref file in `workdir`. */ +export const setLinkedRefFile = (workdir: string, linked: boolean, ref = VALID_REF) => { + const tempDir = join(workdir, "supabase", ".temp"); + if (linked) { + mkdirSync(tempDir, { recursive: true }); + writeFileSync(join(tempDir, "project-ref"), ref); + } else { + rmSync(join(tempDir, "project-ref"), { force: true }); + } +}; + +/** + * A bare run warns exactly once and otherwise prints what the `sameTarget` selector prints; every + * selector run stays silent. `run(undefined)` is the bare run. When `runUnlinked` is given, a bare + * run with no linked project ref configured stays silent and ends the way `sameTarget` does, and + * with `unlinkedFailure` it must fail with exactly that message. + */ +export const assertTargetDefaultWarning = Effect.fnUntraced(function* < + S extends { readonly label: string }, + E, + R, +>(input: { + readonly command: string; + readonly localTarget?: string; + readonly sameTarget: S; + readonly selectors: ReadonlyArray<S>; + readonly run: (selector: S | undefined) => Effect.Effect<TargetRunOutput, E, R>; + readonly runUnlinked?: (selector: S | undefined) => Effect.Effect<TargetRunOutput, E, R>; + readonly unlinkedFailure?: string; +}) { + const warning = `${targetDefaultWarning(input.command, input.localTarget)}\n`; + const bare = yield* input.run(undefined); + const baseline = yield* input.run(input.sameTarget); + + const bareStderr = stripAnsi(bare.stderr); + expect(bareStderr.split(warning)).toHaveLength(2); + expect(bareStderr.replace(warning, "")).toBe(stripAnsi(baseline.stderr)); + expect(bare.stdout).toBe(baseline.stdout); + expect(bare.failure).toBe(baseline.failure); + + for (const selector of input.selectors) { + const { stderr } = yield* input.run(selector); + expect(stripAnsi(stderr), selector.label).not.toContain("by default in the next major version"); + } + + if (input.runUnlinked === undefined) return; + const unlinkedBare = yield* input.runUnlinked(undefined); + const unlinkedBaseline = yield* input.runUnlinked(input.sameTarget); + expect(stripAnsi(unlinkedBare.stderr)).not.toContain("by default in the next major version"); + expect(unlinkedBare.stderr).toBe(unlinkedBaseline.stderr); + expect(unlinkedBare.stdout).toBe(unlinkedBaseline.stdout); + expect(unlinkedBare.failure).toBe(unlinkedBaseline.failure); + if (input.unlinkedFailure !== undefined) { + expect(unlinkedBare.failure).toBe(input.unlinkedFailure); + } +}); From 03149fcc57295f5748beec1d49d9eb7c1bb096c0 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau <avallete@users.noreply.github.com> Date: Tue, 20 Oct 2026 13:04:55 +0000 Subject: [PATCH 04/24] fix(cli): split stack start config and unclassified failure fingerprints (#7211) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR Two `stack start` failure fingerprints each covered many unrelated causes, so telemetry could not say what failed. Each start configuration check now has its own fingerprint, and an unclassified stack failure names the stack operation that failed. ## What changes - `tag:ExperimentalStackStartError:invalid_config` splits by check: `project_config`, `service_config`, `functions_env`, `saved_stack_changes`, `credentials_conflict`, `missing_database`, `missing_credentials`. - Failing to create the Storage uploads, Functions, or Studio snippets directory is classified by its file-system cause (`filesystem` permission, else `filesystem_io`) instead of as a user config error. - A stack failure with no known kind or reason reports `unclassified_<operation>` for the operations `stack start` runs (for example `unclassified_start_composition`, `unclassified_start_service`, `unclassified_ready_service`), covering both the client and the stack owner side. Other operations keep `unclassified`. This applies to every stack command wrapper and `functions serve`. Error messages and output are unchanged. A wrapper that keeps its own fallback for an unclassified stack failure still does. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --- .../stack/destroy/destroy.errors.ts | 2 +- .../stack/install/install.errors.ts | 2 +- .../experimental/stack/logs/logs.errors.ts | 2 +- .../stack/restart/restart.errors.ts | 2 +- .../stack/stack-errors.unit.test.ts | 53 +++++++++++++- .../experimental/stack/start/start.errors.ts | 45 +++++++++++- .../experimental/stack/start/start.handler.ts | 16 +++-- .../stack/status/status.errors.ts | 2 +- .../experimental/stack/stop/stop.errors.ts | 2 +- .../commands/functions/serve/serve.errors.ts | 2 +- .../shared/telemetry/error-actionability.ts | 69 +++++++++++++++++-- 11 files changed, 176 insertions(+), 21 deletions(-) diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts index 4cbe9848d3..3034bb916f 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts @@ -36,7 +36,7 @@ export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStac case "uploads": return { ...actionability.runtimeCrash, fingerprint_suffix: "filesystem_io" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/install/install.errors.ts b/apps/cli/src/commands/experimental/stack/install/install.errors.ts index 9ef0b0d4d6..d47262c23f 100644 --- a/apps/cli/src/commands/experimental/stack/install/install.errors.ts +++ b/apps/cli/src/commands/experimental/stack/install/install.errors.ts @@ -31,7 +31,7 @@ export class StackCommandInstallError extends Data.TaggedError("ExperimentalStac case "runtime": return { ...actionability.dockerNotRunning, fingerprint_suffix: "docker_not_running" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts b/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts index ab35ecdbaa..ea7c300db0 100644 --- a/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts +++ b/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts @@ -22,7 +22,7 @@ export class StackCommandLogsError extends Data.TaggedError("ExperimentalStackLo case "lifecycle": return { ...actionability.invalidConfig, fingerprint_suffix: "lifecycle" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts b/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts index 0f3fd40781..3fa93a3e08 100644 --- a/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts +++ b/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts @@ -25,7 +25,7 @@ export class StackCommandRestartError extends Data.TaggedError("ExperimentalStac case "lifecycle": return { ...actionability.invalidConfig, fingerprint_suffix: "lifecycle" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts b/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts index 3b489ed9fb..0dd77fb4d8 100644 --- a/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts @@ -1,4 +1,5 @@ import { StackError } from "@supabase/stack/effect"; +import { PlatformError } from "effect"; import { describe, expect, it } from "vitest"; import { LocalDbRunningError } from "../../../command-internal/db-bootstrap/local-db-running.ts"; import { DbSetupError } from "../../../command-internal/db-bootstrap/db-setup.ts"; @@ -79,16 +80,62 @@ describe("new stack error telemetry", () => { ).toMatchObject({ error_fingerprint: "tag:ExperimentalStackDestroyError:release_mismatch" }); }); - it("reports a kind from a newer release as unclassified instead of failing", () => { - const cause = new StackError({ operation: "start", message: "x", kind: "from-the-future" }); + it("names an unclassified stack failure by its failed operation, or as unclassified for an unknown operation", () => { + const wrap = (operation: string) => + new StackCommandStartError({ + reason: "stack", + message: "x", + cause: new StackError({ operation, message: "x", kind: "from-the-future" }), + }); - expect(classify(new StackCommandStartError({ reason: "stack", message: "x", cause }))).toEqual({ + expect(classify(wrap("startService"))).toEqual({ + error_fingerprint: "tag:ExperimentalStackStartError:unclassified_start_service", + error_kind: "unknown", + error_category: "unknown", + }); + expect(classify(wrap("a-newer-operation"))).toEqual({ error_fingerprint: "tag:ExperimentalStackStartError:unclassified", error_kind: "unknown", error_category: "unknown", }); }); + it("fingerprints a start configuration check by name, and a directory failure by its user-actionable cause or as I/O", () => { + const configError = (configIssue: "saved-stack-changes" | "missing-credentials") => + classify(new StackCommandStartError({ reason: "invalid-config", configIssue, message: "x" })); + const directoryError = (_tag: "PermissionDenied" | "NotFound" | "Busy") => + classify( + new StackCommandStartError({ + reason: "filesystem", + message: "x", + cause: PlatformError.systemError({ _tag, module: "FileSystem", method: "makeDirectory" }), + }), + ); + + expect(configError("saved-stack-changes")).toEqual({ + error_fingerprint: "tag:ExperimentalStackStartError:saved_stack_changes", + error_kind: "user_actionable", + error_category: "invalid_config", + }); + expect(configError("missing-credentials")).toMatchObject({ + error_fingerprint: "tag:ExperimentalStackStartError:missing_credentials", + }); + expect(directoryError("PermissionDenied")).toEqual({ + error_fingerprint: "tag:ExperimentalStackStartError:filesystem", + error_kind: "user_actionable", + error_category: "permission", + }); + expect(directoryError("NotFound")).toMatchObject({ + error_fingerprint: "tag:ExperimentalStackStartError:not_found", + error_kind: "user_actionable", + }); + expect(directoryError("Busy")).toEqual({ + error_fingerprint: "tag:ExperimentalStackStartError:filesystem_io", + error_kind: "internal_bug", + error_category: "runtime_crash", + }); + }); + it("keeps the declaration of a wrapped CLI-owned error", () => { const cause = new DbSetupError({ reason: "registry_pull", message: "pull failed" }); diff --git a/apps/cli/src/commands/experimental/stack/start/start.errors.ts b/apps/cli/src/commands/experimental/stack/start/start.errors.ts index b4b3d07c04..6535208e78 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.errors.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.errors.ts @@ -7,8 +7,37 @@ import { unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +/** The start check that rejected the project or saved stack configuration. */ +type StartConfigIssue = + | "project-config" + | "service-config" + | "functions-env" + | "saved-stack-changes" + | "credentials-conflict" + | "missing-database" + | "missing-credentials"; + +const configIssueSuffix = { + "project-config": "project_config", + "service-config": "service_config", + "functions-env": "functions_env", + "saved-stack-changes": "saved_stack_changes", + "credentials-conflict": "credentials_conflict", + "missing-database": "missing_database", + "missing-credentials": "missing_credentials", +} as const; + export class StackCommandStartError extends Data.TaggedError("ExperimentalStackStartError")<{ - readonly reason: "invalid-config" | "lock" | "flags" | "runtime" | "lifecycle" | "seed" | "stack"; + readonly reason: + | "invalid-config" + | "filesystem" + | "lock" + | "flags" + | "runtime" + | "lifecycle" + | "seed" + | "stack"; + readonly configIssue?: StartConfigIssue; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -17,7 +46,17 @@ export class StackCommandStartError extends Data.TaggedError("ExperimentalStackS get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { switch (this.reason) { case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return { + ...actionability.invalidConfig, + fingerprint_suffix: + this.configIssue === undefined ? "invalid_config" : configIssueSuffix[this.configIssue], + }; + case "filesystem": { + const declared = causeDeclaration(this.cause); + return declared?.error_kind === "user_actionable" + ? declared + : { ...actionability.runtimeCrash, fingerprint_suffix: "filesystem_io" }; + } case "lock": return ( causeDeclaration(this.cause) ?? { @@ -34,7 +73,7 @@ export class StackCommandStartError extends Data.TaggedError("ExperimentalStackS case "seed": return { ...actionability.seedBuckets, fingerprint_suffix: "seed_buckets" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/start/start.handler.ts b/apps/cli/src/commands/experimental/stack/start/start.handler.ts index ad58bb8163..70e1c19057 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.handler.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.handler.ts @@ -226,7 +226,9 @@ const loadStartConfig = Effect.fn("experimental.stack.start.loadConfig")( Effect.mapError( (error) => new StackCommandStartError({ - reason: Schema.is(LockResolutionError)(error) ? "lock" : "invalid-config", + ...(Schema.is(LockResolutionError)(error) + ? { reason: "lock" as const } + : { reason: "invalid-config" as const, configIssue: "project-config" as const }), message: error.message, cause: error, }), @@ -490,6 +492,7 @@ const incompatibleChange = ( command: replannable ? undefined : command, error: new StackCommandStartError({ reason: "invalid-config", + configIssue: "saved-stack-changes", message: `The saved stack cannot adopt these changes: ${lines.join("; ")}`, suggestion, }), @@ -619,6 +622,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags (error) => new StackCommandStartError({ reason: "invalid-config", + configIssue: "service-config", message: error.message, cause: error, }), @@ -632,6 +636,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags (cause) => new StackCommandStartError({ reason: "invalid-config", + configIssue: "functions-env", message: cause.message, cause, }), @@ -864,6 +869,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags if (rootKey !== undefined && Redacted.value(rootKey) !== savedCredentials.postgresRootKey) return yield* new StackCommandStartError({ reason: "invalid-config", + configIssue: "credentials-conflict", message: "The configured Postgres root key conflicts with the saved stack credentials", }); } @@ -876,7 +882,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags Effect.mapError( (cause) => new StackCommandStartError({ - reason: "invalid-config", + reason: "filesystem", message: `Unable to create the Storage uploads directory: ${cause.message}`, cause, }), @@ -889,7 +895,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags Effect.mapError( (cause) => new StackCommandStartError({ - reason: "invalid-config", + reason: "filesystem", message: `Unable to create the Functions directory: ${cause.message}`, cause, }), @@ -902,7 +908,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags Effect.mapError( (cause) => new StackCommandStartError({ - reason: "invalid-config", + reason: "filesystem", message: `Unable to create the Studio snippets directory: ${cause.message}`, cause, }), @@ -1024,6 +1030,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags if (database === undefined) return yield* new StackCommandStartError({ reason: "invalid-config", + configIssue: "missing-database", message: "The stack composition has no database service", }); const preparation = @@ -1046,6 +1053,7 @@ export const stackStart = Effect.fn("experimental.stack.start")(function* (flags if (stackCredentials === undefined) return yield* new StackCommandStartError({ reason: "invalid-config", + configIssue: "missing-credentials", message: "The stack has no saved credentials", }); if (initialComposition || serviceKindsChanged) { diff --git a/apps/cli/src/commands/experimental/stack/status/status.errors.ts b/apps/cli/src/commands/experimental/stack/status/status.errors.ts index 2cde783332..310f267861 100644 --- a/apps/cli/src/commands/experimental/stack/status/status.errors.ts +++ b/apps/cli/src/commands/experimental/stack/status/status.errors.ts @@ -26,7 +26,7 @@ export class StackCommandStatusError extends Data.TaggedError("ExperimentalStack case "output": return { ...actionability.provideFlags, fingerprint_suffix: "output_format" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts b/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts index 099aec36c4..122ece44e7 100644 --- a/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts +++ b/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts @@ -21,7 +21,7 @@ export class StackCommandStopError extends Data.TaggedError("ExperimentalStackSt case "invalid-config": return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/commands/functions/serve/serve.errors.ts b/apps/cli/src/commands/functions/serve/serve.errors.ts index 85166efdcb..1b2d25fdb4 100644 --- a/apps/cli/src/commands/functions/serve/serve.errors.ts +++ b/apps/cli/src/commands/functions/serve/serve.errors.ts @@ -40,7 +40,7 @@ export class FunctionsServeStackError extends Data.TaggedError("FunctionsServeSt "runtime_stopped", ); case "stack": - return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability; + return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } } } diff --git a/apps/cli/src/shared/telemetry/error-actionability.ts b/apps/cli/src/shared/telemetry/error-actionability.ts index 7760e5a716..7a10a91f9c 100644 --- a/apps/cli/src/shared/telemetry/error-actionability.ts +++ b/apps/cli/src/shared/telemetry/error-actionability.ts @@ -93,6 +93,7 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "connect", "container_configuration", "container_killed", + "credentials_conflict", "daemon_protocol", "daemon_start", "daemon_status", @@ -116,6 +117,7 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "filesystem_io", "flags", "forbidden", + "functions_env", "gateway_auth", "health_check", "health_timeout", @@ -126,6 +128,8 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "invalid_url", "lease_held", "lifecycle", + "missing_credentials", + "missing_database", "network", "not_found", "operation_timeout", @@ -143,6 +147,7 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "process_exit", "process_spawn", "process_stop", + "project_config", "query", "registry_pull", "release_mismatch", @@ -152,10 +157,29 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "request_input", "runtime_stopped", "saml_disabled", + "saved_stack_changes", "seed_buckets", + "service_config", "stack_configuration", "state_file", "unclassified", + "unclassified_configure_composition", + "unclassified_create", + "unclassified_create_service", + "unclassified_credentials", + "unclassified_definition", + "unclassified_endpoint_changes", + "unclassified_open", + "unclassified_plan", + "unclassified_plan_composition", + "unclassified_prepare", + "unclassified_prepare_service", + "unclassified_ready", + "unclassified_ready_service", + "unclassified_start", + "unclassified_start_composition", + "unclassified_start_service", + "unclassified_status", ] as const; type CliErrorFingerprintSuffix = (typeof CLI_ERROR_FINGERPRINT_SUFFIXES)[number]; @@ -538,12 +562,49 @@ const stackErrorReasonActionability = { "release-mismatch": { ...actionability.invalidInput, fingerprint_suffix: "release_mismatch" }, } as const satisfies Record<NonNullable<StackError["reason"]>, CliErrorActionabilityDeclaration>; -/** Reported when nothing in a stack failure classifies it. */ -export const unclassifiedStackFailureActionability: CliErrorActionabilityDeclaration = { +const unclassifiedStackFailure: CliErrorActionabilityDeclaration = { ...actionability.unknown, fingerprint_suffix: "unclassified", }; +// The stack operations `stack start` runs; a failure in any other operation stays `unclassified`. +const unclassifiedStackOperationSuffixes = { + configureComposition: "unclassified_configure_composition", + create: "unclassified_create", + createService: "unclassified_create_service", + credentials: "unclassified_credentials", + definition: "unclassified_definition", + open: "unclassified_open", + plan: "unclassified_plan", + prepare: "unclassified_prepare", + prepareService: "unclassified_prepare_service", + ready: "unclassified_ready", + readyService: "unclassified_ready_service", + start: "unclassified_start", + startComposition: "unclassified_start_composition", + startService: "unclassified_start_service", + startupEndpointChanges: "unclassified_endpoint_changes", + status: "unclassified_status", + supabaseComposition: "unclassified_plan_composition", +} as const satisfies Record<string, CliErrorFingerprintSuffix>; + +/** Reported when nothing in a stack failure classifies it, named by the failed stack operation. */ +export function unclassifiedStackFailureActionability( + cause: unknown, +): CliErrorActionabilityDeclaration { + const error = unwrapNativeFailure(cause); + const operation = + isErrorRecord(error) && readErrorTag(error) === "StackError" + ? ownValue(error, "operation") + : undefined; + return isTableKey(unclassifiedStackOperationSuffixes, operation) + ? { + ...actionability.unknown, + fingerprint_suffix: unclassifiedStackOperationSuffixes[operation], + } + : unclassifiedStackFailure; +} + function isTableKey<T extends object>(table: T, value: unknown): value is keyof T { return typeof value === "string" && Object.hasOwn(table, value); } @@ -971,7 +1032,7 @@ const externalActionabilityByTag: Record<string, ErrorActionabilityAdapter> = { isErrorRecord(outcome) ? stackKindDeclaration(outcome["kind"]) : undefined, ) .find((declaration) => declaration !== undefined) ?? - unclassifiedStackFailureActionability + unclassifiedStackFailure ); }, }; @@ -997,7 +1058,7 @@ export function causeDeclaration(cause: unknown): CliErrorActionabilityDeclarati return undefined; const external = externalActionabilityByTag[tag]?.(error); // An unclassified stack failure leaves the wrapper's own fallback in charge. - return external === unclassifiedStackFailureActionability ? undefined : external; + return external === unclassifiedStackFailure ? undefined : external; } finally { causeDeclarationDepth--; } From b77a14f4ca24a43b3768c93e28f8c51581896f9b Mon Sep 17 00:00:00 2001 From: Andrew Valleteau <avallete@users.noreply.github.com> Date: Tue, 20 Oct 2026 13:19:10 +0000 Subject: [PATCH 05/24] fix(cli): fingerprint stack target-resolution failures by step (#7239) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stacked on #7211. ## TL;DR Every experimental stack command reported every target-resolution failure under one `invalid_config` fingerprint. That covers reading saved stack state, decoding a saved stack, and resolving the project root, so telemetry could not tell these causes apart. Each failed step now has its own fingerprint in every stack command. ## Before ```mermaid flowchart LR A["stack discovery fails"] --> X["<command>:invalid_config"] B["stack lookup fails"] --> X C["prefix match unreadable"] --> X D["project root unresolvable"] --> X ``` ## After ```mermaid flowchart LR A["stack discovery fails"] --> A2["<command>:stack_discovery"] B["stack lookup fails"] --> B2["<command>:stack_lookup"] C["prefix match unreadable"] --> C2["<command>:unreadable_stack"] D["project root unresolvable"] --> D2["<command>:project_root"] ``` ## Why Review on #7211 pointed out that the only remaining `invalid_config` path in `stack start` is target resolution. That path is shared by all stack commands, so it was left for a follow-up. ## What changed - `StackTargetError` carries a closed `targetIssue` that the resolver sets where each failure occurs. - `mapTargetError` carries it into the start, stop, restart, status, destroy, prepare, and logs errors. They report `stack_discovery`, `stack_lookup`, `unreadable_stack`, or `project_root` instead of `invalid_config`. In `stack start`, a configuration check's own fingerprint still takes priority. - prepare, restart, logs, and stop now use the shared `mapTargetError` instead of local copies that would drop the new field. - An invalid-config error without a `targetIssue` still reports `invalid_config`. Error tags, messages, and output are unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --- .../stack/destroy/destroy.errors.ts | 4 +- .../experimental/stack/logs/logs.errors.ts | 4 +- .../experimental/stack/logs/logs.handler.ts | 10 +-- .../stack/restart/restart.errors.ts | 4 +- .../stack/restart/restart.handler.ts | 19 ++---- .../stack/stack-errors.unit.test.ts | 30 +++++++++ .../stack/stack.shared.integration.test.ts | 36 +++++++++- .../experimental/stack/stack.shared.ts | 67 ++++++++++++++----- .../experimental/stack/start/start.errors.ts | 13 ++-- .../stack/status/status.errors.ts | 4 +- .../experimental/stack/stop/stop.errors.ts | 4 +- .../experimental/stack/stop/stop.handler.ts | 37 ++-------- .../shared/telemetry/error-actionability.ts | 4 ++ 13 files changed, 155 insertions(+), 81 deletions(-) diff --git a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts index 3034bb916f..ac5fe598e8 100644 --- a/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts +++ b/apps/cli/src/commands/experimental/stack/destroy/destroy.errors.ts @@ -6,6 +6,7 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStackDestroyError")<{ readonly reason: @@ -16,6 +17,7 @@ export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStac | "runtime" | "stack" | "uploads"; + readonly targetIssue?: StackTargetIssue; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -30,7 +32,7 @@ export class StackCommandDestroyError extends Data.TaggedError("ExperimentalStac case "cancelled": return { ...actionability.cancelled, fingerprint_suffix: "cancelled" }; case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return stackTargetInvalidConfigActionability(this.targetIssue); case "runtime": return { ...actionability.dockerNotRunning, fingerprint_suffix: "docker_not_running" }; case "uploads": diff --git a/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts b/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts index ea7c300db0..97741caa2b 100644 --- a/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts +++ b/apps/cli/src/commands/experimental/stack/logs/logs.errors.ts @@ -6,9 +6,11 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; export class StackCommandLogsError extends Data.TaggedError("ExperimentalStackLogsError")<{ readonly reason: "flags" | "invalid-config" | "lifecycle" | "stack"; + readonly targetIssue?: StackTargetIssue; readonly message: string; readonly suggestion?: string; readonly cause?: unknown; @@ -18,7 +20,7 @@ export class StackCommandLogsError extends Data.TaggedError("ExperimentalStackLo case "flags": return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return stackTargetInvalidConfigActionability(this.targetIssue); case "lifecycle": return { ...actionability.invalidConfig, fingerprint_suffix: "lifecycle" }; case "stack": diff --git a/apps/cli/src/commands/experimental/stack/logs/logs.handler.ts b/apps/cli/src/commands/experimental/stack/logs/logs.handler.ts index 1e7e1e9bd3..d1c227a969 100644 --- a/apps/cli/src/commands/experimental/stack/logs/logs.handler.ts +++ b/apps/cli/src/commands/experimental/stack/logs/logs.handler.ts @@ -13,7 +13,7 @@ import { CommandSettings } from "../../../../config/command-settings.service.ts" import { TelemetryState } from "../../../../telemetry/telemetry-state.service.ts"; import { StackApi, - StackTargetError, + mapTargetError, StackTargetResolver, rejectStackOutput, validateStackTarget, @@ -30,13 +30,7 @@ import { } from "./logs.format.ts"; import { logEvent, type LogSource } from "../../../../command-internal/stack-log-events.ts"; -const targetError = (cause: StackTargetError) => - new StackCommandLogsError({ - reason: cause.reason, - message: cause.message, - ...(cause.suggestion === undefined ? {} : { suggestion: cause.suggestion }), - cause, - }); +const targetError = mapTargetError((props) => new StackCommandLogsError(props)); const logsError = (cause: { readonly message: string }) => new StackCommandLogsError({ reason: "stack", diff --git a/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts b/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts index 3fa93a3e08..86860a8389 100644 --- a/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts +++ b/apps/cli/src/commands/experimental/stack/restart/restart.errors.ts @@ -6,9 +6,11 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; export class StackCommandRestartError extends Data.TaggedError("ExperimentalStackRestartError")<{ readonly reason: "flags" | "not-found" | "invalid-config" | "lifecycle" | "stack"; + readonly targetIssue?: StackTargetIssue; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -21,7 +23,7 @@ export class StackCommandRestartError extends Data.TaggedError("ExperimentalStac case "not-found": return { ...actionability.provideFlags, fingerprint_suffix: "not_found" }; case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return stackTargetInvalidConfigActionability(this.targetIssue); case "lifecycle": return { ...actionability.invalidConfig, fingerprint_suffix: "lifecycle" }; case "stack": diff --git a/apps/cli/src/commands/experimental/stack/restart/restart.handler.ts b/apps/cli/src/commands/experimental/stack/restart/restart.handler.ts index 443c4f1c40..2f8f123c1a 100644 --- a/apps/cli/src/commands/experimental/stack/restart/restart.handler.ts +++ b/apps/cli/src/commands/experimental/stack/restart/restart.handler.ts @@ -10,6 +10,7 @@ import { failedOutcomesDetail, rejectStackOutput, validateStackTarget, + mapTargetError, } from "../stack.shared.ts"; import { StackCommandRestartError } from "./restart.errors.ts"; import type { StackRestartFlags } from "./restart.command.ts"; @@ -34,24 +35,14 @@ export const stackRestart = Effect.fn("experimental.stack.restart")(function* ( const path = yield* Path.Path; const api = yield* StackApi; const resolver = yield* StackTargetResolver; - const mapTargetError = (cause: { - readonly reason: "flags" | "invalid-config"; - readonly message: string; - readonly suggestion?: string; - }) => - new StackCommandRestartError({ - reason: cause.reason, - message: cause.message, - ...(cause.suggestion === undefined ? {} : { suggestion: cause.suggestion }), - cause, - }); + const targetError = mapTargetError((props) => new StackCommandRestartError(props)); yield* rejectStackOutput(yield* Effect.serviceOption(OutputFlag)).pipe( - Effect.mapError(mapTargetError), + Effect.mapError(targetError), ); yield* validateStackTarget({ stack: Option.getOrUndefined(flags.stack), stackId: Option.getOrUndefined(flags.stackId), - }).pipe(Effect.mapError(mapTargetError)); + }).pipe(Effect.mapError(targetError)); const target = yield* resolver .resolve({ projectRoot: settings.workdir, @@ -59,7 +50,7 @@ export const stackRestart = Effect.fn("experimental.stack.restart")(function* ( ...(Option.isSome(flags.stack) ? { name: flags.stack.value } : {}), ...(Option.isSome(flags.stackId) ? { id: flags.stackId.value } : {}), }) - .pipe(Effect.mapError(mapTargetError)); + .pipe(Effect.mapError(targetError)); if (target.id === undefined) return yield* new StackCommandRestartError({ reason: "not-found", diff --git a/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts b/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts index 0dd77fb4d8..cc6031d330 100644 --- a/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack-errors.unit.test.ts @@ -9,6 +9,8 @@ import { DeclarativeApplyError } from "../../db/schema/declarative/declarative.e import { FunctionsServeStackError } from "../../functions/serve/serve.errors.ts"; import { classifyCliErrorActionability } from "../../../shared/telemetry/error-actionability.ts"; import { StackCommandDestroyError } from "./destroy/destroy.errors.ts"; +import { StackCommandLogsError } from "./logs/logs.errors.ts"; +import { mapTargetError, StackTargetError } from "./stack.shared.ts"; import { StackCommandStartError } from "./start/start.errors.ts"; import { StackCommandStatusError } from "./status/status.errors.ts"; @@ -136,6 +138,34 @@ describe("new stack error telemetry", () => { }); }); + it("keeps the failed target-resolution step in the fingerprint of the command that wraps it", () => { + const targetError = (targetIssue?: "stack-lookup" | "project-root") => + new StackTargetError({ + reason: "invalid-config", + message: "x", + ...(targetIssue === undefined ? {} : { targetIssue }), + }); + const asStart = mapTargetError((props) => new StackCommandStartError(props)); + const asLogs = mapTargetError((props) => new StackCommandLogsError(props)); + + expect(classify(targetError("stack-lookup"))).toEqual({ + error_fingerprint: "tag:ExperimentalStackTargetError:stack_lookup", + error_kind: "user_actionable", + error_category: "invalid_config", + }); + expect(classify(asStart(targetError("stack-lookup")))).toEqual({ + error_fingerprint: "tag:ExperimentalStackStartError:stack_lookup", + error_kind: "user_actionable", + error_category: "invalid_config", + }); + expect(classify(asLogs(targetError("project-root")))).toMatchObject({ + error_fingerprint: "tag:ExperimentalStackLogsError:project_root", + }); + expect(classify(asStart(targetError()))).toMatchObject({ + error_fingerprint: "tag:ExperimentalStackStartError:invalid_config", + }); + }); + it("keeps the declaration of a wrapped CLI-owned error", () => { const cause = new DbSetupError({ reason: "registry_pull", message: "pull failed" }); diff --git a/apps/cli/src/commands/experimental/stack/stack.shared.integration.test.ts b/apps/cli/src/commands/experimental/stack/stack.shared.integration.test.ts index 03d7afd658..030bbebe9e 100644 --- a/apps/cli/src/commands/experimental/stack/stack.shared.integration.test.ts +++ b/apps/cli/src/commands/experimental/stack/stack.shared.integration.test.ts @@ -228,7 +228,7 @@ describe("stack target resolver", () => { const failure = yield* resolve({ projectRoot: root, runtime: "auto" }).pipe(Effect.flip); expect(failure).toBeInstanceOf(StackTargetError); - expect(failure.reason).toBe("invalid-config"); + expect(failure).toMatchObject({ reason: "invalid-config", targetIssue: "stack-lookup" }); expect(failure.message).toContain(id); }).pipe(Effect.provide(BunServices.layer)), ); @@ -255,6 +255,8 @@ describe("stack target resolver", () => { runtime: "auto", }).pipe(Effect.flip); + expect(byProject.targetIssue).toBe("stack-lookup"); + expect(byPrefix.targetIssue).toBe("unreadable-stack"); for (const failure of [byProject, byPrefix]) { expect(failure.reason).toBe("invalid-config"); expect(failure.message).toContain(`Stack ${id} could not be read`); @@ -265,6 +267,36 @@ describe("stack target resolver", () => { }).pipe(Effect.provide(BunServices.layer)), ); + it.effect("reports a stack registry that cannot be listed for an id prefix", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const { root, home, resolve } = yield* workspace; + yield* fs.makeDirectory(home); + yield* fs.writeFileString(path.join(home, "stacks"), "not a directory"); + + const failure = yield* resolve({ projectRoot: root, id: "abcd", runtime: "auto" }).pipe( + Effect.flip, + ); + + expect(failure).toMatchObject({ reason: "invalid-config", targetIssue: "stack-discovery" }); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("reports a missing project root as the project root, not as a stack lookup", () => + Effect.gen(function* () { + const path = yield* Path.Path; + const { root, resolve } = yield* workspace; + + const failure = yield* resolve({ + projectRoot: path.join(root, "missing-project"), + runtime: "auto", + }).pipe(Effect.flip); + + expect(failure).toMatchObject({ reason: "invalid-config", targetIssue: "project-root" }); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect("surfaces an unreadable saved stack that an id prefix selects", () => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -279,7 +311,7 @@ describe("stack target resolver", () => { runtime: "auto", }).pipe(Effect.flip); - expect(failure.reason).toBe("invalid-config"); + expect(failure).toMatchObject({ reason: "invalid-config", targetIssue: "unreadable-stack" }); expect(failure.message).toContain(`Stack ${id} could not be read`); }).pipe(Effect.provide(BunServices.layer)), ); diff --git a/apps/cli/src/commands/experimental/stack/stack.shared.ts b/apps/cli/src/commands/experimental/stack/stack.shared.ts index d91576100f..d27ce4fa6c 100644 --- a/apps/cli/src/commands/experimental/stack/stack.shared.ts +++ b/apps/cli/src/commands/experimental/stack/stack.shared.ts @@ -29,24 +29,47 @@ export interface StackTarget { readonly hostRunning: boolean; } +/** The target-resolution step that could not read the saved stack state or project root. */ +export type StackTargetIssue = + | "stack-discovery" + | "stack-lookup" + | "unreadable-stack" + | "project-root"; + +const targetIssueSuffix = { + "stack-discovery": "stack_discovery", + "stack-lookup": "stack_lookup", + "unreadable-stack": "unreadable_stack", + "project-root": "project_root", +} as const; + +export const stackTargetInvalidConfigActionability = ( + targetIssue: StackTargetIssue | undefined, +): CliErrorActionabilityDeclaration => ({ + ...actionability.invalidConfig, + fingerprint_suffix: targetIssue === undefined ? "invalid_config" : targetIssueSuffix[targetIssue], +}); + export class StackTargetError extends Data.TaggedError("ExperimentalStackTargetError")<{ readonly message: string; readonly reason: "flags" | "invalid-config"; + readonly targetIssue?: StackTargetIssue; readonly suggestion?: string; readonly cause?: unknown; }> { get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { return this.reason === "flags" ? { ...actionability.provideFlags, fingerprint_suffix: "flags" } - : { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + : stackTargetInvalidConfigActionability(this.targetIssue); } } -/** Rewraps a `StackTargetError` into a command's own error type, preserving reason, message, and suggestion. */ +/** Rewraps a `StackTargetError` into a command's own error type, preserving its fields. */ export const mapTargetError = <E>( make: (props: { readonly reason: StackTargetError["reason"]; + readonly targetIssue?: StackTargetIssue; readonly message: string; readonly suggestion?: string; readonly cause: StackTargetError; @@ -55,6 +78,7 @@ export const mapTargetError = (error: StackTargetError): E => make({ reason: error.reason, + ...(error.targetIssue === undefined ? {} : { targetIssue: error.targetIssue }), message: error.message, ...(error.suggestion === undefined ? {} : { suggestion: error.suggestion }), cause: error, @@ -116,8 +140,8 @@ export const rejectStackOutput = ( ) : Effect.void; -const stateError = (cause: StackError) => - new StackTargetError({ message: cause.message, reason: "invalid-config", cause }); +const stateError = (targetIssue: StackTargetIssue) => (cause: StackError) => + new StackTargetError({ message: cause.message, reason: "invalid-config", targetIssue, cause }); const runtimeForFlag = (runtime: "auto" | StackRuntime): StackTarget["runtime"] => runtime === "auto" ? undefined : runtime; @@ -135,6 +159,18 @@ export const stackTargetResolverLayer = Layer.effect( const stackApi = yield* StackApi; const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; + const canonicalProjectRoot = (projectRoot: string) => + fs.realPath(projectRoot).pipe( + Effect.mapError( + (cause) => + new StackTargetError({ + message: cause.message, + reason: "invalid-config", + targetIssue: "project-root", + cause, + }), + ), + ); const findByIdPrefix = Effect.fn("StackTargetResolver.findByIdPrefix")(function* ( stateRoot: string, prefix: string, @@ -146,7 +182,7 @@ export const stackTargetResolverLayer = Layer.effect( idPrefix: prefix, onInvalidState: (id, error) => Effect.sync(() => unreadable.push({ id, error })), }) - .pipe(Effect.mapError(stateError)); + .pipe(Effect.mapError(stateError("stack-discovery"))); const matchedIds = [ ...matches.map(({ definition }) => definition.id), ...unreadable.map(({ id }) => id), @@ -160,7 +196,9 @@ export const stackTargetResolverLayer = Layer.effect( }); const [invalid] = unreadable; if (invalid !== undefined) - yield* stackApi.find({ stateRoot, id: invalid.id }).pipe(Effect.mapError(stateError)); + yield* stackApi + .find({ stateRoot, id: invalid.id }) + .pipe(Effect.mapError(stateError("unreadable-stack"))); return matches[0]; }); const resolve = Effect.fn("StackTargetResolver.resolve")(function* (input: { @@ -172,6 +210,9 @@ export const stackTargetResolverLayer = Layer.effect( const id = input.id === undefined ? undefined : yield* validateStackId(input.id); const requestedRuntime = runtimeForFlag(input.runtime); const stateRoot = path.join(settings.supabaseHome, "stacks"); + // Canonicalized before the lookup so a missing project root is not reported as unreadable state. + const lookupRoot = + id === undefined ? yield* canonicalProjectRoot(input.projectRoot) : undefined; const found = id === undefined || isStackId(id) ? yield* stackApi @@ -179,12 +220,12 @@ export const stackTargetResolverLayer = Layer.effect( id === undefined ? { stateRoot, - projectRoot: input.projectRoot, + projectRoot: lookupRoot ?? input.projectRoot, ...(input.name === undefined ? {} : { name: input.name }), } : { stateRoot, id }, ) - .pipe(Effect.map(Option.getOrUndefined), Effect.mapError(stateError)) + .pipe(Effect.map(Option.getOrUndefined), Effect.mapError(stateError("stack-lookup"))) : yield* findByIdPrefix(stateRoot, id); if (id !== undefined && found === undefined) return yield* new StackTargetError({ @@ -204,14 +245,8 @@ export const stackTargetResolverLayer = Layer.effect( // A new stack saves paths under the canonical root its identity derives from. const projectRoot = found?.definition.identity.projectRoot ?? - (yield* fs - .realPath(input.projectRoot) - .pipe( - Effect.mapError( - (cause) => - new StackTargetError({ message: cause.message, reason: "invalid-config", cause }), - ), - )); + lookupRoot ?? + (yield* canonicalProjectRoot(input.projectRoot)); return { projectRoot, ...(found === undefined ? {} : { id: found.definition.id, definition: found.definition }), diff --git a/apps/cli/src/commands/experimental/stack/start/start.errors.ts b/apps/cli/src/commands/experimental/stack/start/start.errors.ts index 6535208e78..ebb98ed577 100644 --- a/apps/cli/src/commands/experimental/stack/start/start.errors.ts +++ b/apps/cli/src/commands/experimental/stack/start/start.errors.ts @@ -6,6 +6,7 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; /** The start check that rejected the project or saved stack configuration. */ type StartConfigIssue = @@ -37,6 +38,7 @@ export class StackCommandStartError extends Data.TaggedError("ExperimentalStackS | "lifecycle" | "seed" | "stack"; + readonly targetIssue?: StackTargetIssue; readonly configIssue?: StartConfigIssue; readonly message: string; readonly detail?: string; @@ -46,11 +48,12 @@ export class StackCommandStartError extends Data.TaggedError("ExperimentalStackS get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { switch (this.reason) { case "invalid-config": - return { - ...actionability.invalidConfig, - fingerprint_suffix: - this.configIssue === undefined ? "invalid_config" : configIssueSuffix[this.configIssue], - }; + return this.configIssue === undefined + ? stackTargetInvalidConfigActionability(this.targetIssue) + : { + ...actionability.invalidConfig, + fingerprint_suffix: configIssueSuffix[this.configIssue], + }; case "filesystem": { const declared = causeDeclaration(this.cause); return declared?.error_kind === "user_actionable" diff --git a/apps/cli/src/commands/experimental/stack/status/status.errors.ts b/apps/cli/src/commands/experimental/stack/status/status.errors.ts index 310f267861..eb244bb379 100644 --- a/apps/cli/src/commands/experimental/stack/status/status.errors.ts +++ b/apps/cli/src/commands/experimental/stack/status/status.errors.ts @@ -6,10 +6,12 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; export class StackCommandStatusError extends Data.TaggedError("ExperimentalStackStatusError")<{ readonly message: string; readonly reason: "flags" | "not-found" | "invalid-config" | "lifecycle" | "output" | "stack"; + readonly targetIssue?: StackTargetIssue; readonly suggestion?: string; readonly cause?: unknown; }> { @@ -20,7 +22,7 @@ export class StackCommandStatusError extends Data.TaggedError("ExperimentalStack case "not-found": return { ...actionability.provideFlags, fingerprint_suffix: "not_found" }; case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return stackTargetInvalidConfigActionability(this.targetIssue); case "lifecycle": return { ...actionability.startStack, fingerprint_suffix: "lifecycle" }; case "output": diff --git a/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts b/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts index 122ece44e7..b69caedb90 100644 --- a/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts +++ b/apps/cli/src/commands/experimental/stack/stop/stop.errors.ts @@ -6,9 +6,11 @@ import { ErrorActionabilityId, unclassifiedStackFailureActionability, } from "../../../../shared/telemetry/error-actionability.ts"; +import { type StackTargetIssue, stackTargetInvalidConfigActionability } from "../stack.shared.ts"; export class StackCommandStopError extends Data.TaggedError("ExperimentalStackStopError")<{ readonly reason: "flags" | "invalid-config" | "stack"; + readonly targetIssue?: StackTargetIssue; readonly message: string; readonly detail?: string; readonly suggestion?: string; @@ -19,7 +21,7 @@ export class StackCommandStopError extends Data.TaggedError("ExperimentalStackSt case "flags": return { ...actionability.provideFlags, fingerprint_suffix: "flags" }; case "invalid-config": - return { ...actionability.invalidConfig, fingerprint_suffix: "invalid_config" }; + return stackTargetInvalidConfigActionability(this.targetIssue); case "stack": return causeDeclaration(this.cause) ?? unclassifiedStackFailureActionability(this.cause); } diff --git a/apps/cli/src/commands/experimental/stack/stop/stop.handler.ts b/apps/cli/src/commands/experimental/stack/stop/stop.handler.ts index dbcbedfdd5..ddcd311e83 100644 --- a/apps/cli/src/commands/experimental/stack/stop/stop.handler.ts +++ b/apps/cli/src/commands/experimental/stack/stop/stop.handler.ts @@ -9,6 +9,7 @@ import { StackTargetResolver, rejectStackOutput, validateStackTarget, + mapTargetError, } from "../stack.shared.ts"; import type { StackStopFlags } from "./stop.command.ts"; import { StackCommandStopError } from "./stop.errors.ts"; @@ -20,6 +21,8 @@ const stopError = (cause: { readonly message: string }) => cause, }); +const targetError = mapTargetError((props) => new StackCommandStopError(props)); + export const stackStop = Effect.fn("experimental.stack.stop")(function* (flags: StackStopFlags) { const telemetry = yield* TelemetryState; return yield* Effect.gen(function* () { @@ -29,15 +32,7 @@ export const stackStop = Effect.fn("experimental.stack.stop")(function* (flags: const api = yield* StackApi; const resolver = yield* StackTargetResolver; yield* rejectStackOutput(yield* Effect.serviceOption(OutputFlag)).pipe( - Effect.mapError( - (cause) => - new StackCommandStopError({ - reason: cause.reason, - message: cause.message, - ...(cause.suggestion === undefined ? {} : { suggestion: cause.suggestion }), - cause, - }), - ), + Effect.mapError(targetError), ); if (Option.isSome(flags.all) && (Option.isSome(flags.stack) || Option.isSome(flags.stackId))) return yield* new StackCommandStopError({ @@ -47,17 +42,7 @@ export const stackStop = Effect.fn("experimental.stack.stop")(function* (flags: yield* validateStackTarget({ stack: Option.getOrUndefined(flags.stack), stackId: Option.getOrUndefined(flags.stackId), - }).pipe( - Effect.mapError( - (cause) => - new StackCommandStopError({ - reason: cause.reason, - message: cause.message, - ...(cause.suggestion === undefined ? {} : { suggestion: cause.suggestion }), - cause, - }), - ), - ); + }).pipe(Effect.mapError(targetError)); const all = Option.getOrElse(flags.all, () => false); const target = all ? undefined @@ -68,17 +53,7 @@ export const stackStop = Effect.fn("experimental.stack.stop")(function* (flags: ...(Option.isSome(flags.stackId) ? { id: flags.stackId.value } : {}), runtime: "auto", }) - .pipe( - Effect.mapError( - (cause) => - new StackCommandStopError({ - reason: cause.reason, - message: cause.message, - ...(cause.suggestion === undefined ? {} : { suggestion: cause.suggestion }), - cause, - }), - ), - ); + .pipe(Effect.mapError(targetError)); const locations = { stateRoot: path.join(settings.supabaseHome, "stacks"), cacheRoot: path.join(settings.supabaseHome, "cache", "stack"), diff --git a/apps/cli/src/shared/telemetry/error-actionability.ts b/apps/cli/src/shared/telemetry/error-actionability.ts index 7a10a91f9c..01824b7ee1 100644 --- a/apps/cli/src/shared/telemetry/error-actionability.ts +++ b/apps/cli/src/shared/telemetry/error-actionability.ts @@ -148,6 +148,7 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "process_spawn", "process_stop", "project_config", + "project_root", "query", "registry_pull", "release_mismatch", @@ -161,6 +162,8 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "seed_buckets", "service_config", "stack_configuration", + "stack_discovery", + "stack_lookup", "state_file", "unclassified", "unclassified_configure_composition", @@ -180,6 +183,7 @@ const CLI_ERROR_FINGERPRINT_SUFFIXES = [ "unclassified_start_composition", "unclassified_start_service", "unclassified_status", + "unreadable_stack", ] as const; type CliErrorFingerprintSuffix = (typeof CLI_ERROR_FINGERPRINT_SUFFIXES)[number]; From 8803a3e90dd18ca14c857026da20f7a073bc6b22 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Tue, 20 Oct 2026 13:36:34 +0000 Subject: [PATCH 06/24] feat(cli): honor `PGSSL*` env on `db push --linked` (CLI-2753) (#7261) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## TL;DR `supabase db push --linked` stops silently dropping `PGSSLMODE` and `PGSSLROOTCERT` when the direct host answers ## whats introduced? `--linked` commands that connect from the CLI itself (`db push`, `migration` commands, `inspect db`, … now read `PGSSLMODE`, `PGSSLROOTCERT` and `PGSSLCERT`/`PGSSLKEY` from the shell on the direct host, the way the pooler fallback already did. ps: the `pg_dump` steps in `db dump`, `db pull` and `migration squash` still don't forward them. `PGSERVICE` and `PGCONNECT_TIMEOUT` apply too, and a missing `PGSSLROOTCERT` file now fails instead of being ignored. host, role and database stay pinned. an invalid mode fails naming the vars on either route, and `sslmode=disable` now dials plaintext without reading cert files. <details> <summary>visual rep</summary> | `supabase db push --linked` | before | now | | ------------------------------------ | -------------------- | ---------------------------- | | direct host | `PGSSL*` ignored | read from the shell | | pooler fallback | read from the shell | unchanged | | invalid `PGSSLMODE` | ignored / IPv6 error | fails naming the vars | </details> ## why this? `--db-url` and the pooler fallback already read these vars. the linked direct host was skipped by mistake during the port, so it now reads them too ## ref: - closes: CLI-2753 - broken/missed in: https://github.com/supabase/cli/pull/5715 --- .../src/command-internal/db-config.errors.ts | 9 + .../db-config.integration.test.ts | 239 +++++++++++++++++- .../src/command-internal/db-config.layer.ts | 55 +++- .../src/command-internal/db-config.parse.ts | 16 +- .../db-config.parse.unit.test.ts | 27 ++ .../src/command-internal/db-config.service.ts | 2 + .../db-connection.sql-pg.integration.test.ts | 34 ++- .../db-connection.sql-pg.layer.ts | 7 +- .../src/commands/bootstrap/SIDE_EFFECTS.md | 3 + .../src/commands/db/advisors/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/db/diff/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/db/dump/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/db/lint/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/db/pull/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/db/push/SIDE_EFFECTS.md | 18 ++ .../cli/src/commands/db/query/SIDE_EFFECTS.md | 2 + .../cli/src/commands/db/reset/SIDE_EFFECTS.md | 2 + .../declarative/generate/SIDE_EFFECTS.md | 2 + .../schema/declarative/sync/SIDE_EFFECTS.md | 3 + .../src/commands/gen/types/SIDE_EFFECTS.md | 9 +- .../src/commands/inspect/db/SIDE_EFFECTS.md | 6 +- .../commands/inspect/report/SIDE_EFFECTS.md | 2 + .../commands/migration/down/SIDE_EFFECTS.md | 2 + .../commands/migration/fetch/SIDE_EFFECTS.md | 2 + .../commands/migration/list/SIDE_EFFECTS.md | 2 + .../commands/migration/repair/SIDE_EFFECTS.md | 2 + .../commands/migration/squash/SIDE_EFFECTS.md | 3 +- .../src/commands/migration/up/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/pull/SIDE_EFFECTS.md | 2 + apps/cli/src/commands/test/db/SIDE_EFFECTS.md | 3 + .../telemetry/__fixtures__/error-tags.txt | 1 + 31 files changed, 444 insertions(+), 21 deletions(-) diff --git a/apps/cli/src/command-internal/db-config.errors.ts b/apps/cli/src/command-internal/db-config.errors.ts index 4d8fe9a781..fb4840f27e 100644 --- a/apps/cli/src/command-internal/db-config.errors.ts +++ b/apps/cli/src/command-internal/db-config.errors.ts @@ -16,6 +16,15 @@ export class DbConfigParseUrlError extends Data.TaggedError("DbConfigParseUrlErr } } +/** The shell libpq `PG*` environment is invalid for a linked connection. */ +export class DbConfigLibpqEnvError extends Data.TaggedError("DbConfigLibpqEnvError")<{ + readonly message: string; +}> { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} + /** * `supabase/config.toml` exists but could not be read or parsed: the read/decode * error aborts the load, rather than silently running against the default local database. diff --git a/apps/cli/src/command-internal/db-config.integration.test.ts b/apps/cli/src/command-internal/db-config.integration.test.ts index 95b7ca5b86..338634959e 100644 --- a/apps/cli/src/command-internal/db-config.integration.test.ts +++ b/apps/cli/src/command-internal/db-config.integration.test.ts @@ -18,11 +18,13 @@ import * as TestClock from "effect/testing/TestClock"; import type { DatabaseInstance, Stack } from "@supabase/stack/effect"; import { vi } from "vitest"; +const fixtureNet = vi.hoisted(() => ({ directHostReachable: false })); + vi.mock("node:net", (importOriginal) => Promise.all([ import("../../tests/helpers/refuse-invalid-db-hosts.ts"), importOriginal<typeof import("node:net")>(), - ]).then(([helper, net]) => helper.refuseInvalidDbHosts(net)), + ]).then(([helper, net]) => helper.refuseInvalidDbHosts(net, fixtureNet)), ); import { @@ -32,7 +34,15 @@ import { mockTelemetryRuntime, mockTty, } from "../../tests/helpers/mocks.ts"; -import { VALID_TOKEN, mockCommandSettings } from "../../tests/helpers/command-mocks.ts"; +import { CommandPlatformApi } from "../auth/command-platform-api.service.ts"; +import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; +import { + VALID_TOKEN, + mockCommandPlatformApiService, + mockCommandSettings, + withConfigEnv, +} from "../../tests/helpers/command-mocks.ts"; +import { withReachableDirectHost } from "../../tests/helpers/refuse-invalid-db-hosts.ts"; import { unusedGateway } from "../../tests/helpers/unused-stack.ts"; import { assertBlockDrift } from "../../tests/helpers/permission-drift.ts"; import { @@ -46,7 +56,9 @@ import { dbConfigLayer, dbConfigPoolerFallback, dbConfigResolverLayer, + resolveLinkedConn, } from "./db-config.layer.ts"; +import { DbConfigIpv6Error, DbConfigLibpqEnvError } from "./db-config.errors.ts"; import { DebugFlag, DnsResolverFlag, @@ -2060,3 +2072,226 @@ describe("dbConfigResolver (stack-mode link document)", () => { }, ); }); + +describe("resolveLinkedConn libpq TLS", () => { + const ref = "cli2753tlsabcdefghij"; + const directHost = `db.${ref}.invalid`; + const poolerHost = "aws-0-us-east-1.pooler.supabase.com"; + const api = mockCommandPlatformApiService({ + v1: { + getPoolerConfig: () => Effect.succeed(primaryPoolerConfigs(ref)), + createLoginRole: () => + Effect.succeed({ role: "cli_login_x", password: "temp", ttl_seconds: 3600 }), + }, + }); + const linkedConnLayer = Layer.mergeAll( + BunServices.layer, + mockOutput().layer, + ConfigProvider.layer(ConfigProvider.fromEnvRecord({}, { preserveEmptyStrings: true })), + mockDebugLogger, + Layer.succeed(CommandPlatformApiFactory, { + make: CommandPlatformApi.pipe(Effect.provide(api.layer)), + }), + Layer.succeed(DbConnection, { connect: () => Effect.die("unexpected connect()") }), + ); + + const projectWorkdir = (dotenv: ReadonlyArray<string>) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const dir = yield* fs.makeTempDirectoryScoped({ prefix: "cli-linked-tls-" }); + yield* fs.makeDirectory(path.join(dir, "supabase"), { recursive: true }); + yield* fs.writeFileString(path.join(dir, "supabase", ".env"), [...dotenv, ""].join("\n")); + return dir; + }); + + const linkedConn = ( + shellEnv: Readonly<Record<string, string>>, + opts: { + readonly workdir?: string; + readonly reachable?: boolean; + readonly password?: Option.Option<string>; + readonly poolerUrl?: string; + } = {}, + ) => + withConfigEnv( + shellEnv, + Effect.gen(function* () { + if (opts.reachable ?? true) yield* withReachableDirectHost(fixtureNet); + return yield* resolveLinkedConn( + ref, + opts.workdir ?? (yield* projectWorkdir([])), + "invalid", + "supabase.com", + "native", + opts.password ?? Option.some("secret"), + { poolerUrl: Option.fromNullishOr(opts.poolerUrl) }, + ); + }), + ).pipe(Effect.scoped); + + const tlsOf = (conn: PgConnInput) => ({ + sslmode: conn.sslmode, + sslrootcert: conn.sslrootcert, + sslcert: conn.sslcert, + sslkey: conn.sslkey, + }); + + it.effect( + "carries shell PGSSL*/PGCONNECT_TIMEOUT while PGHOST/PGUSER/PGPORT/PGDATABASE cannot retarget", + () => + Effect.gen(function* () { + const conn = yield* linkedConn({ + PGSSLMODE: "verify-full", + PGSSLROOTCERT: "/ca.pem", + PGSSLCERT: "/client.crt", + PGSSLKEY: "/client.key", + PGSSLPASSWORD: "keypass", + PGCONNECT_TIMEOUT: "7", + PGHOST: "evil.example", + PGUSER: "not-postgres", + PGPORT: "1", + PGDATABASE: "other", + }); + expect(conn).toEqual({ + host: directHost, + port: 5432, + user: "postgres", + password: "secret", + database: "postgres", + sslmode: "verify-full", + sslrootcert: "/ca.pem", + sslcert: "/client.crt", + sslkey: "/client.key", + sslpassword: "keypass", + connectTimeoutSeconds: 7, + }); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("leaves TLS unset when no libpq TLS variable is set", () => + Effect.gen(function* () { + const conn = yield* linkedConn({}); + expect(conn).toEqual({ + host: directHost, + port: 5432, + user: "postgres", + password: "secret", + database: "postgres", + }); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("lets a service-file verify-full beat PGSSLMODE=disable on the direct host", () => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const dir = yield* fs.makeTempDirectoryScoped({ prefix: "cli-linked-service-" }); + const serviceFile = path.join(dir, "pg_service.conf"); + yield* fs.writeFileString( + serviceFile, + "[svc]\nsslmode=verify-full\nsslrootcert=/svc-ca.pem\n", + ); + const conn = yield* linkedConn({ + PGSERVICE: "svc", + PGSERVICEFILE: serviceFile, + PGSSLMODE: "disable", + }); + expect(tlsOf(conn)).toEqual({ + sslmode: "verify-full", + sslrootcert: "/svc-ca.pem", + sslcert: undefined, + sslkey: undefined, + }); + expect(conn.host).toBe(directHost); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("names the libpq variables when PGSSLMODE is invalid on a reachable direct host", () => + Effect.gen(function* () { + const error = yield* linkedConn({ PGSSLMODE: "verifyfull" }).pipe(Effect.flip); + expect(error).toBeInstanceOf(DbConfigLibpqEnvError); + expect(error.message).toBe( + "invalid libpq environment: check PGSSLMODE, PGSSLCERT/PGSSLKEY, PGSERVICE/PGSERVICEFILE and PGCONNECT_TIMEOUT", + ); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect( + "still reports the IPv6 error for an invalid PGSSLMODE when the direct host is unreachable and no pooler URL is saved", + () => + Effect.gen(function* () { + const error = yield* linkedConn({ PGSSLMODE: "verifyfull" }, { reachable: false }).pipe( + Effect.flip, + ); + expect(error).toBeInstanceOf(DbConfigIpv6Error); + }).pipe(Effect.provide(linkedConnLayer)), + ); + + it.effect("reports an invalid PGSSLMODE, not IPv6, when the saved pooler URL is valid", () => + Effect.gen(function* () { + const savedUrl = `postgres://postgres.${ref}:[YOUR-PASSWORD]@${poolerHost}:6543/postgres`; + const error = yield* linkedConn( + { PGSSLMODE: "verifyfull" }, + { reachable: false, poolerUrl: savedUrl }, + ).pipe(Effect.flip); + expect(error).toBeInstanceOf(DbConfigLibpqEnvError); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("still reports the IPv6 error for a malformed saved pooler URL with a valid env", () => + Effect.gen(function* () { + const error = yield* linkedConn( + { PGSSLMODE: "verify-full" }, + { + reachable: false, + poolerUrl: `postgres://postgres.${ref}@${poolerHost}:notaport/postgres`, + }, + ).pipe(Effect.flip); + expect(error).toBeInstanceOf(DbConfigIpv6Error); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("ignores project .env TLS on the direct host and the saved pooler URL", () => + Effect.gen(function* () { + const savedUrl = `postgres://postgres.${ref}:[YOUR-PASSWORD]@${poolerHost}:6543/postgres`; + const workdir = yield* projectWorkdir([ + "PGSSLMODE=disable", + "PGSSLROOTCERT=/ca-from-dotenv.pem", + ]); + const shellTls = { + sslmode: "verify-full", + sslrootcert: undefined, + sslcert: undefined, + sslkey: undefined, + }; + const direct = yield* linkedConn({ PGSSLMODE: "verify-full" }, { workdir }); + expect(direct.host).toBe(directHost); + expect(tlsOf(direct)).toEqual(shellTls); + const pooler = yield* linkedConn( + { PGSSLMODE: "verify-full" }, + { workdir, reachable: false, poolerUrl: savedUrl }, + ); + expect(pooler.host).toBe(poolerHost); + expect(tlsOf(pooler)).toEqual(shellTls); + }).pipe(Effect.scoped, Effect.provide(linkedConnLayer)), + ); + + it.effect("keeps shell TLS settings on the temp login role for a reachable direct host", () => + Effect.gen(function* () { + const conn = yield* linkedConn( + { PGSSLMODE: "verify-full", PGSSLROOTCERT: "/ca.pem" }, + { password: Option.none() }, + ); + expect(conn).toEqual({ + host: directHost, + port: 5432, + user: "cli_login_x", + password: "temp", + database: "postgres", + sslmode: "verify-full", + sslrootcert: "/ca.pem", + }); + }).pipe(Effect.provide(linkedConnLayer)), + ); +}); diff --git a/apps/cli/src/command-internal/db-config.layer.ts b/apps/cli/src/command-internal/db-config.layer.ts index d11fe6db49..ee59d1bdc8 100644 --- a/apps/cli/src/command-internal/db-config.layer.ts +++ b/apps/cli/src/command-internal/db-config.layer.ts @@ -1,6 +1,17 @@ import * as net from "node:net"; import { BunServices } from "@effect/platform-bun"; -import { Config, Crypto, Duration, Effect, FileSystem, Layer, Option, Path } from "effect"; +import { + Config, + Crypto, + Duration, + Effect, + FileSystem, + Layer, + Option, + Path, + Predicate, + Struct, +} from "effect"; import { CommandPlatformApiFactory } from "../auth/command-platform-api-factory.service.ts"; import { CliArgs } from "../shared/cli/cli-args.service.ts"; @@ -28,6 +39,7 @@ import { import * as Errors from "./db-config.errors.ts"; import { layeredParseEnv, + POOLER_URL_UNPARSABLE, poolerConfigFromConnectionString, parseConnectionString, redactConnectionString, @@ -294,9 +306,13 @@ const waitForTempRole = Effect.fn("DbConfig.waitForTempRole")(function* ( return yield* attempt(1); }); +const LIBPQ_ENV_MESSAGE = + "invalid libpq environment: check PGSSLMODE, PGSSLCERT/PGSSLKEY, PGSERVICE/PGSERVICEFILE and PGCONNECT_TIMEOUT"; + /** * Parse + validate the configured pooler connection string. Returns `None` (treated as "no - * pooler" → IPv6 error) on any validation failure. + * pooler" → IPv6 error) on any validation failure, except an invalid libpq `PG*` environment, + * which fails with {@link Errors.DbConfigLibpqEnvError}. */ const poolerConfigFrom = Effect.fnUntraced(function* ( ref: string, @@ -305,7 +321,15 @@ const poolerConfigFrom = Effect.fnUntraced(function* ( ) { const debug = yield* DebugLogger; const result = yield* poolerConfigFromConnectionString(ref, connectionString, poolerHost); - if (result._tag === "ok") return Option.some(result.conn); + if (Predicate.isTagged(result, "ok")) return Option.some(result.conn); + /* An env-only parse tells an invalid `PG*` setting apart from a malformed pooler URL. */ + if ( + result.reason === POOLER_URL_UNPARSABLE && + (yield* parseConnectionString("postgresql://postgres@localhost:5432/postgres?passfile=")) === + undefined + ) { + return yield* new Errors.DbConfigLibpqEnvError({ message: LIBPQ_ENV_MESSAGE }); + } yield* debug.debug(result.reason); return Option.none(); }); @@ -472,11 +496,30 @@ export const resolveLinkedConn = Effect.fn("DbConfig.resolveLinkedConn")(functio const reachable = yield* tcpReachable(host, DIRECT_PORT); yield* Effect.annotateCurrentSpan("db.direct_reachable", reachable); if (reachable) { - if (base.password.length > 0) { + /* Same libpq TLS and timeout precedence as the pooler URL; host, port, role and database stay + pinned. The empty `passfile=` skips a `.pgpass` read for a password that is discarded. */ + const parsed = yield* parseConnectionString( + `postgresql://postgres@${host}:${DIRECT_PORT}/postgres?passfile=`, + ); + if (parsed === undefined) { + return yield* new Errors.DbConfigLibpqEnvError({ message: LIBPQ_ENV_MESSAGE }); + } + const direct = { + ...base, + ...Struct.pick(parsed, [ + "sslmode", + "sslrootcert", + "sslcert", + "sslkey", + "sslpassword", + "connectTimeoutSeconds", + ]), + }; + if (direct.password.length > 0) { yield* debug.debug("Using database password from env var..."); - return base; + return direct; } - return yield* initLoginRole(ref, base); + return yield* initLoginRole(ref, direct); } // Direct host unreachable (IPv6-only network) → try the pooler. For an ad-hoc `--project-id` diff --git a/apps/cli/src/command-internal/db-config.parse.ts b/apps/cli/src/command-internal/db-config.parse.ts index 7a4c1c58ab..cee27ae343 100644 --- a/apps/cli/src/command-internal/db-config.parse.ts +++ b/apps/cli/src/command-internal/db-config.parse.ts @@ -417,6 +417,8 @@ export type PoolerConfigResult = | { readonly _tag: "ok"; readonly conn: PgConnInput } | { readonly _tag: "invalid"; readonly reason: string }; +export const POOLER_URL_UNPARSABLE = "failed to parse pooler URL"; + /** * Parse + validate a Supabase transaction-pooler URL: strip the dashboard password placeholder, * require the project ref in the tenant user/options, verify the pooler domain belongs to the @@ -430,7 +432,7 @@ export const poolerConfigFromConnectionString = Effect.fnUntraced(function* ( const sanitized = connectionString.replaceAll("[YOUR-PASSWORD]", ""); const parsed = yield* parseConnectionString(sanitized); if (parsed === undefined) { - return { _tag: "invalid", reason: "failed to parse pooler URL" }; + return { _tag: "invalid", reason: POOLER_URL_UNPARSABLE }; } const optionsParam = parsed.options ?? ""; @@ -562,9 +564,10 @@ const parseUrlConnectionString = Effect.fnUntraced(function* ( svc("sslrootcert") ?? libpqEnv(env, "PGSSLROOTCERT") ?? null; - // libpq client cert (query, service, or PGSSLCERT/PGSSLKEY/PGSSLPASSWORD); both or neither, - // else this is a parse error. - const clientCert = resolveClientCert((key) => url.searchParams.get(key), svc, env); + /* libpq client cert (query, service, or PGSSLCERT/PGSSLKEY/PGSSLPASSWORD), skipped under + `sslmode=disable`; otherwise both or neither, else this is a parse error. */ + const clientCert = + sslmode === "disable" ? {} : resolveClientCert((key) => url.searchParams.get(key), svc, env); if (clientCert === "invalid") { return undefined; } @@ -799,8 +802,9 @@ const parseKeywordValueDsn = Effect.fnUntraced(function* ( if (isInvalidSslmode(sslmode)) return undefined; const sslrootcert = params.get("sslrootcert") ?? svc("sslrootcert") ?? libpqEnv(env, "PGSSLROOTCERT"); - // libpq client cert (keyword, service, or PG*); both or neither. - const clientCert = resolveClientCert((key) => params.get(key), svc, env); + /* libpq client cert (keyword, service, or PG*), skipped under `sslmode=disable`; both or neither. */ + const clientCert = + sslmode === "disable" ? {} : resolveClientCert((key) => params.get(key), svc, env); if (clientCert === "invalid") return undefined; const options = params.get("options") ?? svc("options"); // Every other keyword setting (e.g. search_path, statement_timeout) is a startup runtime diff --git a/apps/cli/src/command-internal/db-config.parse.unit.test.ts b/apps/cli/src/command-internal/db-config.parse.unit.test.ts index 0cc3063ab7..cee9bf681d 100644 --- a/apps/cli/src/command-internal/db-config.parse.unit.test.ts +++ b/apps/cli/src/command-internal/db-config.parse.unit.test.ts @@ -249,6 +249,33 @@ layer(parseTestLayer)("parseConnectionString (URL form)", (it) => { }), ); + it.effect("ignores a one-sided client cert under sslmode=disable", () => + Effect.gen(function* () { + expect( + yield* parseConnectionString("postgres://u:pw@h/db?sslmode=disable&sslcert=/c/client.crt"), + ).toEqual({ + host: "h", + port: 5432, + user: "u", + password: "pw", + database: "db", + sslmode: "disable", + }); + expect( + yield* parseConnectionString( + "host=h user=u password=pw sslmode=disable sslkey=/c/client.key", + ), + ).toEqual({ + host: "h", + port: 5432, + user: "u", + password: "pw", + database: "u", + sslmode: "disable", + }); + }), + ); + it.effect("returns undefined for an unparseable URL", () => Effect.gen(function* () { expect(yield* parseConnectionString("postgres://user:pw@ bad host/db")).toBeUndefined(); diff --git a/apps/cli/src/command-internal/db-config.service.ts b/apps/cli/src/command-internal/db-config.service.ts index f8da056666..d4377428d7 100644 --- a/apps/cli/src/command-internal/db-config.service.ts +++ b/apps/cli/src/command-internal/db-config.service.ts @@ -14,6 +14,7 @@ import type { LocalDbRunningError } from "./db-bootstrap/local-db-running.ts"; import type { DbConfigConnectTempRoleError, DbConfigIpv6Error, + DbConfigLibpqEnvError, DbConfigListBansNetworkError, DbConfigListBansStatusError, DbConfigLoadError, @@ -30,6 +31,7 @@ import type { DbConfigFlags, ResolvedDbConfig } from "./db-config.types.ts"; export type DbConfigError = | Config.ConfigError | DbConfigParseUrlError + | DbConfigLibpqEnvError | DbConfigLoadError | LocalDbRunningError | ProjectRefNotLinkedError diff --git a/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts b/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts index 8f1da7de56..d841c1a303 100644 --- a/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts +++ b/apps/cli/src/command-internal/db-connection.sql-pg.integration.test.ts @@ -7,7 +7,7 @@ import * as net from "node:net"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Deferred, Duration, Effect, Layer } from "effect"; +import { Deferred, Duration, Effect, FileSystem, Layer, Path } from "effect"; import { SUGGEST_ENV_VAR, SUGGEST_LOCAL_STACK } from "./connect-errors.ts"; import type { DbConnectError, DbExecError } from "./db-connection.errors.ts"; @@ -1032,4 +1032,36 @@ describe("a local target's explicit TLS request (CLI-2366: honor --db-url's own ); }), ); + + it.live("skips missing CA and client-cert files when sslmode=disable on a remote target", () => + Effect.gen(function* () { + const server = yield* fakeStartupServer(); + yield* Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + /* Uncreated children of a fresh temp dir, so the cert files are guaranteed absent. */ + const dir = yield* fs.makeTempDirectoryScoped({ prefix: "cli-missing-tls-files-" }); + const pool = yield* acquirePgPool( + { + host: "127.0.0.1", + port: server.port, + user: "postgres", + password: SENTINEL_PASSWORD, + database: "postgres", + sslmode: "disable", + sslrootcert: path.join(dir, "ca.pem"), + sslcert: path.join(dir, "client.crt"), + sslkey: path.join(dir, "client.key"), + }, + { isLocal: false, dnsResolver: "native" }, + ); + yield* Effect.tryPromise(() => pool.query("select 1")); + }).pipe( + Effect.scoped, + Effect.provide(BunServices.layer), + Effect.ensuring(Effect.sync(server.close)), + ); + expect(server.sawSslRequest()).toBe(false); + }), + ); }); diff --git a/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts b/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts index 05071a657b..fa99bc7109 100644 --- a/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts +++ b/apps/cli/src/command-internal/db-connection.sql-pg.layer.ts @@ -725,12 +725,15 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC // ({@link sslConfigsFor} already plaintexts socket targets). const rootcertPath = cfg.sslrootcert; const anyTcpTarget = dialTargets.some(({ dialHost }) => !isUnixSocketHost(dialHost)); + /* `sslmode=disable` never dials TLS, so leftover CA or client-cert paths cannot abort it. */ + const plaintextOnly = cfg.sslmode === "disable"; const caCert = cfg.sslrootcertInline !== undefined && cfg.sslrootcertInline.length > 0 && (!isLocal || explicitTls) ? cfg.sslrootcertInline - : rootcertPath !== undefined && + : !plaintextOnly && + rootcertPath !== undefined && rootcertPath.length > 0 && (!isLocal || explicitTls) && anyTcpTarget @@ -750,7 +753,7 @@ const acquirePgPoolConnection = (cfg: PgConnInput, { isLocal, dnsResolver }: DbC const certPath = cfg.sslcert; const keyPath = cfg.sslkey; const clientCert = - certPath !== undefined && keyPath !== undefined && !isLocal && anyTcpTarget + !plaintextOnly && certPath !== undefined && keyPath !== undefined && !isLocal && anyTcpTarget ? { cert: yield* readFileUtf8(fs, certPath).pipe( Effect.mapError( diff --git a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md index dedd2d4caa..5702c4f148 100644 --- a/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/bootstrap/SIDE_EFFECTS.md @@ -20,6 +20,7 @@ command — see Notes). | `<workdir>/supabase/migrations/*.sql` | SQL | native push step, for each pending migration applied | | seed files from `[db.seed].sql_paths` | SQL | native push step (`--include-seed` is always set; gated on `[db.seed].enabled`) | | `<workdir>/supabase/roles.sql` | SQL | native push step (`--include-roles` is always set; existence check + apply) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | native push step, linked project, when set; see `db push` | ## Files Written @@ -72,6 +73,7 @@ neither branch ever reaches the temp-login-role/Management-API path a passwordle | `SUPABASE_YES` | auto-confirm the native push step's prompts, read project-`.env`-aware like the standalone `db push` | no | | `SUPABASE_EXPERIMENTAL_STACK` | blank/`scratch` path only; when `1`, persists `[experimental] stack = true` and omits Docker-era default ports. Empty is unset; other non-empty values fail closed | no | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | native push step, linked-project TLS, timeout, service; see `db push`. An invalid value fails the push step, after project create, link, health poll, and `.env` write; the IPv6-error fallback takes none (see Notes) | no | ## Exit Codes @@ -170,6 +172,7 @@ the project lease. See `link`'s `SIDE_EFFECTS.md` for the format. its best-effort direct-host config rather than aborting — falling back to the same direct-host shape already computed for `.env` (step K's `dbConfig`) so the retry-wrapped push below still gets real reconnect attempts across the backoff window instead of failing bootstrap immediately. + That fallback config takes no `PG*` libpq setting, so it neither applies nor validates them. - **Password**: bootstrap never forwards a password to its internal push call on a separate channel — it always reuses the create-resolved password (`created.dbPassword`). There is no flag-vs-env distinction to preserve once the call is in-process (unlike the former subprocess diff --git a/apps/cli/src/commands/db/advisors/SIDE_EFFECTS.md b/apps/cli/src/commands/db/advisors/SIDE_EFFECTS.md index 2701fc2e69..a82411d588 100644 --- a/apps/cli/src/commands/db/advisors/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/advisors/SIDE_EFFECTS.md @@ -11,6 +11,7 @@ Management API. | `<workdir>/supabase/config.toml` | TOML | local / `--db-url` — to resolve the DB connection config | | `~/.supabase/access-token` | plain text | `--linked` only, when `SUPABASE_ACCESS_TOKEN` unset (keyring → file) | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` only — to resolve the project ref; skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked: `pg_service.conf` if `PGSERVICE` is set; PEM files only on the pooler temp-role probe; see `db push` | ## Files Written @@ -47,6 +48,7 @@ One connection. Within one transaction: `BEGIN` → `set local search_path = ''` | `SUPABASE_PROJECT_ID` | linked project ref override (superseded by `--project-ref` when set) | no | | `SUPABASE_PROFILE` | API profile (built-in name or YAML path) | no | | `PGHOST` / `PGPORT` / … | connection overrides (local / `--db-url`) | no | +| `PG*` libpq | linked: validated; used only by the temp-role probe; see `db push` | no | The API base URL is derived from `SUPABASE_PROFILE`; `SUPABASE_API_URL` is **not** honored (see `command-settings.layer.unit.test.ts`). diff --git a/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md b/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md index 1217eaf8a8..679781b858 100644 --- a/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/diff/SIDE_EFFECTS.md @@ -41,6 +41,7 @@ it, and JSON `null` disables formatting without disabling safe compaction. | `[db.migrations].schema_paths` globs / `<workdir>/supabase/database/**` / `<workdir>/supabase/schemas/**` | SQL | migra engine only, for the local-target declarative-schema fallback; pg-delta always compares the migrations baseline directly to the live target | | `~/.supabase/access-token` | plain text | `--linked` / `--db-url` with no `SUPABASE_ACCESS_TOKEN` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; under migra or pgAdmin the PEM files are read only on the pooler temp-role probe or migra's OOM-fallback schema listing; see `db push` | ## Files Written @@ -119,6 +120,7 @@ of this command's own target resolve, ahead of the differ container. | `SUPABASE_INTERNAL_IMAGE_REGISTRY` | overrides the differ's / shadow's image registry (shell **or** project `.env`, passed to each image operation) | no | | `SUPABASE_DB_ORIOLEDB_VERSION` | overrides `db.orioledb_version`, selecting the OrioleDB shadow image | no | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. The migra and pgadmin containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA outside its OOM fallback) | no | `SUPABASE_DB_SHADOW_PORT`/`SUPABASE_NETWORK_ID`/`--network-id`/`SUPABASE_PROJECT_ID`/ `SUPABASE_DB_HEALTH_TIMEOUT` all apply to `--use-pgadmin` too — its shadow is provisioned diff --git a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md index 0684a7892c..a0f2b22c69 100644 --- a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md @@ -15,6 +15,7 @@ image), to stdout or `--file`. | `~/.supabase/access-token` | plain text | `--linked` when `SUPABASE_ACCESS_TOKEN` unset | | `supabase/.temp/project-ref` | plain text | `--linked` (and the default target) ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`/config.toml `project_id`) is set | | `supabase/.env*` | dotenv | always (project env, feeds `SUPABASE_DB_PASSWORD` / `PG*`) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set: `pg_service.conf`, plus the PEM files on the pooler temp-role probe or a native `pg_dump`; see `db push` | ## Files Written @@ -45,6 +46,7 @@ image), to stdout or `--file`. | `SUPABASE_USE_SLIM_IMAGES` | resolve the current Postgres pin from the slim `ghcr.io/supabase/cli` builds (`true`/`1` enable); majors 13/15 use `15.14.1.167` when the flag is on; historical pins, PG14, OrioleDB tags the catalog does not pin, and flag-off `15.8.1.085` stay on docker.io | | `DOCKER_HOST` | docker daemon endpoint | | `MSYSTEM`, `TERM_PROGRAM` | suppress the piped-stdout non-ASCII warning in MSYS/mintty sessions | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. An invalid value fails before `pg_dump` runs; the compose `pg_dump` container gets none, a native `pg_dump` sees shell values only | ## Exit Codes diff --git a/apps/cli/src/commands/db/lint/SIDE_EFFECTS.md b/apps/cli/src/commands/db/lint/SIDE_EFFECTS.md index 2985353e16..17f8631cde 100644 --- a/apps/cli/src/commands/db/lint/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/lint/SIDE_EFFECTS.md @@ -9,6 +9,7 @@ Checks a database for PL/pgSQL typing errors via the `plpgsql_check` extension. | `<workdir>/supabase/config.toml` | TOML | always — to resolve the local / linked DB connection config | | `~/.supabase/access-token` | plain text | `--linked` only, when `SUPABASE_ACCESS_TOKEN` unset (keyring → file) | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` only, to resolve the project ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -53,6 +54,7 @@ container stacks can enable it. Compose uses the Compose Postgres image. | `SUPABASE_DB_PASSWORD` | linked direct-DB password | no | | `SUPABASE_PROFILE` | API profile (built-in name or YAML path) | no | | `PGHOST` / `PGPORT` / … | connection overrides | no | +| `PG*` libpq | linked project; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md index b1373ac65b..4f3d112773 100644 --- a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md @@ -48,6 +48,7 @@ disables formatting without disabling safe compaction. | `~/.supabase/cache/shadow-baseline/shadow-baseline-<key>.tar.<pid>.partial` | tar | abandoned-partial sweep on every cache-eligible acquire (warm hit and cold export) — enumerated and `stat`ed, and removed when older than 5 minutes (a crashed/SIGKILLed earlier export's leftover) | | `~/.supabase/access-token` | plain text | linked target with no `SUPABASE_ACCESS_TOKEN` | | `<workdir>/supabase/.temp/project-ref` | plain text | linked ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -142,6 +143,7 @@ at all, so nothing is cached for it. | `SUPABASE_SHADOW_CACHE` | shadow baseline cache; on by default, opt-out (`0`/`false`); the shadow's post-baseline state is saved under a managed snapshot key and restored into the next run's fresh stack database (see Notes) | no | | `SUPABASE_EXPERIMENTAL_PG_DELTA` | force pg-delta diff engine | no | | `SUPABASE_EXPERIMENTAL` | selects the deprecated in-process structured-dump export (same as `--declarative`) when `--declarative` is not set | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. The `pg_dump` and migra containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA outside its OOM fallback) | no | ## Exit Codes diff --git a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md index 444a46c2ef..5f066ba674 100644 --- a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md @@ -15,6 +15,8 @@ before migrations unless `--skip-vault` is set. | `<workdir>/supabase/migrations/*.sql` | SQL | for each pending migration, when applied (and not `--dry-run`) | | seed files from `[db.seed].sql_paths` | SQL | when `--include-seed` and `[db.seed].enabled` (paths under `supabase/`) | | `<workdir>/supabase/roles.sql` | SQL | when `--include-roles` (existence check + apply) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | `--db-url` or the linked project, when the DSN or env sets them; no PEM file is read under `sslmode=disable` (see Notes) | +| `.pgpass` (`pgpass.conf` on Windows) | libpq | `--db-url` with no password, or the linked pooler route (password discarded); `PGPASSFILE` overrides the path (see Notes) | ## Files Written @@ -50,6 +52,7 @@ before migrations unless `--skip-vault` is set. | `SUPABASE_PROJECT_ID` | linked-ref resolution override, superseded by `--project-ref` when set (same precedence position) — see Notes | no | | `DOTENV_PRIVATE_KEY*` | decrypts `encrypted:` config secrets; `[db.vault]` values are not decrypted with `--skip-vault` | no | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked project, shell only: `PGSSLMODE`, `PGSSLROOTCERT`, `PGSSLCERT`/`PGSSLKEY`/`PGSSLPASSWORD`, `PGCONNECT_TIMEOUT`, `PGSERVICE`/`PGSERVICEFILE`; `--db-url`: any `PG*` the DSN omits, shell first, then project `.env*` (see Notes) | no | ## Exit Codes @@ -130,3 +133,18 @@ Warning: supabase db push will target the local database by default in the next Prefer idempotent forms (`CREATE INDEX CONCURRENTLY IF NOT EXISTS …`) and isolating such statements in their own migration file. Intentional fix for supabase/cli#5139, adopted into TS in PR supabase/cli#5671 (landed on develop as `b48fad60`). +- **Linked libpq env.** On every linked database connection, `PGSSLMODE`, `PGSSLROOTCERT`, + `PGSSLCERT`/`PGSSLKEY`/`PGSSLPASSWORD`, `PGCONNECT_TIMEOUT`, and `PGSERVICE`/`PGSERVICEFILE` are + read from the shell only, never the project `.env*` (`gen types` keeps its pinned mode and CA, + except on the pooler temp-role probe), on the direct host and the IPv4 pooler alike; host, port, + role, and database stay pinned. As in libpq, a `PGSERVICE` entry's settings outrank these vars. + An invalid mode or `PGCONNECT_TIMEOUT`, an unknown `PGSERVICE`, or a lone `PGSSLCERT`/`PGSSLKEY` + outside `sslmode=disable` fails with `invalid libpq environment` on either route. + `sslmode=disable` dials plaintext and reads no CA or client-cert file; otherwise those PEM files + are read when set and a missing one fails the connection, and a `pg_service.conf` is read + whenever `PGSERVICE` names a service. The pooler route also reads `.pgpass` (`PGPASSFILE`, else + `~/.pgpass`, or `%APPDATA%\postgresql\pgpass.conf` on Windows) and discards that password, and + it keeps `PGAPPNAME` and a service's `options` and runtime keys, which the direct host drops. + Tool containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA + outside its OOM fallback); migra and pgadmin get the timeout as `connect_timeout` and the + pooler's `options` and runtime keys in their URL. diff --git a/apps/cli/src/commands/db/query/SIDE_EFFECTS.md b/apps/cli/src/commands/db/query/SIDE_EFFECTS.md index 71bf687c44..4d2ca3a78f 100644 --- a/apps/cli/src/commands/db/query/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/query/SIDE_EFFECTS.md @@ -14,6 +14,7 @@ the result as a table or JSON. | `~/.supabase/access-token` | plain text | `--linked` when `SUPABASE_ACCESS_TOKEN` unset | | `supabase/.temp/project-ref` | plain text | `--linked` ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | | `supabase/.temp/linked-project.json` | JSON | `--linked` existence check before the cache write (see below) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -34,6 +35,7 @@ the result as a table or JSON. | ----------------------- | ---------------------------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | `--linked` auth | | agent-detection signals | `--agent=auto` (e.g. `CURSOR_*`, `CLAUDECODE`, …) via `@vercel/detect-agent` | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | ## Exit Codes diff --git a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md index 5b7b0a9515..4b83d29bfe 100644 --- a/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/reset/SIDE_EFFECTS.md @@ -76,6 +76,7 @@ or migrations fail, companion services remain stopped. Fix the reported error an | `<workdir>/supabase/buckets/` | files | local path, when storage is up and `[storage.buckets]` configure objects | | `<workdir>/supabase/roles.sql` | SQL | local PG15 path only, via the reused `startSetupLocalDatabase` pipeline — missing file tolerated | | `~/.docker/config.json` + Docker context store (`contexts/meta/<sha256(context)>/meta.json`) | JSON | resolving the daemon endpoint for the local path's running probe (in-process); also read by the `docker`/`podman` CLI itself for registry auth | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -179,6 +180,7 @@ the whole reset** (not just "skip buckets"). | `SUPABASE_NETWORK_ID` (`--network-id`) | forces the recreated container/network onto an existing Docker network — legacy backend only | no | | `SUPABASE_EXPERIMENTAL_STACK` | `1`/`0` selects the stack backend for the local path, overriding `experimental.stack` | no (→ `experimental.stack` → legacy) | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Connection loss during migration apply diff --git a/apps/cli/src/commands/db/schema/declarative/generate/SIDE_EFFECTS.md b/apps/cli/src/commands/db/schema/declarative/generate/SIDE_EFFECTS.md index a8fff1d00e..454898f45b 100644 --- a/apps/cli/src/commands/db/schema/declarative/generate/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/schema/declarative/generate/SIDE_EFFECTS.md @@ -24,6 +24,7 @@ formatting without disabling safe compaction. | `<workdir>/supabase/.temp/postgres-version` | plain text | smart-mode Local flow — the local Postgres image-currency check's version-pin lookup | | `<workdir>/supabase/migrations/*.sql` | SQL | smart mode — detect whether migrations exist | | `~/.supabase/access-token` | plain text | `--linked` (token resolution) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -51,6 +52,7 @@ formatting without disabling safe compaction. | `DOCKER_HOST` | tcp daemon host used as the local DB host fallback | no | | `SUPABASE_USE_SLIM_IMAGES` | resolves the expected local `db` image for the stale-container guard from the slim `ghcr.io/supabase/cli` builds (`true`/`1` enable); majors 13/15 use `15.14.1.167` when the flag is on; historical pins, PG14, OrioleDB tags the catalog does not pin, and flag-off `15.8.1.085` stay on docker.io | no | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md b/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md index 4d47b6e92d..4b21f138f9 100644 --- a/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/schema/declarative/sync/SIDE_EFFECTS.md @@ -33,6 +33,8 @@ disabling safe compaction. | `<workdir>/supabase/schemas/.pgdelta-export.json` | JSON | export metadata, when present | | `~/.supabase/cache/shadow-baseline/shadow-baseline-<key>.tar` | tar | warm shadow-cache hit (migrations/declarative shadows); every cache-eligible acquire (warm hit and successful cold export) also enumerates and `stat`s every `shadow-baseline-*.tar` for LRU keep-3 + 2-day mtime TTL and may delete other keys (`SUPABASE_HOME` overrides the `~/.supabase` root) | | `~/.supabase/cache/shadow-baseline/shadow-baseline-<key>.tar.<pid>.partial` | tar | abandoned-partial sweep on every cache-eligible acquire (warm hit and cold export) — enumerated and `stat`ed, and removed when older than 5 minutes (a crashed/SIGKILLed earlier export's leftover) | +| `PGSSL*` PEM files | PEM | no-files bootstrap on **Linked project**, when set; see `db push` | +| `pg_service.conf` | libpq | no-files bootstrap on **Linked project**, when set; see `db push` | ## Files Written @@ -61,6 +63,7 @@ disabling safe compaction. | `SUPABASE_SERVICES_HOSTNAME` | local DB host for the bootstrap generate | no | | `DOCKER_HOST` | tcp daemon host used as the local DB host fallback | no | | `SUPABASE_USE_SLIM_IMAGES` | resolves the current-pin shadow Postgres and PG15+ realtime/storage/auth migrate-job images from the slim `ghcr.io/supabase/cli` builds (`true`/`1` enable); majors 13/15 use `15.14.1.167` when the flag is on; historical pins, PG14, OrioleDB tags the catalog does not pin, and flag-off `15.8.1.085` stay on docker.io | no | +| `PG*` libpq | no-files bootstrap only, when the target prompt picks **Linked project**: linked-project TLS, timeout, service; see `db push`. Sync itself has no `--linked` flag | no | ## Exit Codes diff --git a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md index f7a3c8fcce..3ea7d6cb1f 100644 --- a/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/gen/types/SIDE_EFFECTS.md @@ -26,6 +26,7 @@ is on, `--local` resolves the project stack through `DbConfigResolver` | `~/.pg_service.conf` (or a `PGSERVICEFILE`/DSN `servicefile=` override) | libpq service file | `--db-url`, only when the DSN or `PGSERVICE` names a `service` — its settings fill in for connection fields the DSN itself omits | | `Package.resolved` (nearest) | JSON | `--lang swift` without `SUPABASE_TYPEGEN`, on macOS or Linux x86_64: the first one found walking up from the working directory, including the copies inside an `.xcodeproj` or `.xcworkspace`; the supabase-swift version it pins picks the `supabase-typegen` release | | `<workdir>/supabase/.temp/rest-version` | plain text | `--local` on the legacy backend only, when `db.major_version > 14` — forces v9 compat if the tag contains `v9`; the stack backend never reads it | +| `$PGSSLROOTCERT`, `$PGSSLCERT` / `$PGSSLKEY`, `pg_service.conf` | PEM, libpq | non-TypeScript `--lang` on `--linked`, `--project-id`, or the implicit linked fallback, when `PGSSLCERT`/`PGSSLKEY` or `PGSERVICE` are set; `$PGSSLROOTCERT` only on the pooler temp-role probe; the PEM files are skipped under `PGSSLMODE=disable` | ## Files Written @@ -106,6 +107,8 @@ way `--db-url` does. | `XDG_CACHE_HOME`, `HOME` | locate the cache directory of the `--lang swift` release binary | | `PATH`, `PATHEXT`, `ComSpec` | find the tool of an out-of-process `--lang`; `PATHEXT` and `ComSpec` matter on Windows only | no (`PATHEXT` defaults to `.COM;.EXE;.BAT;.CMD`, `ComSpec` to `cmd.exe`; without `PATH` the tool is reported as not installed) | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PGSSLMODE`, `PGSSLROOTCERT`, `PGSSLCERT`, `PGSSLKEY`, `PGSSLPASSWORD`, `PGSERVICE`, `PGSERVICEFILE` | non-TypeScript `--lang` on `--linked`, `--project-id`, or the implicit linked fallback: client-cert pair outside `sslmode=disable`; invalid values fail; mode and CA stay pinned except on the pooler temp-role probe (see Notes) | no | +| `PGCONNECT_TIMEOUT` | same paths: the 15s `--query-timeout` default pins it on explicit `--linked`/`--project-id`; the implicit linked fallback keeps it under `--query-timeout 0`; the pooler temp-role probe always uses it | no | ## Exit Codes @@ -167,7 +170,11 @@ Warning: supabase gen types will target the local database by default in the nex - **TLS.** There is no SSL probe on any path. - `--linked` / `--project-id` / the implicit linked fallback / a resolved preview branch connect with `sslmode=require` and the bundled Supabase CA pinned - (promoted to `verify-ca`), matching prior behavior. + (promoted to `verify-ca`), matching prior behavior. On a project ref the `PG*` env + (see `db push`) is validated and supplies the client-cert pair, and `--query-timeout` + sets the connect timeout unless it is `0`; the pooler temp-role probe runs before the pin + and takes the shell `PG*` TLS and timeout settings; a preview branch's direct host takes + no `PG*` setting. - `--db-url` honors the DSN's own `sslmode`/`sslrootcert` when either is set, even against a target classified local (e.g. a TLS tunnel on the loopback stack); otherwise, a known Supabase host gets the Supabase CA pinned the same way, and any diff --git a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md index 9c83fc2cd5..c5cf9fa364 100644 --- a/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md @@ -12,8 +12,9 @@ SQL run and the columns rendered (see the per-subcommand `<name>.query.ts`). | -------------------------------- | ---------- | ----------------------------------------------------------------------------------- | | `<workdir>/supabase/config.toml` | TOML | `--local` (db host/port/password); `--linked` (project ref) | | `~/.supabase/access-token` | plain text | `--linked` only, lazily, when `SUPABASE_ACCESS_TOKEN` unset and keyring unavailable | -| `.pgpass` / `pg_service.conf` | libpq | only if referenced by a `--db-url` connection string | -| `$PGSSLROOTCERT` CA bundle | PEM | only if a `--db-url` sets `sslrootcert` / `PGSSLROOTCERT` | +| `.pgpass` / `pg_service.conf` | libpq | if referenced by a `--db-url` connection string | +| `$PGSSLROOTCERT` CA bundle | PEM | if a `--db-url` sets `sslrootcert` / `PGSSLROOTCERT` | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | Connection resolution and all of the above are handled inside the already-ported `DbConfigResolver` (`command-internal/db-config.layer.ts`); this port adds @@ -42,6 +43,7 @@ no new config reads. | `SUPABASE_ACCESS_TOKEN` | Management API auth (linked only) | no (falls back to keyring / token file) | | `SUPABASE_PROJECT_ID` | project ref fallback (linked), superseded by `--project-ref` when set | no (config resolution fallback) | | libpq vars (`PGSSLROOTCERT`, `PGCONNECT_TIMEOUT`, …) | honored when `--db-url` is used | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Database Queries diff --git a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md index 5054652f91..5bb215309d 100644 --- a/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/inspect/report/SIDE_EFFECTS.md @@ -14,6 +14,7 @@ validating those CSVs. | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` path, without `--project-ref` or `SUPABASE_PROJECT_ID` (linked project ref) | | `~/.supabase/access-token` | plain text | `--linked` path, when `SUPABASE_ACCESS_TOKEN` unset and keyring unavailable | | `<output-dir>/<YYYY-MM-DD>/<name>.csv` ×14 | CSV | read back in-memory for rule evaluation | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | A **missing** `config.toml` is fine (defaults apply); a **malformed** file aborts the command. @@ -72,6 +73,7 @@ resolve the connection (via `DbConfigResolver`). | `SUPABASE_PROFILE` | built-in profile name or YAML file path | no (falls back to `~/.supabase/profile` -> `supabase`) | | `SUPABASE_DB_*` | override `[db]` port / shadow_port / password | no | | `SUPABASE_ENV` | selects which project `.env` files load | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/migration/down/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/down/SIDE_EFFECTS.md index 9833dc7379..3f5b244ddb 100644 --- a/apps/cli/src/commands/migration/down/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/down/SIDE_EFFECTS.md @@ -7,6 +7,7 @@ | `<workdir>/supabase/migrations/` | directory | always, to read migration files | | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and `--linked` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked`, to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -27,6 +28,7 @@ | `SUPABASE_ACCESS_TOKEN` | auth token for `--linked` mode | no (falls back to keyring → `~/.supabase/access-token`) | | `DOTENV_PRIVATE_KEY[_*]` | dotenvx private key(s) to decrypt `encrypted:` `[db.vault]` secrets before upsert | no (required only if a `[db.vault]` value is encrypted) | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md index 93d389918d..21c5f269bd 100644 --- a/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/fetch/SIDE_EFFECTS.md @@ -7,6 +7,7 @@ | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and `--linked` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` (default), to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | | `<workdir>/supabase/.env*`, `<workdir>/.env*` | dotenv | always, to resolve `SUPABASE_YES` (CLI-1878) | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -26,6 +27,7 @@ | ----------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | auth token for `--linked` mode | no (falls back to keyring → `~/.supabase/access-token`) | | `SUPABASE_YES` | auto-confirm the overwrite prompt | no — read from the shell env OR the project `.env`/`.env.local`/`.env.<env>[.local]` files (shell wins; CLI-1878) | +| `PG*` libpq | linked project; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md index 619f33cf95..5607c30257 100644 --- a/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/list/SIDE_EFFECTS.md @@ -7,6 +7,7 @@ | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and `--linked` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` (default), to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | | `<workdir>/supabase/migrations/` | directory | always, to list local migration files | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -26,6 +27,7 @@ | ----------------------- | --------------------------------------- | ------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | auth token for `--linked` mode | no (falls back to keyring → `~/.supabase/access-token`) | | `DB_PASSWORD` | password for direct database connection | no | +| `PG*` libpq | linked project; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md index 7ba716718e..512cf7b1ab 100644 --- a/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/repair/SIDE_EFFECTS.md @@ -6,6 +6,7 @@ | -------------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------- | | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and `--linked` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked` (default), to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -25,6 +26,7 @@ | ----------------------- | --------------------------------------- | ------------------------------------------------------- | | `SUPABASE_ACCESS_TOKEN` | auth token for `--linked` mode | no (falls back to keyring → `~/.supabase/access-token`) | | `DB_PASSWORD` | password for direct database connection | no | +| `PG*` libpq | linked project; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md index 904c587020..62df24b373 100644 --- a/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/squash/SIDE_EFFECTS.md @@ -31,6 +31,7 @@ and uses an automatically assigned port; `db.shadow_port` applies only to the le | `<workdir>/supabase/.temp/{project-ref,postgres-version,pooler-url}` | plain text | `--linked` / linked path — the ref is not taken from `project-ref` when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set; `pooler-url` is used only when the target ref is the linked ref | | `~/.supabase/access-token` | plain text | `--linked` without `--password`/`SUPABASE_ACCESS_TOKEN` | | `~/.docker/config.json` + Docker context store | JSON | resolving the Docker hostname for shadow/pg_dump containers | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -88,7 +89,7 @@ live shadow available. `SUPABASE_YES`, `DB_PASSWORD`, `SUPABASE_ACCESS_TOKEN`, `SUPABASE_SERVICES_HOSTNAME`, `DOCKER_HOST`/`DOCKER_CONTEXT`/`DOCKER_CONFIG`, `SUPABASE_NETWORK_ID`, `SUPABASE_INTERNAL_IMAGE_REGISTRY`, `SUPABASE_USE_SLIM_IMAGES` (current-pin shadow Postgres and PG15+ realtime/storage/auth migrate-job images → slim `ghcr.io/supabase/cli`; historical pins, PG14, OrioleDB tags the catalog does not pin, flag-off `15.8.1.085` stay on docker.io), `SUPABASE_PROJECT_ID`, `SUPABASE_DEBUG`, -`SUPABASE_EXPERIMENTAL`, `SUPABASE_SHADOW_CACHE` (stack shadow baseline cache; on by default, falsy disables restore and publication). +`SUPABASE_EXPERIMENTAL`, `SUPABASE_SHADOW_CACHE` (stack shadow baseline cache; on by default, falsy disables restore and publication), `PG*` libpq on `--linked` (TLS, timeout, service; see `db push`). ## Exit Codes diff --git a/apps/cli/src/commands/migration/up/SIDE_EFFECTS.md b/apps/cli/src/commands/migration/up/SIDE_EFFECTS.md index 7069137693..6ec9ad58b4 100644 --- a/apps/cli/src/commands/migration/up/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/migration/up/SIDE_EFFECTS.md @@ -7,6 +7,7 @@ | `<workdir>/supabase/migrations/` | directory | always, to read pending migration files | | `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` unset and `--linked` | | `<workdir>/supabase/.temp/project-ref` | plain text | `--linked`, to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | ## Files Written @@ -27,6 +28,7 @@ | `SUPABASE_ACCESS_TOKEN` | auth token for `--linked` mode | no (falls back to keyring → `~/.supabase/access-token`) | | `DOTENV_PRIVATE_KEY[_*]` | dotenvx private key(s) to decrypt `encrypted:` `[db.vault]` secrets before upsert | no (required only if a `[db.vault]` value is encrypted) | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push` | no | ## Exit Codes diff --git a/apps/cli/src/commands/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/pull/SIDE_EFFECTS.md index 3cc2cef0e0..51cb806093 100644 --- a/apps/cli/src/commands/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/pull/SIDE_EFFECTS.md @@ -30,6 +30,7 @@ initial-pull `pg_dump` container). See "Database writes" and "Docker" below. | `<workdir>/supabase/config.toml` or `config.json` (reload) | TOML/JSON | re-loaded WITH the `[remotes.*]` overlay applied, only when the resolved target ref matches an existing `[remotes.*]` block — the config step's own conditional reload (`config/pull/SIDE_EFFECTS.md`) | | `<workdir>/supabase/migrations` (directory listing) | filenames | once, in the preview phase, to decide whether the migration-history step auto-runs — skipped when `--with-migration-history` is already set (see Notes); a read failure other than "directory missing" fails the whole command at this point | | same config file, raw on-disk text (TOCTOU re-read) | TOML/JSON | immediately before the config step writes, once the aggregated confirmation is accepted — only reached when the config plan has work; see `config/pull/SIDE_EFFECTS.md`'s TOCTOU row | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set; see `db push` | Each executed sub-step also performs its own file reads exactly as documented in its own `SIDE_EFFECTS.md` (config, db, functions, and — when it runs — migration history), since `pull` @@ -137,6 +138,7 @@ Docker-dependent. | `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | | `SUPABASE_PROFILE` | API profile selection | no | | `env(VAR)` references | interpolated into `config.toml` values by the config step's loader | no | +| `PG*` libpq | db step: linked-project TLS, timeout, service; see `db push`. Containers get no `PG*` TLS settings | no | Once the db and functions steps run, they consume their own established environment variables exactly as documented in their own `SIDE_EFFECTS.md` files (e.g. `SUPABASE_DB_PASSWORD`, diff --git a/apps/cli/src/commands/test/db/SIDE_EFFECTS.md b/apps/cli/src/commands/test/db/SIDE_EFFECTS.md index b85b60921b..15bcd2964c 100644 --- a/apps/cli/src/commands/test/db/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/test/db/SIDE_EFFECTS.md @@ -11,6 +11,8 @@ | `~/.supabase/access-token` | text | `--linked` only, when `SUPABASE_ACCESS_TOKEN` unset | | `<workdir>/supabase/.temp/project-ref` | text | `--linked` only, to resolve the ref — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`) is set | | `<supabaseHome>/stacks/*/state.json` | JSON | managed `--local`: discover the project stack and open its saved definition | +| `PGSSL*` PEM files | PEM | linked project, when set; see `db push` | +| `pg_service.conf` | libpq | linked project, when set; see `db push` | ## Files Written @@ -58,6 +60,7 @@ Managed stack `--local` prove runs the catalog `pg_prove` tool through the ownin | `BITBUCKET_CLONE_DIR` | when set, omit `--security-opt label:disable` (Bitbucket rejects it) | no | | `DEBUG` / `--debug` | append `--verbose` to `pg_prove` | no | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | no | +| `PG*` libpq | `--linked`, the CLI's own connection: linked-project TLS, timeout, service; see `db push`. A native `pg_prove` sees shell values only; `pg_prove` containers get none | no | ## Exit Codes diff --git a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt index c7d23f7af4..c53504bd7a 100644 --- a/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt +++ b/apps/cli/src/shared/telemetry/__fixtures__/error-tags.txt @@ -144,6 +144,7 @@ DbAdvisorsSecurityStatusError DbAdvisorsSetupError DbConfigConnectTempRoleError DbConfigIpv6Error +DbConfigLibpqEnvError DbConfigListBansNetworkError DbConfigListBansStatusError DbConfigLoadError From 3b79d74b1945685bdd75965349338cb2746a9e04 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Wed, 21 Oct 2026 10:55:26 +0000 Subject: [PATCH 07/24] chore(stack): bump edge-runtime to v1.77.8-r0 (#7272) This bumps edge-runtime from v1.77.6-r0 to v1.77.8-r0. https://github.com/supabase/slim-services/releases/tag/edge-runtime-v1.77.8-r0 Planned after supabase/slim-services published v1.77.8-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli-go/pkg/config/templates/Dockerfile | 2 +- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/Dockerfile b/apps/cli-go/pkg/config/templates/Dockerfile index 4104d5223c..19cf46c71d 100644 --- a/apps/cli-go/pkg/config/templates/Dockerfile +++ b/apps/cli-go/pkg/config/templates/Dockerfile @@ -25,7 +25,7 @@ FROM supabase/studio:2026.10.19-sha-b375769 AS studio FROM darthsim/imgproxy:v3.26.0 AS imgproxy -FROM supabase/edge-runtime:v1.77.6 AS edgeruntime +FROM supabase/edge-runtime:v1.77.8 AS edgeruntime FROM timberio/vector:0.58.0-alpine AS vector diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 4104d5223c..19cf46c71d 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -25,7 +25,7 @@ FROM supabase/studio:2026.10.19-sha-b375769 AS studio FROM darthsim/imgproxy:v3.26.0 AS imgproxy -FROM supabase/edge-runtime:v1.77.6 AS edgeruntime +FROM supabase/edge-runtime:v1.77.8 AS edgeruntime FROM timberio/vector:0.58.0-alpine AS vector diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index fe8a57c70a..9f349d8360 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -373,23 +373,23 @@ const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { functions: definition( "edge-runtime", { - upstreamVersion: "v1.77.6", + upstreamVersion: "v1.77.8", revision: 0, image: - "ghcr.io/supabase/cli/edge-runtime:v1.77.6-r0@sha256:6d1a5a63eb817e92bbe9b5083375ac4f702e1fd1120eff717ef56aca79a2ed2c", - upstreamImage: "supabase/edge-runtime:v1.77.6", + "ghcr.io/supabase/cli/edge-runtime:v1.77.8-r0@sha256:a162f950a1f5f69e40afcb2a72a2dd924fe14884cf6b9591723ef69ddcaaedb2", + upstreamImage: "supabase/edge-runtime:v1.77.8", natives: { "darwin-arm64": { - archive: "47c58394d4cde1eda132c18f081dcbbc08863e568f72bf24de1339a9e78a3cbf", - manifest: "f277afe91115a7fcb6a7f2306ba1f52f75f76dc2584df4eb361b3928a825985a", + archive: "6e114f7817beebac350be6bfbd14d12bd3e2b0b2d6e7a2e94eb11ede68213a60", + manifest: "7ee23c9287653a8239963c347a1887f5c541f69bfb356b3f7576cafbf467d528", }, "linux-amd64": { - archive: "2d6d32a063e287bd00fb311ff6d492876621ea00166aa1fe3d695d3eb9443a5c", - manifest: "3a7fa428f7adc563f6f792ca99aefe98a4e3a69ad580963fed3c291eb97950e3", + archive: "af349b253f0ad192d2eaba0b584a0e8a96118ce1b39f514389f40c66ea4e7502", + manifest: "c560ecd5c85bda5ba13f4deb1fc7138291af3149da018d8e83593b64e225a5ab", }, "linux-arm64": { - archive: "e1b9ca5238cb1244e995a116bb344ab94cc151f4b6394a27cbedecb0d8fa601c", - manifest: "e000ae8a38fa3063842cf0e96caed40f50fdfa47cd3114b7b2816111bb7c1056", + archive: "1360ec1f775b62cc81f2469ddfc53840034e9b9e73e02c2594be332ad3218dbe", + manifest: "3208b8a22583cae5522b64cac87b49c8210a32bf29dde6b24a9f7900bffa787d", }, }, }, From 7d2a9a9f840169fa49ea870252081931eb15d692 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Wed, 21 Oct 2026 10:59:09 +0000 Subject: [PATCH 08/24] refactor(config): cover the `io` module with effect lint (CLI-2756) (#7262) ## TL;DR brings `io.ts` and its tests under the effect lint ## whats introduced? - `packages/config/src/io.*` joins the allow list - `writeFileAtomic` reads the temp suffix time from the effect `Clock`, the same value `Date.now()` gave - `io.unit.test.ts` moves to effect: `FileSystem` and `Path`, scoped temp dirs, schema json codecs, and `vi.stubEnv` inside a scoped acquire and release instead of writing `process.env` ## ref - related: CLI-2756 --- .oxlintrc.effect.json | 1 + packages/config/src/io.ts | 5 +- packages/config/src/io.unit.test.ts | 4016 ++++++++++++++------------- 3 files changed, 2023 insertions(+), 1999 deletions(-) diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json index 97dde045f2..cb3630c581 100644 --- a/.oxlintrc.effect.json +++ b/.oxlintrc.effect.json @@ -6,6 +6,7 @@ }, "ignorePatterns": [ "**", + "!packages/config/src/io.*", "!packages/stack/**", "!apps/cli/src/auth/**", "!apps/cli/src/cli/**", diff --git a/packages/config/src/io.ts b/packages/config/src/io.ts index 80e4eb2d3c..3757d3e3ba 100644 --- a/packages/config/src/io.ts +++ b/packages/config/src/io.ts @@ -1,5 +1,5 @@ import { randomBytes } from "node:crypto"; -import { Console, Effect, FileSystem, Path, Predicate, Redacted } from "effect"; +import { Clock, Console, Effect, FileSystem, Path, Predicate, Redacted } from "effect"; import * as SmolToml from "smol-toml"; import { CliConfigSchema, type CliConfig } from "./base.ts"; import { @@ -774,7 +774,8 @@ const DEFAULT_CLI_CONFIG_FILE_MODE = 0o644; const writeFileAtomic = Effect.fnUntraced(function* (filePath: string, content: string) { const fs = yield* FileSystem.FileSystem; - const tmpPath = `${filePath}.tmp.${Date.now()}.${randomBytes(3).toString("hex")}`; + const now = yield* Clock.currentTimeMillis; + const tmpPath = `${filePath}.tmp.${now}.${randomBytes(3).toString("hex")}`; yield* Effect.gen(function* () { const mode = yield* fs.stat(filePath).pipe( diff --git a/packages/config/src/io.unit.test.ts b/packages/config/src/io.unit.test.ts index c71f9c7d7a..3d1af6fe8a 100644 --- a/packages/config/src/io.unit.test.ts +++ b/packages/config/src/io.unit.test.ts @@ -1,10 +1,5 @@ import { afterEach, describe, expect, test, vi, type MockInstance } from "vitest"; import { BunServices } from "@effect/platform-bun"; -import { mkdtempSync } from "node:fs"; -import { chmod, mkdir, readdir, readFile, rm, stat, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { dirname, join } from "node:path"; -import { fileURLToPath } from "node:url"; import { Cause, Deferred, @@ -18,6 +13,7 @@ import { PlatformError, Redacted, Schema, + type Scope, } from "effect"; import { CliConfigSchema, toCliConfigJsonSchema } from "./base.ts"; import { loadCliConfig as loadCliConfigFromBun } from "./bun.ts"; @@ -45,21 +41,40 @@ import { cliConfigStoreLayer } from "./cli-config.layer.ts"; import { CliConfigStore } from "./cli-config.service.ts"; import { CLI_CONFIG_SCHEMA_URL } from "./schema-metadata.ts"; -function makeTempProject(): string { - return mkdtempSync(join(tmpdir(), "supabase-config-")); +function makeTempProject() { + return FileSystem.FileSystem.use((fs) => + fs.makeTempDirectoryScoped({ prefix: "supabase-config-" }), + ); } -const legacyFixturePath = join( - dirname(fileURLToPath(import.meta.url)), - "../testdata/legacy-config.toml", -); +const legacyFixturePath = Effect.gen(function* () { + const path = yield* Path.Path; + return path.join( + path.dirname(yield* path.fromFileUrl(new URL(import.meta.url))), + "../testdata/legacy-config.toml", + ); +}); const decodeCliConfig = Schema.decodeUnknownSync(CliConfigSchema); +const encodeJson = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); +const decodeJson = Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown)); function runConfigEffect<A, E>( - effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path>, + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, ): Promise<A> { - return Effect.runPromise(effect.pipe(Effect.provide(BunServices.layer))); + return Effect.runPromise(effect.pipe(Effect.scoped, Effect.provide(BunServices.layer))); +} + +function stubEnv(name: string, value: string) { + return Effect.acquireRelease( + Effect.sync(() => { + vi.stubEnv(name, value); + }), + () => + Effect.sync(() => { + vi.unstubAllEnvs(); + }), + ); } const sampleConfig = decodeCliConfig({ @@ -72,131 +87,138 @@ const sampleConfig = decodeCliConfig({ }); describe("config io", () => { - test("saves JSON by default when no config exists", async () => { - const cwd = makeTempProject(); - - try { - const saved = await runConfigEffect(saveCliConfig({ cwd, config: sampleConfig })); - expect(saved.format).toBe("json"); - expect(saved.path).toBe(await runConfigEffect(configJsonPath(cwd))); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loads strict JSON", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configJsonPath(cwd)); + test("saves JSON by default when no config exists", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - JSON.stringify({ - project_id: "abc123", - db: { - major_version: 16, - }, - experimental: { stack: true }, - }), - ); - - const loaded = await runConfigEffect(loadCliConfigFile(path)); - expect(loaded.format).toBe("json"); - expect(loaded.config.project_id).toBe("abc123"); - expect(loaded.config.db.major_version).toBe(16); - expect(loaded.config.experimental.stack).toBe(true); - expect(loaded.config.api.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const saved = yield* saveCliConfig({ cwd, config: sampleConfig }); + expect(saved.format).toBe("json"); + expect(saved.path).toBe(yield* configJsonPath(cwd)); + }), + )); - test("populates rawDocument for a .json config", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configJsonPath(cwd)); + test("loads strict JSON", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + yield* encodeJson({ + project_id: "abc123", + db: { + major_version: 16, + }, + experimental: { stack: true }, + }), + ); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - JSON.stringify({ - project_id: "abc123", - db: { - major_version: 16, - }, - }), - ); + const loaded = yield* loadCliConfigFile(path); + expect(loaded.format).toBe("json"); + expect(loaded.config.project_id).toBe("abc123"); + expect(loaded.config.db.major_version).toBe(16); + expect(loaded.config.experimental.stack).toBe(true); + expect(loaded.config.api.enabled).toBe(true); + }), + )); - const loaded = await runConfigEffect(loadCliConfigFile(path)); - expect(loaded.rawDocument).toEqual({ project_id: "abc123", db: { major_version: 16 } }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("populates rawDocument for a .json config", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + yield* encodeJson({ + project_id: "abc123", + db: { + major_version: 16, + }, + }), + ); - test("populates rawText with the exact on-disk bytes for a .json config", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configJsonPath(cwd)); + const loaded = yield* loadCliConfigFile(path); + expect(loaded.rawDocument).toEqual({ project_id: "abc123", db: { major_version: 16 } }); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - const text = JSON.stringify({ project_id: "abc123", db: { major_version: 16 } }); - await writeFile(path, text); + test("populates rawText with the exact on-disk bytes for a .json config", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configJsonPath(cwd); - const loaded = await runConfigEffect(loadCliConfigFile(path)); - expect(loaded.rawText).toBe(text); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + const text = yield* encodeJson({ project_id: "abc123", db: { major_version: 16 } }); + yield* fs.writeFileString(path, text); - test("populates rawText with the exact on-disk bytes for a .toml config", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); + const loaded = yield* loadCliConfigFile(path); + expect(loaded.rawText).toBe(text); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - const text = 'project_id = "abc123"\n\n[db]\nmajor_version = 16\n'; - await writeFile(path, text); + test("populates rawText with the exact on-disk bytes for a .toml config", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); - const loaded = await runConfigEffect(loadCliConfigFile(path)); - expect(loaded.rawText).toBe(text); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + const text = 'project_id = "abc123"\n\n[db]\nmajor_version = 16\n'; + yield* fs.writeFileString(path, text); - test("loads top-level $schema metadata from JSON", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configJsonPath(cwd)); + const loaded = yield* loadCliConfigFile(path); + expect(loaded.rawText).toBe(text); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - JSON.stringify({ - $schema: CLI_CONFIG_SCHEMA_URL, - }), - ); - - const loaded = await runConfigEffect(loadCliConfigFile(path)); - expect(loaded.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); - expect(loaded.config.db.major_version).toBe(17); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("loads top-level $schema metadata from JSON", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + yield* encodeJson({ + $schema: CLI_CONFIG_SCHEMA_URL, + }), + ); - test("rejects JSON comments and trailing commas", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configJsonPath(cwd)); + const loaded = yield* loadCliConfigFile(path); + expect(loaded.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); + expect(loaded.config.db.major_version).toBe(17); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - `{ + test("rejects JSON comments and trailing commas", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + `{ // project ref "project_id": "abc123", "db": { @@ -204,17 +226,13 @@ describe("config io", () => { } } `, - ); + ); - const exit = await Effect.runPromiseExit( - loadCliConfigFile(path).pipe(Effect.provide(BunServices.layer)), - ); + const exit = yield* Effect.exit(loadCliConfigFile(path)); - expect(Exit.isFailure(exit)).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + expect(Exit.isFailure(exit)).toBe(true); + }), + )); test("decodes legacy runtime defaults from an empty config", () => { const config = decodeCliConfig({}); @@ -351,96 +369,91 @@ describe("config io", () => { expect(encodeCliConfigToToml(config).trim()).toBe(""); }); - test("preserves hosted $schema when saving JSON", async () => { - const cwd = makeTempProject(); + test("preserves hosted $schema when saving JSON", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* makeTempProject(); - try { - const saved = await runConfigEffect( - saveCliConfig({ + const saved = yield* saveCliConfig({ cwd, config: decodeCliConfig({}), schemaRef: CLI_CONFIG_SCHEMA_URL, - }), - ); + }); - expect(saved.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); + expect(saved.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); - const content = await readFile(saved.path, "utf8"); - expect(content).toContain(`"$schema": "${CLI_CONFIG_SCHEMA_URL}"`); + const content = yield* fs.readFileString(saved.path); + expect(content).toContain(`"$schema": "${CLI_CONFIG_SCHEMA_URL}"`); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded?.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded?.schemaRef).toBe(CLI_CONFIG_SCHEMA_URL); + }), + )); - test("preserves local $schema when saving JSON over an existing config", async () => { - const cwd = makeTempProject(); - const schemaRef = "./node_modules/@supabase/config/schema.json"; + test("preserves local $schema when saving JSON over an existing config", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* makeTempProject(); + const schemaRef = "./node_modules/@supabase/config/schema.json"; - try { - await runConfigEffect( - saveCliConfig({ + yield* saveCliConfig({ cwd, config: decodeCliConfig({}), schemaRef, - }), - ); + }); - const saved = await runConfigEffect( - saveCliConfig({ + const saved = yield* saveCliConfig({ cwd, config: sampleConfig, - }), - ); + }); - expect(saved.schemaRef).toBe(schemaRef); + expect(saved.schemaRef).toBe(schemaRef); - const content = await readFile(saved.path, "utf8"); - expect(content).toContain(`"$schema": "${schemaRef}"`); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const content = yield* fs.readFileString(saved.path); + expect(content).toContain(`"$schema": "${schemaRef}"`); + }), + )); - test("preserves $schema when saving TOML", async () => { - const cwd = makeTempProject(); - const schemaRef = "./node_modules/@supabase/config/schema.json"; + test("preserves $schema when saving TOML", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* makeTempProject(); + const schemaRef = "./node_modules/@supabase/config/schema.json"; - try { - const saved = await runConfigEffect( - saveCliConfig({ + const saved = yield* saveCliConfig({ cwd, config: decodeCliConfig({}), format: "toml", schemaRef, - }), - ); + }); - expect(saved.schemaRef).toBe(schemaRef); + expect(saved.schemaRef).toBe(schemaRef); - const content = await readFile(saved.path, "utf8"); - expect(content).toContain(`"$schema" = "${schemaRef}"`); + const content = yield* fs.readFileString(saved.path); + expect(content).toContain(`"$schema" = "${schemaRef}"`); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded?.schemaRef).toBe(schemaRef); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded?.schemaRef).toBe(schemaRef); + }), + )); - test("prefers JSON over TOML when both exist", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); + test("prefers JSON over TOML when both exist", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); + const tomlPath = yield* configTomlPath(cwd); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, encodeCliConfigToJson(sampleConfig)); - await writeFile( - tomlPath, - `project_id = "toml-ref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, encodeCliConfigToJson(sampleConfig)); + yield* fs.writeFileString( + tomlPath, + `project_id = "toml-ref" [db] major_version = 16 @@ -448,30 +461,31 @@ major_version = 16 [experimental] stack = true `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded?.format).toBe("json"); - expect(loaded?.config.project_id).toBe("ref_123"); - expect(loaded?.ignoredPaths).toEqual([tomlPath]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded?.format).toBe("json"); + expect(loaded?.config.project_id).toBe("ref_123"); + expect(loaded?.ignoredPaths).toEqual([tomlPath]); + }), + )); // Go-parity callers (legacy `status`/`stop`) pass `tomlOnly: true` so a stray `config.json` // never wins over `config.toml`. - test("loads TOML instead of JSON when tomlOnly is set, even if JSON exists", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, encodeCliConfigToJson(sampleConfig)); - await writeFile( - tomlPath, - `project_id = "toml-ref" + test("loads TOML instead of JSON when tomlOnly is set, even if JSON exists", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); + const tomlPath = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, encodeCliConfigToJson(sampleConfig)); + yield* fs.writeFileString( + tomlPath, + `project_id = "toml-ref" [db] major_version = 16 @@ -479,40 +493,42 @@ major_version = 16 [experimental] stack = true `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd, { tomlOnly: true })); - expect(loaded?.format).toBe("toml"); - expect(loaded?.config.project_id).toBe("toml-ref"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("returns null when tomlOnly is set and only JSON exists", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); + const loaded = yield* loadCliConfig(cwd, { tomlOnly: true }); + expect(loaded?.format).toBe("toml"); + expect(loaded?.config.project_id).toBe("toml-ref"); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, encodeCliConfigToJson(sampleConfig)); + test("returns null when tomlOnly is set and only JSON exists", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); - const loaded = await runConfigEffect(loadCliConfig(cwd, { tomlOnly: true })); - expect(loaded).toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, encodeCliConfigToJson(sampleConfig)); - test("loads TOML when JSON is absent", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); + const loaded = yield* loadCliConfig(cwd, { tomlOnly: true }); + expect(loaded).toBeNull(); + }), + )); + + test("loads TOML when JSON is absent", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - tomlPath, - `project_id = "toml-ref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + tomlPath, + `project_id = "toml-ref" [db] major_version = 16 @@ -520,287 +536,289 @@ major_version = 16 [experimental] stack = true `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded?.format).toBe("toml"); - expect(loaded?.config.project_id).toBe("toml-ref"); - expect(loaded?.config.db.major_version).toBe(16); - expect(loaded?.config.experimental.stack).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("loads the CLI fixture", async () => { - const loaded = await runConfigEffect(loadCliConfigFile(legacyFixturePath)); - const production = loaded.config.remotes.production; - const staging = loaded.config.remotes.staging; - - expect(loaded.format).toBe("toml"); - expect(loaded.config.project_id).toBe("test"); - expect(loaded.config.auth.hook.send_sms.secrets).toBe("env(AUTH_SEND_SMS_SECRETS)"); - expect(loaded.config.edge_runtime.secrets?.test_key).toBe("test_value"); - expect(loaded.config.storage.analytics.buckets).toEqual({ "my-warehouse": {} }); - expect(production).toBeDefined(); - expect(staging).toBeDefined(); - if (!production || !staging) { - throw new Error("Expected legacy remotes to be loaded."); - } - expect(production.project_id).toBe("vpefcjyosynxeiebfscx"); - expect(production.auth.site_url).toBe("http://feature-auth-branch.com/"); - expect(staging.storage?.buckets?.images?.allowed_mime_types).toEqual(["image/png"]); - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded?.format).toBe("toml"); + expect(loaded?.config.project_id).toBe("toml-ref"); + expect(loaded?.config.db.major_version).toBe(16); + expect(loaded?.config.experimental.stack).toBe(true); + }), + )); - test("returns null when no config file exists", async () => { - const cwd = makeTempProject(); + test("loads the CLI fixture", () => + runConfigEffect( + Effect.gen(function* () { + const loaded = yield* loadCliConfigFile(yield* legacyFixturePath); + const production = loaded.config.remotes.production; + const staging = loaded.config.remotes.staging; + + expect(loaded.format).toBe("toml"); + expect(loaded.config.project_id).toBe("test"); + expect(loaded.config.auth.hook.send_sms.secrets).toBe("env(AUTH_SEND_SMS_SECRETS)"); + expect(loaded.config.edge_runtime.secrets?.test_key).toBe("test_value"); + expect(loaded.config.storage.analytics.buckets).toEqual({ "my-warehouse": {} }); + expect(production).toBeDefined(); + expect(staging).toBeDefined(); + if (!production || !staging) { + throw new Error("Expected legacy remotes to be loaded."); + } + expect(production.project_id).toBe("vpefcjyosynxeiebfscx"); + expect(production.auth.site_url).toBe("http://feature-auth-branch.com/"); + expect(staging.storage?.buckets?.images?.allowed_mime_types).toEqual(["image/png"]); + }), + )); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded).toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("returns null when no config file exists", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded).toBeNull(); + }), + )); - test("does not ignore an invalid JSON config when TOML also exists", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); + test("does not ignore an invalid JSON config when TOML also exists", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); + const tomlPath = yield* configTomlPath(cwd); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, `{"project_id": 123}`); - await writeFile( - tomlPath, - `project_id = "toml-ref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, `{"project_id": 123}`); + yield* fs.writeFileString( + tomlPath, + `project_id = "toml-ref" [db] major_version = 16 `, - ); - - await expect(runConfigEffect(loadCliConfig(cwd))).rejects.toThrow(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("returns a typed parse error for invalid JSON", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, `{"project_id": 123}`); + ); - const exit = await Effect.runPromiseExit( - loadCliConfigFile(jsonPath).pipe(Effect.provide(BunServices.layer)), - ); + expect(Exit.isFailure(yield* Effect.exit(loadCliConfig(cwd)))).toBe(true); + }), + )); - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(error.value._tag).toBe("CliConfigParseError"); - if (error.value._tag === "CliConfigParseError") { - expect(error.value.path).toBe(jsonPath); - expect(error.value.format).toBe("json"); + test("returns a typed parse error for invalid JSON", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, `{"project_id": 123}`); + + const exit = yield* Effect.exit(loadCliConfigFile(jsonPath)); + + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(error.value._tag).toBe("CliConfigParseError"); + if (error.value._tag === "CliConfigParseError") { + expect(error.value.path).toBe(jsonPath); + expect(error.value.format).toBe("json"); + } } } - } - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }), + )); - test("redacts edge_runtime.secrets on the CliConfigParseError document", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - // `analytics.port` fails schema decode, which fails the whole `Schema.decodeUnknownSync` - // call while `edge_runtime.secrets` parses fine on its own. `MY_SUPER_SECRET_VALUE` stands - // in for a real secret so the assertion below can confirm it never appears in plaintext. - await writeFile( - tomlPath, - `[analytics] + test("redacts edge_runtime.secrets on the CliConfigParseError document", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + // `analytics.port` fails schema decode, which fails the whole `Schema.decodeUnknownSync` + // call while `edge_runtime.secrets` parses fine on its own. `MY_SUPER_SECRET_VALUE` stands + // in for a real secret so the assertion below can confirm it never appears in plaintext. + yield* fs.writeFileString( + tomlPath, + `[analytics] port = "not-a-number" [edge_runtime.secrets] FOO = "MY_SUPER_SECRET_VALUE" `, - ); - - const exit = await Effect.runPromiseExit( - loadCliConfigFile(tomlPath).pipe(Effect.provide(BunServices.layer)), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { - return; - } - - const edgeRuntime = error.value.document?.edge_runtime; - const secrets = - edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined - ? (edgeRuntime as Record<string, unknown>).secrets - : undefined; - expect(secrets).toBeDefined(); - const foo = (secrets as Record<string, unknown>).FOO; - expect(Redacted.isRedacted(foo)).toBe(true); - expect(Redacted.value(foo as Redacted.Redacted<string>)).toBe("MY_SUPER_SECRET_VALUE"); - // The whole point: a caller that doesn't know to unwrap `Redacted` never sees the raw - // secret, even via JSON.stringify. - expect(JSON.stringify(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const exit = yield* Effect.exit(loadCliConfigFile(tomlPath)); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { + return; + } - test("redacts a non-string edge_runtime.secrets value on the CliConfigParseError document", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - // `FOO` is a TOML array, not a string, so its own schema decode fails, but the raw - // pre-decode value still carries `MY_SUPER_SECRET_VALUE` in plaintext — - // `redactEdgeRuntimeSecrets` must wrap it regardless of shape. - await writeFile( - tomlPath, - `[analytics] + const edgeRuntime = error.value.document?.edge_runtime; + const secrets = + edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined + ? (edgeRuntime as Record<string, unknown>).secrets + : undefined; + expect(secrets).toBeDefined(); + const foo = (secrets as Record<string, unknown>).FOO; + expect(Redacted.isRedacted(foo)).toBe(true); + expect(Redacted.value(foo as Redacted.Redacted<string>)).toBe("MY_SUPER_SECRET_VALUE"); + // The whole point: a caller that doesn't know to unwrap `Redacted` never sees the raw + // secret, even via JSON.stringify. + expect(yield* encodeJson(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); + }), + )); + + test("redacts a non-string edge_runtime.secrets value on the CliConfigParseError document", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + // `FOO` is a TOML array, not a string, so its own schema decode fails, but the raw + // pre-decode value still carries `MY_SUPER_SECRET_VALUE` in plaintext — + // `redactEdgeRuntimeSecrets` must wrap it regardless of shape. + yield* fs.writeFileString( + tomlPath, + `[analytics] port = "not-a-number" [edge_runtime.secrets] FOO = ["MY_SUPER_SECRET_VALUE"] `, - ); - - const exit = await Effect.runPromiseExit( - loadCliConfigFile(tomlPath).pipe(Effect.provide(BunServices.layer)), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { - return; - } - - const edgeRuntime = error.value.document?.edge_runtime; - const secrets = - edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined - ? (edgeRuntime as Record<string, unknown>).secrets - : undefined; - expect(secrets).toBeDefined(); - const foo = (secrets as Record<string, unknown>).FOO; - expect(Redacted.isRedacted(foo)).toBe(true); - expect(Redacted.value(foo as Redacted.Redacted<unknown>)).toEqual(["MY_SUPER_SECRET_VALUE"]); - expect(JSON.stringify(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const exit = yield* Effect.exit(loadCliConfigFile(tomlPath)); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { + return; + } - test("redacts a non-object edge_runtime.secrets field on the CliConfigParseError document", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - // `secrets` itself is a TOML array here, not a table, so the whole field is malformed - // rather than a single bad entry — `redactEdgeRuntimeSecrets` must wrap the field as one - // unit instead of leaving it raw. - await writeFile( - tomlPath, - `[analytics] + const edgeRuntime = error.value.document?.edge_runtime; + const secrets = + edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined + ? (edgeRuntime as Record<string, unknown>).secrets + : undefined; + expect(secrets).toBeDefined(); + const foo = (secrets as Record<string, unknown>).FOO; + expect(Redacted.isRedacted(foo)).toBe(true); + expect(Redacted.value(foo as Redacted.Redacted<unknown>)).toEqual([ + "MY_SUPER_SECRET_VALUE", + ]); + expect(yield* encodeJson(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); + }), + )); + + test("redacts a non-object edge_runtime.secrets field on the CliConfigParseError document", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + // `secrets` itself is a TOML array here, not a table, so the whole field is malformed + // rather than a single bad entry — `redactEdgeRuntimeSecrets` must wrap the field as one + // unit instead of leaving it raw. + yield* fs.writeFileString( + tomlPath, + `[analytics] port = "not-a-number" [edge_runtime] secrets = ["MY_SUPER_SECRET_VALUE"] `, - ); - - const exit = await Effect.runPromiseExit( - loadCliConfigFile(tomlPath).pipe(Effect.provide(BunServices.layer)), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { - return; - } - - const edgeRuntime = error.value.document?.edge_runtime; - const secrets = - edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined - ? (edgeRuntime as Record<string, unknown>).secrets - : undefined; - expect(Redacted.isRedacted(secrets)).toBe(true); - expect(Redacted.value(secrets as Redacted.Redacted<unknown>)).toEqual([ - "MY_SUPER_SECRET_VALUE", - ]); - expect(JSON.stringify(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const exit = yield* Effect.exit(loadCliConfigFile(tomlPath)); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { + return; + } - test("preserves TOML as the active format on save", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); + const edgeRuntime = error.value.document?.edge_runtime; + const secrets = + edgeRuntime !== null && typeof edgeRuntime === "object" && edgeRuntime !== undefined + ? (edgeRuntime as Record<string, unknown>).secrets + : undefined; + expect(Redacted.isRedacted(secrets)).toBe(true); + expect(Redacted.value(secrets as Redacted.Redacted<unknown>)).toEqual([ + "MY_SUPER_SECRET_VALUE", + ]); + expect(yield* encodeJson(error.value.document)).not.toContain("MY_SUPER_SECRET_VALUE"); + }), + )); + + test("preserves TOML as the active format on save", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); + const jsonPath = yield* configJsonPath(cwd); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - tomlPath, - `project_id = "old-ref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + tomlPath, + `project_id = "old-ref" [db] major_version = 16 `, - ); - - const saved = await runConfigEffect(saveCliConfig({ cwd, config: sampleConfig })); - - expect(saved.format).toBe("toml"); - expect(saved.path).toBe(tomlPath); - expect(await Bun.file(jsonPath).exists()).toBe(false); - const content = await readFile(tomlPath, "utf8"); - expect(content).toContain('project_id = "ref_123"'); - expect(content).toContain("[db.pooler]"); - expect(content).not.toContain("major_version"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const saved = yield* saveCliConfig({ cwd, config: sampleConfig }); + + expect(saved.format).toBe("toml"); + expect(saved.path).toBe(tomlPath); + expect(yield* fs.exists(jsonPath)).toBe(false); + const content = yield* fs.readFileString(tomlPath); + expect(content).toContain('project_id = "ref_123"'); + expect(content).toContain("[db.pooler]"); + expect(content).not.toContain("major_version"); + }), + )); - test("preserves JSON as the active format on save", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); + test("preserves JSON as the active format on save", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, encodeCliConfigToJson(sampleConfig)); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, encodeCliConfigToJson(sampleConfig)); - const saved = await runConfigEffect( - saveCliConfig({ + const saved = yield* saveCliConfig({ cwd, config: decodeCliConfig({ project_id: "updated-ref", @@ -808,121 +826,112 @@ major_version = 16 enable_signup: false, }, }), - }), - ); - - expect(saved.format).toBe("json"); - expect(saved.path).toBe(jsonPath); - const content = await readFile(jsonPath, "utf8"); - expect(content).toContain('"project_id": "updated-ref"'); - expect(content).toContain('"enable_signup": false'); - expect(content).not.toContain('"jwt_expiry"'); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }); + + expect(saved.format).toBe("json"); + expect(saved.path).toBe(jsonPath); + const content = yield* fs.readFileString(jsonPath); + expect(content).toContain('"project_id": "updated-ref"'); + expect(content).toContain('"enable_signup": false'); + expect(content).not.toContain('"jwt_expiry"'); + }), + )); - test("supports explicit format override", async () => { - const cwd = makeTempProject(); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(jsonPath, encodeCliConfigToJson(sampleConfig)); - - const saved = await runConfigEffect( - saveCliConfig({ cwd, config: sampleConfig, format: "toml" }), - ); - - expect(saved.format).toBe("toml"); - expect(saved.path).toBe(tomlPath); - expect(await Bun.file(jsonPath).exists()).toBe(false); - const content = await readFile(tomlPath, "utf8"); - expect(content).toContain("[db.pooler]"); - expect(content).not.toContain("[versions]"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("supports explicit format override", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const tomlPath = yield* configTomlPath(cwd); + const jsonPath = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(jsonPath, encodeCliConfigToJson(sampleConfig)); + + const saved = yield* saveCliConfig({ cwd, config: sampleConfig, format: "toml" }); + + expect(saved.format).toBe("toml"); + expect(saved.path).toBe(tomlPath); + expect(yield* fs.exists(jsonPath)).toBe(false); + const content = yield* fs.readFileString(tomlPath); + expect(content).toContain("[db.pooler]"); + expect(content).not.toContain("[versions]"); + }), + )); - test("removes TOML when explicitly switching to JSON", async () => { - const cwd = makeTempProject(); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(tomlPath, encodeCliConfigToToml(sampleConfig)); - - const saved = await runConfigEffect( - saveCliConfig({ cwd, config: sampleConfig, format: "json" }), - ); - - expect(saved.format).toBe("json"); - expect(saved.path).toBe(jsonPath); - expect(await Bun.file(tomlPath).exists()).toBe(false); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("removes TOML when explicitly switching to JSON", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const jsonPath = yield* configJsonPath(cwd); + const tomlPath = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(tomlPath, encodeCliConfigToToml(sampleConfig)); - test("preserves the discovered project format when saving from a nested cwd", async () => { - const cwd = makeTempProject(); - const nestedCwd = join(cwd, "apps", "web", "src"); - const tomlPath = await runConfigEffect(configTomlPath(cwd)); - const jsonPath = await runConfigEffect(configJsonPath(cwd)); + const saved = yield* saveCliConfig({ cwd, config: sampleConfig, format: "json" }); + + expect(saved.format).toBe("json"); + expect(saved.path).toBe(jsonPath); + expect(yield* fs.exists(tomlPath)).toBe(false); + }), + )); - try { - await mkdir(nestedCwd, { recursive: true }); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - tomlPath, - `project_id = "nested-ref" + test("preserves the discovered project format when saving from a nested cwd", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const nestedCwd = join(cwd, "apps", "web", "src"); + const tomlPath = yield* configTomlPath(cwd); + const jsonPath = yield* configJsonPath(cwd); + + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + tomlPath, + `project_id = "nested-ref" [db] major_version = 16 `, - ); + ); - const saved = await runConfigEffect( - saveCliConfig({ + const saved = yield* saveCliConfig({ cwd: nestedCwd, config: decodeCliConfig({ project_id: "nested-updated", }), - }), - ); - - expect(saved.format).toBe("toml"); - expect(saved.path).toBe(tomlPath); - expect(await Bun.file(jsonPath).exists()).toBe(false); - const content = await readFile(tomlPath, "utf8"); - expect(content).toContain('project_id = "nested-updated"'); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }); - test("exposes a CliConfigStore service for the CLI", async () => { - const cwd = makeTempProject(); - const layer = cliConfigStoreLayer.pipe(Layer.provide(BunServices.layer)); + expect(saved.format).toBe("toml"); + expect(saved.path).toBe(tomlPath); + expect(yield* fs.exists(jsonPath)).toBe(false); + const content = yield* fs.readFileString(tomlPath); + expect(content).toContain('project_id = "nested-updated"'); + }), + )); - try { - const loaded = await Effect.runPromise( - Effect.gen(function* () { + test("exposes a CliConfigStore service for the CLI", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + const layer = cliConfigStoreLayer.pipe(Layer.provide(BunServices.layer)); + + const loaded = yield* Effect.gen(function* () { const store = yield* CliConfigStore; yield* store.save({ cwd, config: sampleConfig }); return yield* store.load(cwd); - }).pipe(Effect.provide(layer)), - ); - - expect(loaded?.config.project_id).toBe("ref_123"); - expect(loaded?.config.db.pooler.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }).pipe(Effect.provide(layer)); + + expect(loaded?.config.project_id).toBe("ref_123"); + expect(loaded?.config.db.pooler.enabled).toBe(true); + }), + )); test("encodes sparse TOML for fresh output", () => { const content = encodeCliConfigToToml(sampleConfig); @@ -932,70 +941,62 @@ major_version = 16 expect(content).not.toContain("[versions]"); }); - test("supports the Bun edge entrypoint", async () => { - const cwd = makeTempProject(); - - try { - await saveCliConfig({ cwd, config: sampleConfig }).pipe( - Effect.provide(BunServices.layer), - Effect.runPromise, - ); - const loaded = await loadCliConfigFromBun(cwd); - expect(loaded?.config.project_id).toBe("ref_123"); - expect(loaded?.config.db.pooler.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("supports the Bun edge entrypoint", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); - test("supports the Node edge entrypoint", async () => { - const cwd = makeTempProject(); - - try { - await saveCliConfig({ cwd, config: sampleConfig }).pipe( - Effect.provide(BunServices.layer), - Effect.runPromise, - ); - const loaded = await loadCliConfigFromNode(cwd); - expect(loaded?.config.project_id).toBe("ref_123"); - expect(loaded?.config.db.pooler.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + yield* saveCliConfig({ cwd, config: sampleConfig }); + const loaded = yield* Effect.promise(() => loadCliConfigFromBun(cwd)); + expect(loaded?.config.project_id).toBe("ref_123"); + expect(loaded?.config.db.pooler.enabled).toBe(true); + }), + )); - test("round-trip: save → load → save produces identical config and file content", async () => { - const cwd = makeTempProject(); + test("supports the Node edge entrypoint", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); - try { - const original = decodeCliConfig({ - project_id: "roundtrip-ref", - db: { - major_version: 16, - pooler: { enabled: true }, - }, - auth: { - enable_signup: false, - site_url: "https://example.com", - }, - analytics: { enabled: false }, - }); + yield* saveCliConfig({ cwd, config: sampleConfig }); + const loaded = yield* Effect.promise(() => loadCliConfigFromNode(cwd)); + expect(loaded?.config.project_id).toBe("ref_123"); + expect(loaded?.config.db.pooler.enabled).toBe(true); + }), + )); + + test("round-trip: save → load → save produces identical config and file content", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* makeTempProject(); - const saved1 = await runConfigEffect(saveCliConfig({ cwd, config: original })); - const content1 = await readFile(saved1.path, "utf8"); + const original = decodeCliConfig({ + project_id: "roundtrip-ref", + db: { + major_version: 16, + pooler: { enabled: true }, + }, + auth: { + enable_signup: false, + site_url: "https://example.com", + }, + analytics: { enabled: false }, + }); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded).not.toBeNull(); - expect(loaded!.config).toEqual(original); + const saved1 = yield* saveCliConfig({ cwd, config: original }); + const content1 = yield* fs.readFileString(saved1.path); - const saved2 = await runConfigEffect(saveCliConfig({ cwd, config: loaded!.config })); - const content2 = await readFile(saved2.path, "utf8"); + const loaded = yield* loadCliConfig(cwd); + expect(loaded).not.toBeNull(); + expect(loaded!.config).toEqual(original); - expect(content2).toBe(content1); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const saved2 = yield* saveCliConfig({ cwd, config: loaded!.config }); + const content2 = yield* fs.readFileString(saved2.path); + + expect(content2).toBe(content1); + }), + )); test("includes current keys in generated JSON schema", () => { const schema = toCliConfigJsonSchema(); @@ -1011,14 +1012,17 @@ major_version = 16 expect(schemaString).not.toContain("versions"); }); - test("resolves env() on numeric port fields (CLI-1489)", async () => { - const cwd = makeTempProject(); + test("resolves env() on numeric port fields (CLI-1489)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [api] port = "env(SUPABASE_API_PORT)" @@ -1029,44 +1033,46 @@ port = "env(SUPABASE_DB_PORT)" [analytics] port = "env(SUPABASE_ANALYTICS_PORT)" `, - ); - await writeFile( - join(cwd, "supabase", ".env"), - "SUPABASE_API_PORT=54321\nSUPABASE_DB_PORT=54322\nSUPABASE_ANALYTICS_PORT=54327\n", - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - - expect(loaded).not.toBeNull(); - expect(loaded!.config.api.port).toBe(54321); - expect(loaded!.config.db.port).toBe(54322); - expect(loaded!.config.analytics.port).toBe(54327); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + "SUPABASE_API_PORT=54321\nSUPABASE_DB_PORT=54322\nSUPABASE_ANALYTICS_PORT=54327\n", + ); + + const loaded = yield* loadCliConfig(cwd); + + expect(loaded).not.toBeNull(); + expect(loaded!.config.api.port).toBe(54321); + expect(loaded!.config.db.port).toBe(54322); + expect(loaded!.config.analytics.port).toBe(54327); + }), + )); - test("resolves env() on boolean fields", async () => { - const cwd = makeTempProject(); + test("resolves env() on boolean fields", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [analytics] enabled = "env(SUPABASE_ANALYTICS_ENABLED)" `, - ); - await writeFile(join(cwd, "supabase", ".env"), "SUPABASE_ANALYTICS_ENABLED=false\n"); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.analytics.enabled).toBe(false); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + "SUPABASE_ANALYTICS_ENABLED=false\n", + ); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.analytics.enabled).toBe(false); + }), + )); test.each([ ["1", true], @@ -1078,228 +1084,238 @@ enabled = "env(SUPABASE_ANALYTICS_ENABLED)" ["FALSE", false], ] as const)( "resolves env() on boolean fields using Go's strconv.ParseBool acceptance set (%s -> %s)", - async (envValue, expected) => { - const cwd = makeTempProject(); + (envValue, expected) => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [analytics] enabled = "env(SUPABASE_ANALYTICS_ENABLED)" `, - ); - await writeFile(join(cwd, "supabase", ".env"), `SUPABASE_ANALYTICS_ENABLED=${envValue}\n`); + ); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + `SUPABASE_ANALYTICS_ENABLED=${envValue}\n`, + ); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.analytics.enabled).toBe(expected); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }, + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.analytics.enabled).toBe(expected); + }), + ), ); - test("splits a comma-separated string literal into a slice (Go's StringToSliceHookFunc)", async () => { - // A plain string value for a `[]string` field like `additional_redirect_urls` decodes fine - // when `goViperCompat` is set, not just via `env(...)`. - const cwd = makeTempProject(); + test("splits a comma-separated string literal into a slice (Go's StringToSliceHookFunc)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + // A plain string value for a `[]string` field like `additional_redirect_urls` decodes fine + // when `goViperCompat` is set, not just via `env(...)`. + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] additional_redirect_urls = "http://a,http://b" `, - ); + ); - const loaded = await runConfigEffect(loadCliConfig(cwd, { goViperCompat: true })); - expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { goViperCompat: true }); + expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); + }), + )); - test("splits an env()-substituted comma-separated string into a slice", async () => { - const cwd = makeTempProject(); + test("splits an env()-substituted comma-separated string into a slice", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] additional_redirect_urls = "env(SUPABASE_REDIRECT_URLS)" `, - ); - await writeFile(join(cwd, "supabase", ".env"), "SUPABASE_REDIRECT_URLS=http://a,http://b\n"); - - const loaded = await runConfigEffect(loadCliConfig(cwd, { goViperCompat: true })); - expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + "SUPABASE_REDIRECT_URLS=http://a,http://b\n", + ); + + const loaded = yield* loadCliConfig(cwd, { goViperCompat: true }); + expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); + }), + )); - test("an empty string literal for a slice field decodes to an empty array", async () => { - const cwd = makeTempProject(); + test("an empty string literal for a slice field decodes to an empty array", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] additional_redirect_urls = "" `, - ); + ); - const loaded = await runConfigEffect(loadCliConfig(cwd, { goViperCompat: true })); - expect(loaded!.config.auth.additional_redirect_urls).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { goViperCompat: true }); + expect(loaded!.config.auth.additional_redirect_urls).toEqual([]); + }), + )); - test("an actual array value for a slice field is left untouched", async () => { - const cwd = makeTempProject(); + test("an actual array value for a slice field is left untouched", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] additional_redirect_urls = ["http://a", "http://b"] `, - ); + ); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.auth.additional_redirect_urls).toEqual(["http://a", "http://b"]); + }), + )); - test("preserves env() literals on string fields when the var is unset (Go parity)", async () => { - const cwd = makeTempProject(); + test("preserves env() literals on string fields when the var is unset (Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(MISSING_SECRET)" `, - ); + ); - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.auth.jwt_secret).toBe("env(MISSING_SECRET)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.auth.jwt_secret).toBe("env(MISSING_SECRET)"); + }), + )); - test("preserves env() literals on string fields when the var is set but empty (Go parity)", async () => { - const cwd = makeTempProject(); + test("preserves env() literals on string fields when the var is set but empty (Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(MISSING_SECRET)" `, - ); - await writeFile(join(cwd, "supabase", ".env"), "MISSING_SECRET=\n"); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.auth.jwt_secret).toBe("env(MISSING_SECRET)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + yield* fs.writeFileString(join(cwd, "supabase", ".env"), "MISSING_SECRET=\n"); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.auth.jwt_secret).toBe("env(MISSING_SECRET)"); + }), + )); - test("fails to decode a numeric field when env var is unset", async () => { - const cwd = makeTempProject(); + test("fails to decode a numeric field when env var is unset", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [analytics] port = "env(MISSING_PORT)" `, - ); - - const exit = await Effect.runPromiseExit( - loadCliConfig(cwd).pipe(Effect.provide(BunServices.layer)), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const failure = Cause.findErrorOption(exit.cause); - expect(Option.isSome(failure)).toBe(true); - if (Option.isSome(failure)) { - expect((failure.value as { _tag: string })._tag).toBe("CliConfigParseError"); + ); + + const exit = yield* Effect.exit(loadCliConfig(cwd)); + + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const failure = Cause.findErrorOption(exit.cause); + expect(Option.isSome(failure)).toBe(true); + if (Option.isSome(failure)) { + expect((failure.value as { _tag: string })._tag).toBe("CliConfigParseError"); + } } - } - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }), + )); - test("falls back to ambient process.env when .env is missing", async () => { - const cwd = makeTempProject(); - const previous = process.env.SUPABASE_DB_PORT_TEST; - process.env.SUPABASE_DB_PORT_TEST = "55555"; + test("falls back to ambient process.env when .env is missing", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + yield* stubEnv("SUPABASE_DB_PORT_TEST", "55555"); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [db] port = "env(SUPABASE_DB_PORT_TEST)" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.db.port).toBe(55555); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_DB_PORT_TEST; - } else { - process.env.SUPABASE_DB_PORT_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.db.port).toBe(55555); + }), + )); // Pins pre-Go-parity default behavior, so `packages/stack` and the functions manifest (which // don't pass `goViperCompat`) don't inherit the Go-parity CLI's stricter/wider semantics. - test("loads successfully with a duplicate [remotes.*] project_id when goViperCompat is omitted", async () => { - const cwd = makeTempProject(); + test("loads successfully with a duplicate [remotes.*] project_id when goViperCompat is omitted", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "baseref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "baseref" [remotes.a] project_id = "dupref" @@ -1307,24 +1323,25 @@ project_id = "dupref" [remotes.b] project_id = "dupref" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded).not.toBeNull(); - expect(loaded!.config.project_id).toBe("baseref"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("loads a [remotes.*.compute] section alongside the project's own", async () => { - const cwd = makeTempProject(); + const loaded = yield* loadCliConfig(cwd); + expect(loaded).not.toBeNull(); + expect(loaded!.config.project_id).toBe("baseref"); + }), + )); + + test("loads a [remotes.*.compute] section alongside the project's own", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "baseref" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "baseref" [compute.api] runtime = "node" @@ -1335,166 +1352,159 @@ project_id = "abcdefghijklmnopqrst" [remotes.staging.compute.api] runtime = "deno" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded).not.toBeNull(); - expect(loaded!.config.compute).toEqual({ api: { runtime: "node" } }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("loads successfully with an invalid [remotes.*] project_id format when goViperCompat is omitted", async () => { - const cwd = makeTempProject(); + const loaded = yield* loadCliConfig(cwd); + expect(loaded).not.toBeNull(); + expect(loaded!.config.compute).toEqual({ api: { runtime: "node" } }); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "baseref" + test("loads successfully with an invalid [remotes.*] project_id format when goViperCompat is omitted", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "baseref" [remotes.bad] project_id = "not-a-ref" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded).not.toBeNull(); - expect(loaded!.config.project_id).toBe("baseref"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded).not.toBeNull(); + expect(loaded!.config.project_id).toBe("baseref"); + }), + )); - test("does not split a comma-separated string literal for an array field when goViperCompat is omitted", async () => { - const cwd = makeTempProject(); + test("does not split a comma-separated string literal for an array field when goViperCompat is omitted", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "ref_123" + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "ref_123" [auth] additional_redirect_urls = "http://a,http://b" `, - ); + ); - const exit = await Effect.runPromiseExit( - loadCliConfig(cwd).pipe(Effect.provide(BunServices.layer)), - ); + const exit = yield* Effect.exit(loadCliConfig(cwd)); - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); + } } - } - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("does not warn on a deprecated provider (but still strips it) when goViperCompat is omitted", async () => { - const cwd = makeTempProject(); - const warnings: Array<string> = []; - const errorSpy = vi.spyOn(console, "error").mockImplementation((...args) => { - warnings.push(args.map((a) => String(a)).join(" ")); - }); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "abc123" + test("does not warn on a deprecated provider (but still strips it) when goViperCompat is omitted", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const warnings: Array<string> = []; + const errorSpy = vi.spyOn(console, "error").mockImplementation((...args) => { + warnings.push(args.map((a) => String(a)).join(" ")); + }); + yield* Effect.addFinalizer(() => Effect.sync(() => errorSpy.mockRestore())); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "abc123" [auth.external.slack] enabled = true `, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect("slack" in loaded!.config.auth.external).toBe(false); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - } finally { - errorSpy.mockRestore(); - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("does not resolve a lowercase-named env() reference when goViperCompat is omitted", async () => { - const previous = process.env.lowercase_ref_default_off_test; - process.env.lowercase_ref_default_off_test = "lowercase-ref-value"; - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "env(lowercase_ref_default_off_test)"\n`, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.config.project_id).toBe("env(lowercase_ref_default_off_test)"); - } finally { - if (previous === undefined) { - delete process.env.lowercase_ref_default_off_test; - } else { - process.env.lowercase_ref_default_off_test = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect("slack" in loaded!.config.auth.external).toBe(false); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); - test("resolves a lowercase-named env() reference when goViperCompat is true", async () => { - const previous = process.env.lowercase_ref_default_on_test; - process.env.lowercase_ref_default_on_test = "lowercase-ref-value"; - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "config.toml"), - `project_id = "env(lowercase_ref_default_on_test)"\n`, - ); - - const loaded = await runConfigEffect(loadCliConfig(cwd, { goViperCompat: true })); - expect(loaded!.config.project_id).toBe("lowercase-ref-value"); - } finally { - if (previous === undefined) { - delete process.env.lowercase_ref_default_on_test; - } else { - process.env.lowercase_ref_default_on_test = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + test("does not resolve a lowercase-named env() reference when goViperCompat is omitted", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + yield* stubEnv("lowercase_ref_default_off_test", "lowercase-ref-value"); + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "env(lowercase_ref_default_off_test)"\n`, + ); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.config.project_id).toBe("env(lowercase_ref_default_off_test)"); + }), + )); + + test("resolves a lowercase-named env() reference when goViperCompat is true", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + yield* stubEnv("lowercase_ref_default_on_test", "lowercase-ref-value"); + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + `project_id = "env(lowercase_ref_default_on_test)"\n`, + ); + + const loaded = yield* loadCliConfig(cwd, { goViperCompat: true }); + expect(loaded!.config.project_id).toBe("lowercase-ref-value"); + }), + )); }); describe("saveCliConfig atomic failures", () => { - async function prepareConfigFile() { - const cwd = makeTempProject(); - const directory = join(cwd, "supabase"); - const filePath = join(directory, "config.json"); - await mkdir(directory, { recursive: true }); - await writeFile(filePath, JSON.stringify({ project_id: "old-ref" })); - return { cwd, directory, filePath }; + function prepareConfigFile() { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const directory = join(cwd, "supabase"); + const filePath = join(directory, "config.json"); + yield* fs.makeDirectory(directory, { recursive: true }); + yield* fs.writeFileString(filePath, yield* encodeJson({ project_id: "old-ref" })); + return { cwd, directory, filePath }; + }); } function runSaveWithFileSystem( cwd: string, decorate: (fs: FileSystem.FileSystem) => FileSystem.FileSystem, ) { - return Effect.runPromiseExit( + return Effect.exit( Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; return yield* saveCliConfig({ cwd, config: sampleConfig }).pipe( Effect.provideService(FileSystem.FileSystem, decorate(fs)), ); - }).pipe(Effect.provide(BunServices.layer)), + }), ); } @@ -1508,103 +1518,115 @@ describe("saveCliConfig atomic failures", () => { }); } - test("surfaces a typed PlatformError when the temp-file write fails and leaves the original intact", async () => { - const { cwd, directory, filePath } = await prepareConfigFile(); - - try { - const injectedError = injectedFailure("writeFileString", filePath); - const exit = await runSaveWithFileSystem(cwd, (fs) => ({ - ...fs, - writeFileString: (candidate, content, options) => - candidate.startsWith(`${filePath}.tmp.`) - ? Effect.fail(injectedError) - : fs.writeFileString(candidate, content, options), - })); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) return; - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(error.value).toBe(injectedError); - } - expect(JSON.parse(await readFile(filePath, "utf8"))).toEqual({ project_id: "old-ref" }); - expect( - (await readdir(directory)).filter((entry) => entry.startsWith("config.json.tmp.")), - ).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("surfaces a typed PlatformError when the temp-file write fails and leaves the original intact", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { cwd, directory, filePath } = yield* prepareConfigFile(); + + const injectedError = injectedFailure("writeFileString", filePath); + const exit = yield* runSaveWithFileSystem(cwd, (fs) => ({ + ...fs, + writeFileString: (candidate, content, options) => + candidate.startsWith(`${filePath}.tmp.`) + ? Effect.fail(injectedError) + : fs.writeFileString(candidate, content, options), + })); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) return; + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(error.value).toBe(injectedError); + } + expect(yield* decodeJson(yield* fs.readFileString(filePath))).toEqual({ + project_id: "old-ref", + }); + expect( + (yield* fs.readDirectory(directory)).filter((entry) => + entry.startsWith("config.json.tmp."), + ), + ).toEqual([]); + }), + )); - test("surfaces a typed PlatformError when rename fails and cleans the temp file", async () => { - const { cwd, directory, filePath } = await prepareConfigFile(); - - try { - const injectedError = injectedFailure("rename", filePath); - const exit = await runSaveWithFileSystem(cwd, (fs) => ({ - ...fs, - rename: (from, to) => (to === filePath ? Effect.fail(injectedError) : fs.rename(from, to)), - })); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) return; - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(error.value).toBe(injectedError); - } - expect(JSON.parse(await readFile(filePath, "utf8"))).toEqual({ project_id: "old-ref" }); - expect( - (await readdir(directory)).filter((entry) => entry.startsWith("config.json.tmp.")), - ).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("surfaces a typed PlatformError when rename fails and cleans the temp file", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { cwd, directory, filePath } = yield* prepareConfigFile(); - test("cleans the temp file when interrupted during rename", async () => { - const { cwd, directory, filePath } = await prepareConfigFile(); + const injectedError = injectedFailure("rename", filePath); + const exit = yield* runSaveWithFileSystem(cwd, (fs) => ({ + ...fs, + rename: (from, to) => + to === filePath ? Effect.fail(injectedError) : fs.rename(from, to), + })); - try { - await Effect.runPromise( - Effect.gen(function* () { - const fs = yield* FileSystem.FileSystem; - const renameStarted = yield* Deferred.make<void>(); - const interruptedFs: FileSystem.FileSystem = { - ...fs, - rename: (from, to) => - to === filePath - ? Deferred.succeed(renameStarted, undefined).pipe(Effect.andThen(Effect.never)) - : fs.rename(from, to), - }; - const fiber = yield* Effect.forkChild( - saveCliConfig({ cwd, config: sampleConfig }).pipe( - Effect.provideService(FileSystem.FileSystem, interruptedFs), - ), - { startImmediately: true }, - ); - yield* Deferred.await(renameStarted); - yield* Fiber.interrupt(fiber); - }).pipe(Effect.provide(BunServices.layer)), - ); - - expect(JSON.parse(await readFile(filePath, "utf8"))).toEqual({ project_id: "old-ref" }); - expect( - (await readdir(directory)).filter((entry) => entry.startsWith("config.json.tmp.")), - ).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) return; + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(error.value).toBe(injectedError); + } + expect(yield* decodeJson(yield* fs.readFileString(filePath))).toEqual({ + project_id: "old-ref", + }); + expect( + (yield* fs.readDirectory(directory)).filter((entry) => + entry.startsWith("config.json.tmp."), + ), + ).toEqual([]); + }), + )); + + test("cleans the temp file when interrupted during rename", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { cwd, directory, filePath } = yield* prepareConfigFile(); + + const renameStarted = yield* Deferred.make<void>(); + const interruptedFs: FileSystem.FileSystem = { + ...fs, + rename: (from, to) => + to === filePath + ? Deferred.succeed(renameStarted, undefined).pipe(Effect.andThen(Effect.never)) + : fs.rename(from, to), + }; + const fiber = yield* Effect.forkChild( + saveCliConfig({ cwd, config: sampleConfig }).pipe( + Effect.provideService(FileSystem.FileSystem, interruptedFs), + ), + { startImmediately: true }, + ); + yield* Deferred.await(renameStarted); + yield* Fiber.interrupt(fiber); + + expect(yield* decodeJson(yield* fs.readFileString(filePath))).toEqual({ + project_id: "old-ref", + }); + expect( + (yield* fs.readDirectory(directory)).filter((entry) => + entry.startsWith("config.json.tmp."), + ), + ).toEqual([]); + }), + )); }); describe("config io [remotes.*] merge", () => { - async function writeTomlProject(toml: string): Promise<string> { - const cwd = makeTempProject(); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), toml); - return cwd; + function writeTomlProject(toml: string) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), toml); + return cwd; + }); } // Remote `project_id`s below are valid 20-lowercase-letter refs, since `Config.Validate` @@ -1656,71 +1678,64 @@ enabled = false }; } - test("tracks the source of effective local, remote, and environment values", async () => { - const localCwd = await writeTomlProject(`project_id = "baseref" + test("tracks the source of effective local, remote, and environment values", () => + runConfigEffect( + Effect.gen(function* () { + const localCwd = yield* writeTomlProject(`project_id = "baseref" [api] port = 6001 `); - const envCwd = await writeTomlProject(`project_id = "baseref" + const envCwd = yield* writeTomlProject(`project_id = "baseref" [api] port = "env(API_PORT)" `); - const remoteCwd = await writeTomlProject(`project_id = "baseref" + const remoteCwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "${PREVIEW_REF}" [remotes.preview.db] port = 6002 `); - const remoteEnvCwd = await writeTomlProject(`project_id = "baseref" + const remoteEnvCwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "${PREVIEW_REF}" [remotes.preview.db] port = "env(REMOTE_DB_PORT)" `); - const omittedCwd = await writeTomlProject(`project_id = "baseref" + const omittedCwd = yield* writeTomlProject(`project_id = "baseref" `); - try { - const loaded = await runConfigEffect(loadCliConfig(localCwd)); - const envLoaded = await runConfigEffect( - loadCliConfig(envCwd, { cliProjectEnv: injectedProjectEnv({ API_PORT: "6001" }) }), - ); - const remoteLoaded = await runConfigEffect( - loadCliConfig(remoteCwd, { projectRef: PREVIEW_REF }), - ); - const remoteEnvLoaded = await runConfigEffect( - loadCliConfig(remoteEnvCwd, { + const loaded = yield* loadCliConfig(localCwd); + const envLoaded = yield* loadCliConfig(envCwd, { + cliProjectEnv: injectedProjectEnv({ API_PORT: "6001" }), + }); + const remoteLoaded = yield* loadCliConfig(remoteCwd, { projectRef: PREVIEW_REF }); + const remoteEnvLoaded = yield* loadCliConfig(remoteEnvCwd, { projectRef: PREVIEW_REF, cliProjectEnv: injectedProjectEnv({ REMOTE_DB_PORT: "6003" }), - }), - ); - const omittedLoaded = await runConfigEffect(loadCliConfig(omittedCwd)); - - expect(originAt(loaded, ["api", "port"])).toBe("local"); - expect(originAt(envLoaded, ["api", "port"])).toBe("environment"); - expect(originAt(remoteLoaded, ["db", "port"])).toBe("remote"); - expect(originAt(remoteEnvLoaded, ["db", "port"])).toBe("environment"); - expect(originAt(omittedLoaded, ["studio", "port"])).toBeUndefined(); - } finally { - await Promise.all( - [localCwd, envCwd, remoteCwd, remoteEnvCwd, omittedCwd].map((cwd) => - rm(cwd, { recursive: true, force: true }), - ), - ); - } - }); + }); + const omittedLoaded = yield* loadCliConfig(omittedCwd); + + expect(originAt(loaded, ["api", "port"])).toBe("local"); + expect(originAt(envLoaded, ["api", "port"])).toBe("environment"); + expect(originAt(remoteLoaded, ["db", "port"])).toBe("remote"); + expect(originAt(remoteEnvLoaded, ["db", "port"])).toBe("environment"); + expect(originAt(omittedLoaded, ["studio", "port"])).toBeUndefined(); + }), + )); - test("tracks environment origins for local and selected-remote array leaves", async () => { - const localArrayCwd = await writeTomlProject(`project_id = "baseref" + test("tracks environment origins for local and selected-remote array leaves", () => + runConfigEffect( + Effect.gen(function* () { + const localArrayCwd = yield* writeTomlProject(`project_id = "baseref" [api] schemas = ["env(LOCAL_SCHEMA)"] `); - const remoteArrayCwd = await writeTomlProject(`project_id = "baseref" + const remoteArrayCwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "${PREVIEW_REF}" @@ -1728,49 +1743,41 @@ project_id = "${PREVIEW_REF}" schemas = ["env(REMOTE_SCHEMA)"] `); - try { - const localArrayLoaded = await runConfigEffect( - loadCliConfig(localArrayCwd, { + const localArrayLoaded = yield* loadCliConfig(localArrayCwd, { cliProjectEnv: injectedProjectEnv({ LOCAL_SCHEMA: "local_schema" }), - }), - ); - const remoteArrayLoaded = await runConfigEffect( - loadCliConfig(remoteArrayCwd, { + }); + const remoteArrayLoaded = yield* loadCliConfig(remoteArrayCwd, { projectRef: PREVIEW_REF, cliProjectEnv: injectedProjectEnv({ REMOTE_SCHEMA: "remote_schema" }), - }), - ); - - expect(originAt(localArrayLoaded, ["api", "schemas"])).toBe("environment"); - expect(originAt(remoteArrayLoaded, ["api", "schemas"])).toBe("environment"); - } finally { - await Promise.all( - [localArrayCwd, remoteArrayCwd].map((cwd) => rm(cwd, { recursive: true, force: true })), - ); - } - }); + }); - test("merges the matching remote subtree over the base before decode", async () => { - const cwd = await writeTomlProject(BASE_WITH_REMOTES); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - expect(loaded!.appliedRemote).toBe("preview"); - expect(loaded!.config.project_id).toBe(PREVIEW_REF); - expect(loaded!.config.api.max_rows).toBe(999); - // Array replaced wholesale, not element-merged. - expect(loaded!.config.api.schemas).toEqual(["remote_only"]); - expect(loaded!.config.api.enabled).toBe(true); - expect(loaded!.config.db.major_version).toBe(15); - expect(loaded!.document?.remotes).toBeUndefined(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + expect(originAt(localArrayLoaded, ["api", "schemas"])).toBe("environment"); + expect(originAt(remoteArrayLoaded, ["api", "schemas"])).toBe("environment"); + }), + )); + + test("merges the matching remote subtree over the base before decode", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(BASE_WITH_REMOTES); + + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + expect(loaded!.appliedRemote).toBe("preview"); + expect(loaded!.config.project_id).toBe(PREVIEW_REF); + expect(loaded!.config.api.max_rows).toBe(999); + // Array replaced wholesale, not element-merged. + expect(loaded!.config.api.schemas).toEqual(["remote_only"]); + expect(loaded!.config.api.enabled).toBe(true); + expect(loaded!.config.db.major_version).toBe(15); + expect(loaded!.document?.remotes).toBeUndefined(); + }), + )); - test("rawDocument stays pre-interpolation and pre-merge, with remotes intact, while document is resolved and merged", async () => { - const previous = process.env.SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST; - process.env.SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST = "555"; - const cwd = await writeTomlProject(`project_id = "baseref" + test("rawDocument stays pre-interpolation and pre-merge, with remotes intact, while document is resolved and merged", () => + runConfigEffect( + Effect.gen(function* () { + yield* stubEnv("SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST", "555"); + const cwd = yield* writeTomlProject(`project_id = "baseref" [api] max_rows = "env(SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST)" @@ -1780,155 +1787,135 @@ project_id = "${PREVIEW_REF}" [remotes.preview.api] max_rows = 999 `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - - expect(loaded!.config.api.max_rows).toBe(999); - expect(loaded!.document?.remotes).toBeUndefined(); - - const rawApi = loaded!.rawDocument?.api; - expect(isObject(rawApi) ? rawApi.max_rows : undefined).toBe( - "env(SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST)", - ); - const rawRemotes = loaded!.rawDocument?.remotes; - expect(isObject(rawRemotes) ? Object.keys(rawRemotes) : undefined).toEqual(["preview"]); - const rawPreview = isObject(rawRemotes) ? rawRemotes.preview : undefined; - expect(isObject(rawPreview) ? rawPreview.project_id : undefined).toBe(PREVIEW_REF); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST; - } else { - process.env.SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + + expect(loaded!.config.api.max_rows).toBe(999); + expect(loaded!.document?.remotes).toBeUndefined(); + + const rawApi = loaded!.rawDocument?.api; + expect(isObject(rawApi) ? rawApi.max_rows : undefined).toBe( + "env(SUPABASE_RAW_DOCUMENT_MAX_ROWS_TEST)", + ); + const rawRemotes = loaded!.rawDocument?.remotes; + expect(isObject(rawRemotes) ? Object.keys(rawRemotes) : undefined).toEqual(["preview"]); + const rawPreview = isObject(rawRemotes) ? rawRemotes.preview : undefined; + expect(isObject(rawPreview) ? rawPreview.project_id : undefined).toBe(PREVIEW_REF); + }), + )); // An unselected `remotes` table survives on `document` (interpolated), unlike the matched case // above where it's stripped; callers deciding where to write must still use `rawDocument`. - test("keeps an interpolated remotes table on document when no remote matches", async () => { - const previous = process.env.SUPABASE_UNMATCHED_REMOTE_REF_TEST; - process.env.SUPABASE_UNMATCHED_REMOTE_REF_TEST = STAGING_REF; - const cwd = await writeTomlProject(`project_id = "baseref" + test("keeps an interpolated remotes table on document when no remote matches", () => + runConfigEffect( + Effect.gen(function* () { + yield* stubEnv("SUPABASE_UNMATCHED_REMOTE_REF_TEST", STAGING_REF); + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.staging] project_id = "env(SUPABASE_UNMATCHED_REMOTE_REF_TEST)" `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.appliedRemote).toBeUndefined(); - - const documentRemotes = loaded!.document?.remotes; - const documentStaging = isObject(documentRemotes) ? documentRemotes.staging : undefined; - expect(isObject(documentStaging) ? documentStaging.project_id : undefined).toBe(STAGING_REF); - - const rawRemotes = loaded!.rawDocument?.remotes; - const rawStaging = isObject(rawRemotes) ? rawRemotes.staging : undefined; - expect(isObject(rawStaging) ? rawStaging.project_id : undefined).toBe( - "env(SUPABASE_UNMATCHED_REMOTE_REF_TEST)", - ); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_UNMATCHED_REMOTE_REF_TEST; - } else { - process.env.SUPABASE_UNMATCHED_REMOTE_REF_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.appliedRemote).toBeUndefined(); - test("interpolatedRemotes carries resolved project_id values distinct from the raw literal", async () => { - const previous = process.env.SUPABASE_INTERPOLATED_REMOTES_TEST; - process.env.SUPABASE_INTERPOLATED_REMOTES_TEST = STAGING_REF; - const cwd = await writeTomlProject(`project_id = "baseref" + const documentRemotes = loaded!.document?.remotes; + const documentStaging = isObject(documentRemotes) ? documentRemotes.staging : undefined; + expect(isObject(documentStaging) ? documentStaging.project_id : undefined).toBe( + STAGING_REF, + ); + + const rawRemotes = loaded!.rawDocument?.remotes; + const rawStaging = isObject(rawRemotes) ? rawRemotes.staging : undefined; + expect(isObject(rawStaging) ? rawStaging.project_id : undefined).toBe( + "env(SUPABASE_UNMATCHED_REMOTE_REF_TEST)", + ); + }), + )); + + test("interpolatedRemotes carries resolved project_id values distinct from the raw literal", () => + runConfigEffect( + Effect.gen(function* () { + yield* stubEnv("SUPABASE_INTERPOLATED_REMOTES_TEST", STAGING_REF); + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.staging] project_id = "env(SUPABASE_INTERPOLATED_REMOTES_TEST)" `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd)); - - const interpolatedStaging = loaded!.interpolatedRemotes?.staging; - expect(isObject(interpolatedStaging) ? interpolatedStaging.project_id : undefined).toBe( - STAGING_REF, - ); - - const rawRemotes = loaded!.rawDocument?.remotes; - const rawStaging = isObject(rawRemotes) ? rawRemotes.staging : undefined; - expect(isObject(rawStaging) ? rawStaging.project_id : undefined).toBe( - "env(SUPABASE_INTERPOLATED_REMOTES_TEST)", - ); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_INTERPOLATED_REMOTES_TEST; - } else { - process.env.SUPABASE_INTERPOLATED_REMOTES_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + + const interpolatedStaging = loaded!.interpolatedRemotes?.staging; + expect(isObject(interpolatedStaging) ? interpolatedStaging.project_id : undefined).toBe( + STAGING_REF, + ); + + const rawRemotes = loaded!.rawDocument?.remotes; + const rawStaging = isObject(rawRemotes) ? rawRemotes.staging : undefined; + expect(isObject(rawStaging) ? rawStaging.project_id : undefined).toBe( + "env(SUPABASE_INTERPOLATED_REMOTES_TEST)", + ); + }), + )); - test("carries appliedRemote on CliConfigParseError when the matched remote's decode fails", async () => { - // The remote match/merge notice is owed even when the subsequent decode fails. - // `db.major_version` is an unrelated schema-decode error; the merge must still have - // happened (and be reported) ahead of it. - const cwd = await writeTomlProject( - `${BASE_WITH_REMOTES} + test("carries appliedRemote on CliConfigParseError when the matched remote's decode fails", () => + runConfigEffect( + Effect.gen(function* () { + // The remote match/merge notice is owed even when the subsequent decode fails. + // `db.major_version` is an unrelated schema-decode error; the merge must still have + // happened (and be reported) ahead of it. + const cwd = yield* writeTomlProject( + `${BASE_WITH_REMOTES} [remotes.preview.db] major_version = "not-a-number" `, - ); - try { - const exit = await Effect.runPromiseExit( - loadCliConfig(cwd, { projectRef: PREVIEW_REF }).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { - return; - } - expect(error.value.appliedRemote).toBe("preview"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("loads the base config verbatim when no remote matches", async () => { - const cwd = await writeTomlProject(BASE_WITH_REMOTES); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: "unknownref" })); - expect(loaded!.appliedRemote).toBeUndefined(); - expect(loaded!.config.project_id).toBe("baseref"); - expect(loaded!.config.api.max_rows).toBe(123); - expect(loaded!.config.api.schemas).toEqual(["public", "custom_base"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const exit = yield* Effect.exit(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (!Option.isSome(error) || error.value._tag !== "CliConfigParseError") { + return; + } + expect(error.value.appliedRemote).toBe("preview"); + }), + )); - test("does not merge remotes when no projectRef is requested and none has an empty project_id", async () => { - // `projectRef` defaults to `""`, so this only stays unmerged because neither remote's - // `project_id` is empty. - const cwd = await writeTomlProject(BASE_WITH_REMOTES); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.appliedRemote).toBeUndefined(); - expect(loaded!.config.api.max_rows).toBe(123); - expect(Object.keys(loaded!.config.remotes)).toEqual(["preview", "staging"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("loads the base config verbatim when no remote matches", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(BASE_WITH_REMOTES); - test("rejects duplicate project_id across remotes even when no projectRef is requested", async () => { - // The duplicate-project_id check runs unconditionally on every config load, in the same - // loop that resolves the `[remotes.*]` override — it is not gated on a caller actually - // selecting a remote. - const cwd = await writeTomlProject(`project_id = "baseref" + const loaded = yield* loadCliConfig(cwd, { projectRef: "unknownref" }); + expect(loaded!.appliedRemote).toBeUndefined(); + expect(loaded!.config.project_id).toBe("baseref"); + expect(loaded!.config.api.max_rows).toBe(123); + expect(loaded!.config.api.schemas).toEqual(["public", "custom_base"]); + }), + )); + + test("does not merge remotes when no projectRef is requested and none has an empty project_id", () => + runConfigEffect( + Effect.gen(function* () { + // `projectRef` defaults to `""`, so this only stays unmerged because neither remote's + // `project_id` is empty. + const cwd = yield* writeTomlProject(BASE_WITH_REMOTES); + + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.appliedRemote).toBeUndefined(); + expect(loaded!.config.api.max_rows).toBe(123); + expect(Object.keys(loaded!.config.remotes)).toEqual(["preview", "staging"]); + }), + )); + + test("rejects duplicate project_id across remotes even when no projectRef is requested", () => + runConfigEffect( + Effect.gen(function* () { + // The duplicate-project_id check runs unconditionally on every config load, in the same + // loop that resolves the `[remotes.*]` override — it is not gated on a caller actually + // selecting a remote. + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.a] project_id = "dupref" @@ -1936,25 +1923,22 @@ project_id = "dupref" [remotes.b] project_id = "dupref" `); - try { - const message = await Effect.runPromise( - loadCliConfig(cwd, { goViperCompat: true }).pipe( + + const message = yield* loadCliConfig(cwd, { goViperCompat: true }).pipe( Effect.catchTag("DuplicateRemoteProjectIdError", (error) => Effect.succeed(error.message), ), - Effect.provide(BunServices.layer), - ), - ); - expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); + }), + )); // `goViperCompat` is required even though `projectRef` is passed: the duplicate/format checks // are gated solely on `goViperCompat`, not on whether a remote is being selected. - test("rejects duplicate project_id across remotes with Go's message", async () => { - const cwd = await writeTomlProject(`project_id = "baseref" + test("rejects duplicate project_id across remotes with Go's message", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.a] project_id = "dupref" @@ -1962,25 +1946,25 @@ project_id = "dupref" [remotes.b] project_id = "dupref" `); - try { - const message = await Effect.runPromise( - loadCliConfig(cwd, { projectRef: "dupref", goViperCompat: true }).pipe( + + const message = yield* loadCliConfig(cwd, { + projectRef: "dupref", + goViperCompat: true, + }).pipe( Effect.catchTag("DuplicateRemoteProjectIdError", (error) => Effect.succeed(error.message), ), - Effect.provide(BunServices.layer), - ), - ); - expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); + }), + )); - test("rejects duplicate project_id among remotes that do not match projectRef", async () => { - // The duplicate map is built across all `[remotes.*]` blocks before applying the matching - // override, so a clash between two non-target remotes still fails. - const cwd = await writeTomlProject(`project_id = "baseref" + test("rejects duplicate project_id among remotes that do not match projectRef", () => + runConfigEffect( + Effect.gen(function* () { + // The duplicate map is built across all `[remotes.*]` blocks before applying the matching + // override, so a clash between two non-target remotes still fails. + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.target] project_id = "previewref" @@ -1991,25 +1975,25 @@ project_id = "dupref" [remotes.b] project_id = "dupref" `); - try { - const message = await Effect.runPromise( - loadCliConfig(cwd, { projectRef: "previewref", goViperCompat: true }).pipe( + + const message = yield* loadCliConfig(cwd, { + projectRef: "previewref", + goViperCompat: true, + }).pipe( Effect.catchTag("DuplicateRemoteProjectIdError", (error) => Effect.succeed(error.message), ), - Effect.provide(BunServices.layer), - ), - ); - expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); + }), + )); - test("rejects two remotes that both omit project_id", async () => { - // A missing project_id reads as "", so two remotes that both omit it collide on the empty - // key. - const cwd = await writeTomlProject(`project_id = "baseref" + test("rejects two remotes that both omit project_id", () => + runConfigEffect( + Effect.gen(function* () { + // A missing project_id reads as "", so two remotes that both omit it collide on the empty + // key. + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.a] [remotes.a.api] @@ -2019,66 +2003,63 @@ max_rows = 1 [remotes.b.api] max_rows = 2 `); - try { - const message = await Effect.runPromise( - loadCliConfig(cwd, { projectRef: "previewref", goViperCompat: true }).pipe( + + const message = yield* loadCliConfig(cwd, { + projectRef: "previewref", + goViperCompat: true, + }).pipe( Effect.catchTag("DuplicateRemoteProjectIdError", (error) => Effect.succeed(error.message), ), - Effect.provide(BunServices.layer), - ), - ); - expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(message).toBe("duplicate project_id for [remotes.b] and [remotes.a]"); + }), + )); - test("rejects a remote project_id that is not a valid 20-letter ref, even with no projectRef requested", async () => { - // Every `[remotes.*].project_id` is checked against the ref pattern on every config load, - // not only the one that ends up selected, so this must fail closed even when the caller - // never selects a remote. - const cwd = await writeTomlProject(`project_id = "baseref" + test("rejects a remote project_id that is not a valid 20-letter ref, even with no projectRef requested", () => + runConfigEffect( + Effect.gen(function* () { + // Every `[remotes.*].project_id` is checked against the ref pattern on every config load, + // not only the one that ends up selected, so this must fail closed even when the caller + // never selects a remote. + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.bad] project_id = "not-a-ref" `); - try { - const message = await Effect.runPromise( - loadCliConfig(cwd, { goViperCompat: true }).pipe( + + const message = yield* loadCliConfig(cwd, { goViperCompat: true }).pipe( Effect.catchTag("InvalidRemoteProjectIdError", (error) => Effect.succeed(error.message)), - Effect.provide(BunServices.layer), - ), - ); - expect(message).toBe( - "Invalid config for remotes.bad.project_id. Must be like: abcdefghijklmnopqrst", - ); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(message).toBe( + "Invalid config for remotes.bad.project_id. Must be like: abcdefghijklmnopqrst", + ); + }), + )); - test("the merged document carries pointer sections introduced by the remote", async () => { - const cwd = await writeTomlProject(`project_id = "baseref" + test("the merged document carries pointer sections introduced by the remote", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "${PREVIEW_REF}" [remotes.preview.db.ssl_enforcement] enabled = true `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - // `presenceIn` reads `document` to detect optional pointer sections; - // a remote-introduced `db.ssl_enforcement` must be present there. - const db = loaded!.document?.db; - expect(typeof db === "object" && db !== null && "ssl_enforcement" in db).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - test("forces db.seed.enabled false when the matching remote omits it", async () => { - const cwd = await writeTomlProject(`project_id = "baseref" + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + // `presenceIn` reads `document` to detect optional pointer sections; + // a remote-introduced `db.ssl_enforcement` must be present there. + const db = loaded!.document?.db; + expect(typeof db === "object" && db !== null && "ssl_enforcement" in db).toBe(true); + }), + )); + + test("forces db.seed.enabled false when the matching remote omits it", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(`project_id = "baseref" [db.seed] enabled = true @@ -2088,55 +2069,47 @@ project_id = "${PREVIEW_REF}" [remotes.preview.api] max_rows = 5 `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - expect(loaded!.config.db.seed.enabled).toBe(false); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - test("preserves db.seed.enabled when the matching remote sets it", async () => { - const cwd = await writeTomlProject(`project_id = "baseref" + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + expect(loaded!.config.db.seed.enabled).toBe(false); + }), + )); + + test("preserves db.seed.enabled when the matching remote sets it", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "${PREVIEW_REF}" [remotes.preview.db.seed] enabled = true `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - expect(loaded!.config.db.seed.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - test("resolves env() on a lowercase-named variable, matching Go's case-agnostic matcher", async () => { - // Env-var matching is case-agnostic when `goViperCompat` is set; without it, the strict - // SCREAMING_SNAKE_CASE matcher wouldn't match this lowercase name at all. - const previous = process.env.project_id; - process.env.project_id = "lowercase-ref"; - const cwd = await writeTomlProject(`project_id = "env(project_id)"\n`); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { goViperCompat: true })); - expect(loaded!.config.project_id).toBe("lowercase-ref"); - } finally { - if (previous === undefined) { - delete process.env.project_id; - } else { - process.env.project_id = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + expect(loaded!.config.db.seed.enabled).toBe(true); + }), + )); + + test("resolves env() on a lowercase-named variable, matching Go's case-agnostic matcher", () => + runConfigEffect( + Effect.gen(function* () { + // Env-var matching is case-agnostic when `goViperCompat` is set; without it, the strict + // SCREAMING_SNAKE_CASE matcher wouldn't match this lowercase name at all. + yield* stubEnv("project_id", "lowercase-ref"); + const cwd = yield* writeTomlProject(`project_id = "env(project_id)"\n`); + const loaded = yield* loadCliConfig(cwd, { goViperCompat: true }); + expect(loaded!.config.project_id).toBe("lowercase-ref"); + }), + )); - test("does not match a remote whose project_id is env(REF) against the resolved ref (Go parity)", async () => { - // A `[remotes.x] project_id = "env(REF)"` never matches a caller-supplied, already-resolved - // `REF`: matching compares the literal `env(REF)` string, not what it resolves to. - const previous = process.env.SUPABASE_REMOTE_ENV_REF_TEST; - process.env.SUPABASE_REMOTE_ENV_REF_TEST = PREVIEW_REF; - const cwd = await writeTomlProject(`project_id = "baseref" + test("does not match a remote whose project_id is env(REF) against the resolved ref (Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + // A `[remotes.x] project_id = "env(REF)"` never matches a caller-supplied, already-resolved + // `REF`: matching compares the literal `env(REF)` string, not what it resolves to. + yield* stubEnv("SUPABASE_REMOTE_ENV_REF_TEST", PREVIEW_REF); + const cwd = yield* writeTomlProject(`project_id = "baseref" [api] max_rows = 1 @@ -2146,48 +2119,34 @@ project_id = "env(SUPABASE_REMOTE_ENV_REF_TEST)" [remotes.preview.api] max_rows = 999 `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - expect(loaded!.appliedRemote).toBeUndefined(); - expect(loaded!.config.api.max_rows).toBe(1); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_REMOTE_ENV_REF_TEST; - } else { - process.env.SUPABASE_REMOTE_ENV_REF_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + expect(loaded!.appliedRemote).toBeUndefined(); + expect(loaded!.config.api.max_rows).toBe(1); + }), + )); - test("validates a remote's env(REF) project_id format against its resolved value, not the literal", async () => { - // Format validation runs after `env(...)` resolution, so it validates the resolved - // project_id against the pattern, not the literal `env(REF)` string. - const previous = process.env.SUPABASE_REMOTE_ENV_REF_FORMAT_TEST; - process.env.SUPABASE_REMOTE_ENV_REF_FORMAT_TEST = PREVIEW_REF; - const cwd = await writeTomlProject(`project_id = "baseref" + test("validates a remote's env(REF) project_id format against its resolved value, not the literal", () => + runConfigEffect( + Effect.gen(function* () { + // Format validation runs after `env(...)` resolution, so it validates the resolved + // project_id against the pattern, not the literal `env(REF)` string. + yield* stubEnv("SUPABASE_REMOTE_ENV_REF_FORMAT_TEST", PREVIEW_REF); + const cwd = yield* writeTomlProject(`project_id = "baseref" [remotes.preview] project_id = "env(SUPABASE_REMOTE_ENV_REF_FORMAT_TEST)" `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.appliedRemote).toBeUndefined(); - expect(loaded!.config.project_id).toBe("baseref"); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_REMOTE_ENV_REF_FORMAT_TEST; - } else { - process.env.SUPABASE_REMOTE_ENV_REF_FORMAT_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.appliedRemote).toBeUndefined(); + expect(loaded!.config.project_id).toBe("baseref"); + }), + )); - test("resolves env() references inside the matching remote before merge", async () => { - const previous = process.env.SUPABASE_REMOTE_MAX_ROWS_TEST; - process.env.SUPABASE_REMOTE_MAX_ROWS_TEST = "777"; - const cwd = await writeTomlProject(`project_id = "baseref" + test("resolves env() references inside the matching remote before merge", () => + runConfigEffect( + Effect.gen(function* () { + yield* stubEnv("SUPABASE_REMOTE_MAX_ROWS_TEST", "777"); + const cwd = yield* writeTomlProject(`project_id = "baseref" [api] max_rows = 1 @@ -2197,25 +2156,19 @@ project_id = "${PREVIEW_REF}" [remotes.preview.api] max_rows = "env(SUPABASE_REMOTE_MAX_ROWS_TEST)" `); - try { - const loaded = await runConfigEffect(loadCliConfig(cwd, { projectRef: PREVIEW_REF })); - expect(loaded!.config.api.max_rows).toBe(777); - } finally { - if (previous === undefined) { - delete process.env.SUPABASE_REMOTE_MAX_ROWS_TEST; - } else { - process.env.SUPABASE_REMOTE_MAX_ROWS_TEST = previous; - } - await rm(cwd, { recursive: true, force: true }); - } - }); + const loaded = yield* loadCliConfig(cwd, { projectRef: PREVIEW_REF }); + expect(loaded!.config.api.max_rows).toBe(777); + }), + )); // Business-rule checks (`Auth.External.validate()`, etc.) run exactly once, against the // merged effective config, never iterated over every remote — a non-selected `[remotes.*]` // block's own business-rule violations must not fail the whole config load. - test("loads an unselected remote whose external provider is enabled without a secret", async () => { - const cwd = await writeTomlProject( - `project_id = "baseref" + test("loads an unselected remote whose external provider is enabled without a secret", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject( + `project_id = "baseref" [remotes.staging] project_id = "${STAGING_REF}" @@ -2223,21 +2176,21 @@ project_id = "${STAGING_REF}" [remotes.staging.auth.external.github] enabled = true `, - ); - try { - // [remotes.staging] is never selected/merged, so it's never business-rule-validated, - // even though it decodes fine structurally. - const loaded = await runConfigEffect(loadCliConfig(cwd)); - expect(loaded!.appliedRemote).toBeUndefined(); - expect(loaded!.config.remotes.staging?.auth.external.github.enabled).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + // [remotes.staging] is never selected/merged, so it's never business-rule-validated, + // even though it decodes fine structurally. + const loaded = yield* loadCliConfig(cwd); + expect(loaded!.appliedRemote).toBeUndefined(); + expect(loaded!.config.remotes.staging?.auth.external.github.enabled).toBe(true); + }), + )); - test("still validates the same remote's external provider once it is selected", async () => { - const cwd = await writeTomlProject( - `project_id = "baseref" + test("still validates the same remote's external provider once it is selected", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* writeTomlProject( + `project_id = "baseref" [remotes.staging] project_id = "${STAGING_REF}" @@ -2245,39 +2198,33 @@ project_id = "${STAGING_REF}" [remotes.staging.auth.external.github] enabled = true `, - ); - try { - // Selecting [remotes.staging] merges it into the effective config, which is - // business-rule-validated — a required `client_id`/`secret` is missing, so this must - // still fail. - const exit = await Effect.runPromiseExit( - loadCliConfig(cwd, { projectRef: STAGING_REF }).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("still fails on a structurally malformed value inside an unselected remote", async () => { - // Every remote is structurally decoded regardless of selection — only the - // merged-config-only business rules are skipped for a non-selected remote, not type/shape - // decoding. - const cwd = await writeTomlProject( - `${BASE_WITH_REMOTES} + // Selecting [remotes.staging] merges it into the effective config, which is + // business-rule-validated — a required `client_id`/`secret` is missing, so this must + // still fail. + const exit = yield* Effect.exit(loadCliConfig(cwd, { projectRef: STAGING_REF })); + expect(Exit.isFailure(exit)).toBe(true); + }), + )); + + test("still fails on a structurally malformed value inside an unselected remote", () => + runConfigEffect( + Effect.gen(function* () { + // Every remote is structurally decoded regardless of selection — only the + // merged-config-only business rules are skipped for a non-selected remote, not type/shape + // decoding. + const cwd = yield* writeTomlProject( + `${BASE_WITH_REMOTES} [remotes.staging.db] major_version = "not-a-number" `, - ); - try { - const exit = await Effect.runPromiseExit( - loadCliConfig(cwd).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + const exit = yield* Effect.exit(loadCliConfig(cwd)); + expect(Exit.isFailure(exit)).toBe(true); + }), + )); }); describe("config io deprecated [inbucket] back-compat", () => { @@ -2298,62 +2245,71 @@ describe("config io deprecated [inbucket] back-compat", () => { errorSpy = undefined; }); - async function loadToml(contents: string) { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(path, contents); - try { - return await runConfigEffect(loadCliConfigFile(path)); - } finally { - await rm(cwd, { recursive: true, force: true }); - } + function loadToml(contents: string) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(path, contents); + + return yield* loadCliConfigFile(path); + }); } - test("loads a deprecated [inbucket] section as [local_smtp]", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("loads a deprecated [inbucket] section as [local_smtp]", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [inbucket] enabled = true port = 12345 `, - ); + ); - expect(loaded.config.local_smtp.enabled).toBe(true); - expect(loaded.config.local_smtp.port).toBe(12345); - expect("inbucket" in loaded.config).toBe(false); - expect(loaded.document).not.toHaveProperty("inbucket"); - expect(loaded.document).toHaveProperty("local_smtp"); - expect( - warnings.some((m) => - m.includes( - "WARN: config section [inbucket] is deprecated and will be removed in v3, use [local_smtp] instead.", - ), - ), - ).toBe(true); - }); + expect(loaded.config.local_smtp.enabled).toBe(true); + expect(loaded.config.local_smtp.port).toBe(12345); + expect("inbucket" in loaded.config).toBe(false); + expect(loaded.document).not.toHaveProperty("inbucket"); + expect(loaded.document).toHaveProperty("local_smtp"); + expect( + warnings.some((m) => + m.includes( + "WARN: config section [inbucket] is deprecated and will be removed in v3, use [local_smtp] instead.", + ), + ), + ).toBe(true); + }), + )); - test("fills schema defaults when a deprecated [inbucket] section is partial", async () => { - const loaded = await loadToml( - `project_id = "abc123" + test("fills schema defaults when a deprecated [inbucket] section is partial", () => + runConfigEffect( + Effect.gen(function* () { + const loaded = yield* loadToml( + `project_id = "abc123" [inbucket] port = 9999 `, - ); + ); - // enabled is omitted by the user; the schema default (true) must survive the - // inbucket -> local_smtp rewrite rather than collapsing to a zero value. - expect(loaded.config.local_smtp.enabled).toBe(true); - expect(loaded.config.local_smtp.port).toBe(9999); - }); + // enabled is omitted by the user; the schema default (true) must survive the + // inbucket -> local_smtp rewrite rather than collapsing to a zero value. + expect(loaded.config.local_smtp.enabled).toBe(true); + expect(loaded.config.local_smtp.port).toBe(9999); + }), + )); - test("prefers an explicit [local_smtp] when both sections are present", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("prefers an explicit [local_smtp] when both sections are present", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [inbucket] enabled = true @@ -2363,18 +2319,21 @@ port = 11111 enabled = true port = 22222 `, - ); + ); - expect(loaded.config.local_smtp.port).toBe(22222); - expect(loaded.document).not.toHaveProperty("inbucket"); - // The deprecation warning still fires because the deprecated key was present. - expect(warnings.some((m) => m.includes("[inbucket] is deprecated"))).toBe(true); - }); + expect(loaded.config.local_smtp.port).toBe(22222); + expect(loaded.document).not.toHaveProperty("inbucket"); + // The deprecation warning still fires because the deprecated key was present. + expect(warnings.some((m) => m.includes("[inbucket] is deprecated"))).toBe(true); + }), + )); - test("normalizes a deprecated [remotes.*.inbucket] section", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("normalizes a deprecated [remotes.*.inbucket] section", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [remotes.staging] project_id = "stagingrefaaaaaaaaaa" @@ -2383,34 +2342,38 @@ project_id = "stagingrefaaaaaaaaaa" enabled = true port = 33333 `, - ); + ); - const staging = loaded.config.remotes.staging; - expect(staging?.local_smtp?.port).toBe(33333); - expect(staging).not.toHaveProperty("inbucket"); - expect( - warnings.some((m) => - m.includes( - "WARN: config section [remotes.staging.inbucket] is deprecated and will be removed in v3, use [remotes.staging.local_smtp] instead.", - ), - ), - ).toBe(true); - }); + const staging = loaded.config.remotes.staging; + expect(staging?.local_smtp?.port).toBe(33333); + expect(staging).not.toHaveProperty("inbucket"); + expect( + warnings.some((m) => + m.includes( + "WARN: config section [remotes.staging.inbucket] is deprecated and will be removed in v3, use [remotes.staging.local_smtp] instead.", + ), + ), + ).toBe(true); + }), + )); - test("does not warn when only [local_smtp] is used", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("does not warn when only [local_smtp] is used", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [local_smtp] enabled = true port = 54324 `, - ); + ); - expect(loaded.config.local_smtp.port).toBe(54324); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - }); + expect(loaded.config.local_smtp.port).toBe(54324); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); }); describe("config io deprecated experimental.orioledb_version back-compat", () => { @@ -2429,59 +2392,68 @@ describe("config io deprecated experimental.orioledb_version back-compat", () => errorSpy = undefined; }); - async function loadToml(contents: string) { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(path, contents); - try { - return await runConfigEffect(loadCliConfigFile(path)); - } finally { - await rm(cwd, { recursive: true, force: true }); - } + function loadToml(contents: string) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(path, contents); + + return yield* loadCliConfigFile(path); + }); } - test("promotes a non-empty legacy experimental.orioledb_version to db.orioledb_version", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("promotes a non-empty legacy experimental.orioledb_version to db.orioledb_version", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [experimental] orioledb_version = "15.1.0.150" `, - ); + ); - expect(loaded.config.db.orioledb_version).toBe("15.1.0.150"); - expect(loaded.config.experimental.orioledb_version).toBeUndefined(); - expect(loaded.document).not.toHaveProperty("experimental.orioledb_version"); - expect( - warnings.some((m) => - m.includes( - "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", - ), - ), - ).toBe(true); - }); + expect(loaded.config.db.orioledb_version).toBe("15.1.0.150"); + expect(loaded.config.experimental.orioledb_version).toBeUndefined(); + expect(loaded.document).not.toHaveProperty("experimental.orioledb_version"); + expect( + warnings.some((m) => + m.includes( + "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", + ), + ), + ).toBe(true); + }), + )); - test("does not warn and stays absent when the legacy value is an empty string", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("does not warn and stays absent when the legacy value is an empty string", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [experimental] orioledb_version = "" `, - ); + ); - expect(loaded.config.db.orioledb_version).toBeUndefined(); - expect(loaded.config.experimental.orioledb_version).toBeUndefined(); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - }); + expect(loaded.config.db.orioledb_version).toBeUndefined(); + expect(loaded.config.experimental.orioledb_version).toBeUndefined(); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); - test("promotes a non-empty legacy value when db.orioledb_version is present but empty", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("promotes a non-empty legacy value when db.orioledb_version is present but empty", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [db] orioledb_version = "" @@ -2489,22 +2461,25 @@ orioledb_version = "" [experimental] orioledb_version = "15.1.0.150" `, - ); + ); - expect(loaded.config.db.orioledb_version).toBe("15.1.0.150"); - expect( - warnings.some((m) => - m.includes( - "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", - ), - ), - ).toBe(true); - }); + expect(loaded.config.db.orioledb_version).toBe("15.1.0.150"); + expect( + warnings.some((m) => + m.includes( + "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", + ), + ), + ).toBe(true); + }), + )); - test("an explicit db.orioledb_version wins over a non-empty legacy value, but still warns", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("an explicit db.orioledb_version wins over a non-empty legacy value, but still warns", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [db] orioledb_version = "17.0.0.1" @@ -2512,54 +2487,59 @@ orioledb_version = "17.0.0.1" [experimental] orioledb_version = "15.1.0.150" `, - ); + ); - expect(loaded.config.db.orioledb_version).toBe("17.0.0.1"); - expect( - warnings.some((m) => - m.includes( - "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", - ), - ), - ).toBe(true); - }); + expect(loaded.config.db.orioledb_version).toBe("17.0.0.1"); + expect( + warnings.some((m) => + m.includes( + "WARN: experimental.orioledb_version is deprecated and will be removed in v3, use db.orioledb_version instead.", + ), + ), + ).toBe(true); + }), + )); - test("rejects a non-string legacy experimental.orioledb_version instead of silently discarding it", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - `project_id = "abc123" + test("rejects a non-string legacy experimental.orioledb_version instead of silently discarding it", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + `project_id = "abc123" [experimental] orioledb_version = 1 `, - ); - try { - const exit = await Effect.runPromiseExit( - loadCliConfigFile(path).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(error.value._tag).toBe("CliConfigParseError"); + ); + + const exit = yield* Effect.exit(loadCliConfigFile(path)); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(error.value._tag).toBe("CliConfigParseError"); + } } - } - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }), + )); - test("rejects a non-string canonical db.orioledb_version beside a valid legacy value", async () => { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile( - path, - `project_id = "abc123" + test("rejects a non-string canonical db.orioledb_version beside a valid legacy value", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path, + `project_id = "abc123" [db] orioledb_version = 1 @@ -2567,28 +2547,26 @@ orioledb_version = 1 [experimental] orioledb_version = "15.1.0.150" `, - ); - try { - const exit = await Effect.runPromiseExit( - loadCliConfigFile(path).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(error.value._tag).toBe("CliConfigParseError"); + ); + + const exit = yield* Effect.exit(loadCliConfigFile(path)); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(error.value._tag).toBe("CliConfigParseError"); + } } - } - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + }), + )); - test("normalizes a deprecated remotes.*.experimental.orioledb_version", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("normalizes a deprecated remotes.*.experimental.orioledb_version", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [remotes.staging] project_id = "stagingrefaaaaaaaaaa" @@ -2596,18 +2574,19 @@ project_id = "stagingrefaaaaaaaaaa" [remotes.staging.experimental] orioledb_version = "15.1.0.150" `, - ); + ); - const staging = loaded.config.remotes.staging; - expect(staging?.db?.orioledb_version).toBe("15.1.0.150"); - expect( - warnings.some((m) => - m.includes( - "WARN: remotes.staging.experimental.orioledb_version is deprecated and will be removed in v3, use remotes.staging.db.orioledb_version instead.", - ), - ), - ).toBe(true); - }); + const staging = loaded.config.remotes.staging; + expect(staging?.db?.orioledb_version).toBe("15.1.0.150"); + expect( + warnings.some((m) => + m.includes( + "WARN: remotes.staging.experimental.orioledb_version is deprecated and will be removed in v3, use remotes.staging.db.orioledb_version instead.", + ), + ), + ).toBe(true); + }), + )); }); describe("config io deprecated [auth.external.{linkedin,slack}] back-compat", () => { @@ -2626,95 +2605,110 @@ describe("config io deprecated [auth.external.{linkedin,slack}] back-compat", () errorSpy = undefined; }); - async function loadToml(contents: string, options?: InternalLoadCliConfigOptions) { - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(path, contents); - try { - return await runConfigEffect(loadCliConfigFile(path, options)); - } finally { - await rm(cwd, { recursive: true, force: true }); - } + function loadToml(contents: string, options?: InternalLoadCliConfigOptions) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(path, contents); + + return yield* loadCliConfigFile(path, options); + }); } - test("loads a bare [auth.external.slack] block without required fields", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("loads a bare [auth.external.slack] block without required fields", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [auth.external.slack] enabled = true `, - { goViperCompat: true }, - ); + { goViperCompat: true }, + ); - expect("slack" in loaded.config.auth.external).toBe(false); - expect(loaded.document).not.toHaveProperty("auth.external.slack"); - expect( - warnings.some((m) => - m.includes( - 'WARN: disabling deprecated "slack" provider. It will be removed in v3, use [auth.external.slack_oidc] instead.', - ), - ), - ).toBe(true); - }); + expect("slack" in loaded.config.auth.external).toBe(false); + expect(loaded.document).not.toHaveProperty("auth.external.slack"); + expect( + warnings.some((m) => + m.includes( + 'WARN: disabling deprecated "slack" provider. It will be removed in v3, use [auth.external.slack_oidc] instead.', + ), + ), + ).toBe(true); + }), + )); - test("loads a bare [auth.external.linkedin] block without required fields", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("loads a bare [auth.external.linkedin] block without required fields", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [auth.external.linkedin] enabled = true `, - { goViperCompat: true }, - ); + { goViperCompat: true }, + ); - expect("linkedin" in loaded.config.auth.external).toBe(false); - expect( - warnings.some((m) => - m.includes( - 'WARN: disabling deprecated "linkedin" provider. It will be removed in v3, use [auth.external.linkedin_oidc] instead.', - ), - ), - ).toBe(true); - }); + expect("linkedin" in loaded.config.auth.external).toBe(false); + expect( + warnings.some((m) => + m.includes( + 'WARN: disabling deprecated "linkedin" provider. It will be removed in v3, use [auth.external.linkedin_oidc] instead.', + ), + ), + ).toBe(true); + }), + )); - test("does not warn when the deprecated section is present but disabled", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("does not warn when the deprecated section is present but disabled", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [auth.external.slack] enabled = false `, - ); + ); - expect("slack" in loaded.config.auth.external).toBe(false); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - }); + expect("slack" in loaded.config.auth.external).toBe(false); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); - test("does not warn when only [auth.external.slack_oidc] is used", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("does not warn when only [auth.external.slack_oidc] is used", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [auth.external.slack_oidc] enabled = true client_id = "abc" secret = "shh" `, - ); + ); - expect(loaded.config.auth.external.slack_oidc.enabled).toBe(true); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - }); + expect(loaded.config.auth.external.slack_oidc.enabled).toBe(true); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); - test("strips a deprecated [remotes.*.auth.external.slack] block without warning for an unselected remote", async () => { - captureWarnings(); - const loaded = await loadToml( - `project_id = "abc123" + test("strips a deprecated [remotes.*.auth.external.slack] block without warning for an unselected remote", () => + runConfigEffect( + Effect.gen(function* () { + captureWarnings(); + const loaded = yield* loadToml( + `project_id = "abc123" [remotes.staging] project_id = "stagingrefaaaaaaaaaa" @@ -2722,13 +2716,14 @@ project_id = "stagingrefaaaaaaaaaa" [remotes.staging.auth.external.slack] enabled = true `, - ); + ); - // Not requesting `projectRef` means no remote is selected, so `remotes` survives - // decode verbatim (minus the deprecated key) rather than being merged/dropped. - expect(loaded.config.remotes.staging?.auth.external).not.toHaveProperty("slack"); - expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); - }); + // Not requesting `projectRef` means no remote is selected, so `remotes` survives + // decode verbatim (minus the deprecated key) rather than being merged/dropped. + expect(loaded.config.remotes.staging?.auth.external).not.toHaveProperty("slack"); + expect(warnings.some((m) => m.includes("is deprecated"))).toBe(false); + }), + )); }); describe("remoteNameForProjectRef / remoteProjectIdEntries", () => { @@ -2789,286 +2784,313 @@ describe("remoteNameForProjectRef / remoteProjectIdEntries", () => { }); describe("writeCliConfigDocumentText", () => { - async function tempFileTargets() { - const cwd = makeTempProject(); - const dir = join(cwd, "supabase"); - await mkdir(dir, { recursive: true }); - return { cwd, filePath: join(dir, "config.toml") }; + function tempFileTargets() { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const dir = join(cwd, "supabase"); + yield* fs.makeDirectory(dir, { recursive: true }); + return { cwd, filePath: join(dir, "config.toml") }; + }); } - async function tmpSurvivors(dir: string, baseName: string): Promise<Array<string>> { - const entries = await readdir(dir); - return entries.filter((entry) => entry.startsWith(`${baseName}.tmp.`)); + function tmpSurvivors(dir: string, baseName: string) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const entries = yield* fs.readDirectory(dir); + return entries.filter((entry) => entry.startsWith(`${baseName}.tmp.`)); + }); } - test("replaces the file's content via a temp file in the same directory, then renames over the target", async () => { - const { cwd, filePath } = await tempFileTargets(); + test("replaces the file's content via a temp file in the same directory, then renames over the target", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const { cwd, filePath } = yield* tempFileTargets(); - try { - await writeFile(filePath, 'project_id = "old"\n'); + yield* fs.writeFileString(filePath, 'project_id = "old"\n'); - await runConfigEffect(writeCliConfigDocumentText(filePath, 'project_id = "new"\n')); + yield* writeCliConfigDocumentText(filePath, 'project_id = "new"\n'); - const content = await readFile(filePath, "utf8"); - expect(content).toBe('project_id = "new"\n'); - expect(await tmpSurvivors(join(cwd, "supabase"), "config.toml")).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const content = yield* fs.readFileString(filePath); + expect(content).toBe('project_id = "new"\n'); + expect(yield* tmpSurvivors(join(cwd, "supabase"), "config.toml")).toEqual([]); + }), + )); - test("leaves no temp file behind after a successful write", async () => { - const { cwd, filePath } = await tempFileTargets(); + test("leaves no temp file behind after a successful write", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const { cwd, filePath } = yield* tempFileTargets(); - try { - await writeFile(filePath, 'project_id = "old"\n'); - await runConfigEffect(writeCliConfigDocumentText(filePath, 'project_id = "new"\n')); + yield* fs.writeFileString(filePath, 'project_id = "old"\n'); + yield* writeCliConfigDocumentText(filePath, 'project_id = "new"\n'); - const entries = await readdir(join(cwd, "supabase")); - expect(entries).toEqual(["config.toml"]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const entries = yield* fs.readDirectory(join(cwd, "supabase")); + expect(entries).toEqual(["config.toml"]); + }), + )); - test("preserves the target file's existing mode", async () => { - const { cwd, filePath } = await tempFileTargets(); + test("preserves the target file's existing mode", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { filePath } = yield* tempFileTargets(); - try { - await writeFile(filePath, 'project_id = "old"\n'); - await chmod(filePath, 0o600); + yield* fs.writeFileString(filePath, 'project_id = "old"\n'); + yield* fs.chmod(filePath, 0o600); - await runConfigEffect(writeCliConfigDocumentText(filePath, 'project_id = "new"\n')); + yield* writeCliConfigDocumentText(filePath, 'project_id = "new"\n'); - const info = await stat(filePath); - expect(info.mode & 0o777).toBe(0o600); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + const info = yield* fs.stat(filePath); + expect(info.mode & 0o777).toBe(0o600); + }), + )); - test("fails with a typed CliConfigWriteError (not a defect) when the destination directory doesn't exist, leaving nothing behind", async () => { - const cwd = makeTempProject(); - const filePath = join(cwd, "supabase", "config.toml"); - - try { - const exit = await Effect.runPromiseExit( - writeCliConfigDocumentText(filePath, 'project_id = "new"\n').pipe( - Effect.provide(BunServices.layer), - ), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - // A typed failure, not a defect: findErrorOption only finds `Fail` reasons. - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect((error.value as { _tag: string })._tag).toBe("CliConfigWriteError"); - expect((error.value as { path: string }).path).toBe(filePath); - } - - await expect(readFile(filePath, "utf8")).rejects.toThrow(); - await expect(readdir(join(cwd, "supabase"))).rejects.toThrow(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("fails with a typed CliConfigWriteError (not a defect) when the destination directory doesn't exist, leaving nothing behind", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const filePath = join(cwd, "supabase", "config.toml"); + + const exit = yield* Effect.exit( + writeCliConfigDocumentText(filePath, 'project_id = "new"\n'), + ); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + // A typed failure, not a defect: findErrorOption only finds `Fail` reasons. + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect((error.value as { _tag: string })._tag).toBe("CliConfigWriteError"); + expect((error.value as { path: string }).path).toBe(filePath); + } + + expect(Exit.isFailure(yield* Effect.exit(fs.readFileString(filePath)))).toBe(true); + expect(Exit.isFailure(yield* Effect.exit(fs.readDirectory(join(cwd, "supabase"))))).toBe( + true, + ); + }), + )); // The directory-missing case above fails before any temp file exists, so it never exercises // `Effect.ensuring`'s cleanup removing a real survivor. An existing directory as the // destination lets the temp file get created, so the rename step is what fails instead. - test("fails with a typed CliConfigWriteError when rename fails, and still cleans up the temp file it already created", async () => { - const { cwd, filePath } = await tempFileTargets(); - - try { - await mkdir(filePath); - - const exit = await Effect.runPromiseExit( - writeCliConfigDocumentText(filePath, 'project_id = "new"\n').pipe( - Effect.provide(BunServices.layer), - ), - ); - - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect((error.value as { _tag: string })._tag).toBe("CliConfigWriteError"); - expect((error.value as { path: string }).path).toBe(filePath); - } - - expect(await tmpSurvivors(join(cwd, "supabase"), "config.toml")).toEqual([]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("fails with a typed CliConfigWriteError when rename fails, and still cleans up the temp file it already created", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const { cwd, filePath } = yield* tempFileTargets(); + + yield* fs.makeDirectory(filePath); + + const exit = yield* Effect.exit( + writeCliConfigDocumentText(filePath, 'project_id = "new"\n'), + ); + + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect((error.value as { _tag: string })._tag).toBe("CliConfigWriteError"); + expect((error.value as { path: string }).path).toBe(filePath); + } + + expect(yield* tmpSurvivors(join(cwd, "supabase"), "config.toml")).toEqual([]); + }), + )); }); describe("decodeCliConfigDocumentForValidationEffect", () => { - test("decodes a valid document successfully", async () => { - const config = await runConfigEffect( - decodeCliConfigDocumentForValidationEffect( - { project_id: "abc123" }, - { goViperCompat: true, path: "supabase/config.toml", format: "toml" }, - ), - ); - expect(config.project_id).toBe("abc123"); - }); - - test("fails with a typed CliConfigParseError (SchemaError cause) when a root business-rule check fails", async () => { - const document = { - project_id: "abc123", - auth: { sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } } }, - }; - const exit = await Effect.runPromiseExit( - decodeCliConfigDocumentForValidationEffect(document, { - goViperCompat: true, - path: "supabase/config.toml", - format: "toml", - }).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); - expect(Schema.isSchemaError((error.value as { cause: unknown }).cause)).toBe(true); - } - }); + test("decodes a valid document successfully", () => + runConfigEffect( + Effect.gen(function* () { + const config = yield* decodeCliConfigDocumentForValidationEffect( + { project_id: "abc123" }, + { goViperCompat: true, path: "supabase/config.toml", format: "toml" }, + ); + expect(config.project_id).toBe("abc123"); + }), + )); - test("does not apply business-rule checks to a [remotes.*] block when no remoteName is given (checks disabled)", async () => { - // The same `enabled: true` + empty `account_sid`/`message_service_sid` shape that fails at - // the root above decodes fine inside a `[remotes.*]` block, which decodes with - // `disableChecks: true` since business rules only apply to the merged effective config. - const document = { - project_id: "abc123", - remotes: { - staging: { - project_id: "stagingrefaaaaaaaaaa", + test("fails with a typed CliConfigParseError (SchemaError cause) when a root business-rule check fails", () => + runConfigEffect( + Effect.gen(function* () { + const document = { + project_id: "abc123", auth: { sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } } }, - }, - }, - }; - const config = await runConfigEffect( - decodeCliConfigDocumentForValidationEffect(document, { - goViperCompat: true, - path: "supabase/config.toml", - format: "toml", + }; + const exit = yield* Effect.exit( + decodeCliConfigDocumentForValidationEffect(document, { + goViperCompat: true, + path: "supabase/config.toml", + format: "toml", + }), + ); + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); + expect(Schema.isSchemaError((error.value as { cause: unknown }).cause)).toBe(true); + } }), - ); - expect(config.remotes["staging"]).toBeDefined(); - }); + )); - test("resolves env(VAR) from the project's own .env file, not just the ambient process environment", async () => { - // Resolves `env(VAR)` exactly like `loadCliConfigFile` — including a var that only exists - // in the project's own `.env` file, never in `process.env`. - const cwd = makeTempProject(); - const path = await runConfigEffect(configTomlPath(cwd)); + test("does not apply business-rule checks to a [remotes.*] block when no remoteName is given (checks disabled)", () => + runConfigEffect( + Effect.gen(function* () { + // The same `enabled: true` + empty `account_sid`/`message_service_sid` shape that fails at + // the root above decodes fine inside a `[remotes.*]` block, which decodes with + // `disableChecks: true` since business rules only apply to the merged effective config. + const document = { + project_id: "abc123", + remotes: { + staging: { + project_id: "stagingrefaaaaaaaaaa", + auth: { + sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } }, + }, + }, + }, + }; + const config = yield* decodeCliConfigDocumentForValidationEffect(document, { + goViperCompat: true, + path: "supabase/config.toml", + format: "toml", + }); + expect(config.remotes["staging"]).toBeDefined(); + }), + )); - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(path, 'project_id = "demo"\n'); - await writeFile(join(cwd, "supabase", ".env"), "SUPABASE_VALIDATION_PG_VERSION_TEST=16\n"); + test("resolves env(VAR) from the project's own .env file, not just the ambient process environment", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + // Resolves `env(VAR)` exactly like `loadCliConfigFile` — including a var that only exists + // in the project's own `.env` file, never in `process.env`. + const cwd = yield* makeTempProject(); + const path = yield* configTomlPath(cwd); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(path, 'project_id = "demo"\n'); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + "SUPABASE_VALIDATION_PG_VERSION_TEST=16\n", + ); - const config = await runConfigEffect( - decodeCliConfigDocumentForValidationEffect( + const config = yield* decodeCliConfigDocumentForValidationEffect( { project_id: "abc123", db: { major_version: "env(SUPABASE_VALIDATION_PG_VERSION_TEST)" }, }, { goViperCompat: true, path, format: "toml" }, - ), - ); - expect(config.db.major_version).toBe(16); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + expect(config.db.major_version).toBe(16); + }), + )); - test("leaves an unresolved env(VAR) literal in place rather than failing the field's own required-non-empty check", async () => { - // A required string field spelled as `env(VAR)` with `VAR` unset decodes as the literal - // `env(VAR)` string, which is non-empty, so it still satisfies a required-field check even - // though it never actually resolved. - const document = { - project_id: "abc123", - auth: { - sms: { - twilio: { - enabled: true, - account_sid: "AC123", - message_service_sid: "MG123", - auth_token: "env(SUPABASE_AUTH_SMS_TWILIO_AUTH_TOKEN)", + test("leaves an unresolved env(VAR) literal in place rather than failing the field's own required-non-empty check", () => + runConfigEffect( + Effect.gen(function* () { + // A required string field spelled as `env(VAR)` with `VAR` unset decodes as the literal + // `env(VAR)` string, which is non-empty, so it still satisfies a required-field check even + // though it never actually resolved. + const document = { + project_id: "abc123", + auth: { + sms: { + twilio: { + enabled: true, + account_sid: "AC123", + message_service_sid: "MG123", + auth_token: "env(SUPABASE_AUTH_SMS_TWILIO_AUTH_TOKEN)", + }, + }, }, - }, - }, - }; - const config = await runConfigEffect( - decodeCliConfigDocumentForValidationEffect(document, { - goViperCompat: true, - path: "supabase/config.toml", - format: "toml", + }; + const config = yield* decodeCliConfigDocumentForValidationEffect(document, { + goViperCompat: true, + path: "supabase/config.toml", + format: "toml", + }); + expect(config.auth.sms.twilio.auth_token).toBe("env(SUPABASE_AUTH_SMS_TWILIO_AUTH_TOKEN)"); }), - ); - expect(config.auth.sms.twilio.auth_token).toBe("env(SUPABASE_AUTH_SMS_TWILIO_AUTH_TOKEN)"); - }); + )); - test("remoteName merges the [remotes.*] block over the root and validates the merged projection: a rule violation only visible after the merge fails", async () => { - const document = { - project_id: "abc123", - remotes: { - staging: { - project_id: "stagingrefaaaaaaaaaa", - auth: { sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } } }, - }, - }, - }; - const exit = await Effect.runPromiseExit( - decodeCliConfigDocumentForValidationEffect(document, { - goViperCompat: true, - path: "supabase/config.toml", - format: "toml", - remoteName: "staging", - }).pipe(Effect.provide(BunServices.layer)), - ); - expect(Exit.isFailure(exit)).toBe(true); - if (!Exit.isFailure(exit)) { - return; - } - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); - expect(Schema.isSchemaError((error.value as { cause: unknown }).cause)).toBe(true); - expect((error.value as { appliedRemote?: string }).appliedRemote).toBe("staging"); - } - }); + test("remoteName merges the [remotes.*] block over the root and validates the merged projection: a rule violation only visible after the merge fails", () => + runConfigEffect( + Effect.gen(function* () { + const document = { + project_id: "abc123", + remotes: { + staging: { + project_id: "stagingrefaaaaaaaaaa", + auth: { + sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } }, + }, + }, + }, + }; + const exit = yield* Effect.exit( + decodeCliConfigDocumentForValidationEffect(document, { + goViperCompat: true, + path: "supabase/config.toml", + format: "toml", + remoteName: "staging", + }), + ); + expect(Exit.isFailure(exit)).toBe(true); + if (!Exit.isFailure(exit)) { + return; + } + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect((error.value as { _tag: string })._tag).toBe("CliConfigParseError"); + expect(Schema.isSchemaError((error.value as { cause: unknown }).cause)).toBe(true); + expect((error.value as { appliedRemote?: string }).appliedRemote).toBe("staging"); + } + }), + )); - test("the same document passes when remoteName is omitted — the remote's own rule violation is invisible before the merge", async () => { - const document = { - project_id: "abc123", - remotes: { - staging: { - project_id: "stagingrefaaaaaaaaaa", - auth: { sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } } }, - }, - }, - }; - const config = await runConfigEffect( - decodeCliConfigDocumentForValidationEffect(document, { - goViperCompat: true, - path: "supabase/config.toml", - format: "toml", + test("the same document passes when remoteName is omitted — the remote's own rule violation is invisible before the merge", () => + runConfigEffect( + Effect.gen(function* () { + const document = { + project_id: "abc123", + remotes: { + staging: { + project_id: "stagingrefaaaaaaaaaa", + auth: { + sms: { twilio: { enabled: true, account_sid: "", message_service_sid: "" } }, + }, + }, + }, + }; + const config = yield* decodeCliConfigDocumentForValidationEffect(document, { + goViperCompat: true, + path: "supabase/config.toml", + format: "toml", + }); + expect(config.remotes["staging"]).toBeDefined(); }), - ); - expect(config.remotes["staging"]).toBeDefined(); - }); + )); }); From 588ccc8d9a274ed1a56fa1ca99d9338a1e6be85a Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Wed, 21 Oct 2026 10:59:10 +0000 Subject: [PATCH 09/24] refactor(config): cover the rest of `config/src` with effect lint (CLI-2756) (#7263) ## TL;DR brings the rest of `packages/config/src` under the effect lint ## whats introduced? - `packages/config/src/**` joins the allow list, and the `schema-number` modules stay out until the schema fix on top - `project.ts` yields its parse error directly instead of through `Effect.fail` - the `./io` methods are plain promise returning functions behind `Effect.suspend` instead of `async`, so they resolve and reject exactly as before and never throw synchronously, only `isAsyncFunction` flips - the src tests move to effect, and the entrypoint purity and import contract walkers run on `FileSystem`, skipping entries that vanish mid scan ## ref - related: CLI-2756 --- .oxlintrc.effect.json | 19 +- packages/config/src/compute.unit.test.ts | 2 +- .../config/src/entrypoint-purity.unit.test.ts | 116 +- .../src/functions-manifest.unit.test.ts | 408 ++++--- packages/config/src/io-browser.ts | 10 +- packages/config/src/local.unit.test.ts | 2 +- .../src/monorepo-import-contract.unit.test.ts | 108 +- .../project-config.unit.test.ts | 4 +- .../project-schema.unit.test.ts | 73 +- packages/config/src/project.ts | 4 +- packages/config/src/project.unit.test.ts | 1076 +++++++++-------- .../src/promise-facade.stdin.unit.test.ts | 41 +- packages/config/src/promise-facade.ts | 31 +- .../config/src/promise-facade.unit.test.ts | 497 ++++---- packages/config/src/validate.unit.test.ts | 133 +- 15 files changed, 1363 insertions(+), 1161 deletions(-) diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json index cb3630c581..5fbe226414 100644 --- a/.oxlintrc.effect.json +++ b/.oxlintrc.effect.json @@ -6,7 +6,7 @@ }, "ignorePatterns": [ "**", - "!packages/config/src/io.*", + "!packages/config/src/**", "!packages/stack/**", "!apps/cli/src/auth/**", "!apps/cli/src/cli/**", @@ -136,6 +136,21 @@ "apps/cli/src/command-internal/config-validate.ts", "apps/cli/src/command-internal/config-validate.unit.test.ts", "apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts", - "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts" + "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts", + "packages/config/src/analytics.ts", + "packages/config/src/api.ts", + "packages/config/src/auth/email.ts", + "packages/config/src/auth/index.ts", + "packages/config/src/auth/mfa.ts", + "packages/config/src/auth/rate_limit.ts", + "packages/config/src/auth/sms.ts", + "packages/config/src/compute.ts", + "packages/config/src/db.ts", + "packages/config/src/edge_runtime.ts", + "packages/config/src/inbucket.ts", + "packages/config/src/project-config/api-attributes.ts", + "packages/config/src/realtime.ts", + "packages/config/src/storage.ts", + "packages/config/src/studio.ts" ] } diff --git a/packages/config/src/compute.unit.test.ts b/packages/config/src/compute.unit.test.ts index e402d5b557..691a9a3bd1 100644 --- a/packages/config/src/compute.unit.test.ts +++ b/packages/config/src/compute.unit.test.ts @@ -30,7 +30,7 @@ describe("compute schema", () => { }); test("defaults to an empty section when the key is absent", () => { - expect(Schema.decodeUnknownSync(Schema.Struct({ compute }))({})).toEqual({ compute: {} }); + expect(Schema.decodeSync(Schema.Struct({ compute }))({})).toEqual({ compute: {} }); }); test("drops compute names that are not DNS labels", () => { diff --git a/packages/config/src/entrypoint-purity.unit.test.ts b/packages/config/src/entrypoint-purity.unit.test.ts index fe6bf007f1..baf42a9123 100644 --- a/packages/config/src/entrypoint-purity.unit.test.ts +++ b/packages/config/src/entrypoint-purity.unit.test.ts @@ -1,7 +1,6 @@ import { describe, expect, test } from "vitest"; -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { dirname, join } from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { Data, Effect, Exit, FileSystem, Path, type Scope } from "effect"; import { fileURLToPath } from "node:url"; import * as defaultEntrypoint from "./index.ts"; import * as effectEntrypoint from "./effect.ts"; @@ -13,6 +12,14 @@ import * as internalEntrypoint from "./internal.ts"; // until a platform `Layer` provides them. This test walks the entrypoint's real import graph so // a future edit reintroducing a platform/IO dependency fails here instead of breaking bundling. +function runPlatform<A, E>( + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, +): Promise<A> { + return Effect.runPromise(effect.pipe(Effect.scoped, Effect.provide(BunServices.layer))); +} + +const { dirname, join } = await runPlatform(Effect.service(Path.Path)); + const srcDir = dirname(fileURLToPath(import.meta.url)); const packageRoot = join(srcDir, ".."); @@ -27,7 +34,11 @@ interface TypesBunDefaultExport { readonly default: string; } -const packageJson = JSON.parse(readFileSync(join(packageRoot, "package.json"), "utf8")) as { +const packageJson = JSON.parse( + await runPlatform( + FileSystem.FileSystem.use((fs) => fs.readFileString(join(packageRoot, "package.json"))), + ), +) as { readonly exports: { readonly ".": TypesBunDefaultExport; readonly "./internal": TypesBunDefaultExport; @@ -95,14 +106,19 @@ function findUnresolvableDynamicSpecifiers(rawSource: string): string[] { return unresolvable; } -function readSourceOrThrow(file: string): string { - try { - return readFileSync(file, "utf8"); - } catch { - throw new Error( - `purity walker could not resolve ${file} (imported somewhere in the pure graph)`, - ); - } +class PurityWalkerError extends Data.TaggedError("PurityWalkerError")<{ + readonly message: string; +}> {} + +function readSource(file: string) { + return FileSystem.FileSystem.use((fs) => fs.readFileString(file)).pipe( + Effect.mapError( + () => + new PurityWalkerError({ + message: `purity walker could not resolve ${file} (imported somewhere in the pure graph)`, + }), + ), + ); } interface ImportGraph { @@ -110,7 +126,7 @@ interface ImportGraph { readonly bareSpecifiers: ReadonlySet<string>; } -function collectImportGraph(entryFile: string): ImportGraph { +const collectImportGraph = Effect.fnUntraced(function* (entryFile: string) { const visitedFiles = new Set<string>(); const bareSpecifiers = new Set<string>(); const queue = [entryFile]; @@ -122,16 +138,17 @@ function collectImportGraph(entryFile: string): ImportGraph { } visitedFiles.add(file); - const source = readSourceOrThrow(file); + const source = yield* readSource(file); const unresolvable = findUnresolvableDynamicSpecifiers(source); if (unresolvable.length > 0) { - throw new Error( - `purity walker found a dynamic import()/require() in ${file} whose argument is not a ` + + return yield* new PurityWalkerError({ + message: + `purity walker found a dynamic import()/require() in ${file} whose argument is not a ` + `static string/template literal, so Bun's scanImports() cannot resolve it to a ` + `specifier: ${unresolvable.join(", ")}. Rewrite it as a static specifier so the walker ` + `can verify what it pulls into the bundle.`, - ); + }); } for (const specifier of extractSpecifiers(source)) { @@ -143,8 +160,8 @@ function collectImportGraph(entryFile: string): ImportGraph { } } - return { visitedFiles, bareSpecifiers }; -} + return { visitedFiles, bareSpecifiers } satisfies ImportGraph; +}); describe("extractSpecifiers", () => { test("extracts a dynamic import() specifier", () => { @@ -249,25 +266,28 @@ describe("findUnresolvableDynamicSpecifiers", () => { describe("collectImportGraph", () => { test("throws a named error when a specifier cannot be resolved on disk", () => { const missingFile = join(srcDir, "__does-not-exist__.ts"); - expect(() => collectImportGraph(missingFile)).toThrow( + return expect(runPlatform(collectImportGraph(missingFile))).rejects.toThrow( `purity walker could not resolve ${missingFile} (imported somewhere in the pure graph)`, ); }); - test("throws when a file contains an unresolvable dynamic import()/require() argument", () => { - const cwd = mkdtempSync(join(tmpdir(), "supabase-purity-walker-")); - try { - const entryFile = join(cwd, "entry.ts"); - writeFileSync(entryFile, `const p = "./foo.ts";\nawait import(p);\n`); + test("throws when a file contains an unresolvable dynamic import()/require() argument", () => + runPlatform( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* fs.makeTempDirectoryScoped({ prefix: "supabase-purity-walker-" }); + const entryFile = join(cwd, "entry.ts"); + yield* fs.writeFileString(entryFile, `const p = "./foo.ts";\nawait import(p);\n`); - expect(() => collectImportGraph(entryFile)).toThrow(/cannot resolve it to a specifier/); - } finally { - rmSync(cwd, { recursive: true, force: true }); - } - }); + const error = yield* Effect.flip(collectImportGraph(entryFile)); + expect(error.message).toMatch(/cannot resolve it to a specifier/); + }), + )); }); -const { visitedFiles, bareSpecifiers } = collectImportGraph(join(srcDir, "index.ts")); +const { visitedFiles, bareSpecifiers } = await runPlatform( + collectImportGraph(join(srcDir, "index.ts")), +); // The real, post-partition pure runtime graph: computed by running the walker above and // hardcoded here so both additions and removals are a deliberate, loud review event. @@ -341,7 +361,7 @@ describe("src/index.ts stays browser-safe", () => { // `src/io-browser.ts` (the `browser` condition target for `@supabase/config/io`) must stay just // as bundler-safe as `index.ts` itself — it only adds inert, throw-when-invoked stubs on top of // `export * from "./index.ts"`. Reuses the same walker and allowlist as the `index.ts` suite. -const ioBrowserGraph = collectImportGraph(join(srcDir, "io-browser.ts")); +const ioBrowserGraph = await runPlatform(collectImportGraph(join(srcDir, "io-browser.ts"))); const expectedIoBrowserGraphFiles = [ join(srcDir, "io-browser.ts"), ...expectedPureGraphFiles, @@ -566,14 +586,26 @@ describe("package.json exports map", () => { // The `types`/`default` conditions all point at `dist/` build outputs, which only exist after // `pnpm run build` — intentionally not checked here so this test stays build-independent. // `scripts/build.ts`'s tree-shake/Node-consumer smoke test owns dist correctness instead. - test("the ./io bun condition target exists on disk (its only src target)", () => { - expect(() => readFileSync(join(packageRoot, packageJson.exports["./io"].bun))).not.toThrow(); - }); - - test("the '.', './effect', and './internal' bun condition targets exist on disk", () => { - for (const key of [".", "./effect", "./internal"] as const) { - const target = packageJson.exports[key].bun; - expect(() => readFileSync(join(packageRoot, target))).not.toThrow(); - } - }); + test("the ./io bun condition target exists on disk (its only src target)", () => + runPlatform( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const read = yield* Effect.exit( + fs.readFile(join(packageRoot, packageJson.exports["./io"].bun)), + ); + expect(Exit.isSuccess(read)).toBe(true); + }), + )); + + test("the '.', './effect', and './internal' bun condition targets exist on disk", () => + runPlatform( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + for (const key of [".", "./effect", "./internal"] as const) { + const target = packageJson.exports[key].bun; + const read = yield* Effect.exit(fs.readFile(join(packageRoot, target))); + expect(Exit.isSuccess(read)).toBe(true); + } + }), + )); }); diff --git a/packages/config/src/functions-manifest.unit.test.ts b/packages/config/src/functions-manifest.unit.test.ts index 4538932358..110e1327b9 100644 --- a/packages/config/src/functions-manifest.unit.test.ts +++ b/packages/config/src/functions-manifest.unit.test.ts @@ -1,219 +1,233 @@ import { describe, expect, test } from "vitest"; import { BunServices } from "@effect/platform-bun"; -import { mkdtempSync } from "node:fs"; -import { mkdir, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { Effect, FileSystem, Path, Schema } from "effect"; +import { Effect, FileSystem, Path, Schema, type Scope } from "effect"; import { CliConfigSchema } from "./base.ts"; import { inferFunctionsManifest } from "./functions-manifest.ts"; const decodeCliConfig = Schema.decodeUnknownSync(CliConfigSchema); +const encodeJson = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); -function makeTempProject(): string { - return mkdtempSync(join(tmpdir(), "supabase-functions-manifest-")); +function makeTempProject() { + return FileSystem.FileSystem.use((fs) => + fs.makeTempDirectoryScoped({ prefix: "supabase-functions-manifest-" }), + ); } function runConfigEffect<A, E>( - effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path>, + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, ): Promise<A> { - return Effect.runPromise(effect.pipe(Effect.provide(BunServices.layer))); + return Effect.runPromise(effect.pipe(Effect.scoped, Effect.provide(BunServices.layer))); } describe("functions manifest", () => { - test("detects default functions from the filesystem", async () => { - const cwd = makeTempProject(); - - try { - const functionDir = join(cwd, "supabase", "functions", "hello-world"); - await mkdir(functionDir, { recursive: true }); - await writeFile(join(functionDir, "index.ts"), "Deno.serve(() => new Response())\n"); - await writeFile(join(functionDir, "deno.json"), '{"imports":{}}\n'); - - await expect(runConfigEffect(inferFunctionsManifest({ cwd }))).resolves.toEqual({ - "hello-world": { - enabled: true, - verify_jwt: true, - import_map: "./functions/hello-world/deno.json", - entrypoint: "./functions/hello-world/index.ts", - static_files: [], - env: {}, - }, - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("keeps the default import map when config only customizes other fields", async () => { - const cwd = makeTempProject(); - - try { - const functionDir = join(cwd, "supabase", "functions", "hello-world"); - await mkdir(functionDir, { recursive: true }); - await writeFile(join(functionDir, "index.ts"), "Deno.serve(() => new Response())\n"); - await writeFile(join(functionDir, "deno.json"), '{"imports":{}}\n'); - await writeFile( - join(cwd, "supabase", "config.json"), - JSON.stringify({ + test("detects default functions from the filesystem", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + const functionDir = join(cwd, "supabase", "functions", "hello-world"); + yield* fs.makeDirectory(functionDir, { recursive: true }); + yield* fs.writeFileString( + join(functionDir, "index.ts"), + "Deno.serve(() => new Response())\n", + ); + yield* fs.writeFileString(join(functionDir, "deno.json"), '{"imports":{}}\n'); + + expect(yield* inferFunctionsManifest({ cwd })).toEqual({ + "hello-world": { + enabled: true, + verify_jwt: true, + import_map: "./functions/hello-world/deno.json", + entrypoint: "./functions/hello-world/index.ts", + static_files: [], + env: {}, + }, + }); + }), + )); + + test("keeps the default import map when config only customizes other fields", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + const functionDir = join(cwd, "supabase", "functions", "hello-world"); + yield* fs.makeDirectory(functionDir, { recursive: true }); + yield* fs.writeFileString( + join(functionDir, "index.ts"), + "Deno.serve(() => new Response())\n", + ); + yield* fs.writeFileString(join(functionDir, "deno.json"), '{"imports":{}}\n'); + yield* fs.writeFileString( + join(cwd, "supabase", "config.json"), + yield* encodeJson({ + functions: { + "hello-world": { + verify_jwt: false, + }, + }, + }), + ); + + expect(yield* inferFunctionsManifest({ cwd })).toEqual({ + "hello-world": { + enabled: true, + verify_jwt: false, + import_map: "./functions/hello-world/deno.json", + entrypoint: "./functions/hello-world/index.ts", + static_files: [], + env: {}, + }, + }); + }), + )); + + test("applies config-only custom functions", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + const config = decodeCliConfig({ + functions: { + "custom-entrypoint": { + entrypoint: "./functions/custom-entrypoint/main.ts", + import_map: "./functions/custom-entrypoint/deno.json", + static_files: ["./functions/custom-entrypoint/*.html"], + env: { + OPENAI_API_KEY: "env(OPENAI_API_KEY)", + }, + }, + }, + }); + + expect(yield* inferFunctionsManifest({ cwd, config })).toEqual({ + "custom-entrypoint": { + enabled: true, + verify_jwt: true, + import_map: "./functions/custom-entrypoint/deno.json", + entrypoint: "./functions/custom-entrypoint/main.ts", + static_files: ["./functions/custom-entrypoint/*.html"], + env: { + OPENAI_API_KEY: "env(OPENAI_API_KEY)", + }, + }, + }); + }), + )); + + test("uses slug defaults for config-only functions with non-path overrides", () => + runConfigEffect( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + const config = decodeCliConfig({ functions: { "hello-world": { verify_jwt: false, }, }, - }), - ); - - await expect(runConfigEffect(inferFunctionsManifest({ cwd }))).resolves.toEqual({ - "hello-world": { - enabled: true, - verify_jwt: false, - import_map: "./functions/hello-world/deno.json", - entrypoint: "./functions/hello-world/index.ts", - static_files: [], - env: {}, - }, - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("applies config-only custom functions", async () => { - const cwd = makeTempProject(); - const config = decodeCliConfig({ - functions: { - "custom-entrypoint": { - entrypoint: "./functions/custom-entrypoint/main.ts", - import_map: "./functions/custom-entrypoint/deno.json", - static_files: ["./functions/custom-entrypoint/*.html"], - env: { - OPENAI_API_KEY: "env(OPENAI_API_KEY)", + }); + + expect(yield* inferFunctionsManifest({ cwd, config })).toEqual({ + "hello-world": { + enabled: true, + verify_jwt: false, + import_map: "", + entrypoint: "./functions/hello-world/index.ts", + static_files: [], + env: {}, }, - }, - }, - }); - - try { - await expect(runConfigEffect(inferFunctionsManifest({ cwd, config }))).resolves.toEqual({ - "custom-entrypoint": { - enabled: true, - verify_jwt: true, - import_map: "./functions/custom-entrypoint/deno.json", - entrypoint: "./functions/custom-entrypoint/main.ts", - static_files: ["./functions/custom-entrypoint/*.html"], - env: { - OPENAI_API_KEY: "env(OPENAI_API_KEY)", + }); + }), + )); + + test("keeps disabled filesystem functions in the inferred manifest", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const config = decodeCliConfig({ + functions: { + "hello-world": { + enabled: false, + }, }, - }, - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("uses slug defaults for config-only functions with non-path overrides", async () => { - const cwd = makeTempProject(); - const config = decodeCliConfig({ - functions: { - "hello-world": { - verify_jwt: false, - }, - }, - }); - - try { - await expect(runConfigEffect(inferFunctionsManifest({ cwd, config }))).resolves.toEqual({ - "hello-world": { - enabled: true, - verify_jwt: false, - import_map: "", - entrypoint: "./functions/hello-world/index.ts", - static_files: [], - env: {}, - }, - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("keeps disabled filesystem functions in the inferred manifest", async () => { - const cwd = makeTempProject(); - const config = decodeCliConfig({ - functions: { - "hello-world": { - enabled: false, - }, - }, - }); - - try { - const functionDir = join(cwd, "supabase", "functions", "hello-world"); - await mkdir(functionDir, { recursive: true }); - await writeFile(join(functionDir, "index.ts"), "Deno.serve(() => new Response())\n"); - - await expect(runConfigEffect(inferFunctionsManifest({ cwd, config }))).resolves.toEqual({ - "hello-world": { - enabled: false, - verify_jwt: true, - import_map: "", - entrypoint: "./functions/hello-world/index.ts", - static_files: [], - env: {}, - }, - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("ignores directories that are not default function shapes", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase", "functions", "missing-entrypoint"), { - recursive: true, - }); - await mkdir(join(cwd, "supabase", "functions", "invalid.slug"), { recursive: true }); - await writeFile( - join(cwd, "supabase", "functions", "invalid.slug", "index.ts"), - "Deno.serve(() => new Response())\n", - ); - - await expect(runConfigEffect(inferFunctionsManifest({ cwd }))).resolves.toEqual({}); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("search: false does not climb to an ancestor project's functions", async () => { - const projectRoot = makeTempProject(); - const nestedCwd = join(projectRoot, "nested", "workdir"); - - try { - const functionDir = join(projectRoot, "supabase", "functions", "hello-world"); - await mkdir(functionDir, { recursive: true }); - await writeFile(join(functionDir, "index.ts"), "Deno.serve(() => new Response())\n"); - await writeFile(join(projectRoot, "supabase", "config.json"), "{}\n"); - await mkdir(nestedCwd, { recursive: true }); - - await expect(runConfigEffect(inferFunctionsManifest({ cwd: nestedCwd }))).resolves.toEqual({ - "hello-world": { - enabled: true, - verify_jwt: true, - import_map: "", - entrypoint: "./functions/hello-world/index.ts", - static_files: [], - env: {}, - }, - }); - - await expect( - runConfigEffect(inferFunctionsManifest({ cwd: nestedCwd, search: false })), - ).resolves.toEqual({}); - } finally { - await rm(projectRoot, { recursive: true, force: true }); - } - }); + }); + + const functionDir = join(cwd, "supabase", "functions", "hello-world"); + yield* fs.makeDirectory(functionDir, { recursive: true }); + yield* fs.writeFileString( + join(functionDir, "index.ts"), + "Deno.serve(() => new Response())\n", + ); + + expect(yield* inferFunctionsManifest({ cwd, config })).toEqual({ + "hello-world": { + enabled: false, + verify_jwt: true, + import_map: "", + entrypoint: "./functions/hello-world/index.ts", + static_files: [], + env: {}, + }, + }); + }), + )); + + test("ignores directories that are not default function shapes", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase", "functions", "missing-entrypoint"), { + recursive: true, + }); + yield* fs.makeDirectory(join(cwd, "supabase", "functions", "invalid.slug"), { + recursive: true, + }); + yield* fs.writeFileString( + join(cwd, "supabase", "functions", "invalid.slug", "index.ts"), + "Deno.serve(() => new Response())\n", + ); + + expect(yield* inferFunctionsManifest({ cwd })).toEqual({}); + }), + )); + + test("search: false does not climb to an ancestor project's functions", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const projectRoot = yield* makeTempProject(); + const nestedCwd = join(projectRoot, "nested", "workdir"); + + const functionDir = join(projectRoot, "supabase", "functions", "hello-world"); + yield* fs.makeDirectory(functionDir, { recursive: true }); + yield* fs.writeFileString( + join(functionDir, "index.ts"), + "Deno.serve(() => new Response())\n", + ); + yield* fs.writeFileString(join(projectRoot, "supabase", "config.json"), "{}\n"); + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + + expect(yield* inferFunctionsManifest({ cwd: nestedCwd })).toEqual({ + "hello-world": { + enabled: true, + verify_jwt: true, + import_map: "", + entrypoint: "./functions/hello-world/index.ts", + static_files: [], + env: {}, + }, + }); + + expect(yield* inferFunctionsManifest({ cwd: nestedCwd, search: false })).toEqual({}); + }), + )); }); diff --git a/packages/config/src/io-browser.ts b/packages/config/src/io-browser.ts index 514e5c1e1f..682211bfb4 100644 --- a/packages/config/src/io-browser.ts +++ b/packages/config/src/io-browser.ts @@ -5,11 +5,13 @@ import type { CliConfigIo } from "./promise-facade.ts"; // entrypoint instead. // // Exports the same seven names as `bun.ts`/`node.ts` so bundlers resolve named imports, and each -// export only throws once invoked — so a bundle that imports this module without calling into it +// export only rejects once invoked — so a bundle that imports this module without calling into it // doesn't crash at module evaluation. -async function unavailableInBrowser(): Promise<never> { - throw new Error( - '@supabase/config/io is not available in browser bundles; import the pure surface from "@supabase/config" instead.', +function unavailableInBrowser(): Promise<never> { + return Promise.reject( + new Error( + '@supabase/config/io is not available in browser bundles; import the pure surface from "@supabase/config" instead.', + ), ); } diff --git a/packages/config/src/local.unit.test.ts b/packages/config/src/local.unit.test.ts index 9cbe883c47..5711458ace 100644 --- a/packages/config/src/local.unit.test.ts +++ b/packages/config/src/local.unit.test.ts @@ -5,7 +5,7 @@ import { CliConfigSchema } from "./base.ts"; import { fromConfigDocument } from "./project-config/project-config.ts"; const decode = (toml: string) => - Schema.decodeUnknownSync(CliConfigSchema)(parse(toml), { onExcessProperty: "error" }); + Schema.decodeSync(CliConfigSchema)(parse(toml), { onExcessProperty: "error" }); const services = (entries: string) => `[local.services]\n${entries}\n`; diff --git a/packages/config/src/monorepo-import-contract.unit.test.ts b/packages/config/src/monorepo-import-contract.unit.test.ts index 17702c00a1..a153beb942 100644 --- a/packages/config/src/monorepo-import-contract.unit.test.ts +++ b/packages/config/src/monorepo-import-contract.unit.test.ts @@ -1,8 +1,16 @@ import { describe, expect, test } from "vitest"; -import { readdirSync, readFileSync } from "node:fs"; -import { dirname, join, sep } from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, Exit, FileSystem, Option, Path, type PlatformError } from "effect"; import { fileURLToPath } from "node:url"; +function runPlatform<A, E>( + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path>, +): Promise<A> { + return Effect.runPromise(effect.pipe(Effect.provide(BunServices.layer))); +} + +const { dirname, join, sep } = await runPlatform(Effect.service(Path.Path)); + // Enforces the import rules in `packages/config/AGENTS.md`'s "Monorepo import rule". Forbidden // specifiers are built by concatenation so this file's own source can't self-match the scan. const srcDir = dirname(fileURLToPath(import.meta.url)); @@ -17,43 +25,83 @@ const allowedInternalConsumerPrefix = `${join(repoRoot, "apps", "cli")}${sep}`; const EXCLUDED_DIR_NAMES = new Set(["node_modules", "dist", ".repos"]); const thisPackageDir = join(srcDir, ".."); -function collectTsFiles(dir: string, into: string[]): void { - for (const entry of readdirSync(dir, { withFileTypes: true })) { - const fullPath = join(dir, entry.name); +/* Entries deleted mid-walk by concurrently running tests are skipped, not failures. */ +const orNoneIfNotFound = <A, R>(effect: Effect.Effect<A, PlatformError.PlatformError, R>) => + effect.pipe( + Effect.map(Option.some), + Effect.catchReason("PlatformError", "NotFound", () => Effect.succeedNone), + ); - if (entry.isDirectory()) { - if (EXCLUDED_DIR_NAMES.has(entry.name) || fullPath === thisPackageDir) { +const collectTsFiles: ( + dir: string, + into: string[], +) => Effect.Effect<void, PlatformError.PlatformError, FileSystem.FileSystem> = Effect.fnUntraced( + function* (dir, into) { + const fs = yield* FileSystem.FileSystem; + const names = yield* fs.readDirectory(dir); + for (const name of names) { + const fullPath = join(dir, name); + /* Matches `Dirent` semantics: a symlink is neither a directory nor a file. */ + if (Exit.isSuccess(yield* Effect.exit(fs.readLink(fullPath)))) { + continue; + } + const stat = yield* orNoneIfNotFound(fs.stat(fullPath)); + if (Option.isNone(stat)) { continue; } - collectTsFiles(fullPath, into); - } else if (entry.isFile() && (entry.name.endsWith(".ts") || entry.name.endsWith(".tsx"))) { - into.push(fullPath); + const info = stat.value; + + if (info.type === "Directory") { + if (EXCLUDED_DIR_NAMES.has(name) || fullPath === thisPackageDir) { + continue; + } + yield* orNoneIfNotFound(collectTsFiles(fullPath, into)); + } else if (info.type === "File" && (name.endsWith(".ts") || name.endsWith(".tsx"))) { + into.push(fullPath); + } } - } -} + }, +); -function findViolations(forbiddenSpecifier: string): string[] { +const findViolations = Effect.fnUntraced(function* (forbiddenSpecifier: string) { + const fs = yield* FileSystem.FileSystem; const files: string[] = []; for (const workspaceRoot of ["apps", "packages"]) { - collectTsFiles(join(repoRoot, workspaceRoot), files); + yield* collectTsFiles(join(repoRoot, workspaceRoot), files); } - return files.filter((file) => readFileSync(file, "utf8").includes(forbiddenSpecifier)).sort(); -} + const violations: string[] = []; + for (const file of files) { + const source = yield* orNoneIfNotFound(fs.readFileString(file)); + if (Option.isSome(source) && source.value.includes(forbiddenSpecifier)) { + violations.push(file); + } + } + return violations.sort(); +}); describe("monorepo import contract for @supabase/config", () => { - test("no file outside this package imports the @supabase/config/io entrypoint", () => { - expect(findViolations(forbiddenIoSpecifier)).toEqual([]); - }); - - test("no file outside this package deep-imports @supabase/config/src/*", () => { - expect(findViolations(forbiddenDeepImportPrefix)).toEqual([]); - }); - - test("every @supabase/config/internal import outside this package is under apps/cli/", () => { - const violations = findViolations(internalSpecifier).filter( - (file) => !file.startsWith(allowedInternalConsumerPrefix), - ); - expect(violations).toEqual([]); - }); + test("no file outside this package imports the @supabase/config/io entrypoint", () => + runPlatform( + Effect.gen(function* () { + expect(yield* findViolations(forbiddenIoSpecifier)).toEqual([]); + }), + )); + + test("no file outside this package deep-imports @supabase/config/src/*", () => + runPlatform( + Effect.gen(function* () { + expect(yield* findViolations(forbiddenDeepImportPrefix)).toEqual([]); + }), + )); + + test("every @supabase/config/internal import outside this package is under apps/cli/", () => + runPlatform( + Effect.gen(function* () { + const violations = (yield* findViolations(internalSpecifier)).filter( + (file) => !file.startsWith(allowedInternalConsumerPrefix), + ); + expect(violations).toEqual([]); + }), + )); }); diff --git a/packages/config/src/project-config/project-config.unit.test.ts b/packages/config/src/project-config/project-config.unit.test.ts index d92aee1427..ad8c08e127 100644 --- a/packages/config/src/project-config/project-config.unit.test.ts +++ b/packages/config/src/project-config/project-config.unit.test.ts @@ -5,7 +5,7 @@ import { ProjectConfigParseError } from "../errors.ts"; import { isSecretPath, secretPathPatterns } from "../lib/secret-paths.ts"; import { getDefaultCliConfig, omitDefaultValues, subtractCliConfig } from "../sparse.ts"; import type { EffectiveConfig } from "../sparse.ts"; -import { Schema } from "effect"; +import { DateTime, Schema } from "effect"; import { attachApiResponse, comparableProjectConfigPaths, @@ -2049,7 +2049,7 @@ describe("review round: absent anchors, exponent-free seconds, non-plain values }); test("non-plain structured-cloneable values are rejected before attach", () => { - for (const nonPlain of [new Map(), new Set(), new Date()]) { + for (const nonPlain of [new Map(), new Set(), DateTime.toDateUtc(DateTime.nowUnsafe())]) { let thrown: unknown; try { attachApiResponse({}, { x: nonPlain as unknown as ReadonlyJsonValue } as unknown as Record< diff --git a/packages/config/src/project-config/project-schema.unit.test.ts b/packages/config/src/project-config/project-schema.unit.test.ts index d7b131a7de..445295a748 100644 --- a/packages/config/src/project-config/project-schema.unit.test.ts +++ b/packages/config/src/project-config/project-schema.unit.test.ts @@ -1,8 +1,6 @@ import { describe, expect, test } from "vitest"; -import { readFileSync } from "node:fs"; -import { dirname, join } from "node:path"; -import { fileURLToPath } from "node:url"; -import { Schema, SchemaAST } from "effect"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, FileSystem, Path, Schema, SchemaAST } from "effect"; import * as SmolToml from "smol-toml"; import { CliConfigSchema } from "../base.ts"; import { isSecretPath, secretPathPatterns } from "../lib/secret-paths.ts"; @@ -36,10 +34,13 @@ function findAtPattern( return current; } -const legacyFixturePath = join( - dirname(fileURLToPath(import.meta.url)), - "../../testdata/legacy-config.toml", -); +const legacyFixturePath = Effect.gen(function* () { + const path = yield* Path.Path; + return path.join( + path.dirname(yield* path.fromFileUrl(new URL(import.meta.url))), + "../../testdata/legacy-config.toml", + ); +}); function apiEnvelope(attributes: Record<string, unknown>): unknown { return { data: { type: "project_config", id: "abcdefghijklmnopqrst", attributes } }; @@ -67,12 +68,16 @@ describe("ProjectConfigSchema acceptance", () => { expect(() => decodeProjectConfig(projected)).not.toThrow(); }); - test("fromConfigDocument's output over the real legacy-config.toml fixture validates", () => { - const raw = SmolToml.parse(readFileSync(legacyFixturePath, "utf8")); - const config = decodeCliConfig(raw); - const projected = fromConfigDocument(config); - expect(() => decodeProjectConfig(projected)).not.toThrow(); - }); + test("fromConfigDocument's output over the real legacy-config.toml fixture validates", () => + Effect.runPromise( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const raw = SmolToml.parse(yield* fs.readFileString(yield* legacyFixturePath)); + const config = decodeCliConfig(raw); + const projected = fromConfigDocument(config); + expect(() => decodeProjectConfig(projected)).not.toThrow(); + }).pipe(Effect.provide(BunServices.layer)), + )); test("toProjectConfig's cliConfig arm validates", () => { const projected = toProjectConfig({ cliConfig: { api: { max_rows: 100 } } }); @@ -452,23 +457,29 @@ describe("ProjectConfigSchema Standard Schema interop", () => { expect(ProjectConfigSchema["~standard"].version).toBe(1); }); - test("~standard.validate returns a value on success", async () => { - const outcome = ProjectConfigSchema["~standard"].validate({ - auth: { site_url: "https://example.com" }, - }); - const result = outcome instanceof Promise ? await outcome : outcome; - expect(result.issues).toBeUndefined(); - if (!result.issues) { - expect(result.value).toEqual({ auth: { site_url: "https://example.com" } }); - } - }); - - test("~standard.validate returns issues with paths on failure", async () => { - const outcome = ProjectConfigSchema["~standard"].validate({ auth: { site_url: 123 } }); - const result = outcome instanceof Promise ? await outcome : outcome; - expect(result.issues).toBeDefined(); - expect(result.issues?.[0]?.path).toBeDefined(); - }); + test("~standard.validate returns a value on success", () => + Effect.runPromise( + Effect.gen(function* () { + const outcome = ProjectConfigSchema["~standard"].validate({ + auth: { site_url: "https://example.com" }, + }); + const result = outcome instanceof Promise ? yield* Effect.promise(() => outcome) : outcome; + expect(result.issues).toBeUndefined(); + if (!result.issues) { + expect(result.value).toEqual({ auth: { site_url: "https://example.com" } }); + } + }), + )); + + test("~standard.validate returns issues with paths on failure", () => + Effect.runPromise( + Effect.gen(function* () { + const outcome = ProjectConfigSchema["~standard"].validate({ auth: { site_url: 123 } }); + const result = outcome instanceof Promise ? yield* Effect.promise(() => outcome) : outcome; + expect(result.issues).toBeDefined(); + expect(result.issues?.[0]?.path).toBeDefined(); + }), + )); }); describe("toProjectConfigJsonSchema", () => { diff --git a/packages/config/src/project.ts b/packages/config/src/project.ts index 42819fb68b..0a2e30bb3d 100644 --- a/packages/config/src/project.ts +++ b/packages/config/src/project.ts @@ -120,14 +120,14 @@ function parseDotEnv( const match = dotEnvLinePattern.exec(candidate); if (match === null) { - return yield* Effect.fail(new CliProjectEnvParseError({ path, line: index + 1 })); + return yield* new CliProjectEnvParseError({ path, line: index + 1 }); } const key = match[1]; const rawValue = match[2] ?? ""; if (key === undefined) { - return yield* Effect.fail(new CliProjectEnvParseError({ path, line: index + 1 })); + return yield* new CliProjectEnvParseError({ path, line: index + 1 }); } values[key] = parseDotEnvValue(rawValue); diff --git a/packages/config/src/project.unit.test.ts b/packages/config/src/project.unit.test.ts index f57957c43c..0ddd414b00 100644 --- a/packages/config/src/project.unit.test.ts +++ b/packages/config/src/project.unit.test.ts @@ -1,10 +1,6 @@ import { describe, expect, test } from "vitest"; import { BunServices } from "@effect/platform-bun"; -import { mkdtempSync } from "node:fs"; -import { mkdir, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { Effect, FileSystem, Path, Redacted } from "effect"; +import { Effect, FileSystem, Path, Redacted, type Scope } from "effect"; import { findCliProjectRoot as findCliProjectRootFromBun, loadCliProjectEnvironment as loadCliProjectEnvironmentFromBun, @@ -19,299 +15,326 @@ import { } from "./effect.ts"; import { resolveCliConfigValue as resolveCliConfigValueInternal } from "./internal.ts"; -function makeTempProject(): string { - return mkdtempSync(join(tmpdir(), "supabase-project-config-")); +function makeTempProject() { + return FileSystem.FileSystem.use((fs) => + fs.makeTempDirectoryScoped({ prefix: "supabase-project-config-" }), + ); } function runConfigEffect<A, E>( - effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path>, + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, ): Promise<A> { - return Effect.runPromise(effect.pipe(Effect.provide(BunServices.layer))); + return Effect.runPromise(effect.pipe(Effect.scoped, Effect.provide(BunServices.layer))); } describe("project discovery and lazy env resolution", () => { - test("finds the nearest Supabase project upward", async () => { - const cwd = makeTempProject(); - const repoRoot = join(cwd, "repo"); - const packageRoot = join(repoRoot, "apps", "web"); - const nestedCwd = join(packageRoot, "src", "components"); - - try { - await mkdir(join(repoRoot, "supabase"), { recursive: true }); - await mkdir(join(packageRoot, "supabase"), { recursive: true }); - await mkdir(nestedCwd, { recursive: true }); - await writeFile(join(repoRoot, "supabase", "config.toml"), 'project_id = "repo"\n'); - await writeFile(join(packageRoot, "supabase", "config.toml"), 'project_id = "web"\n'); - - const paths = await runConfigEffect(findCliProjectPaths(nestedCwd)); - - expect(paths?.projectRoot).toBe(packageRoot); - expect(paths?.supabaseDir).toBe(join(packageRoot, "supabase")); - expect(paths?.configPath).toBe(join(packageRoot, "supabase", "config.toml")); - expect(await findCliProjectRootFromBun(nestedCwd)).toBe(packageRoot); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("search: false only checks cwd itself, matching Go's exact-workdir resolution", async () => { - const cwd = makeTempProject(); - const repoRoot = join(cwd, "repo"); - const packageRoot = join(repoRoot, "apps", "web"); - const nestedCwd = join(packageRoot, "src", "components"); - - try { - await mkdir(join(repoRoot, "supabase"), { recursive: true }); - await mkdir(nestedCwd, { recursive: true }); - await writeFile(join(repoRoot, "supabase", "config.toml"), 'project_id = "repo"\n'); - - // nestedCwd has no supabase/ of its own; only an ancestor (repoRoot) does. - const searched = await runConfigEffect(findCliProjectPaths(nestedCwd)); - expect(searched?.projectRoot).toBe(repoRoot); - - const unsearched = await runConfigEffect(findCliProjectPaths(nestedCwd, { search: false })); - expect(unsearched).toBeNull(); - - const configAtRepoRoot = await runConfigEffect( - findCliProjectPaths(repoRoot, { search: false }), - ); - expect(configAtRepoRoot?.projectRoot).toBe(repoRoot); - - expect(await runConfigEffect(loadCliConfig(nestedCwd, { search: false }))).toBeNull(); - expect( - await runConfigEffect(loadCliProjectEnvironment({ cwd: nestedCwd, search: false })), - ).toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("climbs past a FILE named `supabase` in the starting directory instead of failing with ENOTDIR", async () => { - const cwd = makeTempProject(); - const nestedCwd = join(cwd, "child"); - - try { - await mkdir(nestedCwd, { recursive: true }); - await writeFile(join(nestedCwd, "supabase"), "not a directory\n"); - - const paths = await runConfigEffect(findCliProjectPaths(nestedCwd)); - - expect(paths).toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("returns the parent's project when the starting directory has a FILE named `supabase` but the parent has a real config", async () => { - const cwd = makeTempProject(); - const child = join(cwd, "child"); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await mkdir(child, { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "parent"\n'); - await writeFile(join(child, "supabase"), "not a directory\n"); - - const paths = await runConfigEffect(findCliProjectPaths(child)); - - expect(paths?.projectRoot).toBe(cwd); - expect(paths?.configPath).toBe(join(cwd, "supabase", "config.toml")); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loads env from the discovered supabase directory with the right precedence", async () => { - const cwd = makeTempProject(); - const repoRoot = join(cwd, "repo"); - const packageRoot = join(repoRoot, "apps", "web"); - const nestedCwd = join(packageRoot, "src"); - - try { - await mkdir(join(repoRoot, "supabase"), { recursive: true }); - await mkdir(join(packageRoot, "supabase"), { recursive: true }); - await mkdir(nestedCwd, { recursive: true }); - await writeFile(join(repoRoot, "supabase", "config.toml"), 'project_id = "repo"\n'); - await writeFile(join(repoRoot, "supabase", ".env"), "ROOT_ONLY=repo\n"); - await writeFile(join(packageRoot, "supabase", "config.toml"), 'project_id = "web"\n'); - await writeFile( - join(packageRoot, "supabase", ".env"), - "SHARED_ONLY=from-env\nOVERRIDE_ME=from-env\n", - ); - await writeFile( - join(packageRoot, "supabase", ".env.local"), - "LOCAL_ONLY=from-local\nOVERRIDE_ME=from-local\n", - ); - - const projectEnv = await runConfigEffect( - loadCliProjectEnvironment({ + test("finds the nearest Supabase project upward", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const repoRoot = join(cwd, "repo"); + const packageRoot = join(repoRoot, "apps", "web"); + const nestedCwd = join(packageRoot, "src", "components"); + + yield* fs.makeDirectory(join(repoRoot, "supabase"), { recursive: true }); + yield* fs.makeDirectory(join(packageRoot, "supabase"), { recursive: true }); + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + yield* fs.writeFileString( + join(repoRoot, "supabase", "config.toml"), + 'project_id = "repo"\n', + ); + yield* fs.writeFileString( + join(packageRoot, "supabase", "config.toml"), + 'project_id = "web"\n', + ); + + const paths = yield* findCliProjectPaths(nestedCwd); + + expect(paths?.projectRoot).toBe(packageRoot); + expect(paths?.supabaseDir).toBe(join(packageRoot, "supabase")); + expect(paths?.configPath).toBe(join(packageRoot, "supabase", "config.toml")); + expect(yield* Effect.promise(() => findCliProjectRootFromBun(nestedCwd))).toBe(packageRoot); + }), + )); + + test("search: false only checks cwd itself, matching Go's exact-workdir resolution", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const repoRoot = join(cwd, "repo"); + const packageRoot = join(repoRoot, "apps", "web"); + const nestedCwd = join(packageRoot, "src", "components"); + + yield* fs.makeDirectory(join(repoRoot, "supabase"), { recursive: true }); + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + yield* fs.writeFileString( + join(repoRoot, "supabase", "config.toml"), + 'project_id = "repo"\n', + ); + + // nestedCwd has no supabase/ of its own; only an ancestor (repoRoot) does. + const searched = yield* findCliProjectPaths(nestedCwd); + expect(searched?.projectRoot).toBe(repoRoot); + + const unsearched = yield* findCliProjectPaths(nestedCwd, { search: false }); + expect(unsearched).toBeNull(); + + const configAtRepoRoot = yield* findCliProjectPaths(repoRoot, { search: false }); + expect(configAtRepoRoot?.projectRoot).toBe(repoRoot); + + expect(yield* loadCliConfig(nestedCwd, { search: false })).toBeNull(); + expect(yield* loadCliProjectEnvironment({ cwd: nestedCwd, search: false })).toBeNull(); + }), + )); + + test("climbs past a FILE named `supabase` in the starting directory instead of failing with ENOTDIR", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const nestedCwd = join(cwd, "child"); + + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + yield* fs.writeFileString(join(nestedCwd, "supabase"), "not a directory\n"); + + const paths = yield* findCliProjectPaths(nestedCwd); + + expect(paths).toBeNull(); + }), + )); + + test("returns the parent's project when the starting directory has a FILE named `supabase` but the parent has a real config", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const child = join(cwd, "child"); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.makeDirectory(child, { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "parent"\n'); + yield* fs.writeFileString(join(child, "supabase"), "not a directory\n"); + + const paths = yield* findCliProjectPaths(child); + + expect(paths?.projectRoot).toBe(cwd); + expect(paths?.configPath).toBe(join(cwd, "supabase", "config.toml")); + }), + )); + + test("loads env from the discovered supabase directory with the right precedence", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const repoRoot = join(cwd, "repo"); + const packageRoot = join(repoRoot, "apps", "web"); + const nestedCwd = join(packageRoot, "src"); + + yield* fs.makeDirectory(join(repoRoot, "supabase"), { recursive: true }); + yield* fs.makeDirectory(join(packageRoot, "supabase"), { recursive: true }); + yield* fs.makeDirectory(nestedCwd, { recursive: true }); + yield* fs.writeFileString( + join(repoRoot, "supabase", "config.toml"), + 'project_id = "repo"\n', + ); + yield* fs.writeFileString(join(repoRoot, "supabase", ".env"), "ROOT_ONLY=repo\n"); + yield* fs.writeFileString( + join(packageRoot, "supabase", "config.toml"), + 'project_id = "web"\n', + ); + yield* fs.writeFileString( + join(packageRoot, "supabase", ".env"), + "SHARED_ONLY=from-env\nOVERRIDE_ME=from-env\n", + ); + yield* fs.writeFileString( + join(packageRoot, "supabase", ".env.local"), + "LOCAL_ONLY=from-local\nOVERRIDE_ME=from-local\n", + ); + + const projectEnv = yield* loadCliProjectEnvironment({ cwd: nestedCwd, baseEnv: { OVERRIDE_ME: "from-ambient", AMBIENT_ONLY: "from-ambient", }, - }), - ); - - expect(projectEnv).not.toBeNull(); - expect(projectEnv?.values.SHARED_ONLY).toBe("from-env"); - expect(projectEnv?.values.LOCAL_ONLY).toBe("from-local"); - expect(projectEnv?.values.AMBIENT_ONLY).toBe("from-ambient"); - expect(projectEnv?.values.OVERRIDE_ME).toBe("from-ambient"); - expect(projectEnv?.values.ROOT_ONLY).toBeUndefined(); - expect(projectEnv?.sources.OVERRIDE_ME).toBe("ambient"); - expect(projectEnv?.loadedPaths).toEqual([ - join(packageRoot, "supabase", ".env"), - join(packageRoot, "supabase", ".env.local"), - ]); - - const fromBun = await loadCliProjectEnvironmentFromBun({ - cwd: nestedCwd, - baseEnv: { - OVERRIDE_ME: "from-ambient", - }, - }); - - expect(fromBun?.paths.projectRoot).toBe(packageRoot); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("parses a multiline double-quoted .env value (godotenv/Go parity)", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); - await writeFile( - join(cwd, "supabase", ".env"), - [ - 'PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----', - "MIIEpAIBAAKCAQEA1c7+9z5Pad7OejecsQ0bu3aumga", - '-----END RSA PRIVATE KEY-----"', - "OTHER=value", - "", - ].join("\n"), - ); - - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd })); - - expect(projectEnv).not.toBeNull(); - expect(projectEnv?.values.PRIVATE_KEY).toBe( - [ - "-----BEGIN RSA PRIVATE KEY-----", - "MIIEpAIBAAKCAQEA1c7+9z5Pad7OejecsQ0bu3aumga", - "-----END RSA PRIVATE KEY-----", - ].join("\n"), - ); - expect(projectEnv?.values.OTHER).toBe("value"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("parses a multiline single-quoted .env value followed by a trailing comment", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); - await writeFile( - join(cwd, "supabase", ".env"), - ["MULTI='line one", "line two' # trailing comment", "AFTER=ok", ""].join("\n"), - ); - - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd })); - - expect(projectEnv).not.toBeNull(); - expect(projectEnv?.values.MULTI).toBe(["line one", "line two"].join("\n")); - expect(projectEnv?.values.AFTER).toBe("ok"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("still fails a genuinely malformed .env line (not a multiline quote)", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); - await writeFile(join(cwd, "supabase", ".env"), "!!!not-a-valid-line\n"); - - await expect(runConfigEffect(loadCliProjectEnvironment({ cwd }))).rejects.toBeInstanceOf( - CliProjectEnvParseError, - ); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("skipEnvLocal ignores .env.local entirely, matching Go's SUPABASE_ENV=test gate", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); - await writeFile(join(cwd, "supabase", ".env"), "FROM_ENV=1\n"); - // Malformed — would normally throw CliProjectEnvParseError. - await writeFile(join(cwd, "supabase", ".env.local"), "!!!not-a-valid-line\n"); - - const projectEnv = await runConfigEffect( - loadCliProjectEnvironment({ cwd, skipEnvLocal: true }), - ); - - expect(projectEnv).not.toBeNull(); - expect(projectEnv?.values.FROM_ENV).toBe("1"); - expect(projectEnv?.loadedPaths).toEqual([join(cwd, "supabase", ".env")]); - - await expect(runConfigEffect(loadCliProjectEnvironment({ cwd }))).rejects.toBeInstanceOf( - CliProjectEnvParseError, - ); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("leaves [api].auto_expose_new_tables unset by default and round-trips an explicit value", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile(join(projectRoot, "supabase", "config.toml"), `project_id = "ref_123"\n`); - - const defaultLoaded = await runConfigEffect(loadCliConfig(projectRoot)); - expect(defaultLoaded!.config.api.auto_expose_new_tables).toBeUndefined(); - - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123"\n\n[api]\nauto_expose_new_tables = false\n`, - ); - const explicitFalse = await runConfigEffect(loadCliConfig(projectRoot)); - expect(explicitFalse!.config.api.auto_expose_new_tables).toBe(false); - - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123"\n\n[api]\nauto_expose_new_tables = true\n`, - ); - const explicitTrue = await runConfigEffect(loadCliConfig(projectRoot)); - expect(explicitTrue!.config.api.auto_expose_new_tables).toBe(true); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loads raw config without resolving explicit env() references", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + }); + + expect(projectEnv).not.toBeNull(); + expect(projectEnv?.values.SHARED_ONLY).toBe("from-env"); + expect(projectEnv?.values.LOCAL_ONLY).toBe("from-local"); + expect(projectEnv?.values.AMBIENT_ONLY).toBe("from-ambient"); + expect(projectEnv?.values.OVERRIDE_ME).toBe("from-ambient"); + expect(projectEnv?.values.ROOT_ONLY).toBeUndefined(); + expect(projectEnv?.sources.OVERRIDE_ME).toBe("ambient"); + expect(projectEnv?.loadedPaths).toEqual([ + join(packageRoot, "supabase", ".env"), + join(packageRoot, "supabase", ".env.local"), + ]); + + const fromBun = yield* Effect.promise(() => + loadCliProjectEnvironmentFromBun({ + cwd: nestedCwd, + baseEnv: { + OVERRIDE_ME: "from-ambient", + }, + }), + ); + + expect(fromBun?.paths.projectRoot).toBe(packageRoot); + }), + )); + + test("parses a multiline double-quoted .env value (godotenv/Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + [ + 'PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----', + "MIIEpAIBAAKCAQEA1c7+9z5Pad7OejecsQ0bu3aumga", + '-----END RSA PRIVATE KEY-----"', + "OTHER=value", + "", + ].join("\n"), + ); + + const projectEnv = yield* loadCliProjectEnvironment({ cwd }); + + expect(projectEnv).not.toBeNull(); + expect(projectEnv?.values.PRIVATE_KEY).toBe( + [ + "-----BEGIN RSA PRIVATE KEY-----", + "MIIEpAIBAAKCAQEA1c7+9z5Pad7OejecsQ0bu3aumga", + "-----END RSA PRIVATE KEY-----", + ].join("\n"), + ); + expect(projectEnv?.values.OTHER).toBe("value"); + }), + )); + + test("parses a multiline single-quoted .env value followed by a trailing comment", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); + yield* fs.writeFileString( + join(cwd, "supabase", ".env"), + ["MULTI='line one", "line two' # trailing comment", "AFTER=ok", ""].join("\n"), + ); + + const projectEnv = yield* loadCliProjectEnvironment({ cwd }); + + expect(projectEnv).not.toBeNull(); + expect(projectEnv?.values.MULTI).toBe(["line one", "line two"].join("\n")); + expect(projectEnv?.values.AFTER).toBe("ok"); + }), + )); + + test("still fails a genuinely malformed .env line (not a multiline quote)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); + yield* fs.writeFileString(join(cwd, "supabase", ".env"), "!!!not-a-valid-line\n"); + + expect(yield* Effect.flip(loadCliProjectEnvironment({ cwd }))).toBeInstanceOf( + CliProjectEnvParseError, + ); + }), + )); + + test("skipEnvLocal ignores .env.local entirely, matching Go's SUPABASE_ENV=test gate", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "ref_123"\n'); + yield* fs.writeFileString(join(cwd, "supabase", ".env"), "FROM_ENV=1\n"); + // Malformed — would normally throw CliProjectEnvParseError. + yield* fs.writeFileString(join(cwd, "supabase", ".env.local"), "!!!not-a-valid-line\n"); + + const projectEnv = yield* loadCliProjectEnvironment({ cwd, skipEnvLocal: true }); + + expect(projectEnv).not.toBeNull(); + expect(projectEnv?.values.FROM_ENV).toBe("1"); + expect(projectEnv?.loadedPaths).toEqual([join(cwd, "supabase", ".env")]); + + expect(yield* Effect.flip(loadCliProjectEnvironment({ cwd }))).toBeInstanceOf( + CliProjectEnvParseError, + ); + }), + )); + + test("leaves [api].auto_expose_new_tables unset by default and round-trips an explicit value", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123"\n`, + ); + + const defaultLoaded = yield* loadCliConfig(projectRoot); + expect(defaultLoaded!.config.api.auto_expose_new_tables).toBeUndefined(); + + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123"\n\n[api]\nauto_expose_new_tables = false\n`, + ); + const explicitFalse = yield* loadCliConfig(projectRoot); + expect(explicitFalse!.config.api.auto_expose_new_tables).toBe(false); + + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123"\n\n[api]\nauto_expose_new_tables = true\n`, + ); + const explicitTrue = yield* loadCliConfig(projectRoot); + expect(explicitTrue!.config.api.auto_expose_new_tables).toBe(true); + }), + )); + + test("loads raw config without resolving explicit env() references", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(AUTH_JWT_SECRET)" @@ -320,61 +343,68 @@ jwt_secret = "env(AUTH_JWT_SECRET)" enabled = false auth_token = "env(TWILIO_AUTH_TOKEN)" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); - - expect(loaded!.config.auth.jwt_secret).toBe("env(AUTH_JWT_SECRET)"); - expect(loaded!.config.auth.sms.twilio.auth_token).toBe("env(TWILIO_AUTH_TOKEN)"); - expect(projectEnv?.values.AUTH_JWT_SECRET).toBeUndefined(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigValue resolves explicit env() and redacts secret leaves", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); + + expect(loaded!.config.auth.jwt_secret).toBe("env(AUTH_JWT_SECRET)"); + expect(loaded!.config.auth.sms.twilio.auth_token).toBe("env(TWILIO_AUTH_TOKEN)"); + expect(projectEnv?.values.AUTH_JWT_SECRET).toBeUndefined(); + }), + )); + + test("resolveCliConfigValue resolves explicit env() and redacts secret leaves", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(AUTH_JWT_SECRET)" `, - ); - await writeFile(join(projectRoot, "supabase", ".env"), "AUTH_JWT_SECRET=super-secret\n"); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); - - const resolved = await runConfigEffect( - resolveCliConfigValue(loaded!.config.auth.jwt_secret, projectEnv!, "auth.jwt_secret"), - ); - - expect(Redacted.isRedacted(resolved)).toBe(true); - if (!Redacted.isRedacted(resolved)) { - throw new Error("Expected auth.jwt_secret to be redacted."); - } - expect(Redacted.value(resolved)).toBe("super-secret"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigSubtree resolves nested records and remotes lazily", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + yield* fs.writeFileString( + join(projectRoot, "supabase", ".env"), + "AUTH_JWT_SECRET=super-secret\n", + ); + + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); + + const resolved = yield* resolveCliConfigValue( + loaded!.config.auth.jwt_secret, + projectEnv!, + "auth.jwt_secret", + ); + + expect(Redacted.isRedacted(resolved)).toBe(true); + if (!Redacted.isRedacted(resolved)) { + throw new Error("Expected auth.jwt_secret to be redacted."); + } + expect(Redacted.value(resolved)).toBe("super-secret"); + }), + )); + + test("resolveCliConfigSubtree resolves nested records and remotes lazily", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [edge_runtime.secrets] api_key = "env(EDGE_API_KEY)" @@ -385,224 +415,238 @@ project_id = "previewrefaaaaaaaaaa" [remotes.preview.auth] jwt_secret = "env(PREVIEW_JWT_SECRET)" `, - ); - await writeFile( - join(projectRoot, "supabase", ".env"), - "EDGE_API_KEY=edge-secret\nPREVIEW_JWT_SECRET=preview-secret\n", - ); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); - - const edgeRuntime = await runConfigEffect( - resolveCliConfigSubtree(loaded!.config.edge_runtime, projectEnv!, "edge_runtime"), - ); - const previewRemote = await runConfigEffect( - resolveCliConfigSubtree(loaded!.config.remotes.preview, projectEnv!, "remotes.preview"), - ); - - const edgeSecret = edgeRuntime.secrets?.api_key; - expect(Redacted.isRedacted(edgeSecret)).toBe(true); - if (!Redacted.isRedacted(edgeSecret)) { - throw new Error("Expected edge_runtime.secrets.api_key to be redacted."); - } - expect(Redacted.value(edgeSecret)).toBe("edge-secret"); - - const previewSecret = previewRemote!.auth.jwt_secret; - expect(Redacted.isRedacted(previewSecret)).toBe(true); - if (!Redacted.isRedacted(previewSecret)) { - throw new Error("Expected remotes.preview.auth.jwt_secret to be redacted."); - } - expect(Redacted.value(previewSecret)).toBe("preview-secret"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigValue preserves env() literal when the env var is missing (Go parity)", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + yield* fs.writeFileString( + join(projectRoot, "supabase", ".env"), + "EDGE_API_KEY=edge-secret\nPREVIEW_JWT_SECRET=preview-secret\n", + ); + + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); + + const edgeRuntime = yield* resolveCliConfigSubtree( + loaded!.config.edge_runtime, + projectEnv!, + "edge_runtime", + ); + const previewRemote = yield* resolveCliConfigSubtree( + loaded!.config.remotes.preview, + projectEnv!, + "remotes.preview", + ); + + const edgeSecret = edgeRuntime.secrets?.api_key; + expect(Redacted.isRedacted(edgeSecret)).toBe(true); + if (!Redacted.isRedacted(edgeSecret)) { + throw new Error("Expected edge_runtime.secrets.api_key to be redacted."); + } + expect(Redacted.value(edgeSecret)).toBe("edge-secret"); + + const previewSecret = previewRemote!.auth.jwt_secret; + expect(Redacted.isRedacted(previewSecret)).toBe(true); + if (!Redacted.isRedacted(previewSecret)) { + throw new Error("Expected remotes.preview.auth.jwt_secret to be redacted."); + } + expect(Redacted.value(previewSecret)).toBe("preview-secret"); + }), + )); + + test("resolveCliConfigValue preserves env() literal when the env var is missing (Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(MISSING_SECRET)" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); - - const resolved = await runConfigEffect( - resolveCliConfigValue(loaded!.config.auth.jwt_secret, projectEnv!, "auth.jwt_secret"), - ); - - expect(Redacted.isRedacted(resolved)).toBe(false); - expect(resolved).toBe("env(MISSING_SECRET)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); - test("resolveCliConfigValue preserves env() literal when the env var is present but empty (Go parity)", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + const resolved = yield* resolveCliConfigValue( + loaded!.config.auth.jwt_secret, + projectEnv!, + "auth.jwt_secret", + ); + + expect(Redacted.isRedacted(resolved)).toBe(false); + expect(resolved).toBe("env(MISSING_SECRET)"); + }), + )); + + test("resolveCliConfigValue preserves env() literal when the env var is present but empty (Go parity)", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [edge_runtime.secrets] foo = "env(EMPTY_SECRET)" `, - ); - await writeFile(join(projectRoot, "supabase", ".env"), "EMPTY_SECRET=\n"); + ); + yield* fs.writeFileString(join(projectRoot, "supabase", ".env"), "EMPTY_SECRET=\n"); - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); - const resolved = await runConfigEffect( - resolveCliConfigValue( + const resolved = yield* resolveCliConfigValue( loaded!.config.edge_runtime.secrets!.foo, projectEnv!, "edge_runtime.secrets.foo", - ), - ); - - expect(Redacted.isRedacted(resolved)).toBe(false); - expect(resolved).toBe("env(EMPTY_SECRET)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigSubtree preserves env() literals nested inside the selected subtree", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + + expect(Redacted.isRedacted(resolved)).toBe(false); + expect(resolved).toBe("env(EMPTY_SECRET)"); + }), + )); + + test("resolveCliConfigSubtree preserves env() literals nested inside the selected subtree", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth.sms.twilio] enabled = false auth_token = "env(MISSING_SECRET)" `, - ); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); + ); - const resolved = await runConfigEffect( - resolveCliConfigSubtree(loaded!.config.auth.sms.twilio, projectEnv!, "auth.sms.twilio"), - ); + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); - expect(resolved.auth_token).toBe("env(MISSING_SECRET)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("raw config validation still enforces enabled feature requirements", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + const resolved = yield* resolveCliConfigSubtree( + loaded!.config.auth.sms.twilio, + projectEnv!, + "auth.sms.twilio", + ); + + expect(resolved.auth_token).toBe("env(MISSING_SECRET)"); + }), + )); + + test("raw config validation still enforces enabled feature requirements", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth.sms.twilio] enabled = true account_sid = "AC123" `, - ); - - await expect(runConfigEffect(loadCliConfig(projectRoot))).rejects.toBeInstanceOf( - CliConfigParseError, - ); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigValue does not resolve a lowercase-named env() reference by default", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + + expect(yield* Effect.flip(loadCliConfig(projectRoot))).toBeInstanceOf(CliConfigParseError); + }), + )); + + test("resolveCliConfigValue does not resolve a lowercase-named env() reference by default", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(lowercase_secret)" `, - ); - await writeFile(join(projectRoot, "supabase", ".env"), "lowercase_secret=super-secret\n"); - - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); - - const resolved = await runConfigEffect( - resolveCliConfigValue(loaded!.config.auth.jwt_secret, projectEnv!, "auth.jwt_secret"), - ); - - expect(Redacted.isRedacted(resolved)).toBe(true); - if (!Redacted.isRedacted(resolved)) { - throw new Error("Expected auth.jwt_secret to be redacted."); - } - expect(Redacted.value(resolved)).toBe("env(lowercase_secret)"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("resolveCliConfigValue resolves a lowercase-named env() reference when goViperCompat is true", async () => { - const cwd = makeTempProject(); - const projectRoot = join(cwd, "repo"); - - try { - await mkdir(join(projectRoot, "supabase"), { recursive: true }); - await writeFile( - join(projectRoot, "supabase", "config.toml"), - `project_id = "ref_123" + ); + yield* fs.writeFileString( + join(projectRoot, "supabase", ".env"), + "lowercase_secret=super-secret\n", + ); + + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); + + const resolved = yield* resolveCliConfigValue( + loaded!.config.auth.jwt_secret, + projectEnv!, + "auth.jwt_secret", + ); + + expect(Redacted.isRedacted(resolved)).toBe(true); + if (!Redacted.isRedacted(resolved)) { + throw new Error("Expected auth.jwt_secret to be redacted."); + } + expect(Redacted.value(resolved)).toBe("env(lowercase_secret)"); + }), + )); + + test("resolveCliConfigValue resolves a lowercase-named env() reference when goViperCompat is true", () => + runConfigEffect( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const projectRoot = join(cwd, "repo"); + + yield* fs.makeDirectory(join(projectRoot, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(projectRoot, "supabase", "config.toml"), + `project_id = "ref_123" [auth] jwt_secret = "env(lowercase_secret)" `, - ); - await writeFile(join(projectRoot, "supabase", ".env"), "lowercase_secret=super-secret\n"); + ); + yield* fs.writeFileString( + join(projectRoot, "supabase", ".env"), + "lowercase_secret=super-secret\n", + ); - const loaded = await runConfigEffect(loadCliConfig(projectRoot)); - const projectEnv = await runConfigEffect(loadCliProjectEnvironment({ cwd: projectRoot })); + const loaded = yield* loadCliConfig(projectRoot); + const projectEnv = yield* loadCliProjectEnvironment({ cwd: projectRoot }); - const resolved = await runConfigEffect( - resolveCliConfigValueInternal( + const resolved = yield* resolveCliConfigValueInternal( loaded!.config.auth.jwt_secret, projectEnv!, "auth.jwt_secret", { goViperCompat: true }, - ), - ); - - expect(Redacted.isRedacted(resolved)).toBe(true); - if (!Redacted.isRedacted(resolved)) { - throw new Error("Expected auth.jwt_secret to be redacted."); - } - expect(Redacted.value(resolved)).toBe("super-secret"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + ); + + expect(Redacted.isRedacted(resolved)).toBe(true); + if (!Redacted.isRedacted(resolved)) { + throw new Error("Expected auth.jwt_secret to be redacted."); + } + expect(Redacted.value(resolved)).toBe("super-secret"); + }), + )); }); diff --git a/packages/config/src/promise-facade.stdin.unit.test.ts b/packages/config/src/promise-facade.stdin.unit.test.ts index 2ce9e0cb54..93aff90959 100644 --- a/packages/config/src/promise-facade.stdin.unit.test.ts +++ b/packages/config/src/promise-facade.stdin.unit.test.ts @@ -1,10 +1,6 @@ import { describe, expect, test } from "vitest"; import { BunFileSystem, BunPath } from "@effect/platform-bun"; -import { mkdtempSync } from "node:fs"; -import { rm } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { Context, Effect, Layer, Option, Terminal } from "effect"; +import { Context, Effect, FileSystem, Layer, Option, Terminal } from "effect"; import { findCliProjectRoot } from "./bun.ts"; // `bun.ts`'s facade only needs `FileSystem | Path`, so a facade call must never attach the @@ -13,29 +9,32 @@ import { findCliProjectRoot } from "./bun.ts"; // process, since the facade's `ManagedRuntime` is a lazy, module-level singleton — so this // file must make exactly one facade call, and no other test here may call into `bun.ts` before or after it. describe("promise-facade stdin-leak regression (CLI-2231)", () => { - test("the first facade call in this file does not attach a process.stdin 'end' listener", async () => { - const cwd = mkdtempSync(join(tmpdir(), "supabase-promise-facade-stdin-")); - const before = process.stdin.listenerCount("end"); + test("the first facade call in this file does not attach a process.stdin 'end' listener", () => + Effect.runPromise( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const cwd = yield* fs.makeTempDirectoryScoped({ prefix: "supabase-promise-facade-stdin-" }); + const before = process.stdin.listenerCount("end"); - try { - await findCliProjectRoot(cwd); + yield* Effect.promise(() => findCliProjectRoot(cwd)); - expect(process.stdin.listenerCount("end")).toBe(before); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + expect(process.stdin.listenerCount("end")).toBe(before); + }).pipe(Effect.scoped, Effect.provide(BunFileSystem.layer)), + )); }); describe("bun.ts's platform layer excludes Terminal", () => { // Builds the same layer `bun.ts` provides its `ManagedRuntime` independently, so it // doesn't count as a second facade call, and asserts there's no `Terminal` service at all // rather than just that using one doesn't touch stdin today. - test("Layer.mergeAll(BunFileSystem.layer, BunPath.layer) builds a context with no Terminal service", async () => { - const context = await Effect.runPromise( - Layer.build(Layer.mergeAll(BunFileSystem.layer, BunPath.layer)).pipe(Effect.scoped), - ); + test("Layer.mergeAll(BunFileSystem.layer, BunPath.layer) builds a context with no Terminal service", () => + Effect.runPromise( + Effect.gen(function* () { + const context = yield* Layer.build(Layer.mergeAll(BunFileSystem.layer, BunPath.layer)).pipe( + Effect.scoped, + ); - expect(Option.isNone(Context.getOption(context, Terminal.Terminal))).toBe(true); - }); + expect(Option.isNone(Context.getOption(context, Terminal.Terminal))).toBe(true); + }), + )); }); diff --git a/packages/config/src/promise-facade.ts b/packages/config/src/promise-facade.ts index cd45d00b1f..dc2d780aa3 100644 --- a/packages/config/src/promise-facade.ts +++ b/packages/config/src/promise-facade.ts @@ -1,5 +1,5 @@ import type { FileSystem, Path } from "effect"; -import { Layer, ManagedRuntime } from "effect"; +import { Effect, Layer, ManagedRuntime } from "effect"; import type { LoadedCliConfig, LoadCliConfigOptions, @@ -59,19 +59,22 @@ export function makeCliConfigIo( return runtime; } + /* Never throws synchronously: a throw while building the program rejects like any failure. */ + function run<A, E>( + program: () => Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | CliConfigStore>, + ) { + return getRuntime().runPromise(Effect.suspend(program)); + } + return { - loadCliConfig: async (cwd, options) => - getRuntime().runPromise(CliConfigStore.use((store) => store.load(cwd, options))), - findCliProjectRoot: async (cwd) => getRuntime().runPromise(findCliProjectRoot(cwd)), - findCliProjectPaths: async (cwd) => getRuntime().runPromise(findCliProjectPaths(cwd)), - loadCliConfigFile: async (path) => - getRuntime().runPromise(CliConfigStore.use((store) => store.loadFile(path))), - loadCliProjectEnvironment: async (options) => - getRuntime().runPromise( - loadCliProjectEnvironment({ ...options, baseEnv: options.baseEnv ?? process.env }), - ), - saveCliConfig: async (options) => - getRuntime().runPromise(CliConfigStore.use((store) => store.save(options))), - inferFunctionsManifest: async (cwd) => getRuntime().runPromise(inferFunctionsManifest({ cwd })), + loadCliConfig: (cwd, options) => + run(() => CliConfigStore.use((store) => store.load(cwd, options))), + findCliProjectRoot: (cwd) => run(() => findCliProjectRoot(cwd)), + findCliProjectPaths: (cwd) => run(() => findCliProjectPaths(cwd)), + loadCliConfigFile: (path) => run(() => CliConfigStore.use((store) => store.loadFile(path))), + loadCliProjectEnvironment: (options) => + run(() => loadCliProjectEnvironment({ ...options, baseEnv: options.baseEnv ?? process.env })), + saveCliConfig: (options) => run(() => CliConfigStore.use((store) => store.save(options))), + inferFunctionsManifest: (cwd) => run(() => inferFunctionsManifest({ cwd })), }; } diff --git a/packages/config/src/promise-facade.unit.test.ts b/packages/config/src/promise-facade.unit.test.ts index f3648cf802..3fe0ddc226 100644 --- a/packages/config/src/promise-facade.unit.test.ts +++ b/packages/config/src/promise-facade.unit.test.ts @@ -1,10 +1,6 @@ import { describe, expect, test } from "vitest"; import { BunFileSystem, BunPath } from "@effect/platform-bun"; -import { mkdtempSync } from "node:fs"; -import { mkdir, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { Effect, Layer, Schema } from "effect"; +import { Effect, FileSystem, Layer, Path, Schema, type Scope } from "effect"; import { CliConfigSchema } from "./base.ts"; import { CliConfigParseError } from "./errors.ts"; import * as bunFacade from "./bun.ts"; @@ -27,168 +23,199 @@ const decodeCliConfig = Schema.decodeUnknownSync(CliConfigSchema); // Under the OS temp dir, never a path under this repo — otherwise ancestor-search could // walk up into this repo's own `apps/cli/docs/supabase` fixture. -function makeTempProject(): string { - return mkdtempSync(join(tmpdir(), "supabase-promise-facade-")); +function makeTempProject() { + return FileSystem.FileSystem.use((fs) => + fs.makeTempDirectoryScoped({ prefix: "supabase-promise-facade-" }), + ); } -describe("promise-facade via the Bun entrypoint", () => { - test("loadCliConfig resolves null when no Supabase project exists in the tree", async () => { - const cwd = makeTempProject(); - - try { - await expect(loadCliConfig(cwd)).resolves.toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loadCliConfig loads and decodes a real supabase/config.toml", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "facade-loaded-ref"\n'); - - const loaded = await loadCliConfig(cwd); - - expect(loaded?.config.project_id).toBe("facade-loaded-ref"); - expect(loaded?.config.db.major_version).toBe(17); - expect(loaded?.format).toBe("toml"); - expect(loaded?.path).toBe(join(cwd, "supabase", "config.toml")); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("saveCliConfig then loadCliConfigFile roundtrips the same effective config", async () => { - const cwd = makeTempProject(); - - try { - const original = decodeCliConfig({ - project_id: "facade-roundtrip-ref", - db: { pooler: { enabled: true } }, - }); - - const saved = await saveCliConfig({ cwd, config: original }); - expect(saved.format).toBe("json"); - - const loaded = await loadCliConfigFile(saved.path); - - expect(loaded.config).toEqual(original); - expect(loaded.path).toBe(saved.path); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("findCliProjectRoot and findCliProjectPaths resolve from a nested cwd inside a temp project", async () => { - const cwd = makeTempProject(); - const nested = join(cwd, "apps", "web", "src", "components"); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await mkdir(nested, { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "nested-ref"\n'); - - const root = await findCliProjectRoot(nested); - const paths = await findCliProjectPaths(nested); - - expect(root).toBe(cwd); - expect(paths).toEqual({ - projectRoot: cwd, - supabaseDir: join(cwd, "supabase"), - configPath: join(cwd, "supabase", "config.toml"), - envPath: join(cwd, "supabase", ".env"), - envLocalPath: join(cwd, "supabase", ".env.local"), - }); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("findCliProjectRoot and findCliProjectPaths resolve to null when there is no project", async () => { - const cwd = makeTempProject(); - - try { - await expect(findCliProjectRoot(cwd)).resolves.toBeNull(); - await expect(findCliProjectPaths(cwd)).resolves.toBeNull(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loadCliProjectEnvironment reads supabase/.env layered under an explicit baseEnv", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "env-ref"\n'); - await writeFile(join(cwd, "supabase", ".env"), "GREETING=hello-from-dotenv\n"); - - // Explicit empty baseEnv, so no ambient process.env variable can satisfy this assertion. - const projectEnv = await loadCliProjectEnvironment({ cwd, baseEnv: {} }); - - expect(projectEnv?.values.GREETING).toBe("hello-from-dotenv"); - expect(projectEnv?.sources.GREETING).toBe(".env"); - expect(projectEnv?.loadedPaths).toEqual([join(cwd, "supabase", ".env")]); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loadCliProjectEnvironment honors an explicit baseEnv instead of silently defaulting to process.env", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "env-ref"\n'); - await writeFile(join(cwd, "supabase", ".env"), "GREETING=from-dotenv\n"); - - const projectEnv = await loadCliProjectEnvironment({ - cwd, - baseEnv: { GREETING: "from-explicit-base-env" }, - }); - - expect(projectEnv?.values.GREETING).toBe("from-explicit-base-env"); - expect(projectEnv?.sources.GREETING).toBe("ambient"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("inferFunctionsManifest resolves an empty manifest when no functions directory exists", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "functions-ref"\n'); - - await expect(inferFunctionsManifest(cwd)).resolves.toEqual({}); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("two sequential facade calls both succeed, exercising the shared lazy runtime", async () => { - const cwdA = makeTempProject(); - const cwdB = makeTempProject(); - - try { - await mkdir(join(cwdA, "supabase"), { recursive: true }); - await writeFile(join(cwdA, "supabase", "config.toml"), 'project_id = "first-ref"\n'); - await mkdir(join(cwdB, "supabase"), { recursive: true }); - await writeFile(join(cwdB, "supabase", "config.toml"), 'project_id = "second-ref"\n'); - - const first = await loadCliConfig(cwdA); - const second = await loadCliConfig(cwdB); +function runFacadeTest<A, E>( + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, +): Promise<A> { + return Effect.runPromise( + effect.pipe(Effect.scoped, Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layer))), + ); +} - expect(first?.config.project_id).toBe("first-ref"); - expect(second?.config.project_id).toBe("second-ref"); - } finally { - await rm(cwdA, { recursive: true, force: true }); - await rm(cwdB, { recursive: true, force: true }); - } - }); +describe("promise-facade via the Bun entrypoint", () => { + test("loadCliConfig resolves null when no Supabase project exists in the tree", () => + runFacadeTest( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + + yield* Effect.promise(() => expect(loadCliConfig(cwd)).resolves.toBeNull()); + }), + )); + + test("loadCliConfig loads and decodes a real supabase/config.toml", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + 'project_id = "facade-loaded-ref"\n', + ); + + const loaded = yield* Effect.promise(() => loadCliConfig(cwd)); + + expect(loaded?.config.project_id).toBe("facade-loaded-ref"); + expect(loaded?.config.db.major_version).toBe(17); + expect(loaded?.format).toBe("toml"); + expect(loaded?.path).toBe(join(cwd, "supabase", "config.toml")); + }), + )); + + test("saveCliConfig then loadCliConfigFile roundtrips the same effective config", () => + runFacadeTest( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + + const original = decodeCliConfig({ + project_id: "facade-roundtrip-ref", + db: { pooler: { enabled: true } }, + }); + + const saved = yield* Effect.promise(() => saveCliConfig({ cwd, config: original })); + expect(saved.format).toBe("json"); + + const loaded = yield* Effect.promise(() => loadCliConfigFile(saved.path)); + + expect(loaded.config).toEqual(original); + expect(loaded.path).toBe(saved.path); + }), + )); + + test("findCliProjectRoot and findCliProjectPaths resolve from a nested cwd inside a temp project", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const nested = join(cwd, "apps", "web", "src", "components"); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.makeDirectory(nested, { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + 'project_id = "nested-ref"\n', + ); + + const root = yield* Effect.promise(() => findCliProjectRoot(nested)); + const paths = yield* Effect.promise(() => findCliProjectPaths(nested)); + + expect(root).toBe(cwd); + expect(paths).toEqual({ + projectRoot: cwd, + supabaseDir: join(cwd, "supabase"), + configPath: join(cwd, "supabase", "config.toml"), + envPath: join(cwd, "supabase", ".env"), + envLocalPath: join(cwd, "supabase", ".env.local"), + }); + }), + )); + + test("findCliProjectRoot and findCliProjectPaths resolve to null when there is no project", () => + runFacadeTest( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + + yield* Effect.promise(() => expect(findCliProjectRoot(cwd)).resolves.toBeNull()); + yield* Effect.promise(() => expect(findCliProjectPaths(cwd)).resolves.toBeNull()); + }), + )); + + test("loadCliProjectEnvironment reads supabase/.env layered under an explicit baseEnv", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "env-ref"\n'); + yield* fs.writeFileString(join(cwd, "supabase", ".env"), "GREETING=hello-from-dotenv\n"); + + // Explicit empty baseEnv, so no ambient process.env variable can satisfy this assertion. + const projectEnv = yield* Effect.promise(() => + loadCliProjectEnvironment({ cwd, baseEnv: {} }), + ); + + expect(projectEnv?.values.GREETING).toBe("hello-from-dotenv"); + expect(projectEnv?.sources.GREETING).toBe(".env"); + expect(projectEnv?.loadedPaths).toEqual([join(cwd, "supabase", ".env")]); + }), + )); + + test("loadCliProjectEnvironment honors an explicit baseEnv instead of silently defaulting to process.env", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(join(cwd, "supabase", "config.toml"), 'project_id = "env-ref"\n'); + yield* fs.writeFileString(join(cwd, "supabase", ".env"), "GREETING=from-dotenv\n"); + + const projectEnv = yield* Effect.promise(() => + loadCliProjectEnvironment({ + cwd, + baseEnv: { GREETING: "from-explicit-base-env" }, + }), + ); + + expect(projectEnv?.values.GREETING).toBe("from-explicit-base-env"); + expect(projectEnv?.sources.GREETING).toBe("ambient"); + }), + )); + + test("inferFunctionsManifest resolves an empty manifest when no functions directory exists", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + 'project_id = "functions-ref"\n', + ); + + yield* Effect.promise(() => expect(inferFunctionsManifest(cwd)).resolves.toEqual({})); + }), + )); + + test("two sequential facade calls both succeed, exercising the shared lazy runtime", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwdA = yield* makeTempProject(); + const cwdB = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwdA, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwdA, "supabase", "config.toml"), + 'project_id = "first-ref"\n', + ); + yield* fs.makeDirectory(join(cwdB, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwdB, "supabase", "config.toml"), + 'project_id = "second-ref"\n', + ); + + const first = yield* Effect.promise(() => loadCliConfig(cwdA)); + const second = yield* Effect.promise(() => loadCliConfig(cwdB)); + + expect(first?.config.project_id).toBe("first-ref"); + expect(second?.config.project_id).toBe("second-ref"); + }), + )); }); const expectedFacadeFunctionNames = [ @@ -244,86 +271,90 @@ describe("./io is a superset of src/index.ts", () => { describe("io-browser.ts stays side-effect-free", () => { // Dynamic import so a regression back to a top-level throw fails this test // specifically, instead of crashing the whole file at module-load time. - test("importing the module does not throw", async () => { - await expect(import("./io-browser.ts")).resolves.toBeDefined(); - }); + test("importing the module does not throw", () => + expect(import("./io-browser.ts")).resolves.toBeDefined()); - test("calling loadCliConfig rejects with the curated browser-unavailable message", async () => { - await expect(ioBrowserFacade.loadCliConfig("/irrelevant")).rejects.toThrow( + test("calling loadCliConfig rejects with the curated browser-unavailable message", () => + expect(ioBrowserFacade.loadCliConfig("/irrelevant")).rejects.toThrow( '@supabase/config/io is not available in browser bundles; import the pure surface from "@supabase/config" instead.', - ); - }); + )); }); describe("promise-facade singleton runtime", () => { - test("builds the underlying ManagedRuntime exactly once across multiple facade calls", async () => { - const cwd = makeTempProject(); - let builds = 0; - - try { - const countingLayer = Layer.mergeAll( - BunFileSystem.layer, - BunPath.layer, - Layer.effectDiscard( - Effect.sync(() => { - builds += 1; - }), - ), - ); - const io = makeCliConfigIo(countingLayer); - - await io.findCliProjectRoot(cwd); - await io.findCliProjectRoot(cwd); - - expect(builds).toBe(1); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("builds the underlying ManagedRuntime exactly once across multiple facade calls", () => + runFacadeTest( + Effect.gen(function* () { + const cwd = yield* makeTempProject(); + let builds = 0; + + const countingLayer = Layer.mergeAll( + BunFileSystem.layer, + BunPath.layer, + Layer.effectDiscard( + Effect.sync(() => { + builds += 1; + }), + ), + ); + const io = makeCliConfigIo(countingLayer); + + yield* Effect.promise(() => io.findCliProjectRoot(cwd)); + yield* Effect.promise(() => io.findCliProjectRoot(cwd)); + + expect(builds).toBe(1); + }), + )); }); describe("promise-facade via the Node entrypoint", () => { - test("loadCliConfig loads and decodes a real supabase/config.toml", async () => { - const cwd = makeTempProject(); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(join(cwd, "supabase", "config.toml"), 'project_id = "node-facade-ref"\n'); - - const loaded = await nodeFacade.loadCliConfig(cwd); - - expect(loaded?.config.project_id).toBe("node-facade-ref"); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("loadCliConfig loads and decodes a real supabase/config.toml", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString( + join(cwd, "supabase", "config.toml"), + 'project_id = "node-facade-ref"\n', + ); + + const loaded = yield* Effect.promise(() => nodeFacade.loadCliConfig(cwd)); + + expect(loaded?.config.project_id).toBe("node-facade-ref"); + }), + )); }); describe("promise-facade rejection shapes", () => { - test("loadCliConfigFile rejects with a CliConfigParseError for a malformed config.toml", async () => { - const cwd = makeTempProject(); - const configPath = join(cwd, "supabase", "config.toml"); - - try { - await mkdir(join(cwd, "supabase"), { recursive: true }); - await writeFile(configPath, "this is not === valid toml\n"); - - await expect(loadCliConfigFile(configPath)).rejects.toBeInstanceOf(CliConfigParseError); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); - - test("loadCliConfigFile rejects for a nonexistent path", async () => { - const cwd = makeTempProject(); - const configPath = join(cwd, "supabase", "config.toml"); - - try { - await expect(loadCliConfigFile(configPath)).rejects.toThrow(); - } finally { - await rm(cwd, { recursive: true, force: true }); - } - }); + test("loadCliConfigFile rejects with a CliConfigParseError for a malformed config.toml", () => + runFacadeTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const configPath = join(cwd, "supabase", "config.toml"); + + yield* fs.makeDirectory(join(cwd, "supabase"), { recursive: true }); + yield* fs.writeFileString(configPath, "this is not === valid toml\n"); + + yield* Effect.promise(() => + expect(loadCliConfigFile(configPath)).rejects.toBeInstanceOf(CliConfigParseError), + ); + }), + )); + + test("loadCliConfigFile rejects for a nonexistent path", () => + runFacadeTest( + Effect.gen(function* () { + const { join } = yield* Path.Path; + const cwd = yield* makeTempProject(); + const configPath = join(cwd, "supabase", "config.toml"); + + yield* Effect.promise(() => expect(loadCliConfigFile(configPath)).rejects.toThrow()); + }), + )); }); // The stdin-leak guard lives in `promise-facade.stdin.unit.test.ts`, which needs its own diff --git a/packages/config/src/validate.unit.test.ts b/packages/config/src/validate.unit.test.ts index a4e3f3e856..e277ec0c1d 100644 --- a/packages/config/src/validate.unit.test.ts +++ b/packages/config/src/validate.unit.test.ts @@ -9,79 +9,82 @@ const incompleteTwilio = { }; describe("validateCliConfig", () => { - test("accepts a valid effective config", async () => { - const config = await Effect.runPromise( - validateCliConfig({ - auth: { - sms: { - twilio: { - enabled: true, - account_sid: "AC123", - message_service_sid: "MG123", - auth_token: "token", + test("accepts a valid effective config", () => + Effect.runPromise( + Effect.gen(function* () { + const config = yield* validateCliConfig({ + auth: { + sms: { + twilio: { + enabled: true, + account_sid: "AC123", + message_service_sid: "MG123", + auth_token: "token", + }, }, }, - }, - }), - ); + }); - expect(config.auth.sms.twilio.enabled).toBe(true); - expect(config.auth.sms.twilio.account_sid).toBe("AC123"); - }); + expect(config.auth.sms.twilio.enabled).toBe(true); + expect(config.auth.sms.twilio.account_sid).toBe("AC123"); + }), + )); - test("rejects an incomplete enabled provider in the effective config", async () => { - const loaded = await Effect.runPromise( - validateCliConfig({ - auth: { - sms: { - twilio: { - enabled: true, - account_sid: "AC123", - message_service_sid: "MG123", - auth_token: "token", + test("rejects an incomplete enabled provider in the effective config", () => + Effect.runPromise( + Effect.gen(function* () { + const loaded = yield* validateCliConfig({ + auth: { + sms: { + twilio: { + enabled: true, + account_sid: "AC123", + message_service_sid: "MG123", + auth_token: "token", + }, + }, + }, + }); + const overridden = { + ...loaded, + auth: { + ...loaded.auth, + sms: { + ...loaded.auth.sms, + twilio: { ...loaded.auth.sms.twilio, ...incompleteTwilio }, }, }, - }, + }; + const exit = yield* Effect.exit(validateCliConfig(overridden)); + + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const error = Cause.findErrorOption(exit.cause); + expect(Option.isSome(error)).toBe(true); + if (Option.isSome(error)) { + expect(Schema.isSchemaError(error.value)).toBe(true); + } + } }), - ); - const overridden = { - ...loaded, - auth: { - ...loaded.auth, - sms: { - ...loaded.auth.sms, - twilio: { ...loaded.auth.sms.twilio, ...incompleteTwilio }, - }, - }, - }; - const exit = await Effect.runPromiseExit(validateCliConfig(overridden)); + )); - expect(Exit.isFailure(exit)).toBe(true); - if (Exit.isFailure(exit)) { - const error = Cause.findErrorOption(exit.cause); - expect(Option.isSome(error)).toBe(true); - if (Option.isSome(error)) { - expect(Schema.isSchemaError(error.value)).toBe(true); - } - } - }); + test("accepts incomplete unselected remotes but rejects structurally invalid remotes", () => + Effect.runPromise( + Effect.gen(function* () { + const config = yield* validateCliConfig({ + remotes: { + staging: { auth: { sms: { twilio: incompleteTwilio } } }, + }, + }); + expect(config.remotes.staging).toBeDefined(); + if (config.remotes.staging !== undefined) { + expect(config.remotes.staging.auth.sms.twilio.enabled).toBe(true); + } - test("accepts incomplete unselected remotes but rejects structurally invalid remotes", async () => { - const config = await Effect.runPromise( - validateCliConfig({ - remotes: { - staging: { auth: { sms: { twilio: incompleteTwilio } } }, - }, + const exit = yield* Effect.exit( + validateCliConfig({ remotes: { staging: { project_id: 123 } } }), + ); + expect(Exit.isFailure(exit)).toBe(true); }), - ); - expect(config.remotes.staging).toBeDefined(); - if (config.remotes.staging !== undefined) { - expect(config.remotes.staging.auth.sms.twilio.enabled).toBe(true); - } - - const exit = await Effect.runPromiseExit( - validateCliConfig({ remotes: { staging: { project_id: 123 } } }), - ); - expect(Exit.isFailure(exit)).toBe(true); - }); + )); }); From 3509d187e91975b9b5eabae1dad2c47cb1b4a985 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Wed, 21 Oct 2026 10:59:11 +0000 Subject: [PATCH 10/24] refactor(config): cover the config scripts with effect lint (CLI-2757) (#7265) ## TL;DR brings the `packages/config` build and release scripts under the effect lint ## whats introduced? - the allow list widens to `packages/config/**` - `build.ts`, `release-plan.ts` and the semantic release path filter run as effect programs with the same output, exit codes, release outputs and temp dir cleanup - a malformed `package.json` now reports a schema decode message instead of the native json parse one, same exit code ## ref - closes: CLI-2757 --- .oxlintrc.effect.json | 3 +- packages/config/scripts/build.ts | 448 ++++++++++++------ packages/config/scripts/release-plan.ts | 203 +++++--- .../scripts/semantic-release-path-filter.ts | 163 ++++--- .../semantic-release-path-filter.unit.test.ts | 443 +++++++++-------- 5 files changed, 789 insertions(+), 471 deletions(-) diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json index 5fbe226414..d85a5d5b1f 100644 --- a/.oxlintrc.effect.json +++ b/.oxlintrc.effect.json @@ -6,7 +6,7 @@ }, "ignorePatterns": [ "**", - "!packages/config/src/**", + "!packages/config/**", "!packages/stack/**", "!apps/cli/src/auth/**", "!apps/cli/src/cli/**", @@ -137,6 +137,7 @@ "apps/cli/src/command-internal/config-validate.unit.test.ts", "apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts", "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts", + "packages/config/scripts/json-schema-postprocess.unit.test.ts", "packages/config/src/analytics.ts", "packages/config/src/api.ts", "packages/config/src/auth/email.ts", diff --git a/packages/config/scripts/build.ts b/packages/config/scripts/build.ts index a9d7a1cb80..79fd55421f 100644 --- a/packages/config/scripts/build.ts +++ b/packages/config/scripts/build.ts @@ -1,6 +1,5 @@ -import { mkdir, mkdtemp, realpath, rm, symlink } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import path from "node:path"; +import { BunFileSystem, BunPath } from "@effect/platform-bun"; +import { Console, Data, Effect, FileSystem, Layer, Path, type PlatformError, Schema } from "effect"; import { CliConfigSchema, toCliConfigJsonSchema } from "../src/base.ts"; import { ProjectConfigSchema, @@ -9,15 +8,61 @@ import { import { CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL } from "../src/schema-metadata.ts"; import { collapseNonFiniteNumberUnions, withSchemaMetadata } from "./json-schema-postprocess.ts"; -const packageRoot = path.resolve(import.meta.dir, ".."); +const packageRoot = Effect.gen(function* () { + const path = yield* Path.Path; + return path.resolve(import.meta.dir, ".."); +}); -async function runCommand(cmd: readonly string[], cwd: string = packageRoot): Promise<void> { - const child = Bun.spawn([...cmd], { cwd, stdout: "inherit", stderr: "inherit" }); - const exitCode = await child.exited; +const encodeJson = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown)); +const encodeIndentedJson = Schema.encodeEffect(Schema.fromJsonString(Schema.Unknown, { space: 2 })); + +class StdinWriteError extends Data.TaggedError("StdinWriteError")<{ readonly cause: unknown }> {} + +class BuildInputError extends Data.TaggedError("BuildInputError")<{ + readonly source: string; + readonly message: string; + readonly cause?: unknown; +}> {} + +const buildInputError = + (source: string) => + <A, R>(effect: Effect.Effect<A, Schema.SchemaError, R>) => + effect.pipe( + Effect.mapError((cause) => new BuildInputError({ source, message: cause.message, cause })), + ); + +/* Dies with the platform's own error object, so a host failure prints as the raw OS error. */ +const dieWithPlatformCause = (error: PlatformError.PlatformError) => + Effect.die(error.cause ?? error); + +/* Like `try`/`finally`: a cleanup failure replaces the body's outcome rather than joining it. */ +const withTempDirectory = <A, E, R, E2, R2>( + acquire: Effect.Effect<string, E, R>, + use: (dir: string) => Effect.Effect<A, E2, R2>, +) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + return yield* Effect.acquireUseRelease( + acquire, + (dir) => Effect.exit(use(dir)), + (dir) => + fs + .remove(dir, { recursive: true, force: true }) + .pipe(Effect.catchTag("PlatformError", dieWithPlatformCause)), + ); + }).pipe(Effect.flatten); + +const runCommand = Effect.fnUntraced(function* (cmd: readonly string[], cwd?: string) { + const child = Bun.spawn([...cmd], { + cwd: cwd ?? (yield* packageRoot), + stdout: "inherit", + stderr: "inherit", + }); + const exitCode = yield* Effect.promise(() => child.exited); if (exitCode !== 0) { - throw new Error(`\`${cmd.join(" ")}\` failed with exit code ${exitCode}`); + return yield* Effect.die(new Error(`\`${cmd.join(" ")}\` failed with exit code ${exitCode}`)); } -} +}); function isRecord(value: unknown): value is Record<string, unknown> { return typeof value === "object" && value !== null; @@ -47,48 +92,67 @@ function readStringAt(root: unknown, path: ReadonlyArray<string>): string { * `as` cast, since a non-object result here would mean the collapse walk * itself is broken. */ -async function renderCollapsedJsonSchema( +const renderCollapsedJsonSchema = Effect.fnUntraced(function* ( outputPath: string, document: unknown, rootAst: Parameters<typeof collapseNonFiniteNumberUnions>[1], metadata: Parameters<typeof withSchemaMetadata>[1], -): Promise<void> { +) { const collapsed = collapseNonFiniteNumberUnions(document, rootAst); if (!isRecord(collapsed)) { - throw new Error(`collapseNonFiniteNumberUnions did not return an object for ${outputPath}`); + return yield* Effect.die( + new Error(`collapseNonFiniteNumberUnions did not return an object for ${outputPath}`), + ); } - await renderJsonSchema(outputPath, withSchemaMetadata(collapsed, metadata)); -} + yield* renderJsonSchema(outputPath, withSchemaMetadata(collapsed, metadata)); +}); -async function renderJsonSchema(outputPath: string, json: Record<string, unknown>): Promise<void> { - const schema = `${JSON.stringify(json, null, 2)}\n`; +const renderJsonSchema = Effect.fnUntraced(function* ( + outputPath: string, + json: Record<string, unknown>, +) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const schema = `${yield* encodeIndentedJson(json)}\n`; const formatter = Bun.spawn(["bun", "x", "oxfmt", `--stdin-filepath=${outputPath}`], { stdin: "pipe", stdout: "pipe", stderr: "pipe", }); - try { - await formatter.stdin.write(schema); - await formatter.stdin.end(); - } catch (error) { - if (!(error instanceof Error && "code" in error && error.code === "EPIPE")) { - throw error; - } - } + yield* Effect.tryPromise({ + try: () => Promise.resolve(formatter.stdin.write(schema)), + catch: (cause) => new StdinWriteError({ cause }), + }).pipe( + Effect.andThen( + Effect.tryPromise({ + try: () => Promise.resolve(formatter.stdin.end()), + catch: (cause) => new StdinWriteError({ cause }), + }), + ), + Effect.catchTag("StdinWriteError", ({ cause: error }) => + error instanceof Error && "code" in error && error.code === "EPIPE" + ? Effect.void + : Effect.die(error), + ), + ); - const [exitCode, formatted, stderr] = await Promise.all([ - formatter.exited, - new Response(formatter.stdout).text(), - new Response(formatter.stderr).text(), - ]); + const [exitCode, formatted, stderr] = yield* Effect.promise(() => + Promise.all([ + formatter.exited, + new Response(formatter.stdout).text(), + new Response(formatter.stderr).text(), + ]), + ); if (exitCode !== 0) { - throw new Error(`oxfmt failed with exit code ${exitCode}: ${stderr.trim()}`); + return yield* Effect.die( + new Error(`oxfmt failed with exit code ${exitCode}: ${stderr.trim()}`), + ); } - await mkdir(path.dirname(outputPath), { recursive: true }); - await Bun.write(outputPath, formatted); -} + yield* fs.makeDirectory(path.dirname(outputPath), { recursive: true }); + yield* fs.writeFileString(outputPath, formatted); +}); /** * Proves `sideEffects: false` against the real compiled `dist/index.js`, not by @@ -97,41 +161,50 @@ async function renderJsonSchema(outputPath: string, json: Record<string, unknown * tree-shake away. A positive-control probe confirms the test's marker is * actually detectable by this method before trusting its absence as meaningful. */ -async function verifyTreeShaking(): Promise<void> { - const distIndexPath = await realpath(path.join(packageRoot, "dist", "index.js")); - const distInternalPath = await realpath(path.join(packageRoot, "dist", "internal.js")); +const verifyTreeShaking = Effect.fnUntraced(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* packageRoot; + const distIndexPath = yield* fs.realPath(path.join(root, "dist", "index.js")); + const distInternalPath = yield* fs.realPath(path.join(root, "dist", "internal.js")); // `mkdtemp` can return a path through a symlinked prefix (e.g. macOS's // `/var` -> `/private/var`) that Bun's bundler resolves canonically; compute // the relative specifier against that same canonical form, or `path.relative` // produces a specifier with one too many `../` segments. - const probeDir = await realpath( - await mkdtemp(path.join(tmpdir(), "supabase-config-tree-shake-")), - ); + const makeProbeDir = fs + .makeTempDirectory({ prefix: "supabase-config-tree-shake-" }) + .pipe(Effect.flatMap((dir) => fs.realPath(dir))); - try { + const probeInDirectory = Effect.fnUntraced(function* (probeDir: string) { function relativeSpecifierFor(target: string): string { const relative = path.relative(probeDir, target).split(path.sep).join("/"); return relative.startsWith(".") ? relative : `./${relative}`; } - async function bundle(entryName: string, source: string): Promise<string> { + const bundle = Effect.fnUntraced(function* (entryName: string, source: string) { const probeEntry = path.join(probeDir, entryName); - await Bun.write(probeEntry, source); - const result = await Bun.build({ - entrypoints: [probeEntry], - target: "browser", - minify: false, - }); + yield* fs.writeFileString(probeEntry, source); + const result = yield* Effect.promise(() => + Bun.build({ + entrypoints: [probeEntry], + target: "browser", + minify: false, + }), + ); if (!result.success) { const messages = result.logs.map((log) => log.message).join("\n"); - throw new Error(`tree-shake probe failed to bundle ${entryName}:\n${messages}`); + return yield* Effect.die( + new Error(`tree-shake probe failed to bundle ${entryName}:\n${messages}`), + ); } const [output] = result.outputs; if (!output) { - throw new Error(`tree-shake probe produced no bundle output for ${entryName}`); + return yield* Effect.die( + new Error(`tree-shake probe produced no bundle output for ${entryName}`), + ); } - return output.text(); - } + return yield* Effect.promise(() => output.text()); + }); // Only appears in `src/project-config/registry*.ts`; `CliConfigSchema`'s own // field names (`base.ts`/`api.ts`) use `schemas`, not `db_schema`. @@ -147,46 +220,52 @@ async function verifyTreeShaking(): Promise<void> { "description", ]); - const positiveControlCode = await bundle( + const positiveControlCode = yield* bundle( "positive-control.js", `export { projectConfigMappingRows } from "${relativeSpecifierFor(distInternalPath)}";\n`, ); if (!positiveControlCode.includes(REGISTRY_ONLY_MARKER)) { - throw new Error( - `tree-shake probe's positive control failed: bundling { projectConfigMappingRows } from ` + - `dist/internal.js did not include marker ${JSON.stringify(REGISTRY_ONLY_MARKER)} — this ` + - `probe methodology can no longer detect the marker it's meant to prove absent below, so its ` + - `absence from the CliConfigSchema-only probe would be meaningless.`, + return yield* Effect.die( + new Error( + `tree-shake probe's positive control failed: bundling { projectConfigMappingRows } from ` + + `dist/internal.js did not include marker ${yield* encodeJson(REGISTRY_ONLY_MARKER)} — this ` + + `probe methodology can no longer detect the marker it's meant to prove absent below, so its ` + + `absence from the CliConfigSchema-only probe would be meaningless.`, + ), ); } - const code = await bundle( + const code = yield* bundle( "probe.js", `export { CliConfigSchema } from "${relativeSpecifierFor(distIndexPath)}";\n`, ); if (code.includes(REGISTRY_ONLY_MARKER)) { - throw new Error( - `tree-shake probe failed: bundling only { CliConfigSchema } from dist/index.js still pulled in ` + - `registry-only code (found marker ${JSON.stringify(REGISTRY_ONLY_MARKER)} from ` + - `project-config/registry.ts). "sideEffects": false is not holding for this package — ` + - `investigate before trusting the tree-shaking claim.`, + return yield* Effect.die( + new Error( + `tree-shake probe failed: bundling only { CliConfigSchema } from dist/index.js still pulled in ` + + `registry-only code (found marker ${yield* encodeJson(REGISTRY_ONLY_MARKER)} from ` + + `project-config/registry.ts). "sideEffects": false is not holding for this package — ` + + `investigate before trusting the tree-shaking claim.`, + ), ); } if (!code.includes(SCHEMA_MARKER)) { - throw new Error( - `tree-shake probe failed: expected schema marker ${JSON.stringify(SCHEMA_MARKER)} is missing from ` + - `the bundle output — the probe isn't actually exercising CliConfigSchema.`, + return yield* Effect.die( + new Error( + `tree-shake probe failed: expected schema marker ${yield* encodeJson(SCHEMA_MARKER)} is missing from ` + + `the bundle output — the probe isn't actually exercising CliConfigSchema.`, + ), ); } - console.log( + yield* Console.log( `[build] tree-shake probe OK (${code.length} bytes; registry-only marker absent, schema marker present, positive control passed).`, ); - } finally { - await rm(probeDir, { recursive: true, force: true }); - } -} + }); + + yield* withTempDirectory(makeProbeDir, probeInDirectory); +}); const SMOKE_TEST_RUNTIME_DEPS = [ "effect", @@ -234,7 +313,10 @@ function buildSmokeTestScript(): string { * Extracts with `tar` (not `npm install <tarball>`) and symlinks pnpm's * already-resolved deps in, keeping the check hermetic and network-free. */ -async function runPackAndInstallSmokeTest(): Promise<void> { +const runPackAndInstallSmokeTest = Effect.fnUntraced(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* packageRoot; const npmPath = Bun.which("npm"); const nodePath = Bun.which("node"); const tarPath = Bun.which("tar"); @@ -244,37 +326,39 @@ async function runPackAndInstallSmokeTest(): Promise<void> { nodePath === null ? "node" : null, tarPath === null ? "tar" : null, ].filter((name) => name !== null); - throw new Error( - `the pack-and-install smoke test (CLI-2234) requires ${missing.join(", ")} on PATH — install ` + - "it (mise provides node/npm; tar ships with every supported OS) before running `pnpm build`.", + return yield* Effect.die( + new Error( + `the pack-and-install smoke test (CLI-2234) requires ${missing.join(", ")} on PATH — install ` + + "it (mise provides node/npm; tar ships with every supported OS) before running `pnpm build`.", + ), ); } - const scratchDir = await mkdtemp(path.join(tmpdir(), "supabase-config-pack-smoke-")); - try { + const packAndInstall = Effect.fnUntraced(function* (scratchDir: string) { const packResult = Bun.spawn([npmPath, "pack", "--json", "--pack-destination", scratchDir], { - cwd: packageRoot, + cwd: root, stdout: "pipe", stderr: "inherit", }); - const [packExitCode, packStdout] = await Promise.all([ - packResult.exited, - new Response(packResult.stdout).text(), - ]); + const [packExitCode, packStdout] = yield* Effect.promise(() => + Promise.all([packResult.exited, new Response(packResult.stdout).text()]), + ); if (packExitCode !== 0) { - throw new Error(`\`npm pack\` failed with exit code ${packExitCode}`); + return yield* Effect.die(new Error(`\`npm pack\` failed with exit code ${packExitCode}`)); } - const packEntries = JSON.parse(packStdout) as ReadonlyArray<{ readonly filename: string }>; + const packEntries = yield* decodePackEntries(packStdout).pipe( + buildInputError("`npm pack --json` output"), + ); const [packEntry] = packEntries; if (packEntry === undefined) { - throw new Error("`npm pack --json` produced no tarball entries"); + return yield* Effect.die(new Error("`npm pack --json` produced no tarball entries")); } const tarballPath = path.join(scratchDir, packEntry.filename); const consumerDir = path.join(scratchDir, "consumer"); const consumerConfigDir = path.join(consumerDir, "node_modules", "@supabase", "config"); - await mkdir(consumerConfigDir, { recursive: true }); - await runCommand([ + yield* fs.makeDirectory(consumerConfigDir, { recursive: true }); + yield* runCommand([ tarPath, "-xzf", tarballPath, @@ -283,113 +367,167 @@ async function runPackAndInstallSmokeTest(): Promise<void> { "--strip-components=1", ]); - await Bun.write( + yield* fs.writeFileString( path.join(consumerDir, "package.json"), - `${JSON.stringify( - { name: "supabase-config-pack-smoke", version: "0.0.0", private: true, type: "module" }, - null, - 2, - )}\n`, + `${yield* encodeIndentedJson({ + name: "supabase-config-pack-smoke", + version: "0.0.0", + private: true, + type: "module", + })}\n`, ); const consumerNodeModules = path.join(consumerDir, "node_modules"); for (const name of SMOKE_TEST_RUNTIME_DEPS) { - const real = await realpath(path.join(packageRoot, "node_modules", name)); + const real = yield* fs.realPath(path.join(root, "node_modules", name)); const dest = path.join(consumerNodeModules, name); - await mkdir(path.dirname(dest), { recursive: true }); - await symlink(real, dest, "dir"); + yield* fs.makeDirectory(path.dirname(dest), { recursive: true }); + yield* fs.symlink(real, dest); } - await runCommand([nodePath, "--input-type=module", "-e", buildSmokeTestScript()], consumerDir); - } finally { - await rm(scratchDir, { recursive: true, force: true }); - } -} + yield* runCommand([nodePath, "--input-type=module", "-e", buildSmokeTestScript()], consumerDir); + }); -interface ExportsMap { - readonly [subpath: string]: ExportsNode; -} -type ExportsNode = string | { readonly [condition: string]: ExportsNode }; + yield* withTempDirectory( + fs.makeTempDirectory({ prefix: "supabase-config-pack-smoke-" }), + packAndInstall, + ); +}); -function collectDistTargets(node: ExportsNode, into: Set<string>): void { +const decodePackageJson = Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown)); + +const decodePackEntries = Schema.decodeEffect( + Schema.fromJsonString(Schema.Array(Schema.Struct({ filename: Schema.String }))), +); + +const collectDistTargets: ( + node: unknown, + into: Set<string>, +) => Effect.Effect<void, BuildInputError> = Effect.fnUntraced(function* (node, into) { if (typeof node === "string") { if (node.startsWith("./dist/")) { into.add(node); } return; } + if (node === null || node === undefined) { + return yield* new BuildInputError({ + source: "package.json", + message: `an exports target is ${node}`, + }); + } + if (!isRecord(node)) { + return; + } for (const [condition, value] of Object.entries(node)) { // `bun` conditions point at `src/*.ts`, which trivially exists at every // commit (it's source, not a build output) — nothing to verify here. if (condition === "bun") { continue; } - collectDistTargets(value, into); + yield* collectDistTargets(value, into); } -} +}); /** Every `types`/`default`/JSON-artifact target the exports map declares must exist once the build finishes. */ -async function verifyExportsMapTargetsExist(): Promise<void> { - const packageJson = JSON.parse(await Bun.file(path.join(packageRoot, "package.json")).text()) as { - readonly exports: ExportsMap; - }; +const verifyExportsMapTargetsExist = Effect.fnUntraced(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* packageRoot; + const packageJson = yield* decodePackageJson( + yield* fs.readFileString(path.join(root, "package.json")), + ).pipe(buildInputError("package.json")); + const exportsField = isRecord(packageJson) ? packageJson.exports : undefined; + if (exportsField === null || exportsField === undefined) { + return yield* new BuildInputError({ + source: "package.json", + message: `"exports" is ${exportsField === null ? "null" : "missing"}`, + }); + } const targets = new Set<string>(); - for (const node of Object.values(packageJson.exports)) { - collectDistTargets(node, targets); + if (isRecord(exportsField)) { + for (const node of Object.values(exportsField)) { + yield* collectDistTargets(node, targets); + } } const missing: string[] = []; for (const target of targets) { - if (!(await Bun.file(path.join(packageRoot, target)).exists())) { + /* A directory at a target path does not count as an emitted file. */ + const isFile = yield* fs.stat(path.join(root, target)).pipe( + Effect.map((info) => info.type === "File"), + Effect.orElseSucceed(() => false), + ); + if (!isFile) { missing.push(target); } } if (missing.length > 0) { - throw new Error( - `the following dist targets declared in package.json's exports map are missing after the ` + - `build: ${missing.join(", ")}`, + return yield* Effect.die( + new Error( + `the following dist targets declared in package.json's exports map are missing after the ` + + `build: ${missing.join(", ")}`, + ), ); } - console.log(`[build] verified ${targets.size} exports-map dist targets exist on disk.`); -} + yield* Console.log(`[build] verified ${targets.size} exports-map dist targets exist on disk.`); +}); + +const build = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* packageRoot; + + yield* Console.log("[build] removing stale dist/ (stale modules from renames must not ship)..."); + yield* fs.remove(path.join(root, "dist"), { recursive: true, force: true }); + + yield* Console.log("[build] compiling TypeScript project (tsconfig.build.json)..."); + yield* runCommand(["pnpm", "exec", "tsc", "-p", "tsconfig.build.json"]); + + yield* Console.log("[build] rendering JSON Schema artifacts..."); + yield* renderCollapsedJsonSchema( + path.join(root, "dist", "schema.json"), + toCliConfigJsonSchema(), + CliConfigSchema.ast, + { + id: CLI_CONFIG_SCHEMA_URL, + title: "Supabase CLI config (CliConfig)", + description: + "The Supabase CLI's local project config document (supabase/config.toml or supabase/config.json).", + }, + ); + yield* renderCollapsedJsonSchema( + path.join(root, "dist", "project-schema.json"), + toProjectConfigJsonSchema(), + ProjectConfigSchema.ast, + { + id: PROJECT_CONFIG_SCHEMA_URL, + title: "Supabase hosted project config (ProjectConfig)", + description: + "The sparse, hosted-project subset of CliConfig that a Supabase project manages.", + }, + ); -console.log("[build] removing stale dist/ (stale modules from renames must not ship)..."); -await rm(path.join(packageRoot, "dist"), { recursive: true, force: true }); - -console.log("[build] compiling TypeScript project (tsconfig.build.json)..."); -await runCommand(["pnpm", "exec", "tsc", "-p", "tsconfig.build.json"]); - -console.log("[build] rendering JSON Schema artifacts..."); -await renderCollapsedJsonSchema( - path.join(packageRoot, "dist", "schema.json"), - toCliConfigJsonSchema(), - CliConfigSchema.ast, - { - id: CLI_CONFIG_SCHEMA_URL, - title: "Supabase CLI config (CliConfig)", - description: - "The Supabase CLI's local project config document (supabase/config.toml or supabase/config.json).", - }, -); -await renderCollapsedJsonSchema( - path.join(packageRoot, "dist", "project-schema.json"), - toProjectConfigJsonSchema(), - ProjectConfigSchema.ast, - { - id: PROJECT_CONFIG_SCHEMA_URL, - title: "Supabase hosted project config (ProjectConfig)", - description: "The sparse, hosted-project subset of CliConfig that a Supabase project manages.", - }, -); + yield* Console.log("[build] verifying every exports-map dist target exists..."); + yield* verifyExportsMapTargetsExist(); -console.log("[build] verifying every exports-map dist target exists..."); -await verifyExportsMapTargetsExist(); + yield* Console.log("[build] verifying the sideEffects:false tree-shaking claim..."); + yield* verifyTreeShaking(); -console.log("[build] verifying the sideEffects:false tree-shaking claim..."); -await verifyTreeShaking(); + yield* Console.log("[build] running the pack-and-install smoke test..."); + yield* runPackAndInstallSmokeTest(); -console.log("[build] running the pack-and-install smoke test..."); -await runPackAndInstallSmokeTest(); + yield* Console.log("[build] done."); +}); -console.log("[build] done."); +await Effect.runPromise( + build.pipe( + Effect.catchTags({ + PlatformError: dieWithPlatformCause, + BuildInputError: ({ source, message, cause }) => + Effect.die(new Error(`[build] malformed ${source}: ${message}`, { cause })), + }), + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layer)), + ), +); diff --git a/packages/config/scripts/release-plan.ts b/packages/config/scripts/release-plan.ts index 7730bd2f41..bf51ecde1a 100644 --- a/packages/config/scripts/release-plan.ts +++ b/packages/config/scripts/release-plan.ts @@ -13,17 +13,44 @@ * that no release was found. */ -import { appendFile } from "node:fs/promises"; -import path from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { + Cause, + Config, + Console, + Data, + Effect, + FileSystem, + Path, + PlatformError, + Predicate, + Schema, +} from "effect"; import process from "node:process"; -import { fileURLToPath } from "node:url"; import { parseArgs } from "node:util"; import { PACKAGE_PATH_PREFIX } from "./semantic-release-path-filter.ts"; // Not `import.meta.dir`: that Bun-ism doesn't survive vitest's module // transform, and this module is imported by its unit test. -const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); +const packageRoot = Effect.gen(function* () { + const path = yield* Path.Path; + return path.resolve(path.dirname(yield* path.fromFileUrl(new URL(import.meta.url))), ".."); +}); + +class ReleasePlanError extends Data.TaggedError("ReleasePlanError")<{ + readonly message: string; +}> {} + +/* Keeps the platform's own error text in the `[release-plan]` failure report. */ +const hostFailure = Effect.mapError( + (error: PlatformError.PlatformError) => + new ReleasePlanError({ + message: error.cause instanceof Error ? error.cause.message : error.message, + }), +); + +const decodeJson = Schema.decodeEffect(Schema.fromJsonString(Schema.Unknown)); interface ConfigPackageJson { readonly name: string; @@ -60,29 +87,38 @@ function isRecord(value: unknown): value is Record<string, unknown> { return typeof value === "object" && value !== null; } -async function readPackageJson(): Promise<ConfigPackageJson> { - const packageJsonPath = path.join(packageRoot, "package.json"); - const parsed: unknown = JSON.parse(await Bun.file(packageJsonPath).text()); +const readPackageJson = Effect.fnUntraced(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const packageJsonPath = path.join(yield* packageRoot, "package.json"); + const parsed: unknown = yield* decodeJson( + yield* fs.readFileString(packageJsonPath).pipe(hostFailure), + ).pipe( + Effect.mapError( + (error) => + new ReleasePlanError({ message: `${packageJsonPath} is malformed: ${error.message}` }), + ), + ); if ( !isRecord(parsed) || typeof parsed.name !== "string" || (parsed.private !== undefined && typeof parsed.private !== "boolean") ) { - throw new Error( - `${packageJsonPath} is malformed: expected a string "name" and an optional boolean "private".`, - ); + return yield* new ReleasePlanError({ + message: `${packageJsonPath} is malformed: expected a string "name" and an optional boolean "private".`, + }); } - return { name: parsed.name, private: parsed.private }; -} + const packageJson: ConfigPackageJson = { name: parsed.name, private: parsed.private }; + return packageJson; +}); -async function runGit( - args: readonly string[], - cwd: string, -): Promise<{ exitCode: number; stdout: string }> { +const runGit = Effect.fnUntraced(function* (args: readonly string[], cwd: string) { const proc = Bun.spawn(["git", ...args], { cwd, stdout: "pipe", stderr: "pipe" }); - const [stdout, exitCode] = await Promise.all([new Response(proc.stdout).text(), proc.exited]); + const [stdout, exitCode] = yield* Effect.promise(() => + Promise.all([new Response(proc.stdout).text(), proc.exited]), + ); return { exitCode, stdout: stdout.trim() }; -} +}); type HeadDivergence = "current" | "behind-remote" | "diverged" | "unknown"; @@ -93,14 +129,16 @@ type HeadDivergence = "current" | "behind-remote" | "diverged" | "unknown"; * `"unknown"` — never blocking the plan — when the remote ref can't be * resolved, e.g. a local-only clone with no `origin` fetched. */ -async function resolveHeadDivergence(cwd: string): Promise<HeadDivergence> { +const resolveHeadDivergence = Effect.fnUntraced(function* ( + cwd: string, +): Effect.fn.Return<HeadDivergence> { const remoteRef = `origin/${RELEASE_BRANCHES[0]}`; - const remote = await runGit(["rev-parse", "--verify", remoteRef], cwd); + const remote = yield* runGit(["rev-parse", "--verify", remoteRef], cwd); if (remote.exitCode !== 0 || remote.stdout === "") { return "unknown"; } - const head = await runGit(["rev-parse", "HEAD"], cwd); + const head = yield* runGit(["rev-parse", "HEAD"], cwd); if (head.exitCode !== 0 || head.stdout === "") { return "unknown"; } @@ -108,7 +146,7 @@ async function resolveHeadDivergence(cwd: string): Promise<HeadDivergence> { return "current"; } - const ancestor = await runGit(["merge-base", "--is-ancestor", "HEAD", remoteRef], cwd); + const ancestor = yield* runGit(["merge-base", "--is-ancestor", "HEAD", remoteRef], cwd); if (ancestor.exitCode === 0) { return "behind-remote"; } @@ -116,7 +154,7 @@ async function resolveHeadDivergence(cwd: string): Promise<HeadDivergence> { return "diverged"; } return "unknown"; -} +}); /** * Runs semantic-release in dry-run mode against this package's own history. @@ -127,21 +165,24 @@ async function resolveHeadDivergence(cwd: string): Promise<HeadDivergence> { * apart via git rather than parsing its log output. Otherwise * `result.nextRelease` carries the computed version, bump type, and notes. */ -async function computeReleasePlan(isPrivate: boolean): Promise<ReleasePlan> { - const { default: semanticRelease } = await import("semantic-release"); - const result = await semanticRelease( - { - branches: [...RELEASE_BRANCHES], - tagFormat: "config-v${version}", - dryRun: true, - plugins: ["./scripts/semantic-release-path-filter.ts"], - }, - { cwd: packageRoot, env: process.env }, +const computeReleasePlan = Effect.fnUntraced(function* (isPrivate: boolean) { + const { default: semanticRelease } = yield* Effect.promise(() => import("semantic-release")); + const cwd = yield* packageRoot; + const result = yield* Effect.promise(() => + semanticRelease( + { + branches: [...RELEASE_BRANCHES], + tagFormat: "config-v${version}", + dryRun: true, + plugins: ["./scripts/semantic-release-path-filter.ts"], + }, + { cwd, env: process.env }, + ), ); if (result === false) { - const divergence = await resolveHeadDivergence(packageRoot); - return { + const divergence = yield* resolveHeadDivergence(cwd); + const plan: ReleasePlan = { due: false, reason: divergence === "behind-remote" @@ -150,6 +191,7 @@ async function computeReleasePlan(isPrivate: boolean): Promise<ReleasePlan> { ? "branch-diverged" : "no-releasable-commits", }; + return plan; } // With no config-v* tag on the branch, semantic-release would cut 1.0.0 @@ -157,13 +199,14 @@ async function computeReleasePlan(isPrivate: boolean): Promise<ReleasePlan> { // changelog of every commit that ever touched packages/config/. Refuse until a // baseline tag exists (see AGENTS.md "One-time setup"); the escape hatch is // for an intentional first cut. - if (!result.lastRelease.gitTag && !process.env.CONFIG_RELEASE_ALLOW_NO_BASELINE) { - throw new Error( - "no config-v* baseline tag found on this branch: semantic-release would release " + + if (!result.lastRelease.gitTag && !(yield* readEnv("CONFIG_RELEASE_ALLOW_NO_BASELINE"))) { + return yield* new ReleasePlanError({ + message: + "no config-v* baseline tag found on this branch: semantic-release would release " + `${result.nextRelease.version} with notes generated from the entire monorepo history. ` + "Push a baseline tag first (e.g. config-v0.1.0 — see packages/config/AGENTS.md), or set " + "CONFIG_RELEASE_ALLOW_NO_BASELINE=1 to proceed deliberately.", - ); + }); } const version = result.nextRelease.version; @@ -171,33 +214,45 @@ async function computeReleasePlan(isPrivate: boolean): Promise<ReleasePlan> { // title — refuse anything that isn't the plain stable x.y.z this // stable-only train can produce. if (!/^\d+\.\d+\.\d+$/.test(version)) { - throw new Error(`computed version "${version}" is not a plain x.y.z stable version.`); + return yield* new ReleasePlanError({ + message: `computed version "${version}" is not a plain x.y.z stable version.`, + }); } - return { + const plan: ReleasePlan = { due: true, version, bumpType: result.nextRelease.type, notes: result.nextRelease.notes ?? "", isPrivate, }; + return plan; +}); + +function readEnv(name: string) { + return Config.String(name).pipe(Config.withDefault("")); } -async function appendGithubOutput(lines: readonly string[]): Promise<void> { - const outputPath = process.env.GITHUB_OUTPUT; +const appendFile = Effect.fnUntraced(function* (filePath: string, data: string) { + const fs = yield* FileSystem.FileSystem; + yield* fs.writeFileString(filePath, data, { flag: "a" }).pipe(hostFailure); +}); + +const appendGithubOutput = Effect.fnUntraced(function* (lines: readonly string[]) { + const outputPath = yield* readEnv("GITHUB_OUTPUT"); if (!outputPath) { return; } - await appendFile(outputPath, `${lines.join("\n")}\n`); -} + yield* appendFile(outputPath, `${lines.join("\n")}\n`); +}); -async function appendStepSummary(markdown: string): Promise<void> { - const summaryPath = process.env.GITHUB_STEP_SUMMARY; +const appendStepSummary = Effect.fnUntraced(function* (markdown: string) { + const summaryPath = yield* readEnv("GITHUB_STEP_SUMMARY"); if (!summaryPath) { return; } - await appendFile(summaryPath, `${markdown}\n`); -} + yield* appendFile(summaryPath, `${markdown}\n`); +}); /** * A fence long enough that no backtick run inside `content` can close it. @@ -292,36 +347,56 @@ function renderLocalPlan(plan: ReleasePlan): string { return `[release-plan] @supabase/config would release ${plan.version} (${plan.bumpType})${privateNote}.`; } -async function main(): Promise<void> { +const main = Effect.gen(function* () { const { values } = parseArgs({ options: { "notes-out": { type: "string" } } }); const notesOutPath = values["notes-out"]; - const packageJson = await readPackageJson(); + const packageJson = yield* readPackageJson(); const isPrivate = packageJson.private === true; - const plan = await computeReleasePlan(isPrivate); + const plan = yield* computeReleasePlan(isPrivate); if (plan.due && notesOutPath) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; // Guarantee the trailing newline: the notes end up as a GH release // body_path file, and a missing final newline is the kind of upstream // formatting detail nothing else pins. - await Bun.write(notesOutPath, plan.notes.endsWith("\n") ? plan.notes : `${plan.notes}\n`); + const notes = plan.notes.endsWith("\n") ? plan.notes : `${plan.notes}\n`; + /* `--notes-out` may name a file in a directory that does not exist yet; as with + Bun.write, a failed mkdir reports the original open error. */ + yield* fs.writeFileString(notesOutPath, notes).pipe( + Effect.catchIf( + (error) => Predicate.isTagged(error.reason, "NotFound"), + (error) => + fs.makeDirectory(path.dirname(notesOutPath), { recursive: true }).pipe( + Effect.mapError(() => error), + Effect.andThen(fs.writeFileString(notesOutPath, notes)), + ), + ), + hostFailure, + ); } - if (process.env.GITHUB_OUTPUT) { - await appendGithubOutput(toGithubOutputLines(plan)); + if (yield* readEnv("GITHUB_OUTPUT")) { + yield* appendGithubOutput(toGithubOutputLines(plan)); } else { - console.log(renderLocalPlan(plan)); + yield* Console.log(renderLocalPlan(plan)); } - await appendStepSummary(renderStepSummary(plan)); -} + yield* appendStepSummary(renderStepSummary(plan)); +}); if (import.meta.main) { - try { - await main(); - } catch (error) { - console.error(`[release-plan] ${error instanceof Error ? error.message : String(error)}`); - process.exit(1); - } + await Effect.runPromise( + main.pipe( + Effect.catchCause((cause) => { + const error = Cause.squash(cause); + return Console.error( + `[release-plan] ${error instanceof Error ? error.message : String(error)}`, + ).pipe(Effect.andThen(Effect.sync(() => process.exit(1)))); + }), + Effect.provide(BunServices.layer), + ), + ); } diff --git a/packages/config/scripts/semantic-release-path-filter.ts b/packages/config/scripts/semantic-release-path-filter.ts index 0e2678a51c..fdec2dff1a 100644 --- a/packages/config/scripts/semantic-release-path-filter.ts +++ b/packages/config/scripts/semantic-release-path-filter.ts @@ -12,6 +12,7 @@ * standard Angular commit analysis, just scoped to its own history. */ +import { Data, Effect } from "effect"; import process from "node:process"; import type { AnalyzeCommitsContext, GenerateNotesContext } from "semantic-release"; @@ -45,6 +46,10 @@ const releaseNotesGenerator: ReleaseNotesGeneratorPlugin = require("@semantic-re export const PACKAGE_PATH_PREFIX = "packages/config/"; +class GitDiffTreeStdinError extends Data.TaggedError("GitDiffTreeStdinError")<{ + readonly cause: unknown; +}> {} + /** * Resolves which of `commits` touch a path under {@link PACKAGE_PATH_PREFIX} * using one batched `git diff-tree --stdin -r --root --name-only -z` @@ -56,86 +61,116 @@ export const PACKAGE_PATH_PREFIX = "packages/config/"; * rather than mis-parsed; `--root` makes a root commit diff against the empty * tree instead of also printing nothing. */ -export async function filterCommitsToPackage<T extends { hash: string }>( +export function filterCommitsToPackage<T extends { hash: string }>( commits: readonly T[], cwd: string, ): Promise<T[]> { - if (commits.length === 0) { - return []; - } - - const hashes = commits.map((commit) => commit.hash); - // `git diff-tree --stdin` echoes each commit's full object ID, and the header - // recognition below matches echoed elements against the input set; an - // abbreviated hash would never match its own echo, silently dropping that - // commit's paths. Refuse anything but full OIDs up front — semantic-release - // always supplies them, this guards other callers. - const invalidHashes = hashes.filter((hash) => !/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/.test(hash)); - if (invalidHashes.length > 0) { - throw new Error( - `filterCommitsToPackage requires full lowercase hex object IDs; got: ${invalidHashes - .slice(0, 3) - .join(", ")}${invalidHashes.length > 3 ? ", …" : ""}`, - ); - } - const knownHashes = new Set(hashes); - - const proc = Bun.spawn(["git", "diff-tree", "--stdin", "-r", "--root", "--name-only", "-z"], { - cwd, - stdin: "pipe", - stdout: "pipe", - stderr: "pipe", - }); - // Starts draining stdout/stderr before writing stdin: Bun buffers subprocess - // output eagerly so this can't deadlock today, but a future runtime change - // could reintroduce the classic full-pipe deadlock, so don't rely on that. - const stdoutText = new Response(proc.stdout).text(); - const stderrText = new Response(proc.stderr).text(); - try { - await proc.stdin.write(`${hashes.join("\n")}\n`); - await proc.stdin.end(); - } catch (error) { - if (!(error instanceof Error && "code" in error && error.code === "EPIPE")) { - throw error; + return Effect.runPromise(filterCommitsToPackageEffect(commits, cwd)); +} + +function filterCommitsToPackageEffect<T extends { hash: string }>( + commits: readonly T[], + cwd: string, +) { + return Effect.gen(function* () { + if (commits.length === 0) { + return []; } - } - - const [exitCode, stdout, stderr] = await Promise.all([proc.exited, stdoutText, stderrText]); - if (exitCode !== 0) { - throw new Error(`git diff-tree --stdin failed with exit code ${exitCode}: ${stderr.trim()}`); - } - - const touchedHashes = new Set<string>(); - let currentHash: string | null = null; - for (const element of stdout.split("\0")) { - if (element.length === 0) { - continue; + + const hashes = commits.map((commit) => commit.hash); + // `git diff-tree --stdin` echoes each commit's full object ID, and the header + // recognition below matches echoed elements against the input set; an + // abbreviated hash would never match its own echo, silently dropping that + // commit's paths. Refuse anything but full OIDs up front — semantic-release + // always supplies them, this guards other callers. + const invalidHashes = hashes.filter((hash) => !/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/.test(hash)); + if (invalidHashes.length > 0) { + return yield* Effect.die( + new Error( + `filterCommitsToPackage requires full lowercase hex object IDs; got: ${invalidHashes + .slice(0, 3) + .join(", ")}${invalidHashes.length > 3 ? ", …" : ""}`, + ), + ); } - if (knownHashes.has(element)) { - currentHash = element; - continue; + const knownHashes = new Set(hashes); + + const proc = Bun.spawn(["git", "diff-tree", "--stdin", "-r", "--root", "--name-only", "-z"], { + cwd, + stdin: "pipe", + stdout: "pipe", + stderr: "pipe", + }); + // Starts draining stdout/stderr before writing stdin: Bun buffers subprocess + // output eagerly so this can't deadlock today, but a future runtime change + // could reintroduce the classic full-pipe deadlock, so don't rely on that. + const stdoutText = new Response(proc.stdout).text(); + const stderrText = new Response(proc.stderr).text(); + yield* Effect.tryPromise({ + try: () => Promise.resolve(proc.stdin.write(`${hashes.join("\n")}\n`)), + catch: (cause) => new GitDiffTreeStdinError({ cause }), + }).pipe( + Effect.andThen( + Effect.tryPromise({ + try: () => Promise.resolve(proc.stdin.end()), + catch: (cause) => new GitDiffTreeStdinError({ cause }), + }), + ), + Effect.catchTag("GitDiffTreeStdinError", ({ cause: error }) => + error instanceof Error && "code" in error && error.code === "EPIPE" + ? Effect.void + : Effect.die(error), + ), + ); + + const [exitCode, stdout, stderr] = yield* Effect.promise(() => + Promise.all([proc.exited, stdoutText, stderrText]), + ); + if (exitCode !== 0) { + return yield* Effect.die( + new Error(`git diff-tree --stdin failed with exit code ${exitCode}: ${stderr.trim()}`), + ); } - if (currentHash !== null && element.startsWith(PACKAGE_PATH_PREFIX)) { - touchedHashes.add(currentHash); + + const touchedHashes = new Set<string>(); + let currentHash: string | null = null; + for (const element of stdout.split("\0")) { + if (element.length === 0) { + continue; + } + if (knownHashes.has(element)) { + currentHash = element; + continue; + } + if (currentHash !== null && element.startsWith(PACKAGE_PATH_PREFIX)) { + touchedHashes.add(currentHash); + } } - } - return commits.filter((commit) => touchedHashes.has(commit.hash)); + return commits.filter((commit) => touchedHashes.has(commit.hash)); + }); } -async function withFilteredCommits<C extends AnalyzeCommitsContext | GenerateNotesContext, R>( +function withFilteredCommits<C extends AnalyzeCommitsContext | GenerateNotesContext, R>( context: C, step: string, delegate: (filteredContext: C) => Promise<R>, ): Promise<R> { - const filtered = await filterCommitsToPackage(context.commits, context.cwd ?? process.cwd()); - context.logger.log( - `${step}: ${filtered.length} of ${context.commits.length} commits touch ${PACKAGE_PATH_PREFIX}`, + return Effect.runPromise( + Effect.gen(function* () { + const filtered = yield* filterCommitsToPackageEffect( + context.commits, + context.cwd ?? process.cwd(), + ); + context.logger.log( + `${step}: ${filtered.length} of ${context.commits.length} commits touch ${PACKAGE_PATH_PREFIX}`, + ); + return yield* Effect.promise(() => delegate({ ...context, commits: filtered })); + }), ); - return delegate({ ...context, commits: filtered }); } -export async function analyzeCommits( +export function analyzeCommits( pluginConfig: PluginConfig, context: AnalyzeCommitsContext, ): Promise<string | null> { @@ -144,7 +179,7 @@ export async function analyzeCommits( ); } -export async function generateNotes( +export function generateNotes( pluginConfig: PluginConfig, context: GenerateNotesContext, ): Promise<string> { diff --git a/packages/config/scripts/semantic-release-path-filter.unit.test.ts b/packages/config/scripts/semantic-release-path-filter.unit.test.ts index f79a9a61ec..abeda51437 100644 --- a/packages/config/scripts/semantic-release-path-filter.unit.test.ts +++ b/packages/config/scripts/semantic-release-path-filter.unit.test.ts @@ -1,6 +1,5 @@ -import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { dirname, join } from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, FileSystem, Path, type Scope } from "effect"; import process from "node:process"; import type { AnalyzeCommitsContext, Commit, GenerateNotesContext } from "semantic-release"; import { afterAll, beforeAll, describe, expect, test } from "vitest"; @@ -22,36 +21,48 @@ const GIT_HERMETIC_CONFIG = [ "commit.gpgsign=false", ]; -async function git(cwd: string, args: string[]): Promise<string> { - const proc = Bun.spawn(["git", ...GIT_HERMETIC_CONFIG, ...args], { - cwd, - stdout: "pipe", - stderr: "pipe", +function runTest<A, E>( + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | Scope.Scope>, +): Promise<A> { + return Effect.runPromise(effect.pipe(Effect.scoped, Effect.provide(BunServices.layer))); +} + +function git(cwd: string, args: string[]) { + return Effect.gen(function* () { + const proc = Bun.spawn(["git", ...GIT_HERMETIC_CONFIG, ...args], { + cwd, + stdout: "pipe", + stderr: "pipe", + }); + const [exitCode, stdout, stderr] = yield* Effect.promise(() => + Promise.all([ + proc.exited, + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + ]), + ); + if (exitCode !== 0) { + return yield* Effect.die( + new Error(`git ${args.join(" ")} failed with exit code ${exitCode}: ${stderr.trim()}`), + ); + } + return stdout; }); - const [exitCode, stdout, stderr] = await Promise.all([ - proc.exited, - new Response(proc.stdout).text(), - new Response(proc.stderr).text(), - ]); - if (exitCode !== 0) { - throw new Error(`git ${args.join(" ")} failed with exit code ${exitCode}: ${stderr.trim()}`); - } - return stdout; } -async function commitFiles( - cwd: string, - files: Record<string, string>, - message: string, -): Promise<string> { - for (const [relativePath, content] of Object.entries(files)) { - const fullPath = join(cwd, relativePath); - await mkdir(dirname(fullPath), { recursive: true }); - await writeFile(fullPath, content); - } - await git(cwd, ["add", ...Object.keys(files)]); - await git(cwd, ["commit", "-m", message]); - return (await git(cwd, ["rev-parse", "HEAD"])).trim(); +function commitFiles(cwd: string, files: Record<string, string>, message: string) { + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const { dirname, join } = yield* Path.Path; + for (const [relativePath, content] of Object.entries(files)) { + const fullPath = join(cwd, relativePath); + yield* fs.makeDirectory(dirname(fullPath), { recursive: true }); + yield* fs.writeFileString(fullPath, content); + } + yield* git(cwd, ["add", ...Object.keys(files)]); + yield* git(cwd, ["commit", "-m", message]); + return (yield* git(cwd, ["rev-parse", "HEAD"])).trim(); + }); } function fakeCommit(hash: string, message: string): Commit { @@ -103,175 +114,230 @@ describe("semantic-release-path-filter", () => { let hashNonAsciiPath: string; let hashMerge: string; - beforeAll(async () => { - repoDir = await mkdtemp(join(tmpdir(), "semantic-release-path-filter-")); - await git(repoDir, ["init", "-b", "main", "-q"]); - - // Seed a plain root commit so the main scenario commits all exercise the - // ordinary (parented) diff path; the root-commit case (`--root`) has its - // own dedicated test below. - await commitFiles(repoDir, { "README.md": "seed\n" }, "chore: seed repo root commit"); - - hashConfigOnly = await commitFiles( - repoDir, - { "packages/config/src/foo.ts": "export const foo = 1;\n" }, - "chore: seed packages/config/src/foo.ts", - ); - hashCliOnly = await commitFiles( - repoDir, - { "apps/cli/src/bar.ts": "export const bar = 1;\n" }, - "chore: seed apps/cli/src/bar.ts", - ); - hashBoth = await commitFiles( - repoDir, - { - "packages/config/src/foo2.ts": "export const foo2 = 1;\n", - "apps/cli/src/bar2.ts": "export const bar2 = 1;\n", - }, - "chore: seed both packages/config and apps/cli files", - ); - hashPrefixTrap = await commitFiles( - repoDir, - { "packages/config-other/x.ts": "export const x = 1;\n" }, - "chore: seed packages/config-other, a prefix-adjacent trap", - ); - hashNonAsciiPath = await commitFiles( - repoDir, - { "packages/config/src/café.ts": "export const café = 1;\n" }, - "chore: seed a non-ASCII path under packages/config", - ); - - await git(repoDir, ["checkout", "-b", "feature", "-q"]); - await commitFiles( - repoDir, - { "packages/config/src/on-branch.ts": "export const onBranch = 1;\n" }, - "chore: seed a commit on the feature branch", - ); - await git(repoDir, ["checkout", "main", "-q"]); - await git(repoDir, ["merge", "--no-ff", "feature", "-m", "merge: merge feature branch"]); - hashMerge = (await git(repoDir, ["rev-parse", "HEAD"])).trim(); - }); + beforeAll(() => + runTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + repoDir = yield* fs.makeTempDirectory({ prefix: "semantic-release-path-filter-" }); + yield* git(repoDir, ["init", "-b", "main", "-q"]); + + // Seed a plain root commit so the main scenario commits all exercise the + // ordinary (parented) diff path; the root-commit case (`--root`) has its + // own dedicated test below. + yield* commitFiles(repoDir, { "README.md": "seed\n" }, "chore: seed repo root commit"); + + hashConfigOnly = yield* commitFiles( + repoDir, + { "packages/config/src/foo.ts": "export const foo = 1;\n" }, + "chore: seed packages/config/src/foo.ts", + ); + hashCliOnly = yield* commitFiles( + repoDir, + { "apps/cli/src/bar.ts": "export const bar = 1;\n" }, + "chore: seed apps/cli/src/bar.ts", + ); + hashBoth = yield* commitFiles( + repoDir, + { + "packages/config/src/foo2.ts": "export const foo2 = 1;\n", + "apps/cli/src/bar2.ts": "export const bar2 = 1;\n", + }, + "chore: seed both packages/config and apps/cli files", + ); + hashPrefixTrap = yield* commitFiles( + repoDir, + { "packages/config-other/x.ts": "export const x = 1;\n" }, + "chore: seed packages/config-other, a prefix-adjacent trap", + ); + hashNonAsciiPath = yield* commitFiles( + repoDir, + { "packages/config/src/café.ts": "export const café = 1;\n" }, + "chore: seed a non-ASCII path under packages/config", + ); - afterAll(async () => { - await rm(repoDir, { recursive: true, force: true }); - }); + yield* git(repoDir, ["checkout", "-b", "feature", "-q"]); + yield* commitFiles( + repoDir, + { "packages/config/src/on-branch.ts": "export const onBranch = 1;\n" }, + "chore: seed a commit on the feature branch", + ); + yield* git(repoDir, ["checkout", "main", "-q"]); + yield* git(repoDir, ["merge", "--no-ff", "feature", "-m", "merge: merge feature branch"]); + hashMerge = (yield* git(repoDir, ["rev-parse", "HEAD"])).trim(); + }), + ), + ); + + afterAll(() => + runTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + yield* fs.remove(repoDir, { recursive: true, force: true }); + }), + ), + ); test("PACKAGE_PATH_PREFIX is the packages/config/ prefix the release train filters commits against", () => { expect(PACKAGE_PATH_PREFIX).toBe("packages/config/"); }); describe("filterCommitsToPackage", () => { - test("keeps only commits whose diff touches packages/config/**, preserving the input's order", async () => { - const shuffledInput = [hashCliOnly, hashBoth, hashPrefixTrap, hashMerge, hashConfigOnly].map( - (hash) => ({ - hash, + test("keeps only commits whose diff touches packages/config/**, preserving the input's order", () => + runTest( + Effect.gen(function* () { + const shuffledInput = [ + hashCliOnly, + hashBoth, + hashPrefixTrap, + hashMerge, + hashConfigOnly, + ].map((hash) => ({ + hash, + })); + + const result = yield* Effect.promise(() => + filterCommitsToPackage(shuffledInput, repoDir), + ); + + expect(result).toEqual([{ hash: hashBoth }, { hash: hashConfigOnly }]); }), - ); - - const result = await filterCommitsToPackage(shuffledInput, repoDir); - - expect(result).toEqual([{ hash: hashBoth }, { hash: hashConfigOnly }]); - }); - - test("excludes a merge commit even though the branch it merged touched packages/config/**", async () => { - const result = await filterCommitsToPackage([{ hash: hashMerge }], repoDir); - - expect(result).toEqual([]); - }); + )); - test("does not treat packages/config-other/ as a match for the packages/config/ prefix", async () => { - const result = await filterCommitsToPackage([{ hash: hashPrefixTrap }], repoDir); + test("excludes a merge commit even though the branch it merged touched packages/config/**", () => + runTest( + Effect.gen(function* () { + const result = yield* Effect.promise(() => + filterCommitsToPackage([{ hash: hashMerge }], repoDir), + ); - expect(result).toEqual([]); - }); + expect(result).toEqual([]); + }), + )); - test("returns an empty array for empty input", async () => { - const result = await filterCommitsToPackage([], repoDir); + test("does not treat packages/config-other/ as a match for the packages/config/ prefix", () => + runTest( + Effect.gen(function* () { + const result = yield* Effect.promise(() => + filterCommitsToPackage([{ hash: hashPrefixTrap }], repoDir), + ); - expect(result).toEqual([]); - }); + expect(result).toEqual([]); + }), + )); - test("includes a commit whose only config path is non-ASCII (core.quotePath would C-quote it without -z)", async () => { - const result = await filterCommitsToPackage([{ hash: hashNonAsciiPath }], repoDir); + test("returns an empty array for empty input", () => + runTest( + Effect.gen(function* () { + const result = yield* Effect.promise(() => filterCommitsToPackage([], repoDir)); - expect(result).toEqual([{ hash: hashNonAsciiPath }]); - }); - - test("includes a root commit that touches packages/config/** (--root diffs it against the empty tree)", async () => { - const rootRepoDir = await mkdtemp(join(tmpdir(), "semantic-release-path-filter-root-")); - try { - await git(rootRepoDir, ["init", "-b", "main", "-q"]); - const rootHash = await commitFiles( - rootRepoDir, - { "packages/config/src/first.ts": "export const first = 1;\n" }, - "chore: repo root commit touching packages/config", - ); + expect(result).toEqual([]); + }), + )); - const result = await filterCommitsToPackage([{ hash: rootHash }], rootRepoDir); + test("includes a commit whose only config path is non-ASCII (core.quotePath would C-quote it without -z)", () => + runTest( + Effect.gen(function* () { + const result = yield* Effect.promise(() => + filterCommitsToPackage([{ hash: hashNonAsciiPath }], repoDir), + ); - expect(result).toEqual([{ hash: rootHash }]); - } finally { - await rm(rootRepoDir, { recursive: true, force: true }); - } - }); + expect(result).toEqual([{ hash: hashNonAsciiPath }]); + }), + )); + + test("includes a root commit that touches packages/config/** (--root diffs it against the empty tree)", () => + runTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const rootRepoDir = yield* fs.makeTempDirectoryScoped({ + prefix: "semantic-release-path-filter-root-", + }); + + yield* git(rootRepoDir, ["init", "-b", "main", "-q"]); + const rootHash = yield* commitFiles( + rootRepoDir, + { "packages/config/src/first.ts": "export const first = 1;\n" }, + "chore: repo root commit touching packages/config", + ); + + const result = yield* Effect.promise(() => + filterCommitsToPackage([{ hash: rootHash }], rootRepoDir), + ); + + expect(result).toEqual([{ hash: rootHash }]); + }), + )); - test("rejects an abbreviated hash instead of silently dropping its commit (diff-tree echoes full OIDs)", async () => { - await expect( + test("rejects an abbreviated hash instead of silently dropping its commit (diff-tree echoes full OIDs)", () => + expect( filterCommitsToPackage([{ hash: hashConfigOnly.slice(0, 7) }], repoDir), - ).rejects.toThrow(/full lowercase hex object IDs/); - }); - - test("rejects with a descriptive error when git diff-tree exits non-zero", async () => { - const notARepo = await mkdtemp(join(tmpdir(), "semantic-release-path-filter-not-a-repo-")); - try { - await expect(filterCommitsToPackage([{ hash: "a".repeat(40) }], notARepo)).rejects.toThrow( - /^git diff-tree --stdin failed with exit code 128: /, - ); - } finally { - await rm(notARepo, { recursive: true, force: true }); - } - }); + ).rejects.toThrow(/full lowercase hex object IDs/)); + + test("rejects with a descriptive error when git diff-tree exits non-zero", () => + runTest( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const notARepo = yield* fs.makeTempDirectoryScoped({ + prefix: "semantic-release-path-filter-not-a-repo-", + }); + + yield* Effect.promise(() => + expect(filterCommitsToPackage([{ hash: "a".repeat(40) }], notARepo)).rejects.toThrow( + /^git diff-tree --stdin failed with exit code 128: /, + ), + ); + }), + )); }); describe("analyzeCommits", () => { - test('resolves "minor" for a feat(config) commit whose diff touches packages/config', async () => { - const context = fakeAnalyzeCommitsContext( - [fakeCommit(hashConfigOnly, "feat(config): add a new config option")], - repoDir, - ); + test('resolves "minor" for a feat(config) commit whose diff touches packages/config', () => + runTest( + Effect.gen(function* () { + const context = fakeAnalyzeCommitsContext( + [fakeCommit(hashConfigOnly, "feat(config): add a new config option")], + repoDir, + ); - const result = await analyzeCommits({}, context); + const result = yield* Effect.promise(() => analyzeCommits({}, context)); - expect(result).toBe("minor"); - }); - - test("resolves null when the only commits are fix commits touching apps/cli, not packages/config", async () => { - const context = fakeAnalyzeCommitsContext( - [fakeCommit(hashCliOnly, "fix: correct an unrelated cli bug")], - repoDir, - ); - - const result = await analyzeCommits({}, context); - - expect(result).toBeNull(); - }); + expect(result).toBe("minor"); + }), + )); - test('resolves "patch", not "major", because the breaking-change commit outside packages/config is filtered out', async () => { - const context = fakeAnalyzeCommitsContext( - [ - fakeCommit(hashConfigOnly, "fix: correct a bug in the config parser"), - fakeCommit( - hashCliOnly, - "feat!: drop legacy cli flag\n\nBREAKING CHANGE: removes the legacy flag entirely", - ), - ], - repoDir, - ); + test("resolves null when the only commits are fix commits touching apps/cli, not packages/config", () => + runTest( + Effect.gen(function* () { + const context = fakeAnalyzeCommitsContext( + [fakeCommit(hashCliOnly, "fix: correct an unrelated cli bug")], + repoDir, + ); - const result = await analyzeCommits({}, context); + const result = yield* Effect.promise(() => analyzeCommits({}, context)); - expect(result).toBe("patch"); - }); + expect(result).toBeNull(); + }), + )); + + test('resolves "patch", not "major", because the breaking-change commit outside packages/config is filtered out', () => + runTest( + Effect.gen(function* () { + const context = fakeAnalyzeCommitsContext( + [ + fakeCommit(hashConfigOnly, "fix: correct a bug in the config parser"), + fakeCommit( + hashCliOnly, + "feat!: drop legacy cli flag\n\nBREAKING CHANGE: removes the legacy flag entirely", + ), + ], + repoDir, + ); + + const result = yield* Effect.promise(() => analyzeCommits({}, context)); + + expect(result).toBe("patch"); + }), + )); }); describe("generateNotes", () => { @@ -298,19 +364,22 @@ describe("semantic-release-path-filter", () => { }; } - test("the notes mention the config commit and omit the commit that only touched apps/cli", async () => { - const context = fakeGenerateNotesContext( - [ - fakeCommit(hashConfigOnly, "feat(config): add a new config option"), - fakeCommit(hashCliOnly, "fix(cli): unrelated cli bug that must not appear"), - ], - repoDir, - ); - - const notes = await generateNotes({}, context); - - expect(notes).toContain("add a new config option"); - expect(notes).not.toContain("unrelated cli bug that must not appear"); - }); + test("the notes mention the config commit and omit the commit that only touched apps/cli", () => + runTest( + Effect.gen(function* () { + const context = fakeGenerateNotesContext( + [ + fakeCommit(hashConfigOnly, "feat(config): add a new config option"), + fakeCommit(hashCliOnly, "fix(cli): unrelated cli bug that must not appear"), + ], + repoDir, + ); + + const notes = yield* Effect.promise(() => generateNotes({}, context)); + + expect(notes).toContain("add a new config option"); + expect(notes).not.toContain("unrelated cli bug that must not appear"); + }), + )); }); }); From b1c2e5c1beeb9968752b976ead984c3c32e407f1 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Wed, 21 Oct 2026 10:59:12 +0000 Subject: [PATCH 11/24] fix(config): explain config parse errors and reject inf/nan (CLI-2755) (#7267) ## TL;DR a broken config now says what is wrong and where, and `inf` or `nan` numbers no longer load ## whats hurting the user? `supabase start`, `status`, `stop` and `functions serve` print a bare `CliConfigParseError` for any broken config. `config push`, `gen types`, `seed buckets` and the rest print the raw parser output, which quotes nearby lines of the file, secrets included. `max_rows = inf` or a `nan` rate limit loads quietly and reaches the containers as `Infinity` ## now fixed by? every command prints the field path or the line and column, never the file contents. number fields only take finite values, so the json schema post processing that hid the `inf` and `nan` variants goes away. > [!NOTE] > a config with `inf` or `nan` in a number field used to load and now fails, so set a real number instead. ## ref: - closes: CLI-2756 - closes: CLI-2755 --- .oxlintrc.effect.json | 18 +- .../command-internal/db-config.toml-read.ts | 3 +- .../command-internal/local-project-context.ts | 19 +- .../local-project-context.unit.test.ts | 31 +++ apps/cli/src/command-internal/seed-buckets.ts | 4 +- apps/cli/src/commands/config/config.load.ts | 3 +- .../config/push/push.integration.test.ts | 6 +- .../commands/gen/gen.signing-keys-config.ts | 7 +- .../src/commands/gen/types/types.handler.ts | 3 +- .../commands/inspect/report/report.config.ts | 3 +- .../src/commands/secrets/set/set.handler.ts | 12 +- .../secrets/set/set.integration.test.ts | 4 +- .../status/status.integration.test.ts | 3 + .../cli/src/commands/storage/storage.frame.ts | 8 +- apps/cli/src/shared/output/normalize-error.ts | 21 ++ .../output/normalize-error.unit.test.ts | 20 +- packages/config/AGENTS.md | 8 +- .../scripts/build-artifacts.unit.test.ts | 49 ++--- packages/config/scripts/build.ts | 47 +--- .../config/scripts/json-schema-postprocess.ts | 202 ------------------ .../json-schema-postprocess.unit.test.ts | 108 +--------- packages/config/src/analytics.ts | 4 +- packages/config/src/api.ts | 4 +- packages/config/src/auth/email.ts | 6 +- packages/config/src/auth/index.ts | 6 +- packages/config/src/auth/mfa.ts | 4 +- packages/config/src/auth/rate_limit.ts | 14 +- packages/config/src/auth/sms.ts | 4 +- packages/config/src/compute.ts | 15 +- packages/config/src/db.ts | 28 +-- packages/config/src/edge_runtime.ts | 4 +- packages/config/src/inbucket.ts | 6 +- packages/config/src/io.unit.test.ts | 5 + .../src/project-config/api-attributes.ts | 41 ++-- .../project-config.unit.test.ts | 13 +- .../src/project-config/registry-auth.ts | 2 +- .../config/src/project-config/registry.ts | 5 +- packages/config/src/realtime.ts | 2 +- packages/config/src/storage.ts | 12 +- packages/config/src/studio.ts | 2 +- 40 files changed, 252 insertions(+), 504 deletions(-) diff --git a/.oxlintrc.effect.json b/.oxlintrc.effect.json index d85a5d5b1f..edf2b3ef1c 100644 --- a/.oxlintrc.effect.json +++ b/.oxlintrc.effect.json @@ -136,22 +136,6 @@ "apps/cli/src/command-internal/config-validate.ts", "apps/cli/src/command-internal/config-validate.unit.test.ts", "apps/cli/src/command-internal/db-bootstrap/local-db-running.integration.test.ts", - "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts", - "packages/config/scripts/json-schema-postprocess.unit.test.ts", - "packages/config/src/analytics.ts", - "packages/config/src/api.ts", - "packages/config/src/auth/email.ts", - "packages/config/src/auth/index.ts", - "packages/config/src/auth/mfa.ts", - "packages/config/src/auth/rate_limit.ts", - "packages/config/src/auth/sms.ts", - "packages/config/src/compute.ts", - "packages/config/src/db.ts", - "packages/config/src/edge_runtime.ts", - "packages/config/src/inbucket.ts", - "packages/config/src/project-config/api-attributes.ts", - "packages/config/src/realtime.ts", - "packages/config/src/storage.ts", - "packages/config/src/studio.ts" + "apps/cli/src/command-internal/db-bootstrap/local-db-running.ts" ] } diff --git a/apps/cli/src/command-internal/db-config.toml-read.ts b/apps/cli/src/command-internal/db-config.toml-read.ts index cb9cbc4558..5134fbd1b2 100644 --- a/apps/cli/src/command-internal/db-config.toml-read.ts +++ b/apps/cli/src/command-internal/db-config.toml-read.ts @@ -33,6 +33,7 @@ import { import { ramInBytes } from "./size-units.ts"; import { collectDotenvPrivateKeys, decryptSecret, isEncryptedSecret } from "./vault-decrypt.ts"; import { Output } from "../shared/output/output.service.ts"; +import { configParseCauseMessage } from "../shared/output/normalize-error.ts"; /** Resolves a config `env(VAR)` reference: shell env first, then project `.env`. */ type EnvLookup = (name: string) => string | undefined; @@ -1153,7 +1154,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( try: () => asRecord(SmolToml.parse(maybeContent.value)), catch: (cause) => new DbConfigLoadError({ - message: `failed to load config: ${cause instanceof Error ? cause.message : String(cause)}`, + message: `failed to load config: ${configParseCauseMessage(cause)}`, }), }); // Same per-section promotion as the config loader, before the remote merge; the loader owns diff --git a/apps/cli/src/command-internal/local-project-context.ts b/apps/cli/src/command-internal/local-project-context.ts index d758c13f71..b223e10253 100644 --- a/apps/cli/src/command-internal/local-project-context.ts +++ b/apps/cli/src/command-internal/local-project-context.ts @@ -5,7 +5,17 @@ import { type CliConfig, } from "@supabase/config/effect"; import { loadCliConfig } from "@supabase/config/internal"; -import { Config, Console, Crypto, Effect, FileSystem, Option, Path, Schema } from "effect"; +import { + Config, + Console, + Crypto, + Effect, + FileSystem, + Option, + Path, + Predicate, + Schema, +} from "effect"; import { recordOrioleDbTelemetry } from "./db-image.ts"; import { resolveLocalProjectId, sanitizeProjectId } from "./docker-ids.ts"; @@ -17,6 +27,7 @@ import { envOverrideOrioleDBVersion, } from "./local-config-values.ts"; import { resolveProjectEnvironmentValues } from "./project-environment.ts"; +import { configParseCauseMessage } from "../shared/output/normalize-error.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; /** Config, resolved project env values, hostname, and sanitized project id for a command. */ @@ -87,7 +98,11 @@ export const loadLocalProjectContext = <E>( goViperCompat: true, projectRef, }).pipe( - Effect.mapError((cause) => mapConfigLoadError(`failed to read config: ${String(cause)}`)), + Effect.mapError((cause) => + mapConfigLoadError( + `failed to read config: ${Predicate.isTagged(cause, "CliConfigParseError") ? configParseCauseMessage(cause.cause) : String(cause)}`, + ), + ), ); // Printed beside the loader's deprecated-config WARNs, on the same real console, so it shares // their once-per-context-load cadence. diff --git a/apps/cli/src/command-internal/local-project-context.unit.test.ts b/apps/cli/src/command-internal/local-project-context.unit.test.ts index fbf3c9d9aa..6491930d7c 100644 --- a/apps/cli/src/command-internal/local-project-context.unit.test.ts +++ b/apps/cli/src/command-internal/local-project-context.unit.test.ts @@ -267,6 +267,37 @@ describe("loadLocalProjectContext", () => { ); }); + it.effect("names the failing field when supabase/config.toml fails to decode", () => { + const workdir = tempRoot.current; + return Effect.gen(function* () { + yield* writeConfigToml(workdir, '[db]\nport = "abc"\n'); + + const error = yield* loadLocalProjectContext( + workdir, + (message) => new TestError({ message }), + ).pipe(Effect.flip); + expect(error.message).toBe('failed to read config: Expected number\n at ["db"]["port"]'); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))); + }); + + it.effect("reports a TOML syntax error's position without quoting the file", () => { + const workdir = tempRoot.current; + return Effect.gen(function* () { + yield* writeConfigToml( + workdir, + '[edge_runtime.secrets]\nAPI_KEY = "sk_live_123"\nbroken =\n', + ); + + const error = yield* loadLocalProjectContext( + workdir, + (message) => new TestError({ message }), + ).pipe(Effect.flip); + expect(error.message).toBe( + "failed to read config: Invalid TOML document: incomplete declaration: value expected (line 3, column 9)", + ); + }).pipe(Effect.provide(Layer.mergeAll(BunServices.layer, runtimeInfoLayer))); + }); + it.effect("keeps the Error: prefix when a project-root .env is malformed", () => { const workdir = tempRoot.current; return Effect.gen(function* () { diff --git a/apps/cli/src/command-internal/seed-buckets.ts b/apps/cli/src/command-internal/seed-buckets.ts index 48cf490acf..3d0b307e4d 100644 --- a/apps/cli/src/command-internal/seed-buckets.ts +++ b/apps/cli/src/command-internal/seed-buckets.ts @@ -9,6 +9,7 @@ import { resolveYesWithProjectEnv } from "./global-flags.ts"; import { CommandSettings } from "../config/command-settings.service.ts"; import { bold, yellow } from "./colors.ts"; import { loadProjectEnv } from "./db-config.toml-read.ts"; +import { relativeConfigPath } from "./workdir-project.ts"; import { shouldSearchAncestors } from "./workdir-search.ts"; import { promptYesNo } from "./prompt-yes-no.ts"; import { @@ -33,6 +34,7 @@ import { } from "../commands/seed/buckets/buckets.classify.ts"; import { SeedConfigLoadError } from "../commands/seed/buckets/buckets.errors.ts"; import { bucketObjectKey } from "../commands/seed/buckets/buckets.upload.ts"; +import { configParseCauseMessage } from "../shared/output/normalize-error.ts"; const CONFIG_PATH = "supabase/config.toml"; const UPLOAD_CONCURRENCY = 5; @@ -178,7 +180,7 @@ export const seedBucketsRun = Effect.fnUntraced(function* (opts: { "CliConfigParseError", (cause) => new SeedConfigLoadError({ - message: `failed to parse supabase/config.toml: ${String(cause.cause)}`, + message: `failed to parse ${relativeConfigPath(workdir, cause.path)}: ${configParseCauseMessage(cause.cause)}`, }), ), ); diff --git a/apps/cli/src/commands/config/config.load.ts b/apps/cli/src/commands/config/config.load.ts index 164356ec55..6d4881a37b 100644 --- a/apps/cli/src/commands/config/config.load.ts +++ b/apps/cli/src/commands/config/config.load.ts @@ -6,6 +6,7 @@ import { relativeConfigPath, } from "../../command-internal/workdir-project.ts"; import { shouldSearchAncestors } from "../../command-internal/workdir-search.ts"; +import { configParseCauseMessage } from "../../shared/output/normalize-error.ts"; export { relativeConfigPath }; @@ -31,7 +32,7 @@ export function loadLocalConfig<E>( CliConfigParseError: (cause) => Effect.fail( makeError( - `failed to parse ${relativeConfigPath(cliSettings.workdir, cause.path)}: ${String(cause.cause)}`, + `failed to parse ${relativeConfigPath(cliSettings.workdir, cause.path)}: ${configParseCauseMessage(cause.cause)}`, ), ), DuplicateRemoteProjectIdError: (cause) => Effect.fail(makeError(cause.message)), diff --git a/apps/cli/src/commands/config/push/push.integration.test.ts b/apps/cli/src/commands/config/push/push.integration.test.ts index ed38a5e943..43e036e71c 100644 --- a/apps/cli/src/commands/config/push/push.integration.test.ts +++ b/apps/cli/src/commands/config/push/push.integration.test.ts @@ -479,7 +479,9 @@ describe("config push integration", () => { const message = yield* configPush({ projectRef: Option.none() }).pipe( Effect.catchTag("ConfigPushLoadConfigError", (error) => Effect.succeed(error.message)), ); - expect(message).toContain("failed to parse supabase/config.toml:"); + expect(message).toBe( + "failed to parse supabase/config.toml: Invalid TOML document: incomplete key-value: cannot find end of key (line 1, column 1)", + ); expect(api.requests).toHaveLength(0); }).pipe(Effect.provide(layer)); }); @@ -504,7 +506,7 @@ describe("config push integration", () => { const message = yield* configPush({ projectRef: Option.none() }).pipe( Effect.catchTag("ConfigPushLoadConfigError", (error) => Effect.succeed(error.message)), ); - expect(message).toContain("failed to parse supabase/config.json:"); + expect(message).toBe("failed to parse supabase/config.json: JSON Parse error: Expected '}'"); expect(api.requests).toHaveLength(0); }).pipe(Effect.provide(layer)); }); diff --git a/apps/cli/src/commands/gen/gen.signing-keys-config.ts b/apps/cli/src/commands/gen/gen.signing-keys-config.ts index 6dcfb4f769..8d2eed3dfe 100644 --- a/apps/cli/src/commands/gen/gen.signing-keys-config.ts +++ b/apps/cli/src/commands/gen/gen.signing-keys-config.ts @@ -4,6 +4,7 @@ import { Config, Effect, FileSystem, Option, Path, Predicate } from "effect"; import { assertDecodableJwkAlgorithm } from "../../command-internal/go-jwt.ts"; import { goJsonKindName } from "../../command-internal/go-json.ts"; import { resolveProjectEnvironmentValues } from "../../command-internal/project-environment.ts"; +import { configParseCauseMessage } from "../../shared/output/normalize-error.ts"; /** * Shared `[auth].signing_keys_path` config-loading logic for `gen signing-key` and `gen @@ -347,7 +348,11 @@ export const resolveSigningKeysConfigPaths = Effect.fnUntraced(function* <E>( tomlOnly: true, }).pipe( Effect.catchTag("CliConfigParseError", (cause) => - Effect.fail(onConfigParseError(`failed to parse ${cause.path}: ${String(cause.cause)}`)), + Effect.fail( + onConfigParseError( + `failed to parse ${cause.path}: ${configParseCauseMessage(cause.cause)}`, + ), + ), ), ); if (loaded === null) { diff --git a/apps/cli/src/commands/gen/types/types.handler.ts b/apps/cli/src/commands/gen/types/types.handler.ts index f4aebf1422..1455185b46 100644 --- a/apps/cli/src/commands/gen/types/types.handler.ts +++ b/apps/cli/src/commands/gen/types/types.handler.ts @@ -76,6 +76,7 @@ import { parseQueryTimeoutMillis, rootCaBundle, } from "./types.shared.ts"; +import { configParseCauseMessage } from "../../../shared/output/normalize-error.ts"; const mapProjectTypesError = mapHttpError({ networkError: GenTypesNetworkError, @@ -274,7 +275,7 @@ export const genTypes = Effect.fn("gen.types")(function* (flags: GenTypesFlags) Effect.catchTags({ CliConfigParseError: (cause) => new GenTypesParseConfigError({ - message: `failed to parse ${toRelativeConfigPath(cause.path)}: ${String(cause.cause)}`, + message: `failed to parse ${toRelativeConfigPath(cause.path)}: ${configParseCauseMessage(cause.cause)}`, }), DuplicateRemoteProjectIdError: (cause) => new GenTypesParseConfigError({ message: cause.message }), diff --git a/apps/cli/src/commands/inspect/report/report.config.ts b/apps/cli/src/commands/inspect/report/report.config.ts index f522556f75..f6cb38470e 100644 --- a/apps/cli/src/commands/inspect/report/report.config.ts +++ b/apps/cli/src/commands/inspect/report/report.config.ts @@ -8,6 +8,7 @@ import { loadProjectEnv, } from "../../../command-internal/db-config.toml-read.ts"; import type { InspectRule } from "./report.rules.ts"; +import { configParseCauseMessage } from "../../../shared/output/normalize-error.ts"; type RawDoc = { readonly [key: string]: unknown }; @@ -65,7 +66,7 @@ export const readInspectRules = Effect.fnUntraced(function* ( try: () => asRecord(SmolToml.parse(content)), catch: (cause) => new DbConfigLoadError({ - message: `failed to load config: ${cause instanceof Error ? cause.message : String(cause)}`, + message: `failed to load config: ${configParseCauseMessage(cause)}`, }), }); diff --git a/apps/cli/src/commands/secrets/set/set.handler.ts b/apps/cli/src/commands/secrets/set/set.handler.ts index 934a7176eb..6e848c0196 100644 --- a/apps/cli/src/commands/secrets/set/set.handler.ts +++ b/apps/cli/src/commands/secrets/set/set.handler.ts @@ -17,6 +17,7 @@ import { TelemetryState } from "../../../telemetry/telemetry-state.service.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; import { mapHttpError } from "../../../command-internal/http-errors.ts"; +import { configParseCauseMessage } from "../../../shared/output/normalize-error.ts"; import { InvalidSecretPairError, SecretsEnvFileOpenError, @@ -129,13 +130,10 @@ export const secretsSet = Effect.fn("secrets.set")(function* (flags: SecretsSetF }), Effect.catchTags({ CliConfigParseError: (cause) => { - // `smol-toml` embeds a source codeblock (which can include real secret values) after a - // blank-line separator on a raw parse failure; truncate before it. A schema-decode - // error puts the rejected value inline instead, with no such separator, so use a fixed, - // content-free message there. + /* A schema-decode issue path names the failing entry, which can be a secret's name. */ const shortMessage = cause.document === undefined - ? String(cause.cause).split("\n\n")[0] + ? configParseCauseMessage(cause.cause) : "schema validation failed"; // Printed here too since a matching `[remotes.*]` block is found before decode runs, // even though decode then failed. Emitted ahead of the debug log below to preserve @@ -146,7 +144,9 @@ export const secretsSet = Effect.fn("secrets.set")(function* (flags: SecretsSetF : Effect.void ).pipe( Effect.andThen( - debugLogger.debug(`failed to parse supabase/config.toml: ${shortMessage}`), + debugLogger.debug( + `failed to parse supabase/${path.basename(cause.path)}: ${shortMessage}`, + ), ), Effect.as(recoverEdgeRuntimeConfig(cause)), ); diff --git a/apps/cli/src/commands/secrets/set/set.integration.test.ts b/apps/cli/src/commands/secrets/set/set.integration.test.ts index a4a720c6ef..21d6ecb32c 100644 --- a/apps/cli/src/commands/secrets/set/set.integration.test.ts +++ b/apps/cli/src/commands/secrets/set/set.integration.test.ts @@ -926,8 +926,8 @@ project_id = "not-a-valid-ref" it.effect( "does not echo a literal secret value from config.toml into the debug log on a schema-decode error", () => { - // Unlike the syntax-error case above, a schema-decode failure has no separator to - // truncate: the rejected value appears inline on one line. The bad entry sits inside + // Unlike the syntax-error case above, a schema-decode issue path names the failing entry, + // which here is the secret's name. The bad entry sits inside // `[edge_runtime.secrets]` itself, so this also exercises the per-entry recovery path — // `PLANTED_SECRET` is dropped, but the CLI-arg secret still goes through. const { layer, api, debugLogger } = setup(); diff --git a/apps/cli/src/commands/status/status.integration.test.ts b/apps/cli/src/commands/status/status.integration.test.ts index 0859cd518f..8af2abe9b5 100644 --- a/apps/cli/src/commands/status/status.integration.test.ts +++ b/apps/cli/src/commands/status/status.integration.test.ts @@ -594,6 +594,9 @@ project_id = "previewrefaaaaaaaaaa" expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { expect(Cause.pretty(exit.cause)).toContain("StatusConfigLoadError"); + expect(Cause.pretty(exit.cause)).toContain( + "duplicate project_id for [remotes.b] and [remotes.a]", + ); } expect(child.spawned).toEqual([]); }).pipe(Effect.provide(layer)); diff --git a/apps/cli/src/commands/storage/storage.frame.ts b/apps/cli/src/commands/storage/storage.frame.ts index 4efae5ac45..595ad4dc91 100644 --- a/apps/cli/src/commands/storage/storage.frame.ts +++ b/apps/cli/src/commands/storage/storage.frame.ts @@ -14,7 +14,10 @@ import { parseStorageUrl, } from "../../command-internal/storage-url.ts"; import { StorageConfigError } from "../../command-internal/storage-credentials.errors.ts"; -import { missingProjectConfigMessageEffect } from "../../command-internal/workdir-project.ts"; +import { + missingProjectConfigMessageEffect, + relativeConfigPath, +} from "../../command-internal/workdir-project.ts"; import { shouldSearchAncestors } from "../../command-internal/workdir-search.ts"; import { validateWorkdirIsDirectory } from "../../command-internal/workdir-validation.ts"; import { @@ -23,6 +26,7 @@ import { StorageUrlParseError, StorageWorkdirError, } from "./storage.errors.ts"; +import { configParseCauseMessage } from "../../shared/output/normalize-error.ts"; /** * Shared plumbing for the four `storage` subcommands: resolving the project ref @@ -58,7 +62,7 @@ export const loadStorageConfig = Effect.fn("Storage.loadConfig")(function* ( "CliConfigParseError", (cause) => new StorageConfigError({ - message: `failed to parse supabase/config.toml: ${String(cause.cause)}`, + message: `failed to parse ${relativeConfigPath(cliSettings.workdir, cause.path)}: ${configParseCauseMessage(cause.cause)}`, }), ), ); diff --git a/apps/cli/src/shared/output/normalize-error.ts b/apps/cli/src/shared/output/normalize-error.ts index 6619fda663..3ed617daac 100644 --- a/apps/cli/src/shared/output/normalize-error.ts +++ b/apps/cli/src/shared/output/normalize-error.ts @@ -2,6 +2,7 @@ import { Cause, Option } from "effect"; import { CliError } from "effect/cli"; import { unwrapNativeFailure } from "../telemetry/error-actionability.ts"; import { formatInvalidValueMessage } from "../cli/invalid-value-message.ts"; +import { stripControlSequences } from "./strip-control-sequences.ts"; import type { CliErrorSuggestionContext } from "../cli/subcommand-flag-suggestions.ts"; import { formatCliErrorsForDisplay } from "../cli/subcommand-flag-suggestions.ts"; @@ -31,6 +32,17 @@ const readRawString = (value: ErrorRecord, key: string): string | undefined => { return typeof field === "string" ? field : undefined; }; +/** A config parse failure's reason and position, without the source text it quotes, which can hold secrets. */ +export function configParseCauseMessage(cause: unknown): string { + const text = cause instanceof Error ? cause.message : String(cause); + const reason = (stripControlSequences(text).split("\n\n")[0] ?? "").trimEnd(); + /* Only smol-toml's error carries a `codeblock`; Bun sets `line`/`column` on every `Error`. */ + if (isErrorRecord(cause) && typeof cause["codeblock"] === "string") { + return `${reason} (line ${String(cause["line"])}, column ${String(cause["column"])})`; + } + return cause instanceof SyntaxError ? reason.replace(/"[\s\S]*"/u, '"<redacted>"') : reason; +} + const mappedError = ( error: ErrorRecord, context?: CliErrorSuggestionContext, @@ -72,6 +84,15 @@ const mappedError = ( } return undefined; } + case "CliConfigParseError": { + const path = readString(error, "path"); + return path === undefined + ? undefined + : { + code: tag, + message: `failed to parse ${path}: ${configParseCauseMessage(error["cause"])}`, + }; + } case "UnknownSubcommand": return { code: "UnknownSubcommand", diff --git a/apps/cli/src/shared/output/normalize-error.unit.test.ts b/apps/cli/src/shared/output/normalize-error.unit.test.ts index ed2b48e619..8ab8ede6d2 100644 --- a/apps/cli/src/shared/output/normalize-error.unit.test.ts +++ b/apps/cli/src/shared/output/normalize-error.unit.test.ts @@ -26,16 +26,30 @@ describe("normalizeCliError", () => { }); }); - test("CliConfigParseError falls back to its bare tag as both code and message", () => { + test("CliConfigParseError keeps its tag as code and names the file and cause in its message", () => { const error = new CliConfigParseError({ path: "supabase/config.toml", format: "toml", - cause: new Error("unexpected token"), + cause: new Error("unexpected \u001b[2Jtoken\n"), }); expect(normalizeCliError(error)).toEqual({ code: "CliConfigParseError", - message: "CliConfigParseError", + message: "failed to parse supabase/config.toml: unexpected token", + }); + }); + + test("CliConfigParseError hides the token a JSON parse error quotes", () => { + const error = new CliConfigParseError({ + path: "supabase/config.json", + format: "json", + cause: new SyntaxError('JSON Parse error: Unexpected identifier "sk_live_123"'), + }); + + expect(normalizeCliError(error)).toEqual({ + code: "CliConfigParseError", + message: + 'failed to parse supabase/config.json: JSON Parse error: Unexpected identifier "<redacted>"', }); }); diff --git a/packages/config/AGENTS.md b/packages/config/AGENTS.md index 7d9f384033..f1c0b4eb45 100644 --- a/packages/config/AGENTS.md +++ b/packages/config/AGENTS.md @@ -94,8 +94,7 @@ surface must update that test deliberately — it is not meant to be a silent pa every `dist`/`types`/`default` export condition points at. 2. Renders both generated JSON Schema artifacts (`dist/schema.json`, `dist/project-schema.json`) from `toCliConfigJsonSchema()`/`toProjectConfigJsonSchema()`, post-processed (via - `scripts/json-schema-postprocess.ts`) to collapse Effect's non-finite-number `anyOf` encoding - back to a plain `number`/`integer` node and to add `$id`/`title`/`description`, then formatted + `scripts/json-schema-postprocess.ts`) to add `$id`/`title`/`description`, then formatted through `oxfmt`. 3. Verifies every `types`/non-`bun` `default` target (plus both JSON artifacts) declared in package.json's `exports` map actually exists on disk. @@ -145,8 +144,9 @@ own guarantees and must stay green after any entrypoint or type-surface change: directory. - `src/lib/resolve.unit.test.ts` — behavioral coverage of the public sync resolvers. - `scripts/json-schema-postprocess.unit.test.ts` / `scripts/build-artifacts.unit.test.ts` — the - JSON Schema post-processing `renderJsonSchema` applies (non-finite-number `anyOf` collapse, - `$id`/`title`/`description`), the second against the real generated documents. + `$id`/`title`/`description` metadata `withSchemaMetadata` adds, and, against the real generated + documents, that every config number schema is finite (no non-finite `anyOf` split reaches the + published artifacts). ## Releases (CLI-2233) diff --git a/packages/config/scripts/build-artifacts.unit.test.ts b/packages/config/scripts/build-artifacts.unit.test.ts index 28ed07aeff..f8da54e665 100644 --- a/packages/config/scripts/build-artifacts.unit.test.ts +++ b/packages/config/scripts/build-artifacts.unit.test.ts @@ -1,15 +1,12 @@ import { describe, expect, test } from "vitest"; -import { CliConfigSchema, toCliConfigJsonSchema } from "../src/base.ts"; -import { - ProjectConfigSchema, - toProjectConfigJsonSchema, -} from "../src/project-config/project-schema.ts"; +import { toCliConfigJsonSchema } from "../src/base.ts"; +import { toProjectConfigJsonSchema } from "../src/project-config/project-schema.ts"; import { CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL } from "../src/schema-metadata.ts"; -import { collapseNonFiniteNumberUnions, withSchemaMetadata } from "./json-schema-postprocess.ts"; +import { withSchemaMetadata } from "./json-schema-postprocess.ts"; -// Regression coverage for the post-processing `scripts/build.ts` applies to -// both `dist/schema.json` and `dist/project-schema.json`, generated in-memory -// here via the same pure functions the build script itself calls. +// Regression coverage for `dist/schema.json` and `dist/project-schema.json`, built in +// memory via the same pure functions `scripts/build.ts` calls: every number schema is +// finite, so no non-finite `anyOf` split reaches them. function isRecord(value: unknown): value is Record<string, unknown> { return typeof value === "object" && value !== null; @@ -46,31 +43,19 @@ function findAnyOfWithNonFiniteEnum(node: unknown, into: Array<unknown>): void { } } -const cliDocument = withSchemaMetadata( - collapseNonFiniteNumberUnions(toCliConfigJsonSchema(), CliConfigSchema.ast) as Record< - string, - unknown - >, - { - id: CLI_CONFIG_SCHEMA_URL, - title: "Supabase CLI config (CliConfig)", - description: "test", - }, -); +const cliDocument = withSchemaMetadata(toCliConfigJsonSchema(), { + id: CLI_CONFIG_SCHEMA_URL, + title: "Supabase CLI config (CliConfig)", + description: "test", +}); -const projectDocument = withSchemaMetadata( - collapseNonFiniteNumberUnions(toProjectConfigJsonSchema(), ProjectConfigSchema.ast) as Record< - string, - unknown - >, - { - id: PROJECT_CONFIG_SCHEMA_URL, - title: "Supabase hosted project config (ProjectConfig)", - description: "test", - }, -); +const projectDocument = withSchemaMetadata(toProjectConfigJsonSchema(), { + id: PROJECT_CONFIG_SCHEMA_URL, + title: "Supabase hosted project config (ProjectConfig)", + description: "test", +}); -describe("generated JSON Schema artifacts, post-processed", () => { +describe("published JSON Schema artifacts", () => { test.each([ ["schema.json", cliDocument], ["project-schema.json", projectDocument], diff --git a/packages/config/scripts/build.ts b/packages/config/scripts/build.ts index 79fd55421f..b3a7a01d3c 100644 --- a/packages/config/scripts/build.ts +++ b/packages/config/scripts/build.ts @@ -1,12 +1,9 @@ import { BunFileSystem, BunPath } from "@effect/platform-bun"; import { Console, Data, Effect, FileSystem, Layer, Path, type PlatformError, Schema } from "effect"; -import { CliConfigSchema, toCliConfigJsonSchema } from "../src/base.ts"; -import { - ProjectConfigSchema, - toProjectConfigJsonSchema, -} from "../src/project-config/project-schema.ts"; +import { toCliConfigJsonSchema } from "../src/base.ts"; +import { toProjectConfigJsonSchema } from "../src/project-config/project-schema.ts"; import { CLI_CONFIG_SCHEMA_URL, PROJECT_CONFIG_SCHEMA_URL } from "../src/schema-metadata.ts"; -import { collapseNonFiniteNumberUnions, withSchemaMetadata } from "./json-schema-postprocess.ts"; +import { withSchemaMetadata } from "./json-schema-postprocess.ts"; const packageRoot = Effect.gen(function* () { const path = yield* Path.Path; @@ -85,28 +82,6 @@ function readStringAt(root: unknown, path: ReadonlyArray<string>): string { return current; } -/** - * Runs a schema document through {@link collapseNonFiniteNumberUnions} and - * {@link withSchemaMetadata}, then writes it via {@link renderJsonSchema}. - * Narrows the collapse step's `unknown` result with `isRecord` instead of an - * `as` cast, since a non-object result here would mean the collapse walk - * itself is broken. - */ -const renderCollapsedJsonSchema = Effect.fnUntraced(function* ( - outputPath: string, - document: unknown, - rootAst: Parameters<typeof collapseNonFiniteNumberUnions>[1], - metadata: Parameters<typeof withSchemaMetadata>[1], -) { - const collapsed = collapseNonFiniteNumberUnions(document, rootAst); - if (!isRecord(collapsed)) { - return yield* Effect.die( - new Error(`collapseNonFiniteNumberUnions did not return an object for ${outputPath}`), - ); - } - yield* renderJsonSchema(outputPath, withSchemaMetadata(collapsed, metadata)); -}); - const renderJsonSchema = Effect.fnUntraced(function* ( outputPath: string, json: Record<string, unknown>, @@ -486,27 +461,23 @@ const build = Effect.gen(function* () { yield* runCommand(["pnpm", "exec", "tsc", "-p", "tsconfig.build.json"]); yield* Console.log("[build] rendering JSON Schema artifacts..."); - yield* renderCollapsedJsonSchema( + yield* renderJsonSchema( path.join(root, "dist", "schema.json"), - toCliConfigJsonSchema(), - CliConfigSchema.ast, - { + withSchemaMetadata(toCliConfigJsonSchema(), { id: CLI_CONFIG_SCHEMA_URL, title: "Supabase CLI config (CliConfig)", description: "The Supabase CLI's local project config document (supabase/config.toml or supabase/config.json).", - }, + }), ); - yield* renderCollapsedJsonSchema( + yield* renderJsonSchema( path.join(root, "dist", "project-schema.json"), - toProjectConfigJsonSchema(), - ProjectConfigSchema.ast, - { + withSchemaMetadata(toProjectConfigJsonSchema(), { id: PROJECT_CONFIG_SCHEMA_URL, title: "Supabase hosted project config (ProjectConfig)", description: "The sparse, hosted-project subset of CliConfig that a Supabase project manages.", - }, + }), ); yield* Console.log("[build] verifying every exports-map dist target exists..."); diff --git a/packages/config/scripts/json-schema-postprocess.ts b/packages/config/scripts/json-schema-postprocess.ts index ce2fc444ca..0c2365a749 100644 --- a/packages/config/scripts/json-schema-postprocess.ts +++ b/packages/config/scripts/json-schema-postprocess.ts @@ -1,210 +1,8 @@ -import { SchemaAST } from "effect"; - /** * Pure JSON Schema post-processing used by `build.ts`'s `renderJsonSchema` on * both generated artifacts (`dist/schema.json`, `dist/project-schema.json`). */ -function isRecord(value: unknown): value is Record<string, unknown> { - return typeof value === "object" && value !== null && !Array.isArray(value); -} - -const NON_FINITE_ENUM_VALUES: ReadonlySet<string> = new Set(["Infinity", "-Infinity", "NaN"]); - -function isNonFiniteStringEnumNode(node: unknown): boolean { - if (!isRecord(node) || node["type"] !== "string") { - return false; - } - const values = node["enum"]; - return ( - Array.isArray(values) && - values.length > 0 && - values.every((value) => typeof value === "string" && NON_FINITE_ENUM_VALUES.has(value)) - ); -} - -function isPlainNumberNode(node: unknown): node is Record<string, unknown> { - return isRecord(node) && node["type"] === "number"; -} - -interface RecoveredAnnotations { - readonly description?: string; - readonly default?: unknown; -} - -/** - * `description`/`default` for every `Schema.Number` leaf reachable from `ast`, - * keyed by dotted property path (`"*"` for a record/array element) — the only - * two annotations `Schema.toJsonSchemaDocument` drops when it splits a plain - * `Schema.Number` into the `anyOf` union {@link collapseNonFiniteNumberUnions} - * collapses back down. - */ -function collectNumberLeafAnnotations( - ast: SchemaAST.AST, - prefix: ReadonlyArray<string> = [], - into: Map<string, RecoveredAnnotations> = new Map(), -): Map<string, RecoveredAnnotations> { - if (SchemaAST.isObjects(ast)) { - for (const property of ast.propertySignatures) { - collectNumberLeafAnnotations(property.type, [...prefix, String(property.name)], into); - } - for (const indexSignature of ast.indexSignatures) { - collectNumberLeafAnnotations(indexSignature.type, [...prefix, "*"], into); - } - } else if (SchemaAST.isArrays(ast)) { - for (const element of ast.elements) { - collectNumberLeafAnnotations(element, [...prefix, "*"], into); - } - for (const rest of ast.rest) { - collectNumberLeafAnnotations(rest, [...prefix, "*"], into); - } - } else if (SchemaAST.isUnion(ast)) { - for (const member of ast.types) { - collectNumberLeafAnnotations(member, prefix, into); - } - } else if (SchemaAST.isNumber(ast)) { - const description = ast.annotations?.["description"]; - const defaultValue = ast.annotations?.["default"]; - if (typeof description === "string" || defaultValue !== undefined) { - into.set(prefix.join("."), { - ...(typeof description === "string" ? { description } : {}), - ...(defaultValue !== undefined ? { default: defaultValue } : {}), - }); - } - } - return into; -} - -function tryCollapseNonFiniteNumberUnion( - node: Record<string, unknown>, - path: ReadonlyArray<string>, - annotationsByPath: ReadonlyMap<string, RecoveredAnnotations>, -): Record<string, unknown> | undefined { - const anyOf = node["anyOf"]; - if (!Array.isArray(anyOf) || anyOf.length !== 2) { - return undefined; - } - const [first, second] = anyOf; - const numberNode = isPlainNumberNode(first) - ? first - : isPlainNumberNode(second) - ? second - : undefined; - const enumNode = isNonFiniteStringEnumNode(first) - ? first - : isNonFiniteStringEnumNode(second) - ? second - : undefined; - if (numberNode === undefined || enumNode === undefined) { - return undefined; - } - - const { anyOf: _anyOf, ...siblings } = node; - const merged: Record<string, unknown> = { ...numberNode, ...siblings }; - const recovered = annotationsByPath.get(path.join(".")); - if (merged["description"] === undefined && recovered?.description !== undefined) { - merged["description"] = recovered.description; - } - if (merged["default"] === undefined && recovered?.default !== undefined) { - merged["default"] = recovered.default; - } - return merged; -} - -function collapseSchemaNode( - node: unknown, - path: ReadonlyArray<string>, - annotationsByPath: ReadonlyMap<string, RecoveredAnnotations>, -): unknown { - if (!isRecord(node)) { - return node; - } - - const collapsed = tryCollapseNonFiniteNumberUnion(node, path, annotationsByPath); - if (collapsed !== undefined) { - return collapsed; - } - - const result: Record<string, unknown> = { ...node }; - - const properties = node["properties"]; - if (isRecord(properties)) { - result["properties"] = Object.fromEntries( - Object.entries(properties).map(([name, child]) => [ - name, - collapseSchemaNode(child, [...path, name], annotationsByPath), - ]), - ); - } - - const patternProperties = node["patternProperties"]; - if (isRecord(patternProperties)) { - result["patternProperties"] = Object.fromEntries( - Object.entries(patternProperties).map(([pattern, child]) => [ - pattern, - collapseSchemaNode(child, [...path, "*"], annotationsByPath), - ]), - ); - } - - const additionalProperties = node["additionalProperties"]; - if (isRecord(additionalProperties)) { - result["additionalProperties"] = collapseSchemaNode( - additionalProperties, - [...path, "*"], - annotationsByPath, - ); - } - - const items = node["items"]; - if (items !== undefined) { - result["items"] = collapseSchemaNode(items, [...path, "*"], annotationsByPath); - } - - const prefixItems = node["prefixItems"]; - if (Array.isArray(prefixItems)) { - result["prefixItems"] = prefixItems.map((item) => - collapseSchemaNode(item, [...path, "*"], annotationsByPath), - ); - } - - for (const combinator of ["anyOf", "allOf", "oneOf"] as const) { - const branches = node[combinator]; - if (Array.isArray(branches)) { - result[combinator] = branches.map((branch) => - collapseSchemaNode(branch, path, annotationsByPath), - ); - } - } - - const defs = node["$defs"]; - if (isRecord(defs)) { - result["$defs"] = Object.fromEntries( - Object.entries(defs).map(([name, child]) => [ - name, - // `$defs` members are keyed by ref name, not position, so `path` passes - // through unchanged; neither generated document currently emits `$defs`, - // so this is inert. - collapseSchemaNode(child, path, annotationsByPath), - ]), - ); - } - - return result; -} - -/** - * Collapses every `anyOf: [{ type: "number", ... }, { type: "string", enum: - * [subset of "Infinity"/"-Infinity"/"NaN"] }]` node in `document` down to the - * plain `{ type: "number", ... }` branch, re-attaching `description`/`default` - * from `rootAst` when the union node doesn't already carry them. `rootAst` must - * be the same schema `document` was rendered from. - */ -export function collapseNonFiniteNumberUnions(document: unknown, rootAst: SchemaAST.AST): unknown { - const annotationsByPath = collectNumberLeafAnnotations(rootAst); - return collapseSchemaNode(document, [], annotationsByPath); -} - /** * Injects `$id`/`title`/`description` right after `$schema`, ahead of the * document's own keys. `metadata` is authoritative: any `$id`/`title`/ diff --git a/packages/config/scripts/json-schema-postprocess.unit.test.ts b/packages/config/scripts/json-schema-postprocess.unit.test.ts index 294963c62f..8b17d79cf6 100644 --- a/packages/config/scripts/json-schema-postprocess.unit.test.ts +++ b/packages/config/scripts/json-schema-postprocess.unit.test.ts @@ -1,111 +1,5 @@ import { describe, expect, test } from "vitest"; -import { Schema } from "effect"; -import { collapseNonFiniteNumberUnions, withSchemaMetadata } from "./json-schema-postprocess.ts"; - -describe("collapseNonFiniteNumberUnions", () => { - test("collapses a top-level anyOf-with-non-finite-enum node to a plain number", () => { - const schema = Schema.Struct({ port: Schema.Number }); - const document = { - properties: { - port: { - anyOf: [{ type: "number" }, { type: "string", enum: ["Infinity", "-Infinity", "NaN"] }], - }, - }, - }; - - const result = collapseNonFiniteNumberUnions(document, schema.ast); - - expect(result).toEqual({ properties: { port: { type: "number" } } }); - }); - - test("re-attaches description/default from the source AST when missing on the union node", () => { - const schema = Schema.Struct({ - max_rows: Schema.Number.annotate({ description: "Row limit.", default: 1000 }), - }); - const document = { - properties: { - max_rows: { - anyOf: [{ type: "number" }, { type: "string", enum: ["Infinity", "-Infinity", "NaN"] }], - }, - }, - }; - - const result = collapseNonFiniteNumberUnions(document, schema.ast) as { - properties: { max_rows: Record<string, unknown> }; - }; - - expect(result.properties.max_rows).toEqual({ - type: "number", - description: "Row limit.", - default: 1000, - }); - }); - - test("does not override description/default already present on the union node", () => { - const schema = Schema.Struct({ - max_rows: Schema.Number.annotate({ description: "Row limit.", default: 1000 }), - }); - const document = { - properties: { - max_rows: { - anyOf: [{ type: "number" }, { type: "string", enum: ["Infinity", "-Infinity", "NaN"] }], - description: "Overridden description.", - }, - }, - }; - - const result = collapseNonFiniteNumberUnions(document, schema.ast) as { - properties: { max_rows: Record<string, unknown> }; - }; - - expect(result.properties.max_rows["description"]).toBe("Overridden description."); - expect(result.properties.max_rows["default"]).toBe(1000); - }); - - test("leaves an unrelated anyOf (e.g. object-or-null) untouched", () => { - const schema = Schema.Struct({ compute: Schema.Unknown }); - const document = { - properties: { - compute: { anyOf: [{ type: "object" }, { type: "null" }] }, - }, - }; - - const result = collapseNonFiniteNumberUnions(document, schema.ast); - - expect(result).toEqual(document); - }); - - test("descends through properties, patternProperties, items, and additionalProperties", () => { - const schema = Schema.Struct({ - list: Schema.Array(Schema.Number), - table: Schema.Record(Schema.String, Schema.Number), - }); - const nonFiniteNumberNode = { - anyOf: [{ type: "number" }, { type: "string", enum: ["Infinity", "-Infinity", "NaN"] }], - }; - const document = { - properties: { - list: { type: "array", items: nonFiniteNumberNode }, - table: { type: "object", patternProperties: { ".*": nonFiniteNumberNode } }, - }, - }; - - const result = collapseNonFiniteNumberUnions(document, schema.ast); - - expect(result).toEqual({ - properties: { - list: { type: "array", items: { type: "number" } }, - table: { type: "object", patternProperties: { ".*": { type: "number" } } }, - }, - }); - }); - - test("leaves a plain non-object document untouched", () => { - expect(collapseNonFiniteNumberUnions("not-a-schema-doc", Schema.String.ast)).toBe( - "not-a-schema-doc", - ); - }); -}); +import { withSchemaMetadata } from "./json-schema-postprocess.ts"; describe("withSchemaMetadata", () => { test("inserts $id/title/description right after $schema, ahead of the rest of the document", () => { diff --git a/packages/config/src/analytics.ts b/packages/config/src/analytics.ts index fd3cf12128..29f61f39f7 100644 --- a/packages/config/src/analytics.ts +++ b/packages/config/src/analytics.ts @@ -22,7 +22,7 @@ export const analytics = Schema.Struct({ tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnabled))), - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPort, description: "Port to the local Logflare service.", tags, @@ -40,7 +40,7 @@ export const analytics = Schema.Struct({ links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultBackend))), vector_port: Schema.optionalKey( - Schema.Number.annotate({ + Schema.Finite.annotate({ description: "Port to the local syslog ingest service.", tags, }), diff --git a/packages/config/src/api.ts b/packages/config/src/api.ts index eee055f05b..082a8a7a31 100644 --- a/packages/config/src/api.ts +++ b/packages/config/src/api.ts @@ -24,7 +24,7 @@ export const api = Schema.Struct({ tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnabled))), - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPort, description: "Port to use for the API URL.", tags, @@ -49,7 +49,7 @@ export const api = Schema.Struct({ ) .annotate({ default: defaultExtraSearchPath }) .pipe(Schema.withDecodingDefaultKey(Effect.succeed([...defaultExtraSearchPath]))), - max_rows: Schema.Number.annotate({ + max_rows: Schema.Finite.annotate({ default: defaultMaxRows, description: "The maximum number of rows returned from a view, table, or stored procedure. Limits payload size for accidental or malicious requests.", diff --git a/packages/config/src/auth/email.ts b/packages/config/src/auth/email.ts index 342d77e12f..eebc58d221 100644 --- a/packages/config/src/auth/email.ts +++ b/packages/config/src/auth/email.ts @@ -109,13 +109,13 @@ export const email = Schema.Struct({ tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultMaxFrequency))), - otp_length: Schema.Number.annotate({ + otp_length: Schema.Finite.annotate({ default: defaultOtpLength, description: "Number of characters used in the email OTP.", tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultOtpLength))), - otp_expiry: Schema.Number.annotate({ + otp_expiry: Schema.Finite.annotate({ default: defaultOtpExpiry, description: "Number of seconds before the email OTP expires.", tags, @@ -133,7 +133,7 @@ export const email = Schema.Struct({ }), ), port: Schema.optionalKey( - Schema.Number.annotate({ + Schema.Finite.annotate({ description: "Port number of the SMTP server.", }), ), diff --git a/packages/config/src/auth/index.ts b/packages/config/src/auth/index.ts index 2730a7d01c..6c3cff2625 100644 --- a/packages/config/src/auth/index.ts +++ b/packages/config/src/auth/index.ts @@ -66,7 +66,7 @@ export const auth = Schema.Struct({ links: [links.auth], }) .pipe(Schema.withDecodingDefaultKey(Effect.succeed([...defaultAdditionalRedirectUrls]))), - jwt_expiry: Schema.Number.annotate({ + jwt_expiry: Schema.Finite.annotate({ default: defaultJwtExpiry, description: "How long tokens are valid for, in seconds. Defaults to 3600 (1 hour), maximum 604,800 seconds (one week).", @@ -93,7 +93,7 @@ export const auth = Schema.Struct({ tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnableRefreshTokenRotation))), - refresh_token_reuse_interval: Schema.Number.annotate({ + refresh_token_reuse_interval: Schema.Finite.annotate({ default: defaultRefreshTokenReuseInterval, description: "Allows refresh tokens to be reused after expiry, up to the specified interval in seconds.", @@ -118,7 +118,7 @@ export const auth = Schema.Struct({ tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnableAnonymousSignIns))), - minimum_password_length: Schema.Number.annotate({ + minimum_password_length: Schema.Finite.annotate({ default: defaultMinimumPasswordLength, description: "Passwords shorter than this value will be rejected as weak.", tags, diff --git a/packages/config/src/auth/mfa.ts b/packages/config/src/auth/mfa.ts index 8a638a092f..c364059508 100644 --- a/packages/config/src/auth/mfa.ts +++ b/packages/config/src/auth/mfa.ts @@ -60,7 +60,7 @@ export const mfa = Schema.Struct({ tags, links: [links.phone], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPhoneVerifyEnabled))), - otp_length: Schema.Number.annotate({ + otp_length: Schema.Finite.annotate({ default: defaultPhoneOtpLength, description: "The length of the OTP code.", tags, @@ -93,7 +93,7 @@ export const mfa = Schema.Struct({ links: [links.mfa], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultWebAuthnVerifyEnabled))), }).pipe(Schema.withDecodingDefaultKey(Effect.succeed({ ...defaultWebAuthn }))), - max_enrolled_factors: Schema.Number.annotate({ + max_enrolled_factors: Schema.Finite.annotate({ default: defaultMaxEnrolledFactors, description: "The maximum number of MFA factors a user can enroll in.", tags, diff --git a/packages/config/src/auth/rate_limit.ts b/packages/config/src/auth/rate_limit.ts index e5ab60de50..e1a4ae1a8b 100644 --- a/packages/config/src/auth/rate_limit.ts +++ b/packages/config/src/auth/rate_limit.ts @@ -19,45 +19,45 @@ const defaultTokenVerifications = 30; const defaultWeb3 = 30; export const rate_limit = Schema.Struct({ - email_sent: Schema.Number.annotate({ + email_sent: Schema.Finite.annotate({ default: defaultEmailSent, description: "Number of emails that can be sent per hour.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEmailSent))), - sms_sent: Schema.Number.annotate({ + sms_sent: Schema.Finite.annotate({ default: defaultSmsSent, description: "Number of SMS messages that can be sent per hour.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultSmsSent))), - anonymous_users: Schema.Number.annotate({ + anonymous_users: Schema.Finite.annotate({ default: defaultAnonymousUsers, description: "Number of anonymous sign-ins that can be made per hour per IP address.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultAnonymousUsers))), - token_refresh: Schema.Number.annotate({ + token_refresh: Schema.Finite.annotate({ default: defaultTokenRefresh, description: "Number of sessions that can be refreshed in a 5 minute interval per IP address.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultTokenRefresh))), - sign_in_sign_ups: Schema.Number.annotate({ + sign_in_sign_ups: Schema.Finite.annotate({ default: defaultSignInSignUps, description: "Number of sign up and sign-in requests that can be made in a 5 minute interval per IP address.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultSignInSignUps))), - token_verifications: Schema.Number.annotate({ + token_verifications: Schema.Finite.annotate({ default: defaultTokenVerifications, description: "Number of OTP or magic link verifications that can be made in a 5 minute interval per IP address.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultTokenVerifications))), - web3: Schema.Number.annotate({ + web3: Schema.Finite.annotate({ default: defaultWeb3, description: "Number of Web3 logins that can be made in a 5 minute interval per IP address.", tags, diff --git a/packages/config/src/auth/sms.ts b/packages/config/src/auth/sms.ts index 439ad8ee58..b638f0c525 100644 --- a/packages/config/src/auth/sms.ts +++ b/packages/config/src/auth/sms.ts @@ -163,13 +163,13 @@ export const sms = Schema.Struct({ tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultMaxFrequency))), - otp_length: Schema.Number.annotate({ + otp_length: Schema.Finite.annotate({ default: defaultOtpLength, description: "Number of characters used in the SMS OTP.", tags, links: [links.auth], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultOtpLength))), - otp_expiry: Schema.Number.annotate({ + otp_expiry: Schema.Finite.annotate({ default: defaultOtpExpiry, description: "Number of seconds before the SMS OTP expires.", tags, diff --git a/packages/config/src/compute.ts b/packages/config/src/compute.ts index a19cc3dc95..57ed4b5427 100644 --- a/packages/config/src/compute.ts +++ b/packages/config/src/compute.ts @@ -47,18 +47,17 @@ const computeEntry = Schema.Struct({ instances: Schema.optionalKey( // Bounded to match the Management API's input schema; an out-of-range value would // be dropped rather than sent, deploying a different count than the config asked for. - Schema.Number.check( - Schema.isInt().annotate({ expected: "a whole number of instances" }), - Schema.isGreaterThanOrEqualTo(0).annotate({ expected: "zero or more instances" }), - ).annotate({ - description: dedent` + Schema.Int.annotate({ expected: "a whole number of instances" }) + .check(Schema.isGreaterThanOrEqualTo(0).annotate({ expected: "zero or more instances" })) + .annotate({ + description: dedent` Number of instances to run. Every deploy sends a complete spec, so a count recorded here is what keeps a scaled compute scaled; \`--instances\` overrides it for one deploy. Defaults to 1. `, - examples: [3], - tags, - }), + examples: [3], + tags, + }), ), source: Schema.optionalKey( Schema.String.annotate({ diff --git a/packages/config/src/db.ts b/packages/config/src/db.ts index 67f5d3219f..d78d17f759 100644 --- a/packages/config/src/db.ts +++ b/packages/config/src/db.ts @@ -40,18 +40,18 @@ const settings = Schema.Struct({ effective_cache_size: Schema.optionalKey(Schema.String), logical_decoding_work_mem: Schema.optionalKey(Schema.String), maintenance_work_mem: Schema.optionalKey(Schema.String), - max_connections: Schema.optionalKey(Schema.Number), - max_locks_per_transaction: Schema.optionalKey(Schema.Number), - max_parallel_maintenance_workers: Schema.optionalKey(Schema.Number), - max_parallel_workers: Schema.optionalKey(Schema.Number), - max_parallel_workers_per_gather: Schema.optionalKey(Schema.Number), - max_replication_slots: Schema.optionalKey(Schema.Number), + max_connections: Schema.optionalKey(Schema.Finite), + max_locks_per_transaction: Schema.optionalKey(Schema.Finite), + max_parallel_maintenance_workers: Schema.optionalKey(Schema.Finite), + max_parallel_workers: Schema.optionalKey(Schema.Finite), + max_parallel_workers_per_gather: Schema.optionalKey(Schema.Finite), + max_replication_slots: Schema.optionalKey(Schema.Finite), max_slot_wal_keep_size: Schema.optionalKey(Schema.String), max_standby_archive_delay: Schema.optionalKey(Schema.String), max_standby_streaming_delay: Schema.optionalKey(Schema.String), max_wal_size: Schema.optionalKey(Schema.String), - max_wal_senders: Schema.optionalKey(Schema.Number), - max_worker_processes: Schema.optionalKey(Schema.Number), + max_wal_senders: Schema.optionalKey(Schema.Finite), + max_worker_processes: Schema.optionalKey(Schema.Finite), session_replication_role: Schema.optionalKey( stringEnum(["origin", "replica", "local"], { description: "Session replication role.", @@ -68,13 +68,13 @@ const settings = Schema.Struct({ }).pipe(Schema.withDecodingDefaultKey(Effect.succeed({}))); export const db = Schema.Struct({ - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPort, description: "Port to use for the local database URL.", tags, links: [links.postgres], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPort))), - shadow_port: Schema.Number.annotate({ + shadow_port: Schema.Finite.annotate({ default: defaultShadowPort, description: "Port used by db diff command to initialize the shadow database.", tags, @@ -85,7 +85,7 @@ export const db = Schema.Struct({ "Maximum amount of time to wait for health check when starting the local database.", tags, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultHealthTimeout))), - major_version: Schema.Number.annotate({ + major_version: Schema.Finite.annotate({ default: defaultMajorVersion, description: "The database major version to use. This has to be the same as your remote database's.", @@ -106,7 +106,7 @@ export const db = Schema.Struct({ tags, links: [links.pgbouncer()], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPoolerEnabled))), - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPoolerPort, description: "Port to use for the local connection pooler.", tags, @@ -118,13 +118,13 @@ export const db = Schema.Struct({ tags, links: [links.pgbouncer("pool_mode")], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPoolMode))), - default_pool_size: Schema.Number.annotate({ + default_pool_size: Schema.Finite.annotate({ default: defaultPoolSize, description: "How many server connections to allow per user/database pair.", tags, links: [links.pgbouncer("default_pool_size")], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPoolSize))), - max_client_conn: Schema.Number.annotate({ + max_client_conn: Schema.Finite.annotate({ default: defaultMaxClientConn, description: "Maximum number of client connections allowed.", tags, diff --git a/packages/config/src/edge_runtime.ts b/packages/config/src/edge_runtime.ts index e5a78e0b5a..5ca240439d 100644 --- a/packages/config/src/edge_runtime.ts +++ b/packages/config/src/edge_runtime.ts @@ -20,12 +20,12 @@ export const edge_runtime = Schema.Struct({ description: "Configure the supported request policy.", tags, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPolicy))), - inspector_port: Schema.Number.annotate({ + inspector_port: Schema.Finite.annotate({ default: defaultInspectorPort, description: "Port to run the Edge Functions inspector on.", tags, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultInspectorPort))), - deno_version: Schema.Number.annotate({ + deno_version: Schema.Finite.annotate({ default: defaultDenoVersion, description: "The Deno major version to use.", tags, diff --git a/packages/config/src/inbucket.ts b/packages/config/src/inbucket.ts index 7070b816e0..4ca3f920d2 100644 --- a/packages/config/src/inbucket.ts +++ b/packages/config/src/inbucket.ts @@ -20,7 +20,7 @@ export const inbucket = Schema.Struct({ tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnabled))), - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPort, description: dedent` Port to use for the email testing server web interface. @@ -31,14 +31,14 @@ export const inbucket = Schema.Struct({ links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultPort))), smtp_port: Schema.optionalKey( - Schema.Number.annotate({ + Schema.Finite.annotate({ description: "Optional SMTP port to expose for local testing.", tags, links, }), ), pop3_port: Schema.optionalKey( - Schema.Number.annotate({ + Schema.Finite.annotate({ description: "Optional POP3 port to expose for local testing.", tags, links, diff --git a/packages/config/src/io.unit.test.ts b/packages/config/src/io.unit.test.ts index 3d1af6fe8a..07f5ac5332 100644 --- a/packages/config/src/io.unit.test.ts +++ b/packages/config/src/io.unit.test.ts @@ -271,6 +271,11 @@ describe("config io", () => { ).toThrow(); }); + test("rejects a non-finite number during decode", () => { + expect(() => decodeCliConfig({ db: { port: Infinity } })).toThrow("Expected a finite number"); + expect(() => decodeCliConfig({ api: { max_rows: NaN } })).toThrow("Expected a finite number"); + }); + test("only validates the highest-priority enabled sms provider during decode (Go switch parity)", () => { // Providers are validated in a fixed priority order (twilio, twilio_verify, messagebird, // textlocal, vonage); only the first enabled one is checked, so a complete, higher-priority diff --git a/packages/config/src/project-config/api-attributes.ts b/packages/config/src/project-config/api-attributes.ts index dca32a9ec1..2967b7408c 100644 --- a/packages/config/src/project-config/api-attributes.ts +++ b/packages/config/src/project-config/api-attributes.ts @@ -1,4 +1,4 @@ -import { Schema } from "effect"; +import { Predicate, Schema } from "effect"; /** * A lenient, hand-mirrored copy of the v2 project-config API's `data.attributes` shape (not @@ -8,6 +8,9 @@ import { Schema } from "effect"; * `Schema.Unknown`, not omitted, so drift detection and passthrough can still see them. */ +/** Any JS number, `±Infinity`/`NaN` included: the registry's own gates do the narrowing. */ +const ApiNumber = Schema.declare(Predicate.isNumber, { expected: "number" }); + // See `./registry.ts` for which of these fields are mapped; the rest are `Schema.Unknown`. const postgresSettingsAttributes = Schema.Struct({ effective_cache_size: Schema.optionalKey(Schema.String), @@ -26,20 +29,20 @@ const postgresSettingsAttributes = Schema.Struct({ log_temp_files: Schema.optionalKey(Schema.Unknown), maintenance_work_mem: Schema.optionalKey(Schema.String), track_activity_query_size: Schema.optionalKey(Schema.String), - max_connections: Schema.optionalKey(Schema.Number), - max_locks_per_transaction: Schema.optionalKey(Schema.Number), + max_connections: Schema.optionalKey(ApiNumber), + max_locks_per_transaction: Schema.optionalKey(ApiNumber), max_logical_replication_workers: Schema.optionalKey(Schema.Unknown), - max_parallel_maintenance_workers: Schema.optionalKey(Schema.Number), - max_parallel_workers: Schema.optionalKey(Schema.Number), - max_parallel_workers_per_gather: Schema.optionalKey(Schema.Number), - max_replication_slots: Schema.optionalKey(Schema.Number), + max_parallel_maintenance_workers: Schema.optionalKey(ApiNumber), + max_parallel_workers: Schema.optionalKey(ApiNumber), + max_parallel_workers_per_gather: Schema.optionalKey(ApiNumber), + max_replication_slots: Schema.optionalKey(ApiNumber), max_slot_wal_keep_size: Schema.optionalKey(Schema.String), max_standby_archive_delay: Schema.optionalKey(Schema.String), max_standby_streaming_delay: Schema.optionalKey(Schema.String), max_sync_workers_per_subscription: Schema.optionalKey(Schema.Unknown), max_wal_size: Schema.optionalKey(Schema.String), - max_wal_senders: Schema.optionalKey(Schema.Number), - max_worker_processes: Schema.optionalKey(Schema.Number), + max_wal_senders: Schema.optionalKey(ApiNumber), + max_worker_processes: Schema.optionalKey(ApiNumber), session_replication_role: Schema.optionalKey(Schema.String), shared_buffers: Schema.optionalKey(Schema.String), statement_timeout: Schema.optionalKey(Schema.String), @@ -71,7 +74,7 @@ const networkRestrictionsAttributes = Schema.Struct({ }); const databaseAttributes = Schema.Struct({ - major_version: Schema.optionalKey(Schema.Number), + major_version: Schema.optionalKey(ApiNumber), ssl_enforced: Schema.optionalKey(Schema.Boolean), network_restrictions: Schema.optionalKey(networkRestrictionsAttributes), postgres_settings: Schema.optionalKey(postgresSettingsAttributes), @@ -85,8 +88,8 @@ const poolerAttributes = Schema.Struct({ server_lifetime: Schema.optionalKey(Schema.Unknown), query_wait_timeout: Schema.optionalKey(Schema.Unknown), reserve_pool_size: Schema.optionalKey(Schema.Unknown), - default_pool_size: Schema.optionalKey(Schema.Number), - max_client_conn: Schema.optionalKey(Schema.Number), + default_pool_size: Schema.optionalKey(ApiNumber), + max_client_conn: Schema.optionalKey(ApiNumber), }); // `db_schema`/`db_extra_search_path`/`max_rows` are mapped (`./registry.ts`); the rest aren't. @@ -95,7 +98,7 @@ const poolerAttributes = Schema.Struct({ const apiAttributes = Schema.Struct({ db_schema: Schema.optionalKey(Schema.String), db_extra_search_path: Schema.optionalKey(Schema.String), - max_rows: Schema.optionalKey(Schema.Number), + max_rows: Schema.optionalKey(ApiNumber), db_pool_acquisition_timeout: Schema.optionalKey(Schema.Unknown), db_pool: Schema.optionalKey(Schema.Unknown), }); @@ -128,16 +131,16 @@ const storageFeaturesAttributes = Schema.Struct({ iceberg_catalog: Schema.optionalKey( Schema.Struct({ enabled: Schema.optionalKey(Schema.Boolean), - max_namespaces: Schema.optionalKey(Schema.Number), - max_tables: Schema.optionalKey(Schema.Number), - max_catalogs: Schema.optionalKey(Schema.Number), + max_namespaces: Schema.optionalKey(ApiNumber), + max_tables: Schema.optionalKey(ApiNumber), + max_catalogs: Schema.optionalKey(ApiNumber), }), ), vector_buckets: Schema.optionalKey( Schema.Struct({ enabled: Schema.optionalKey(Schema.Boolean), - max_buckets: Schema.optionalKey(Schema.Number), - max_indexes: Schema.optionalKey(Schema.Number), + max_buckets: Schema.optionalKey(ApiNumber), + max_indexes: Schema.optionalKey(ApiNumber), }), ), }); @@ -148,7 +151,7 @@ const storageCapabilitiesAttributes = Schema.Unknown; // `file_size_limit` is mapped; `capabilities`/`upstream_target`/`migration_version` aren't. const storageAttributes = Schema.Struct({ - file_size_limit: Schema.optionalKey(Schema.Number), + file_size_limit: Schema.optionalKey(ApiNumber), features: Schema.optionalKey(storageFeaturesAttributes), capabilities: Schema.optionalKey(storageCapabilitiesAttributes), upstream_target: Schema.optionalKey(Schema.Unknown), diff --git a/packages/config/src/project-config/project-config.unit.test.ts b/packages/config/src/project-config/project-config.unit.test.ts index ad8c08e127..7be2435506 100644 --- a/packages/config/src/project-config/project-config.unit.test.ts +++ b/packages/config/src/project-config/project-config.unit.test.ts @@ -771,6 +771,13 @@ describe("fromApiProjectConfig — api section", () => { // default), not merely "not caller_misuse". expect((thrown as ProjectConfigParseError).reason ?? "api_response").toBe("api_response"); expect((thrown as ProjectConfigParseError).apiPath).toEqual(["api", "max_rows"]); + expect(thrown).toHaveProperty( + "message", + expect.stringContaining( + "at data.attributes.api.max_rows: expected a finite number, got number", + ), + ); + expect(thrown).not.toHaveProperty("detail"); }); }); @@ -2741,9 +2748,9 @@ describe("review round: exactness parsing, cross-arm disabled sentinels (CLI-223 ["-0", -0], ["negative", -5], ["-Infinity", Number.NEGATIVE_INFINITY], - // TOML's `nan` literal is a real reachable document value (`smol-toml` parses - // `max_rows = nan` to `Number.NaN`); `NaN <= 0` is `false`, so a naive non-positive check - // would miss it, but `!(value > 0)` catches it since `NaN > 0` is also `false`. + // Config decode rejects TOML `nan`/`inf` (`Schema.Finite`), so `NaN` only reaches this arm + // through a caller-built `EffectiveConfig`; `NaN <= 0` is `false`, so a naive non-positive + // check would miss it, but `!(value > 0)` catches it since `NaN > 0` is also `false`. ["NaN", Number.NaN], ])("api.max_rows: %s is omitted on the document arm", (_description, value) => { const doc = fromConfigDocument({ api: { enabled: true, max_rows: value } }); diff --git a/packages/config/src/project-config/registry-auth.ts b/packages/config/src/project-config/registry-auth.ts index 28ef052255..8039221c08 100644 --- a/packages/config/src/project-config/registry-auth.ts +++ b/packages/config/src/project-config/registry-auth.ts @@ -641,7 +641,7 @@ const smtpRows: ReadonlyArray<ProjectConfigMappingRow> = [ const port = parseUint16(expectString(value, smtpPortPath)); return smtpExplicitlyDisabledInAttributes(attributes) ? undefined : port; }, - // The config schema allows an unrestricted `Schema.Number` for port, but the push wrapper + // The config schema allows any finite number for port, but the push wrapper // stringifies it verbatim while this row's transform only ever produces what `parseUint16` // accepts. Replaying the same String→parseUint16 round trip here predicts that: a fractional // or out-of-range document port omits the field, matching what the API arm reports post-push. diff --git a/packages/config/src/project-config/registry.ts b/packages/config/src/project-config/registry.ts index dbce597062..4843d1e9e2 100644 --- a/packages/config/src/project-config/registry.ts +++ b/packages/config/src/project-config/registry.ts @@ -51,8 +51,9 @@ function clampDocumentUint(value: unknown): unknown { * Omits (rather than clamps) `api.max_rows` when the document value is non-positive * (including `0` and `NaN`): push only ever sends `max_rows` when it is strictly positive, * so any other value never round-trips and asserting it would be drift. Uses `!(value > 0)` - * rather than `value <= 0` specifically because `NaN <= 0` is `false` — a TOML `nan` value - * would otherwise ride through and poison every downstream comparison. + * rather than `value <= 0` specifically because `NaN <= 0` is `false` — a `NaN` in a caller-built + * `EffectiveConfig` (config decode already rejects TOML `nan`) would otherwise ride through and + * poison every downstream comparison. */ function normalizeDocumentMaxRows(value: unknown): unknown { return typeof value === "number" && !(value > 0) ? undefined : value; diff --git a/packages/config/src/realtime.ts b/packages/config/src/realtime.ts index 10299ad7fa..ff11cfc30c 100644 --- a/packages/config/src/realtime.ts +++ b/packages/config/src/realtime.ts @@ -32,7 +32,7 @@ export const realtime = Schema.Struct({ }, ], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultIpVersion))), - max_header_length: Schema.Number.annotate({ + max_header_length: Schema.Finite.annotate({ default: defaultMaxHeaderLength, description: "Maximum length of the HTTP header.", tags, diff --git a/packages/config/src/storage.ts b/packages/config/src/storage.ts index 01a5756fda..876e527727 100644 --- a/packages/config/src/storage.ts +++ b/packages/config/src/storage.ts @@ -39,7 +39,7 @@ const emptyTable = Schema.Union([ // Accepts a human-readable string (`"50MiB"`) or a bare byte count (`5000000`); a numeric // value is normalized to its decimal string so the decoded type is always a `string`. -const fileSizeLimit = Schema.Union([Schema.String, Schema.Number]).pipe( +const fileSizeLimit = Schema.Union([Schema.String, Schema.Finite]).pipe( Schema.decodeTo(Schema.String, { decode: SchemaGetter.transform((value) => (typeof value === "number" ? String(value) : value)), encode: SchemaGetter.transform((value) => value), @@ -126,19 +126,19 @@ export const storage = Schema.Struct({ tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultAnalyticsEnabled))), - max_namespaces: Schema.Number.annotate({ + max_namespaces: Schema.Finite.annotate({ default: defaultMaxNamespaces, description: "Maximum number of analytics namespaces.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultMaxNamespaces))), - max_tables: Schema.Number.annotate({ + max_tables: Schema.Finite.annotate({ default: defaultMaxTables, description: "Maximum number of analytics tables.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultMaxTables))), - max_catalogs: Schema.Number.annotate({ + max_catalogs: Schema.Finite.annotate({ default: defaultMaxCatalogs, description: "Maximum number of analytics catalogs.", tags, @@ -162,13 +162,13 @@ export const storage = Schema.Struct({ tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultVectorEnabled))), - max_buckets: Schema.Number.annotate({ + max_buckets: Schema.Finite.annotate({ default: defaultMaxBuckets, description: "Maximum number of vector buckets.", tags, links, }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultMaxBuckets))), - max_indexes: Schema.Number.annotate({ + max_indexes: Schema.Finite.annotate({ default: defaultMaxIndexes, description: "Maximum number of vector indexes.", tags, diff --git a/packages/config/src/studio.ts b/packages/config/src/studio.ts index 16fc90ec0d..876411d7fb 100644 --- a/packages/config/src/studio.ts +++ b/packages/config/src/studio.ts @@ -25,7 +25,7 @@ export const studio = Schema.Struct({ tags, links: [links.studio], }).pipe(Schema.withDecodingDefaultKey(Effect.succeed(defaultEnabled))), - port: Schema.Number.annotate({ + port: Schema.Finite.annotate({ default: defaultPort, description: "Port to use for Supabase Studio.", tags, From 7e3035c644f4528833dd7a6a992535ded7158a8f Mon Sep 17 00:00:00 2001 From: Prashansa Kulshrestha <prashansa.kulshrestha@supabase.io> Date: Wed, 21 Oct 2026 12:42:39 +0000 Subject: [PATCH 12/24] feat(cli): show token permissions in command help (#7268) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a PERMISSIONS section to `--help` for every command, listing the personal access token permissions it needs. Every command already declares the Management API operations it may call. The help section resolves those declarations against the permissions the API spec lists for each operation, so users can see which token to create before running a command. - Permissions come from a generated `operationId` to `x-fga-permissions` / `x-oauth-scope` table in `@supabase/api/permissions`. - Conditions are worded against the command's own flags: a flag the command lacks is never named, equivalent cases merge, and a permission already shown under a broader condition is not repeated. Requirements that depend on something other than a flag show as "May be required", without a reason. - `--output-format json` help carries the same section under a top-level `permissions` field, with each group's flag conditions as data. The raw permission declaration is no longer serialized into `doc.annotations`. - Each permission scheme is resolved and rendered on its own, so OAuth scopes can be added later by registering one more scheme. Output sample: ``` $ supabase projects api-keys --help DESCRIPTION List all API keys for a Supabase project. ... rest of the content ... PERMISSIONS Personal access token (FGA): Required with --reveal: api_gateway_keys_read + api_gateway_keys_secret_read Required without --reveal: api_gateway_keys_read May be required (without --project-ref): projects_read Optional (command still runs without these): project_admin_read ``` ``` $ supabase projects api-keys --help --output-format json ... "permissions": { "status": "mapped", "schemes": [ { "scheme": "fga", "title": "Personal access token (FGA)", "groups": [ { "heading": "Required with --reveal", "need": "required", "when": [ { "all": [ { "flag": "reveal", "values": [ "true" ] } ] } ], "rows": [ { "anyOf": [ [ "api_gateway_keys_read", "api_gateway_keys_secret_read" ] ] } ] }, { "heading": "Required without --reveal", "need": "required", "when": [ { "all": [ { "flag": "reveal", "values": [ "false" ] } ] } ], "rows": [ { "anyOf": [ [ "api_gateway_keys_read" ] ] } ] }, { "heading": "May be required (without --project-ref)", "need": "maybe", "when": [ { "all": [ { "flag": "project-ref", "values": [ "unset", "empty" ] } ] } ], "rows": [ { "anyOf": [ [ "projects_read" ] ] } ] }, { "heading": "Optional (command still runs without these)", "need": "optional", "rows": [ { "anyOf": [ [ "project_admin_read" ] ] } ] } ] } ] } ... ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> --- apps/cli/src/cli/root.ts | 4 +- .../command-permissions.annotation.ts | 32 +- .../command-permissions/fga-request-sets.ts | 102 ++ .../fga-request-sets.unit.test.ts | 115 ++ .../help-section.integration.test.ts | 147 ++ .../help-section.tree.unit.test.ts | 255 ++++ .../command-permissions/help-section.ts | 334 +++++ .../help-section.unit.test.ts | 1211 +++++++++++++++++ .../command-permissions/model.ts | 8 +- .../permission-conditions.ts | 439 ++++++ .../command-permissions/permission-flags.ts | 104 ++ .../command-permissions/permission-schemes.ts | 106 ++ apps/cli/src/command-internal/global-flags.ts | 3 + .../branches/create/create.command.ts | 12 +- apps/cli/src/shared/output/json-formatter.ts | 13 +- apps/cli/src/shared/output/text-formatter.ts | 9 + apps/cli/tests/helpers/permission-drift.ts | 60 +- .../helpers/permission-drift.unit.test.ts | 56 +- packages/api/package.json | 1 + packages/api/scripts/generate.ts | 33 + .../generated-output-sync.unit.test.ts | 19 +- packages/api/src/entrypoints.unit.test.ts | 1 + packages/api/src/generated/permissions.ts | 423 ++++++ 23 files changed, 3456 insertions(+), 31 deletions(-) create mode 100644 apps/cli/src/command-internal/command-permissions/fga-request-sets.ts create mode 100644 apps/cli/src/command-internal/command-permissions/fga-request-sets.unit.test.ts create mode 100644 apps/cli/src/command-internal/command-permissions/help-section.integration.test.ts create mode 100644 apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts create mode 100644 apps/cli/src/command-internal/command-permissions/help-section.ts create mode 100644 apps/cli/src/command-internal/command-permissions/help-section.unit.test.ts create mode 100644 apps/cli/src/command-internal/command-permissions/permission-conditions.ts create mode 100644 apps/cli/src/command-internal/command-permissions/permission-flags.ts create mode 100644 apps/cli/src/command-internal/command-permissions/permission-schemes.ts create mode 100644 packages/api/src/generated/permissions.ts diff --git a/apps/cli/src/cli/root.ts b/apps/cli/src/cli/root.ts index 03a6fa9a48..4d2e43ee0a 100644 --- a/apps/cli/src/cli/root.ts +++ b/apps/cli/src/cli/root.ts @@ -68,7 +68,7 @@ import { extractCommandPath, type CliRootCommand } from "../shared/cli/run.ts"; import { isBuiltInTextRequest, resolveAgentOutputFormat } from "../shared/cli/agent-output.ts"; import type { OutputFormat } from "../shared/output/types.ts"; import { - GLOBAL_FLAGS, + ROOT_GLOBAL_FLAGS, AgentFlag, CreateTicketFlag, DebugFlag, @@ -276,7 +276,7 @@ export const rootCommandForFeatures = (options: RootCommandFeatures = {}) => }), ), ), - Command.withGlobalFlags([OutputFormatFlag, ...GLOBAL_FLAGS]), + Command.withGlobalFlags(ROOT_GLOBAL_FLAGS), ) satisfies CliRootCommand; /** Pairs the root with the agent default `runCli` applies before this root parses. */ diff --git a/apps/cli/src/command-internal/command-permissions/command-permissions.annotation.ts b/apps/cli/src/command-internal/command-permissions/command-permissions.annotation.ts index 095ea6e2d1..c741160c9e 100644 --- a/apps/cli/src/command-internal/command-permissions/command-permissions.annotation.ts +++ b/apps/cli/src/command-internal/command-permissions/command-permissions.annotation.ts @@ -1,8 +1,10 @@ import { Context, Option } from "effect"; import { Command } from "effect/cli"; +import { commandInternals, userGlobalFlagParams } from "../../docs/docs-introspection.ts"; import { mapped } from "./model.ts"; import type { CommandPermissions, PermissionFragment } from "./model.ts"; +import { CommandFlagsAnnotation, permissionFlagsOf } from "./permission-flags.ts"; /** Context key under which a command's declared permissions are stored in `Command.annotations`. */ export class CommandPermissionsAnnotation extends Context.Service< @@ -12,13 +14,31 @@ export class CommandPermissionsAnnotation extends Context.Service< /** * Declares the Management API operations a command may call. Takes a full `CommandPermissions`, - * or a fragment (e.g. the result of `compose`) that is declared as mapped. + * or a fragment (e.g. the result of `compose`) that is declared as mapped. Also lets help read the + * command's own flags as they are at this call, so it can tell a flag the command lacks from one it + * hides; apply it after the command's flags are defined. */ -export const withPermissions = (permissions: CommandPermissions | PermissionFragment) => - Command.annotate( - CommandPermissionsAnnotation, - "status" in permissions ? permissions : mapped(permissions), - ); +export const withPermissions = + (permissions: CommandPermissions | PermissionFragment) => + <Name extends string, Input, ContextInput, E, R>( + self: Command.Command<Name, Input, ContextInput, E, R>, + ): Command.Command<Name, Input, ContextInput, E, R> => { + const readOwnFlags = () => { + const internals = commandInternals(self); + return permissionFlagsOf([ + ...internals.config.flags, + ...internals.contextConfig.flags, + ...userGlobalFlagParams(self), + ]); + }; + return self.pipe( + Command.annotate( + CommandPermissionsAnnotation, + "status" in permissions ? permissions : mapped(permissions), + ), + Command.annotate(CommandFlagsAnnotation, readOwnFlags), + ); + }; /** The permissions declared on `command` with {@link withPermissions}, or none if it has no declaration. */ export function readPermissions(command: Command.Command.Any): Option.Option<CommandPermissions> { diff --git a/apps/cli/src/command-internal/command-permissions/fga-request-sets.ts b/apps/cli/src/command-internal/command-permissions/fga-request-sets.ts new file mode 100644 index 0000000000..7a75a58268 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/fga-request-sets.ts @@ -0,0 +1,102 @@ +/** + * CLI-side corrections to the API spec's FGA permissions, for operations the API enforces + * differently from what the spec's sets suggest. The spec lists a plain "any one of these sets" + * and carries no marker for a set that is only needed for one kind of request, for a set that + * is always needed alongside another, or for a choice that depends on the resource addressed. + */ + +import type { RequestDetails } from "./model.ts"; + +/** The set an operation needs for a request carrying `request`, in place of the spec's sets. */ +interface RequestSet { + readonly operationId: string; + readonly request: RequestDetails; + readonly permissions: ReadonlyArray<string>; +} + +/** + * The spec lists the secret-read set for the api-keys operation but not which request needs it; + * `reveal=true`, which returns full secret keys, is taken to be that request. Creating a branch + * with data needs the production permission. + */ +export const fgaRequestSets: ReadonlyArray<RequestSet> = [ + { + operationId: "v1-get-project-api-keys", + request: { query: { reveal: "true" } }, + permissions: ["api_gateway_keys_read", "api_gateway_keys_secret_read"], + }, + { + operationId: "v1-create-a-branch", + request: { body: { with_data: "true" } }, + permissions: ["branching_production_create"], + }, +]; + +/** + * An operation that always needs `required` and, for requests that may write, also `extra`; without + * `extra` the request still runs, with reduced privileges. + */ +interface ConditionalExtra { + readonly operationId: string; + readonly required: ReadonlyArray<string>; + readonly extra: ReadonlyArray<string>; +} + +export const fgaConditionalExtras: ReadonlyArray<ConditionalExtra> = [ + { + operationId: "v1-run-a-query", + required: ["database_read"], + extra: ["database_write"], + }, +]; + +/** What a request addressing one kind of resource needs; `for` names that resource. */ +interface ResourceCase { + readonly for: string; + readonly allOf: ReadonlyArray<string>; +} + +interface ResourceCases { + readonly operationId: string; + readonly cases: ReadonlyArray<ResourceCase>; +} + +const branchCases = (action: string): ReadonlyArray<ResourceCase> => [ + { for: "a preview branch", allOf: [`branching_development_${action}`] }, + { for: "the main branch", allOf: [`branching_production_${action}`] }, +]; + +/** A branch addressed by id needs the production permission only when it is the project's main branch. */ +export const fgaResourceCases: ReadonlyArray<ResourceCases> = [ + { operationId: "v1-get-a-branch", cases: branchCases("read") }, + { operationId: "v1-get-a-branch-config", cases: branchCases("read") }, + { operationId: "v1-update-a-branch-config", cases: branchCases("write") }, + { operationId: "v1-delete-a-branch", cases: branchCases("delete") }, +]; + +/** + * What to say about an operation for which the spec lists no FGA set. The profile endpoint rejects + * project- and organization-scoped tokens. + */ +export const fgaOperationNotes: ReadonlyMap<string, string> = new Map([ + ["v1-get-profile", "a user-scoped token (project- and organization-scoped tokens are rejected)"], +]); + +const matches = ( + expected: Readonly<Record<string, string>> | undefined, + sent: Readonly<Record<string, string>> | undefined, +) => Object.entries(expected ?? {}).every(([key, value]) => sent?.[key] === value); + +/** The set an operation needs for `request`, or `undefined` when no correction applies to it. */ +export function fgaSetForRequest( + operationId: string, + request: RequestDetails | undefined, +): ReadonlyArray<string> | undefined { + if (request === undefined) return undefined; + return fgaRequestSets.find( + (entry) => + entry.operationId === operationId && + matches(entry.request.query, request.query) && + matches(entry.request.body, request.body), + )?.permissions; +} diff --git a/apps/cli/src/command-internal/command-permissions/fga-request-sets.unit.test.ts b/apps/cli/src/command-internal/command-permissions/fga-request-sets.unit.test.ts new file mode 100644 index 0000000000..72371f570f --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/fga-request-sets.unit.test.ts @@ -0,0 +1,115 @@ +import { operationPermissions } from "@supabase/api/permissions"; +import { Exit, Option } from "effect"; +import { describe, expect, it } from "vitest"; + +import { commandTreeFor } from "../../../tests/helpers/command-flags.ts"; +import type { PermissionVariant } from "../../../tests/helpers/permission-registry.ts"; +import { readPermissions } from "./command-permissions.annotation.ts"; +import { + fgaConditionalExtras, + fgaOperationNotes, + fgaRequestSets, + fgaResourceCases, + fgaSetForRequest, +} from "./fga-request-sets.ts"; + +const VARIANTS: ReadonlyArray<PermissionVariant> = [ + undefined, + { + projectState: Exit.succeed("stack"), + stackBackend: "stack", + computeEnabled: true, + } satisfies Required<NonNullable<PermissionVariant>>, +]; + +const isSpecSet = (operationId: string, permissions: ReadonlyArray<string>) => + (operationPermissions[operationId]?.fga ?? []).some( + (set) => set.length === permissions.length && permissions.every((p) => set.includes(p)), + ); + +const declaredOperationIds = () => + new Set( + VARIANTS.flatMap((variant) => + [...commandTreeFor(variant).values()].flatMap((node) => { + const declared = Option.getOrUndefined(readPermissions(node.command)); + return declared?.status === "mapped" + ? declared.operations.map((entry) => entry.operationId) + : []; + }), + ), + ); + +describe("FGA request sets", () => { + it("only name permission sets the API spec lists for the operation", () => { + for (const { operationId, permissions } of fgaRequestSets) { + expect( + isSpecSet(operationId, permissions), + `${operationId} spec sets do not include ${permissions.join(" + ")}`, + ).toBe(true); + } + for (const { operationId, required, extra } of fgaConditionalExtras) { + expect(isSpecSet(operationId, required), `${operationId} required ${required}`).toBe(true); + expect(isSpecSet(operationId, extra), `${operationId} extra ${extra}`).toBe(true); + } + for (const { operationId, cases } of fgaResourceCases) { + expect(cases.length, `${operationId} cases`).toBeGreaterThan(1); + for (const { allOf } of cases) { + expect(isSpecSet(operationId, allOf), `${operationId} case ${allOf}`).toBe(true); + } + } + }); + + it("only correct operations some command declares", () => { + const declared = declaredOperationIds(); + const corrected = [ + ...fgaRequestSets.map((entry) => entry.operationId), + ...fgaConditionalExtras.map((entry) => entry.operationId), + ...fgaResourceCases.map((entry) => entry.operationId), + ]; + for (const operationId of corrected) { + expect(declared.has(operationId), `no command declares ${operationId}`).toBe(true); + } + }); + + it("only add a note for an operation the API spec lists no FGA set for", () => { + expect(fgaOperationNotes.size).toBeGreaterThan(0); + for (const operationId of fgaOperationNotes.keys()) { + expect(operationPermissions[operationId]?.fga, `${operationId} spec sets`).toEqual([]); + } + }); + + it("cover every request detail a command declaration makes", () => { + for (const variant of VARIANTS) { + for (const [path, node] of commandTreeFor(variant)) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared?.status !== "mapped") continue; + for (const entry of declared.operations) { + if (entry.request === undefined) continue; + expect( + fgaSetForRequest(entry.operationId, entry.request), + `"${path}" sends ${JSON.stringify(entry.request)} to ${entry.operationId}`, + ).toBeDefined(); + } + } + } + }); + + it("leaves a request without the detail on the operation's own sets", () => { + expect(fgaSetForRequest("v1-get-project-api-keys", undefined)).toBeUndefined(); + expect( + fgaSetForRequest("v1-get-project-api-keys", { query: { reveal: "false" } }), + ).toBeUndefined(); + expect( + fgaSetForRequest("v1-create-a-branch", { body: { with_data: "false" } }), + ).toBeUndefined(); + expect( + fgaSetForRequest("v1-create-a-branch", { query: { with_data: "true" } }), + ).toBeUndefined(); + }); + + it("matches a request detail in the body", () => { + expect(fgaSetForRequest("v1-create-a-branch", { body: { with_data: "true" } })).toEqual([ + "branching_production_create", + ]); + }); +}); diff --git a/apps/cli/src/command-internal/command-permissions/help-section.integration.test.ts b/apps/cli/src/command-internal/command-permissions/help-section.integration.test.ts new file mode 100644 index 0000000000..74c989c158 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/help-section.integration.test.ts @@ -0,0 +1,147 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Console, Effect, Layer, Option } from "effect"; +import { CliOutput, Command } from "effect/cli"; + +import { emptyEnv, fakeConsole, mockOutput } from "../../../tests/helpers/mocks.ts"; +import { projectsCommand } from "../../commands/projects/projects.command.ts"; +import { CliArgs } from "../../shared/cli/cli-args.service.ts"; +import { jsonCliOutputFormatter } from "../../shared/output/json-formatter.ts"; +import { textCliOutputFormatter } from "../../shared/output/text-formatter.ts"; +import { + DebugFlag, + DnsResolverFlag, + ExperimentalFlag, + OutputFlag, + ProfileFlag, + WorkdirFlag, + YesFlag, +} from "../global-flags.ts"; + +const layerFor = ( + args: ReadonlyArray<string>, + console: Console.Console, + formatter: CliOutput.Formatter, +) => + Layer.mergeAll( + emptyEnv(), + CliOutput.layer(formatter), + Layer.succeed(CliArgs, { args }), + Layer.succeed(Console.Console, console), + Layer.succeed(DebugFlag, false), + Layer.succeed(OutputFlag, Option.none()), + Layer.succeed(ExperimentalFlag, false), + Layer.succeed(ProfileFlag, "supabase"), + Layer.succeed(WorkdirFlag, Option.none()), + Layer.succeed(YesFlag, false), + Layer.succeed(DnsResolverFlag, "native"), + mockOutput({ format: "text" }).layer, + ); + +const helpFor = ( + args: ReadonlyArray<string>, + formatter: CliOutput.Formatter = textCliOutputFormatter(), +) => + Effect.gen(function* () { + const { console, calls } = fakeConsole(); + yield* Effect.scoped( + Command.runWith(projectsCommand, { version: "0.0.0-test" })(args).pipe( + Effect.provide(layerFor(args, console, formatter)), + ), + ); + return calls.join("\n"); + }); + +describe("token permissions in command help", () => { + it.effect("shows the permissions a leaf command needs, with the --reveal case separate", () => + Effect.gen(function* () { + const text = yield* helpFor(["api-keys", "--help"]); + + expect(text.slice(text.indexOf("PERMISSIONS"))).toContain( + [ + "PERMISSIONS", + " Scoped access token:", + " Required with --reveal:", + " api_gateway_keys_read + api_gateway_keys_secret_read", + " Required without --reveal:", + " api_gateway_keys_read", + " May be required (without --project-ref):", + " projects_read", + " Optional (command still runs without these):", + " project_admin_read", + ].join("\n"), + ); + }), + ); + + it.effect("carries the same section as data in JSON help, without the raw declaration", () => + Effect.gen(function* () { + const text = yield* helpFor(["api-keys", "--help"]); + const printed = yield* helpFor(["api-keys", "--help"], jsonCliOutputFormatter()); + const json = JSON.parse(printed.replace(/^log:/, "")); + + expect(json.permissions).toEqual({ + status: "mapped", + schemes: [ + { + scheme: "fga", + title: "Scoped access token", + groups: [ + { + heading: "Required with --reveal", + need: "required", + when: [{ all: [{ flag: "reveal", values: ["true"] }] }], + rows: [{ anyOf: [["api_gateway_keys_read", "api_gateway_keys_secret_read"]] }], + }, + { + heading: "Required without --reveal", + need: "required", + when: [{ all: [{ flag: "reveal", values: ["false"] }] }], + rows: [{ anyOf: [["api_gateway_keys_read"]] }], + }, + { + heading: "May be required (without --project-ref)", + need: "maybe", + when: [{ all: [{ flag: "project-ref", values: ["unset", "empty"] }] }], + rows: [{ anyOf: [["projects_read"]] }], + }, + { + heading: "Optional (command still runs without these)", + need: "optional", + rows: [{ anyOf: [["project_admin_read"]] }], + }, + ], + }, + ], + }); + expect(JSON.stringify(json.doc)).not.toContain("supabase/Command"); + + const block = text.slice(text.indexOf("PERMISSIONS")); + for (const scheme of json.permissions.schemes) { + for (const group of scheme.groups) { + expect(block).toContain(`${group.heading}:`); + for (const row of group.rows) { + const shown: ReadonlyArray<string> = + "anyOf" in row + ? row.anyOf.map((alternative: ReadonlyArray<string>) => alternative.join(" + ")) + : "cases" in row + ? row.cases.map( + (item: { for: string; allOf: ReadonlyArray<string> }) => + `${item.allOf.join(" + ")} for ${item.for}`, + ) + : [row.note]; + for (const text of shown) expect(block).toContain(text); + } + } + } + }), + ); + + it.effect("shows no permissions section on a command group", () => + Effect.gen(function* () { + const text = yield* helpFor(["--help"]); + + expect(text).toContain("api-keys"); + expect(text).not.toContain("PERMISSIONS"); + }), + ); +}); diff --git a/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts b/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts new file mode 100644 index 0000000000..b48f858f26 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/help-section.tree.unit.test.ts @@ -0,0 +1,255 @@ +import { BunServices } from "@effect/platform-bun"; +import { Context, Effect, Exit, Option } from "effect"; +import type { Command } from "effect/cli"; +import { describe, expect, it } from "vitest"; + +import { commandTreeFor } from "../../../tests/helpers/command-flags.ts"; +import type { PermissionVariant } from "../../../tests/helpers/permission-registry.ts"; +import { commandInternals, userGlobalFlagParams } from "../../docs/docs-introspection.ts"; +import { readPermissions } from "./command-permissions.annotation.ts"; +import { renderPermissionsSection, resolvePermissionSection } from "./help-section.ts"; +import { CommandFlagsAnnotation, permissionFlagsOf } from "./permission-flags.ts"; +import type { FlagCatalog } from "./permission-flags.ts"; +import { rootCommandForFeatures } from "../../cli/root.ts"; +import { unwrapParam } from "../param-introspection.ts"; + +const VARIANTS: ReadonlyArray<{ readonly name: string; readonly variant: PermissionVariant }> = [ + { name: "default backend", variant: undefined }, + { + name: "stack backend", + variant: { + projectState: Exit.succeed("stack"), + stackBackend: "stack", + computeEnabled: true, + } satisfies Required<NonNullable<PermissionVariant>>, + }, +]; + +const ownParams = (command: Command.Command.Any) => { + const internals = commandInternals(command); + return [ + ...internals.config.flags, + ...internals.contextConfig.flags, + ...userGlobalFlagParams(command), + ]; +}; + +describe("permission help section across the command tree", () => { + for (const { name, variant } of VARIANTS) { + describe(name, () => { + const tree = commandTreeFor(variant); + const rootFlags = permissionFlagsOf(userGlobalFlagParams(rootCommandForFeatures(variant))); + const leaves = [...tree].filter(([, node]) => node.isLeaf); + + // Help lists only visible flags of a parent group, but the command's own flags come with + // their hidden ones from what `withPermissions` recorded. + const catalogFor = (path: string): FlagCatalog => { + const parts = path.split(" "); + const inherited = parts.slice(0, -1).flatMap((_, index) => { + const ancestor = tree.get(parts.slice(0, index + 1).join(" ")); + return ancestor === undefined + ? [] + : permissionFlagsOf(ownParams(ancestor.command)).filter((flag) => !flag.hidden); + }); + const own = tree.get(path); + const ownFlags = own === undefined ? [] : permissionFlagsOf(ownParams(own.command)); + return new Map([...rootFlags, ...inherited, ...ownFlags].map((flag) => [flag.name, flag])); + }; + + const paramsFor = (path: string) => { + const parts = path.split(" "); + return new Map( + [ + ...userGlobalFlagParams(rootCommandForFeatures(variant)), + ...parts.flatMap((_, index) => { + const ancestor = tree.get(parts.slice(0, index + 1).join(" ")); + return ancestor === undefined ? [] : ownParams(ancestor.command); + }), + ].map((param) => [unwrapParam(param)?.single.name, param] as const), + ); + }; + + it("gives a flag a default only when parsing no arguments yields that value", async () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared?.status !== "mapped") continue; + const params = paramsFor(path); + // Help rebuilds a parent group's flags without defaults, so only these carry one. + const known = new Map( + [...rootFlags, ...permissionFlagsOf(ownParams(node.command))].map((flag) => [ + flag.name, + flag, + ]), + ); + const tested = new Set( + declared.operations.flatMap((entry) => + (entry.when ?? []).flatMap((condition) => + condition.oneOf === undefined ? [] : [condition.flag], + ), + ), + ); + for (const flag of tested) { + const param = params.get(flag); + if (flag === undefined || param === undefined) continue; + const exit = await Effect.runPromiseExit( + param.parse({ flags: {}, arguments: [] }).pipe(Effect.provide(BunServices.layer)), + ); + const parsed = Exit.isSuccess(exit) ? exit.value[1] : undefined; + expect( + known.get(flag)?.default, + `"${path}" --${flag} default differs from parsing no arguments`, + ).toBe(typeof parsed === "string" ? parsed : undefined); + } + } + }); + + it("never tests the presence of a text flag that has a default", () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared?.status !== "mapped") continue; + const catalog = catalogFor(path); + for (const entry of declared.operations) { + for (const condition of entry.when ?? []) { + if (condition.flag === undefined || condition.present === undefined) continue; + expect( + catalog.get(condition.flag)?.default, + `"${path}" tests whether --${condition.flag} is given, but it has a default`, + ).toBeUndefined(); + } + } + } + }); + + it("never turns on by default a switch a condition tests", async () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared?.status !== "mapped") continue; + const params = paramsFor(path); + const tested = new Set( + declared.operations.flatMap((entry) => + (entry.when ?? []).flatMap((condition) => + condition.is === undefined ? [] : [condition.flag], + ), + ), + ); + for (const flag of tested) { + const param = params.get(flag); + if (param === undefined || catalogFor(path).get(flag)?.hidden !== false) continue; + const [, value] = await Effect.runPromise( + param.parse({ flags: {}, arguments: [] }).pipe(Effect.provide(BunServices.layer)), + ); + expect(value, `"${path}" --${flag} defaults to on`).not.toBe(true); + } + } + }); + + it("records every flag a command accepts when it declares its permissions", () => { + for (const [path, node] of leaves) { + const recorded = Context.getOption(node.command.annotations, CommandFlagsAnnotation); + const expected = permissionFlagsOf(ownParams(node.command)).map((flag) => flag.name); + expect(Option.isSome(recorded), `"${path}" records no flags`).toBe(true); + expect( + Option.match(recorded, { onNone: () => [], onSome: (read) => read() }).map( + (flag) => flag.name, + ), + `"${path}" declares permissions before its flags are complete`, + ).toEqual(expected); + } + }); + + // Help sees a parent group's flags only as visible names, so a condition cannot depend on + // a parent's hidden flag or on its choice values. + it("only conditions on flags a parent group shares when help can describe them", () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared?.status !== "mapped") continue; + const parts = path.split(" "); + const own = new Set(permissionFlagsOf(ownParams(node.command)).map((flag) => flag.name)); + const root = new Set(rootFlags.map((flag) => flag.name)); + const inherited = new Map( + parts + .slice(0, -1) + .flatMap((_, index) => { + const ancestor = tree.get(parts.slice(0, index + 1).join(" ")); + return ancestor === undefined ? [] : permissionFlagsOf(ownParams(ancestor.command)); + }) + .map((flag) => [flag.name, flag]), + ); + for (const entry of declared.operations) { + for (const condition of entry.when ?? []) { + const flag = condition.flag === undefined ? undefined : inherited.get(condition.flag); + if (flag === undefined || own.has(flag.name) || root.has(flag.name)) continue; + expect( + !flag.hidden && flag.kind !== "choice", + `"${path}" conditions on --${flag.name}, a hidden or choice flag of a parent group that help cannot describe`, + ).toBe(true); + } + } + } + }); + + it("renders a section for every command that calls an API operation", () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared === undefined) continue; + const section = resolvePermissionSection(declared, catalogFor(path)); + const callsApi = declared.status === "unmapped" || declared.operations.length > 0; + expect(section === undefined, `"${path}" section presence`).toBe(!callsApi); + if (section?.status !== "mapped") continue; + for (const scheme of section.schemes) { + expect( + scheme.groups.length, + `"${path}" has an empty ${scheme.scheme} block`, + ).toBeGreaterThan(0); + for (const group of scheme.groups) { + expect( + group.rows.length, + `"${path}" "${group.heading}" lists no rows`, + ).toBeGreaterThan(0); + } + } + } + }); + + it("names only flags the command shows in its help", () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared === undefined) continue; + const catalog = catalogFor(path); + const section = resolvePermissionSection(declared, catalog); + if (section === undefined) continue; + const text = renderPermissionsSection(section); + for (const [, flag] of text.matchAll(/--([a-z][a-z0-9-]*)/g)) { + expect( + flag !== undefined && catalog.get(flag)?.hidden === false, + `"${path}" section names --${flag}, which its help does not show`, + ).toBe(true); + } + } + }); + + it("prints each heading once, and never lists a row as both required and optional", () => { + for (const [path, node] of leaves) { + const declared = Option.getOrUndefined(readPermissions(node.command)); + if (declared === undefined) continue; + const section = resolvePermissionSection(declared, catalogFor(path)); + if (section?.status !== "mapped") continue; + for (const scheme of section.schemes) { + const headings = scheme.groups.map((group) => group.heading); + expect(new Set(headings).size, `"${path}" repeats a heading`).toBe(headings.length); + const rowsOf = (matches: (need: string) => boolean) => + scheme.groups + .filter((group) => matches(group.need)) + .flatMap((group) => group.rows.map((row) => JSON.stringify(row))); + const required = new Set(rowsOf((need) => need !== "optional")); + for (const row of rowsOf((need) => need === "optional")) { + expect(required.has(row), `"${path}" lists ${row} as required and optional`).toBe( + false, + ); + } + } + } + }); + }); + } +}); diff --git a/apps/cli/src/command-internal/command-permissions/help-section.ts b/apps/cli/src/command-internal/command-permissions/help-section.ts new file mode 100644 index 0000000000..85a3e9c4cd --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/help-section.ts @@ -0,0 +1,334 @@ +import { Context, Option } from "effect"; +import type { HelpDoc } from "effect/cli"; + +import { CommandPermissionsAnnotation } from "./command-permissions.annotation.ts"; +import { + alternativesOf, + conditionContext, + commonConditions, + conditionImplies, + conditionText, + joinAnd, + joinOr, + onlyLinked, + simplify, + TARGET_VARIABLE, + termOf, + withoutVariables, +} from "./permission-conditions.ts"; +import type { + Alternative, + Condition, + ConditionContext, + ConditionData, + Term, +} from "./permission-conditions.ts"; +import { CommandFlagsAnnotation, flagCatalogFor } from "./permission-flags.ts"; +import type { FlagCatalog } from "./permission-flags.ts"; +import { permissionSchemes } from "./permission-schemes.ts"; +import type { Alternatives, PermissionScheme, Requirement } from "./permission-schemes.ts"; +import type { CommandPermissions, OperationEntry } from "./model.ts"; + +type PermissionKind = OperationEntry["kind"]; + +/** One thing a heading lists: what an operation needs, or why a scheme lists nothing. */ +type PermissionRow = Requirement | { readonly note: string }; + +interface PermissionGroup { + /** The heading the text section prints, without its colon. */ + readonly heading: string; + /** + * `required` fails or loses the command's result without it; `maybe` is needed only in + * circumstances the heading does not spell out; `optional` leaves the command working without it. + */ + readonly need: "required" | "maybe" | "optional"; + /** The ways the group applies, any one of which suffices; absent when it always applies. */ + readonly when?: ReadonlyArray<Alternative>; + readonly rows: ReadonlyArray<PermissionRow>; +} + +interface SchemeSection { + readonly scheme: string; + readonly title: string; + readonly groups: ReadonlyArray<PermissionGroup>; +} + +/** What help shows about a command's token permissions. */ +export type PermissionSection = + | { readonly status: "unmapped" } + | { + readonly status: "mapped"; + /** What the whole section depends on, in words. */ + readonly note?: string; + /** The conditions behind `note`, all of which must hold for any group to apply. */ + readonly when?: ReadonlyArray<ConditionData>; + readonly schemes: ReadonlyArray<SchemeSection>; + }; + +interface Bucket { + readonly row: PermissionRow; + readonly kind: PermissionKind; + readonly situational: boolean; + condition: Condition; +} + +const rowKey = (row: PermissionRow) => JSON.stringify(row); + +/** An entry the command can reach; `hedged` when a prerequisite or a hidden flag qualifies it. */ +interface ReachableEntry { + readonly entry: OperationEntry; + readonly term: Term; + readonly hedged: boolean; +} + +const strength = (kind: PermissionKind) => (kind === "required" ? 1 : 0); + +interface RowPart { + readonly row: PermissionRow; + readonly hedged: boolean; +} + +function rowPartsFor(scheme: PermissionScheme, entry: OperationEntry): ReadonlyArray<RowPart> { + const parts = scheme.requirement(entry.operationId, entry.request); + return parts === undefined + ? [{ row: { note: scheme.missing(entry.kind, entry.operationId) }, hedged: false }] + : parts.map(({ requirement, hedged }) => ({ row: requirement, hedged })); +} + +const sameOrBroader = (bucket: Bucket, other: Bucket) => + strength(other.kind) >= strength(bucket.kind) && + (bucket.kind === "best-effort" || !other.situational || bucket.situational) && + conditionImplies(bucket.condition, other.condition); + +/** + * Whether holding what `other` lists always satisfies `row`: the same row, or an any-of row whose + * alternatives are exactly the per-resource cases of `other`. + */ +function rowImpliedBy(row: PermissionRow, other: PermissionRow): boolean { + if (rowKey(row) === rowKey(other)) return true; + if (!("anyOf" in row) || !("cases" in other)) return false; + return other.cases.every(({ allOf }) => + row.anyOf.some((alternative) => JSON.stringify(alternative) === JSON.stringify(allOf)), + ); +} + +/** + * Whether `other` already shows this bucket's row: under a condition at least as broad or, for an + * optional row, as required under any condition, so no heading says the command still runs + * without a permission that another heading requires. + */ +function absorbedBy(bucket: Bucket, other: Bucket): boolean { + if (other === bucket || !rowImpliedBy(bucket.row, other.row)) return false; + return ( + (bucket.kind === "best-effort" && other.kind === "required") || sameOrBroader(bucket, other) + ); +} + +function headingFor( + kind: PermissionKind, + situational: boolean, + flags: string, + remote: boolean, +): string { + const inner = [flags, ...(remote ? ["remote only"] : [])].filter(Boolean).join(", "); + if (kind === "required" && !situational) { + return `Required${flags === "" ? "" : ` ${flags}`}${remote ? " (remote only)" : ""}`; + } + if (kind === "required") return `May be required${inner === "" ? "" : ` (${inner})`}`; + return `Optional${inner === "" ? "" : `, ${inner}`} (command still runs without these)`; +} + +const needOf = (group: { kind: PermissionKind; situational: boolean }) => + group.kind === "best-effort" ? "optional" : group.situational ? "maybe" : "required"; + +const rank = (group: { kind: PermissionKind; situational: boolean; unconditional: boolean }) => + (group.kind === "required" ? 0 : 3) + (group.situational ? 2 : group.unconditional ? 0 : 1); + +function schemeSection( + scheme: PermissionScheme, + terms: ReadonlyArray<ReachableEntry>, + context: ConditionContext, +): SchemeSection { + const buckets = new Map<string, Bucket>(); + for (const { entry, term, hedged } of terms) { + for (const part of rowPartsFor(scheme, entry)) { + const { row } = part; + const situational = (hedged || part.hedged) && entry.kind === "required"; + const key = `${entry.kind}\u0000${situational}\u0000${rowKey(row)}`; + const bucket = buckets.get(key); + if (bucket === undefined) { + buckets.set(key, { row, kind: entry.kind, situational, condition: [term] }); + } else { + bucket.condition = [...bucket.condition, term]; + } + } + } + const simplified = [...buckets.values()].map((bucket) => ({ + ...bucket, + condition: simplify(bucket.condition, context), + })); + const shown = simplified.filter( + (bucket) => !simplified.some((other) => absorbedBy(bucket, other)), + ); + + const groups = new Map< + string, + { + heading: string; + kind: PermissionKind; + situational: boolean; + unconditional: boolean; + when: ReadonlyArray<Alternative> | undefined; + rows: Array<PermissionRow>; + } + >(); + for (const bucket of shown) { + const { flags, remote } = conditionText(bucket.condition, context); + const heading = headingFor(bucket.kind, bucket.situational, flags, remote); + const when = alternativesOf(bucket.condition, context); + // A group holds only rows whose conditions are identical, so one row's condition never + // stands for another's even when their headings read alike. + const key = `${heading}\u0000${JSON.stringify(when)}`; + const group = groups.get(key) ?? { + heading, + kind: bucket.kind, + situational: bucket.situational, + unconditional: flags === "" && !remote, + when, + rows: [], + }; + group.rows.push(bucket.row); + groups.set(key, group); + } + return { + scheme: scheme.id, + title: scheme.title, + groups: [...groups.values()] + .sort((a, b) => rank(a) - rank(b)) + .map((group) => ({ + heading: group.heading, + need: needOf(group), + ...(group.when === undefined ? {} : { when: group.when }), + rows: group.rows, + })), + }; +} + +function notNeededWithOtherTargets(catalog: FlagCatalog): string { + const flags = ["local", "db-url"].filter((name) => catalog.get(name)?.hidden === false); + return flags.length === 0 ? "" : `Not needed with ${joinOr(flags.map((flag) => `--${flag}`))}.`; +} + +/** + * What help shows for a command's declared permissions, or `undefined` when the command calls no + * API operation. Each registered scheme is resolved on its own. + */ +export function resolvePermissionSection( + permissions: CommandPermissions, + catalog: FlagCatalog, + schemes: ReadonlyArray<PermissionScheme> = permissionSchemes, +): PermissionSection | undefined { + if (permissions.status === "unmapped") return { status: "unmapped" }; + const context = conditionContext(permissions.operations, catalog, permissions.dbTargetDefault); + const reachable = permissions.operations.flatMap((entry): ReadonlyArray<ReachableEntry> => { + const reduced = termOf(entry.when, context); + return reduced === undefined + ? [] + : [{ entry, term: reduced.term, hedged: entry.context !== undefined || reduced.widened }]; + }); + if (reachable.length === 0) return undefined; + + const { dbTargetDefault } = permissions; + const liftTarget = dbTargetDefault === "linked" && onlyLinked(reachable.map(({ term }) => term)); + const withoutLifted = liftTarget + ? reachable.map((item) => ({ ...item, term: withoutVariables(item.term, [TARGET_VARIABLE]) })) + : reachable; + const common = commonConditions( + withoutLifted.map(({ term }) => term), + context, + ); + const terms = withoutLifted.map((item) => ({ + ...item, + term: withoutVariables(item.term, common.variables), + })); + const note = [ + ...(liftTarget ? [notNeededWithOtherTargets(catalog)] : []), + ...(common.positive.length > 0 ? [`Only needed with ${joinAnd(common.positive)}.`] : []), + ...(common.negative.length > 0 ? [`Not needed with ${joinOr(common.negative)}.`] : []), + ] + .filter((sentence) => sentence !== "") + .join(" "); + const when: ReadonlyArray<ConditionData> = [ + ...(liftTarget ? [{ target: ["linked"] }] : []), + ...common.conditions, + ]; + return { + status: "mapped", + ...(note === "" ? {} : { note }), + ...(when.length === 0 ? {} : { when }), + schemes: schemes.map((scheme) => schemeSection(scheme, terms, context)), + }; +} + +const WRAP_AT = 56; + +const altText = (alternative: ReadonlyArray<string>) => alternative.join(" + "); + +const wraps = (anyOf: Alternatives) => + anyOf.some((alternative) => altText(alternative).length > WRAP_AT) || + (anyOf.length > 1 && anyOf.map(altText).join(" or ").length > WRAP_AT); + +function renderRow(row: PermissionRow): Array<string> { + if ("note" in row) return [` ${row.note}`]; + if ("cases" in row) { + return row.cases.map( + ({ for: target, allOf }, index) => + ` ${index === 0 ? "" : " or "}${altText(allOf)} for ${target}`, + ); + } + if (!wraps(row.anyOf)) return [` ${row.anyOf.map(altText).join(" or ")}`]; + return row.anyOf.flatMap((alternative, index) => { + const [head, ...more] = + altText(alternative).length > WRAP_AT ? alternative : [altText(alternative)]; + return [ + index === 0 ? ` ${head}` : ` or ${head}`, + ...more.map((permission) => ` ${index === 0 ? "" : " "}+ ${permission}`), + ]; + }); +} + +/** The PERMISSIONS help section as text. */ +export function renderPermissionsSection(section: PermissionSection): string { + if (section.status === "unmapped") { + return "PERMISSIONS\n Token permissions for this command are not documented yet."; + } + const lines = ["PERMISSIONS"]; + if (section.note !== undefined) lines.push(` ${section.note}`); + for (const scheme of section.schemes) { + lines.push(` ${scheme.title}:`); + for (const group of scheme.groups) { + lines.push(` ${group.heading}:`); + for (const row of group.rows) lines.push(...renderRow(row)); + } + } + return lines.join("\n"); +} + +/** The section for the command a help document describes, when it declares permissions. */ +export function permissionSectionForHelp(doc: HelpDoc.HelpDoc): PermissionSection | undefined { + return Option.flatMap( + Context.getOption(doc.annotations, CommandPermissionsAnnotation), + (declared) => Option.fromNullishOr(resolvePermissionSection(declared, flagCatalogFor(doc))), + ).pipe(Option.getOrUndefined); +} + +/** The help document without the annotations that hold a command's raw permission declaration. */ +export function withoutPermissionAnnotations(doc: HelpDoc.HelpDoc): HelpDoc.HelpDoc { + return { + ...doc, + annotations: Context.omit( + CommandPermissionsAnnotation, + CommandFlagsAnnotation, + )(doc.annotations), + }; +} diff --git a/apps/cli/src/command-internal/command-permissions/help-section.unit.test.ts b/apps/cli/src/command-internal/command-permissions/help-section.unit.test.ts new file mode 100644 index 0000000000..1374151e27 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/help-section.unit.test.ts @@ -0,0 +1,1211 @@ +import { Flag } from "effect/cli"; +import type { Param } from "effect/cli"; +import { describe, expect, it } from "vitest"; + +import { renderPermissionsSection, resolvePermissionSection } from "./help-section.ts"; +import { mapped } from "./model.ts"; +import type { CommandPermissions, OperationEntry } from "./model.ts"; +import { permissionFlagsOf } from "./permission-flags.ts"; +import type { FlagCatalog } from "./permission-flags.ts"; +import { fgaScheme, oauthScheme } from "./permission-schemes.ts"; +import type { PermissionScheme } from "./permission-schemes.ts"; + +const catalogOf = (...params: ReadonlyArray<Param.AnyFlag>): FlagCatalog => + new Map(permissionFlagsOf(params).map((flag) => [flag.name, flag])); + +const switchFlag = (name: string) => Flag.Boolean(name).pipe(Flag.withDefault(false)); +const projectRef = Flag.String("project-ref").pipe(Flag.optional); + +interface Declaration { + readonly operations: ReadonlyArray<OperationEntry>; + readonly dbTargetDefault?: "local" | "linked"; +} + +function render( + declaration: Declaration, + catalog: FlagCatalog = catalogOf(), + schemes?: ReadonlyArray<PermissionScheme>, +): string | undefined { + const section = resolvePermissionSection(mapped(declaration), catalog, schemes); + return section === undefined ? undefined : renderPermissionsSection(section); +} + +const lines = (...text: ReadonlyArray<string>) => text.join("\n"); +const fga = " Scoped access token:"; + +describe("permission help section", () => { + describe("conditions", () => { + it("drops a switch condition when the operation is declared with and without it", () => { + const text = render( + { + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "dry", is: true }], + }, + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "dry", is: false }], + }, + ], + }, + catalogOf(switchFlag("dry")), + ); + + expect(text).toBe(lines("PERMISSIONS", fga, " Required:", " projects_read")); + }); + + it("merges a flag's given, empty and unset cases into one unconditional entry", () => { + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const text = render( + { + operations: [ + { ...operation, when: [{ flag: "project-ref" }] }, + { ...operation, when: [{ flag: "project-ref", oneOf: [""] }] }, + { ...operation, when: [{ flag: "project-ref", present: false }] }, + ], + }, + catalogOf(projectRef), + ); + + expect(text).toBe( + lines("PERMISSIONS", fga, " Required:", " database_pooling_config_read"), + ); + }); + + it("words an empty value as the flag being given", () => { + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const text = render( + { + operations: [ + { ...operation, when: [{ flag: "project-ref" }] }, + { ...operation, when: [{ flag: "project-ref", oneOf: [""] }] }, + ], + }, + catalogOf(projectRef), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required with --project-ref:", + " database_pooling_config_read", + ), + ); + }); + + it("drops a condition once a choice flag's values cover every choice", () => { + const type = Flag.Literals("type", ["security", "performance", "all"] as const).pipe( + Flag.optional, + ); + const text = render( + { + operations: [ + { + operationId: "v1-get-security-advisors", + kind: "required", + when: [{ flag: "type", oneOf: ["all", "security"] }], + }, + { + operationId: "v1-get-security-advisors", + kind: "required", + when: [{ flag: "type", present: false }], + }, + { + operationId: "v1-get-performance-advisors", + kind: "required", + when: [{ flag: "type", oneOf: ["all", "performance"] }], + }, + { + operationId: "v1-get-performance-advisors", + kind: "required", + when: [{ flag: "type", present: false }], + }, + ], + }, + catalogOf(type), + ); + + expect(text).toBe(lines("PERMISSIONS", fga, " Required:", " advisors_read")); + }); + + it("names the values of a choice flag when they do not cover every choice", () => { + const lang = Flag.Literals("lang", ["typescript", "go", "swift"] as const).pipe( + Flag.withDefault("typescript" as const), + ); + const text = render( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { + operationId: "v1-get-project", + kind: "required", + when: [{ flag: "lang", oneOf: ["go", "swift"] }], + }, + ], + }, + catalogOf(lang), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Required with --lang=go|swift:", + " project_admin_read", + ), + ); + }); + + it("adds 'or unset' to text and the default to data, only when the condition includes the default", () => { + const lang = Flag.Literals("lang", ["typescript", "go", "swift"] as const).pipe( + Flag.withDefault("typescript" as const), + ); + const declaration = (values: ReadonlyArray<string>): Declaration => ({ + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { + operationId: "v1-get-project", + kind: "required", + when: [{ flag: "lang", oneOf: values }], + }, + ], + }); + const catalog = catalogOf(lang); + const groupOf = (values: ReadonlyArray<string>) => { + const section = resolvePermissionSection(mapped(declaration(values)), catalog); + if (section?.status !== "mapped") throw new Error("expected a mapped section"); + return section.schemes[0]?.groups[1]; + }; + + expect(groupOf(["typescript"])).toMatchObject({ + heading: "Required with --lang=typescript or unset", + when: [{ all: [{ flag: "lang", values: ["typescript"], default: "typescript" }] }], + }); + expect(groupOf(["typescript", "go"])).toMatchObject({ + heading: "Required with --lang=typescript|go or unset", + }); + expect(groupOf(["go"])?.when).toEqual([{ all: [{ flag: "lang", values: ["go"] }] }]); + expect(groupOf(["go"])?.heading).toBe("Required with --lang=go"); + }); + + it("ignores a condition on a flag the command lacks when it holds for an unset flag", () => { + const text = render({ + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "password", present: false }], + }, + { + operationId: "v1-get-project", + kind: "best-effort", + when: [{ flag: "gone", is: false }], + }, + ], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Optional (command still runs without these):", + " project_admin_read", + ), + ); + }); + + it("drops an entry whose condition needs a flag the command lacks", () => { + const text = render({ + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { operationId: "v1-get-project", kind: "required", when: [{ flag: "gone", is: true }] }, + ], + }); + + expect(text).toBe(lines("PERMISSIONS", fga, " Required:", " projects_read")); + }); + + it("never names a hidden flag, and hedges what it gated", () => { + const hidden = Flag.Boolean("legacy").pipe(Flag.withDefault(false), Flag.withHidden); + const text = render( + { + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "legacy", is: true }], + }, + ], + }, + catalogOf(hidden), + ); + + expect(text).toBe(lines("PERMISSIONS", fga, " May be required:", " projects_read")); + }); + + it("covers every value of a text flag when a literal, any value and no value all apply", () => { + const from = Flag.String("from").pipe(Flag.optional); + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const text = render( + { + operations: [ + { ...operation, when: [{ flag: "from", oneOf: ["linked"] }] }, + { ...operation, when: [{ flag: "from" }] }, + { ...operation, when: [{ flag: "from", present: false }] }, + ], + }, + catalogOf(from), + ); + + expect(text).toBe( + lines("PERMISSIONS", fga, " Required:", " database_pooling_config_read"), + ); + }); + + it("words a text flag's literal and any given value as the flag being given", () => { + const from = Flag.String("from").pipe(Flag.optional); + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const text = render( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { ...operation, when: [{ flag: "from", oneOf: ["linked"] }] }, + { ...operation, when: [{ flag: "from" }] }, + ], + }, + catalogOf(from), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Required with --from:", + " database_pooling_config_read", + ), + ); + }); + + it("keeps a condition on an optional choice flag even when it names every choice", () => { + const type = Flag.Literals("type", ["security", "performance"] as const).pipe(Flag.optional); + const text = render( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { operationId: "v1-get-project", kind: "required", when: [{ flag: "type" }] }, + ], + }, + catalogOf(type), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Required with --type=security|performance:", + " project_admin_read", + ), + ); + }); + + it("separates alternatives that each combine several flags", () => { + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const text = render( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { + ...operation, + when: [ + { flag: "a", is: true }, + { flag: "b", is: true }, + ], + }, + { + ...operation, + when: [ + { flag: "c", is: true }, + { flag: "d", is: false }, + ], + }, + ], + }, + catalogOf(switchFlag("a"), switchFlag("b"), switchFlag("c"), switchFlag("d")), + ); + + expect(text).toContain( + " Required with --a and --b; or with --c, without --d:\n database_pooling_config_read", + ); + }); + + it("words alternative flags as one list", () => { + const text = render( + { + operations: [ + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "from", oneOf: ["linked"] }], + }, + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "to", oneOf: ["linked"] }], + }, + ], + }, + catalogOf(Flag.String("from").pipe(Flag.optional), Flag.String("to").pipe(Flag.optional)), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required with --from=linked or --to=linked:", + " database_pooling_config_read", + ), + ); + }); + }); + + describe("grouping", () => { + it("lists a permission once, under the strongest group that covers it", () => { + const text = render( + { + operations: [ + { operationId: "v1-get-project", kind: "best-effort" }, + { operationId: "v1-get-project", kind: "required" }, + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "dry", is: false }], + }, + { operationId: "v1-get-pooler-config", kind: "required" }, + ], + }, + catalogOf(switchFlag("dry")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " project_admin_read", + " database_pooling_config_read", + ), + ); + }); + + it("drops an Optional row that a May be required group already lists", () => { + const text = render({ + operations: [ + { operationId: "v1-get-project", kind: "required", context: "a prompt picks it" }, + { operationId: "v1-get-project", kind: "best-effort" }, + ], + }); + + expect(text).toBe( + lines("PERMISSIONS", fga, " May be required:", " project_admin_read"), + ); + }); + + it("keeps a permission under Required when a situational group also lists it", () => { + const text = render({ + operations: [ + { operationId: "v1-get-project", kind: "required" }, + { operationId: "v1-get-project", kind: "required", context: "a prompt picks it" }, + ], + }); + + expect(text).toBe(lines("PERMISSIONS", fga, " Required:", " project_admin_read")); + }); + + it("does not call a permission optional when another heading requires it", () => { + const text = render( + { + operations: [ + { + operationId: "v1-get-project", + kind: "required", + when: [{ flag: "strict", is: true }], + }, + { operationId: "v1-get-project", kind: "best-effort" }, + { operationId: "v1-list-all-projects", kind: "required" }, + ], + }, + catalogOf(switchFlag("strict")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Required with --strict:", + " project_admin_read", + ), + ); + }); + + it("never shows the reason behind a situational entry", () => { + const text = render({ + operations: [ + { + operationId: "v1-get-project", + kind: "required", + context: "the user picks the project", + }, + ], + }); + + expect(text).toBe( + lines("PERMISSIONS", fga, " May be required:", " project_admin_read"), + ); + expect(text).not.toContain("picks"); + }); + + it("orders required before optional and unconditional before conditional", () => { + const text = render( + { + operations: [ + { operationId: "v1-get-project", kind: "best-effort" }, + { operationId: "v1-get-pooler-config", kind: "required", context: "needs a ref" }, + { + operationId: "v1-get-database-metadata", + kind: "required", + when: [{ flag: "dry", is: true }], + }, + { operationId: "v1-list-all-projects", kind: "required" }, + ], + }, + catalogOf(switchFlag("dry")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " projects_read", + " Required with --dry:", + " database_read", + " May be required:", + " database_pooling_config_read", + " Optional (command still runs without these):", + " project_admin_read", + ), + ); + }); + }); + + describe("shared conditions", () => { + it("says a command is not needed with the flags every entry excludes", () => { + const text = render( + { + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "local", is: false }], + }, + { + operationId: "v1-get-project", + kind: "best-effort", + when: [{ flag: "local", is: false }], + }, + ], + }, + catalogOf(switchFlag("local")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + " Not needed with --local.", + fga, + " Required:", + " projects_read", + " Optional (command still runs without these):", + " project_admin_read", + ), + ); + }); + + it("says a command is only needed with the flag every entry requires", () => { + const text = render( + { + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "linked", is: true }], + }, + ], + }, + catalogOf(switchFlag("linked")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + " Only needed with --linked.", + fga, + " Required:", + " projects_read", + ), + ); + }); + + it("says it is not needed with the other targets when the default is the linked project", () => { + const text = render( + { + dbTargetDefault: "linked", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + ], + }, + catalogOf( + switchFlag("local"), + Flag.String("db-url").pipe(Flag.optional), + switchFlag("linked"), + ), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + " Not needed with --local or --db-url.", + fga, + " Required:", + " projects_read", + ), + ); + }); + + it("says it is only needed with --linked when the default is the local database", () => { + const text = render( + { + dbTargetDefault: "local", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + ], + }, + catalogOf(switchFlag("local"), switchFlag("linked")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + " Only needed with --linked.", + fga, + " Required:", + " projects_read", + ), + ); + }); + + it("keeps the target in the heading when some entries do not depend on it", () => { + const text = render( + { + dbTargetDefault: "linked", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + { operationId: "v1-get-project", kind: "required" }, + ], + }, + catalogOf(switchFlag("local"), switchFlag("linked")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " project_admin_read", + " Required (remote only):", + " projects_read", + ), + ); + }); + + it("drops a target condition the command cannot vary", () => { + const text = render({ + dbTargetDefault: "linked", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + ], + }); + + expect(text).toBe(lines("PERMISSIONS", fga, " Required:", " projects_read")); + }); + }); + + describe("schemes", () => { + it("shows the secret-read permission when the request reveals secret keys", () => { + const text = render( + { + operations: [ + { + operationId: "v1-get-project-api-keys", + kind: "required", + when: [{ flag: "reveal", is: true }], + request: { query: { reveal: "true" } }, + }, + { + operationId: "v1-get-project-api-keys", + kind: "required", + when: [{ flag: "reveal", is: false }], + }, + ], + }, + catalogOf(switchFlag("reveal")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required with --reveal:", + " api_gateway_keys_read + api_gateway_keys_secret_read", + " Required without --reveal:", + " api_gateway_keys_read", + ), + ); + }); + + it("says when the spec lists no token permission for an operation", () => { + const text = render({ + operations: [{ operationId: "v1-authorize-user", kind: "required" }], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " no token permission listed in the API spec", + ), + ); + }); + + it("asks for a user-scoped token where the profile endpoint rejects other tokens", () => { + const text = render({ operations: [{ operationId: "v1-get-profile", kind: "required" }] }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " a user-scoped token (project- and organization-scoped tokens are rejected)", + ), + ); + }); + + it("keeps alternatives on one line while they fit the help width", () => { + const text = render({ + operations: [{ operationId: "v1-list-all-branches", kind: "required" }], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " branching_development_read or branching_production_read", + ), + ); + }); + + it("puts each alternative on its own line once together they exceed the help width", () => { + const first = ["a".repeat(30)]; + const second = ["b".repeat(30)]; + const scheme: PermissionScheme = { + ...fgaScheme, + requirement: () => [{ requirement: { anyOf: [first, second] }, hedged: false }], + }; + const text = render( + { operations: [{ operationId: "v1-get-project", kind: "required" }] }, + catalogOf(), + [scheme], + ); + + expect(text).toBe( + lines("PERMISSIONS", fga, " Required:", ` ${first[0]}`, ` or ${second[0]}`), + ); + }); + + it("shows a permission the API needs only for some requests apart from the one it always needs", () => { + const text = render( + { + operations: [ + { operationId: "v1-run-a-query", kind: "required", when: [{ flag: "dry", is: true }] }, + ], + }, + catalogOf(switchFlag("dry")), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + " Only needed with --dry.", + fga, + " Required:", + " database_read", + " May be required:", + " database_write", + ), + ); + }); + + it("lists a conditional extra once when several entries add it", () => { + const text = render({ + operations: [ + { operationId: "v1-run-a-query", kind: "required" }, + { operationId: "v1-run-a-query", kind: "required", context: "a prompt picks it" }, + ], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " database_read", + " May be required:", + " database_write", + ), + ); + }); + + it("hedges a conditional extra of an entry that is already situational", () => { + const section = resolvePermissionSection( + mapped({ + operations: [ + { operationId: "v1-run-a-query", kind: "required", context: "a prompt picks it" }, + ], + }), + catalogOf(), + ); + + expect( + section?.status === "mapped" && + section.schemes[0]?.groups.map((group) => [group.need, group.rows]), + ).toEqual([["maybe", [{ anyOf: [["database_read"]] }, { anyOf: [["database_write"]] }]]]); + }); + + it("names the resource that decides which permission an operation needs", () => { + const section = resolvePermissionSection( + mapped({ operations: [{ operationId: "v1-get-a-branch", kind: "required" }] }), + catalogOf(), + ); + if (section?.status !== "mapped") throw new Error("expected a mapped section"); + + expect(section.schemes[0]?.groups[0]?.rows).toEqual([ + { + cases: [ + { for: "a preview branch", allOf: ["branching_development_read"] }, + { for: "the main branch", allOf: ["branching_production_read"] }, + ], + }, + ]); + expect(renderPermissionsSection(section)).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " branching_development_read for a preview branch", + " or branching_production_read for the main branch", + ), + ); + }); + + it("drops an any-of row that a per-resource row over the same permissions already covers", () => { + const text = render({ + operations: [ + { operationId: "v1-list-all-branches", kind: "required" }, + { operationId: "v1-get-a-branch", kind: "required" }, + ], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " branching_development_read for a preview branch", + " or branching_production_read for the main branch", + ), + ); + }); + + it("keeps an any-of row when the per-resource row asks for different permissions", () => { + const text = render({ + operations: [ + { operationId: "v1-list-all-branches", kind: "required" }, + { operationId: "v1-delete-a-branch", kind: "required" }, + ], + }); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " branching_development_read or branching_production_read", + " branching_development_delete for a preview branch", + " or branching_production_delete for the main branch", + ), + ); + }); + + it("shows the permission a request body detail needs next to the plain set", () => { + const withData = Flag.Boolean("with-data").pipe(Flag.optional); + const text = render( + { + operations: [ + { operationId: "v1-create-a-branch", kind: "required" }, + { + operationId: "v1-create-a-branch", + kind: "required", + when: [{ flag: "with-data" }], + request: { body: { with_data: "true" } }, + }, + ], + }, + catalogOf(withData), + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " branching_development_create", + " or branching_production_create", + " Required with --with-data:", + " branching_production_create", + ), + ); + }); + + it("breaks a single alternative that is too long across lines", () => { + const longSet = ["a".repeat(30), "b".repeat(30)]; + const scheme: PermissionScheme = { + ...fgaScheme, + requirement: () => [{ requirement: { anyOf: [longSet, ["c"]] }, hedged: false }], + }; + const text = render( + { operations: [{ operationId: "v1-get-project", kind: "required" }] }, + catalogOf(), + [scheme], + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + ` ${longSet[0]}`, + ` + ${longSet[1]}`, + " or c", + ), + ); + }); + + it("renders each registered scheme as its own block", () => { + const text = render( + { + operations: [ + { operationId: "v1-get-project", kind: "required" }, + { operationId: "v1-get-storage-config", kind: "best-effort" }, + ], + }, + catalogOf(), + [fgaScheme, oauthScheme], + ); + + expect(text).toBe( + lines( + "PERMISSIONS", + fga, + " Required:", + " project_admin_read", + " Optional (command still runs without these):", + " storage_config_read", + " OAuth token (scopes):", + " Required:", + " projects:read", + " Optional (command still runs without these):", + " skipped with OAuth tokens (not available yet)", + ), + ); + }); + + it("says an operation is not available with OAuth tokens yet when it has no scope", () => { + const text = render( + { operations: [{ operationId: "v1-get-storage-config", kind: "required" }] }, + catalogOf(), + [oauthScheme], + ); + + expect(text).toContain("not available with OAuth tokens yet"); + }); + }); + + describe("need", () => { + it("tells required, maybe and optional apart", () => { + const section = resolvePermissionSection( + mapped({ + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { operationId: "v1-get-pooler-config", kind: "required", context: "a prompt picks it" }, + { operationId: "v1-get-project", kind: "best-effort", context: "a cache is cold" }, + ], + }), + catalogOf(), + ); + + expect( + section?.status === "mapped" && + section.schemes[0]?.groups.map((group) => [group.heading, group.need]), + ).toEqual([ + ["Required", "required"], + ["May be required", "maybe"], + ["Optional (command still runs without these)", "optional"], + ]); + }); + }); + + describe("structured conditions", () => { + const whenOf = (declaration: Declaration, catalog: FlagCatalog) => { + const section = resolvePermissionSection(mapped(declaration), catalog); + if (section?.status !== "mapped") throw new Error("expected a mapped section"); + return { + shared: section.when, + groups: section.schemes.flatMap((scheme) => + scheme.groups.map((group) => [group.heading, group.when] as const), + ), + }; + }; + + it("gives each group the flag values it applies to, naming an unset or empty flag", () => { + const { groups } = whenOf( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { + operationId: "v1-get-project", + kind: "required", + when: [{ flag: "dry", is: false }], + }, + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "project-ref", present: false }], + }, + ], + }, + catalogOf(switchFlag("dry"), projectRef), + ); + + expect(groups).toEqual([ + ["Required", undefined], + ["Required without --dry", [{ all: [{ flag: "dry", values: ["false"] }] }]], + [ + "Required without --project-ref", + [{ all: [{ flag: "project-ref", values: ["unset", "empty"] }] }], + ], + ]); + }); + + it("lists each way a group can apply as its own alternative", () => { + const operation = { operationId: "v1-get-pooler-config", kind: "required" } as const; + const { groups } = whenOf( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { ...operation, when: [{ flag: "from", oneOf: ["linked"] }] }, + { + ...operation, + when: [ + { flag: "to", oneOf: ["linked"] }, + { flag: "dry", is: true }, + ], + }, + ], + }, + catalogOf( + Flag.String("from").pipe(Flag.optional), + Flag.String("to").pipe(Flag.optional), + switchFlag("dry"), + ), + ); + + expect(groups[1]?.[1]).toEqual([ + { all: [{ flag: "from", values: ["linked"] }] }, + { + all: [ + { flag: "to", values: ["linked"] }, + { flag: "dry", values: ["true"] }, + ], + }, + ]); + }); + + it("reports what holds for every group as one shared condition", () => { + const { shared, groups } = whenOf( + { + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ flag: "local", is: false }], + }, + { + operationId: "v1-get-project", + kind: "best-effort", + when: [{ flag: "local", is: false }], + }, + ], + }, + catalogOf(switchFlag("local")), + ); + + expect(shared).toEqual([{ flag: "local", values: ["false"] }]); + expect(groups.map(([, when]) => when)).toEqual([undefined, undefined]); + }); + + it("names the linked project as the shared target when it is the default", () => { + const { shared } = whenOf( + { + dbTargetDefault: "linked", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + ], + }, + catalogOf(switchFlag("local"), switchFlag("linked")), + ); + + expect(shared).toEqual([{ target: ["linked"] }]); + }); + + it("keeps rows under different conditions apart even when the headings read alike", () => { + const { groups } = whenOf( + { + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { operationId: "v1-get-project", kind: "required", when: [{ flag: "project-ref" }] }, + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "project-ref", oneOf: [""] }], + }, + ], + }, + catalogOf(projectRef), + ); + + expect(groups.slice(1)).toEqual([ + ["Required with --project-ref", [{ all: [{ flag: "project-ref", values: ["set"] }] }]], + ['Required with --project-ref=""', [{ all: [{ flag: "project-ref", values: ["empty"] }] }]], + ]); + }); + + it("treats naming a target flag as selecting that target", () => { + const { groups } = whenOf( + { + dbTargetDefault: "local", + operations: [ + { operationId: "v1-list-all-projects", kind: "required" }, + { + operationId: "v1-get-project", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + { + operationId: "v1-get-pooler-config", + kind: "required", + when: [{ flag: "linked", is: true }], + }, + ], + }, + catalogOf(switchFlag("local"), switchFlag("linked")), + ); + + expect(groups).toEqual([ + ["Required", undefined], + ["Required with --linked", [{ all: [{ target: ["linked"] }] }]], + ]); + }); + + it("names a target in a group's condition when not every entry depends on it", () => { + const { shared, groups } = whenOf( + { + dbTargetDefault: "local", + operations: [ + { + operationId: "v1-list-all-projects", + kind: "required", + when: [{ dbTarget: ["linked"] }], + }, + { operationId: "v1-get-project", kind: "required" }, + ], + }, + catalogOf(switchFlag("local"), switchFlag("linked")), + ); + + expect(shared).toBeUndefined(); + expect(groups[1]).toEqual(["Required with --linked", [{ all: [{ target: ["linked"] }] }]]); + }); + }); + + describe("status", () => { + it("says a command's permissions are not documented when it is unmapped", () => { + const unmapped: CommandPermissions = { status: "unmapped", reason: "go-delegated" }; + const section = resolvePermissionSection(unmapped, catalogOf()); + + expect(section).toEqual({ status: "unmapped" }); + expect(section && renderPermissionsSection(section)).toBe( + lines("PERMISSIONS", " Token permissions for this command are not documented yet."), + ); + }); + + it("shows nothing for a command that calls no API operation", () => { + expect(render({ operations: [] })).toBeUndefined(); + }); + }); +}); diff --git a/apps/cli/src/command-internal/command-permissions/model.ts b/apps/cli/src/command-internal/command-permissions/model.ts index 9d65e3fb61..8f063ab9ea 100644 --- a/apps/cli/src/command-internal/command-permissions/model.ts +++ b/apps/cli/src/command-internal/command-permissions/model.ts @@ -71,6 +71,12 @@ interface DbTargetCondition { export type EntryCondition = FlagCondition | DbTargetCondition; +/** The parts of a request a permission can depend on, string-encoded like the request on the wire. */ +export interface RequestDetails { + readonly query?: Readonly<Record<string, string>>; + readonly body?: Readonly<Record<string, string>>; +} + export interface OperationEntry { /** The spec's `operationId`, e.g. `"v1-list-all-secrets"`. */ readonly operationId: string; @@ -80,7 +86,7 @@ export interface OperationEntry { /** A condition that isn't a flag, e.g. "only without a saved .temp/pooler-url". */ readonly context?: string; /** Request details a permission can depend on, e.g. `reveal=true` on the api-keys call. */ - readonly request?: { readonly query?: Readonly<Record<string, string>> }; + readonly request?: RequestDetails; /** The building block this entry came from (set by {@link block}); absent for a command's own entry. */ readonly source?: string; } diff --git a/apps/cli/src/command-internal/command-permissions/permission-conditions.ts b/apps/cli/src/command-internal/command-permissions/permission-conditions.ts new file mode 100644 index 0000000000..020dd48408 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/permission-conditions.ts @@ -0,0 +1,439 @@ +import type { DbTarget, EntryCondition, OperationEntry } from "./model.ts"; +import type { FlagCatalog, PermissionFlag } from "./permission-flags.ts"; + +const ABSENT = "\u0000absent"; +const EMPTY = "\u0000empty"; +const SET = "\u0000set"; +const TARGET = "\u0000target"; + +const TARGET_FLAGS = ["db-url", "local", "linked"] as const satisfies ReadonlyArray<DbTarget>; + +const isLiteral = (value: string) => ![ABSENT, EMPTY, SET].includes(value); + +/** A literal value is a case of "given any value", so a set holding both keeps only `SET`. */ +function withoutSubsumedLiterals(values: ReadonlySet<string>): ReadonlySet<string> { + return values.has(SET) ? new Set([...values].filter((value) => !isLiteral(value))) : values; +} + +function intersect(a: ReadonlySet<string>, b: ReadonlySet<string>): ReadonlySet<string> { + const common = [...a].filter((value) => b.has(value) || (isLiteral(value) && b.has(SET))); + const fromB = [...b].filter((value) => isLiteral(value) && a.has(SET)); + return new Set([...common, ...fromB]); +} + +/** + * What a run must look like for an entry to apply: each listed variable (a flag, or the database + * target) takes one of the listed values, and unlisted variables are unrestricted. + */ +export type Term = ReadonlyMap<string, ReadonlySet<string>>; + +/** The OR of terms. A single empty term means "always". */ +export type Condition = ReadonlyArray<Term>; + +export interface ConditionContext { + readonly catalog: FlagCatalog; + /** The target a run uses when no target flag is given; set with `withDbTarget`. */ + readonly targetDefault: "local" | "linked" | undefined; + /** The values a variable can take; `undefined` for a variable the command does not have. */ + readonly domainOf: (variable: string) => ReadonlyArray<string> | undefined; + /** The database targets a run can select. */ + readonly targets: ReadonlyArray<string>; +} + +export function conditionContext( + entries: ReadonlyArray<OperationEntry>, + catalog: FlagCatalog, + targetDefault: "local" | "linked" | undefined, +): ConditionContext { + const conditions = entries.flatMap((entry) => entry.when ?? []); + const targets = [ + ...TARGET_FLAGS.filter((flag) => catalog.has(flag)), + ...(targetDefault !== undefined && !catalog.has(targetDefault) ? [targetDefault] : []), + ]; + const domainOf = (variable: string): ReadonlyArray<string> | undefined => { + if (variable === TARGET) return targets; + const flag = catalog.get(variable); + if (flag === undefined) return undefined; + if (flag.kind === "boolean") return ["true", "false"]; + if (flag.kind === "choice") { + const absenceIsTested = conditions.some( + (condition) => condition.flag === variable && condition.present === false, + ); + return flag.optional || absenceIsTested ? [...flag.choices, ABSENT] : flag.choices; + } + return [ABSENT, EMPTY, SET]; + }; + return { catalog, targetDefault, domainOf, targets }; +} + +type Constraint = + | { readonly kind: "vacuous" } + | { readonly kind: "unreachable" } + | { readonly kind: "restrict"; readonly variable: string; readonly values: ReadonlySet<string> }; + +const VACUOUS: Constraint = { kind: "vacuous" }; +const UNREACHABLE: Constraint = { kind: "unreachable" }; + +function restrict( + context: ConditionContext, + variable: string, + values: ReadonlyArray<string>, +): Constraint { + const unique = withoutSubsumedLiterals(new Set(values)); + if (unique.size === 0) return UNREACHABLE; + const domain = context.domainOf(variable); + if (domain !== undefined && domain.every((value) => unique.has(value))) return VACUOUS; + return { kind: "restrict", variable, values: unique }; +} + +function targetConstraint(context: ConditionContext, targets: ReadonlyArray<DbTarget>): Constraint { + if (context.targets.length === 0) return VACUOUS; + return restrict( + context, + TARGET, + targets.filter((target) => context.targets.includes(target)), + ); +} + +function missingFlagConstraint(condition: EntryCondition): Constraint { + if (condition.is !== undefined) return condition.is ? UNREACHABLE : VACUOUS; + if (condition.oneOf !== undefined) return UNREACHABLE; + return (condition.present ?? true) ? UNREACHABLE : VACUOUS; +} + +function flagConstraint( + context: ConditionContext, + flag: PermissionFlag, + condition: EntryCondition, +): Constraint { + if (flag.hidden) return VACUOUS; + const variable = flag.name; + // Naming a target flag selects that target, so it is the same restriction as the target condition. + const selected = TARGET_FLAGS.find((target) => target === variable); + if ( + selected !== undefined && + context.targets.length > 1 && + condition.oneOf === undefined && + requiresFlag(condition) + ) { + return targetConstraint(context, [selected]); + } + if (flag.kind === "boolean") { + if (condition.is !== undefined) return restrict(context, variable, [String(condition.is)]); + if (condition.oneOf !== undefined) return restrict(context, variable, condition.oneOf); + return restrict(context, variable, [String(condition.present ?? true)]); + } + if (condition.oneOf !== undefined) { + const values = condition.oneOf.map((value) => + value === "" && flag.kind === "text" ? EMPTY : value, + ); + return restrict(context, variable, values); + } + if (condition.present !== undefined || condition.is === undefined) { + const present = condition.present ?? true; + if (flag.kind === "choice") { + return restrict(context, variable, present ? flag.choices : [ABSENT]); + } + return restrict(context, variable, present ? [SET] : [ABSENT, EMPTY]); + } + return VACUOUS; +} + +/** Whether the condition holds only when the flag is given or on. */ +const requiresFlag = (condition: EntryCondition) => + condition.is === true || + condition.oneOf !== undefined || + (condition.is === undefined && condition.present !== false); + +/** + * Reduces an entry's `when` to one term against the command's flags, or `undefined` when no run can + * satisfy it. A condition on a flag the command lacks is vacuous when it holds for an unset flag + * (`is: false`, absence) and unreachable otherwise. A condition on a hidden flag is dropped; when + * it said the entry applies only with that flag, `widened` reports it so the caller can hedge. + */ +export function termOf( + conditions: ReadonlyArray<EntryCondition> | undefined, + context: ConditionContext, +): { readonly term: Term; readonly widened: boolean } | undefined { + const term = new Map<string, ReadonlySet<string>>(); + let widened = false; + for (const condition of conditions ?? []) { + const flag = condition.flag === undefined ? undefined : context.catalog.get(condition.flag); + const constraint = + condition.dbTarget !== undefined + ? targetConstraint(context, condition.dbTarget) + : flag === undefined + ? missingFlagConstraint(condition) + : flagConstraint(context, flag, condition); + if (constraint.kind === "unreachable") return undefined; + if (flag?.hidden === true && requiresFlag(condition)) widened = true; + if (constraint.kind === "vacuous") continue; + const existing = term.get(constraint.variable); + const values = + existing === undefined ? constraint.values : intersect(existing, constraint.values); + if (values.size === 0) return undefined; + term.set(constraint.variable, values); + } + return { term, widened }; +} + +/** Whether every run matching `strong` also matches `weak`. */ +function implies(strong: Term, weak: Term): boolean { + return [...weak].every(([variable, allowed]) => { + const values = strong.get(variable); + return ( + values !== undefined && + [...values].every((value) => allowed.has(value) || (isLiteral(value) && allowed.has(SET))) + ); + }); +} + +const sameValues = (a: ReadonlySet<string>, b: ReadonlySet<string>) => + a.size === b.size && [...a].every((value) => b.has(value)); + +function mergeTerms(a: Term, b: Term, context: ConditionContext): Term | undefined { + if (a.size !== b.size) return undefined; + let differing: string | undefined; + for (const [variable, values] of a) { + const other = b.get(variable); + if (other === undefined) return undefined; + if (sameValues(values, other)) continue; + if (differing !== undefined) return undefined; + differing = variable; + } + if (differing === undefined) return a; + const union = withoutSubsumedLiterals( + new Set([...(a.get(differing) ?? []), ...(b.get(differing) ?? [])]), + ); + const merged = new Map(a); + const domain = context.domainOf(differing); + if (domain !== undefined && domain.every((value) => union.has(value))) merged.delete(differing); + else merged.set(differing, union); + return merged; +} + +/** + * Collapses an OR of terms: drops a term another term makes redundant and merges two terms that + * differ in one variable into one that takes the union of its values, dropping a variable whose + * values cover its whole domain. + */ +export function simplify(terms: Condition, context: ConditionContext): Condition { + let current = terms; + for (;;) { + const kept = current.filter( + (term, index) => + !current.some( + (other, otherIndex) => + otherIndex !== index && + implies(term, other) && + (!implies(other, term) || otherIndex < index), + ), + ); + const merge = kept.flatMap((a, i) => + kept.slice(i + 1).flatMap((b, offset) => { + const merged = mergeTerms(a, b, context); + return merged === undefined ? [] : [{ i, j: i + 1 + offset, merged }]; + }), + )[0]; + if (merge === undefined) return kept; + current = [...kept.filter((_, index) => index !== merge.i && index !== merge.j), merge.merged]; + } +} + +/** Whether every run matching `strong` also matches `weak`, term by term. */ +export function conditionImplies(strong: Condition, weak: Condition): boolean { + return strong.every((term) => weak.some((other) => implies(term, other))); +} + +export const TARGET_VARIABLE = TARGET; + +export function withoutVariables(term: Term, variables: ReadonlyArray<string>): Term { + return new Map([...term].filter(([variable]) => !variables.includes(variable))); +} + +/** Whether every term restricts the target to the linked project alone. */ +export function onlyLinked(terms: ReadonlyArray<Term>): boolean { + return ( + terms.length > 0 && + terms.every((term) => { + const values = term.get(TARGET); + return values !== undefined && values.size === 1 && values.has("linked"); + }) + ); +} + +export const joinAnd = (items: ReadonlyArray<string>) => + items.length < 2 ? items.join("") : `${items.slice(0, -1).join(", ")} and ${items.at(-1)}`; + +export const joinOr = (items: ReadonlyArray<string>) => + items.length < 2 ? items.join("") : `${items.slice(0, -1).join(", ")} or ${items.at(-1)}`; + +interface TermText { + readonly positive: ReadonlyArray<string>; + readonly negative: ReadonlyArray<string>; + readonly remote: boolean; +} + +function textOfVariable( + variable: string, + values: ReadonlySet<string>, + context: ConditionContext, +): Partial<TermText> { + if (variable === TARGET) { + if (values.size === 1 && values.has("linked")) { + return context.targetDefault === "linked" ? { remote: true } : { positive: ["--linked"] }; + } + const named = context.targets.filter((target) => values.has(target)); + return { positive: [joinOr(named.map((target) => `--${target}`))] }; + } + const flag = context.catalog.get(variable); + const name = `--${variable}`; + if (flag?.kind === "boolean") { + return values.has("true") ? { positive: [name] } : { negative: [name] }; + } + const literals = + flag?.kind === "choice" + ? flag.choices.filter((value) => values.has(value)) + : [...values].filter((value) => ![ABSENT, EMPTY, SET].includes(value)).sort(); + if (literals.length > 0) { + const orUnset = + values.has(ABSENT) || (flag?.default !== undefined && literals.includes(flag.default)); + return { positive: [`${name}=${literals.join("|")}${orUnset ? " or unset" : ""}`] }; + } + if (values.has(ABSENT)) return values.has(SET) ? {} : { negative: [name] }; + if (values.has(EMPTY) && !values.has(SET)) return { positive: [`${name}=""`] }; + return { positive: [name] }; +} + +function termText(term: Term, context: ConditionContext): TermText { + const parts = [...term].map(([variable, values]) => textOfVariable(variable, values, context)); + return { + positive: parts.flatMap((part) => part.positive ?? []), + negative: parts.flatMap((part) => part.negative ?? []), + remote: parts.some((part) => part.remote === true), + }; +} + +const flagsPhrase = ({ positive, negative }: TermText) => + [ + ...(positive.length > 0 ? [`with ${joinAnd(positive)}`] : []), + ...(negative.length > 0 ? [`without ${joinAnd(negative)}`] : []), + ].join(", "); + +/** The flag condition as help prints it, and whether it limits the command to the linked project. */ +export function conditionText( + condition: Condition, + context: ConditionContext, +): { readonly flags: string; readonly remote: boolean } { + const texts = condition.map((term) => termText(term, context)); + const [only] = texts; + if (texts.length === 1 && only !== undefined) { + return { flags: flagsPhrase(only), remote: only.remote }; + } + if (texts.every((text) => text.negative.length === 0 && !text.remote)) { + const common = texts[0]?.positive.filter((part) => + texts.every((t) => t.positive.includes(part)), + ); + const rest = texts.map((text) => text.positive.filter((part) => !common?.includes(part))); + if (common !== undefined && rest.every((parts) => parts.length === 1)) { + const alternatives = joinOr(rest.flat()); + return { + flags: `with ${joinAnd(common.length === 0 ? [alternatives] : [...common, `one of ${alternatives}`])}`, + remote: false, + }; + } + } + const phrases = texts.map((text) => + [flagsPhrase(text), ...(text.remote ? ["remote only"] : [])].filter(Boolean).join(", "), + ); + const separator = phrases.some((phrase) => /,| and /.test(phrase)) ? "; or " : " or "; + return { flags: phrases.join(separator), remote: false }; +} + +/** + * The variables every term restricts the same way, split by how help words them: `positive` ones a + * run must have (`--linked`), `negative` ones it must lack (`--local`). + */ +export function commonConditions( + terms: ReadonlyArray<Term>, + context: ConditionContext, +): { + variables: ReadonlyArray<string>; + conditions: ReadonlyArray<ConditionData>; + positive: ReadonlyArray<string>; + negative: ReadonlyArray<string>; +} { + const [first] = terms; + const shared = [...(first ?? [])].filter(([variable, values]) => + terms.every((term) => { + const other = term.get(variable); + return other !== undefined && sameValues(values, other); + }), + ); + const described = shared.map(([variable, values]) => ({ + variable, + text: textOfVariable(variable, values, context), + })); + const usable = described.filter( + ({ text }) => (text.positive?.length ?? 0) + (text.negative?.length ?? 0) > 0, + ); + return { + variables: usable.map(({ variable }) => variable), + conditions: usable.flatMap(({ variable }) => { + const values = first?.get(variable); + return values === undefined ? [] : [conditionData(variable, values, context)]; + }), + positive: usable.flatMap(({ text }) => text.positive ?? []), + negative: usable.flatMap(({ text }) => text.negative ?? []), + }; +} + +/** One restriction as data: a flag takes one of `values`, or the database target is one of `target`. */ +export type ConditionData = + | { + readonly flag: string; + readonly values: ReadonlyArray<string>; + /** The flag's value when absent, present only when `values` includes it. */ + readonly default?: string; + } + | { readonly target: ReadonlyArray<string> }; + +/** One way a group can apply: every condition holds. */ +export interface Alternative { + readonly all: ReadonlyArray<ConditionData>; +} + +const SENTINEL_NAMES = new Map([ + [ABSENT, "unset"], + [EMPTY, "empty"], + [SET, "set"], +]); + +function conditionData( + variable: string, + values: ReadonlySet<string>, + context: ConditionContext, +): ConditionData { + const domain = context.domainOf(variable) ?? []; + const ordered = [...values].sort((a, b) => { + const rank = domain.indexOf(a) - domain.indexOf(b); + return rank === 0 ? a.localeCompare(b) : rank; + }); + const names = ordered.map((value) => SENTINEL_NAMES.get(value) ?? value); + if (variable === TARGET) return { target: names }; + const fallback = context.catalog.get(variable)?.default; + return fallback !== undefined && names.includes(fallback) + ? { flag: variable, values: names, default: fallback } + : { flag: variable, values: names }; +} + +/** The condition as the alternatives it allows, or `undefined` when it always holds. */ +export function alternativesOf( + condition: Condition, + context: ConditionContext, +): ReadonlyArray<Alternative> | undefined { + if (condition.some((term) => term.size === 0)) return undefined; + return condition.map((term) => ({ + all: [...term].map(([variable, values]) => conditionData(variable, values, context)), + })); +} diff --git a/apps/cli/src/command-internal/command-permissions/permission-flags.ts b/apps/cli/src/command-internal/command-permissions/permission-flags.ts new file mode 100644 index 0000000000..43b4898c28 --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/permission-flags.ts @@ -0,0 +1,104 @@ +import { Context, Option } from "effect"; +import type { HelpDoc, Param } from "effect/cli"; + +import { choiceKeysOf } from "../../docs/docs-introspection.ts"; +import { ROOT_GLOBAL_FLAGS } from "../global-flags.ts"; +import { unwrapParam } from "../param-introspection.ts"; + +/** What a permission condition needs to know about a flag: its value shape and whether help shows it. */ +export interface PermissionFlag { + readonly name: string; + readonly kind: "boolean" | "choice" | "text"; + readonly hidden: boolean; + /** The accepted values of a `choice` flag; empty otherwise. */ + readonly choices: ReadonlyArray<string>; + /** Declared with `Flag.optional` outermost, so it can be unset; a flag with a default is not. */ + readonly optional: boolean; + /** The value a `choice` or `text` flag takes when absent, when it is declared with `Flag.withDefault`. */ + readonly default?: string; +} + +/** The flags a command accepts, by name. */ +export type FlagCatalog = ReadonlyMap<string, PermissionFlag>; + +/** + * Context key under which {@link withPermissions} stores a function that reads the annotated + * command's own flags, so the command's internals are inspected only when help is rendered. + */ +export class CommandFlagsAnnotation extends Context.Service< + CommandFlagsAnnotation, + () => ReadonlyArray<PermissionFlag> +>()("supabase/CommandFlags") {} + +function isMapParam(param: Param.Any): param is Param.Map<Param.ParamKind, unknown, unknown> { + return param._tag === "Map" && "param" in param && "f" in param && typeof param.f === "function"; +} + +/** + * The value `Flag.withDefault` gives an absent flag, read from the `Map` over `Optional` it builds. + * Any other shape, a throwing mapper or a non-string result yields no default. + */ +function defaultOf(param: Param.Any): string | undefined { + if (!isMapParam(param) || param.param._tag !== "Optional") return undefined; + try { + const value = param.f(Option.none()); + return typeof value === "string" ? value : undefined; + } catch { + return undefined; + } +} + +export function permissionFlagsOf( + params: ReadonlyArray<Param.AnyFlag>, +): ReadonlyArray<PermissionFlag> { + return params.flatMap((param) => { + const single = unwrapParam(param)?.single; + if (single === undefined) return []; + const choices = choiceKeysOf(single.primitiveType); + const kind = + single.primitiveType._tag === "Boolean" + ? "boolean" + : choices === undefined + ? "text" + : "choice"; + const fallback = kind === "boolean" ? undefined : defaultOf(param); + const known = fallback !== undefined && (choices === undefined || choices.includes(fallback)); + return [ + { + name: single.name, + kind, + hidden: single.hidden, + choices: choices ?? [], + optional: param._tag === "Optional", + ...(known ? { default: fallback } : {}), + }, + ]; + }); +} + +const globalFlags = permissionFlagsOf(ROOT_GLOBAL_FLAGS.map((entry) => entry.flag)); + +/** + * Every flag a help document's command accepts: the root's globals, the flags captured by + * `withPermissions` (which override a global of the same name), and any other visible flag the help + * document lists, such as one a parent group shares. + */ +export function flagCatalogFor(doc: HelpDoc.HelpDoc): FlagCatalog { + const catalog = new Map<string, PermissionFlag>(globalFlags.map((flag) => [flag.name, flag])); + const own = Option.match(Context.getOption(doc.annotations, CommandFlagsAnnotation), { + onNone: () => [], + onSome: (readFlags) => readFlags(), + }); + for (const flag of own) catalog.set(flag.name, flag); + for (const flag of [...doc.flags, ...(doc.globalFlags ?? [])]) { + if (catalog.has(flag.name)) continue; + catalog.set(flag.name, { + name: flag.name, + kind: flag.type === "boolean" ? "boolean" : "text", + hidden: false, + choices: [], + optional: !flag.required, + }); + } + return catalog; +} diff --git a/apps/cli/src/command-internal/command-permissions/permission-schemes.ts b/apps/cli/src/command-internal/command-permissions/permission-schemes.ts new file mode 100644 index 0000000000..f7e843f70e --- /dev/null +++ b/apps/cli/src/command-internal/command-permissions/permission-schemes.ts @@ -0,0 +1,106 @@ +import { operationPermissions } from "@supabase/api/permissions"; + +import { + fgaConditionalExtras, + fgaOperationNotes, + fgaResourceCases, + fgaSetForRequest, +} from "./fga-request-sets.ts"; +import type { OperationEntry } from "./model.ts"; + +type PermissionKind = OperationEntry["kind"]; + +/** The alternatives for one operation: a token needs every permission in at least one of them. */ +export type Alternatives = ReadonlyArray<ReadonlyArray<string>>; + +/** What a token needs when the resource a request addresses decides which permissions apply. */ +interface PermissionCase { + readonly for: string; + readonly allOf: ReadonlyArray<string>; +} + +/** What a token needs for an operation: any one of several sets, or a set chosen by resource. */ +export type Requirement = + | { readonly anyOf: Alternatives } + | { readonly cases: ReadonlyArray<PermissionCase> }; + +/** One part of an operation's requirement; `hedged` when it applies only to some requests. */ +interface RequirementPart { + readonly requirement: Requirement; + readonly hedged: boolean; +} + +/** One way of expressing what a token needs: FGA permissions, OAuth scopes, and so on. */ +export interface PermissionScheme { + readonly id: string; + readonly title: string; + /** The parts of an operation's requirement, or `undefined` when the scheme lists none. */ + readonly requirement: ( + operationId: string, + request: OperationEntry["request"], + ) => ReadonlyArray<RequirementPart> | undefined; + /** What to say about an operation for which {@link requirement} is `undefined`. */ + readonly missing: (kind: PermissionKind, operationId: string) => string; +} + +const NO_LISTING = "no token permission listed in the API spec"; + +const always = (requirement: Requirement): RequirementPart => ({ requirement, hedged: false }); + +const isSubset = (a: ReadonlyArray<string>, b: ReadonlyArray<string>) => + a.every((permission) => b.includes(permission)); + +/** Drops any set that another set already covers, since a token holding the smaller set suffices. */ +function withoutSupersets(sets: Alternatives): Alternatives { + return sets.filter( + (set, index) => + !sets.some( + (other, otherIndex) => + otherIndex !== index && + isSubset(other, set) && + (!isSubset(set, other) || otherIndex < index), + ), + ); +} + +export const fgaScheme: PermissionScheme = { + id: "fga", + title: "Scoped access token", + requirement: (operationId, request) => { + const forRequest = fgaSetForRequest(operationId, request); + if (forRequest !== undefined) return [always({ anyOf: [forRequest] })]; + const extra = fgaConditionalExtras.find((entry) => entry.operationId === operationId); + if (extra !== undefined) { + return [ + always({ anyOf: [extra.required] }), + { requirement: { anyOf: [extra.extra] }, hedged: true }, + ]; + } + const resource = fgaResourceCases.find((entry) => entry.operationId === operationId); + if (resource !== undefined) { + return [always({ cases: resource.cases })]; + } + const sets = operationPermissions[operationId]?.fga ?? []; + return sets.length === 0 ? undefined : [always({ anyOf: withoutSupersets(sets) })]; + }, + missing: (_kind, operationId) => fgaOperationNotes.get(operationId) ?? NO_LISTING, +}; + +/** Resolves OAuth scopes; register it in {@link permissionSchemes} to show them in help. */ +export const oauthScheme: PermissionScheme = { + id: "oauth", + title: "OAuth token (scopes)", + requirement: (operationId) => { + const scope = operationPermissions[operationId]?.oauthScope; + return scope === undefined ? undefined : [always({ anyOf: [[scope]] })]; + }, + missing: (kind, operationId) => { + if ((operationPermissions[operationId]?.fga.length ?? 0) === 0) return NO_LISTING; + return kind === "required" + ? "not available with OAuth tokens yet" + : "skipped with OAuth tokens (not available yet)"; + }, +}; + +/** Add a scheme here to show it in every command's help. */ +export const permissionSchemes: ReadonlyArray<PermissionScheme> = [fgaScheme]; diff --git a/apps/cli/src/command-internal/global-flags.ts b/apps/cli/src/command-internal/global-flags.ts index 7a2578c039..df9c33e193 100644 --- a/apps/cli/src/command-internal/global-flags.ts +++ b/apps/cli/src/command-internal/global-flags.ts @@ -134,6 +134,9 @@ export const GLOBAL_FLAGS = [ AgentFlag, ] as const; +/** The flags the root command registers: every global flag plus `--output-format`. */ +export const ROOT_GLOBAL_FLAGS = [OutputFormatFlag, ...GLOBAL_FLAGS] as const; + /** * Resolves the current value of every global/persistent flag above, keyed by its own CLI flag * name. Used by `telemetry/command-telemetry.ts` as a fallback for a global flag not present in a diff --git a/apps/cli/src/commands/branches/create/create.command.ts b/apps/cli/src/commands/branches/create/create.command.ts index 51d4cfa7bb..1615cb7c83 100644 --- a/apps/cli/src/commands/branches/create/create.command.ts +++ b/apps/cli/src/commands/branches/create/create.command.ts @@ -100,8 +100,16 @@ export const branchesCreateCommand = Command.make("create", config).pipe( withPermissions( compose( { - operations: [{ operationId: "v1-create-a-branch", kind: "required" }], - noApiEffectFlags: ["region", "size", "persistent", "with-data", "notify-url", "git-branch"], + operations: [ + { operationId: "v1-create-a-branch", kind: "required" }, + { + operationId: "v1-create-a-branch", + kind: "required", + when: [{ flag: "with-data" }], + request: { body: { with_data: "true" } }, + }, + ], + noApiEffectFlags: ["region", "size", "persistent", "notify-url", "git-branch"], }, projectRefPicker, upgradeSuggestion, diff --git a/apps/cli/src/shared/output/json-formatter.ts b/apps/cli/src/shared/output/json-formatter.ts index 18795fc7da..35a424eabe 100644 --- a/apps/cli/src/shared/output/json-formatter.ts +++ b/apps/cli/src/shared/output/json-formatter.ts @@ -1,10 +1,21 @@ import type { CliOutput, HelpDoc } from "effect/cli"; +import { + permissionSectionForHelp, + withoutPermissionAnnotations, +} from "../../command-internal/command-permissions/help-section.ts"; import type { CliErrorSuggestionContext } from "../cli/subcommand-flag-suggestions.ts"; import { cliErrorCode, formatCliErrorsForDisplay } from "../cli/subcommand-flag-suggestions.ts"; export function jsonCliOutputFormatter(context?: CliErrorSuggestionContext): CliOutput.Formatter { return { - formatHelpDoc: (doc: HelpDoc.HelpDoc) => JSON.stringify({ _tag: "Help", doc }), + formatHelpDoc: (doc: HelpDoc.HelpDoc) => { + const permissions = permissionSectionForHelp(doc); + return JSON.stringify({ + _tag: "Help", + doc: withoutPermissionAnnotations(doc), + ...(permissions === undefined ? {} : { permissions }), + }); + }, formatCliError: (error) => JSON.stringify({ _tag: "Error", diff --git a/apps/cli/src/shared/output/text-formatter.ts b/apps/cli/src/shared/output/text-formatter.ts index dc314545af..8ff6d4fca2 100644 --- a/apps/cli/src/shared/output/text-formatter.ts +++ b/apps/cli/src/shared/output/text-formatter.ts @@ -1,4 +1,8 @@ import { CliOutput } from "effect/cli"; +import { + permissionSectionForHelp, + renderPermissionsSection, +} from "../../command-internal/command-permissions/help-section.ts"; import type { CliErrorSuggestionContext, FormattedCliError, @@ -17,6 +21,11 @@ export function textCliOutputFormatter(context?: CliErrorSuggestionContext): Cli return { ...base, + formatHelpDoc: (doc) => { + const text = base.formatHelpDoc(doc); + const section = permissionSectionForHelp(doc); + return section === undefined ? text : `${text}\n\n${renderPermissionsSection(section)}`; + }, formatErrors: (errors) => { const formatted = formatCliErrorsForDisplay(errors, context); if (!formatted.changed) return base.formatErrors(errors); diff --git a/apps/cli/tests/helpers/permission-drift.ts b/apps/cli/tests/helpers/permission-drift.ts index 1faac0f4c7..21978b8f45 100644 --- a/apps/cli/tests/helpers/permission-drift.ts +++ b/apps/cli/tests/helpers/permission-drift.ts @@ -25,6 +25,8 @@ interface RecordedMethodUrl { readonly url: string; /** Query parameters Effect's HttpClient keeps apart from `url` (a recorded request's `urlParams`). */ readonly urlParams?: string; + /** The request body as the mock recorded it: parsed JSON when it parses. */ + readonly body?: unknown; } export interface DriftCheckOptions { @@ -83,38 +85,66 @@ function queryOf(request: RecordedMethodUrl, apiUrl: string | undefined): URLSea return params; } +const DETAIL_KINDS = ["query", "body"] as const; + +type DetailKind = (typeof DETAIL_KINDS)[number]; + +/** The string-encoded top-level body fields of a recorded JSON object body. */ +function bodyFieldsOf(body: unknown): ReadonlyMap<string, string> { + if (typeof body !== "object" || body === null || Array.isArray(body)) return new Map(); + return new Map( + Object.entries(body).map(([key, value]) => [ + key, + typeof value === "string" ? value : JSON.stringify(value), + ]), + ); +} + /** - * A request must carry the qualifiers of at least one applicable entry for its operation, and no - * qualifier value the declaration names for the operation without an applicable entry declaring it. + * A request must carry the qualifiers (query parameters and body fields) of at least one + * applicable entry for its operation, and no qualifier value the declaration names for the + * operation without an applicable entry declaring it. */ function assertQualifiersMatch( prefix: string, request: RecordedMethodUrl, - sent: URLSearchParams, + sentQuery: URLSearchParams, applicable: ReadonlyArray<OperationEntry>, declaredForOperation: ReadonlyArray<OperationEntry>, ): void { const target = `${request.method} ${new URL(request.url, "https://api.supabase.com").pathname}`; + const sentBody = bodyFieldsOf(request.body); + const sent = (kind: DetailKind, key: string): string | undefined => + kind === "query" ? (sentQuery.get(key) ?? undefined) : sentBody.get(key); + const listed = (entry: OperationEntry, kind: DetailKind) => + Object.entries(entry.request?.[kind] ?? {}); if ( !applicable.some((entry) => - Object.entries(entry.request?.query ?? {}).every(([key, value]) => sent.get(key) === value), + DETAIL_KINDS.every((kind) => + listed(entry, kind).every(([key, value]) => sent(kind, key) === value), + ), ) ) { throw new Error( - `${prefix}: request ${target} was sent without the query qualifiers its declaration lists (${applicable - .map((entry) => JSON.stringify(entry.request?.query)) + `${prefix}: request ${target} was sent without the request qualifiers its declaration lists (${applicable + .map((entry) => JSON.stringify(entry.request)) .join(" or ")}).`, ); } - const mentioned = new Set( - declaredForOperation.flatMap((entry) => Object.keys(entry.request?.query ?? {})), - ); - for (const key of mentioned) { - const value = sent.get(key); - if (value !== null && !applicable.some((entry) => entry.request?.query?.[key] === value)) { - throw new Error( - `${prefix}: request ${target} carries ${key}=${value}, which no applicable declaration entry lists for this operation.`, - ); + for (const kind of DETAIL_KINDS) { + const mentioned = new Set( + declaredForOperation.flatMap((entry) => listed(entry, kind).map(([key]) => key)), + ); + for (const key of mentioned) { + const value = sent(kind, key); + if ( + value !== undefined && + !applicable.some((entry) => entry.request?.[kind]?.[key] === value) + ) { + throw new Error( + `${prefix}: request ${target} carries ${kind === "body" ? "body " : ""}${key}=${value}, which no applicable declaration entry lists for this operation.`, + ); + } } } } diff --git a/apps/cli/tests/helpers/permission-drift.unit.test.ts b/apps/cli/tests/helpers/permission-drift.unit.test.ts index d8c4f0f750..2f0e50ddf1 100644 --- a/apps/cli/tests/helpers/permission-drift.unit.test.ts +++ b/apps/cli/tests/helpers/permission-drift.unit.test.ts @@ -221,7 +221,7 @@ describe("assertDriftAgainst", () => { }); const check = ( lookup: DeclaredPermissionsLookup, - request: { method: string; url: string; urlParams?: string }, + request: { method: string; url: string; urlParams?: string; body?: unknown }, activeFlags: Record<string, boolean> = {}, ) => assertDriftAgainst(lookup, { command: "keys", activeFlags, requests: [request] }); @@ -236,11 +236,61 @@ describe("assertDriftAgainst", () => { it("rejects a request sent without the declared qualifier or with another value", () => { expect(() => check(withReveal, { method: "GET", url: apiKeys })).toThrow( - /sent without the query qualifiers its declaration lists/, + /sent without the request qualifiers its declaration lists/, ); expect(() => check(withReveal, { method: "GET", url: apiKeys, urlParams: "reveal=false" }), - ).toThrow(/sent without the query qualifiers/); + ).toThrow(/sent without the request qualifiers/); + }); + + describe("in the body", () => { + const branches = `${API}/v1/projects/${REF}/branches`; + const withData = declared({ + status: "mapped", + operations: [ + { operationId: "v1-create-a-branch", kind: "required" }, + { + operationId: "v1-create-a-branch", + kind: "required", + when: [{ flag: "with-data" }], + request: { body: { with_data: "true" } }, + }, + ], + noApiEffectFlags: [], + }); + const create = (body: unknown, activeFlags: Record<string, boolean> = {}) => + check(withData, { method: "POST", url: branches, body }, activeFlags); + + it("accepts a body carrying the declared field under the flag that declares it", () => { + expect(() => create({ with_data: true }, { "with-data": true })).not.toThrow(); + expect(() => create({ branch_name: "x" })).not.toThrow(); + }); + + it("rejects a body field the run's flags do not declare", () => { + expect(() => create({ with_data: true })).toThrow( + /carries body with_data=true, which no applicable declaration entry lists/, + ); + }); + + it("rejects a body without the field a declaration requires", () => { + const required = declared({ + status: "mapped", + operations: [ + { + operationId: "v1-create-a-branch", + kind: "required", + request: { body: { with_data: "true" } }, + }, + ], + noApiEffectFlags: [], + }); + expect(() => + check(required, { method: "POST", url: branches, body: { with_data: false } }), + ).toThrow(/sent without the request qualifiers/); + expect(() => + check(required, { method: "POST", url: branches, body: { with_data: true } }), + ).not.toThrow(); + }); }); it("rejects a qualifier the declaration lists only under a flag the run does not use", () => { diff --git a/packages/api/package.json b/packages/api/package.json index 56d9501921..0963ffb643 100644 --- a/packages/api/package.json +++ b/packages/api/package.json @@ -9,6 +9,7 @@ "default": "./src/node.ts" }, "./effect": "./src/effect.ts", + "./permissions": "./src/generated/permissions.ts", "./openapi.json": "./src/generated/openapi.json" }, "scripts": { diff --git a/packages/api/scripts/generate.ts b/packages/api/scripts/generate.ts index 4e5f22f03f..db7c7b4d7c 100644 --- a/packages/api/scripts/generate.ts +++ b/packages/api/scripts/generate.ts @@ -29,6 +29,8 @@ export type OpenApiOperation = { readonly parameters?: ReadonlyArray<OpenApiParameter>; readonly requestBody?: OpenApiRequestBody; readonly responses?: Record<string, OpenApiResponse>; + readonly "x-fga-permissions"?: ReadonlyArray<ReadonlyArray<string>>; + readonly "x-oauth-scope"?: string; }; type OpenApiRequestBody = { @@ -1150,6 +1152,36 @@ export type GeneratedEffectOperations = typeof versionedEffectOperations; `; } +export function renderPermissions(document: OpenApiDocument): string { + const rows: Array<string> = []; + for (const pathItem of Object.values(document.paths)) { + for (const method of httpMethodOrder) { + const operation = pathItem[method]; + if (!operation?.operationId) continue; + const fga = operation["x-fga-permissions"] ?? []; + const oauthScope = operation["x-oauth-scope"]; + rows.push( + ` ${JSON.stringify(operation.operationId)}: { fga: ${JSON.stringify(fga)}${ + oauthScope === undefined ? "" : `, oauthScope: ${JSON.stringify(oauthScope)}` + } },`, + ); + } + } + return `/** Token permissions an operation declares in the API spec. */ +export interface OperationPermissions { + /** Alternative permission sets: a token needs every permission in at least one set. */ + readonly fga: ReadonlyArray<ReadonlyArray<string>>; + /** OAuth scope for the operation; absent when OAuth tokens cannot call it yet. */ + readonly oauthScope?: string; +} + +/** Permissions keyed by OpenAPI operation id. */ +export const operationPermissions: Readonly<Record<string, OperationPermissions>> = { +${rows.join("\n")} +}; +`; +} + function main() { const document = loadSpec(); const operations = extractOperations(document); @@ -1159,6 +1191,7 @@ function main() { writeFileSync(path.join(generatedDir, "contracts.ts"), renderContracts(document, operations)); writeFileSync(path.join(generatedDir, "effect-client.ts"), renderEffectClient(operations)); + writeFileSync(path.join(generatedDir, "permissions.ts"), renderPermissions(document)); writeFileSync(path.join(generatedDir, "openapi.json"), `${JSON.stringify(document, null, 2)}\n`); console.log(`Generated ${operations.length} API operations in ${generatedDir}`); diff --git a/packages/api/scripts/generated-output-sync.unit.test.ts b/packages/api/scripts/generated-output-sync.unit.test.ts index 355380b25c..c9f5f3e802 100644 --- a/packages/api/scripts/generated-output-sync.unit.test.ts +++ b/packages/api/scripts/generated-output-sync.unit.test.ts @@ -4,7 +4,13 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; import { describe, expect, test } from "vitest"; -import { extractOperations, loadSpec, renderContracts, renderEffectClient } from "./generate.ts"; +import { + extractOperations, + loadSpec, + renderContracts, + renderEffectClient, + renderPermissions, +} from "./generate.ts"; // Full-fidelity drift guard: re-renders every generated file from the committed // snapshot, formats it with the same oxfmt the pipeline uses, and requires byte @@ -91,6 +97,17 @@ describe("generated output sync", () => { }, ); + test( + "permissions.ts is byte-identical to the generator's render of the committed snapshot", + { timeout: RENDER_TIMEOUT_MS }, + () => { + expectSameSource( + formatWithOxfmt(renderPermissions(document), "permissions.ts"), + "permissions.ts", + ); + }, + ); + test( "openapi.json is byte-identical to the generator's normalized rewrite of itself", { timeout: RENDER_TIMEOUT_MS }, diff --git a/packages/api/src/entrypoints.unit.test.ts b/packages/api/src/entrypoints.unit.test.ts index a52d44664e..ca5dadfcac 100644 --- a/packages/api/src/entrypoints.unit.test.ts +++ b/packages/api/src/entrypoints.unit.test.ts @@ -46,6 +46,7 @@ describe("@supabase/api entrypoints", () => { default: "./src/node.ts", }); expect(packageJson.exports["./effect"]).toBe("./src/effect.ts"); + expect(packageJson.exports["./permissions"]).toBe("./src/generated/permissions.ts"); expect(packageJson.exports["./openapi.json"]).toBe("./src/generated/openapi.json"); expect(packageJson.exports["./bun"]).toBeUndefined(); expect(packageJson.exports["./node"]).toBeUndefined(); diff --git a/packages/api/src/generated/permissions.ts b/packages/api/src/generated/permissions.ts new file mode 100644 index 0000000000..42196b9d25 --- /dev/null +++ b/packages/api/src/generated/permissions.ts @@ -0,0 +1,423 @@ +/** Token permissions an operation declares in the API spec. */ +export interface OperationPermissions { + /** Alternative permission sets: a token needs every permission in at least one set. */ + readonly fga: ReadonlyArray<ReadonlyArray<string>>; + /** OAuth scope for the operation; absent when OAuth tokens cannot call it yet. */ + readonly oauthScope?: string; +} + +/** Permissions keyed by OpenAPI operation id. */ +export const operationPermissions: Readonly<Record<string, OperationPermissions>> = { + "v1-get-a-branch-config": { + fga: [["branching_development_read"], ["branching_production_read"]], + oauthScope: "environment:read", + }, + "v1-update-a-branch-config": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-delete-a-branch": { + fga: [["branching_development_delete"], ["branching_production_delete"]], + oauthScope: "environment:write", + }, + "v1-push-a-branch": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-merge-a-branch": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-reset-a-branch": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-restore-a-branch": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-diff-a-branch": { + fga: [["branching_development_write"], ["branching_production_write"]], + oauthScope: "environment:write", + }, + "v1-list-all-projects": { fga: [["projects_read"]], oauthScope: "projects:read" }, + "v1-create-a-project": { fga: [["organization_projects_create"]], oauthScope: "projects:write" }, + "v1-get-available-regions": { fga: [], oauthScope: "organizations:read" }, + "v1-list-all-organizations": { fga: [["organizations_read"]], oauthScope: "organizations:read" }, + "v1-create-an-organization": { fga: [["organizations_create"]] }, + "v1-authorize-user": { fga: [] }, + "v1-exchange-oauth-token": { fga: [] }, + "v1-revoke-token": { fga: [] }, + "v1-oauth-authorize-project-claim": { + fga: [["organization_admin_write", "project_admin_write"]], + }, + "v1-list-all-snippets": { fga: [["snippets_read"]], oauthScope: "database:read" }, + "v1-get-a-snippet": { fga: [["snippets_read"]], oauthScope: "database:read" }, + "v1-get-profile": { fga: [] }, + "v1-list-action-runs": { fga: [["action_runs_read"]], oauthScope: "environment:read" }, + "v1-count-action-runs": { fga: [["action_runs_read"]], oauthScope: "environment:read" }, + "v1-get-action-run": { fga: [["action_runs_read"]], oauthScope: "environment:read" }, + "v1-update-action-run-status": { fga: [["action_runs_write"]], oauthScope: "environment:write" }, + "v1-get-action-run-logs": { fga: [["action_runs_read"]], oauthScope: "environment:read" }, + "v1-get-project-api-keys": { + fga: [["api_gateway_keys_read"], ["api_gateway_keys_read", "api_gateway_keys_secret_read"]], + oauthScope: "secrets:read", + }, + "v1-create-project-api-key": { fga: [["api_gateway_keys_write"]], oauthScope: "secrets:write" }, + "v1-get-project-legacy-api-keys": { + fga: [["api_gateway_keys_read"]], + oauthScope: "secrets:read", + }, + "v1-update-project-legacy-api-keys": { + fga: [["api_gateway_keys_write"]], + oauthScope: "secrets:write", + }, + "v1-get-project-api-key": { + fga: [["api_gateway_keys_read"], ["api_gateway_keys_read", "api_gateway_keys_secret_read"]], + oauthScope: "secrets:read", + }, + "v1-update-project-api-key": { fga: [["api_gateway_keys_write"]], oauthScope: "secrets:write" }, + "v1-delete-project-api-key": { fga: [["api_gateway_keys_write"]], oauthScope: "secrets:write" }, + "v1-list-all-branches": { + fga: [["branching_development_read"], ["branching_production_read"]], + oauthScope: "environment:read", + }, + "v1-create-a-branch": { + fga: [["branching_development_create"], ["branching_production_create"]], + oauthScope: "environment:write", + }, + "v1-disable-preview-branching": { + fga: [["branching_production_delete"]], + oauthScope: "environment:write", + }, + "v1-get-a-branch": { + fga: [["branching_development_read"], ["branching_production_read"]], + oauthScope: "environment:read", + }, + "v1-get-hostname-config": { fga: [["custom_domain_read"]], oauthScope: "domains:read" }, + "v1-Delete hostname config": { fga: [["custom_domain_write"]], oauthScope: "domains:write" }, + "v1-update-hostname-config": { fga: [["custom_domain_write"]], oauthScope: "domains:write" }, + "v1-verify-dns-config": { fga: [["custom_domain_write"]], oauthScope: "domains:write" }, + "v1-activate-custom-hostname": { fga: [["custom_domain_write"]], oauthScope: "domains:write" }, + "v1-get-jit-access-config": { fga: [["project_admin_read"]], oauthScope: "database:read" }, + "v1-update-jit-access-config": { fga: [["project_admin_write"]], oauthScope: "database:write" }, + "v1-list-all-network-bans": { + fga: [["database_network_bans_read"]], + oauthScope: "projects:read", + }, + "v1-list-all-network-bans-enriched": { + fga: [["database_network_bans_read"]], + oauthScope: "projects:read", + }, + "v1-delete-network-bans": { + fga: [["database_network_bans_write"]], + oauthScope: "projects:write", + }, + "v1-get-network-restrictions": { + fga: [["database_network_restrictions_read"]], + oauthScope: "projects:read", + }, + "v1-patch-network-restrictions": { + fga: [["database_network_restrictions_write"]], + oauthScope: "projects:write", + }, + "v1-update-network-restrictions": { + fga: [["database_network_restrictions_write"]], + oauthScope: "projects:write", + }, + "v1-get-pgsodium-config": { fga: [["project_admin_write"]], oauthScope: "secrets:read" }, + "v1-update-pgsodium-config": { fga: [["project_admin_write"]], oauthScope: "secrets:write" }, + "v1-get-postgrest-service-config": { + fga: [["data_api_config_read"], ["data_api_config_read", "data_api_config_secret_read"]], + oauthScope: "rest:read", + }, + "v1-update-postgrest-service-config": { + fga: [["data_api_config_write"]], + oauthScope: "rest:write", + }, + "v1-get-project": { fga: [["project_admin_read"]], oauthScope: "projects:read" }, + "v1-update-a-project": { fga: [["project_admin_write"]], oauthScope: "projects:write" }, + "v1-delete-a-project": { fga: [["project_admin_write"]], oauthScope: "projects:write" }, + "v1-list-all-secrets": { fga: [["edge_functions_secrets_read"]], oauthScope: "secrets:read" }, + "v1-bulk-create-secrets": { + fga: [["edge_functions_secrets_write"]], + oauthScope: "secrets:write", + }, + "v1-bulk-delete-secrets": { + fga: [["edge_functions_secrets_write"]], + oauthScope: "secrets:write", + }, + "v1-get-ssl-enforcement-config": { + fga: [["database_ssl_config_read"]], + oauthScope: "database:read", + }, + "v1-update-ssl-enforcement-config": { + fga: [["database_ssl_config_write"]], + oauthScope: "database:write", + }, + "v1-generate-typescript-types": { fga: [["database_read"]], oauthScope: "database:read" }, + "v1-get-vanity-subdomain-config": { + fga: [["vanity_subdomain_read"]], + oauthScope: "domains:read", + }, + "v1-deactivate-vanity-subdomain-config": { + fga: [["vanity_subdomain_write"]], + oauthScope: "domains:write", + }, + "v1-check-vanity-subdomain-availability": { + fga: [["vanity_subdomain_write"]], + oauthScope: "domains:write", + }, + "v1-activate-vanity-subdomain-config": { + fga: [["vanity_subdomain_write"]], + oauthScope: "domains:write", + }, + "v1-upgrade-postgres-version": { + fga: [["project_admin_write", "database_write"]], + oauthScope: "projects:write", + }, + "v1-get-postgres-upgrade-eligibility": { + fga: [["project_admin_read", "database_read"]], + oauthScope: "projects:read", + }, + "v1-get-postgres-upgrade-status": { + fga: [["project_admin_read", "database_read"]], + oauthScope: "projects:read", + }, + "v1-get-readonly-mode-status": { + fga: [["database_readonly_config_read"]], + oauthScope: "database:read", + }, + "v1-disable-readonly-mode-temporarily": { + fga: [["database_readonly_config_write"]], + oauthScope: "database:write", + }, + "v1-setup-a-read-replica": { fga: [["infra_read_replicas_write"]] }, + "v1-remove-a-read-replica": { fga: [["infra_read_replicas_write"]] }, + "v1-get-services-health": { fga: [["project_admin_read"]], oauthScope: "projects:read" }, + "v1-get-legacy-signing-key": { fga: [["auth_signing_keys_read"]], oauthScope: "secrets:read" }, + "v1-create-legacy-signing-key": { + fga: [["auth_signing_keys_write"]], + oauthScope: "secrets:write", + }, + "v1-get-project-signing-keys": { fga: [["auth_signing_keys_read"]], oauthScope: "secrets:read" }, + "v1-create-project-signing-key": { + fga: [["auth_signing_keys_write"]], + oauthScope: "secrets:write", + }, + "v1-get-project-signing-key": { fga: [["auth_signing_keys_read"]] }, + "v1-update-project-signing-key": { + fga: [["auth_signing_keys_write"]], + oauthScope: "secrets:write", + }, + "v1-remove-project-signing-key": { + fga: [["auth_signing_keys_write"]], + oauthScope: "secrets:write", + }, + "v1-get-auth-service-config": { fga: [["auth_config_read"]], oauthScope: "auth:read" }, + "v1-update-auth-service-config": { + fga: [["auth_config_write", "project_admin_write"]], + oauthScope: "auth:write", + }, + "v1-list-project-tpa-integrations": { fga: [["auth_config_read"]], oauthScope: "auth:read" }, + "v1-create-project-tpa-integration": { fga: [["auth_config_write"]], oauthScope: "auth:write" }, + "v1-get-project-tpa-integration": { fga: [["auth_config_read"]], oauthScope: "auth:read" }, + "v1-delete-project-tpa-integration": { fga: [["auth_config_write"]], oauthScope: "auth:write" }, + "v1-pause-a-project": { fga: [["project_admin_write"]], oauthScope: "projects:write" }, + "v1-restart-a-project": { fga: [["project_admin_write"]], oauthScope: "projects:write" }, + "v1-list-available-restore-versions": { + fga: [["project_admin_read"]], + oauthScope: "projects:read", + }, + "v1-restore-a-project": { fga: [["project_admin_write"]], oauthScope: "projects:write" }, + "v1-cancel-a-project-restoration": { + fga: [["project_admin_write"]], + oauthScope: "projects:write", + }, + "v1-list-project-addons": { fga: [["infra_add_ons_read"]] }, + "v1-apply-project-addon": { fga: [["infra_add_ons_write"]] }, + "v1-remove-project-addon": { fga: [["infra_add_ons_write"]] }, + "v1-get-project-claim-token": { fga: [["project_admin_read"]] }, + "v1-create-project-claim-token": { fga: [["organization_admin_write", "project_admin_write"]] }, + "v1-delete-project-claim-token": { fga: [["organization_admin_write", "project_admin_write"]] }, + "v1-get-performance-advisors": { fga: [["advisors_read"]], oauthScope: "database:read" }, + "v1-get-security-advisors": { fga: [["advisors_read"]], oauthScope: "database:read" }, + "v1-get-project-logs-all": { fga: [["analytics_logs_read"]], oauthScope: "analytics:read" }, + "v1-get-project-logs": { fga: [["analytics_logs_read"]], oauthScope: "analytics:read" }, + "v1-get-project-usage-api-count": { fga: [["analytics_usage_read"]] }, + "v1-get-project-usage-request-count": { fga: [["analytics_usage_read"]] }, + "v1-get-project-function-combined-stats": { fga: [["analytics_usage_read"]] }, + "v1-scrape-project-metrics": { fga: [["analytics_logs_read"]], oauthScope: "analytics:read" }, + "v1-create-login-role": { fga: [["database_write"]], oauthScope: "database:write" }, + "v1-delete-login-roles": { fga: [["database_write"]], oauthScope: "database:write" }, + "v1-list-migration-history": { fga: [["database_migrations_read"]], oauthScope: "database:read" }, + "v1-apply-a-migration": { fga: [["database_migrations_write"]], oauthScope: "database:write" }, + "v1-upsert-a-migration": { fga: [["database_migrations_write"]], oauthScope: "database:write" }, + "v1-rollback-migrations": { fga: [["database_migrations_write"]], oauthScope: "database:write" }, + "v1-get-a-migration": { fga: [["database_migrations_read"]], oauthScope: "database:read" }, + "v1-patch-a-migration": { fga: [["database_migrations_write"]], oauthScope: "database:write" }, + "v1-run-a-query": { fga: [["database_read"], ["database_write"]], oauthScope: "database:write" }, + "v1-read-only-query": { fga: [["database_read"]], oauthScope: "database:read" }, + "v1-enable-database-webhook": { + fga: [["database_webhooks_config_write"]], + oauthScope: "database:write", + }, + "v1-get-database-metadata": { fga: [["database_read"]], oauthScope: "projects:read" }, + "v1-update-database-password": { fga: [["database_config_write"]], oauthScope: "database:write" }, + "v1-get-jit-access": { fga: [["database_jit_read"]], oauthScope: "database:read" }, + "v1-authorize-jit-access": { fga: [["database_jit_read"]], oauthScope: "database:read" }, + "v1-update-jit-access": { fga: [["database_jit_write"]] }, + "v1-list-jit-access": { fga: [["database_jit_write"]] }, + "v1-invite-external-jit-access": { fga: [["database_jit_write"]] }, + "v1-accept-invite-external-jit-access": { fga: [] }, + "v1-delete-invite-external-jit-access": { fga: [["database_jit_write"]] }, + "v1-delete-jit-access": { fga: [["database_jit_write"]] }, + "v1-get-database-openapi": { fga: [["database_read"]], oauthScope: "database:read" }, + "v1-list-all-functions": { fga: [["edge_functions_read"]], oauthScope: "edge_functions:read" }, + "v1-create-a-function": { fga: [["edge_functions_write"]], oauthScope: "edge_functions:write" }, + "v1-bulk-update-functions": { + fga: [["edge_functions_write"]], + oauthScope: "edge_functions:write", + }, + "v1-deploy-a-function": { fga: [["edge_functions_write"]], oauthScope: "edge_functions:write" }, + "v1-get-a-function": { fga: [["edge_functions_read"]], oauthScope: "edge_functions:read" }, + "v1-update-a-function": { fga: [["edge_functions_write"]], oauthScope: "edge_functions:write" }, + "v1-delete-a-function": { fga: [["edge_functions_write"]], oauthScope: "edge_functions:write" }, + "v1-get-a-function-body": { fga: [["edge_functions_read"]], oauthScope: "edge_functions:read" }, + "v1-list-all-buckets": { fga: [["storage_read"]], oauthScope: "storage:read" }, + "v1-get-database-disk": { fga: [["infra_disk_config_read"]] }, + "v1-modify-database-disk": { fga: [["infra_disk_config_write"]] }, + "v1-get-disk-utilization": { fga: [["infra_disk_config_read"]] }, + "v1-get-project-disk-autoscale-config": { fga: [["infra_disk_config_read"]] }, + "v1-get-storage-config": { fga: [["storage_config_read"]] }, + "v1-update-storage-config": { fga: [["storage_config_write"]] }, + "v1-get-project-pgbouncer-config": { fga: [["database_read"]], oauthScope: "database:read" }, + "v1-get-pooler-config": { fga: [["database_pooling_config_read"]], oauthScope: "database:read" }, + "v1-update-pooler-config": { + fga: [["database_pooling_config_write"]], + oauthScope: "database:write", + }, + "v1-get-postgres-config": { fga: [["database_config_read"]], oauthScope: "database:read" }, + "v1-update-postgres-config": { fga: [["database_config_write"]], oauthScope: "database:write" }, + "v1-get-realtime-config": { fga: [["realtime_config_read"]] }, + "v1-update-realtime-config": { fga: [["realtime_config_write"]] }, + "v1-shutdown-realtime": { fga: [["realtime_config_write"]] }, + "v1-list-all-sso-provider": { fga: [["auth_config_read"]], oauthScope: "auth:read" }, + "v1-create-a-sso-provider": { fga: [["auth_config_write"]], oauthScope: "auth:write" }, + "v1-get-a-sso-provider": { fga: [["auth_config_read"]], oauthScope: "auth:read" }, + "v1-update-a-sso-provider": { fga: [["auth_config_write"]], oauthScope: "auth:write" }, + "v1-delete-a-sso-provider": { fga: [["auth_config_write"]], oauthScope: "auth:write" }, + "v1-list-all-backups": { fga: [["backups_read"]], oauthScope: "database:read" }, + "v1-restore-pitr-backup": { fga: [["backups_write"]], oauthScope: "database:write" }, + "v1-get-restore-point": { fga: [["backups_read"]], oauthScope: "database:read" }, + "v1-create-restore-point": { fga: [["backups_write"]], oauthScope: "database:write" }, + "v1-restore-physical-backup": { fga: [["backups_write"]], oauthScope: "database:write" }, + "v1-get-backup-schedule": { fga: [["backups_read"]], oauthScope: "database:read" }, + "v1-update-backup-schedule": { fga: [["backups_write"]], oauthScope: "database:write" }, + "v1-undo": { fga: [["backups_write"]], oauthScope: "database:write" }, + "v1-get-organization-entitlements": { + fga: [["organization_admin_read"]], + oauthScope: "organizations:read", + }, + "v1-list-organization-members": { fga: [["members_read"]], oauthScope: "organizations:read" }, + "v1-get-an-organization": { + fga: [["organization_admin_read"]], + oauthScope: "organizations:read", + }, + "v1-get-organization-project-claim": { fga: [["organization_admin_write"]] }, + "v1-claim-project-for-organization": { fga: [["organization_admin_write"]] }, + "v1-get-all-projects-for-organization": { + fga: [["organization_projects_read"]], + oauthScope: "projects:read", + }, + "v2-list-log-drains": { fga: [["analytics_config_read"]], oauthScope: "analytics_config:read" }, + "v2-create-log-drain": { + fga: [["analytics_config_write"]], + oauthScope: "analytics_config:write", + }, + "v2-update-log-drain": { + fga: [["analytics_config_write"]], + oauthScope: "analytics_config:write", + }, + "v2-delete-log-drain": { + fga: [["analytics_config_write"]], + oauthScope: "analytics_config:write", + }, + "v2-run-project-advisors": { fga: [["advisors_read"]] }, + "v2-create-a-branch": { + fga: [["branching_development_create"], ["branching_production_create"]], + oauthScope: "environment:write", + }, + "v2-get-project-config": { + fga: [ + [ + "database_config_read", + "database_read", + "database_ssl_config_read", + "database_network_restrictions_read", + "auth_config_read", + "data_api_config_read", + "realtime_config_read", + "storage_config_read", + ], + ], + }, + "v2-list-notebooks": { fga: [["project_notebooks_read"]], oauthScope: "database:read" }, + "v2-create-notebook": { fga: [["project_notebooks_write"]], oauthScope: "database:write" }, + "v2-get-notebook": { fga: [["project_notebooks_read"]], oauthScope: "database:read" }, + "v2-update-notebook": { fga: [["project_notebooks_write"]], oauthScope: "database:write" }, + "v2-delete-notebook": { fga: [["project_notebooks_write"]], oauthScope: "database:write" }, + "v2-preview-a-project-transfer": { fga: [["project_admin_read"]] }, + "v2-transfer-a-project": { fga: [["organization_admin_write"]] }, + "v2-list-private-link-associations": { fga: [["project_admin_read"]] }, + "v2-create-private-link-association": { fga: [["project_admin_write"]] }, + "v2-delete-private-link-association": { fga: [["project_admin_write"]] }, + "v2-delete-private-link-association-for-database": { fga: [["project_admin_write"]] }, + "v2-list-all-compute-instances": { fga: [["workers_read"]], oauthScope: "edge_functions:read" }, + "v2-get-a-compute-instance": { fga: [["workers_read"]], oauthScope: "edge_functions:read" }, + "v2-delete-a-compute-instance": { fga: [["workers_write"]], oauthScope: "edge_functions:write" }, + "v2-create-compute-instance-upload": { + fga: [["workers_write"]], + oauthScope: "edge_functions:write", + }, + "v2-deploy-a-compute-instance": { fga: [["workers_write"]], oauthScope: "edge_functions:write" }, + "v2-list-organization-members": { fga: [["members_read"]], oauthScope: "organizations:read" }, + "v2-assign-organization-member-role": { fga: [["organization_admin_write"]] }, + "v2-list-organization-roles": { fga: [["members_read"]], oauthScope: "organizations:read" }, + "v2-create-organization-invitations": { + fga: [["members_write"]], + oauthScope: "organizations:write", + }, + "v2-delete-organization-invitations": { + fga: [["members_write"]], + oauthScope: "organizations:write", + }, + "v2-list-organization-projects": { + fga: [["organization_projects_read"]], + oauthScope: "projects:read", + }, + "v2-list-organization-github-connections": { + fga: [["organization_projects_read"]], + oauthScope: "projects:read", + }, + "v2-projects-ref-webhooks-endpoints-get": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-post": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-delete": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-id-get": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-id-patch": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-id-delete": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-id-deliveries-get": { fga: [] }, + "v2-projects-ref-webhooks-endpoints-id-test-post": { fga: [] }, + "v2-projects-ref-webhooks-deliveries-id-get": { fga: [] }, + "v2-projects-ref-webhooks-deliveries-id-retry-post": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-get": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-post": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-delete": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-id-get": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-id-patch": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-id-delete": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-id-deliveries-get": { fga: [] }, + "v2-organizations-slug-webhooks-endpoints-id-test-post": { fga: [] }, + "v2-organizations-slug-webhooks-deliveries-id-get": { fga: [] }, + "v2-organizations-slug-webhooks-deliveries-id-retry-post": { fga: [] }, +}; From cbf85a913b9d889df7dc81a7922fd94cf4c52e8b Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 09:28:59 +0000 Subject: [PATCH 13/24] chore(stack): bump realtime to v2.147.1-r0 (#7278) This bumps realtime from v2.145.5-r0 to v2.147.1-r0. https://github.com/supabase/slim-services/releases/tag/realtime-v2.147.1-r0 Planned after supabase/slim-services published v2.147.0-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli-go/pkg/config/templates/Dockerfile | 2 +- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/Dockerfile b/apps/cli-go/pkg/config/templates/Dockerfile index 19cf46c71d..7420a83022 100644 --- a/apps/cli-go/pkg/config/templates/Dockerfile +++ b/apps/cli-go/pkg/config/templates/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.145.5 AS realtime +FROM supabase/realtime:v2.147.1 AS realtime FROM supabase/storage-api:v1.84.0 AS storage diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 19cf46c71d..7420a83022 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -33,7 +33,7 @@ FROM supabase/supavisor:2.9.13 AS supavisor FROM supabase/gotrue:v2.197.0 AS gotrue -FROM supabase/realtime:v2.145.5 AS realtime +FROM supabase/realtime:v2.147.1 AS realtime FROM supabase/storage-api:v1.84.0 AS storage diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 9f349d8360..9bd9f61750 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -296,23 +296,23 @@ const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { realtime: definition( "realtime", { - upstreamVersion: "v2.145.5", + upstreamVersion: "v2.147.1", revision: 0, image: - "ghcr.io/supabase/cli/realtime:v2.145.5-r0@sha256:d1b3dda7f8a9a4045b830d083ec8ac8242569a43c78577af9da36bffa5e65968", - upstreamImage: "supabase/realtime:v2.145.5", + "ghcr.io/supabase/cli/realtime:v2.147.1-r0@sha256:49a10b8b4e6e5ee4d94e31477ad97f462ef220a842e5859b76114b15e41b53e1", + upstreamImage: "supabase/realtime:v2.147.1", natives: { "darwin-arm64": { - archive: "c6fd679b107bb436a0f2794ddfe94ea46aaa52de0bd5523d7f4c553e688aca48", - manifest: "36f6727174ec8e522508d6b011b265c9fbbfb710fa6e48270bf221adc34561cd", + archive: "fd05e59cf29c6e8e0523f60ca0f3e823d045178fa7ebdcba934d116c0889c15a", + manifest: "f85b0ae9b9a10671f892298e68912ed0fc2e6828427e7d039d0651edd5600339", }, "linux-amd64": { - archive: "c5dcbd8837daa539838bab8adcf4a8e3b08ad60b6d2a677ac2020de7de81d710", - manifest: "899a989b00499186527994e468511d359230ce8860635f7fe055294e78503883", + archive: "4f50539770db276c890dd582fef5239a3800a4d074311ec739227738e5f3f576", + manifest: "43651c3933e26580605390cb000fdc36603213e10b2a5fbbeaf519178e6eea9c", }, "linux-arm64": { - archive: "35be9b17d46bbfe50502f41ba1f68d561d073f634912347ff7e3b5f2f6d73a70", - manifest: "7b18c917fa46fbdb93468078040ca173b7a66420b86121a0f7f20cd278a3394b", + archive: "f04da0eaec820690f24a37da0d2b0787e41ef8c6f0d28935639cf6c87d2d0162", + manifest: "705d7c7321537acbb6cbb1b2b3efef5f19dc9a167c31d960082e0323ab200120", }, }, }, From d37b30a5569f7585c6bd7c62f19e03f1842cde40 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 09:38:21 +0000 Subject: [PATCH 14/24] chore(stack): bump storage to v1.84.17-r0 (#7277) This bumps storage from v1.84.0-r0 to v1.84.17-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.84.17-r0 Planned after supabase/slim-services published v1.84.17-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli-go/pkg/config/templates/Dockerfile | 2 +- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/Dockerfile b/apps/cli-go/pkg/config/templates/Dockerfile index 7420a83022..40f21a699f 100644 --- a/apps/cli-go/pkg/config/templates/Dockerfile +++ b/apps/cli-go/pkg/config/templates/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.147.1 AS realtime -FROM supabase/storage-api:v1.84.0 AS storage +FROM supabase/storage-api:v1.84.17 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 7420a83022..40f21a699f 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.147.1 AS realtime -FROM supabase/storage-api:v1.84.0 AS storage +FROM supabase/storage-api:v1.84.17 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 9bd9f61750..aa22bdcd2a 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -322,23 +322,23 @@ const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { storage: definition( "storage", { - upstreamVersion: "v1.84.0", + upstreamVersion: "v1.84.17", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.84.0-r0@sha256:837c299c73d5fa0ada05cb4d9648186e1bb1d500d5892db7d12601bde5674f7b", - upstreamImage: "supabase/storage-api:v1.84.0", + "ghcr.io/supabase/cli/storage:v1.84.17-r0@sha256:b1ad3c089bcb7d28437d4b2bc9d9053bed9234d2b2ad175fbc302dc82fd1f316", + upstreamImage: "supabase/storage-api:v1.84.17", natives: { "darwin-arm64": { - archive: "f3024fa9b6fc34407acbad41b3b99be5e527869a29e575870bc0a3e92a2bc49f", - manifest: "e088b3add2c659771ceeb73288720e632adbd34571b0424e93f217ed4a70f4ab", + archive: "4059c5433aab210a60cbb2548b3b1dff1e24587e5c040427986dff78e0f0e2a9", + manifest: "d90ef9c66cd48f46d49b2d485a74d205de430efb922c730eb303b08127581b28", }, "linux-amd64": { - archive: "685dcdb7ada3c4551de9c517844b51ae6c94dbc6112b14d9cb1ca57c0f110327", - manifest: "5db6aa659348a72757ba808c3f0829f7c3b0aa31405c327f2bfb45e7bd5dd1a4", + archive: "2f95354e7a45041acfd75b25f9d19fb9b5006827464284909af67624f2cf6cbb", + manifest: "b8b86c57e228ec90c163d1bea92a337c35f87ff856b8e483ccc3967d0edb5985", }, "linux-arm64": { - archive: "82e8c364480c28310990a3573ea8b487e46282d7895decbf6a4ef6ef345aecda", - manifest: "a1ae9e8846df92c49c7cb4570c9d0b0e1641c685fcfeffbac77c185de982d7fb", + archive: "639c0b897610319ad2518e1671c574c792c9fe7e5a20ab2ce6fd0b2b18990c53", + manifest: "115450d2a0473d6491aa15c1677086c9947c913edd8a022faf17d1d2f63c7a8e", }, }, }, From cce8808a0f03f374ee26e1a792a7362607d3168c Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 10:39:46 +0000 Subject: [PATCH 15/24] chore: sync API types from infrastructure (#7273) This PR was automatically created to sync API types from the infrastructure repository. Changes were detected after refreshing `apps/cli-go/api/v1-openapi.yaml` from the latest spec from infrastructure, and `apps/cli-go/pkg/api` is regenerated from it. The snapshot diff is the upstream API change and is worth reading: it is marked `linguist-generated`, so GitHub collapses it behind a "Load diff" click rather than hiding it. Co-authored-by: supabase-cli-releaser[bot] <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> --- apps/cli-go/api/v1-openapi.yaml | 2 ++ apps/cli-go/pkg/api/types.gen.go | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/apps/cli-go/api/v1-openapi.yaml b/apps/cli-go/api/v1-openapi.yaml index 210608ae7c..f0b48a3fad 100644 --- a/apps/cli-go/api/v1-openapi.yaml +++ b/apps/cli-go/api/v1-openapi.yaml @@ -13688,6 +13688,8 @@ components: - log_auth_error_rate_high - log_storage_error_rate_high - log_edge_function_error_rate_high + - realtime_authorization_timeouts + - realtime_postgres_changes_spike - project_not_active - advisor_check_unavailable title: diff --git a/apps/cli-go/pkg/api/types.gen.go b/apps/cli-go/pkg/api/types.gen.go index 4ab675df0a..f221e809fb 100644 --- a/apps/cli-go/pkg/api/types.gen.go +++ b/apps/cli-go/pkg/api/types.gen.go @@ -4698,6 +4698,8 @@ const ( PitrNotEnabled V1ProjectAdvisorsResponseOutputLintsName = "pitr_not_enabled" PolicyExistsRlsDisabled V1ProjectAdvisorsResponseOutputLintsName = "policy_exists_rls_disabled" ProjectNotActive V1ProjectAdvisorsResponseOutputLintsName = "project_not_active" + RealtimeAuthorizationTimeouts V1ProjectAdvisorsResponseOutputLintsName = "realtime_authorization_timeouts" + RealtimePostgresChangesSpike V1ProjectAdvisorsResponseOutputLintsName = "realtime_postgres_changes_spike" RlsDisabledInPublic V1ProjectAdvisorsResponseOutputLintsName = "rls_disabled_in_public" RlsEnabledNoPolicy V1ProjectAdvisorsResponseOutputLintsName = "rls_enabled_no_policy" RlsReferencesUserMetadata V1ProjectAdvisorsResponseOutputLintsName = "rls_references_user_metadata" @@ -4778,6 +4780,10 @@ func (e V1ProjectAdvisorsResponseOutputLintsName) Valid() bool { return true case ProjectNotActive: return true + case RealtimeAuthorizationTimeouts: + return true + case RealtimePostgresChangesSpike: + return true case RlsDisabledInPublic: return true case RlsEnabledNoPolicy: From 4ddc57dbb8192d7dd1d05120f3903a652e14c9b4 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Thu, 22 Oct 2026 13:44:27 +0000 Subject: [PATCH 16/24] fix(cli): stop `start` and `stop` failing on podman (CLI-2696) (#7270) ## TL;DR fixes `supabase start` and `supabase stop` failing every few runs on podman machine ## whats hurting the user? on macOS with podman machine and no `docker` on PATH, `supabase start` fails every few runs with `failed to inspect docker image` and `ssh: handshake failed: EOF` `supabase stop` fails the same way with `failed to stop container: exit 125` and leaves the stack behind both fire one `podman` call per image or container all at once, and each call opens its own ssh session into the VM, whose sshd drops new ones once ten are still in the handshake ## now fixed by? capping those calls at 8 in flight so every handshake gets accepted, rather than retrying, which would only hide the drops and add delay > [!NOTE] > docker shares the same cap, so stopping more than 8 containers that ignore their stop signal, so it can take an extra smol grace period... <details><summary>visual rep:</summary> ```mermaid flowchart LR subgraph before["before"] A["start / stop"] --> B["one podman call per image or container, all at once"] B --> C["sshd drops past ten handshakes"] C --> D["start / stop fail"] end subgraph now["now"] E["start / stop"] --> F["at most 8 podman calls in flight"] F --> G["all handshakes accepted"] end ``` </details> ## ref: - broken/missed in: https://github.com/supabase/cli/pull/5847 - spotted while working on: https://github.com/supabase/cli/pull/7089 --- .../cli/src/command-internal/container-cli.ts | 4 ++ .../db-bootstrap/image-prepull.ts | 6 ++- .../db-bootstrap/image-prepull.unit.test.ts | 32 +++++++++------ .../src/command-internal/docker-remove-all.ts | 3 +- apps/cli/src/commands/stop/SIDE_EFFECTS.md | 4 +- .../commands/stop/stop.integration.test.ts | 41 +++++++++++++++++++ 6 files changed, 73 insertions(+), 17 deletions(-) diff --git a/apps/cli/src/command-internal/container-cli.ts b/apps/cli/src/command-internal/container-cli.ts index 05687938f6..c6f165ea50 100644 --- a/apps/cli/src/command-internal/container-cli.ts +++ b/apps/cli/src/command-internal/container-cli.ts @@ -21,6 +21,10 @@ import { type Spawner = ChildProcessSpawner["Service"]; +/* Max container CLI calls a fan-out keeps in flight: each Podman-machine call opens its own ssh + connection, and the VM's sshd starts dropping them past ten mid-handshake; 8 leaves headroom. */ +export const CONTAINER_CLI_CONCURRENCY = 8; + /** * Raised when neither `docker` nor `podman` can be spawned at all, as opposed to a spawned * process exiting non-zero. Callers fold this into their own tagged error via diff --git a/apps/cli/src/command-internal/db-bootstrap/image-prepull.ts b/apps/cli/src/command-internal/db-bootstrap/image-prepull.ts index 6970f4ef54..81b05dc57d 100644 --- a/apps/cli/src/command-internal/db-bootstrap/image-prepull.ts +++ b/apps/cli/src/command-internal/db-bootstrap/image-prepull.ts @@ -12,6 +12,7 @@ import { type CliErrorActionabilityDeclaration, ErrorActionabilityId, } from "../../shared/telemetry/error-actionability.ts"; +import { CONTAINER_CLI_CONCURRENCY } from "../container-cli.ts"; import { makeDockerImageResolver } from "../docker-image-resolve.ts"; import { SUGGEST_DOCKER_INSTALL, isDockerDaemonUnreachable } from "../docker-suggest.ts"; @@ -40,7 +41,8 @@ export class ImagePrepullError extends Data.TaggedError("ImagePrepullError")<{ } /** - * Resolves every image in `images` concurrently through the shared registry-fallback resolver. + * Resolves every image in `images` concurrently, at most {@link CONTAINER_CLI_CONCURRENCY} at a + * time, through the shared registry-fallback resolver. * * Returns a map from each original image reference to the resolved image URL callers must use * to reference that image afterward — re-resolving the same image could land on a different @@ -62,7 +64,7 @@ export function ensureImagesCached( Effect.withSpan("ImagePrepull.resolveImage", { attributes: { "image.name": image } }), ), ), - { concurrency: "unbounded" }, + { concurrency: CONTAINER_CLI_CONCURRENCY }, ); const resolved = new Map<string, string>(); diff --git a/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts b/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts index 638702f1cd..2e4835d52e 100644 --- a/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts +++ b/apps/cli/src/command-internal/db-bootstrap/image-prepull.unit.test.ts @@ -3,6 +3,7 @@ import { Deferred, Effect, Fiber, Ref, Sink, Stream } from "effect"; import * as TestClock from "effect/testing/TestClock"; import { ChildProcessSpawner } from "effect/process"; +import { CONTAINER_CLI_CONCURRENCY } from "../container-cli.ts"; import { ImagePrepullError, ensureImagesCached } from "./image-prepull.ts"; /** Per-call variant of `docker-lifecycle.unit.test.ts`'s `mockSpawner`, so each argv can respond differently. */ @@ -105,23 +106,23 @@ describe("ensureImagesCached", () => { ); }); - it.live("resolves every image concurrently rather than one at a time", () => + it.live("resolves images concurrently, at most CONTAINER_CLI_CONCURRENCY at a time", () => Effect.gen(function* () { - const started = yield* Ref.make(0); - const bothStarted = yield* Deferred.make<void>(); + const inFlight = yield* Ref.make(0); + const peak = yield* Ref.make(0); + const capReached = yield* Deferred.make<void>(); const spawner = ChildProcessSpawner.make((command) => Effect.gen(function* () { const args = command._tag === "StandardCommand" ? command.args : []; if (args[0] === "image" && args[1] === "inspect") { - const count = yield* Ref.updateAndGet(started, (n) => n + 1); - if (count < 2) { - // A sequential implementation would hang here forever, since the second call - // never starts until this one returns. - yield* Deferred.await(bothStarted); - } else { - yield* Deferred.succeed(bothStarted, undefined); - } + const count = yield* Ref.updateAndGet(inFlight, (n) => n + 1); + yield* Ref.update(peak, (n) => Math.max(n, count)); + if (count === CONTAINER_CLI_CONCURRENCY) yield* Deferred.succeed(capReached, undefined); + yield* Deferred.await(capReached); + /* Yield before decrementing so an unbounded fan-out exceeds the cap. */ + yield* Effect.yieldNow; + yield* Ref.update(inFlight, (n) => n - 1); } const exitDeferred = yield* Deferred.make<ChildProcessSpawner.ExitCode>(); yield* Deferred.succeed(exitDeferred, ChildProcessSpawner.ExitCode(0)); @@ -141,8 +142,13 @@ describe("ensureImagesCached", () => { }), ); - const resolved = yield* ensureImagesCached(spawner, ["supabase/a:1", "supabase/b:1"]); - expect(resolved.size).toBe(2); + const images = Array.from( + { length: CONTAINER_CLI_CONCURRENCY + 4 }, + (_, index) => `supabase/image-${index}:1`, + ); + const resolved = yield* ensureImagesCached(spawner, images); + expect(resolved.size).toBe(images.length); + expect(yield* Ref.get(peak)).toBe(CONTAINER_CLI_CONCURRENCY); }), ); diff --git a/apps/cli/src/command-internal/docker-remove-all.ts b/apps/cli/src/command-internal/docker-remove-all.ts index 9612139963..a37893b1fd 100644 --- a/apps/cli/src/command-internal/docker-remove-all.ts +++ b/apps/cli/src/command-internal/docker-remove-all.ts @@ -7,6 +7,7 @@ import { ErrorActionabilityId, } from "../shared/telemetry/error-actionability.ts"; import { + CONTAINER_CLI_CONCURRENCY, containerCliExitCode, containerCliOutput, describeContainerCliFailure, @@ -171,7 +172,7 @@ export const dockerRemoveAll = ( stderr: "ignore", }).pipe(Effect.result), ), - { concurrency: "unbounded" }, + { concurrency: CONTAINER_CLI_CONCURRENCY }, ); const failedStop = stopResults.find( (result) => Result.isFailure(result) || result.success !== 0, diff --git a/apps/cli/src/commands/stop/SIDE_EFFECTS.md b/apps/cli/src/commands/stop/SIDE_EFFECTS.md index 97585d1337..bd206c8d98 100644 --- a/apps/cli/src/commands/stop/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/stop/SIDE_EFFECTS.md @@ -172,7 +172,9 @@ flags.noBackup`. `--backup=false` alone does **not** delete volumes; only volume by default, so dropping it there is lossless. Podman itself is a TS-only fallback (the old Go CLI talked to the Docker Engine API directly rather than shelling out to a `docker`/`podman` binary), so there's no parity concern here either way. -- Containers are stopped concurrently (`Effect.all(..., { concurrency: "unbounded" })`). +- Containers are stopped concurrently, at most `CONTAINER_CLI_CONCURRENCY` in flight on Docker and + Podman alike (shared with start image prepull), so stopping more than that many containers that + ignore their stop signal can take more than one 10 s grace period. Every container's failure is checked before failing the command (rather than stopping at the first failure) — though the surfaced message is a single fixed string rather than a joined list of per-container errors, since Docker CLI subprocess stderr isn't diff --git a/apps/cli/src/commands/stop/stop.integration.test.ts b/apps/cli/src/commands/stop/stop.integration.test.ts index 3a25eebdf3..b8cef7f80e 100644 --- a/apps/cli/src/commands/stop/stop.integration.test.ts +++ b/apps/cli/src/commands/stop/stop.integration.test.ts @@ -11,6 +11,7 @@ import { Option, Path, PlatformError, + Ref, Sink, Stream, } from "effect"; @@ -25,6 +26,7 @@ import { useTempWorkdir, withEnvVar, } from "../../../tests/helpers/command-mocks.ts"; +import { CONTAINER_CLI_CONCURRENCY } from "../../command-internal/container-cli.ts"; import { DebugFlag } from "../../command-internal/global-flags.ts"; import { runtimeInfoLayer } from "../../shared/runtime/runtime-info.layer.ts"; import { stop } from "./stop.handler.ts"; @@ -89,6 +91,7 @@ function mockRoutedContainerCliSpawner( // Fails both docker and podman spawn attempts matching this predicate, simulating a // runtime that cannot be spawned at all (vs. a spawned process exiting non-zero). readonly failSpawnFor?: (args: ReadonlyArray<string>) => boolean; + readonly onSpawn?: (args: ReadonlyArray<string>) => Effect.Effect<void>; } = {}, ) { const spawned: Array<SpawnRecord> = []; @@ -101,6 +104,7 @@ function mockRoutedContainerCliSpawner( const cmd = standard ? command.command : ""; const args = standard ? command.args : []; spawned.push({ command: cmd, args }); + if (opts.onSpawn !== undefined) yield* opts.onSpawn(args); if (opts.dockerMissing === true && cmd === "docker") { return yield* PlatformError.systemError({ @@ -197,6 +201,7 @@ interface SetupOpts { readonly route?: (args: ReadonlyArray<string>) => RouteResult; readonly dockerMissing?: boolean; readonly failSpawnFor?: (args: ReadonlyArray<string>) => boolean; + readonly onSpawn?: (args: ReadonlyArray<string>) => Effect.Effect<void>; readonly configuredProjectId?: string; readonly skipConfig?: boolean; /** Defaults to `tempRoot.current` — override for `--workdir`-resolution tests. */ @@ -220,6 +225,7 @@ const setup = (opts: SetupOpts = {}) => const child = mockRoutedContainerCliSpawner(opts.route ?? defaultRoute(), { dockerMissing: opts.dockerMissing, failSpawnFor: opts.failSpawnFor, + onSpawn: opts.onSpawn, }); const layer = Layer.mergeAll( @@ -1195,6 +1201,41 @@ enabled = true }).pipe(Effect.provide(BunServices.layer)), ); + it.live("stops many containers in order, at most CONTAINER_CLI_CONCURRENCY at a time", () => + Effect.gen(function* () { + const containerIds = Array.from( + { length: CONTAINER_CLI_CONCURRENCY + 6 }, + (_, index) => `c${index + 1}`, + ); + const inFlight = yield* Ref.make(0); + const peak = yield* Ref.make(0); + const capReached = yield* Deferred.make<void>(); + const { layer, out, child } = yield* setup({ + configuredProjectId: "demo", + route: defaultRoute({ containerIds }), + onSpawn: (args) => + args[0] !== "stop" + ? Effect.void + : Effect.gen(function* () { + const count = yield* Ref.updateAndGet(inFlight, (n) => n + 1); + yield* Ref.update(peak, (n) => Math.max(n, count)); + if (count === CONTAINER_CLI_CONCURRENCY) { + yield* Deferred.succeed(capReached, undefined); + } + yield* Deferred.await(capReached); + /* Yield before decrementing so an unbounded fan-out exceeds the cap. */ + yield* Effect.yieldNow; + yield* Ref.update(inFlight, (n) => n - 1); + }), + }); + yield* stop(flags()).pipe(Effect.provide(layer)); + const stopCalls = child.spawned.filter((s) => s.args[0] === "stop"); + expect(stopCalls.map((s) => s.args)).toEqual(containerIds.map((id) => ["stop", id])); + expect(yield* Ref.get(peak)).toBe(CONTAINER_CLI_CONCURRENCY); + expect(out.stdoutText).toContain("Stopped"); + }).pipe(Effect.provide(BunServices.layer)), + ); + it.effect("falls back to podman when docker is absent", () => Effect.gen(function* () { const { layer, child } = yield* setup({ From 2e9975e4982dcec07ee919a9d5c4f3332a44c822 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Thu, 22 Oct 2026 13:44:41 +0000 Subject: [PATCH 17/24] fix(cli): add the country code to the `test_otp` example (CLI-2820) (#7279) ## TL;DR fixes the `[auth.sms.test_otp]` example that `supabase init` writes, so its phone gets the test code instead of a real SMS ## whats hurts? `supabase init` writes `4152127777 = "123456"` as the test OTP example, without the country code auth strips the leading `+` from the phone and looks it up as an exact key, so `+14152127777` never matches `4152127777` uncommenting the example and sending that phone calls the configured SMS provider instead of returning the test code ## now fixed by? the example key is now `14152127777` with a comment saying keys are the phone number with country code and without the `+` prefix, so `+14152127777` gets `123456` ## ref: - closes: CLI-2820 - related: https://github.com/supabase/supabase/pull/52118 --- apps/cli-go/pkg/config/templates/config.toml | 3 ++- apps/cli/src/shared/init/project-init.templates.ts | 3 ++- apps/cli/src/shared/init/testdata/go-templates/config.toml | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/config.toml b/apps/cli-go/pkg/config/templates/config.toml index c26043d5c9..937c65bfc2 100644 --- a/apps/cli-go/pkg/config/templates/config.toml +++ b/apps/cli-go/pkg/config/templates/config.toml @@ -267,8 +267,9 @@ template = "Your code is {{ `{{ .Code }}` }}" max_frequency = "5s" # Use pre-defined map of phone number to OTP for testing. +# Keys are the phone number with country code and without the + prefix. # [auth.sms.test_otp] -# 4152127777 = "123456" +# 14152127777 = "123456" # Configure logged in session timeouts. # [auth.sessions] diff --git a/apps/cli/src/shared/init/project-init.templates.ts b/apps/cli/src/shared/init/project-init.templates.ts index 0bd00b1537..2d4e0ef25b 100644 --- a/apps/cli/src/shared/init/project-init.templates.ts +++ b/apps/cli/src/shared/init/project-init.templates.ts @@ -269,8 +269,9 @@ template = "Your code is {{ .Code }}" max_frequency = "5s" # Use pre-defined map of phone number to OTP for testing. +# Keys are the phone number with country code and without the + prefix. # [auth.sms.test_otp] -# 4152127777 = "123456" +# 14152127777 = "123456" # Configure logged in session timeouts. # [auth.sessions] diff --git a/apps/cli/src/shared/init/testdata/go-templates/config.toml b/apps/cli/src/shared/init/testdata/go-templates/config.toml index c26043d5c9..937c65bfc2 100644 --- a/apps/cli/src/shared/init/testdata/go-templates/config.toml +++ b/apps/cli/src/shared/init/testdata/go-templates/config.toml @@ -267,8 +267,9 @@ template = "Your code is {{ `{{ .Code }}` }}" max_frequency = "5s" # Use pre-defined map of phone number to OTP for testing. +# Keys are the phone number with country code and without the + prefix. # [auth.sms.test_otp] -# 4152127777 = "123456" +# 14152127777 = "123456" # Configure logged in session timeouts. # [auth.sessions] From e76408c48da3012f76f8b504706675e886c04eef Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Thu, 22 Oct 2026 13:44:50 +0000 Subject: [PATCH 18/24] fix(cli): upload multiline dynamic imports on deploy (CLI-2827) (#7282) ## TL;DR `functions deploy --use-api` now uploads modules behind dynamic imports written across lines or with import attributes, so the deployed function no longer throws `Module not found` ## whats improved/fixed? the deploy only uploads files its import scanner finds, and that scanner skipped any `import()` where `)` didn't follow the closing quote right away so a specifier on its own line, a trailing comma, or `{ with: { type: "json" } }` left the module out while the deploy still exited 0... ## how? the `import()` branch now allows whitespace and `,` after the specifier and pairs its quotes, so a concatenated path like `"./locales/" + lang + ".json"` is never read as a file that fails `functions serve` ## ref: - closes: https://github.com/supabase/cli/issues/7280 --- .../deploy/deploy.integration.test.ts | 36 +++++++++++++++++++ apps/cli/src/shared/functions/deploy.ts | 4 +-- 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts index 198c545795..2e47137c5f 100644 --- a/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts +++ b/apps/cli/src/commands/functions/deploy/deploy.integration.test.ts @@ -1269,6 +1269,42 @@ describe("functions deploy", () => { }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); }); + it.effect("uploads modules loaded by a dynamic import() split across lines", () => { + const { out, layer } = apiDeployHarness(deployedHelloWorld); + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* writeCliConfig(tempRoot.current); + yield* writeLocalFunction( + tempRoot.current, + "hello-world", + [ + 'await import(\n "./multi.ts"\n);', + 'await import(\n "./comma.ts",\n);', + 'await import(\n "./data.json",\n { with: { type: "json" } },\n);', + 'await import(\n "./locales/" + lang + ".json",\n { with: { type: "json" } },\n);', + 'await import("./it\'s.ts");', + "", + ].join("\n"), + ); + const imported = ["multi.ts", "comma.ts", "data.json", "it's.ts"]; + const functionDir = path.join(tempRoot.current, "supabase", "functions", "hello-world"); + for (const file of imported) { + yield* fs.writeFileString(path.join(functionDir, file), "{}\n"); + } + + yield* functionsDeploy(baseFlags); + + for (const file of imported) { + expect(out.stderrText).toContain( + `Uploading asset (hello-world): supabase/functions/hello-world/${file}\n`, + ); + } + expect(out.stderrText).not.toContain("WARN:"); + }).pipe(Effect.provide(layer), Effect.ensuring(removeTempRoot)); + }); + it.live("reports a bundle output directory that cannot be created as an unknown error", () => { const out = mockOutput({ format: "text" }); const child = mockDockerBundleSpawner(); diff --git a/apps/cli/src/shared/functions/deploy.ts b/apps/cli/src/shared/functions/deploy.ts index 4c9d45e554..a3c5b63d7e 100644 --- a/apps/cli/src/shared/functions/deploy.ts +++ b/apps/cli/src/shared/functions/deploy.ts @@ -79,7 +79,7 @@ const SUPABASE_FUNCTIONS_DIR = "supabase/functions"; const IMPORT_MAP_GUIDE_URL = "https://supabase.com/docs/guides/functions/import-maps"; const WINDOWS_ABSOLUTE_PATH = /^[A-Za-z]:\//; const importPathPattern = - /(?:import|export)\s+(?:type\s+)?(?:{[^{}]+}|.*?)\s*(?:from)?\s*['"](.*?)['"]|import\(\s*['"](.*?)['"]\)/gi; + /(?:import|export)\s+(?:type\s+)?(?:{[^{}]+}|.*?)\s*(?:from)?\s*['"](.*?)['"]|import\(\s*(?:"([^"\r\n]*)"|'([^'\r\n]*)')\s*[,)]/gi; export function shouldChmodBundleOutputDirectory(platform: NodeJS.Platform) { return platform !== "win32"; @@ -857,7 +857,7 @@ const walkImportPaths = Effect.fnUntraced(function* < const text = new TextDecoder().decode(contents.value); importPathPattern.lastIndex = 0; for (const match of text.matchAll(importPathPattern)) { - const raw = match[1] ?? match[2]; + const raw = match[1] ?? match[2] ?? match[3]; if (raw === undefined) { continue; } From 37e309e8ed628091c9e4624ac13adf8c8769e51f Mon Sep 17 00:00:00 2001 From: Andrew Valleteau <avallete@users.noreply.github.com> Date: Thu, 22 Oct 2026 16:09:10 +0000 Subject: [PATCH 19/24] chore(cli): bump pg-delta to 1.0.0-alpha.60 (#7283) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates `@supabase/pg-delta` to `1.0.0-alpha.60` and moves its `minimumReleaseAgeExclude` entry to the new version. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --- apps/cli/package.json | 2 +- pnpm-lock.yaml | 10 +++++----- pnpm-workspace.yaml | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/apps/cli/package.json b/apps/cli/package.json index 794563d8bc..a9a6612162 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -57,7 +57,7 @@ "@napi-rs/keyring": "^1.3.0", "@supabase/api": "workspace:*", "@supabase/config": "workspace:*", - "@supabase/pg-delta": "1.0.0-alpha.59", + "@supabase/pg-delta": "1.0.0-alpha.60", "@supabase/pg-topo": "1.0.0-alpha.8", "@supabase/stack": "workspace:*", "@supabase/supabase-js": "catalog:", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 783fc5e73f..1109b76646 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -331,8 +331,8 @@ importers: specifier: workspace:* version: link:../../packages/config '@supabase/pg-delta': - specifier: 1.0.0-alpha.59 - version: 1.0.0-alpha.59(@supabase/pg-topo@1.0.0-alpha.8(supports-color@7.2.0))(supports-color@7.2.0) + specifier: 1.0.0-alpha.60 + version: 1.0.0-alpha.60(@supabase/pg-topo@1.0.0-alpha.8(supports-color@7.2.0))(supports-color@7.2.0) '@supabase/pg-topo': specifier: 1.0.0-alpha.8 version: 1.0.0-alpha.8(supports-color@7.2.0) @@ -2859,8 +2859,8 @@ packages: resolution: {integrity: sha512-DQ0aVH8wSQAccVqNoEkec62qCu2QRNyoGN53RqsVZ1k6F1zq4/v8scrlR6LNT2RJmT97apiTmORijPVhErCS2g==} engines: {node: '>=22.0.0'} - '@supabase/pg-delta@1.0.0-alpha.59': - resolution: {integrity: sha512-8uXrWOhInH5mCNByPM74uJm5RjBkF+1zMvK/QkGvxxGIhYtz/pt/DqnwpcPRsGPumicbaYLPGqKoySrw7IDMGw==} + '@supabase/pg-delta@1.0.0-alpha.60': + resolution: {integrity: sha512-pUO6OPOCOhcz89tHL+oOGWUPkVgPiw50n7ftnpUoozdOhqsPEANj81SvRMXClCepgkysvI+CJSN868vtj6ueog==} engines: {node: '>=20.0.0'} hasBin: true peerDependencies: @@ -8565,7 +8565,7 @@ snapshots: dependencies: tslib: 2.8.1 - '@supabase/pg-delta@1.0.0-alpha.59(@supabase/pg-topo@1.0.0-alpha.8(supports-color@7.2.0))(supports-color@7.2.0)': + '@supabase/pg-delta@1.0.0-alpha.60(@supabase/pg-topo@1.0.0-alpha.8(supports-color@7.2.0))(supports-color@7.2.0)': dependencies: '@types/debug': 4.1.13 '@types/pg': 8.23.1 diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 39d3369a86..ab9ce8003d 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -107,7 +107,7 @@ minimumReleaseAgeExclude: - "@effect/platform-node-shared@4.0.2" - "@effect/sql-pg@4.0.2" - "@effect/vitest@4.0.2" - - "@supabase/pg-delta@1.0.0-alpha.59" + - "@supabase/pg-delta@1.0.0-alpha.60" - "@supabase/pg-topo@1.0.0-alpha.8" - "@supabase/postgrest-typegen@0.6.0" - "@supabase/typegen@0.3.1" From 522a516ba1eb582115aa23bd48f01b17791aae94 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 22:38:44 +0000 Subject: [PATCH 20/24] chore(stack): bump storage to v1.84.19-r0 (#7284) This bumps storage from v1.84.17-r0 to v1.84.19-r0. https://github.com/supabase/slim-services/releases/tag/storage-v1.84.19-r0 Planned after supabase/slim-services published v1.84.19-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli-go/pkg/config/templates/Dockerfile | 2 +- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 18 +++++++++--------- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/Dockerfile b/apps/cli-go/pkg/config/templates/Dockerfile index 40f21a699f..6141335710 100644 --- a/apps/cli-go/pkg/config/templates/Dockerfile +++ b/apps/cli-go/pkg/config/templates/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.147.1 AS realtime -FROM supabase/storage-api:v1.84.17 AS storage +FROM supabase/storage-api:v1.84.19 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 40f21a699f..6141335710 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -35,7 +35,7 @@ FROM supabase/gotrue:v2.197.0 AS gotrue FROM supabase/realtime:v2.147.1 AS realtime -FROM supabase/storage-api:v1.84.17 AS storage +FROM supabase/storage-api:v1.84.19 AS storage FROM supabase/logflare:1.50.15 AS logflare # One-shot job images (not started as long-running containers); differ's alias diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index aa22bdcd2a..1f630dd144 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -322,23 +322,23 @@ const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { storage: definition( "storage", { - upstreamVersion: "v1.84.17", + upstreamVersion: "v1.84.19", revision: 0, image: - "ghcr.io/supabase/cli/storage:v1.84.17-r0@sha256:b1ad3c089bcb7d28437d4b2bc9d9053bed9234d2b2ad175fbc302dc82fd1f316", - upstreamImage: "supabase/storage-api:v1.84.17", + "ghcr.io/supabase/cli/storage:v1.84.19-r0@sha256:ac022dd6fba7ef5f84d676bbe776445c5e1955b786fe14b75a5669015ea771f2", + upstreamImage: "supabase/storage-api:v1.84.19", natives: { "darwin-arm64": { - archive: "4059c5433aab210a60cbb2548b3b1dff1e24587e5c040427986dff78e0f0e2a9", - manifest: "d90ef9c66cd48f46d49b2d485a74d205de430efb922c730eb303b08127581b28", + archive: "019aa9887ffeddf03a1465ae3a59eaaa983a09aecd75b1ad1371aa40cd72f2ea", + manifest: "73395585a635aad109912ae78206f68939c7c927cc2b69db9923f15a7804dfa2", }, "linux-amd64": { - archive: "2f95354e7a45041acfd75b25f9d19fb9b5006827464284909af67624f2cf6cbb", - manifest: "b8b86c57e228ec90c163d1bea92a337c35f87ff856b8e483ccc3967d0edb5985", + archive: "61c049488a4deec67a8b4da947057924391f7efa0e7717b0648ca170255ee998", + manifest: "77f281f5684dd16df93162fb38740ade98464295902247403f2f8607bd5d9320", }, "linux-arm64": { - archive: "639c0b897610319ad2518e1671c574c792c9fe7e5a20ab2ce6fd0b2b18990c53", - manifest: "115450d2a0473d6491aa15c1677086c9947c913edd8a022faf17d1d2f63c7a8e", + archive: "689340dcd56b740335a144bf9b3ddb1146e3f13973d0a593df565d22e8ce151a", + manifest: "7d9e3b1d4aa8f91c2707641b64d44d8efe32a61d2923e0c0940729193199fef9", }, }, }, From ce77899cb45383dfca7f4ee5f44a285f342536c0 Mon Sep 17 00:00:00 2001 From: Andrew Valleteau <avallete@users.noreply.github.com> Date: Fri, 23 Oct 2026 07:57:31 +0000 Subject: [PATCH 21/24] ci(cli): retry rate-limited registry probes when detecting unmirrored images (#7289) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The mirror-template-images detect step checks public.ecr.aws anonymously, and the shared per-IP quota can answer `429 Too Many Requests`, which aborted the run. Rate-limited probes now retry with backoff (10s, 30s, 60s) before failing with the registry error; other failures still abort immediately. This avoids giving the detect job, which installs npm dependencies, the production AWS role. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --- apps/cli/scripts/detect-unmirrored-images.ts | 52 +++++++++++++++---- .../detect-unmirrored-images.unit.test.ts | 24 +++++++++ 2 files changed, 67 insertions(+), 9 deletions(-) diff --git a/apps/cli/scripts/detect-unmirrored-images.ts b/apps/cli/scripts/detect-unmirrored-images.ts index e1d0bd7010..ee055e499d 100644 --- a/apps/cli/scripts/detect-unmirrored-images.ts +++ b/apps/cli/scripts/detect-unmirrored-images.ts @@ -83,16 +83,50 @@ export function inspectResultIsMirrored( throw new Error(`could not inspect ${target} (exit ${exit}): ${stderr.trim()}`); } -function imageExistsOnMirror(target: string): Promise<boolean> { - const result = spawnSync("docker", ["buildx", "imagetools", "inspect", target], { - encoding: "utf8", - stdio: ["ignore", "ignore", "pipe"], - }); - if (result.error) { - throw result.error; +export interface InspectResult { + readonly exit: number | NodeJS.Signals | null; + readonly stderr: string; +} + +/** Backoff before each retry of a rate-limited (HTTP 429) registry probe. */ +export const RATE_LIMIT_RETRY_DELAYS_MS: ReadonlyArray<number> = [10_000, 30_000, 60_000]; + +/** + * Runs a registry probe, retrying with backoff while the registry answers 429; the anonymous + * public.ecr.aws quota is shared per runner IP and usually recovers within a minute. + */ +export async function inspectWithRateLimitRetry( + target: string, + inspect: () => InspectResult, + sleep: (ms: number) => Promise<void>, + delaysMs: ReadonlyArray<number> = RATE_LIMIT_RETRY_DELAYS_MS, +): Promise<boolean> { + for (let attempt = 0; ; attempt++) { + const { exit, stderr } = inspect(); + const delay = delaysMs[attempt]; + if (exit !== 0 && stderr.includes("429 Too Many Requests") && delay !== undefined) { + console.error(`rate limited inspecting ${target}, retrying in ${delay}ms`); + await sleep(delay); + continue; + } + return inspectResultIsMirrored(target, exit, stderr); } - return Promise.resolve( - inspectResultIsMirrored(target, result.status ?? result.signal, result.stderr), +} + +function imageExistsOnMirror(target: string): Promise<boolean> { + return inspectWithRateLimitRetry( + target, + () => { + const result = spawnSync("docker", ["buildx", "imagetools", "inspect", target], { + encoding: "utf8", + stdio: ["ignore", "ignore", "pipe"], + }); + if (result.error) { + throw result.error; + } + return { exit: result.status ?? result.signal, stderr: result.stderr }; + }, + (ms) => new Promise((resolve) => setTimeout(resolve, ms)), ); } diff --git a/apps/cli/scripts/detect-unmirrored-images.unit.test.ts b/apps/cli/scripts/detect-unmirrored-images.unit.test.ts index 2daedd6175..a3c2d93172 100644 --- a/apps/cli/scripts/detect-unmirrored-images.unit.test.ts +++ b/apps/cli/scripts/detect-unmirrored-images.unit.test.ts @@ -1,6 +1,7 @@ import { describe, expect, test } from "vitest"; import { inspectResultIsMirrored, + inspectWithRateLimitRetry, MIRROR_REGISTRIES, mirrorImageTarget, mirrorImageTargets, @@ -77,4 +78,27 @@ describe("detect unmirrored images", () => { `could not inspect ${target} (exit 1): ${stderr.trim()}`, ); }); + + test("a rate-limited probe is retried with backoff and only aborts once retries run out", async () => { + const target = "public.ecr.aws/supabase/postgres:17.11.0.006"; + const rateLimited = { + exit: 1, + stderr: "ERROR: httpReadSeeker: failed open: unexpected status: 429 Too Many Requests\n", + }; + const slept: number[] = []; + const sleep = (ms: number) => { + slept.push(ms); + return Promise.resolve(); + }; + + const responses = [rateLimited, rateLimited, { exit: 0, stderr: "" }]; + await expect( + inspectWithRateLimitRetry(target, () => responses.shift()!, sleep, [1, 2, 3]), + ).resolves.toBe(true); + expect(slept).toEqual([1, 2]); + + await expect(inspectWithRateLimitRetry(target, () => rateLimited, sleep, [1])).rejects.toThrow( + "429 Too Many Requests", + ); + }); }); From 7a6fd005d55fad509cba8acb043133e792cfdb24 Mon Sep 17 00:00:00 2001 From: Vaibhav <117663341+7ttp@users.noreply.github.com> Date: Fri, 23 Oct 2026 08:05:53 +0000 Subject: [PATCH 22/24] feat(cli): honor tls settings in `pg_dump` (CLI-2753) (#7266) ## TL;DR remote `db dump` and the first `db pull` seed now pass the resolved TLS settings to `pg_dump` ## whats introduced? `pg_dump` only got host, port, user, password and database, so a container dump ignored `PGSSLMODE=verify-full`, trusted any CA, and got turned away by servers that require a client cert. for the linked project and a remote `--db-url`, it now gets the resolved `sslmode` and connect timeout, the root cert under `require` or `verify-*`, and the client cert unless the key needs `sslpassword` In a container the cert files are mounted read only, and a missing one fails before docker runs > [!NOTE] > a remote dump that only worked because verification was skipped now fails. local targets and runs without TLS or timeout settings are unchanged <details> <summary>visual rep:</summary> ```mermaid flowchart TD conn["resolved connection"] --> remote{"remote target?"} remote -- no --> plain["host, port, user, password, database"] remote -- yes --> tls["+ sslmode and connect timeout<br/>+ root cert under require or verify-*<br/>+ system root cert forces verify-full<br/>+ client cert unless the key needs sslpassword"] tls --> where{"where pg_dump runs"} where -- container --> box["cert files checked, then mounted read only<br/>key copied to a private file<br/>verify-* without root cert, or system, uses the image CA bundle"] where -- native --> host["host paths kept<br/>verify-* without root cert uses ~/.postgresql/root.crt, else the host CA bundle<br/>system uses the host CA bundle"] plain --> dump["pg_dump"] box --> dump host --> dump ``` </details> ## ref - spotted while working on: https://github.com/supabase/cli/pull/7261 - related: CLI-2753 --- apps/cli/src/command-internal/db-pull-run.ts | 1 + apps/cli/src/command-internal/pg-dump.env.ts | 31 +- .../command-internal/pg-dump.env.unit.test.ts | 42 +++ apps/cli/src/command-internal/pg-dump.run.ts | 127 ++++++++- .../command-internal/pg-dump.run.unit.test.ts | 2 + apps/cli/src/commands/db/dump/SIDE_EFFECTS.md | 19 +- apps/cli/src/commands/db/dump/dump.handler.ts | 1 + .../commands/db/dump/dump.integration.test.ts | 267 ++++++++++++++++++ apps/cli/src/commands/db/pull/SIDE_EFFECTS.md | 2 +- .../commands/db/pull/pull.integration.test.ts | 16 ++ apps/cli/src/commands/db/push/SIDE_EFFECTS.md | 3 +- apps/cli/src/commands/pull/SIDE_EFFECTS.md | 2 +- 12 files changed, 495 insertions(+), 18 deletions(-) diff --git a/apps/cli/src/command-internal/db-pull-run.ts b/apps/cli/src/command-internal/db-pull-run.ts index c10db0437a..1fc8f4cbfe 100644 --- a/apps/cli/src/command-internal/db-pull-run.ts +++ b/apps/cli/src/command-internal/db-pull-run.ts @@ -511,6 +511,7 @@ export const runDbPull = Effect.fn("db.pull.run")(function* ( keepComments: false, excludeTable: [], columnInsert: false, + remote: !resolved.isLocal, }; const toDumpOpenError = (cause: { readonly message: string }) => new DbPullDumpError({ diff --git a/apps/cli/src/command-internal/pg-dump.env.ts b/apps/cli/src/command-internal/pg-dump.env.ts index 2ec6e70a90..0c35380e21 100644 --- a/apps/cli/src/command-internal/pg-dump.env.ts +++ b/apps/cli/src/command-internal/pg-dump.env.ts @@ -111,21 +111,42 @@ export interface DumpOptions { readonly excludeTable: ReadonlyArray<string>; /** `true` emits `--column-inserts` instead of `COPY` statements. */ readonly columnInsert: boolean; + /** `true` forwards the conn's TLS settings to `pg_dump`. */ + readonly remote?: boolean; } -export function toDumpEnv(conn: PgConnInput): Record<string, string> { - return { +const TLS_DEMANDING_SSLMODES = new Set(["require", "verify-ca", "verify-full"]); + +export function toDumpEnv(conn: PgConnInput, remote = false): Record<string, string> { + const env: Record<string, string> = { PGHOST: conn.host, PGPORT: String(conn.port), PGUSER: conn.user, PGPASSWORD: conn.password, PGDATABASE: conn.database, }; + if (!remote) return env; + /* libpq's `system` root cert implies and only allows `verify-full`. */ + const sslmode = + conn.sslrootcert === "system" && conn.sslmode !== "disable" ? "verify-full" : conn.sslmode; + if (sslmode !== undefined) env["PGSSLMODE"] = sslmode; + if (conn.connectTimeoutSeconds !== undefined) + env["PGCONNECT_TIMEOUT"] = String(conn.connectTimeoutSeconds); + if (sslmode === "disable") return env; + /* Under `prefer`, libpq falls back to plaintext when the root cert does not match. */ + if (conn.sslrootcert !== undefined && TLS_DEMANDING_SSLMODES.has(sslmode ?? "")) + env["PGSSLROOTCERT"] = conn.sslrootcert; + /* libpq takes no key passphrase from the env, so an encrypted key would stop `pg_dump`. */ + if (conn.sslcert !== undefined && conn.sslkey !== undefined && conn.sslpassword === undefined) { + env["PGSSLCERT"] = conn.sslcert; + env["PGSSLKEY"] = conn.sslkey; + } + return env; } /** Env assembly for a schema-only dump. */ export function buildSchemaDumpEnv(conn: PgConnInput, opt: DumpOptions): Record<string, string> { - const env = toDumpEnv(conn); + const env = toDumpEnv(conn, opt.remote); if (opt.schema.length > 0) { // Must append flag because empty string results in error. env["EXTRA_FLAGS"] = `--schema=${opt.schema.join("|")}`; @@ -140,7 +161,7 @@ export function buildSchemaDumpEnv(conn: PgConnInput, opt: DumpOptions): Record< /** Env assembly for a data-only dump. */ export function buildDataDumpEnv(conn: PgConnInput, opt: DumpOptions): Record<string, string> { - const env = toDumpEnv(conn); + const env = toDumpEnv(conn, opt.remote); if (opt.schema.length > 0) { env["INCLUDED_SCHEMAS"] = opt.schema.join("|"); } else { @@ -170,7 +191,7 @@ export function quoteUpperCase(table: string): string { /** Env assembly for a role dump. */ export function buildRoleDumpEnv(conn: PgConnInput, opt: DumpOptions): Record<string, string> { - const env = toDumpEnv(conn); + const env = toDumpEnv(conn, opt.remote); env["RESERVED_ROLES"] = RESERVED_ROLES.join("|"); env["ALLOWED_CONFIGS"] = ALLOWED_CONFIGS.join("|"); if (!opt.keepComments) { diff --git a/apps/cli/src/command-internal/pg-dump.env.unit.test.ts b/apps/cli/src/command-internal/pg-dump.env.unit.test.ts index 2d4209c49f..024abf2be6 100644 --- a/apps/cli/src/command-internal/pg-dump.env.unit.test.ts +++ b/apps/cli/src/command-internal/pg-dump.env.unit.test.ts @@ -40,6 +40,48 @@ describe("toDumpEnv", () => { PGDATABASE: "postgres", }); }); + + it("forwards a remote sslmode=disable without any PEM path", () => { + const env = toDumpEnv( + { + ...CONN, + sslmode: "disable", + sslcert: "/certs/client.crt", + sslkey: "/certs/client.key", + connectTimeoutSeconds: 3, + }, + true, + ); + expect(env["PGSSLMODE"]).toBe("disable"); + expect(env["PGCONNECT_TIMEOUT"]).toBe("3"); + expect(env["PGSSLCERT"]).toBeUndefined(); + expect(env["PGSSLKEY"]).toBeUndefined(); + }); + + it("pins verify-full for libpq's system root cert", () => { + for (const sslmode of [undefined, "require", "verify-ca"]) + expect(toDumpEnv({ ...CONN, sslmode, sslrootcert: "system" }, true)).toMatchObject({ + PGSSLMODE: "verify-full", + PGSSLROOTCERT: "system", + }); + }); + + it("keeps the root cert and an encrypted key out of a remote dump", () => { + const env = toDumpEnv( + { + ...CONN, + sslmode: "prefer", + sslrootcert: "/certs/root.crt", + sslcert: "/certs/client.crt", + sslkey: "/certs/client.key", + sslpassword: "secret", + }, + true, + ); + expect(env["PGSSLROOTCERT"]).toBeUndefined(); + expect(env["PGSSLCERT"]).toBeUndefined(); + expect(env["PGSSLKEY"]).toBeUndefined(); + }); }); describe("buildSchemaDumpEnv", () => { diff --git a/apps/cli/src/command-internal/pg-dump.run.ts b/apps/cli/src/command-internal/pg-dump.run.ts index 15ba8c4f2a..8232be1e4a 100644 --- a/apps/cli/src/command-internal/pg-dump.run.ts +++ b/apps/cli/src/command-internal/pg-dump.run.ts @@ -1,4 +1,4 @@ -import { Effect, Option } from "effect"; +import { Config, Effect, FileSystem, Option, Path } from "effect"; import type { Stack } from "@supabase/stack/effect"; import type { ArtifactPin } from "@supabase/stack/internal/artifacts"; import { postgres } from "@supabase/stack/commands"; @@ -11,6 +11,7 @@ import { viperEnvStringWithProjectFallback } from "./viper-env.ts"; import { RuntimeInfo } from "../shared/runtime/runtime-info.service.ts"; import { DockerRun } from "./docker-run.service.ts"; import { DockerRunError } from "./docker-run.errors.ts"; +import { DbConnectError } from "./db-connection.errors.ts"; import { getRegistryImageUrl } from "./docker-registry.ts"; import { BundledPostgresClient, resolveBundledPostgresRuntime } from "./bundled-postgres-client.ts"; import { Output } from "../shared/output/output.service.ts"; @@ -58,6 +59,7 @@ export const streamPgDump = Effect.fnUntraced(function* <E>(params: { params.projectEnvValues ?? {}, ); const extraHosts = runtimeInfo.platform === "linux" ? ["host.docker.internal:host-gateway"] : []; + const tls = yield* containerTlsFiles(params.env, params.script); const image = yield* getRegistryImageUrl(params.image, params.projectEnvValues).pipe( Effect.mapError( @@ -73,11 +75,11 @@ export const streamPgDump = Effect.fnUntraced(function* <E>(params: { return yield* docker.runStream<E>( { image, - cmd: ["bash", "-c", params.script, "--"], - env: params.env, - binds: [], + cmd: ["bash", "-c", tls.script, "--"], + env: tls.env, + binds: tls.mounts.map((mount) => `${mount.source}:${mount.target}:ro`), workingDir: Option.none(), - securityOpt: [], + securityOpt: tls.securityOpt, extraHosts, network, projectEnvValues: params.projectEnvValues, @@ -86,6 +88,102 @@ export const streamPgDump = Effect.fnUntraced(function* <E>(params: { ); }); +const CONTAINER_TLS_FILES = [ + ["PGSSLROOTCERT", "sslrootcert", "/run/supabase/tls/root.crt"], + ["PGSSLCERT", "sslcert", "/run/supabase/tls/client.crt"], + ["PGSSLKEY", "sslkey", "/run/supabase/tls/client.key"], +] as const; + +const CONTAINER_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"; +const CONTAINER_PRIVATE_KEY = "/tmp/supabase-client.key"; + +const HOST_CA_BUNDLES = [ + "/etc/ssl/cert.pem", + "/etc/ssl/certs/ca-certificates.crt", + "/etc/pki/tls/certs/ca-bundle.crt", + "/etc/ssl/ca-bundle.pem", +] as const; + +/* Verifying without a root cert, or with `system`, needs a CA bundle libpq can read. */ +const usesDefaultTrustRoots = (env: Readonly<Record<string, string>>): boolean => + env["PGSSLROOTCERT"] === "system" || + ((env["PGSSLMODE"] === "verify-ca" || env["PGSSLMODE"] === "verify-full") && + env["PGSSLROOTCERT"] === undefined); + +const opensslOverride = (name: string) => + Config.option(Config.NonEmptyString(name)).pipe( + Effect.orElseSucceed(() => Option.none<string>()), + ); + +const nativeTlsEnv = Effect.fnUntraced(function* ( + env: Readonly<Record<string, string>>, + homeDir: string, + clientVersion: string, +) { + if (!usesDefaultTrustRoots(env)) return env; + if (env["PGSSLROOTCERT"] === "system") { + /* `system` means OpenSSL's verify paths, which honour these; libpq reads `system` from 16. */ + const certFile = yield* opensslOverride("SSL_CERT_FILE"); + const certDir = yield* opensslOverride("SSL_CERT_DIR"); + if ( + Number.parseInt(clientVersion, 10) >= 16 && + (Option.isSome(certFile) || Option.isSome(certDir)) + ) + return env; + if (Option.isSome(certFile)) return { ...env, PGSSLROOTCERT: certFile.value }; + } + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exists = (file: string) => fs.exists(file).pipe(Effect.orElseSucceed(() => false)); + /* libpq already trusts ~/.postgresql/root.crt by default, so leave it in charge. */ + if ( + env["PGSSLROOTCERT"] === undefined && + (yield* exists(path.join(homeDir, ".postgresql", "root.crt"))) + ) + return env; + for (const bundle of HOST_CA_BUNDLES) + if (yield* exists(bundle)) return { ...env, PGSSLROOTCERT: bundle }; + return env; +}); + +const containerTlsFiles = Effect.fnUntraced(function* ( + env: Readonly<Record<string, string>>, + script: string, +) { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const mounted = { ...env }; + const mounts: Array<{ source: string; target: string; readOnly: true }> = []; + for (const [key, setting, target] of CONTAINER_TLS_FILES) { + const value = env[key]; + if (value === undefined || (key === "PGSSLROOTCERT" && value === "system")) continue; + const source = path.resolve(value); + const unreadable = (reason: string) => + new DbConnectError({ message: `failed to read ${setting} ${value}: ${reason}` }); + /* A colon past a Windows drive prefix would split the `-v` spec into extra fields. */ + if (source.replace(/^[A-Za-z]:[\\/]/u, "").includes(":")) + return yield* unreadable("a Docker bind mount cannot contain ':'"); + /* `docker run -v` would create a missing source as a host directory. */ + const info = yield* fs + .stat(source) + .pipe(Effect.mapError((error) => unreadable(String(error.reason.cause ?? error.message)))); + if (info.type !== "File") return yield* unreadable("not a file"); + mounts.push({ source, target, readOnly: true }); + mounted[key] = target; + } + if (usesDefaultTrustRoots(env)) mounted["PGSSLROOTCERT"] = CONTAINER_CA_BUNDLE; + const key = mounted["PGSSLKEY"]; + let prelude = ""; + if (key !== undefined) { + /* libpq rejects a key Docker Desktop shares with loose permissions; use a private copy. */ + prelude = `(umask 077 && cp ${key} ${CONTAINER_PRIVATE_KEY}) || exit 1\n`; + mounted["PGSSLKEY"] = CONTAINER_PRIVATE_KEY; + } + /* SELinux blocks label-confined containers from reading host files. */ + const securityOpt = mounts.length > 0 ? ["label:disable"] : []; + return { env: mounted, mounts, securityOpt, script: prelude + script }; +}); + export type PgDumpClient = | { readonly kind: "container" } | { @@ -332,11 +430,26 @@ export const streamPgDumpWithClient = Effect.fn("streamPgDumpWithClient")(functi runtime.kind === "container" && runtimeInfo.platform === "linux" ? ["host.docker.internal:host-gateway"] : []; + const tls = + runtime.kind === "container" + ? yield* containerTlsFiles(params.env, params.script) + : { + env: yield* nativeTlsEnv( + params.env, + runtimeInfo.homeDir, + params.client.pin.upstreamVersion, + ), + mounts: [], + securityOpt: [], + script: params.script, + }; return yield* bundled.run({ pin: params.client.pin, runtime, - argv: ["bash", "-c", params.script, "--"], - env: params.env, + argv: ["bash", "-c", tls.script, "--"], + env: tls.env, + mounts: tls.mounts, + securityOpt: tls.securityOpt, network: yield* bundledDumpNetwork( Option.getOrUndefined(networkIdFlag), params.forceHostNetwork === true, diff --git a/apps/cli/src/command-internal/pg-dump.run.unit.test.ts b/apps/cli/src/command-internal/pg-dump.run.unit.test.ts index 967e83658a..e3bbd74131 100644 --- a/apps/cli/src/command-internal/pg-dump.run.unit.test.ts +++ b/apps/cli/src/command-internal/pg-dump.run.unit.test.ts @@ -1,4 +1,5 @@ import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; import { ConfigProvider, Effect, Layer, Option } from "effect"; import { DockerRun } from "./docker-run.service.ts"; @@ -40,6 +41,7 @@ describe("streamPgDump", () => { mockRuntimeInfo({ platform: "linux" }), Layer.succeed(NetworkIdFlag, Option.none()), ConfigProvider.layer(configProvider), + BunServices.layer, ), ), ); diff --git a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md index a0f2b22c69..a35d069505 100644 --- a/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/dump/SIDE_EFFECTS.md @@ -15,7 +15,7 @@ image), to stdout or `--file`. | `~/.supabase/access-token` | plain text | `--linked` when `SUPABASE_ACCESS_TOKEN` unset | | `supabase/.temp/project-ref` | plain text | `--linked` (and the default target) ref resolution — skipped when `--project-ref` (or `SUPABASE_PROJECT_ID`/config.toml `project_id`) is set | | `supabase/.env*` | dotenv | always (project env, feeds `SUPABASE_DB_PASSWORD` / `PG*`) | -| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | linked project, when set: `pg_service.conf`, plus the PEM files on the pooler temp-role probe or a native `pg_dump`; see `db push` | +| `PGSSL*` PEM, `pg_service.conf` | PEM, libpq | remote target, when set: `pg_service.conf`, plus the PEM files on the pooler temp-role probe and in `pg_dump` | ## Files Written @@ -41,12 +41,13 @@ image), to stdout or `--file`. | `SUPABASE_ACCESS_TOKEN` | `--linked` auth | | `SUPABASE_DB_ORIOLEDB_VERSION` | overrides `db.orioledb_version`, selecting the OrioleDB `pg_dump` image | | `SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION` | deprecated alias of `SUPABASE_DB_ORIOLEDB_VERSION`, used only while that is unset; when set, stderr gets `WARN: SUPABASE_EXPERIMENTAL_ORIOLEDB_VERSION is deprecated and will be removed in v3, use SUPABASE_DB_ORIOLEDB_VERSION instead.` | -| `BITBUCKET_CLONE_DIR` | (no-op for dump — no `--security-opt` is set) | +| `BITBUCKET_CLONE_DIR` | clears the `label:disable` security option that mounted TLS files add | | `SUPABASE_INTERNAL_IMAGE_REGISTRY` | rewrite the pg image registry for compose dumps; stack dumps use the catalog image pin unchanged | | `SUPABASE_USE_SLIM_IMAGES` | resolve the current Postgres pin from the slim `ghcr.io/supabase/cli` builds (`true`/`1` enable); majors 13/15 use `15.14.1.167` when the flag is on; historical pins, PG14, OrioleDB tags the catalog does not pin, and flag-off `15.8.1.085` stay on docker.io | | `DOCKER_HOST` | docker daemon endpoint | | `MSYSTEM`, `TERM_PROGRAM` | suppress the piped-stdout non-ASCII warning in MSYS/mintty sessions | -| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. An invalid value fails before `pg_dump` runs; the compose `pg_dump` container gets none, a native `pg_dump` sees shell values only | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. An invalid value fails before `pg_dump` runs; a remote target also hands them to `pg_dump` (see Notes) | +| `SSL_CERT_FILE`, `SSL_CERT_DIR` | a native `pg_dump` on libpq 16+ keeps `sslrootcert=system` when either is set; libpq 15 uses only `SSL_CERT_FILE` | ## Exit Codes @@ -118,6 +119,18 @@ Warning: supabase db dump will target the local database by default in the next suggestion uses the generic `ipv6Suggestion()` text rather than one that prefills the project's specific pooler connection string. Surfacing that exact URL needs the pooler string exposed at this seam. +- **TLS settings in `pg_dump`.** A remote target (linked project or remote `--db-url`) + gets the resolved `sslmode` and connect timeout, the root cert only under + `require`/`verify-ca`/`verify-full`, and the client cert and key unless the key needs + `sslpassword` (libpq reads no passphrase from the environment). `system` pins + `verify-full`. Verifying without a root cert, or with `system`, uses the image's trust + store in a container and the host's CA bundle for a native `pg_dump`, which keeps + libpq's `~/.postgresql/root.crt` when no root cert is set, keeps `system` on libpq 16+ + when `SSL_CERT_FILE` or `SSL_CERT_DIR` is set (libpq 15 uses only `SSL_CERT_FILE`), + and also inherits the shell's `PG*`. A container mounts the PEM files read-only under + `/run/supabase/tls` (the Docker daemon must see the same paths) and copies the key to + a private file. A missing or non-file path, or one containing `:` past a Windows + drive, fails before the container starts. - **Stack backend host rewrite.** Stack dumps always use catalog `pg_dump` (native artifact or a one-shot of the same image). `--local` native rewrites loopback to `127.0.0.1`. Native `--linked` / `--db-url` keep diff --git a/apps/cli/src/commands/db/dump/dump.handler.ts b/apps/cli/src/commands/db/dump/dump.handler.ts index 8a5d426551..fee9feb3fa 100644 --- a/apps/cli/src/commands/db/dump/dump.handler.ts +++ b/apps/cli/src/commands/db/dump/dump.handler.ts @@ -290,6 +290,7 @@ export const dbDump = Effect.fn("db.dump")(function* (flags: DbDumpFlags) { keepComments, excludeTable: flags.exclude, columnInsert: !flags.useCopy, + remote: !isLocal, }; // The script + diagnostic verb are connection-independent; the env is rebuilt // per connection so the pooler-fallback retry can target a different host. diff --git a/apps/cli/src/commands/db/dump/dump.integration.test.ts b/apps/cli/src/commands/db/dump/dump.integration.test.ts index 1dec38c4e7..fe1c2f9435 100644 --- a/apps/cli/src/commands/db/dump/dump.integration.test.ts +++ b/apps/cli/src/commands/db/dump/dump.integration.test.ts @@ -942,6 +942,254 @@ describe("db dump integration", () => { }, ); + const writePem = (name: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const file = path.join(tmp.current, name); + yield* fs.writeFileString(file, "-----BEGIN CERTIFICATE-----\n"); + return file; + }); + + it.effect("hands the linked project's TLS settings to the pg_dump container", () => + Effect.gen(function* () { + const path = yield* Path.Path; + const root = yield* writePem("root.crt"); + const cert = yield* writePem("client.crt"); + const key = yield* writePem("client.key"); + const { layer, docker } = setup({ + conn: { + ...REMOTE_CONN, + sslmode: "verify-full", + sslrootcert: root, + sslcert: path.relative(process.cwd(), cert), + sslkey: key, + connectTimeoutSeconds: 7, + }, + isLocal: false, + }); + yield* dbDump(flags({ linked: Option.some(true) })).pipe(Effect.provide(layer)); + expect(docker.lastOpts?.env).toMatchObject({ + PGHOST: REMOTE_CONN.host, + PGSSLMODE: "verify-full", + PGSSLROOTCERT: "/run/supabase/tls/root.crt", + PGSSLCERT: "/run/supabase/tls/client.crt", + PGSSLKEY: "/tmp/supabase-client.key", + PGCONNECT_TIMEOUT: "7", + }); + expect(docker.lastOpts?.cmd[2]).toContain( + "(umask 077 && cp /run/supabase/tls/client.key /tmp/supabase-client.key) || exit 1\n", + ); + expect(docker.lastOpts?.binds).toEqual([ + `${root}:/run/supabase/tls/root.crt:ro`, + `${cert}:/run/supabase/tls/client.crt:ro`, + `${key}:/run/supabase/tls/client.key:ro`, + ]); + expect(docker.lastOpts?.securityOpt).toEqual(["label:disable"]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("fails before pg_dump runs when a TLS file is missing", () => + Effect.gen(function* () { + const path = yield* Path.Path; + const missing = path.join(tmp.current, "missing.crt"); + const { layer, docker } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert: missing }, + isLocal: false, + }); + const exit = yield* dbDump(flags({ linked: Option.some(true) })).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(failMessage(exit)).toContain(`failed to read sslrootcert ${missing}`); + expect(docker.allOpts).toHaveLength(0); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("uses the image CA bundle without a host root cert", () => + Effect.forEach(["system", undefined], (sslrootcert) => { + const { layer, docker } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert }, + isLocal: false, + }); + return Effect.gen(function* () { + yield* dbDump(flags({ linked: Option.some(true) })); + expect(docker.lastOpts?.env["PGSSLROOTCERT"]).toBe("/etc/ssl/certs/ca-certificates.crt"); + expect(docker.lastOpts?.binds).toEqual([]); + expect(docker.lastOpts?.securityOpt).toEqual([]); + }).pipe(Effect.provide(layer)); + }), + ); + + it.effect("mounts the linked project's TLS files into a Windows tool container", () => + Effect.gen(function* () { + const root = yield* writePem("root.crt"); + const { layer, bundled } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert: root }, + isLocal: false, + platform: "win32", + stdout: "", + }); + yield* dbDump(flags({ linked: Option.some(true) })).pipe( + Effect.provide(Layer.mergeAll(layer, stackBackendLayer("stack"))), + ); + expect(bundled.lastOpts?.runtime).toEqual({ kind: "container", engine: "docker" }); + expect(bundled.lastOpts?.env).toMatchObject({ + PGSSLMODE: "verify-full", + PGSSLROOTCERT: "/run/supabase/tls/root.crt", + }); + expect(bundled.lastOpts?.mounts).toEqual([ + { source: root, target: "/run/supabase/tls/root.crt", readOnly: true }, + ]); + expect(bundled.lastOpts?.securityOpt).toEqual(["label:disable"]); + }).pipe(Effect.provide(BunServices.layer)), + ); + + it.effect("rejects a TLS file path that a Docker bind mount cannot carry", () => { + const { layer, docker } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert: "/certs/a:b/root.crt" }, + isLocal: false, + }); + return Effect.gen(function* () { + const exit = yield* dbDump(flags({ linked: Option.some(true) })).pipe(Effect.exit); + expect(failMessage(exit)).toBe( + "failed to read sslrootcert /certs/a:b/root.crt: a Docker bind mount cannot contain ':'", + ); + expect(docker.allOpts).toHaveLength(0); + }).pipe(Effect.provide(layer)); + }); + + const withHostCaBundles = (present: ReadonlyArray<string>) => + Layer.effect( + FileSystem.FileSystem, + Effect.map(FileSystem.FileSystem, (fs) => ({ + ...fs, + exists: (path: string) => + path.startsWith("/etc/") || path.startsWith("/home/user/") + ? Effect.succeed(present.includes(path)) + : fs.exists(path), + })), + ); + + it.effect("keeps host paths and falls back to the host's CA bundle for a native pg_dump", () => + Effect.forEach( + [ + { sslrootcert: "/certs/root.crt", present: [], expected: "/certs/root.crt" }, + { + sslrootcert: undefined, + present: ["/etc/ssl/certs/ca-certificates.crt"], + expected: "/etc/ssl/certs/ca-certificates.crt", + }, + { sslrootcert: undefined, present: [], expected: undefined }, + { + sslrootcert: undefined, + present: ["/home/user/.postgresql/root.crt", "/etc/ssl/cert.pem"], + expected: undefined, + }, + ], + ({ sslrootcert, present, expected }) => { + const { layer, bundled } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert }, + isLocal: false, + platform: "darwin", + stdout: "", + }); + return Effect.gen(function* () { + yield* dbDump(flags({ linked: Option.some(true) })).pipe( + Effect.provide(withHostCaBundles(present)), + ); + expect(bundled.lastOpts?.runtime).toEqual({ kind: "native" }); + expect(bundled.lastOpts?.env["PGSSLROOTCERT"]).toBe(expected); + expect(bundled.lastOpts?.mounts).toEqual([]); + }).pipe(Effect.provide(Layer.mergeAll(layer, stackBackendLayer("stack")))); + }, + ), + ); + + it.effect( + "follows OpenSSL's trust store overrides for sslrootcert=system in a native pg_dump", + () => { + const cases: ReadonlyArray<{ + sslrootcert: string | undefined; + env: Readonly<Record<string, string>>; + major?: 15; + expected: string; + }> = [ + { sslrootcert: "system", env: {}, expected: "/etc/ssl/cert.pem" }, + { sslrootcert: "system", env: { SSL_CERT_FILE: "/certs/ca.pem" }, expected: "system" }, + { sslrootcert: "system", env: { SSL_CERT_DIR: "/certs" }, expected: "system" }, + { + sslrootcert: "system", + env: { SSL_CERT_FILE: "/certs/ca.pem", SSL_CERT_DIR: "/certs" }, + expected: "system", + }, + { + sslrootcert: "system", + env: { SSL_CERT_FILE: "/certs/ca.pem" }, + major: 15, + expected: "/certs/ca.pem", + }, + { + sslrootcert: "system", + env: { SSL_CERT_DIR: "/certs" }, + major: 15, + expected: "/etc/ssl/cert.pem", + }, + { + sslrootcert: undefined, + env: { SSL_CERT_FILE: "/certs/ca.pem" }, + expected: "/etc/ssl/cert.pem", + }, + ]; + return Effect.forEach(cases, ({ sslrootcert, env, major, expected }) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + if (major !== undefined) { + yield* fs.makeDirectory(path.join(tmp.current, "supabase"), { recursive: true }); + yield* fs.writeFileString( + path.join(tmp.current, "supabase", "config.toml"), + `[db]\nmajor_version = ${major}\n`, + ); + } + const { layer, bundled } = setup({ + conn: { ...REMOTE_CONN, sslmode: "verify-full", sslrootcert }, + isLocal: false, + platform: "darwin", + stdout: "", + env, + ...(major === undefined ? {} : { workdir: tmp.current }), + }); + yield* dbDump(flags({ linked: Option.some(true) })).pipe( + Effect.provide( + withHostCaBundles(["/etc/ssl/cert.pem"]).pipe( + Layer.provideMerge(Layer.mergeAll(layer, stackBackendLayer("stack"))), + ), + ), + ); + expect(bundled.lastOpts?.pin.upstreamVersion.startsWith(String(major ?? 17))).toBe(true); + expect(bundled.lastOpts?.env).toMatchObject({ + PGSSLMODE: "verify-full", + PGSSLROOTCERT: expected, + }); + }).pipe(Effect.provide(BunServices.layer)), + ); + }, + ); + + it.effect("never hands TLS settings to a local pg_dump", () => { + const { layer, docker } = setup({ + conn: { ...LOCAL_CONN, sslmode: "verify-full", sslrootcert: "/certs/root.crt" }, + isLocal: true, + }); + return Effect.gen(function* () { + yield* dbDump(flags({ local: Option.some(true) })); + expect(docker.lastOpts?.env["PGSSLMODE"]).toBeUndefined(); + expect(docker.lastOpts?.env["PGSSLROOTCERT"]).toBeUndefined(); + expect(docker.lastOpts?.binds).toEqual([]); + }).pipe(Effect.provide(layer)); + }); + it.effect("caches the linked project even when connection resolution fails (Go PostRun)", () => { // The project ref is resolved before the connection is built, and the // linked-project cache is refreshed unconditionally afterward. So an @@ -1128,6 +1376,25 @@ describe("db dump integration", () => { }).pipe(Effect.provide(layer)); }); + it.effect("linked: keeps the pooler's TLS settings on the IPv4 retry", () => { + const { layer, docker } = setup({ + conn: REMOTE_CONN, + isLocal: false, + poolerFallback: Option.some({ ...POOLER_CONN, sslmode: "verify-full" }), + results: [ + { exitCode: 1, stderr: IPV6_STDERR }, + { exitCode: 0, stdout: "CREATE SCHEMA x;\n" }, + ], + }); + return Effect.gen(function* () { + yield* dbDump(flags()); + expect(docker.allOpts[1]?.env).toMatchObject({ + PGHOST: POOLER_CONN.host, + PGSSLMODE: "verify-full", + }); + }).pipe(Effect.provide(layer)); + }); + it.effect("linked: preserves the original dump error when the pooler fallback fails", () => { const { layer, resolver, docker } = setup({ conn: REMOTE_CONN, diff --git a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md index 4f3d112773..9155ce69a8 100644 --- a/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/pull/SIDE_EFFECTS.md @@ -143,7 +143,7 @@ at all, so nothing is cached for it. | `SUPABASE_SHADOW_CACHE` | shadow baseline cache; on by default, opt-out (`0`/`false`); the shadow's post-baseline state is saved under a managed snapshot key and restored into the next run's fresh stack database (see Notes) | no | | `SUPABASE_EXPERIMENTAL_PG_DELTA` | force pg-delta diff engine | no | | `SUPABASE_EXPERIMENTAL` | selects the deprecated in-process structured-dump export (same as `--declarative`) when `--declarative` is not set | no | -| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. The `pg_dump` and migra containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA outside its OOM fallback) | no | +| `PG*` libpq | linked-project TLS, timeout, service; see `db push`. The `pg_dump` seed gets them (see `db dump`); the migra container gets none of these `PG*` TLS settings (migra gets the CLI's own bundled CA outside its OOM fallback) | no | ## Exit Codes diff --git a/apps/cli/src/commands/db/pull/pull.integration.test.ts b/apps/cli/src/commands/db/pull/pull.integration.test.ts index ff2be7539e..249c6eb24f 100644 --- a/apps/cli/src/commands/db/pull/pull.integration.test.ts +++ b/apps/cli/src/commands/db/pull/pull.integration.test.ts @@ -196,6 +196,7 @@ interface SetupOpts { readonly historyUpdateFailWith?: string; // Runs the real resolver and linked-project cache; `workdir` must be a real directory. readonly realLayers?: boolean; + readonly remoteTls?: { readonly sslmode: string }; } function setup(workdir: string, opts: SetupOpts = {}) { @@ -424,6 +425,7 @@ function setup(workdir: string, opts: SetupOpts = {}) { user: "postgres", password: "x", database: "postgres", + ...opts.remoteTls, }, isLocal: connType === "local", ref: opts.resolvedRef !== undefined ? Option.some(opts.resolvedRef) : Option.none(), @@ -1257,6 +1259,20 @@ describe("db pull", () => { }, ); + it.effect("an initial pull hands the linked project's TLS settings to pg_dump", () => { + const s = setup(tmp.current, { + remoteVersions: [], + dumpStdout: "create table dumped ();\n", + edgeStdout: "create table diffed ();\n", + yes: true, + remoteTls: { sslmode: "verify-full" }, + }); + return Effect.gen(function* () { + yield* dbPull(flags()); + expect(s.dumpCalls[0]?.env).toMatchObject({ PGSSLMODE: "verify-full" }); + }).pipe(Effect.provide(s.layer)); + }); + it.effect("an initial pull in json mode emits a native structured envelope", () => { const s = setup(tmp.current, { format: "json", diff --git a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md index 5f066ba674..f62909c5c5 100644 --- a/apps/cli/src/commands/db/push/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/db/push/SIDE_EFFECTS.md @@ -145,6 +145,7 @@ Warning: supabase db push will target the local database by default in the next whenever `PGSERVICE` names a service. The pooler route also reads `.pgpass` (`PGPASSFILE`, else `~/.pgpass`, or `%APPDATA%\postgresql\pgpass.conf` on Windows) and discards that password, and it keeps `PGAPPNAME` and a service's `options` and runtime keys, which the direct host drops. - Tool containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA + `db dump` and the `pg_dump` seed of a first `db pull` also get them (see `db dump`). Other + tool containers get none of these `PG*` TLS settings (migra gets the CLI's own bundled CA outside its OOM fallback); migra and pgadmin get the timeout as `connect_timeout` and the pooler's `options` and runtime keys in their URL. diff --git a/apps/cli/src/commands/pull/SIDE_EFFECTS.md b/apps/cli/src/commands/pull/SIDE_EFFECTS.md index 51cb806093..f39632b727 100644 --- a/apps/cli/src/commands/pull/SIDE_EFFECTS.md +++ b/apps/cli/src/commands/pull/SIDE_EFFECTS.md @@ -138,7 +138,7 @@ Docker-dependent. | `SUPABASE_ACCESS_TOKEN` | auth token (bypasses credential file/keyring lookup) | no (falls back to keyring → `~/.supabase/access-token`) | | `SUPABASE_PROFILE` | API profile selection | no | | `env(VAR)` references | interpolated into `config.toml` values by the config step's loader | no | -| `PG*` libpq | db step: linked-project TLS, timeout, service; see `db push`. Containers get no `PG*` TLS settings | no | +| `PG*` libpq | db step: linked-project TLS, timeout, service; see `db push`. The `pg_dump` seed also gets them | no | Once the db and functions steps run, they consume their own established environment variables exactly as documented in their own `SIDE_EFFECTS.md` files (e.g. `SUPABASE_DB_PASSWORD`, From 531c926e4c971c770c635a52fb956181bb6fcad0 Mon Sep 17 00:00:00 2001 From: "supabase-cli-releaser[bot]" <246109035+supabase-cli-releaser[bot]@users.noreply.github.com> Date: Fri, 23 Oct 2026 08:20:52 +0000 Subject: [PATCH 23/24] chore(stack): bump postgres to 15.19.0.007-r0 (#7291) This bumps postgres from 15.19.0.006-r0 to 15.19.0.007-r0. https://github.com/supabase/slim-services/releases/tag/postgres-15.19.0.007-r0 Planned after supabase/slim-services published 17.11.0.007-r0. This branch is rewritten from develop whenever a newer relevant release arrives. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- apps/cli-go/pkg/config/templates/Dockerfile | 2 +- apps/cli/src/shared/services/Dockerfile | 2 +- packages/stack/src/Artifacts.ts | 20 ++++++++++---------- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/apps/cli-go/pkg/config/templates/Dockerfile b/apps/cli-go/pkg/config/templates/Dockerfile index 6141335710..74b0373f1d 100644 --- a/apps/cli-go/pkg/config/templates/Dockerfile +++ b/apps/cli-go/pkg/config/templates/Dockerfile @@ -7,7 +7,7 @@ # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. FROM supabase/postgres:17.11.0.006 AS pg -FROM supabase/postgres:15.19.0.006 AS pg15 +FROM supabase/postgres:15.19.0.007 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). FROM supabase/postgres:14.1.0.89 AS pg14 # New always-on service alias: extend SERVICE_IMAGE_ALIASES in diff --git a/apps/cli/src/shared/services/Dockerfile b/apps/cli/src/shared/services/Dockerfile index 6141335710..74b0373f1d 100644 --- a/apps/cli/src/shared/services/Dockerfile +++ b/apps/cli/src/shared/services/Dockerfile @@ -7,7 +7,7 @@ # adjacent lines, and catalog PRs for neighbouring aliases are often open at the same time. FROM supabase/postgres:17.11.0.006 AS pg -FROM supabase/postgres:15.19.0.006 AS pg15 +FROM supabase/postgres:15.19.0.007 AS pg15 # Postgres 14 has no slim build; bumped by hand (Dependabot ignores supabase/postgres). FROM supabase/postgres:14.1.0.89 AS pg14 # New always-on service alias: extend SERVICE_IMAGE_ALIASES in diff --git a/packages/stack/src/Artifacts.ts b/packages/stack/src/Artifacts.ts index 1f630dd144..1f18982d6d 100644 --- a/packages/stack/src/Artifacts.ts +++ b/packages/stack/src/Artifacts.ts @@ -157,24 +157,24 @@ const definitions: Readonly<Record<ArtifactKind, ArtifactDefinition>> = { "bin/psql", ], { - "15.19.0.006": { - upstreamVersion: "15.19.0.006", + "15.19.0.007": { + upstreamVersion: "15.19.0.007", revision: 0, image: - "ghcr.io/supabase/cli/postgres:15.19.0.006-r0@sha256:dd5d757785930d2daf216ccc2ea871e2337657d11c42f7cc709309774fa663f1", - upstreamImage: "supabase/postgres:15.19.0.006", + "ghcr.io/supabase/cli/postgres:15.19.0.007-r0@sha256:7698932a5a3878d8eba9b5cc8ce75ff29162744b7878b5892b33849c310437cf", + upstreamImage: "supabase/postgres:15.19.0.007", natives: { "darwin-arm64": { - archive: "0350c9ff12f84061f1a35de8218400fb8e6059dedd36d037bd01e9b7adde3e4b", - manifest: "a2d941b715b0fa09918353164f13fac47751973ab5ae4ceca978bb8263c41f62", + archive: "3c35ed10ac30ccd645eb684777d723bc7855d4fc5a39bacd7100337679a30a4d", + manifest: "a1f11b1d68c83d4905c48ad9851eaef7a68584f4dd3df1d520f107d5e23c3595", }, "linux-amd64": { - archive: "3e0126f154c174a88c45a9fa496441e1a15e2f328e4ad3b61fcc02c9caead69c", - manifest: "af86fad4f63a9bafbc853e8291275d95762da609e9775129f7debbdcdd35c1bb", + archive: "61f1f74855e52bcabe95db07c70ac54a086b5fbd113c007810a0bee2ad072051", + manifest: "64c70c66bc3b71324727a059921074e0722284a4cbb077914d71a22d513b4e55", }, "linux-arm64": { - archive: "ec1f08c01a110aa267c646a7ab69dc63e2e544c5d9d0c0536cfb974423280b3a", - manifest: "7527e460894f9ef76c57ec03359db5e8cc4d8fdb7043f727f4213344d9f3e76c", + archive: "5b45fb61144b04154f3568dcedae941e765b6da32b7a1ea7d0c5ae6becadc048", + manifest: "fed2eac98a385378a3c9c4e94dd03d05d2f5acff4900e2071649f98164807aed", }, }, }, From c3d6a6de854c9604c1b0516d1280080c7f68c9f2 Mon Sep 17 00:00:00 2001 From: avallete <andrew.valleteau@supabase.io> Date: Fri, 23 Oct 2026 12:56:55 +0200 Subject: [PATCH 24/24] test(cli): drop pooler URL from db toml precedence goldens The merged resolver reads the pooler URL from the target, not the TOML reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- .../testdata/config-precedence/db-toml/a-base-only.json | 1 - .../testdata/config-precedence/db-toml/b-matched-remote.json | 1 - .../config-precedence/db-toml/b2-remote-block-without-ref.json | 1 - .../testdata/config-precedence/db-toml/c-shell-env.json | 1 - .../testdata/config-precedence/db-toml/d-project-dotenv.json | 1 - .../config-precedence/db-toml/d2-shell-beats-dotenv.json | 1 - .../testdata/config-precedence/db-toml/e-remote-beats-shell.json | 1 - .../config-precedence/db-toml/e2-remote-beats-dotenv.json | 1 - .../db-toml/f-remote-matched-by-env-project-id.json | 1 - .../db-toml/f2-remote-env-match-project-id-is-block-literal.json | 1 - .../db-toml/g-captcha-secret-from-env-only.json | 1 - .../db-toml/h-storage-realtime-env-disable.json | 1 - .../db-toml/i-project-id-env-with-matched-remote.json | 1 - .../db-toml/i2-project-id-env-without-remote.json | 1 - 14 files changed, 14 deletions(-) diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json index 55cf899a7b..3972c220f2 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/a-base-only.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json index 7ad4bb8138..6fccaca7c5 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b-matched-remote.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "abcdefghijklmnopqrst", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json index 55cf899a7b..3972c220f2 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/b2-remote-block-without-ref.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json index c9beabc890..8bdfd08325 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/c-shell-env.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 56001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json index 718f334759..35a5fa5f17 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d-project-dotenv.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 57001, "projectEnv": { "SUPABASE_API_SCHEMAS": "public,dotenv_api", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json index c9beabc890..8bdfd08325 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/d2-shell-beats-dotenv.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 56001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json index 05857cdf83..f15845d793 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e-remote-beats-shell.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 56001, "projectEnv": {}, "projectId": "abcdefghijklmnopqrst", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json index b92d9dde59..b7a808c7b7 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/e2-remote-beats-dotenv.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 57001, "projectEnv": { "SUPABASE_API_SCHEMAS": "public,dotenv_api", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json index 3ad1b199d6..04064981ef 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f-remote-matched-by-env-project-id.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "zyxwvutsrqponmlkjihg", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json index 3ad1b199d6..04064981ef 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/f2-remote-env-match-project-id-is-block-literal.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "zyxwvutsrqponmlkjihg", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json index 55cf899a7b..3972c220f2 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/g-captcha-secret-from-env-only.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json index acbdf90332..3458b23c6b 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/h-storage-realtime-env-disable.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "golden-base", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json index b579052d57..edcceaa606 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i-project-id-env-with-matched-remote.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "shell-project-id", diff --git a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json index afff4284fc..dc1a4610b5 100644 --- a/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json +++ b/apps/cli/src/command-internal/testdata/config-precedence/db-toml/i2-project-id-env-without-remote.json @@ -22,7 +22,6 @@ "enabled": false, "formatOptions": null }, - "poolerConnectionString": null, "port": 55001, "projectEnv": {}, "projectId": "shell-project-id",