feat(core): reclaim deleted event aggregates

.specgit.yaml

@@ -1,8 +1,8 @@
 version: 1
-delivery: macos-integrity-boundary
+delivery: event-retention-reclamation
 context:
   kind: branch
-  branch: feat/498-macos-integrity-boundary
+  branch: feat/524-event-retention-reclamation
 issues:
-  - 498
-pr: 536
+  - 524
+pr: 537

docs/adr/0001-event-residue-scrub-and-sqlite-auto-vacuum-reclamation.md

@@ -0,0 +1,118 @@
+# ADR 0001: Event residue scrub and SQLite auto-vacuum reclamation
+
+- **Status:** Accepted
+- **Date:** 2026-09-04
+- **Issue:** #524 (delivery: PR #537 → `dev`)
+- **Supersedes:** none
+
+## Context
+
+The durable event store had no reclamation semantics. `Event.remove(aggregateID)` ran only on
+explicit session removal and covered the session aggregate alone, and SQLite ran without
+`auto_vacuum`, so deleted rows never returned pages to the filesystem. Two failure shapes
+motivated the decision:
+
+- A crash between the session-row delete and any cleanup stranded durable event aggregates
+  whose `SessionTable` and `WorkflowTable` read models were both gone. Replaying such an
+  aggregate is impossible: the `WorkflowCreated` projector INSERT dies on the
+  `workflow.session_id` foreign key once the session row is gone (pinned by
+  `packages/opencode/test/dag/dag-replay-idempotency.test.ts`), so residue can neither be
+  replayed nor re-materialized — it can only be deleted.
+- `database.ts` initialized WAL and pragmas after driver open, so an application-level
+  `PRAGMA auto_vacuum` could not take effect: after WAL initialization the pragma silently
+  yields NONE even on an empty database.
+
+The decision checkpoint was approved on 2026-09-04 with the scope locked below.
+
+## Decisions
+
+1. **Explicit session + dag scrub.** `Session.remove` captures every related dag aggregate ID
+   before the `Deleted` publish (the projector's session-row delete FK-cascades the workflow
+   rows inside the publish transaction, so a post-publish lookup would see nothing) and removes
+   each dag event aggregate after the session aggregate — terminal workflows included. The
+   per-dag scrub is soft-degrading (a failure is logged and the aggregate is left for the
+   startup sweep) but preserves interruption (`Cause.hasInterrupts` re-raise, the
+   `EventResidueSweep` sibling discipline).
+2. **Guarded default-on startup sweep.** `EventResidueSweep` runs one pass per process start,
+   forked into the layer scope so it can neither block nor fail startup. Eligibility is the
+   zero-live-read-model rule: an aggregate in `event_sequence` with neither a `session` nor a
+   `workflow` row. Removal is a single atomic guarded `DELETE` that re-evaluates both
+   NOT EXISTS guards inside the statement (no select-then-delete TOCTOU window), so an
+   aggregate recreated concurrently survives. Wired into `AppLayer` and the HttpApiApp node
+   graph, so every serving process sweeps; the pass is idempotent.
+3. **New databases: FULL before WAL.** Both SQLite drivers (`sqlite.bun.ts`, `sqlite.node.ts`)
+   set `auto_vacuum=FULL` at the driver layer, before `journal_mode=WAL`, and only on a
+   genuinely empty (0-page) file. An immediate SQLITE_BUSY from a second opener racing the
+   first is tolerated: the pragma is a persistent header property and runs before any
+   WAL/migration write, so the first-write winner sets FULL for the database.
+4. **Existing databases: explicit conversion only.** Legacy `auto_vacuum=NONE` databases are
+   never converted at startup — startup is detect-only (a warning pointing at the command). The
+   only conversion path is `opencode db vacuum --db <path>`: the target must be named
+   explicitly and must already exist as a regular file (vacuum never creates a database), runs
+   FULL → VACUUM → `wal_checkpoint(TRUNCATE)` outside any startup path, and fails nonzero
+   unless the `PRAGMA auto_vacuum` readback is exactly FULL. Exclusive access is a hard
+   requirement (a concurrent writer fails VACUUM with SQLITE_BUSY).
+5. **Archived-session retention: off and deferred.** No retention policy for archived sessions
+   ships in this decision (Phase 3).
+6. **Active truncation: rejected.** Truncating active/retained session event history and event
+   snapshot folding are rejected; incremental replay (`seq > after`, ascending) and sync
+   cursors must keep observing unbroken per-aggregate histories.
+7. **`incremental_vacuum` is forbidden.** A disposable bun:sqlite prototype reproduced an
+   exit-139 crash under the incremental mode; no code path may enable it.
+
+## Consequences and risks
+
+- Deleting events on legacy NONE databases still does not shrink the file until an operator
+  runs the explicit conversion; disk usage grows until then.
+- `auto_vacuum=FULL` pays its known SQLite overhead (pointer-map pages, per-update mapping) on
+  every new database in exchange for automatic page reclamation.
+- The sweep runs once per process start: residue created and abandoned within a single process
+  lifetime waits for the next start. This is accepted because the shapes it targets are
+  crash/in-flight zombies.
+- The conversion command requires exclusive access; the error guidance says to close running
+  opencode processes and retry.
+- Replay and sync contracts are preserved by construction: only whole aggregates with no live
+  read model are ever removed, and such aggregates are unreplayable anyway (FK death), so no
+  consumer can observe the removal as a gap in a replayable history.
+
+## Alternatives considered
+
+- **Rely on replay instead of scrubbing** — rejected: a wiped dag aggregate whose session row
+  is gone dies on the workflow foreign key during re-materialization, so replay cannot replace
+  deletion.
+- **Silent startup conversion of legacy databases** — rejected: converting requires a blocking
+  full VACUUM; startup stays non-blocking and detect-only.
+- **`PRAGMA incremental_vacuum`** — rejected (decision 7).
+- **A recurring background reaper** — rejected in favor of one idempotent guarded pass per
+  process start; residue is crash-shaped, not steady-state throughput.
+- **Truncate or fold active event histories** — rejected (decision 6).
+
+## Rollout and rollback
+
+Rollout lands as ordinary PRs through `dev` per the release train; no operator action is
+required — new databases get FULL automatically, legacy databases keep working unchanged (with
+a detect-only warning), and the sweep is default-on. Rollback is removing the sweep from the
+app graphs and reverting the driver pragma: the sweep is additive and idempotent, and legacy
+databases were never written by any of this. A database created with FULL keeps its header
+mode; reverting one is itself an explicit operator VACUUM and is not automated.
+
+## Acceptance
+
+- Active/retained session replay is unchanged; only zero-live-read-model aggregates are
+  removed (guarded delete re-checked inside the statement).
+- Cleanup failures never block the application path; interruption is preserved, not logged as
+  failure.
+- Disposable-file tests demonstrate page reclamation and the new/existing database behavior;
+  no startup-time full VACUUM exists.
+- `bun run test:dag-core`, focused event/session tests, package typecheck, and migration
+  freshness checks pass in CI.
+
+## Non-goals
+
+- **No global bounded-retention claim.** Live and retained sessions keep their full event
+  history indefinitely; this decision bounds nothing by age, size, or count.
+- **No tombstones, unarchive, or sync changes.** Offline deletion tombstones, unarchive
+  semantics, and sync cursor/protocol changes stay out of scope.
+- **No authorization for #531 or live-database work.** This decision does not authorize running
+  VACUUM or any cleanup against a live local database; the destructive operator procedure
+  remains the human-only issue #531.

packages/core/src/database/database.ts

@@ -2,7 +2,8 @@ export * as Database from "./database"
 
 import { EffectDrizzleSqlite } from "@opencode-ai/effect-drizzle-sqlite"
 import { layer as sqliteLayer } from "#sqlite"
-import { Context, Effect, Layer } from "effect"
+import { Cause, Context, Effect, Layer } from "effect"
+import { sql } from "drizzle-orm"
 import { Global } from "../global"
 import { Flag } from "../flag/flag"
 import { isAbsolute, join } from "path"
@@ -29,6 +30,27 @@ export const layer = Layer.effect(
     yield* db.run("PRAGMA busy_timeout = 5000")
     yield* db.run("PRAGMA cache_size = -64000")
     yield* db.run("PRAGMA foreign_keys = ON")
+    // #524: genuinely new databases were switched to auto_vacuum=FULL by the
+    // sqlite driver BEFORE WAL init. A legacy database keeps its NONE mode —
+    // converting one silently at startup would need a blocking full VACUUM —
+    // so it is only detected and surfaced softly here; conversion is the
+    // explicit user-triggered `opencode db vacuum --db <path>` command.
+    // Detect-only means detect-only: a failed readback degrades to a warning
+    // (the layer body is orDie'd, so an unhandled failure would kill startup),
+    // while an interruption is always re-raised.
+    const autoVacuum = yield* db.get<{ auto_vacuum: number }>(sql`PRAGMA auto_vacuum`).pipe(
+      Effect.catchCause((cause) =>
+        Cause.hasInterrupts(cause)
+          ? Effect.interrupt
+          : Effect.logWarning("database auto_vacuum readback failed — skipping the detect-only check", { cause }).pipe(
+              Effect.as(undefined),
+            ),
+      ),
+    )
+    if (autoVacuum?.auto_vacuum === 0)
+      yield* Effect.logWarning(
+        "database auto_vacuum is NONE — deleted pages stay allocated until converted; run `opencode db vacuum --db <path>` (prints its path with `opencode db path`)",
+      )
     yield* db.run("PRAGMA wal_checkpoint(PASSIVE)")
     yield* DatabaseMigration.apply(db)
 

packages/core/src/database/sqlite.bun.ts

@@ -161,11 +161,40 @@ const nativeLayer = (config: Config) =>
         create: config.create ?? true,
       })
       yield* Effect.addFinalizer(() => Effect.sync(() => native.close()))
+      // #524: auto_vacuum must be set BEFORE any WAL initialization — after
+      // WAL init the pragma silently yields NONE even on an empty database.
+      // Only a genuinely empty (0-page) file is eligible: on any existing
+      // database the pragma would be a no-op at best, so legacy NONE
+      // databases are never written here (startup stays detect-only; the
+      // explicit conversion lives in ./vacuum).
+      native.run("PRAGMA busy_timeout = 5000;")
+      if (config.readonly !== true) setAutoVacuumFull(native)
       if (config.disableWAL !== true) native.run("PRAGMA journal_mode = WAL;")
       return native
     }),
   )
 
+/**
+ * Setting auto_vacuum needs the write lock of a read-header-then-write
+ * upgrade, which SQLite fails with an immediate SQLITE_BUSY the busy handler
+ * cannot retry — a second opener racing the first one's initialization on the
+ * same new file hits it. Skipping on BUSY is safe: auto_vacuum is a
+ * persistent header property and every opener runs this pragma BEFORE any
+ * WAL/migration write, so whichever connection wins the first-write race sets
+ * FULL for the database.
+ */
+function setAutoVacuumFull(native: Database) {
+  const page = native.query<{ page_count: number }, []>("PRAGMA page_count").get()
+  if (!page || page.page_count !== 0) return
+  try {
+    native.run("PRAGMA auto_vacuum = FULL;")
+  } catch (cause) {
+    if (!isSqliteBusy(cause)) throw cause
+  }
+}
+
+const isSqliteBusy = (cause: unknown) => cause instanceof Error && /SQLITE_BUSY|database is locked/i.test(cause.message)
+
 const sqliteLayer = (config: Config) => Layer.effect(Client.SqlClient, make(config))
 
 const drizzleLayer = Layer.effect(

packages/core/src/database/sqlite.node.ts

@@ -156,11 +156,40 @@ const nativeLayer = (config: Config) =>
         open: true,
       })
       yield* Effect.addFinalizer(() => Effect.sync(() => native.close()))
+      // #524: auto_vacuum must be set BEFORE any WAL initialization — after
+      // WAL init the pragma silently yields NONE even on an empty database.
+      // On an existing non-empty database the pragma is a SQLite no-op, so
+      // legacy NONE databases are never converted here (startup stays
+      // detect-only; the explicit conversion lives in ./vacuum).
+      native.exec("PRAGMA busy_timeout = 5000;")
+      if (config.readonly !== true) setAutoVacuumFull(native)
       if (config.disableWAL !== true && config.readonly !== true) native.exec("PRAGMA journal_mode = WAL;")
       return native
     }),
   )
 
+/**
+ * Setting auto_vacuum needs the write lock of a read-header-then-write
+ * upgrade, which SQLite fails with an immediate SQLITE_BUSY the busy handler
+ * cannot retry — a second opener racing the first one's initialization on the
+ * same new file hits it. Skipping on BUSY is safe: auto_vacuum is a
+ * persistent header property and every opener runs this pragma BEFORE any
+ * WAL/migration write, so whichever connection wins the first-write race sets
+ * FULL for the database.
+ */
+function setAutoVacuumFull(native: DatabaseSync) {
+  const page: unknown = native.prepare("PRAGMA page_count").get()
+  const pageCount = typeof page === "object" && page !== null && "page_count" in page ? page.page_count : undefined
+  if (typeof pageCount !== "number" || pageCount !== 0) return
+  try {
+    native.exec("PRAGMA auto_vacuum = FULL;")
+  } catch (cause) {
+    if (!isSqliteBusy(cause)) throw cause
+  }
+}
+
+const isSqliteBusy = (cause: unknown) => cause instanceof Error && /SQLITE_BUSY|database is locked/i.test(cause.message)
+
 const sqliteLayer = (config: Config) => Layer.effect(Client.SqlClient, make(config))
 
 const drizzleLayer = Layer.effect(

packages/core/src/database/vacuum.ts

@@ -0,0 +1,97 @@
+// SPDX-FileCopyrightText: 2026 LeXwDeX
+// SPDX-License-Identifier: AGPL-3.0-or-later
+
+export * as Vacuum from "./vacuum"
+
+import { EffectDrizzleSqlite } from "@opencode-ai/effect-drizzle-sqlite"
+import { NodeFileSystem } from "@effect/platform-node"
+import { Effect, FileSystem, Schema } from "effect"
+import { sql } from "drizzle-orm"
+import { layer } from "#sqlite"
+
+const makeDb = EffectDrizzleSqlite.makeWithDefaults()
+
+export interface ConvertResult {
+  /** Readback of `PRAGMA auto_vacuum` after conversion: 1 == FULL. */
+  readonly autoVacuum: number
+}
+
+export class Refused extends Schema.TaggedErrorClass<Refused>()("VacuumRefused", {
+  filename: Schema.String,
+  reason: Schema.String,
+}) {
+  override get message() {
+    return `refusing to vacuum ${this.filename}: ${this.reason}`
+  }
+}
+
+export class NotFull extends Schema.TaggedErrorClass<NotFull>()("VacuumNotFull", {
+  filename: Schema.String,
+  // -1 encodes an unreadable readback (the pragma returned no row).
+  autoVacuum: Schema.Number,
+}) {
+  override get message() {
+    return `vacuum did not take full effect for ${this.filename}: PRAGMA auto_vacuum reads back ${this.autoVacuum}, expected 1 (FULL) — close running opencode processes that use this file and retry`
+  }
+}
+
+/**
+ * #524 Phase 2 gate and the only success path of `convertToFull`: after the
+ * FULL -> VACUUM -> wal_checkpoint(TRUNCATE) sequence the readback must be
+ * exactly FULL (1), otherwise the conversion silently failed (e.g. a writer
+ * kept the file alive through VACUUM) and must surface as a nonzero failure
+ * with actionable diagnostics — never as a success result. Exported as the
+ * deterministic seam that lets tests prove a non-FULL readback cannot report
+ * success.
+ */
+export const verifyFull = (filename: string, readback: number | undefined): Effect.Effect<ConvertResult, NotFull> =>
+  readback === 1
+    ? Effect.succeed({ autoVacuum: readback })
+    : Effect.fail(new NotFull({ filename, autoVacuum: readback ?? -1 }))
+
+// #524: refuse every target that is not an existing regular file BEFORE any
+// SQLite open — the driver opens with create enabled, so a typo'd path would
+// otherwise silently materialize a fresh empty database.
+const validateTarget = Effect.fn("Vacuum.validateTarget")(function* (fs: FileSystem.FileSystem, filename: string) {
+  if (filename === ":memory:") yield* new Refused({ filename, reason: ":memory: is not a file on disk" })
+  const info = yield* fs.stat(filename).pipe(Effect.catch(() => Effect.void))
+  if (info === undefined) {
+    yield* new Refused({
+      filename,
+      reason: "no such file — vacuum never creates a database (print the default path with `opencode db path`)",
+    })
+    return
+  }
+  if (info.type !== "File") yield* new Refused({ filename, reason: `not a regular file (${info.type})` })
+})
+
+const convert = (filename: string) =>
+  Effect.gen(function* () {
+    const db = yield* makeDb
+    yield* db.run(sql`PRAGMA busy_timeout = 5000`)
+    yield* db.run(sql`PRAGMA auto_vacuum = FULL`)
+    yield* db.run(sql`VACUUM`)
+    yield* db.run(sql`PRAGMA wal_checkpoint(TRUNCATE)`)
+    const mode = yield* db.get<{ auto_vacuum: number }>(sql`PRAGMA auto_vacuum`)
+    return mode?.auto_vacuum
+  }).pipe(Effect.provide(layer({ filename })))
+
+/**
+ * #524 Phase 2: explicit, user-triggered conversion of a legacy
+ * auto_vacuum=NONE database to FULL. Runs OUTSIDE startup and never touches a
+ * default database path implicitly — the caller names the file (the CLI
+ * surface requires an explicit `--db`, so tests only ever pass disposable
+ * temp paths), and the target must already exist as a regular file: vacuum
+ * never creates a database. The FULL → VACUUM → wal_checkpoint(TRUNCATE)
+ * sequence rebuilds the database with FULL enabled and truncates the WAL; a
+ * concurrent writer makes VACUUM fail with SQLITE_BUSY instead of corrupting
+ * anything, and a non-FULL readback fails via `verifyFull`. Incremental
+ * auto-vacuum is deliberately never used anywhere.
+ */
+export const convertToFull = (filename: string) =>
+  Effect.gen(function* () {
+    const fs = yield* FileSystem.FileSystem
+    yield* validateTarget(fs, filename)
+    const readback = yield* convert(filename)
+    return yield* verifyFull(filename, readback)
+  }).pipe(Effect.provide(NodeFileSystem.layer))