FEA-1550: Cut Agent Dashboard over to PGlite by mikeangstadt · Pull Request #275 · closedloop-ai/closedloop-electron

mikeangstadt · 2026-06-06T12:49:43Z

Summary

Replaces the ClosedLoop-owned Agent Dashboard SQLite store with a fresh PGlite data directory at agent-dashboard.pgdata; agent-dashboard.sqlite is intentionally not read, copied, renamed, or migrated.
Removes the generated Agent Monitor sidecar/vendor tree, sidecar build process, third-party dashboard package wiring, and SQLite dashboard repository code.
Keeps Agent Dashboard behind the existing dashboard feature flag, with the design-system dashboard path enabled automatically when that feature is on.
Moves dashboard-backed sessions, metered usage, packs, skills, tools, subagents, plans, pull requests, workflow data, and OpenCode ingestion onto the in-process PGlite runtime.
Keeps OpenCode ingestion by reading OpenCode's own external opencode.db; SQLite is not used as a ClosedLoop app database.
Removes sandbox filtering from dashboard ingest and cloud sync so all local sessions can import/sync, regardless of working directory.
Sanitizes every cloud-synced agent-session payload and makes oversized-payload chunking always-on.
Fixes PGlite/PostgreSQL strict GROUP BY workflow queries by grouping on subtype columns and aggregating fallback names.
Fixes large token analytics by avoiding PostgreSQL int casts and using bigint token arithmetic for aggregate totals.
Aligns the dashboard shell, metric cards, tables, form controls, empty/loading states, and visualization containers on design-system primitives wherever possible.
Ports deleted SQLite/sidecar tests that still apply to first-party PGlite behavior, including token reconciliation, pagination, lifecycle/import, parser, catchup-cache, identity, and sync edge cases.
Bumps the desktop package minor version to 0.16.0.

Notes

First start with this build creates an empty PGlite database and lets hooks/collectors repopulate it from live and file-based scrapers.
PGlite ingest state uses a fresh namespace so legacy sidecar/SQLite catchup caches cannot suppress first-start refill.
Packs remain core dashboard functionality; their data now comes from the PGlite-backed dashboard APIs.
Local Electron runs from apps/desktop/dist/main; after the bigint analytics fix, rebuild/restart the app so the stale ::int analytics query is not still loaded from old dist output.
Deleted tests tied only to generated sidecar/vendor wiring, ClosedLoop-owned SQLite contention, sandbox-only sync filtering, or removed third-party UI code remain intentionally deleted.

Testing

pnpm -C apps/desktop build
pnpm -C apps/desktop typecheck
pnpm -C apps/desktop lint
./apps/desktop/node_modules/.bin/tsx --test apps/desktop/test/pglite-agent-dashboard-database.test.ts
./apps/desktop/node_modules/.bin/tsx --test apps/desktop/test/pglite-agent-dashboard-database.test.ts apps/desktop/test/agent-session-sync-service.test.ts apps/desktop/test/agent-session-sync-sanitization.test.ts apps/desktop/test/collectors-parsers.test.ts apps/desktop/test/ingest-orchestrator.test.ts apps/desktop/test/reconciliation-worker.test.ts apps/desktop/test/feature-flags.test.ts apps/desktop/test/agent-dashboard-boundary.test.ts apps/desktop/test/agent-monitor-listener.test.ts

Risks

Existing agent-dashboard.sqlite data is not migrated by design; the new PGlite DB starts empty and repopulates from scrapers.
The full desktop test suite was not rerun after the final legacy sidecar/SQLite cleanup; the validation above covers the touched dashboard, ingest, sync, parser, feature-flag, boundary, and reconciliation paths.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc4e0e7bc4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

shafty023 · 2026-06-07T03:17:04Z

Several follow-up replies say fixes landed in the latest push, but the PR head is still 56a7efb21378ec76094831a3d2be307bed953088 and refs/pull/275/head points to the same SHA. At that head, the previously commented code paths still show the unfixed implementation: startup still calls prepareAgentDashboardDatabaseStartup({ backend: "sqlite" }), migration still uses SELECT * ... .all(), and backup cleanup still uses recursive removal. Please push the commit containing the referenced fixes or update the threads to point to the correct commit before resolving them.

- Add a recoverable SQLite to PGlite migration layer for agent dashboard data - Wire dashboard startup through database preparation before backend handoff - Copy supported tables with row-count verification and .bak retention handling - Cover successful migration, fallback, skip, startup, and backup cleanup paths - Bump desktop minor version to 0.16.0 for the database technology transition Testing: Full desktop test suite, typecheck, lint, and focused migration tests passed Risks: Migration currently covers the in-process Agent Dashboard tables present in this repo

- Restrict backup cleanup to exact naming pattern, regular files only, no recursive delete - Stamp backup mtime with migration time instead of preserving SQLite mtime - Stage PGlite in temp directory, promote only after backup succeeds - Sanitize absolute paths and error stacks from logs - Batch inserts (500 rows) instead of per-row INSERT - Validate all SQLite tables are managed before migration - Require .bak + .pgdata for already_migrated; detect downgrade scenario - Wire migration into startup with keepSource, non-blocking background execution - Pass startup result to runtime for PGlite readiness signaling Testing: Full migration test suite (9 tests), typecheck, and lint pass

- Preserve live SQLite when prior backup and PGlite data already exist. - Copy SQLite source rows in bounded batches instead of materializing entire tables. - Sanitize the remaining SQLite path in migration failure logs. - Add regression coverage for recreated SQLite, multi-batch event copy, and sanitized logs. Testing: Focused SQLite-to-PGlite migration tests, desktop typecheck, desktop lint, desktop build, and full desktop test suite passed. Risks: PGlite remains a prepared background copy while SQLite is the active runtime until async stores cut over.

- Replace the design-system Agent Dashboard SQLite runtime with a fresh PGlite data directory - Remove the SQLite-to-PGlite migration startup path and migration tests - Add async PGlite-backed dashboard stores, hook writes, collector imports, session sync, and reconciliation reads - Cover fresh PGlite startup and first hook ingestion Testing: Desktop typecheck, lint, focused PGlite dashboard tests, sync tests, and boot-recovery test passed; full desktop suite progressed through touched coverage but was interrupted after hanging in boot-recovery, which passed in isolation Risks: Existing agent-dashboard.sqlite data is intentionally not migrated; the new PGlite database starts empty and repopulates from hooks and collectors after first start

- Remove the generated Agent Monitor sidecar, third-party dashboard package wiring, and ClosedLoop-owned SQLite repository. - Keep Agent Dashboard on the PGlite runtime with fresh ingest state, ungated session import/sync, always-on chunked sync, and sanitized payloads. - Add PGlite coverage for dashboard parity, metered usage rows, and sync sanitization while retaining OpenCode ingestion from its external database. Testing: Desktop typecheck, lint, and focused dashboard/sync/reconciliation/parser tests passed. Risks: Full desktop test suite was not rerun after removing the legacy sidecar and SQLite-only tests.

github-actions · 2026-06-08T05:46:34Z

Compatibility Smoke Test Results ⚠️

Status: skipped
Electron SHA: 39c121c2afd058a984285720ceaa738fbf82492d
Symphony Alpha SHA (last-known-good): 24741ed9be45cc761195d7a7b6613bd30cdfce84
Note: Skipped because the stage GitHub App credentials are not configured for this workflow run.

View Actions run

- Use bigint token arithmetic in PGlite analytics and session totals, and carry hook identity fields into sync payloads. - Align dashboard pages, tables, controls, and visualizations on design-system primitives. - Port deleted SQLite/sidecar coverage that still maps to first-party PGlite behavior, including token reconciliation, pagination, lifecycle/import, parser, catchup-cache, and sync edge cases. Testing: Desktop typecheck, lint, build, focused PGlite dashboard tests, and expanded dashboard/PGlite/sync/parser/reconciliation/boundary tests passed. Risks: Full desktop test suite was not rerun; generated sidecar/vendor-only tests remain intentionally deleted because the sidecar path was removed.

- Add a PGlite-era boot probe for the disabled Agent Dashboard flag.\n- Wire the CI-invoked boot-off package script back into desktop package scripts.\n\nTesting:\nFull desktop test suite, typecheck, lint, and CI boot-off checks passed.\n\nRisks:\nLow; test-only script wiring and boot assertion coverage.

- Force the boot-off probe child process to exit on both success and failure.\n- Add a parent-side timeout so CI reports probe failures instead of hanging.\n\nTesting:\nTargeted boot-off script passed locally.\n\nRisks:\nLow; test-only probe control flow.

- Restore old_string, new_string, patch, arguments to STRIPPED_LEAF_KEYS to prevent leaking sensitive file diffs and shell args to cloud relay - Replace LIKE with ILIKE in PGlite session search for case-insensitive parity with the old SQLite behavior - Make runtime close() async and await it in both shutdown paths to prevent data loss from fire-and-forgotten PGlite close - Update apps/desktop/CLAUDE.md security model to document intentional sandbox gate removal (FEA-1550 ungated local import/sync) - Add agentSessionChunkedSyncEnabled cleanup in settings-store migration - Add TODO(FEA-1550) comment to settings migration blocks for future removal tracking - Wire getUserIdentity through runtime options to openPgliteAgentDatabase - Fix sweepStaleSessions to use injected now parameter instead of Date.now() for deterministic stale session cutoff - Expand sanitization test to cover all 12 stripped keys Testing: typecheck (no new errors), lint clean on changed files, agent-session-sync-sanitization tests pass (2/2), pglite-agent-dashboard-database tests pass (10/10) Risks: close() is now awaited in shutdown — if PGlite.close() hangs, it could delay app exit (mitigated by existing shutdown timeout). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Use design-system Tabs, Switch, Input, Checkbox, Badge, and Button primitives across active renderer screens. - Remove the obsolete monolithic renderer HTML and stop staging it for packaging. Testing: Desktop typecheck, renderer build, focused renderer boundary tests, staging script syntax check, and raw control scan passed. Risks: Low; visual controls now come from the shared design system, and the removed HTML was no longer loaded by the main window.

thadeusb

Massive cleanup, the sidecar removal alone is worth celebrating. A few things inline on the new PGlite paths, biggest one being boot can crash if PGlite init fails.

- Degrade Agent Monitor startup when PGlite runtime init fails instead of crashing desktop boot. - Drain queued PGlite writes before close and cap live hook event data at the existing 64KB import limit. - Strip cloud-synced event summaries and update sanitization coverage. - Refresh stale Agent Monitor docs and delete dead pull request E2E audit/contract files. Testing: Full desktop typecheck, lint, focused PGlite and sync sanitization tests, and full desktop test suite passed. Risks: Low; cloud sync now omits event summaries, which reduces relay detail but avoids leaking raw tool error text.

…Glite app - Add 10 new PGlite tables: pack_catalog, pack_catalog_history, pack_install_runs, agent_packs, skills, project_pack_associations, plans, plan_versions, pull_requests, pr_backfill_seen - Port catalog-store (seed, list, fetch result, install runs) from CJS/SQLite to TypeScript/PGlite with jsonb columns and positional params - Port pack-store (upsert, list, skills, usage, sessions) with PG-native json operators replacing json_extract, string_agg replacing GROUP_CONCAT - Port pack-scanner (filesystem discovery of 5 harness skill roots) and 6 catalog detector adapters (RTK, SuperClaude, voltagent, etc.) - Port install-orchestrator with security-hardened child env, 10min timeout, concurrency guard, and IPC streaming replacing SSE - Port catalog-fetcher (GitHub API stats with gh CLI + REST fallback) and catalog-contents (7 content type scrapers with 7-day TTL) - Port plan-store with extraction from hook events (ExitPlanMode, file writes, Codex plans) and startup backfill from ~/.claude/plans/ - Port pr-store with command-gated detection of gh pr create and startup transcript backfill with mtime caching - Add 25+ IPC handlers for catalog, packs, plans, and PR features - Add preload bridge wrappers and shared DTO types for all new features - Build full React UI: PacksCatalog with install/uninstall cards, InstallModal with streaming output, Sparkline, PlansView with version history and confirm/reject, PullRequestsView with stat pills - Wire startup lifecycle: catalog seed → pack scan → GitHub fetch - Schedule 24h recurring catalog fetch with cleanup on close - Expose constrained storeDb accessor (PgliteExecutor, not raw client) Testing: typecheck clean, lint clean, 26/26 tests pass Risks: First run creates empty catalog/inventory tables; data populates from filesystem scan and GitHub fetch on next boot cycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Import design-system globals.css before local globals.css so DS styles are the base layer and local overrides apply on top - Replace arbitrary size-[18px] span wrapper with standard size-4 class directly on SVG elements, matching the design system's own icon sizing convention and ensuring Tailwind generates the class Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@theme

…er CSS - Add fade-in, slide-up, and pulse-slow animations via @theme inline (ported from sidecar tailwind.config.js, candidates for DS upstream) - Extend font fallback chains to match sidecar (Inter, Fira Code) The design system already provides surface-*, accent, border color tokens via @theme inline — no color additions needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Align pack install IPC results and streamed output payloads across main, preload, and renderer types - Handle failed catalog mutation starts without opening a stale install modal - Pass selected recent project cwd through project-scoped install and uninstall actions - Keep local-only agent session user and org identity explicitly null until server-owned identity is available Testing: Desktop typecheck, lint, and renderer build passed Risks: Low; scoped to pack catalog install UI flow and typed IPC contracts

- Register dashboard database IPC handlers before PGlite startup completes and wait on the shared database promise per invocation. - Route pack scanner/catalog warnings through gateway logging and resolve catalog probe binaries through the login-shell resolver. - Retry stale worktree directory removal and fall back from invalid Git overrides before materializing PRD branches. - Add focused guards for early IPC registration, binary resolution fallback, and stale PRD worktree cleanup. Testing: Full desktop test suite, desktop typecheck, and desktop lint passed. Risks: Low; dashboard IPC now backpressures on database startup, and invalid Git overrides fall back to the login-shell Git path.

mikeangstadt · 2026-06-08T13:40:43Z

Code Review Results (21 findings, 1 rejected)

Verdict: NEEDS_ATTENTION

HIGH (3)

install-orchestrator.ts:445 — single_install === 1 compares against number but PGlite returns boolean true. Dead code path.
pglite.ts:1590 — Re-imported sessions never refresh metadata (plans, PRs, diffStats become permanently stale).
agent-dashboard-design-system-runtime.ts:185 — Collector emit skips handleSessionMutation(), leaving historicalDetailsCache stale → duplicate sessions.

MEDIUM (18)

install-orchestrator.ts:252 — Uses /usr/bin/which instead of resolveBinaryFromLoginShellSync()
catalog-contents.ts:122 — ghApi() uses bare "gh" instead of login-shell-resolved path
catalog-contents.ts:441 — Non-null assertions on optional ContentsSpec fields
catalog-fetcher.ts:336 — Non-null assertion on optional plugin_path
pglite.ts:599 — handleSessionMutation doesn't invalidate cache on reactivation
agent-session-sync-service.ts:738 — summary: null wire contract change undocumented
pack-store.ts:545 — LIKE metacharacters not escaped in filesystem paths
install-orchestrator.ts:370 — Dead validateTrustedAction export (no callers)
settings-store.ts:99 — Migration uses FEA-1550 instead of cleanup ticket
pack-store.ts:294 — Stale comment about non-existent slash filter
PacksCatalog.tsx:570 — formatDate duplicated across 4 files
agent-dashboard-design-system-runtime.ts:381 — IPC handler ignores 7-day TTL
pglite.ts:2127 — getSkills() unbounded query
pglite-agent-dashboard-database.test.ts:16 — Invalid BillingMode in test stub
package.json:26 — CI test script missing packs/plans/PR coverage
pglite.ts:72 — No schema migration system
catalog-store.ts:22 — DbClient type duplicated across 7 files
catalog-contents.ts:110 — GitHub API helpers duplicated with catalog-fetcher

Dismissed (1)

pack-store.ts:294 — Hygiene false positive on example path in comment

🤖 Generated with Claude Code

- Map pack catalog SQL rows into the shared renderer CatalogEntry DTO before crossing IPC. - Update catalog contents, install runs, and install orchestration to consume the same camelCase catalog contract. - Guard Packs UI rendering against malformed catalog array fields so partial data cannot crash the screen. - Add focused catalog-store contract coverage for catalog, history, and install-run DTO mapping. Testing: Catalog-store contract test, desktop typecheck, desktop lint, and renderer production build passed. Risks: Low; catalog IPC now uses the documented shared DTO shape consistently.

- Replace the Packs recent-projects DISTINCT query with a Postgres-compatible GROUP BY query. - Filter blank cwd values and preserve most-recent ordering by MAX(started_at). - Add a boundary guard for the invalid DISTINCT/ORDER BY pattern. Testing: Agent dashboard boundary test, desktop typecheck, and desktop lint passed. Risks: Low; only affects the recent project picker query used by pack actions.

wongk · 2026-06-08T13:58:20Z

is this file generated?

Correction from my earlier reply: apps/desktop/src/main/database/pglite.ts is hand-written source, not generated. I added a top-of-file note in 19235e6 to make that explicit.

how do we maintain this file? how do we handle DB migrations/schema changes?

- Map installed packs, skills, plans, and pull requests into shared renderer DTOs before IPC. - Add renderer array guards for ported screens so stale or malformed arrays cannot crash map/length usage. - Add ported-screen store contract coverage for the IPC-facing DTO shapes. Testing: Full desktop test suite passed; focused ported-screen store contract tests, catalog contract tests, dashboard boundary tests, desktop typecheck, desktop lint, and renderer production build passed. Risks: Low; ported screen IPC now returns documented shared DTO shapes and renderer guards tolerate empty or malformed arrays.

mikeangstadt · 2026-06-08T14:35:22Z

Handled the latest summary-only review feedback in 19235e6:

Refreshed sessions.metadata on re-import so metadata-backed Plans/PRs/diff stats update after initial import.
Invalidated historical session details after collector imports before notifying the renderer.
Removed the duplicate getSkills() scan from getCoreFeatures() and capped the skill-event query.
Switched pack install/catalog GitHub CLI checks to resolveBinaryFromLoginShellSync().
Guarded missing plugin_path instead of using a non-null assertion.
Escaped SQL LIKE metacharacters for pack usage path matching.
Honored the 7-day catalog contents TTL in the IPC handler.
Removed the dead trusted-action install helper.
Updated the persisted-settings cleanup TODO to PLN-116-cleanup.
Shared the duplicated ported-screen date formatter.
Added the pglite.ts hand-written source note and corrected the inline generated-file thread.

Validation: full desktop test suite passed (2019 tests), focused dashboard/store/boundary tests passed, desktop typecheck passed, desktop lint passed, and renderer production build passed.

mikeangstadt requested a review from a team June 6, 2026 12:49

chatgpt-codex-connector Bot reviewed Jun 6, 2026

View reviewed changes

Comment thread apps/desktop/src/main/database/sqlite-to-pglite-migration.ts Outdated

mikeangstadt force-pushed the fea-1550-sqlite-pglite-migration branch 3 times, most recently from 880ed3e to b8030f7 Compare June 6, 2026 13:24

mikeangstadt requested a review from thadeusb June 6, 2026 13:31