You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1412034
Browse filesBrowse the repository at this point in the historyBrowse files
feat: dashboard chat reply is the agent's own words, not a canned string (#102)
## Problem
The dashboard build chat replied with the **same hardcoded string on
every completed build** — "Done — built and verified against your data.
Saved as a draft — tell me what to change." — regardless of what the
user said. It showed up identically against unrelated messages *and*
against corrections. In the reported case a user said the dashboard
showed ~3k active hotels when they expected 6k+, and the reply was still
a blanket "built and verified": a confident claim of success over a
change that may never have landed. That's worse than templated.
Root cause: the agent's task contract literally said **"no prose"**, and
both the backend
([`SavedDashboardService.java`](../blob/main/backend/src/main/java/com/dbaagent/service/SavedDashboardService.java))
and the frontend store
([`useDashboardChatStore.js`](../blob/main/src/lib/stores/useDashboardChatStore.js))
hardcoded the constant. There was no agent reply to show — the model was
forbidden from speaking.
## Change
- **Contract** (`DashboardAgentService`): the final message now ends
with a required ` ```dashboard-note ` fence — 1–3 sentences in plain
business language: what *this* turn changed, anything it could not
do/verify or deliberately skipped, and on a correction whether the
disputed value actually moved. Bound by the existing "never show
internals" security rule (no table/column/SQL/UUID in the reply).
`extractSummary` reads it; `stripNotes` removes it **before** HTML
extraction so prose can't be mistaken for the artifact. The note rides
in `config.summary`.
- **Rendering**: `SavedDashboardService.buildReplyText` (persistence)
and the frontend store's `buildReplyText` (live tab) both render the
note, falling back to a now-shared `DEFAULT_BUILD_REPLY` constant only
when a turn produced no note — so an agent still on the old contract
keeps working. The two helpers are kept textually in sync (documented in
CLAUDE.md).
## Verification (live stack, not code-reading)
Real build + a correction turn + chat-only path, driven through the
browser (chrome-devtools MCP), with DB read-back and network
confirmation that calls hit the local backend:
- **Happy path** — reply is the agent's own note ("…I did not add richer
breakdowns because the recent activity is very sparse…"), persisted and
rendered; not the canned string.
- **Correction case** (the original bug) — challenged "20"; reply:
*"…The earlier 20 was not a broken count; it was the full conversation
total, and the updated headline now shows sent replies separately, which
is 10…"* — addresses the dispute directly.
- **Security** — leakage scan across all replies: clean.
- **State consistency** — full reload shows the persisted honest notes,
no fallback.
- **Regression** — dashboards still build fully (real widgets via
`deepsql.query`); chat-only path still returns the agent's words and
never appends the canned line.
- **Sync invariant** — backend and frontend fallback strings
byte-identical.
Full hands-on QA: 11/11 scenarios PASS, zero blocking issues.
## Notes
- Backend recompiled (`BUILD SUCCESS`); frontend lint clean.
- **Not a code change but required to run the feature locally:** the
agent handoff needs `AGENT_WEBUI_URL=http://deepsql-agent:8787` (the
compose default). A local `.env` with `http://localhost:8787` makes
*every* build fail as "agent unavailable" — worth aligning
`.env.example`/docs, but outside this PR's committed files.
- This makes replies **honest**, not the underlying data more correct —
a genuinely wrong figure is now surfaced in words, but fixing the
SQL/grounding behind it is a separate follow-up.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: CLAUDE.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -255,7 +255,8 @@ their native runners. See `desktop/README.md` for the full picture.
255
255
Dashboards are **generated by the embedded DeepSQL Agent acting as a coding agent**
256
256
(customized Hermes runtime — see [`agent/README.md`](agent/README.md)) — it writes the whole dashboard as a single self-contained HTML document, not a JSON spec. The earlier spec+renderer model (metrics/charts/tables + a `{{placeholder}}` substitution engine + `DashboardBuilder.js`) was thrown away: the rigid `col BETWEEN {{name}}` convention couldn't express real SQL (e.g. a Unix-epoch date filter → `near '{range.start}'` syntax errors) and boxed the agent in.
257
257
258
-
-`DashboardAgentService` is a thin broker: `ensureProfileForUser` → `ensureSession` (fresh session) → `sendAndAwait` with an **artifact contract**. The agent grounds on the brain/schema, verifies every query with `execute_sql`, then emits ONE HTML doc (in a ` ```html ` block). The broker extracts the HTML and returns `{version:3, renderMode:"artifact", title, html, trace}`, stored verbatim in `saved_dashboards.dashboardConfig`.
258
+
-`DashboardAgentService` is a thin broker: `ensureProfileForUser` → `ensureSession` (fresh session) → `sendAndAwait` with an **artifact contract**. The agent grounds on the brain/schema, verifies every query with `execute_sql`, then emits ONE HTML doc (in a ` ```html ` block). The broker extracts the HTML and returns `{version:3, renderMode:"artifact", title, html, summary, trace}`, stored verbatim in `saved_dashboards.dashboardConfig`.
259
+
- **The chat reply is the agent's own words, not a template.** The build contract ends with a required ` ```dashboard-note ` fence (1–3 sentences): what THIS turn changed, plus anything it could not do/verify or deliberately skipped, and on a correction turn whether the disputed value actually changed. `DashboardAgentService.extractSummary` reads it (and `stripNotes` removes it *before* HTML extraction, so prose can't be mistaken for the artifact); it rides in `config.summary`. Both persistence (`SavedDashboardService.buildReplyText`) and the live tab (`useDashboardChatStore.buildReplyText`) render it, falling back to `DEFAULT_BUILD_REPLY` ("Done — built and verified…") only when a turn produced no note. Before this, the contract banned prose and both sites hardcoded that constant, so every build — including one answering a user's correction ("you're showing 3k, I expected 6k+") — reported the same confident "built and verified", which is worse than templated: it claimed success over an unverified change. **The two `buildReplyText` helpers must stay textually in sync** (backend Java + frontend JS) — the live bubble and the persisted reply are read from the same `summary`.
259
260
- The agent loads the **`dashboard-design` skill** (`agent/skills/dashboard-design/SKILL.md`, v2 — artifact contract, the `deepsql.query` runtime, composition/UX rules, an **intent checklist**, and Unix-epoch date handling).
260
261
-**Rendering + data access**: `DashboardArtifact.jsx` renders the HTML in a **sandboxed iframe** (`sandbox="allow-scripts"`, opaque origin + a strict CSP — no external network). The artifact fetches data only through an injected `deepsql.query(sql)` bridge that `postMessage`s to the parent; the parent calls **`POST /api/dashboards/query`** (`DashboardQueryController`), which is **read-only twice over** (`McpSqlGuardService.validateReadOnlySql` + `QueryExecutionContext.api` = `READ_ONLY_ONLY`) and access-scoped via `assertCanReadConnectionContent`. So the agent's code has full creative freedom while every query stays guarded and sandboxed. The bridge also auto-sizes the iframe and forwards runtime errors.
261
262
- Generation endpoints unchanged (`POST /api/dashboards/generate` + `/generate/stream`). `DashboardBuilder.js`/`DashboardInputs.js` remain only because `tabs/Core/PreviewTab.js` still uses them — the dashboard *creation* path no longer touches them.
0 commit comments