diff --git a/README.md b/README.md index bcecc84..055a33e 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ plugins/tableau/ skills/tableau-analytics/ # read-oriented: querying/exploring content skills/tableau-content-viewer/ # find a view/workbook and render it, no data querying or editing skills/tableau-workbook-authoring/ # generate/modify workbooks by editing TWB XML + skills/author-data-app/ # scaffold, wire, author, package, and publish a Tableau data app (viz extension) skills/shared/rendering.md # shared render-in-side-panel steps, used by both of the above schemas//twb_.xsd # per-Tableau-version TWB XSD schemas (2018.1-2026.2) skills/tableau-workbook-authoring/scripts/validate_workbook.py # validates a .twb/.twbx against the matching XSD diff --git a/plugins/tableau/skills/author-data-app/SKILL.md b/plugins/tableau/skills/author-data-app/SKILL.md new file mode 100644 index 0000000..0825106 --- /dev/null +++ b/plugins/tableau/skills/author-data-app/SKILL.md @@ -0,0 +1,297 @@ +--- +name: author-data-app +description: End-to-end workflow for building a Tableau data app — scaffold a new app with the scaffold-data-app MCP tool and finalize its returned postUnzip plan, author the extension's query + visualization yourself from the human's stated criteria/vibe, then package the workspace into a .twbx and publish it with the MCP publish-workbook flow. Use whenever a user wants to create, build, or publish a Tableau data app. +--- + +# Author Data App + +Builds a Tableau data app from nothing to published. Walk the phases top to +bottom. + +``` +1. Scaffold + finalize → 1.5 Wire datasource* → 2. Author (you) → 3. Package → 4. Publish +``` + +\* Phase 1.5 is a prerequisite to a *working* app: the name-only +`scaffold-data-app` ships an empty ``, so a scaffolded app reaches +no datasource at runtime and renders "no data source found." Wire the target +published datasource into the `.twb` before authoring against it. (Publishing the +starter as-is to prove packaging works does not need it.) + +**Division of labor: you write ALL the code, every phase, always — including +`app.js`.** The human "vibe codes" by describing what they want (criteria, +theme, vibe, target insights) — they do not write `app.js` themselves. Read +the two bundled guides before authoring: [design-data-app.md](design-data-app.md) +(what to build) and [build-data-app.md](build-data-app.md) (how, using this +skill's local tools). Only skip authoring `app.js` if the human explicitly says +they want to write it themselves for this app (rare) — in that case hand off +the workspace path and point them at both guides as their own reference. + +There is intentionally **no separate validation phase** — a TWBX cannot be +pre-validated (Tableau validates extracts/extensions at publish time), so +`publish-workbook` surfaces any errors when you reach phase 4. The one thing you +must get right before then is package *layout* (phase 3), or the workbook won't +open at all. + +--- + +## Phase 1 — Scaffold + finalize + +Call the `scaffold-data-app` MCP tool with the app name: + +> scaffold-data-app({ datappName: "Sales Demo" }) + +**Both transports return the same static, un-substituted template *zip* plus an +identical `postUnzip` plan.** They no longer differ in what's returned or how +it's finalized — only in how the zip gets onto disk: + +- **local (stdio):** result has `filePath` + a `postUnzip` plan. `filePath` + points at the same static template **zip** the S3 path serves — it is *not* a + pre-finalized workspace directory. Unzip it to a temp dir, then apply the plan + there. No download needed, but unzip still is. +- **remote (http):** result has `s3URL` + a `postUnzip` plan. Download the zip + from `s3URL` first, then unzip it to a temp dir and apply the plan there. + +Past the fetch step the two are identical: same unzip, same plan — including the +root-dir rename (`Data App Name` → ``), which now applies to both. +Apply the plan deterministically with the bundled script — applying it freehand +leaves half-replaced `TODO-MANIFEST-ID` / `TODO App Name` tokens or interleaves +edits and renames in the wrong order. + +### Finalizing — apply the postUnzip plan (both transports) + +Save the plan, unzip the template into a temp dir (remote downloads first; +local doesn't), then run `apply-plan.mjs` against that directory — the apply +step itself is identical for both transports. + +```bash +SKILL_DIR="" +WORK="$(mktemp -d -t dataapp)" + +# 1. Save the postUnzip object verbatim (do NOT reformat — find tokens must match byte-for-byte) +cat > "$WORK/plan.json" <<'PLAN_JSON' +{ …paste the result's postUnzip object here… } +PLAN_JSON +``` + +**Remote (http) — download, then unzip:** + +```bash +curl -fsSL "" -o "$WORK/template.zip" +mkdir -p "$WORK/unzipped" +unzip -q "$WORK/template.zip" -d "$WORK/unzipped" +``` + +**Local (stdio) — unzip only, no download** (`filePath` is already the same +template zip, just sitting on local disk instead of behind a presigned URL): + +```bash +mkdir -p "$WORK/unzipped" +unzip -q "" -d "$WORK/unzipped" +``` + +**Both transports — apply the plan** (edits first, then renames; verifies no +placeholders survive): + +```bash +node "$SKILL_DIR/apply-plan.mjs" "$WORK/unzipped" "$WORK/plan.json" +``` + +`apply-plan.mjs` prints the finalized workspace root on stdout. See +[apply-plan.mjs](apply-plan.mjs) for the full contract; it hard-fails if a `find` +token is missing (the zip is stale / out of sync with the plan) rather than +emitting a broken workspace. + +At the end of phase 1 you have a finalized workspace directory: +``` +/ + .twb + Packages/com.tableau.mcp./ + manifest.json + extensions/data-app.trex + content/index.html + content/src/app.js ← the authoring surface + content/src/… +``` + +--- + +## Phase 1.5 — Wire the published datasource (prerequisite for a working app) + +The scaffolded `.twb` ships an **empty ``** (both at the workbook +root and inside the worksheet ``). At runtime the app calls +`getAllDataSourcesAsync()` and finds nothing → it renders **"no data source found +in the workbook."** To query live data the workbook must have a real published +datasource wired in. + +Do this once the user has named a target published datasource; it is +skippable if the user only wants to publish the starter to prove packaging. + +Do **not** hand-edit the XML — the wiring spans four coordinated locations (root +datasource `name`, root `relation connection`, view `datasource name`, +`datasource-dependencies datasource`) that must all carry the identical +`sqlproxy.` join key, and both empty anchors must be filled. Use the bundled +[wire-datasource.mjs](wire-datasource.mjs) script, which does all four edits +atomically and hard-fails rather than emitting a half-wired workbook. + +1. **Get the datasource's identity** with `list-datasources` (LUID, name/caption, + contentUrl, and the server host + site) and `get-datasource-metadata({ datasourceLuid })` + (field names + datatypes). The published DS **contentUrl** is the + `repositoryId`. +2. **Write a descriptor** listing *only the fields the app will query* (name + + datatype + role), e.g.: + + ```bash + cat > "$WORK/descriptor.json" <<'DS_JSON' + { + "caption": "", + "repositoryId": "", + "site": "", + "server": "", + "channel": "https", "port": 443, + "fields": [ + { "name": "Profit", "datatype": "real", "role": "measure" }, + { "name": "Region", "datatype": "string", "role": "dimension" } + ] + } + DS_JSON + ``` + +3. **Run the wiring script** (it prints the wired `.twb` path, and generates a + consistent `sqlproxy.` unless you supply `connectionName`): + + ```bash + node "$SKILL_DIR/wire-datasource.mjs" "/.twb" "$WORK/descriptor.json" + ``` + +The script hard-fails if an anchor is missing (already wired / template drifted), +if any empty `` survives, or if the join key isn't referenced ≥4×. +Trust that failure over patching the XML by hand. `datatype` maps to the column +`type` (`real`/`integer` → quantitative, `date`/`datetime` → ordinal, else +nominal); `role: "measure"` gets a `Sum` aggregation, `dimension` a `Count`. + +--- + +## Phase 2 — Author + +**Always author `app.js` yourself — this is the fixed default, not a +fallback.** The human vibe-codes: they describe what they want (criteria, +theme, target insights, audience) and you turn that into the actual +`ds.queryAsync(...)` → chart implementation. Read +[design-data-app.md](design-data-app.md) (what to build) and +[build-data-app.md](build-data-app.md) (how) first, then: + +1. **Introspect the datasource.** `list-datasources` → find the LUID → + `get-datasource-metadata({ datasourceLuid })` for fields/model/params → + `query-datasource({ datasourceLuid, query, limit })` to preview real VDS + `{ data: [...] }` rows and confirm field captions/types before committing to a + chart. (Ensure Phase 1.5 wiring is done — the app can't query without it.) +2. **Design what to build** using [design-data-app.md](design-data-app.md): pick + the archetype by audience, lead with the message (BLUF), choose the mark by the + perception hierarchy, keep graphical integrity (zero baseline, "as of" + provenance), use action titles + direct labels, and restrained color (grey + + one accent, colorblind-safe). +3. **Edit `content/src/app.js` on disk** (there is no upsert tool). Inside the + `AUTHOR YOUR APP HERE` block, replace the `renderStarter(...)` call with a real + `ds.queryAsync(query)` → `extractData(result)` → build a Vega-Lite spec → + `vegaEmbed(el, spec)`; match columns by field name. Vendor + vega/vega-lite/vega-embed locally under `content/src/` and load them from + `index.html` (mirror how `tableau.extensions.1.latest.js` is already vendored + relative and loaded before `app.js`). +4. **Follow the sandbox rules in the `AUTHOR YOUR APP HERE` comment** — that + comment is the source of truth (render-first/initialize-second, surface every + error via `renderError`, no CDN, 2D over WebGL, `textContent`/`createElement` + never `innerHTML` with live values). Don't re-derive them. +5. **There is no local preview.** You cannot see the app render against live data + while authoring — the visual review happens live in Tableau after publish + (phase 4). + +### Rare exception: the human wants to write `app.js` themselves + +Only skip authoring if the human explicitly says they want to write `app.js` +themselves for this app — an override of the default, not the norm. In that +case, stop and hand off: tell them the workspace path +(`Packages/com.tableau.mcp./content/src/app.js`) and point them at +[design-data-app.md](design-data-app.md) and [build-data-app.md](build-data-app.md) +as their own reference. Resume at phase 3 once they say it's authored. + +--- + +## Phase 3 — Package into a .twbx + +A `.twbx` is a zip of the workspace **contents** with the `.twb` and `Packages/` +at the **archive root** — never nested inside the `/` folder. Nesting +is the #1 cause of `NativeException: An unexpected error occurred opening the +packaged workbook` and `PackageValidationException: Package directory contains no +extension .trex files under extensions/`. + +Package with the proven two-step zip (run from *inside* the workspace dir so +paths are root-relative), excluding OS cruft: + +```bash +cd "" # the finalized workspace dir +OUT="../.twbx" +rm -f "$OUT" +zip -X "$OUT" ".twb" # .twb at root, first +zip -rX "$OUT" Packages -x '*.DS_Store' '*/.DS_Store' '__MACOSX*' # package tree, no cruft +unzip -l "$OUT" # sanity: .twb + Packages/… at top level, no / prefix +``` + +The listing must show `.twb` and `Packages/com.tableau.mcp./…` +at the top level with no wrapping folder and no `.DS_Store`/`__MACOSX` entries. + +> The template these workspaces come from is already publish-valid (`.twb` +> extension wired into a pane, `.trex` with `author email`, `` block, +> ``, `min-api-version`). Packaging is the only structural step you own. + +--- + +## Phase 4 — Publish + +Uses the MCP publish tools (gated by the `authoring-tools` feature; not available +to Slack clients). + +1. **Find the target project LUID:** + > list-projects({}) + Pick the project the user wants (ask if ambiguous). + +2. **Publish.** Two paths — pick based on transport: + + - **Local (stdio), simplest:** the `.twbx` is on the MCP server's own + filesystem, so pass it directly: + > publish-workbook({ workbookFilePath: "", name: "", projectId: "", overwrite: false }) + + - **Remote (http) / staged uploads configured:** stage the bytes first, then + publish by id: + > request-workbook-upload({ filename: ".twbx" }) → returns an upload URL + workbookUploadId + > (upload the .twbx bytes to the returned URL — staged-workbook-upload) + > publish-workbook({ workbookUploadId: "", name: "", projectId: "", overwrite: false }) + +3. **Report the outcome.** On success `publish-workbook` returns + `status: "published"` with the workbook `url` and any `warnings` — give the + user the URL. If it returns `status: "invalid"` (or an error), surface the + `errors`/`warnings` verbatim; common causes trace back to `.twb`/`.trex` + wiring, not packaging. Set `overwrite: true` only if the user wants to replace + an existing workbook of the same name. + +--- + +## Common Mistakes + +- **Nesting the workspace folder in the .twbx.** Zip the *contents* (`.twb` + + `Packages/` at root), not the `/` directory. Always `unzip -l` to confirm. +- **Applying a postUnzip plan freehand.** Use `apply-plan.mjs` — edits before + renames, renames deepest-first, verified. See its Common Mistakes section. +- **Hand-editing the `` wiring.** Use `wire-datasource.mjs` — freehand + edits mismatch the `sqlproxy.` join key across its four locations or leave an + empty `` anchor, and the app silently reaches no data. +- **Assuming a local result's `filePath` is already substituted, or skipping + unzip because it's local.** `filePath` points at the same static, + un-substituted template **zip** the S3 path serves — not a finalized + workspace directory. Unzip it (no download needed, but unzip still is) and + apply the plan before authoring, exactly like the remote path. +- **Handing off `app.js` to the human unprompted.** Phase 2 authoring is the + fixed default — always write `app.js` yourself from the human's stated + criteria/vibe. Only hand off when the human explicitly says they want to + write it themselves. +- **Shipping OS cruft.** Exclude `.DS_Store` / `__MACOSX` from the `.twbx`. diff --git a/plugins/tableau/skills/author-data-app/apply-plan.mjs b/plugins/tableau/skills/author-data-app/apply-plan.mjs new file mode 100644 index 0000000..73760f0 --- /dev/null +++ b/plugins/tableau/skills/author-data-app/apply-plan.mjs @@ -0,0 +1,149 @@ +#!/usr/bin/env node +/** + * Deterministically finalize a scaffolded data app workspace by applying the + * `postUnzip` plan returned by the `scaffold-data-app` MCP tool. + * + * The remote (http) transport returns a plan of the shape: + * { instructions, edits: [{ file, replacements: [{ find, replace, occurrence }] }], + * renames: [{ from, to }], wiresDatasource } + * where every `file`/`from`/`to` path is relative to the unzip directory and + * includes the template root dir prefix (e.g. "Data App Name/..."). + * + * Order matters and is the whole reason this is a script rather than freehand + * edits: apply EVERY edit first (literal, non-regex find/replace on file + * contents), THEN apply the renames in the given order (deepest paths first, + * the root dir last). Doing renames before edits would invalidate the edit + * paths; reordering renames would rename a parent out from under a child. + * + * Each replacement's `occurrence` is `'first'` (replace only the first + * remaining occurrence — used to resolve two textually-identical anchors to + * two different values in sequence) or `'all'`/omitted (replace every + * occurrence, the default). When `wiresDatasource` is truthy, the plan's + * `.twb` edit included datasource-wiring replacements, so the residual-token + * check below also verifies no empty `` anchor survived. + * + * Usage: + * node apply-plan.mjs + * + * Exits non-zero with a diagnostic on any failure, and after applying, + * verifies no residual placeholder tokens remain in the finalized files. + */ + +import { readFileSync, renameSync, writeFileSync } from 'node:fs'; +import { join, resolve } from 'node:path'; + +const PLACEHOLDER_TOKENS = ['TODO-MANIFEST-ID', 'TODO App Name', 'TODO Username via Tableau MCP']; +const WIRING_ANCHOR_TOKEN = ''; + +function die(message) { + console.error(`✗ ${message}`); + process.exit(1); +} + +const [, , unzipDirArg, planPathArg] = process.argv; +if (!unzipDirArg || !planPathArg) { + die('Usage: node apply-plan.mjs '); +} + +const unzipDir = resolve(unzipDirArg); + +let plan; +try { + plan = JSON.parse(readFileSync(planPathArg, 'utf8')); +} catch (error) { + die(`Could not read/parse plan JSON at ${planPathArg}: ${error.message}`); +} + +const edits = Array.isArray(plan.edits) ? plan.edits : []; +const renames = Array.isArray(plan.renames) ? plan.renames : []; +if (edits.length === 0 && renames.length === 0) { + die('Plan has no edits or renames — did you pass the full postUnzip object?'); +} + +// Guard against a plan path escaping the unzip dir. +function safeJoin(rel) { + const abs = resolve(unzipDir, rel); + if (abs !== unzipDir && !abs.startsWith(unzipDir + '/')) { + die(`Plan path escapes the unzip directory: ${rel}`); + } + return abs; +} + +// 1) Apply edits: literal (non-regex) find/replace on each file's contents. +for (const { file, replacements } of edits) { + const abs = safeJoin(file); + let content; + try { + content = readFileSync(abs, 'utf8'); + } catch (error) { + die(`Edit target missing: ${file} (${error.message})`); + } + for (const { find, replace, occurrence } of replacements ?? []) { + if (!content.includes(find)) { + die(`Placeholder "${find}" not found in ${file} — template/plan out of sync.`); + } + if (occurrence === 'first') { + const idx = content.indexOf(find); + content = content.slice(0, idx) + replace + content.slice(idx + find.length); + } else { + content = content.split(find).join(replace); + } + } + writeFileSync(abs, content, 'utf8'); + console.error(` edited ${file}`); +} + +// 2) Apply renames in the given order (deepest-first, root last). +for (const { from, to } of renames) { + try { + renameSync(safeJoin(from), safeJoin(to)); + } catch (error) { + die(`Rename failed: ${from} -> ${to} (${error.message})`); + } + console.error(` renamed ${from} -> ${to}`); +} + +// 3) The finalized workspace root is the target of the last rename. +const finalRootRel = renames.at(-1)?.to; +const finalRoot = finalRootRel ? safeJoin(finalRootRel) : unzipDir; + +// 4) Verify no placeholder tokens survived in the substituted text files. +// Recompute each edited file's final path by applying every rename's +// from->to prefix substitution, in order (a file may be moved by more +// than one rename, e.g. a root-dir rename AND a nested package-dir rename). +function remapThroughRenames(relPath) { + let current = relPath; + for (const { from, to } of renames) { + if (current === from) { + current = to; + } else if (current.startsWith(from + '/')) { + current = to + current.slice(from.length); + } + } + return current; +} + +const residual = []; +for (const { file } of edits) { + const finalRel = remapThroughRenames(file); + let content; + try { + content = readFileSync(safeJoin(finalRel), 'utf8'); + } catch (error) { + die(`Finalized file missing for placeholder check: ${finalRel} (${error.message})`); + } + const tokens = plan.wiresDatasource + ? [...PLACEHOLDER_TOKENS, WIRING_ANCHOR_TOKEN] + : PLACEHOLDER_TOKENS; + for (const token of tokens) { + if (content.includes(token)) { + residual.push(`${finalRel}: "${token}"`); + } + } +} +if (residual.length > 0) { + die(`Residual placeholders after finalize:\n ${residual.join('\n ')}`); +} + +console.error(`✓ Finalized workspace at ${join(finalRoot)}`); +console.log(finalRoot); diff --git a/plugins/tableau/skills/author-data-app/build-data-app.md b/plugins/tableau/skills/author-data-app/build-data-app.md new file mode 100644 index 0000000..455a877 --- /dev/null +++ b/plugins/tableau/skills/author-data-app/build-data-app.md @@ -0,0 +1,104 @@ +# Build a data app — the HOW (workflow + mechanics) + +> The **HOW** layer of authoring, adapted for this skill's local MCP tools. Read +> [design-data-app.md](design-data-app.md) first for **WHAT** to build. This file owns the workflow; +> the in-code mechanics (sandbox rules, VDS query shape, safe DOM, Vega-Lite-local) are owned by the +> `AUTHOR YOUR APP HERE` comment block in the scaffolded `content/src/app.js` — follow it there, this +> file only points at it. + +## Live-query model (read this first) + +**The data app queries its datasource live via the Tableau Extensions API — there is NO embedded data +snapshot.** The shipped app calls `readMetadataAsync()` / `queryAsync()` at view time against the +published datasource, so it always reflects current data. Two consequences: + +1. **The app reaches datasources through the workbook, and queries them directly.** It is a viz + extension hosted on a worksheet, so it uses + `tableau.extensions.workbook.getAllDataSourcesAsync()` to reach every datasource wired into the + workbook, then calls `ds.readMetadataAsync()` / `ds.queryAsync(query)`. It does **not** read + marks-card summary data and the host worksheet declares **no encodings** — the app builds its own + VDS query. Results come back in the standard VizQL Data Service shape `{ data: [...] }`; the + starter's `extractData()` helper reads `result.data` — use it, and match columns by field name + (not position). + - **Prerequisite:** the workbook must actually have that datasource wired in. Our name-only + `scaffold-data-app` ships an **empty ``**, so a freshly scaffolded workbook + reaches *nothing* at runtime and renders "no data source found." **Phase 1.5 of the SKILL wires + the datasource into the `.twb`** — do that before authoring, or the live query has no target. +2. **You cannot run the live query yourself.** A live query only executes inside the Tableau host, so + you cannot see real rows until the app is published and opened in Tableau. While authoring, + introspect the datasource with `get-datasource-metadata` / `query-datasource` to design and + sanity-check the query; do the **visual** review in Tableau **after** publishing. + +## 1. Detect intent + +Author a real app (beyond the starter) when the user asks to "chart", "visualize", "build a +dashboard", or otherwise wants a live visual over a specific published datasource. Skip it when the +answer is a single-value lookup or text and the user hasn't signaled interest in a reusable visual — +then the starter handoff (SKILL Phase 2 default) is enough. + +## 2. Identify the published datasource(s) + +Find the target published datasource and its LUID with `list-datasources` (ask the user which one if +ambiguous). You can wire more than one if the app genuinely needs it. This LUID drives both the +Phase 1.5 `.twb` wiring and your introspection queries. + +## 3. Introspect the datasource + +- `get-datasource-metadata({ datasourceLuid })` → fields, data model, parameters. This is the field + list the app will match by name. +- `query-datasource({ datasourceLuid, query, limit })` → preview real VDS `{ data: [...] }` rows so + you can sanity-check the exact query the app will run before you commit to a chart. Match columns + by field caption/name, not by position. + +## 4. Author `content/src/app.js` (direct file edit) + +There is no upsert tool — **edit the file on disk.** Inside the `AUTHOR YOUR APP HERE` block, replace +the `renderStarter(...)` call with the real flow: + +- Build a VDS query (fields + optional filters/aggregations) and call `ds.queryAsync(query)`. +- Read rows with the provided `extractData()` helper (returns `result.data`); match columns by field + name. +- Render with a chart. **Default library: Vega-Lite** — build a spec from the rows and render with + `vegaEmbed(el, spec)`. + +Prefer to derive new fields / change data shapes **at query time** rather than in JS. There is no +required file layout, chart count, or palette — a good app clearly addresses the user's objective. +See [design-data-app.md](design-data-app.md) for encoding, narrative, integrity, and color. + +### Sandbox & lifecycle rules — follow the in-file comment + +The published app runs inside the Tableau viz-extension sandbox, not a browser. The authoritative +rules live in the `AUTHOR YOUR APP HERE` comment in the scaffolded `app.js` and are **not restated +here** to avoid drift. In brief, that comment requires: surface every error on-screen (no visible +console — use the starter's `renderError`); render first / initialize second; **vendor libraries +locally, no CDN** (add vega/vega-lite/vega-embed under `content/src/` and load them from +`index.html` with relative paths, mirroring how `tableau.extensions.1.latest.js` is already vendored +and loaded before `app.js`); prefer 2D (SVG/Canvas/DOM) over WebGL; use safe DOM APIs +(`textContent` / `createElement`), never `innerHTML` with live values. + +## 5. Package (SKILL Phase 3) + +There is **no separate validation tool and no local preview.** A `.twbx` cannot be pre-validated — +Tableau validates at publish time. Package the workspace into a flat `.twbx` per SKILL Phase 3 +(`.twb` + `Packages/` at the archive root; `unzip -l` to confirm; no `.DS_Store`/`__MACOSX`). + +## 6. Ask explicitly before publishing + +Never auto-publish. Ask, in plain language, whether the user wants this app published — publishing +creates content on their Tableau site, and that is their decision. "Looks good" is not consent to +publish; get a clear yes to publishing specifically. If there is no clear yes, stop. + +## 7. Publish (SKILL Phase 4) + +On an explicit yes, publish with `publish-workbook` (via `list-projects` for the target project +LUID). Surface the returned canonical `url` verbatim and report any warnings. `publish-workbook` +surfaces any structural/extension errors at this point (it is where validation effectively happens). + +## 8. Review the live app in Tableau (the only "preview") + +There is **no local preview** — the live query only runs inside Tableau. Open the published workbook +(the user's personal space is the natural iteration target) and confirm the viz-extension worksheet +renders the real chart — not "no data source found" (Phase 1.5 wiring missing) or "Live query +unavailable" (query/render error). A one-time extension trust prompt on first load is expected, not a +failure. Apply the design checks in [design-data-app.md](design-data-app.md) (5-second + takeaway +test). To iterate: edit `app.js` (step 4), re-package (step 5), republish (steps 6–7). diff --git a/plugins/tableau/skills/author-data-app/design-data-app.md b/plugins/tableau/skills/author-data-app/design-data-app.md new file mode 100644 index 0000000..1c4f9be --- /dev/null +++ b/plugins/tableau/skills/author-data-app/design-data-app.md @@ -0,0 +1,173 @@ +# Design a compelling, trustworthy Tableau data app + +> The **WHAT/why** layer of authoring — decide what to show and why one encoding reads more +> truthfully than another. The companion [build-data-app.md](build-data-app.md) owns the **HOW** +> (introspect → author `app.js` → package → publish). Read this first when the user asks you to +> author. + +## What this is + +The design layer for a Tableau **data app** — a custom HTML/JS/CSS web app, bundled as a viz +(worksheet) extension, that queries a published datasource **live**. + +**You render everything yourself.** There is no Marks card, no Show Me, no Analytics pane, no +drag-to-Rows. When this guide says "put the measure on position" or "grey the non-focal marks," that +is an instruction about the SVG/Canvas/DOM you write and the CSS you apply — not a Tableau UI action. +Treat these as strong, well-argued defaults ("schools of thought, not commandments"): name the +context when you depart, and remember the only decisive test is whether the intended viewer reaches +the intended conclusion. + +## Start with the message, not the chart + +Decide *what to say* before you decide how to draw it. + +- **Lead with the answer (BLUF).** The app's headline (`

`/header) is the bottom line, stated as a + sentence: "Recommendation: discontinue Product X (−$500K/yr)", not a topic like "Product + Profitability." Don't make a decision-maker hunt for the punchline. +- **Structure like a pyramid.** Answer on top → about three **MECE** supporting points (mutually + exclusive, collectively exhaustive; the rule of three respects working memory) → detail beneath + each. In a multi-panel app: a hero number/statement at the top, then a small number of panels that + each prove one non-overlapping point, with drill/detail available on demand. +- **Overview first, then zoom/filter, then details on demand** (Shneiderman). Open on the summary; + let the viewer narrow; reveal row-level detail last (hover, expand, a detail panel). +- **Apply the "so what?" test to every panel.** If a view has no answer to "so what should I do with + this?", cut it. A pile of correct charts with no Big Idea is clutter. + +## Choose the app's role (archetype) + +A data app's **purpose** drives its density and interactivity. Name the role before composing; most +bad apps come from handing one shape to the wrong audience. + +| Role | Reader & question | Density | Interactivity | +|---|---|---|---| +| **Strategic** | Executives — "Are we on track against goals?" | Low: a few KPIs *with context* | Minimal — glance, maybe one filter | +| **Operational** | Front-line ops — "Is anything wrong now, and what do I do?" | Moderate, ruthlessly prioritized to the actionable | Alerting + light triage drill | +| **Analytical** | Analysts — "Why did this happen? What if we change X?" | High: many marks, fine granularity | Rich — filters, parameters, select-to-compare, drill | + +**Hybrids are the norm.** A strategic KPI strip on top (glance) over operational/analytical panels +below (act/investigate), with progressive disclosure so complexity appears only when summoned. Name +which role *each panel* serves and design that panel's density and interaction to its role. + +**Compose it like a real dashboard.** Hero view upper-left; align comparable panels on shared scales; +group related panels with whitespace/padding (proximity), not heavy borders; keep one consistent mark +type and palette for like data (similarity). Use a CSS grid / flex layout for small multiples and +coordinated panels; reveal drill panels on click rather than showing everything at once. + +## Encode by the judgment the viewer must make + +The perception hierarchy (Cleveland & McGill; broadly replicated) ranks how *accurately* people +decode encodings, most → least accurate: + +1. **Position along a common scale** (aligned bars, dot plots) +2. **Position along non-aligned identical scales** (small multiples) +3. **Length, direction, angle** (unaligned bars; pie slice angles) +4. **Area** (bubbles, treemaps) +5. **Volume, curvature** (3-D) +6. **Color saturation / shading** — least accurate + +**So:** for any value the viewer must read *precisely*, use **position** — map the key measure to an +x/y axis. Use **color and size only for secondary, low-precision** encoding (category hue, rough +magnitude). A shared axis beats a dual axis (the second axis manufactures crossings). Avoid 3-D +entirely. **Caveat:** for "spot the cluster/outlier/shape" (not precise readout) a dense scatter or +heatmap can beat a long bar list — match the encoding to the question. + +**Mark choice, in priority order** when several would work: + +1. **Bar** — comparison, ranking, composition, distribution. When uncertain, a *sorted* horizontal + bar chart is the safest default. +2. **Line** — trends over a continuous (usually time) axis. +3. **Scatter (circle)** — relationship between two measures. +4. **Text/number** — exact value lookup for small data (a big KPI number). +5. **Heatmap (square)** — dense matrix patterns. +6. **Area** — only when volume emphasis adds meaning; stacked for part-to-whole. +7. **Pie** — rarely best; use a bar. Acceptable only for a 2–3-slice "roughly half" read. + +**Common encoding mistakes to avoid:** lines on categorical (unordered) data (implies false +continuity); unsorted bars (make the viewer scan); >7 color categories (indistinguishable — group +into "Other"); non-stacked areas with 3+ series (occlusion); filled maps when a bar answers more +precisely (area dominates perception); fine distinctions encoded by bubble size (Weber's law — a ~10% +size change is the just-noticeable difference). + +## Don't lie: graphical integrity + +- **Zero baseline on bars.** Bar length encodes from a common baseline; a truncated axis inflates the + ratio and lies. Keep zero when you draw bars. Lines encode by position, so a non-zero baseline can + be legitimate *if labeled*. +- **No deceptive dual axis.** Two unrelated measures on independent scales can be made to "cross" + anywhere. Prefer a shared/blended axis or index both series to % change; if you must dual, + synchronize and label. +- **Rough-only for area/size.** Doubling a value doubles a bubble's *area* but radius grows as √, so + viewers over-read big bubbles. +- **Show provenance and uncertainty.** Because the app queries **live**, surface the datasource name + and an "as of " note in the app chrome; where relevant, show a reference band / + confidence range and sample size. A number with no target/prior/benchmark is meaningless. +- **Don't imply causation** from a scatter + trend line without the caveat. + +## Title and annotate for the takeaway (highest-leverage move) + +- **Action/insight titles, not topic titles.** "West region drove 60% of Q3 growth" — not "Sales by + Region." Restate the sentence dynamically as the viewer filters if you can. +- **Direct labels beat a legend.** Put the category name next to its mark (e.g. at a line's end) to + remove the eye's round-trip. +- **Reference lines/bands deliver context** — target, prior period, average, good/bad range. Draw + them in your chart. +- **Callouts spotlight the climax.** Annotate the one mark that carries the insight with the insight + *and the implied action* — not just a value. + +## Color + +Get the palette *type* right first — it's the most common color mistake. + +- **Continuous data → sequential or diverging gradient.** Sequential (light → dark) for one + direction; diverging (e.g. blue ↔ orange through a neutral midpoint) for two directions from a + meaningful center (pin the midpoint to the real center, e.g. zero for profit — not the data + average). Steps must be visibly distinct; use ~5–7 steps, not a subtle 9-step ramp. +- **Categorical data → distinct hues**, ≤ 5–7 of them. Beyond that, the legend *becomes* the chart — + group small categories into "Other." +- **Grey is the most important color.** Default most marks to a medium grey (`#999`/`#aaa`) and spend + **one accent hue** on the mark that carries the insight. One blue bar among eleven grey ones + communicates the ranking instantly; twelve different colors communicate nothing. This is + pre-attentive pop-out — use *one* attribute (color) so the key mark is found in <250ms; a + conjunction (red *and* square) forces slow serial search. +- **Never use hue for ordered data** (hue has no natural order) — use a sequential lightness ramp. +- **Accessibility outranks minimalism.** ~8% of men have red-green color-vision deficiency. Never + encode meaning by red-vs-green alone — add a label, icon, or position cue. Prefer ColorBrewer + palettes (colorblind-tested, print-safe); ship a hardcoded palette in the app rather than a rainbow + default. +- **Text contrast.** WCAG AA is 4.5:1 (normal) / 3:1 (large). Dark fills need light labels and vice + versa; make annotation text one shade darker than its mark color so it stays legible. +- **Keep legends close, or skip them.** Prefer direct labels; when a legend is needed, place it + adjacent to the chart it serves. Color only the *category noun* in an annotation, not the whole + sentence. A 2–4-category chart can use its title as the legend (color the category words to match + the marks). + +## Declutter — within reason + +Maximize the share of ink that carries data: cut chartjunk (decoration that dominates data), heavy +gridlines, borders, and gradients that don't encode anything; direct-label instead of a distant +legend. **But** minimalism is not settled dogma — faint gridlines aid value lookup, controlled +embellishment can aid *retention* (not just comprehension), and accessibility always wins over a +lower ink count. Interactivity lets you keep the overview clean and put detail on demand. + +## Verify — the only test that counts + +You **cannot** see the app render against live data while authoring — a live query only runs inside +the Tableau host. So the design review is deferred: **publish the app (phase 4), open it in Tableau, +and look at it running against live data.** Then apply the classic checks there: + +- **The 5-second test:** glance for ~5 seconds, look away, and name what you remember and where your + eyes went first. If the hero metric isn't what's recalled, fix prominence (size, top-left position, + the lone accent) before adding anything. +- **The takeaway test:** confirm a representative viewer reaches the intended conclusion *and action* + in seconds. If not, redesign — don't re-explain. + +If it doesn't read, iterate the workspace files and republish. The audience, not the author, judges +whether it works. + +## Source + +Design rationale adapted with permission from field design references by Jon Plax and prior Tableau +authoring knowledge; underlying frameworks are Cleveland & McGill, Bertin, Tufte, Few, Knaflic +(*Storytelling with Data*), Cairo (*The Truthful Art* / *How Charts Lie*), the Minto Pyramid, +Shneiderman's mantra, Gestalt, and Lisa Charlotte Muth's color guide (Datawrapper). All native-Desktop +authoring mechanics have been re-expressed for a custom-rendered web data app. diff --git a/plugins/tableau/skills/author-data-app/wire-datasource.mjs b/plugins/tableau/skills/author-data-app/wire-datasource.mjs new file mode 100644 index 0000000..660f590 --- /dev/null +++ b/plugins/tableau/skills/author-data-app/wire-datasource.mjs @@ -0,0 +1,246 @@ +#!/usr/bin/env node +/** + * Deterministically wire a published datasource into a scaffolded data-app `.twb`. + * + * The `scaffold-data-app` MCP tool emits a workbook with TWO empty + * `` anchors — one at the workbook root and one inside the + * worksheet ``. Until they are filled, the running extension calls + * `getAllDataSourcesAsync()`, finds nothing, and renders "no data source found + * in the workbook." This script fills both anchors with a single published + * `sqlproxy` (Data Server) datasource, keeping the `sqlproxy.` join key + * byte-identical everywhere it must appear. + * + * It is a script rather than freehand XML for the same reason as apply-plan.mjs: + * the wiring spans four coordinated locations (root datasource `name`, root + * `relation connection`, view `datasource name`, `datasource-dependencies + * datasource`) that must agree exactly, and it's easy to leave one empty anchor + * behind. Get any of that wrong and the workbook silently reaches no data. + * + * Usage: + * node wire-datasource.mjs + * + * descriptor.json (Claude assembles from list-datasources + get-datasource-metadata; + * list ONLY the fields the app will query): + * { + * "caption": "Superstore Datasource", + * "repositoryId": "SuperstoreDatasource", // published DS contentUrl (== repo-location id / dbname) + * "site": "mcp-test", + * "server": "10ax.online.tableau.com", + * "channel": "https", // optional, default https + * "port": 443, // optional, default 443 (use http/80 for on-prem) + * "connectionName": "sqlproxy.", // optional, generated if omitted + * "fields": [ + * { "name": "Profit", "datatype": "real", "role": "measure" }, + * { "name": "Region", "datatype": "string", "role": "dimension" } + * ] + * } + * + * Exits non-zero with a diagnostic on any failure (missing/already-filled + * anchor, empty fields, drifted template) rather than emitting a broken workbook. + * Prints the wired `.twb` path on stdout. + */ + +import { readFileSync, writeFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +// The exact empty anchors emitted by the scaffold template. Matched literally. +const EMPTY_ANCHOR = ''; + +function die(message) { + console.error(`✗ ${message}`); + process.exit(1); +} + +// XML attribute-value escaping (single-quoted attrs + element text). +function esc(value) { + return String(value) + .replace(/&/g, '&') + .replace(//g, '>') + .replace(/'/g, ''') + .replace(/"/g, '"'); +} + +// datatype -> Tableau column `type`. +function typeOf(datatype) { + switch (String(datatype).toLowerCase()) { + case 'real': + case 'integer': + return 'quantitative'; + case 'date': + case 'datetime': + return 'ordinal'; + default: + return 'nominal'; // string and anything unrecognized + } +} + +// A field's derived attributes, computed once and reused across all blocks so +// the root metadata-record, the view column, and the column-instance agree. +function deriveField(field, ordinal) { + const name = field?.name; + if (!name || typeof name !== 'string') { + die(`Every field needs a string "name" (field #${ordinal} was ${JSON.stringify(field)}).`); + } + const datatype = String(field.datatype || 'string').toLowerCase(); + const role = field.role === 'measure' ? 'measure' : 'dimension'; + const isMeasure = role === 'measure'; + const type = typeOf(datatype); + return { + name, + datatype, + role, + type, + ordinal, + aggregation: isMeasure ? 'Sum' : 'Count', + // role attribute: 0 = dimension, 1 = measure + roleAttr: isMeasure ? 1 : 0, + localName: `[${name}]`, + // column-instance derivation + name token: [sum:Profit:qk] / [none:Region:nk] + derivation: isMeasure ? 'Sum' : 'None', + instanceName: isMeasure ? `[sum:${name}:qk]` : `[none:${name}:nk]`, + }; +} + +// --- args --------------------------------------------------------------- + +const [, , twbPathArg, descriptorPathArg] = process.argv; +if (!twbPathArg || !descriptorPathArg) { + die('Usage: node wire-datasource.mjs '); +} +const twbPath = resolve(twbPathArg); + +let descriptor; +try { + descriptor = JSON.parse(readFileSync(descriptorPathArg, 'utf8')); +} catch (error) { + die(`Could not read/parse descriptor JSON at ${descriptorPathArg}: ${error.message}`); +} + +const { caption, repositoryId, site, server } = descriptor; +for (const [key, value] of Object.entries({ caption, repositoryId, site, server })) { + if (!value || typeof value !== 'string') { + die(`Descriptor is missing required string "${key}".`); + } +} +const channel = descriptor.channel || 'https'; +const port = descriptor.port ?? (channel === 'https' ? 443 : 80); + +const fieldsIn = Array.isArray(descriptor.fields) ? descriptor.fields : []; +if (fieldsIn.length === 0) { + die('Descriptor "fields" must list at least one field the app will query.'); +} +const fields = fieldsIn.map(deriveField); + +// Single source of truth for the join key. +const connectionName = + descriptor.connectionName || + `sqlproxy.${Math.random().toString(36).slice(2)}${Math.random().toString(36).slice(2)}`.slice(0, 37); +if (!connectionName.startsWith('sqlproxy.')) { + die(`connectionName must start with "sqlproxy." (got "${connectionName}").`); +} + +// --- build the XML blocks ---------------------------------------------- + +const metadataRecords = fields + .map( + (f) => ` + ${esc(f.name)} + ${f.type === 'quantitative' ? 5 : 129} + ${esc(f.localName)} + [sqlproxy] + ${esc(f.name)} + ${f.ordinal} + true + ${esc(f.datatype)} + ${f.aggregation} + true + + 1 + ${f.roleAttr} + + `, + ) + .join('\n'); + +const rootDatasource = ` + + + + + + + +${metadataRecords} + + + + `; + +const viewColumns = fields + .map( + (f) => + ` `, + ) + .join('\n'); + +const viewColumnInstances = fields + .map( + (f) => + ` `, + ) + .join('\n'); + +const viewDatasources = ` + + + +${viewColumns} +${viewColumnInstances} + `; + +// --- apply, splitting on so each anchor is unambiguous ------ + +let content; +try { + content = readFileSync(twbPath, 'utf8'); +} catch (error) { + die(`Could not read .twb at ${twbPath}: ${error.message}`); +} + +const splitIdx = content.indexOf(''); +if (splitIdx === -1) { + die('No element found — is this a scaffolded data-app .twb?'); +} +let head = content.slice(0, splitIdx); +let tail = content.slice(splitIdx); + +// Root anchor lives in the head (before ). +if (!head.includes(EMPTY_ANCHOR)) { + die(`Root "${EMPTY_ANCHOR}" anchor not found before — already wired or template drifted.`); +} +head = head.replace(EMPTY_ANCHOR, rootDatasource); + +// View anchor is the first empty inside the worksheets section. +if (!tail.includes(EMPTY_ANCHOR)) { + die(`View "${EMPTY_ANCHOR}" anchor not found inside — already wired or template drifted.`); +} +tail = tail.replace(EMPTY_ANCHOR, viewDatasources); + +const wired = head + tail; + +// --- verify before writing ---------------------------------------------- + +if (wired.includes(EMPTY_ANCHOR)) { + die('An empty anchor survived wiring — refusing to write a half-wired workbook.'); +} +// name appears: root datasource name, root relation connection, view datasource +// name, datasource-dependencies datasource = at least 4 references. +const refCount = wired.split(`'${connectionName}'`).length - 1; +if (refCount < 4) { + die(`Expected the connection name to appear >=4 times, saw ${refCount} — wiring incomplete.`); +} + +writeFileSync(twbPath, wired, 'utf8'); +console.error(`✓ Wired datasource '${caption}' (${connectionName}) with ${fields.length} field(s) into ${twbPath}`); +console.log(twbPath);