Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
217 changes: 209 additions & 8 deletions README.md

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,14 @@ The server version and token features are probed once on `auth login` / `auth li

To find the right `minVersion` for a new endpoint, validate against `../metabase` at `origin/release-x.58.x` — route files `src/metabase/api_routes/routes.clj` and `enterprise/backend/src/metabase_enterprise/api_routes/routes.clj`. Token-feature keys are the underscored map keys in `src/metabase/premium_features/settings.clj`.

## Worktree scope and pinning

A git-sync worktree is a self-contained checkout of one branch's content in the same app DB, tagged with `worktree_id`; the main app is `worktree_id = null`. Every command therefore runs either against the main app or inside exactly one worktree, and `defineMetabaseCommand` makes each one say which it tolerates: `worktree: "scoped"` honours a resolved scope, `"any"` is indifferent to one, and `"main-only"` changes or runs main-app state and refuses to execute while a scope is in force. The field is required, so a new command cannot silently inherit a classification.

A scope comes from `--worktree <id|branch>`, then `MB_WORKTREE`, then the profile's pin (`worktree: { id, branch }` on the profile record). A pin is a lock rather than a default: while it stands, the flag and the env var may only re-state it, and naming a different worktree is a `ConfigError`. The server has no per-worktree credential, so the CLI's isolation boundary is the credential set the process can reach: a harness gives an agent an `XDG_CONFIG_HOME` holding one pinned profile and no `MB_URL`/`MB_API_KEY`, and every command that process can run is then confined to that worktree — main-app writes refuse, and reads and writes of scoped resources carry the scope.

Both the lock and the main-only refusal are enforced in `getClient()`, before the transport is used, and they need only the profile record and the environment to decide. Resolving a scope _ref_ into an id/branch pair may cost a request, so `ctx.getWorktree()` is memoised and returns the pin's own pair without asking the server.

## The e2e stack

The suite drives the built binary against a real Metabase in docker compose, with no mocks. `scripts/e2e-matrix.ts` namespaces the compose project and its volumes per stack, so each matrix entry gets its own server, app-db volume, bootstrap artifact and snapshot. `bun run e2e:up` sets no project name and therefore always targets the default stack's containers, whatever `METABASE_CLI_E2E_STACK` says — reach for `e2e:matrix --stack=<id>` when you need a genuinely separate server.
Expand Down
28 changes: 15 additions & 13 deletions packages/cli/skill-data/core/SKILL.md

Large diffs are not rendered by default.

92 changes: 91 additions & 1 deletion packages/cli/skill-data/git-sync/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: git-sync
description: Round-trip Metabase content (cards, dashboards, transforms, snippets, collections, Library-published table/field metadata) between an instance and a git remote via `mb git-sync …` — status, dirty / has-remote-changes checks, import, export (with branch guard), branches, stash, add/remove a collection from sync. Load when the user wants to "import the latest changes", "export to git", "push my changes to the repo", "open a PR with my Metabase changes", "git sync", "dirty check", "stash before pulling", "add a collection to sync", or anything `mb git-sync …`.
description: Round-trip Metabase content (cards, dashboards, transforms, snippets, collections, Library-published table/field metadata) between an instance and a git remote via `mb git-sync …` — status, dirty / has-remote-changes checks, import, export (with branch guard), branches, stash, add/remove a collection from sync. Load when the user wants to "import the latest changes", "export to git", "push my changes to the repo", "open a PR with my Metabase changes", "git sync", "dirty check", "stash before pulling", "add a collection to sync", or anything `mb git-sync …` / `mb worktree …`. Also covers worktrees — isolated per-branch checkouts of transforms and other content — for "work on this branch in isolation", "create a worktree", "pin the profile to a worktree", "export the worktree and open a PR".
allowed-tools: Read, Write, Edit, Bash, AskUserQuestion
---

Expand Down Expand Up @@ -40,6 +40,7 @@ Pulls the configured branch and applies it to the instance. Polls until the task
| `--branch <name>` | Defaults to the `remote-sync-branch` setting; override per-call. |
| `--no-wait` | Return as soon as the task is queued; combine with `mb git-sync wait` later. |
| `--force` | **Discards local Metabase-side dirty changes** (lossy). Confirm with the user first. |
| `--merge` | Three-way merge remote changes instead of failing on divergence. Confirm first. |
| `--timeout <ms>` | Polling deadline. Default 600 000. |
| `--interval <ms>` | Polling cadence. Default 2 000. |

Expand All @@ -62,6 +63,7 @@ Pushes Metabase-side changes back to the configured remote. `-m` is the commit m
| `--branch <name>` | Push to a specific branch instead of the configured one. |
| `-m, --message <s>` | Commit message. |
| `--force` | Force-push / overwrite remote. Confirm with the user. |
| `--merge` | Three-way merge instead of failing on divergence. |
| `--no-wait` | Don't poll. |

Workflow:
Expand Down Expand Up @@ -96,6 +98,94 @@ mb git-sync stash --profile <n> # export current state

`stash` is the safe move when the instance has team work you don't want to lose, but you need to pivot to a different branch (`import` would discard, `export --force` would overwrite). It exports current state to a fresh branch first.

## Worktrees (isolated branch checkouts)

A **worktree** is a self-contained checkout of one branch's content inside the same instance. Transforms, transform tags, snippets, `transforms`-namespace collections, the Library, cards, dashboards, and documents are tagged with the worktree's id; tables and fields stay shared with the main app. It is how a chain of transforms gets built, exported, and reviewed as a PR without touching production content. Admin-only, and needs Metabase v64+ with the `remote_sync` feature.

Two hard limits, up front:

- **A worktree's transforms cannot be run.** `mb transform run` refuses while a worktree scope is in force, and the server rejects the run anyway. There is no workaround to find — run the transform in the main app after the branch is merged and imported, and say so rather than trying to validate the SQL by running it in the worktree.
- **A worktree is bound to its branch for life.** No branch switching inside one; a different branch means a different worktree.

### Where am I? Check before touching import/export

```bash
mb auth status --profile <n> --json | jq '.worktree' # {id, branch} when the profile is pinned, else null
mb git-sync status --profile <n> --json | jq '.worktree' # the scope this command ran under
mb worktree list --profile <n> --json # every worktree on the instance
```

A pinned profile pushes to the worktree's branch; an unpinned one pushes to whatever `remote-sync-branch` points at. Confusing the two is how worktree work lands on the team's tracked branch, so read the pin before the first `export`.

### Preconditions before creating one

1. **The user named the branch.** Create a worktree only for a branch the user asked you to work on.
2. **A branch holds at most one worktree.** Check `mb worktree list --json` first; a duplicate fails with the server's `A worktree for branch '<b>' already exists.` — reuse it with `mb worktree pin <branch>` instead.
3. **A pinned profile is already confined.** `mb worktree create` for any other branch is refused, because a second worktree from a pinned session would escape the pin.

### The workflow

```bash
# 1. Create the worktree and pin the profile to it. Mints the branch on the remote when missing
# (--no-checkout, so the main app's tracked branch is untouched), then pulls the branch in.
mb worktree create feat/order-metrics --pin --profile <n> --json

# 2. Edit transforms. Scoped commands need no extra flag once the profile is pinned.
mb transform create --file ./.scratch/transform.json --profile <n> --json
mb transform list --profile <n> --json # only this worktree's transforms
mb transform update <id> --file ./.scratch/patch.json --profile <n> --json

# 3. Read state before pushing — same rule as the main app.
mb git-sync status --profile <n> --json # branch, dirty flag, current task, all scoped
mb git-sync dirty --profile <n> --json # exactly what will be committed

# 4. Dry-run the push, read the answer, then push.
mb git-sync export-preflight --profile <n> --json
mb git-sync export -m "add order metrics transforms" --profile <n>

# 5. Open the PR from the branch with plain git / gh, and let a human review and merge it.

# 6. Retire the worktree once the PR is merged. This clears the pin it was holding.
mb worktree delete feat/order-metrics --profile <n> --json
```

`export-preflight` answers `{has_changes, clean, conflicts, summary: {added, updated, removed}, force_push_casualties: {deleted, overwritten}, reason}`. Read it before every worktree export: `clean: true` with empty `conflicts` is a push that applies as-is; a non-empty `conflicts` or `force_push_casualties` is a conversation with the user, not a `--force`.

The main app pulls the merged branch from a **different, unpinned** profile — `git-sync import` into the main app and `transform run` are both main-app operations and a pinned profile refuses them:

```bash
mb git-sync import --profile <main-app-profile>
mb transform run <id> --wait --profile <main-app-profile> --json
```

### Scope precedence and what a pinned profile refuses

The scope comes from `--worktree <id|branch>`, else `MB_WORKTREE`, else the profile's pin. **The pin is a lock, not a default:** while it stands, the flag and the env var may only re-state it, and naming a different worktree exits 2 with `profile "<p>" is pinned to worktree <id> (<branch>); refusing --worktree <x>`.

Under a scope, a command that changes or runs main-app state refuses before any request and exits 2 — `transform run` / `cancel`, `card` / `library` writes, and the main-app git-sync verbs `stash`, `create-branch`, `add-collection`, `remove-collection`:

```
transform run is not available inside a worktree (scope: worktree 3 (feat/order-metrics) from the profile pin); it changes main-app content. Unpin the profile (`mb worktree unpin`) or drop MB_WORKTREE to run it against the main app.
```

Two more refusals belong to the scope:

- Fetching a row that lives elsewhere: `transform 12 is not in worktree 3 (feat/order-metrics); refusing to touch main-app content`.
- `--branch` on `import` / `export` / `export-preflight`: `a worktree is pinned to its branch; drop --branch`.

### Confirmations these flags need

- `--force` (on `import`, `export`, and `worktree delete`) and `--merge` (on `import` / `export`) are **lossy or history-rewriting** — ask the user first, with `AskUserQuestion`, naming what gets discarded. `export-preflight`'s `force_push_casualties` is the list to show them.
- `mb worktree delete` refuses a worktree holding unpushed changes: `worktree <id> (<branch>) has unpushed changes; push them with mb git-sync export or pass --force to discard`. Export first, or ask before forcing — the content in a worktree exists nowhere else once it is gone.

### Don't (worktrees)

- **Don't unpin to reach the main app.** The pin is the isolation boundary a harness put there; `mb worktree unpin` to get past a refusal turns an isolated session into one that can write production content. Report the refusal and what you would need instead.
- **Don't run a worktree's transforms**, and don't route around the refusal with `mb query`, a native card, or a second profile. Validate the query shape with `--dry-run` (see `mbql`) and run it for real in the main app after the merge.
- **Don't create worktrees for branches you were not asked to touch.** A worktree checks a whole branch's content into the instance; an unwanted one is content to clean up, not a free experiment.
- **Don't `git-sync export` from the main app while the work lives in a worktree.** It pushes the main app's state to the tracked branch and the worktree's edits are not in it. Check `.worktree` in `git-sync status` before exporting.
- **Don't hand-write the branch's YAML in the repo to "help" the PR.** Worktree content round-trips through `export` exactly like main-app content — see the first entry of the general "Don't" list below.

## Polling and cancelling

```bash
Expand Down
10 changes: 9 additions & 1 deletion packages/cli/skill-data/transform/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ mb transform run "$TRANSFORM_ID" --wait --profile <name> --json
- `--wait` polls until status is `succeeded` or `failed`. Without it you get only `{message: "Transform run started", run_id, final: null}` and must poll yourself — don't put bare `transform run` in a tight loop; let `--wait` do the polling.
- `--sync` implies `--wait`, then waits until the run registers its output table (the run registers it itself — no `db sync-schema` needed), adding `target_table_id` to the envelope. Use it when you'll build MBQL on the output (see "Inspect").
- The `--json` envelope is shape-stable: `{message, run_id, final}` (plus `target_table_id` under `--sync` — a number, or `null` if the table didn't register before the timeout). `final` is `null` when `--wait` is omitted or the run never started, otherwise a full `TransformRun` with `status` and `message`. On a failed run (`final.status` ∈ {`failed`, `timeout`, `canceled`}) the CLI exits 1 and writes a one-line `transform run <id> failed` to stderr; the failure detail lives only in `final.message` on stdout, so `jq -r '.final.message'` is where to look.
- `transform create --json` returns the agent-facing compact projection: `{id, name, description, source_type, target: {type, database, schema, name}, target_db_id}`. Read `target.schema`/`target.name` directly off it — no follow-up `transform get`.
- `transform create --json` returns the agent-facing compact projection: `{id, name, description, source_type, target: {type, database, schema, name}, target_db_id, worktree_id}`. Read `target.schema`/`target.name` directly off it — no follow-up `transform get`.
- If a transform with the same `name` already has a YAML representation on disk under the configured remote-sync repo, `create` mints a `_2` suffix on the exported filename (the new transform gets a fresh `entity_id`; the prior one isn't touched). For "iterate on the same concept", prefer `transform update <id>` — see "Iterating on a failing transform".
- **`collection_id` only accepts a collection in the `:transforms` namespace.** Transforms aren't filed next to cards and dashboards — a normal analytics collection id fails create/update with `collection_id: A Transform can only go in Collections in the :transforms namespace.` Omit `collection_id` to leave the transform uncollected (the common case), or provision one with `mb collection create --body '{"name":"…"}' --namespace transforms --json` (see `core`) and pass the returned `id`. Cards and dashboards you build **on top of** the output table go in ordinary collections — so "put the transform and its dashboard in collection X" means _X holds the dashboard + cards; the transform stays in the transforms namespace._

Expand All @@ -71,6 +71,14 @@ On `target_table_id: null` (still syncing when the poll timed out; exit 0) re-po

Columns and types are inferred from the result set; change the SELECT shape and the next run fails on a column mismatch — drop the table first (`transform delete-table <id>`). A changed shape also needs a re-run with `--sync` before MBQL sees the new/renamed columns.

## Editing transforms safely in a worktree

A **worktree** is an isolated checkout of one git branch's content inside the same instance — the way to build a chain of transforms for PR review without touching production transforms or their tables. The lifecycle (create → pin → export-preflight → export → PR → delete) lives in the `git-sync` skill, section "Worktrees": **`mb skills get git-sync`**. What matters while authoring:

- **`worktree_id` says where a transform lives.** `transform get` and `transform list` carry it: an integer for a worktree's transform, `null` for a main-app one. Read it before editing anything you did not create in this session — under a scope, `update` / `delete` / `delete-table` of a main-app transform is refused with `transform <id> is not in worktree <n> (<branch>); refusing to touch main-app content`.
- **The scope is implicit once the profile is pinned.** With `--worktree <id|branch>`, `MB_WORKTREE`, or a pin in force, `transform list` shows only that worktree's transforms and `transform create` files what it creates into it. Without a scope both address the main app, whose listing excludes every worktree's rows.
- **`transform run` is refused under a scope**, and the server rejects worktree runs anyway: `transform run is not available inside a worktree (…); it changes main-app content.` The create → run → fix loop therefore does not close inside a worktree. Validate the query with `--dry-run` (see `mbql`) before creating, iterate the body with `transform update`, and run it in the main app once the branch is merged and imported. Don't unpin, don't switch to another profile, and don't stand in a run with `mb query` — tell the user the run waits for the merge.

## Inspect runs and cancel an in-flight run

```bash
Expand Down
12 changes: 7 additions & 5 deletions packages/cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ import { hoistGlobalFlags } from "./commands/global-flags";
import { trustSystemCa } from "./core/system-ca";
import main from "./main";
import { reportError } from "./output/error";
import { findUnknownCommand, resolveBreadcrumb, showUsage, showUsageJson } from "./output/help";
import { resolveInvocation, showUsage, showUsageJson } from "./output/help";
import { installInterruptHandler } from "./runtime/interrupt";
import { setVerbChain } from "./runtime/verb-chain";

const HELP_FLAGS: ReadonlySet<string> = new Set(["--help", "-h"]);
const JSON_HELP_FLAG = "--json";
Expand All @@ -24,7 +25,7 @@ async function run(): Promise<void> {
cmd: CommandDef<T>,
parent?: CommandDef<T>,
): Promise<void> => {
const breadcrumb = await resolveBreadcrumb(main, rawArgs);
const { breadcrumb } = await resolveInvocation(main, rawArgs);
if (wantsJsonHelp) {
await showUsageJson(cmd, breadcrumb);
return;
Expand All @@ -37,11 +38,12 @@ async function run(): Promise<void> {
return;
}
if (!rawArgs.some((arg) => HELP_FLAGS.has(arg))) {
const unknown = await findUnknownCommand(main, rawArgs);
if (unknown !== null) {
reportError(new ConfigError(`unknown command: ${unknown}`));
const { unknownToken, verbs } = await resolveInvocation(main, rawArgs);
if (unknownToken !== null) {
reportError(new ConfigError(`unknown command: ${unknownToken}`));
return;
}
setVerbChain(verbs);
}
await runMain(main, { showUsage: showUsageWithBreadcrumb, rawArgs });
}
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/commands/alert/archive.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export default defineMetabaseCommand({
details:
"Deactivates the alert and drops its scheduled trigger. `mb alert list --include-inactive` still shows it, and `mb alert update <id> --body '{\"active\":true}'` brings it back.",
capabilities: { minVersion: 58 },
worktree: "main-only",
args: {
...outputFlags,
...profileFlag,
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/commands/alert/create.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export default defineMetabaseCommand({
'each recipient is `{type: "notification-recipient/user", user_id}` or `{type: "notification-recipient/raw-value", details: {value: "a@b.com"}}`.',
].join(" "),
capabilities: { minVersion: 58 },
worktree: "main-only",
args: {
...outputFlags,
...profileFlag,
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/commands/alert/get.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export default defineMetabaseCommand({
details:
"`--full` includes the hydrated card the alert watches, alongside its schedules and handlers.",
capabilities: { minVersion: 58 },
worktree: "any",
args: {
...outputFlags,
...profileFlag,
Expand Down
Loading
Loading