diff --git a/docs/kane-cli-assurance-automation.md b/docs/kane-cli-assurance-automation.md index 5514bce70..b1a6547c9 100644 --- a/docs/kane-cli-assurance-automation.md +++ b/docs/kane-cli-assurance-automation.md @@ -168,11 +168,11 @@ extract: no TTY — pass an explicit --mode agent|ci|override to run headless | Mode | Questions | stdout | |---|---|---| | `interactive` | asked in the chat (TTY default) | the Ink chat UI | -| `agent` | low/medium-risk defaults are auto-taken (each reported); a **high-risk** question pauses the session — exit `3`, resumable | **NDJSON events** (one JSON object per line); prose diagnostics go to stderr | -| `ci` | any high-risk question **fails closed** — exit `1`, error code `HIGH_RISK_CI` | prose transcript | +| `agent` | **every** question pauses the session — exit `3`, resumable; the agent that drives the run answers at resume or escalates to its person | **NDJSON events** (one JSON object per line); prose diagnostics go to stderr | +| `ci` | any high-risk question **fails closed** — exit `1`, error code `HIGH_RISK_CI`; low- and medium-risk questions take their recommended default (each reported) | prose transcript | | `override` | every default is auto-taken, including high-risk (each flagged in the commit record) | prose transcript | -Rule of thumb: `agent` when something can read the pause and answer (an AI agent, a human on the next shift); `ci` when a pipeline must never guess; `override` when you accept the recommended defaults wholesale and want one unattended pass. +Rule of thumb: `agent` when something can read the pause and answer (an AI agent, a human on the next shift); `ci` when a pipeline must fail rather than guess on anything high-risk; `override` when you accept the recommended defaults wholesale and want one unattended pass. The same matrix drives `maintain reconcile`, with two reconcile-specific rules: no headless mode ever archives anything — ARCHIVE decisions wait for an interactive session — and a `ci`-mode run that hits a decision needing a human **stores the plan and exits `2`** (the work isn't lost; walk the stored plan interactively or apply it in `agent` mode). @@ -202,7 +202,9 @@ With `--mode agent`, stdout speaks a versioned NDJSON vocabulary — envelope `{ | `corpus` | extract: the `sources[]` this run covers + already-extracted `skipped[]` | | `source_start` / `source_skipped` | `source_id`, `index`/`total`, `resumed` / `reason` | | `plan` | the `--plan` transcription payload | -| `assumed_default` | a question auto-answered with its recommended default: `id`, `selected_index`, `risk` | +| `assumed_default` | a question auto-answered with its recommended default: `id`, `selected_index`, `risk`. `ci` and `override` runs only, since an `agent` run pauses instead | +| `variables_declared` | design: the stubs a phase commit wrote — `file` (the pool file) and `variables[]` (`name`, `description`, `secret`). Only names that existed in no variable file, and it fires after the `commit` that minted the tests | +| `variables_summary` | design, at the end of the run: every name still needing a value, in the same shape as `variables_declared` | | `agent_activity` | progress: `kind` (`tool` / `decision` / `progress` / `thinking_done`) + a display `label` | | `agent_message` *(0.7.2)* | the agent's narrative `text` — the conversational lead-in before a question batch and the closing statement at the end of a run | | `warning` *(0.7.2)* | an actionable non-fatal condition: `code` (`ZERO_USE_CASES`, `SAVE_FAILED`) + `message` | @@ -334,6 +336,31 @@ The rule stands: there is no auto-approve. These paths land **your** decisions f ## Coverage on the stream *(0.7.1)* +## The sync verbs on the stream {#the-sync-verbs-on-the-stream} + +`kane-cli context sync`, `kane-cli context push`, `kane-cli context pull`, `kane-cli context clone` and the subcommands `kane-cli context sync add`, `kane-cli context sync list`, `kane-cli context sync remove`, `kane-cli context sync status` and `kane-cli context sync doctor` all take `--mode agent` and speak the same strict envelope with `verb: "sync"`: stdout is NDJSON only, stderr stays empty, and `done` is last. None of them calls the agent or spends credits. `kane-cli context sync setup` is the one command that needs a terminal, and under `--mode agent` it answers `error{code: TTY_REQUIRED}` naming `kane-cli context sync add` and `kane-cli context clone` as the alternatives. What these commands do is in [Sharing the context graph with your team](/support/docs/kane-cli-assurance-sharing/). + +| type | payload highlights | +|---|---| +| `sync_probe_started` / `sync_probe` | `kane-cli context sync add` and `kane-cli context clone`, around the location check (the slow part on a first GitHub fetch): `name`, `kind`, then `tier` (`1` means the location can publish, `2` or `3` download only) and `detail` | +| `sync_status` | `kane-cli context sync status`: `name`, `relation` — `kind` (`up-to-date`, `behind`, `ahead`, `diverged`, `empty-storage`, `empty-local`, `foreign-lineage`), `local` and `storage` (each `{seq, hash}` of that side's last record, or null when that side is empty) and, on `diverged`, `at` (the first record number where the two sides differ, so the last shared record is `at - 1`) — then `rebase_id` (the open rebase, or null) and `decisions[]` (each in the `sync_rebase_decision` shape). The event needs the location: when it cannot be reached the command refuses instead, and the decisions are not listed | +| `sync_status` with `--show ` | needs no location: `record[]` (the saved record in full, one line each), `decision`, `rebase_id`, `decisions[]`, while `name` and `relation` are null | +| `sync_locations` / `sync_removed` | `kane-cli context sync list` / `kane-cli context sync remove` | +| `sync_pull_done` | `kane-cli context pull`, `kane-cli context sync`, `kane-cli context clone`: `name`, `imported`, `from`, `to`, `blobs`, `proposals`, and `clone` adds `dir` | +| `sync_push_done` | `kane-cli context push`, `kane-cli context sync`: `name`, `from`, `to`, `pushed`, `blobs`, `proposals`, `already_there` | +| `sync_rebase_started` | `kane-cli context pull --rebase --yes`: `rebase_id`, `from`, `backup_path`, `moved[]`, `quarantined_tests[]` | +| `sync_rebase_item` | one per saved record: `seq`, `intent` (`extract`, `review`, `names`, `retire` and so on), `outcome` (`reapplied`, `already-present`, `not-reapplied`, `decision`), `reason?` | +| `sync_rebase_decision` | one per open decision, in dependency order: `decision_id`, `kind`, `intent`, `seq`, `label`, `location`, `mine` and `theirs` (one line per side, the local change and the location's), `answers[]` (only the legal ones, each `{answer, consequence}`) | +| `sync_rebase_done` | once per walk, and once when `kane-cli context sync doctor --abort` closes a readable rebase: the four counts, `decisions_open`, `status` (`complete`, `paused`, `aborted`) | +| `sync_rebase_open` | `kane-cli context push` or `kane-cli context sync` while a rebase is still open, followed by `sync_error{SYNC_REBASE_PENDING}` | +| `sync_doctor` / `sync_rebase_exported` | `kane-cli context sync doctor` / `kane-cli context sync doctor --export` | +| `sync_behind` | the advisory from `kane-cli context extract`, `kane-cli design tests` and `kane-cli maintain reconcile` when a teammate has pushed since you pulled: `name`, `local_seq`, `storage_seq`, `text`. Nothing is refused | +| `gitignore_updated` | `kane-cli context clone` and `kane-cli context sync doctor --export` (and `kane-cli context ingest` on its `extract` stream) when the new store's folder is inside a git repository and `.context/` was added to its `.gitignore`: `path`. When the line could not be written, the store is still created and a `warning{message}` says why | + +Exit codes keep their meanings, with one addition: exit `3` is also **a person has to decide something**, which covers a `kane-cli context push` or `kane-cli context pull` refused because you are behind or diverged (the remedy names the command), and a rebase that stopped on open decisions (answer them with `--answer`, or on a terminal). Exit `2` is a precondition (a location that cannot be reached, a rebase still open, missing keys), and exit `1` a record that cannot be used. A refusal says what stopped, not that nothing happened. + +## Coverage on the stream + `cover --mode agent` and `cover gaps --mode agent` speak the same envelope (`verb: "cover"` / `"gaps"`): the full `--json` payload arrives as **one** `coverage` (or `gaps`) event — *(0.8.2)* `cover gaps ` emits the document closed over that use-case — and `done` closes the stream carrying the worklist's ready-to-paste commands in `next[]`. `--mode ci` speaks the identical stream. Any refusal is an `error` event + `done` with exit `2`. ## When releases don't match diff --git a/docs/kane-cli-assurance-context.md b/docs/kane-cli-assurance-context.md index 18fc6e2be..320b91218 100644 --- a/docs/kane-cli-assurance-context.md +++ b/docs/kane-cli-assurance-context.md @@ -512,7 +512,8 @@ Freshness is orthogonal: `fresh` / `stale` (the source snapshot moved) / `orphan ``` .context/ -├── meta.json # store identity + format version +├── meta.json # format version (schema and canonicalizer) +├── store.json # this store's own identity ids ├── commits/ # append-only records — the truth ├── blobs/ # write-once source snapshots ├── derived/ # regenerable read caches (delete any time; rebuild restores) @@ -525,7 +526,11 @@ Freshness is orthogonal: `fresh` / `stale` (the source snapshot moved) / `orphan └── signals.ndjson # internal review bookkeeping (appears once recorded) ``` -Two rules worth repeating from the [overview](/support/docs/kane-cli-assurance/#the-store-context): the store is **single-writer**, and it is **not git-mergeable** — gitignore it and share by re-ingesting sources. +Two rules worth repeating from the [overview](/support/docs/kane-cli-assurance/#the-store-context): the store is **single-writer**, and it is **never merged with git**. Kane CLI adds `.context/` to your `.gitignore` when it creates the store inside a git repository: `kane-cli context ingest`, `kane-cli context clone` and `kane-cli context sync doctor --export` all do it, and say `added .context/ to .gitignore` once. When the line cannot be written, the store is still created and `warning: could not add .context/ to .gitignore: ` says why, so add the line yourself. Set `KANE_CONTEXT_GITIGNORE=0` to keep it out. + +To share the store with your team, bind it to a **location**, a GitHub repository, an S3-compatible bucket, or a folder on a shared drive, and use `kane-cli context push`, `kane-cli context pull`, `kane-cli context sync` and `kane-cli context clone`. See [Sharing the context graph with your team](/support/docs/kane-cli-assurance-sharing/). A teammate who has pulled your records has the same use-cases, cited lines and review verdicts you committed, and nothing needs to be re-ingested. + +`.context/sync/` holds what sharing adds: the location list (no secret in it), a backup and a receipt for every rebase, and the location's on-disk state for this machine. ### Tracing a run @@ -535,11 +540,12 @@ Every extract run also writes a per-run trace to `.context/logs/extract-.log ## For agents and CI -Headless extraction (`--mode agent|ci|override`), the NDJSON event stream, exit codes, and the pause/resume contract are documented in [Automation](/support/docs/kane-cli-assurance-automation/). +Headless extraction (`--mode agent|ci|override`), the NDJSON event stream, exit codes, and the pause/resume contract are documented in [Agents and CI](/support/docs/kane-cli-assurance-automation/). A pipeline that works on a shared store clones it once, pulls before each run and pushes after: the sync verbs speak the same stream, see [the sync verbs on the stream](/support/docs/kane-cli-assurance-automation/#the-sync-verbs-on-the-stream), and the CI shape is in [CI/CD](/support/docs/kane-cli-cicd/#a-shared-context-store-in-ci). ## Next steps - [Requirement sources](/support/docs/kane-cli-assurance-sources/) — every accepted file type and remote URL. - [Designing tests](/support/docs/kane-cli-assurance-design/) — turn a trusted use-case into ACs, scenarios, and runnable tests. - [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/) — what to do when a source changes. +- [Sharing the context graph with your team](/support/docs/kane-cli-assurance-sharing/) — one location; publishing, taking your teammates' records, cloning, and what happens when two people change the same thing. - [Automation](/support/docs/kane-cli-assurance-automation/) — the headless contract. diff --git a/docs/kane-cli-assurance-sharing.md b/docs/kane-cli-assurance-sharing.md new file mode 100644 index 000000000..c34b5a62b --- /dev/null +++ b/docs/kane-cli-assurance-sharing.md @@ -0,0 +1,405 @@ +--- +id: kane-cli-assurance-sharing +title: Sharing the context graph with your team +sidebar_label: Sharing with Your Team +description: "Share a .context store with your team through a GitHub repository, an S3-compatible bucket or a shared folder: kane-cli context push, pull, sync, clone, and the decisions when two people change the same thing." +keywords: + - kane cli context sync + - kane cli context push + - kane cli context pull + - kane cli context clone + - shared context graph + - assurance + - testmu ai +url: https://www.testmuai.com/support/docs/kane-cli-assurance-sharing/ +site_name: TestMu AI +slug: kane-cli-assurance-sharing/ +canonical: https://www.testmuai.com/support/docs/kane-cli-assurance-sharing/ +--- +import VerifiedTag from '@site/src/component/verifiedTag'; + + + + +`kane-cli context sync` shares your `.context/` store with your team through a **location**: a GitHub repository, an S3-compatible bucket, or a folder on a shared drive. Every person keeps their own store on their own machine, and the location holds the team's shared history. `kane-cli context push` publishes your new records, `kane-cli context pull` takes your teammates', and `kane-cli context clone` makes a new store from the location. Nothing on a location is ever overwritten or deleted, so a published record can never be un-published by mistake. + +Sharing starts from a store you already have. `kane-cli context ingest --mode ci` creates one from a first document, see [Building the context graph](/support/docs/kane-cli-assurance-context/). Then: + + + +```bash +kane-cli context sync setup # guided: share this store, or join your team's +kane-cli context sync origin # every day after that: take the team's new records, publish your own +``` + +:::warning +Two rules stand behind every command here: **never merge `.context/` with git**, and never edit it by hand. Sharing goes through a location, and only through the commands on this page. +::: + +## Three kinds of location {#locations} + +| Kind | Address you type | How you sign in | Who can read it | +|---|---|---|---| +| **GitHub** (recommended) | `https://github.com//.git` or `git@github.com:/.git`; add `#` for a branch other than `main`, `?prefix=` to share one repository between stores | the Git sign-in you already have (SSH key, or HTTPS through a credential helper); CI uses `KANE_SYNC_GIT_TOKEN` or a deploy key | whoever the repository's visibility and permissions allow, everyone if the repository is public | +| **S3-compatible** (AWS S3, MinIO, R2 and similar) | `s3:///?region=&endpoint=`, where `prefix`, `region` (default `us-east-1`) and `endpoint` (only for a non-AWS service) are optional | an access key pair, passed with `--credential-env` or `--credential-file` | everyone granted the bucket in the storage console | +| **Folder** | a path: `/Volumes/team/context` or `../shared/team-context` | none, the file system's own permissions | everyone who can open the folder | + +**GitHub.** Use a **dedicated repository**, not your code repository, and make it private if the requirements are. Kane CLI needs Git 2.31 or newer on the machine. Setup and `kane-cli context sync add` check the repository by writing a few small permanent connection-check files and commits; they never publish your context. The check also pushes to a scratch reference under `refs/kane/probe/`. When that check cannot finish (`SYNC_PROBE_INCONCLUSIVE`), repository rules that block the reference are the usual cause: ask the repository owner to allow it, then run the command again. Each push is one ordinary commit on the branch, and nothing is force-pushed, merged or rebased on the server. GitHub does not accept a single object above 100 MB, so when a source document is larger than that, choose an S3-compatible bucket. For HTTPS without a stored login, the GitHub CLI does it in two commands: `gh auth login --hostname github.com --git-protocol https --web`, then `gh auth setup-git --hostname github.com`. Another Git server works the same way with that server's sign-in. + +**S3-compatible.** Ask your storage administrator for an existing bucket and a key pair; Kane CLI creates neither. `--credential-env TEAM_S3` reads `TEAM_S3_ID` and `TEAM_S3_SECRET`, and `--credential-file keys.json` reads `{"accessKeyId": "…", "secretAccessKey": "…"}`. Either way the pair is saved in `~/.testmuai/kaneai/context-sync/.json`, readable by you only (mode `0600`), and `kane-cli context sync remove ` deletes it. That file belongs to the location's *name*, not to one store: every store on this machine whose location is called `origin` reads the same `origin.json`, so binding a second bucket as `origin` from another store replaces the keys the first one uses. When one machine works with more than one bucket, give each its own name (`team-s3`, `archive`). `KANE_SYNC_S3_ACCESS_KEY_ID` and `KANE_SYNC_S3_SECRET_ACCESS_KEY`, both set, win over the file at use time and are never written to disk, which is the CI form. A key pair the location refuses never replaces one that worked. + +```bash +kane-cli context sync add origin "s3://team-context/assurance?region=eu-west-1" --credential-env TEAM_S3 +kane-cli context sync add origin "s3://team-context/assurance?endpoint=https://minio.example.internal:9000" --credential-file ./keys.json +``` + +**Folder.** A mounted shared drive serves a team; a path on your own disk serves only you, since sending the path does not send its contents. A folder that does not exist yet is created by `kane-cli context sync add` when its parent exists. + +## Guided setup {#setup} + +```bash +kane-cli context sync setup +``` + +Setup needs a terminal: headless it refuses with `TTY_REQUIRED` and names `kane-cli context sync add` and `kane-cli context clone` as the alternatives. It never publishes your context, since publishing is always the separate `kane-cli context push`. The first screen asks **What do you want to do?**: **Share my context with my team** or **Join my team's context**. + +**Share.** Setup asks where the shared context should live, **GitHub — Recommended**, **S3-compatible storage** or **Folder**, then for that kind's details. GitHub: the address of an existing repository, or a row that opens GitHub in the browser so you create a dedicated repository there and come back with its address; missing Git or GitHub CLI shows an installation link and a retry row. S3: the bucket name, the region (prefilled `us-east-1`), an optional folder inside the bucket, the service (Amazon S3, or another S3-compatible service, which then asks for its endpoint), and how to reach it, either credentials already configured on this computer or the access keys from your storage administrator typed in (the secret access key is masked) and saved in your local Kane CLI credential store, never in the shared context. Folder: the path, and the screen reminds you that everyone needs the same mounted drive. + +The location is bound under the name `origin`. If this store already has a location called `origin`, setup asks for another name and suggests `team`. The last screen shows the address, says that the connection check adds small permanent files (and, for a repository, commits) to the location, and waits for **Connect location**. Setup then reports whether you can publish: with write access it shows the `kane-cli context push ` line to run when you are ready, otherwise it asks you to obtain write access and check the connection again. Then it prints `Share this location with teammates:` followed by the complete address (for a Git location, the repository address with the branch and prefix you chose), and one closing line saying who can read it. After a refusal, **Use a different address** re-enters the same prompts with what you typed kept, and **Choose a different storage** returns to the picker. Esc or Ctrl+C leave setup with exit `3`: *Setup left. Your context has not been published. Any completed connection checks remain in the location.* + +**Join.** Setup asks you to **Paste the location your teammate shared**, then for a new folder under **Download into a new folder** (prefilled `team-context`; a folder that already exists is refused, so none of your files is replaced), then waits for **Connect and download**. It downloads the verified team context into that folder and closes with `Next: open that folder and run kane-cli context sync status.` Join never creates a location: an address that holds no records yet is refused until the teammate who shared it has pushed. `kane-cli context clone
` run by hand is less strict about the folder: it accepts one that already exists as long as `/.context` is not there yet, which is how the CI recipe clones into `.`. + +## The commands {#commands} + +| Command | When to use it | What it prints | +|---|---|---| +| `kane-cli context sync setup` | the first time, on a terminal | the screens above | +| `kane-cli context sync add
[--credential-env \| --credential-file ]` | bind a location by hand (scripts, CI, a second location). Re-running with the same name replaces its address, and its S3 keys when you pass new ones, which is how you re-key | `added origin: dir tier 1 (all probes passed)`, and the line ends with what the checks found (the table below). A changed address also prints `dropped the witness at position 4: the location changed from … to …`, where the witness is the last record this machine saw on that location, and a new address starts without one | +| `kane-cli context sync list [--json]` | which locations this store knows | one line per location: `origin dir tier 1 …/shared/team-context`, meaning name, kind, what the checks found, address | +| `kane-cli context sync remove ` | forget a location; for an S3-compatible location its saved keys are deleted too | `removed origin`, then `kept its witness at position 4: re-adding the same storage keeps its rollback protection` when it had one | +| `kane-cli context sync status [name] [--show ] [--json]` | where this store stands against the location; never writes | one line: `up to date with origin: position 4 on both sides`, `behind origin: this store is at position 1, origin at 2 — run kane-cli context pull origin`, `ahead of origin: … — run kane-cli context push origin`, `origin is empty: nothing has been pushed there yet — run …`, `this store is empty; origin is at position 2 — run …`, or `you and origin both added work after position 2 — run …`, then any open decisions. `--show ` prints saved record `n` in full | +| `kane-cli context sync doctor [--abort] [--export [--from ]] [--json]` | what the store looks like after an interruption; close an open rebase; rebuild the pre-rebase store beside | the chain state and every rebase with its id | +| `kane-cli context push [name]` | publish your new records | `pushed 2 records (3..4), 1 blobs, 0 proposals`; with nothing new to publish, `up to date` | +| `kane-cli context pull [name]` | take your teammates' new records | `pulled 1 records (3..3), 0 blobs, 0 proposals` | +| `kane-cli context pull --rebase [--yes]` | you and the location both added work | the walk below | +| `kane-cli context sync [name] [--answer =]` | the everyday verb: finish an open rebase, pull, then push | the pull and push lines | +| `kane-cli context clone
[dir] [--credential-env \| --credential-file ]` | join: make a new store from a location, bound as `origin`. `dir` defaults to the location's last path segment (`team-context` for `../shared/team-context`), and the two flags pass an S3 key pair the same way `kane-cli context sync add` takes it | `cloned into …/bob: 2 records (1..2), 2 blobs, 0 proposals` | + +`[name]` defaults to the only location; with several, to the one called `origin`; otherwise the command asks you to name one. Every command takes `--mode agent` for an NDJSON stream, see [Agents and CI](/support/docs/kane-cli-assurance-automation/#the-sync-verbs-on-the-stream). + +`kane-cli context sync add` and `kane-cli context clone` check a location before binding it: that it can be read, and whether a small test write lands there and reads back. `kane-cli context sync add` prints `added : tier ()`, `kane-cli context clone` prints `bound origin: …` in the same shape, and each line of `kane-cli context sync list` repeats the kind and tier: + +| The line says | It means | +|---|---| +| `dir tier 1 (all probes passed)`; for a bucket, `s3 tier 1 (all probes passed)`; for a repository, `git tier 1 (Can download and publish; independent reads and concurrent writes verified.)` | **can publish**: the location could be read, and the test write landed and read back, so publishing, taking and cloning all work | +| `dir tier 3 (P1 exclusive create: …/ro-location cannot be written: this account has no write access)`; for a repository, `git tier 3 (This location can be downloaded, but publication was refused.)` | **download only**: the location could be read, but the test write was refused (no permission, a read-only mount, a read-only key pair), so clone and pull work and push refuses. `kane-cli context sync add` follows with `read-only: this location can be cloned and pulled, never pushed`, and `kane-cli context clone` with `pull-only: this location can be cloned and pulled, never pushed` | +| `tier 2` | **download only** as well: the checks did not prove everything safe publishing needs, and the detail in the line names the check that failed. `kane-cli context sync add` follows with `pull-only: this storage did not prove atomic create`, and `kane-cli context clone` with the same `pull-only:` line as tier 3 | + +Joining an S3-compatible location with keys from the environment: + +```bash +kane-cli context clone "s3://team-context/assurance?region=eu-west-1" team-context --credential-env TEAM_S3 +``` + +## The everyday loop {#loop} + +Alice shares a store she already built and Bob joins it. Then each of them works, pulls, and pushes. Alice and Bob are normally two machines; here they are two folders side by side, with the location as a third: + +``` +team/ +├── alice/ alice's store (.context/ inside) +├── bob-home/ bob's machine; his store lands in bob/ after the clone +└── shared/team-context the location +``` + +Alice runs her commands in `alice/`, Bob in `bob-home/`, and, once the clone exists, inside `bob/`. + +``` +[alice] $ kane-cli context sync add origin ../shared/team-context +checking origin: read access and safe publishing… +added origin: dir tier 1 (all probes passed) + +[alice] $ kane-cli context push origin +wrote kane-context/v1/meta.json on origin +pushed 1 records (1..1), 1 blobs, 0 proposals + +[bob] $ kane-cli context clone ../shared/team-context bob +checking origin: read access and safe publishing… +bound origin: dir tier 1 (all probes passed) +cloned into …/bob: 1 records (1..1), 1 blobs, 0 proposals + +[bob] $ cd bob +``` + +In the push and clone lines, *blobs* are the source-document snapshots the records cite and *proposals* the extraction artifacts behind them; both travel with the records, see [What travels](#travels). A proposal travels when its file exists, and a missing one never holds up a push or a pull. + +Bob ingests a second document and publishes it. Alice, who has not pulled yet, tries to push. In the output, `position 2` is the second record of the shared history, which this page calls record 2: + +``` +[bob] $ kane-cli context ingest brief.md --mode ci +created brief source sha256:f2e4… blob sha256:60af… +landed 1 source(s) — run kane-cli context extract to extract them + +[bob] $ kane-cli context push origin +pushed 1 records (2..2), 1 blobs, 0 proposals + +[alice] $ kane-cli context sync status origin +behind origin: this store is at position 1, origin at 2 — run kane-cli context pull origin + +[alice] $ kane-cli context push origin +error: origin holds 2 records, this store 1: it has records this store has not seen +next: kane-cli context pull origin + +[alice] $ kane-cli context pull origin +pulled 1 records (2..2), 1 blobs, 0 proposals + +[alice] $ kane-cli context sync origin +up to date +``` + +The habit that avoids almost every refusal: **pull before you start, push when you finish**, or just run `kane-cli context sync origin` at both ends. When a teammate has pushed since you last pulled, a command that writes to the store first prints one advisory line, `origin has moved past this machine — run kane-cli context pull origin`, or, when both sides moved, `this store and origin have diverged — run kane-cli context pull origin --rebase`, and carries on. The line is a best-effort courtesy, not a check you can rely on: it refuses nothing, it gives up silently after 1.5 seconds or when the location cannot be reached, and it is not printed under `--mode agent` (`kane-cli context extract`, `kane-cli design tests` and `kane-cli maintain reconcile` emit a `sync_behind` event there instead). `kane-cli context review` prints it unless `--json` or `--mode agent` is given, and a test run writes it to its session log. `KANE_SYNC_GUARD=0` turns it off. + +Ingest, extract, review and design all work the same way; the location only changes where records go afterwards. What a push sends, and what stays on your machine, is listed in [What travels](#travels). + +## What travels and what never travels {#travels} + +| Travels to the location | Never travels | +|---|---| +| every record of the graph (sources, use-cases, criteria, scenarios, tests, reviews, names, retirements, run results) | `.context/derived/`, the read caches, since every store rebuilds them from the records | +| the source documents those records cite (the snapshot bytes) | paused sessions, locks and per-run logs, which are one machine's state | +| the extraction proposals behind the records, when their files exist, and a missing one never blocks a push | your `_test.md` files, which are working files in your project, and the records already carry what a test is. A teammate who wants to run them takes them from the project's code checkout, and a rebase writes again the ones its reapplied records created | +| | your keys, the location list's local bindings, and the store's own identity ids | +| | everything under `.context/sync/`, meaning rebase backups, saved decisions and receipts, and the test results waiting on this machine to be recorded. A location never holds an unfinished rebase: finish or close it here | + +The sync commands never call the agent: **publishing, taking, cloning and a rebase spend no credits.** A store that pulls a use-case gets the same records the extractor committed, cited lines included. + +## When a command refuses {#refusals} + +Every sync refusal prints a plain reason, nearly all add a `next:` line with the command to run, and on the agent stream the same refusal is a `sync_error` event with a stable code, the reason and the remedy. The common ones: + +| You see | Code | It means | Next | +|---|---|---|---| +| `origin holds 2 records, this store 1: it has records this store has not seen` | `SYNC_BEHIND` | you are behind | `kane-cli context pull origin`, then push again | +| `you and origin both added work after position 2` | `SYNC_DIVERGED` | you and a teammate both worked from the same record | `kane-cli context pull origin --rebase` (add `--yes` without a terminal) | +| a push says a teammate's record took the place it was writing to, and names the last record that landed | `SYNC_POSITION_TAKEN` | a teammate's push landed while your push was running, and what landed stays | `kane-cli context pull origin --rebase`, then push again | +| a push at the very first record says a teammate's record took the place it was writing to | `SYNC_POSITION_TAKEN` | the location was empty when you started and a teammate filled it first, so there is no shared history between you | `kane-cli context clone
`, then ingest your documents there again | +| a pull says this store changed while it ran | `SYNC_LOCAL_MOVED` | something appended to the store while the pull was being prepared or landing. When part of the pull had landed, the message names the range that did, a valid prefix, kept | run the pull again | +| `nothing answered at /no/such/parent/team-context: that folder does not exist and cannot be created`, or `no repository answered at that address, or your account cannot see it; a private repository looks missing until access is granted` | `SYNC_LOCATION_UNREACHABLE` | the address is wrong, the host is down, or a private repository is not shared with you | check the address, then run the command again | +| `the bucket … did not accept origin's access keys, even for reading`; over SSH, `github.com did not accept your SSH key` or `this computer has not accepted github.com's SSH host key yet`; over HTTPS, `sign in is needed, or your account has no access to this repository on github.com` | `SYNC_LOCATION_DENIED` | the location answered and refused you | follow the message: for a bucket, ask the owner for access or bind again with the right keys (`kane-cli context sync add origin
--credential-env `); for an SSH key, load it with `ssh-add`; for a host key, run `ssh -T git@github.com` once and answer yes; over HTTPS, sign in through Git | +| `--credential-env TEAM_S3: TEAM_S3_ID is not set` | `CREDENTIALS_MISSING` | half a key pair | set `TEAM_S3_ID` and `TEAM_S3_SECRET`, then run the command again | +| `nothing to clone at …: it holds no records` | `SYNC_LOCATION_EMPTY` | the location is empty and nobody has pushed yet | ask the teammate who shared the address to run `kane-cli context push origin` first, then clone again | +| `archive is a tier 3 location: it can be cloned and pulled, never pushed` | `SYNC_READ_ONLY` | the location can be downloaded from only (no write permission, a read-only mount, a read-only key pair). `kane-cli context sync` meets this too, after its pull has already landed | take records from it with `kane-cli context pull`, and publish to a location that can publish | +| `a rebase (…) is open with 2 decisions unresolved` | `SYNC_REBASE_PENDING` | the store is fenced until the rebase is finished or closed | `kane-cli context sync` to answer, or `kane-cli context sync doctor --abort` to close it | +| `This location needs Git 2.31 or newer.` | `SYNC_GIT_REQUIRED` | Git is missing or too old for a GitHub location | install or update Git, then run the command again | +| `The server did not complete the concurrent connection check. Its scratch-ref policy may differ from the storage branch.` | `SYNC_PROBE_INCONCLUSIVE` | the concurrent connection check did not finish, and repository rules that block the scratch reference it pushes under `refs/kane/probe/` are the usual cause. Nothing was bound | ask the repository owner to check that the reference is allowed, then run the command again | +| `origin refused publication`, `A context object is too large for this Git location.` or `Git attributes transform context bytes (…)` | `SYNC_LOCATION_REFUSED_WRITE` | the location took the reads but refused the write: no write permission or a branch rule, a document above 100 MB, or a repository whose `.gitattributes` converts files | fix what the message names, whether write access or the rule, an S3-compatible bucket for the large document, or a dedicated repository with no conversions, then push again | +| `session … is live (pid …)`, `lock … is held by …` or `a reset is already in progress` | `SYNC_STORE_BUSY` | a live session, a held lock, a reset in progress, or a store that changed while the rebase was being prepared stopped the rebase from starting | follow the message: finish or pause the live session; wait for the command holding the lock; `kane-cli context sync doctor` for a reset in progress; when the store changed during preparation, run `kane-cli context pull origin --rebase` again | +| a message saying a record on the location is stamped with a newer version than this Kane CLI reads (on clone: `the storage was written with canonicalizer …`) | `SYNC_CLIENT_TOO_OLD` | a teammate published from a newer Kane CLI than yours | update Kane CLI, then run the command again | +| a rebase says this store is not the one origin was bound from | `SYNC_COPIED_STORE_UNRESOLVED` | you copied or moved the project folder since binding the location | `kane-cli context sync add origin
` in this folder, then run the command again | +| `origin holds a different history (its first record is not this store's)` | `SYNC_FOREIGN_LINEAGE` | that location was started from another store, and there is no merge | to work on it, clone it; to share this store, add an empty location | +| a push to a repository says publication could not be confirmed | `SYNC_PUBLICATION_UNKNOWN` | the network dropped mid-push, and the batch may have landed | check connectivity and run the same command again, which reconciles what landed and never duplicates it | +| `the storage kept growing while its tail was being found` | `SYNC_REMOTE_HOLE` | teammates were publishing while your command read the location, and nothing was changed on your side | run the command again in a moment | +| a message naming a hole at the location, a record or blob that should be there is missing | `SYNC_REMOTE_HOLE` | something outside Kane CLI deleted files at the location | nothing was changed on your side. Restore the location from a copy that holds the missing records, then run the command again | +| `the storage was rolled back: position … was seen there with …` | `SYNC_REMOTE_ROLLED_BACK` | the location was restored from an older snapshot, or rebuilt | nothing was changed on your side. Restore the location from a copy that holds the records this machine saw, or, once a person has confirmed it was rebuilt on purpose, `kane-cli context sync remove origin` then `kane-cli context sync add origin
` | +| `the tail at position … changed while this pull ran` | `SYNC_REMOTE_CHAIN_BROKEN` | the last record on the location read back with different contents during this pull, and nothing was changed on your side | run the pull again | +| a message saying a record on the location is sealed onto a hash its predecessor does not have | `SYNC_REMOTE_CHAIN_BROKEN` | the location's history was edited outside Kane CLI | nothing was changed on your side, and a person has to look at the location | +| a message saying an object *on the location* does not hash to its name | `SYNC_OBJECT_CORRUPT` | a file at the location is damaged | nothing was changed on your side, and a person has to look at the location before anything more is pushed to it | +| a message saying a *local* record or blob does not hash to its file name | `SYNC_LOCAL_RECORD_CORRUPT` or `SYNC_LOCAL_BLOB_CORRUPT` | a file in your own store is damaged, which says nothing about the location's copy | run `kane-cli context fsck`, then restore that record or blob from a backup or a clone of your store | + +Exit `3` means a person has to decide something, exit `2` a precondition, and exit `1` a record that cannot be used. A refusal says what stopped, not that nothing happened: `kane-cli context sync` finishes its pull before its push can be refused, and a push can land before its confirmation is lost. A refusal this table does not list still says what stopped and what to run next. How an agent handles these codes is in [Agents and CI](/support/docs/kane-cli-assurance-automation/#the-sync-verbs-on-the-stream), and the entries a person meets most are in [Troubleshooting](/support/docs/kane-cli-troubleshooting/#context-sync-a-location-cannot-be-reached-or-refuses-you). + +## When two people changed the same thing {#rebase} + +Alice and Bob both worked after the same shared record (record 2). Bob named the source `brief` "spec" and re-ingested a new version of `prd`, then pushed. Alice, without pulling, named `prd` "spec" and retired `prd`. Now `kane-cli context sync status origin` says `you and origin both added work after position 2` and names the command to run, `kane-cli context push origin` refuses with the same words (exit `3`), and so does a plain pull. + +`kane-cli context pull origin --rebase` does three things: it **saves** Alice's records after the shared record in a backup, **takes** origin's records, and **reapplies** the saved records on top, one by one. Nothing on origin is rewritten. On a terminal it asks first, as a panel with the facts and two rows: + +``` +rebase from origin + local 2 records from position 3 saved to .context/sync/backups/2026-09-14T10-45-07-658Z-reset + origin records 3..4 taken into this store + then the local records reapplied on top; a disagreement becomes a decision you answer + never history on origin rewritten + +? continue + +❯ 1. continue — save, take, reapply + 2. not now — nothing changes + + ↑↓ select · ⏎ confirm · esc not now +``` + +Without a terminal the same command refuses and names the line to run: `kane-cli context pull origin --rebase --yes`. `--yes` confirms exactly this, and never answers a decision. **Not now** prints `no rebase started — run kane-cli context pull origin --rebase when you are ready` and exits `3`, with nothing saved or taken. + +Most saved records reapply without a question, one line each: a record the other side does not touch is reapplied, and a record the other side already made the same way is skipped as already there. A **real disagreement about one thing** reaches you as a **decision card**, one at a time. The two sides are always called **local** and the location's name: + +``` +decision 1 of 2 · slug on your names from record 3 + local named prd "spec" at 2026-09-14T10:44:20.253Z + origin "spec" names brief on origin (by bob at 2026-09-14T10:44:15.529Z) + +? what should happen to the local names from record 3 + +❯ 1. keep origin's version — the local change stays in the backup + 2. apply the local version — the name moves to the local node; the node that holds it on origin is reached by its id again + 3. decide later — leaves the rebase open; kane-cli context sync or kane-cli context pull asks again + + ↑↓ select · ⏎ confirm · 1–9 jump · v full record · a decide everything later · ctrl+c pause +``` + +The name after `by` is the author id recorded with that change, a person's user name as the machine reported it, or an agent. `someone` means the record carried no author, a re-ingested source for one. + +**Keep origin's version** is always offered and always the default: it writes nothing, and the local change stays in the backup. **Apply the local version** writes the local change as a new record on top of origin's, and the row says the consequence. For a newly created item whose match origin has since retired, the card offers **add the local version as new**, *a new node beside origin's (duplicate check first)*, in place of **apply the local version**. **Decide later** leaves the card open, and `kane-cli context sync` or `kane-cli context pull` asks again. `v` shows the full record under the facts, `a` leaves every remaining card for later, and Ctrl+C pauses; none of these is an error. Each answered card leaves one line in the scrollback (`decision h3: kept origin's version`), and the run ends with a summary counting what was reapplied, already there, not reapplied and undecided. + +Some saved records cannot be reapplied at all, and their card offers only **keep origin's version**: a record this build cannot read or replay, one that refers to an item origin no longer has, or one the store's own checks refuse. The change stays in the backup, and that piece of work has to be done again once the rebase is finished. Keeping origin's version can also bring a later card back, since a saved record that was built on the one you set aside is looked at again. + +**Without a terminal** (a pipe, CI, `--mode agent`) no card is shown. Every open decision is one line, its id, the kind of record, the local change in words, and the answers it takes, and `kane-cli context sync status` repeats them. When the location cannot be reached, the human output still lists them before the refusal, and `--show ` needs no location at all: + +``` +[alice] $ kane-cli context sync status origin +up to date with origin: position 4 on both sides +rebase 2026-09-14T10-45-07-658Z-reset open: 2 decisions waiting + h3 names named prd "spec" at 2026-09-14T10:44:20.253Z · keep-theirs | apply-mine + h4 retire retired prd at sha256:fb3e3e5a6cfa (2026-09-14T10:44:21.854Z) · keep-theirs | apply-mine +next: kane-cli context sync · kane-cli context sync status --show 3 (one record in full) +``` + +`kane-cli context sync status --show ` prints saved record `n` in full: what it did, what origin holds instead, the answers it accepts, and the raw detail behind the card. + +``` +[alice] $ kane-cli context sync status --show 3 +record 3: names spec +origin sha256:1a52… +outcome: decision h3 (slug) +local: named prd "spec" at 2026-09-14T10:44:20.253Z +origin: "spec" names brief on origin (by bob at 2026-09-14T10:44:15.529Z) +choose: keep-theirs, apply-mine +{ + "slug": "spec", + "mine": "sha256:fb3e…", + "theirs": "sha256:f2e4…" +} +``` + +The second line, `origin sha256:…`, is the saved record's own hash: the word there means the original record, not the location, and the location's side is the `origin:` row below it. + +Answer by id with `--answer =` on `kane-cli context sync` or `kane-cli context pull`, where `keep-theirs` keeps origin's version, `apply-mine` applies the local version, and `apply-mine-as-new` adds it as new. Each run answers what you gave it, and with cards still open it stops again and says so (`not synced: 1 decision waiting — run kane-cli context sync origin to answer it, or kane-cli context sync origin --answer h4=`). The last answer finishes the rebase, and `kane-cli context sync` then pulls and pushes as usual, while `kane-cli context pull` only pulls. Here, after `--answer h3=keep-theirs` in the run before: + +``` +[alice] $ kane-cli context sync origin --answer h4=apply-mine +finishing rebase 2026-09-14T10-45-07-658Z-reset first: 1 decision waiting +reapplied your retire from record 4 + +rebase summary — complete + rebase 2026-09-14T10-45-07-658Z-reset + reapplied 1 + already there 0 + not reapplied 1 + test files 0 restored +pushed 1 records (5..5), 0 blobs, 0 proposals +``` + +To see what an answer did, ask for the saved record again once the rebase is closed, and `kane-cli context sync status --show ` names the record it became: + +``` +[alice] $ kane-cli context sync status --show 4 +record 4: retire prd +origin sha256:f972… +outcome: reapplied → record 5 +``` + +:::warning +**While a rebase is open, the store takes no other change**, exactly like git mid-rebase. `kane-cli context extract`, `kane-cli design tests`, `kane-cli maintain reconcile`, `kane-cli context ingest`, `kane-cli context review`, `kane-cli context name`, `kane-cli context retire` and `kane-cli context revert` refuse until the rebase is finished or closed, all with the same code (`SYNC_REBASE_PENDING`) and the same two ways out. +::: + +``` +[alice] $ kane-cli context name prd other +error: a rebase (2026-09-14T10-45-07-658Z-reset) is open with 2 decisions unresolved; this store takes no other change until it is finished — run kane-cli context sync or kane-cli context pull to continue, or kane-cli context sync doctor --abort to close it +next: run kane-cli context sync or kane-cli context pull to continue, or kane-cli context sync doctor --abort to close it +``` + +`kane-cli context push` refuses with the same code and the same `next:` line, and its reason names the rebase, counts the decisions unresolved, and says nothing was pushed. A test run does not refuse: the results it records are kept to the side and land on the first run after the rebase is finished. + +`kane-cli context sync doctor` shows the store's state and every rebase with its id, open or closed. Three of its words come from the mechanics: a rebase id ends in `-reset`; the first line, `no reset in progress`, is about the import step of a rebase, and it reads `reset in progress` only when an import was interrupted before it finished; and `sentinel: absent` is the healthy state, where the sentinel is the marker a running import leaves in the store. With the two decisions above still open: + +``` +[alice] $ kane-cli context sync doctor +no reset in progress +sentinel: absent +chain: ok, tail at position 4 +rebase 2026-09-14T10-45-07-658Z-reset (2026-09-14 10:45:07Z): open, 0 reapplied, 0 already present, 0 not reapplied, 2 decisions open +next: kane-cli context sync (or kane-cli context pull) to finish rebase 2026-09-14T10-45-07-658Z-reset; kane-cli context sync doctor --abort to close it instead +``` + +`kane-cli context sync doctor --abort` closes the open one, keeping what was already reapplied and leaving the unanswered decisions in the backup (exit `3` when decisions were left). From another run, with one decision open: + +``` +[alice] $ kane-cli context sync doctor --abort +closed rebase 2026-09-14T10-45-52-997Z-reset: 0 reapplied stay, 1 unresolved decision left in the backup (receipt: sync/replay/2026-09-14T10-45-52-997Z-reset.receipt.json) +``` + +A rebase that was interrupted before origin's records had been imported is undone by the same `kane-cli context sync doctor --abort`: your records are put back and the store is as it was. Once the import has landed, the rebase can only be finished (`SYNC_RESET_IMPORTED`): `kane-cli context sync` or `kane-cli context pull` finishes it before doing anything else, and then you can close it with `kane-cli context sync doctor --abort` if you still want to. + +**The way back.** The live store holds records your teammates published, so it is never rewound in place. `kane-cli context sync doctor --export ` rebuilds your store *as it was before the rebase*, beside it, as its own store, for inspection or as a fresh start: + +``` +[alice] $ kane-cli context sync doctor --export ../alice-before-rebase +exported rebase 2026-09-14T10-45-52-997Z-reset: 3 records (0 test files, 0 session files) into …/alice-before-rebase/.context; it is its own store +``` + +Give the export a fresh folder outside the live store and its backups; a folder that already holds a store is refused. The exported folder is then a store like any other: open it and run `kane-cli context list` or `kane-cli context explain`. It has no location bound, so run `kane-cli context sync add` there first if you want to share from it. Without `--from ` the export takes the open rebase, or the only one; with several closed rebases, name one. Everything a rebase saves stays under `.context/sync/backups//` and `.context/sync/replay/`, and a receipt there lists every reapplied, skipped and undecided record. + +A `_test.md` that a reapplied record created is **restored**, and the summary's `test files … restored` line counts them. If that path now holds someone else's file, Kane CLI leaves it alone and `kane-cli context sync status` lists it: rename or move that file, then run `kane-cli context sync` again. + +## What happens when {#what-happens-when} + +`origin` is the location, and `local` means the records saved from your store. + +| local… | origin, meanwhile… | Result of `kane-cli context pull origin --rebase` | +|---|---|---| +| ingested a new document, or extracted a new use-case | did something unrelated | reapplied, no question asked; a short id that origin has since used (`uc-5`) is renumbered (`uc-6`) and the alias is in the receipt | +| extracted a use-case | extracted the same use-case | kept once, origin's, and your review verdicts and evidence attach to it | +| edited a use-case | edited it the same way | not reapplied: already done | +| edited a use-case | edited it differently | a decision, and both versions are kept until you answer | +| reviewed an item (approve, reject) | reviewed the same item differently | a decision | +| named a node "spec" | named a *different* node "spec" | a decision: keep origin's name, or move the name to your node (origin's node is still reachable by its id) | +| archived an item | built new scenarios or tests on it | a decision that offers only **keep origin's version**: the archive stays in the backup, so reconsider it after the rebase is finished | +| retired a source | re-ingested a new version of that source | a decision: keep origin's new version, or retire the source as it is now, new version included | +| ran tests and recorded results | ran the same tests | both results apply, each to the exact version that ran | +| designed tests for a use-case | designed tests for the same use-case | a decision: keep origin's and run `kane-cli design tests` again afterwards if you want both | + +## For agents and CI + +Every sync command in the table above takes `--mode agent` and speaks NDJSON: `sync_status`, `sync_pull_done`, `sync_push_done`, the `sync_rebase_*` family, `sync_error{code, detail, remedy}`, and `done` last, and a rebase that stops on decisions ends with `done` carrying `paused` and exit `3`. The contract and the events are in [Agents and CI](/support/docs/kane-cli-assurance-automation/#the-sync-verbs-on-the-stream), and the CI recipe with a GitHub token or deploy key is in [CI/CD](/support/docs/kane-cli-cicd/#a-shared-context-store-in-ci). + +## FAQ + +**Two stores that were never connected, can I merge them?** No. A location holds one history, a store started separately is `a different history`, and the answer is to clone one of them and re-ingest the other's documents into it. + +## Next Steps + +- [Building the context graph](/support/docs/kane-cli-assurance-context/) for ingest, extract and review +- [Agents and CI](/support/docs/kane-cli-assurance-automation/) for the headless contract, including the sync verbs +- [Configuration](/support/docs/kane-cli-configuration/#context-sync-environment-variables) for the sync environment variables and on-disk paths +- [Troubleshooting](/support/docs/kane-cli-troubleshooting/) for the sync entries diff --git a/docs/kane-cli-assurance.md b/docs/kane-cli-assurance.md index 1ae878d6e..7f0ae10c0 100644 --- a/docs/kane-cli-assurance.md +++ b/docs/kane-cli-assurance.md @@ -149,6 +149,7 @@ Every stage is a separate command, so you can stop, review, and resume at any po | Execute | [`kane-cli testmd run`](/support/docs/kane-cli-testmd/), `kane-cli testrun run` | Author and replay the designed tests; every run seals an evidence pack | | Measure | [`kane-cli cover`](/support/docs/kane-cli-assurance-coverage/) | Two axes: what a pack **proved** vs what the design still **owes** | | Maintain | [`kane-cli maintain`](/support/docs/kane-cli-assurance-maintain/) | Reconcile the suite when a source document changes | +| Share | [`kane-cli context sync`](/support/docs/kane-cli-assurance-sharing/) | Share the store with your team through a location, a GitHub repository, an S3-compatible bucket, or a folder, with `kane-cli context push`, `kane-cli context pull` and `kane-cli context clone` | ## The vocabulary @@ -177,9 +178,10 @@ If you have a PRD and care about coverage accounting, start with assurance. If y The assurance commands work over a local store in your project directory, created on first `ingest`: -- It is **append-only**: nothing is ever deleted or rewritten. Edits create new versions; mistakes are reverted with compensation records. `kane-cli context explain` can replay the full history of any node. -- It is **yours and local**: sources, use-cases, designs, and review verdicts live in your project, not on a server. The extract and design agents run against the KaneAI service using your login, but the store they commit to is on your disk. -- **Keep `.context/` out of git merges.** The store is single-writer and not git-mergeable — two branches appending records will corrupt it on the next read. Gitignore it; share by re-ingesting sources. +- It is **append-only** in everyday use: no command that writes to it deletes or rewrites a record. Edits create new versions, and mistakes are reverted with compensation records. `kane-cli context explain` can replay the full history of any node. The one exception is a sync rebase, which replaces your records after the last shared one with the location's and reapplies your own on top, and every replaced record is kept in a backup. +- It is **yours and local**: sources, use-cases, designs, and review verdicts live in your project. The extract and design agents run against the KaneAI service using your login and send it what they need to read, but the store they commit to is on your disk, and the store itself goes elsewhere only when you publish it to a location your team shares. +- **Never merge `.context/` with git.** The store is single-writer, and two branches appending records will corrupt it on the next read. Kane CLI adds `.context/` to your `.gitignore` when it creates the store inside a git repository, and `KANE_CONTEXT_GITIGNORE=0` keeps it out. +- **Share it through a location**: bind the store to a GitHub repository, an S3-compatible bucket, or a folder on a shared drive, then `kane-cli context push` publishes your records, `kane-cli context pull` takes your teammates', and `kane-cli context clone` gives a new teammate a store of their own. Nothing on a location is ever overwritten or deleted, and when two people change the same thing the disagreement is a decision you answer, never a silent overwrite. See [Sharing the context graph with your team](/support/docs/kane-cli-assurance-sharing/). - `kane-cli context fsck` verifies the whole store; `kane-cli context rebuild` regenerates the read caches from the verified records. ## What costs credits diff --git a/docs/kane-cli-cicd.md b/docs/kane-cli-cicd.md index 901343ecc..db45b9468 100644 --- a/docs/kane-cli-cicd.md +++ b/docs/kane-cli-cicd.md @@ -178,11 +178,29 @@ These patterns apply to every CI system; the platform-specific recipes below dif - **Always pass `--headless`**. CI runners have no display. - **Always set `--timeout `**. A hung run cannot be allowed to block the pipeline. - **Authenticate with `--username` and `--access-key`** from CI secrets. Do not call `kane-cli login` in CI. That flow opens a browser for OAuth and will not work on a runner. -- **Load test data with `--variables-file `**. Check the file into your repo (without secret values), or generate it before the step. +- **Load test data with `--variables-file `**. Check the file into your repo (without secret values), or generate it before the step. A `{{name}}` with no value fails the job with exit `2` **before** any browser starts, with a receipt naming the variable, so fill values from CI secrets in that step, not later. +- **Run the suite on the cloud grid when the runner cannot run it.** `kane-cli testrun run … --remote` turns the suite into one HyperExecute job: the grid supplies Chrome on a macOS runner, or, for mobile `_test.md` members, a virtual Android emulator or iOS simulator, so the runner needs no Chrome, Xcode or Android Studio, and `--parallel N` spreads the members across N grid runners. The recordings and the evidence pack come back to the checkout as if the suite had run on your machine. + + ```bash + kane-cli plugin install remote-execution + + # a web suite on 4 grid runners + kane-cli testrun run tests/web/ --remote --parallel 4 \ + --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY" \ + --on-failure fail-fast + + # a mobile suite, from a Linux runner + kane-cli testrun run tests/app/ --remote \ + --device-name "Pixel 7" --os-version 14 \ + --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY" \ + --on-failure fail-fast + ``` + + It needs a plan with HyperExecute macOS runners. Web and mobile members go in separate runs. Pick devices with `kane-cli devices list --target emulator|simulator --remote`, and allow a timeout of about 10 minutes. See [Remote Runs](/support/docs/kane-cli-remote-execution/). - **Check the exit code**. `0` passed, `1` failed, `2` error, `3` timeout or cancellation. :::warning Chrome Requirement -The runner spawns Chrome itself, so the CI image must have Chrome available on `PATH`. If your runner image cannot install Chrome, point Kane CLI at a remote browser with `--cdp-endpoint ` or `--ws-endpoint ` (for example, a `wss://` endpoint). +The runner spawns Chrome itself, so the CI image must have Chrome available on `PATH`. If your runner image cannot install Chrome, you have two options. For a single `kane-cli run` or `testmd run`, point Kane CLI at a remote browser with `--cdp-endpoint ` or `--ws-endpoint ` (for example, a `wss://` endpoint), where Kane CLI still runs on the runner and drives that browser. For a whole suite, `kane-cli testrun run … --remote` moves the run itself to the grid, as above. ::: ## Authentication in CI/CD @@ -401,7 +419,7 @@ kane-cli run "Open the pricing page and verify the Pro plan is listed" \ --variables-file ./tests/variables.json ``` -If your CI image cannot install Chrome (for example, a minimal Node Alpine image), point Kane CLI at a remote browser instead: +If your CI image cannot install Chrome (for example, a minimal Node Alpine image), either run a whole suite on the grid with `kane-cli testrun run … --remote` (see [Remote Runs](/support/docs/kane-cli-remote-execution/)), or point a single run at a remote browser: ```bash kane-cli run "Open the pricing page and verify the Pro plan is listed" \ @@ -491,3 +509,37 @@ kane-cli run "Log in as {{email}} with {{password}} and verify dashboard" \ --access-key $LT_ACCESS_KEY \ --headless --agent ``` + +## A shared context store in CI {#a-shared-context-store-in-ci} + +When your team [shares the context graph](/support/docs/kane-cli-assurance-sharing/) through a location, a pipeline works on the same store: clone it once, pull before each run, and push the facts the run produced after. The sync commands never call the agent or spend credits, while the run between them, `kane-cli context extract` below, is an ordinary extraction and consumes credits like any other. + +- **Sign in without a person.** For a GitHub location over HTTPS, set `KANE_SYNC_GIT_TOKEN` from a CI secret: a repository-scoped personal access token with Contents read and write, or a GitHub App installation token. A workflow's own `GITHUB_TOKEN` only reaches the workflow's repository, so a separate context repository needs its own token. Over SSH, an SSH deploy key on the context repository works with no token. Kane CLI never answers an SSH prompt, so the runner must have the key loaded and the host's key already accepted (`ssh-keyscan github.com >> ~/.ssh/known_hosts`). Repository rules must also allow the connection check's scratch reference under `refs/kane/probe/`, see [Three kinds of location](/support/docs/kane-cli-assurance-sharing/#locations). For an S3-compatible location, set `KANE_SYNC_S3_ACCESS_KEY_ID` and `KANE_SYNC_S3_SECRET_ACCESS_KEY`. Neither is ever written to disk by Kane CLI. See [Context sync environment variables](/support/docs/kane-cli-configuration/#context-sync-environment-variables). +- **Use `--mode agent`** for structured output, and read the exit code: `0` done, `3` a person has to decide (the runner is behind or diverged, or a rebase stopped on decisions), `2` a precondition (the location cannot be reached, keys missing, a rebase still open). +- **Do not answer decisions blindly.** A rebase that stops on a decision is a job that stops. Save `kane-cli context sync status origin --json` as a build artifact: it is a report of what is waiting, not something a later job can replay on its own, because answers go to the store that holds the open rebase. The follow-up job must run on the same persisted `.context/`, a workspace or cache that survives between jobs, where a person or an agent answers with `kane-cli context sync origin --answer =`. `keep-theirs` writes nothing, but it is a choice: the local change stays in the backup. The full contract is in [Agents and CI](/support/docs/kane-cli-assurance-automation/#the-sync-verbs-on-the-stream). +- **Keep `.context/` out of version control.** The store never goes through a git merge, and the location is where it is shared. + + + +```bash +set -e # stop at the first failing command, so every exit code below is read + +# GitHub Actions step, where the secret CONTEXT_REPO_TOKEN grants Contents read and write on the context repository +export KANE_SYNC_GIT_TOKEN="$CONTEXT_REPO_TOKEN" + +# first run on this runner: a store from the team location +[ -d .context ] || kane-cli context clone https://github.com/example-org/team-context.git . --mode agent + +# before the run: take the team's new records, and stop the job if a person has to decide +kane-cli context pull origin --mode agent > pull.ndjson || { + code=$? + [ "$code" -eq 3 ] && kane-cli context sync status origin --json > sync-status.json + exit "$code" +} + +# the run itself: it fails on a high-risk question and takes the recommended default for the rest +kane-cli context extract --mode ci + +# after the run: publish what landed +kane-cli context push origin --mode agent +``` diff --git a/docs/kane-cli-cli-reference.md b/docs/kane-cli-cli-reference.md index 48e0c5888..3b3c3c815 100644 --- a/docs/kane-cli-cli-reference.md +++ b/docs/kane-cli-cli-reference.md @@ -116,7 +116,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "kane-cli doctor", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli doctor # required checks, each with a fix if it fails\nkane-cli doctor --install # install the test tooling Kane CLI manages\nkane-cli doctor --targets # also list the emulators and simulators available" + "text": "kane-cli doctor --target emulator # required checks, each with a fix if it fails\nkane-cli doctor --target simulator --install # install the test tooling Kane CLI manages" }, { "@type": "SoftwareSourceCode", @@ -158,7 +158,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "kane-cli config", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli config show # Show all settings\nkane-cli config set-window x # Set browser window size\nkane-cli config set-mode # Set run mode\nkane-cli config chrome-profile [path] # Set Chrome profile (interactive picker if no path)\nkane-cli config project [id] # Set Test Manager project (interactive picker if no id)\nkane-cli config folder [id] # Set Test Manager folder (interactive picker if no id)\nkane-cli config set-target # Set run target: desktop | emulator | simulator\nkane-cli config set-device # Set default mobile device\nkane-cli config set-app # Set default app under test for mobile runs" + "text": "kane-cli config show # Show all settings\nkane-cli config set-window x # Set browser window size\nkane-cli config set-mode # Set run mode\nkane-cli config chrome-profile [path] # Set Chrome profile (interactive picker if no path)\nkane-cli config project [id] # Set Test Manager project (interactive picker if no id)\nkane-cli config folder [id] # Set Test Manager folder (interactive picker if no id)\nkane-cli config set-target # Set run target: desktop | emulator | simulator\nkane-cli config set-device-name # Set default mobile device\nkane-cli config set-os-version # Set the default mobile device's OS version\nkane-cli config set-app # Set default app under test for mobile runs" }, { "@type": "SoftwareSourceCode", @@ -231,26 +231,53 @@ kane-cli run "" [options] | Flag | Description | Default | |------|-------------|---------| | `--target ` | Which target to run against: `desktop`, `emulator`, or `simulator` | Saved session target, otherwise `desktop` | -| `--device ` | Pick a device by name, serial, `ip:port`, or udid | TTY opens a one-time picker and saves the choice; non-interactive runs require it | +| `--device-name ` | Pick a device as `kane-cli devices list --target emulator\|simulator` prints it. A name needs a version | Config value. When unset, a TTY run opens a one-time picker and saves the choice. Non-interactive runs require a device | +| `--os-version ` | The device's OS version, for example `14` or `17.5`. On its own, it matches any device running that version | Config value | | `--app ` | The app under test. A build (emulator `.apk`, simulator `.zip`) or an uploaded app id | Config value. Required for every mobile run | -On the `desktop` target, `--device` and `--app` are ignored. See [Mobile Testing](/support/docs/kane-cli-mobile/) for setup. +On the `desktop` target, the device flags and `--app` are ignored. See [Mobile Testing](/support/docs/kane-cli-mobile/) for setup. --- ### `kane-cli doctor` -Check the mobile tooling on this machine, and install the tooling Kane CLI manages. +Check the mobile tooling this machine needs for one target, `emulator` or `simulator`, and install the tooling Kane CLI manages for it. `--target` is required. ```bash -kane-cli doctor # required checks, each with a fix if it fails -kane-cli doctor --install # install the test tooling Kane CLI manages -kane-cli doctor --targets # also list the emulators and simulators available +kane-cli doctor --target emulator # required checks, each with a fix if it fails +kane-cli doctor --target simulator --install # install the test tooling Kane CLI manages ``` -`doctor` prints one line per required check. Run `kane-cli login` before `--install`. See [Mobile Testing](/support/docs/kane-cli-mobile/). +`doctor` prints one line per required check, each with a fix. Run `kane-cli login` before `--install`. See [Mobile Testing](/support/docs/kane-cli-mobile/). + +--- + +### `kane-cli devices list` + +List the emulators or simulators Kane CLI can run against. `--target` is required. Address a device on a run with `--device-name` and `--os-version`, as the list prints them. + + + +```bash +kane-cli devices list --target emulator # the emulators on this machine +kane-cli devices list --target simulator --remote # the simulators the cloud grid can provision +``` + +See [Remote Runs](/support/docs/kane-cli-remote-execution/) for the grid catalog. + +--- + +### `kane-cli apps list` + +List the uploaded builds your account can use for a target. The **APP ID** column is what `--app` and `kane-cli config set-app` accept. + + + +```bash +kane-cli apps list --target emulator +``` --- @@ -340,7 +367,8 @@ kane-cli config chrome-profile [path] # Set Chrome profile (interactive pic kane-cli config project [id] # Set Test Manager project (interactive picker if no id) kane-cli config folder [id] # Set Test Manager folder (interactive picker if no id) kane-cli config set-target # Set run target: desktop | emulator | simulator -kane-cli config set-device # Set default mobile device +kane-cli config set-device-name # Set default mobile device, as kane-cli devices list prints it +kane-cli config set-os-version # Set the default mobile device's OS version kane-cli config set-app # Set default app under test for mobile runs ``` diff --git a/docs/kane-cli-configuration.md b/docs/kane-cli-configuration.md index e53c43961..0512c79a5 100644 --- a/docs/kane-cli-configuration.md +++ b/docs/kane-cli-configuration.md @@ -172,7 +172,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "Mobile Target", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli config set-target emulator # desktop | emulator | simulator\nkane-cli config set-device pixel-7 # name, serial, ip:port, or udid\nkane-cli config set-app ./builds/app-debug.apk" + "text": "kane-cli config set-target emulator # desktop | emulator | simulator\nkane-cli config set-device-name \"Pixel 7 API 35\" # as kane-cli devices list --target emulator prints it\nkane-cli config set-os-version 15\nkane-cli config set-app ./builds/app-debug.apk" }, { "@type": "SoftwareSourceCode", @@ -273,7 +273,8 @@ Empty fields are shown as `(none)`. The `chrome` path is empty by default, in wh | `folder_name` | string \| null | `null` | Display name of the selected folder | Set by `kane-cli config folder` | | `mode` | `"action"` \| `"testing"` | `"testing"` | Agent behaviour on auth walls, blocked pages, or error pages. | `kane-cli config set-mode ` | | `target` | `"desktop"` \| `"emulator"` \| `"simulator"` | `"desktop"` | Default run target. `desktop` runs the Chrome browser; `emulator` and `simulator` run against a virtual Android or iOS device (macOS Apple Silicon only). See [Mobile Target](#mobile-target). | `kane-cli config set-target ` | -| `device` | string \| null | `null` | Default mobile device, by name, serial, `ip:port`, or udid. When empty, a TTY run prompts once and saves the choice; a non-interactive run needs `--device` or this key set. Ignored on the `desktop` target. | `kane-cli config set-device ` | +| `device_name` | string \| null | `null` | Default mobile device, by the name `kane-cli devices list --target ` prints. Pairs with `os_version`. When empty, a TTY run prompts once and saves the choice; a non-interactive run needs `--device-name` or this key set. Ignored on the `desktop` target. | `kane-cli config set-device-name ` | +| `os_version` | string \| null | `null` | OS version of the default mobile device (`14`, `17.5`). Required alongside `device_name`. | `kane-cli config set-os-version ` | | `app` | string \| null | `null` | Default app under test for mobile runs: a build path (`.apk` or `.zip`) or an uploaded app id. Ignored on the `desktop` target. | `kane-cli config set-app ` | | `bug_detection` | `"off"` \| `"stop"` \| `"continue"` | `"off"` | Whether the agent flags suspected product bugs while authoring. See [Bug detection](#bug-detection). | `kane-cli config set-bug-detection `, or per-run `--bug-detection` | | `code_export.enabled` | boolean | `false` | Generate code export after upload completes. | TUI menu, or `--code-export` flag | @@ -386,21 +387,22 @@ You can override the saved mode for a single run with `--mode ` ### Mobile Target -On macOS Apple Silicon, Kane CLI can run against a virtual mobile device instead of the desktop browser. Three settings persist the default target and how to reach it. They are a **separate axis** from `mode` above: `mode` tunes agent behaviour, while these choose *what device* a run drives. +On macOS Apple Silicon, Kane CLI can run against a virtual mobile device on this machine instead of the desktop browser. Four settings persist the default target and how to reach it. They are a **separate axis** from `mode` above: `mode` tunes agent behaviour, while these choose *what device* a run drives. ```bash -kane-cli config set-target emulator # desktop | emulator | simulator -kane-cli config set-device pixel-7 # name, serial, ip:port, or udid +kane-cli config set-target emulator # desktop | emulator | simulator +kane-cli config set-device-name "Pixel 7 API 35" # as `kane-cli devices list --target emulator` prints it +kane-cli config set-os-version 15 kane-cli config set-app ./builds/app-debug.apk ``` - **`target`**: `desktop`, the default, runs Chrome. `emulator` runs a virtual Android device and `simulator` a virtual iOS device. Existing web runs are unaffected. -- **`device`**: the device a mobile run selects, by name, serial, `ip:port`, or udid. When unset, a TTY run prompts once and saves the choice. Non-interactive runs need it set, either here or with `--device`. +- **`device_name`** and **`os_version`**: the device a mobile run selects, in the vocabulary of `kane-cli devices list --target `. A name needs a version, and a version on its own matches any device running it. When unset, a TTY run prompts once and saves the choice. Non-interactive runs need them set, either here or with `--device-name` and `--os-version`. - **`app`**: the app under test for a mobile run, a build path (emulator `.apk`, simulator `.zip`) or an uploaded app id, `APP` followed by six or more digits. Required for every mobile run. On the `desktop` target, `device` and `app` are ignored. -A run reads these as its defaults. Override any of them for a single run with `--target`, `--device`, and `--app`. Setup and the full list of accepted app formats are in [Mobile Testing](/support/docs/kane-cli-mobile/). +A run reads these as its defaults. Override any of them for a single run with `--target`, `--device-name`, `--os-version`, and `--app`. Setup and the full list of accepted app formats are in [Mobile Testing](/support/docs/kane-cli-mobile/). These defaults describe devices on this machine. A [`testrun run --remote`](/support/docs/kane-cli-remote-execution/) run names its device from the grid catalog with the same two flags. ### Bug detection @@ -489,6 +491,26 @@ A handful of environment variables control how kane-cli locates and launches Chr The CDP timeout and retry settings only affect transient launch failures (Chrome started but did not become reachable in time) — a missing or invalid binary fails immediately without retrying. See [Chrome failed to launch](/support/docs/kane-cli-troubleshooting/#chrome-failed-to-launch) for the matching troubleshooting steps. +### Context sync environment variables + +[Sharing the context graph](/support/docs/kane-cli-assurance-sharing/) reads a few environment variables, never `tui-config.json`. + +| Variable | Effect | +|----------|--------| +| `KANE_SYNC_GIT_TOKEN` | A token for a GitHub location over HTTPS in CI: a repository-scoped personal access token with **Contents read and write**, or a GitHub App installation token. Read at use time, never written to disk or put on a command line. A workflow's own `GITHUB_TOKEN` reaches only that workflow's repository, so a separate context repository needs its own token or an SSH deploy key. | +| `KANE_SYNC_S3_ACCESS_KEY_ID` and `KANE_SYNC_S3_SECRET_ACCESS_KEY` | The access key pair for an S3-compatible location, read at use time. Both must be set, and together they win over the saved credential file. Never written to disk, which is the CI form. | +| `KANE_SYNC_GUARD` | `0` turns off the advisory line a write command prints when a teammate has pushed past this machine (`origin has moved past this machine — run kane-cli context pull origin`, or `this store and origin have diverged — run kane-cli context pull origin --rebase`). The check writes nothing, refuses nothing, and gives up silently after 1.5 seconds. | +| `KANE_SYNC_GIT_TRANSFER_TIMEOUT_SECONDS` | How long one Git transfer (fetch, push) may take, `60` to `3600`. Default 15 minutes. Raise it for a slow link or a very large first fetch. | +| `KANE_SYNC_GIT_HTTP_POST_BUFFER` | Git's HTTP upload buffer in bytes for that command (1 MiB to 512 MiB), for an HTTPS proxy that rejects chunked uploads, where `33554432` is 32 MiB. Larger values cost memory, and the default is unchanged. | +| `KANE_CONTEXT_GITIGNORE` | `0` stops Kane CLI from adding `.context/` to your `.gitignore` when it creates the store inside a git repository. | + +Two places on disk belong to sharing and are not touched by a settings reset: + +| Path | Holds | +|------|-------| +| `~/.testmuai/kaneai/context-sync/.json` | the saved S3 key pair for the location named ``, readable by you only (mode `0600`). `kane-cli context sync remove ` deletes it. The file belongs to the name, not to one store: every store on this machine whose location is called `` reads it, and binding another bucket under that name from any store replaces it, so give each bucket its own name. | +| `~/.testmuai/kaneai/context-sync/mirrors/` | Kane CLI's own cache of each GitHub location (bare Git objects, no checkout). Safe to delete, and the next command fetches again. | + --- ## Resetting Settings diff --git a/docs/kane-cli-evidence-merge.md b/docs/kane-cli-evidence-merge.md index 797f28c86..a766bbb78 100644 --- a/docs/kane-cli-evidence-merge.md +++ b/docs/kane-cli-evidence-merge.md @@ -134,7 +134,6 @@ Targets are execution ids or pack paths, and **order matters**. Earlier targets | `--title ` | Title for the merged run | first eligible pack's | | `--no-finalize` | Keep the merged pack live instead of sealing it | seals by default | | `--json` | Machine-readable merge report | off | -| `--env <name>` | Environment (`prod` or `stage`) | active env | `--rules` and `--on-collision` are mutually exclusive. diff --git a/docs/kane-cli-evidence-viewing.md b/docs/kane-cli-evidence-viewing.md index 4850cdb32..2cf7a4be1 100644 --- a/docs/kane-cli-evidence-viewing.md +++ b/docs/kane-cli-evidence-viewing.md @@ -178,7 +178,6 @@ Open the `viewer` URL in your browser. |---|---|---| | `--port <n>` | Pin the local port | ephemeral | | `--viewer-url <base>` | Override the hosted viewer base URL | environment's viewer | -| `--env <name>` | Environment (`prod` or `stage`) | active profile's env | `serve` accepts sealed `.evidence` files only. A live, unsealed pack directory is rejected. diff --git a/docs/kane-cli-evidence.md b/docs/kane-cli-evidence.md index 43e5b6902..9cc18380c 100644 --- a/docs/kane-cli-evidence.md +++ b/docs/kane-cli-evidence.md @@ -162,6 +162,7 @@ What lands in the project store depends on the surface: | `kane-cli run` / TUI session | Only when the session is **named** (`--name`, or the save prompt at exit) | | `kane-cli testmd run` | Always | | `kane-cli testrun run` | Always (the pack is created directly in the store) | +| `kane-cli testrun run --remote` | Always. The suite's pack is sealed on the grid and downloaded into the store when the job ends, alongside the members' `output-<stem>/` recordings. See [Remote Runs](/support/docs/kane-cli-remote-execution/) | An interactive TUI session maps to one pack: it accumulates every run in the session and seals when you `/exit` or start over with `/new`. diff --git a/docs/kane-cli-generate.md b/docs/kane-cli-generate.md index 8f81730ed..047dbc607 100644 --- a/docs/kane-cli-generate.md +++ b/docs/kane-cli-generate.md @@ -176,7 +176,7 @@ The request id (`23271` above) is printed at the end of each generation and is h | `--files <paths>` | Comma-separated local files to attach as context (new generations and refines only). See [Attaching files for context](#attaching-files-for-context). | | `--project <id>` / `--folder <id>` | Test Manager project / folder. | | `--agent` | Emit structured NDJSON on stdout (auto-on when run non-interactively / piped). | -| `--env`, `--username`, `--access-key` | Environment and authentication, same as [`kane-cli run`](/support/docs/kane-cli-quickstart/). See [Authentication](/support/docs/kane-cli-authentication/). | +| `--username`, `--access-key` | Authentication, same as [`kane-cli run`](/support/docs/kane-cli-quickstart/). See [Authentication](/support/docs/kane-cli-authentication/). | ## Attaching files for context diff --git a/docs/kane-cli-installation.md b/docs/kane-cli-installation.md index 42a826c47..a807ab96a 100644 --- a/docs/kane-cli-installation.md +++ b/docs/kane-cli-installation.md @@ -245,7 +245,7 @@ If the command is not found, your shell is not seeing the npm global `bin` direc | Windows | x64 | ✅ | :::note -**Mobile testing**, the iOS Simulator and the Android Emulator, is supported on **macOS Apple Silicon (arm64) only** for the initial release. See [Mobile Testing](/support/docs/kane-cli-mobile/) for the simulator and emulator prerequisites. +**Mobile testing**, the iOS Simulator and the Android Emulator, runs on your own machine on **macOS Apple Silicon (arm64) only**. See [Mobile Testing](/support/docs/kane-cli-mobile/) for the simulator and emulator prerequisites. On every other platform, mobile suites run on the cloud grid with [`testrun run --remote`](/support/docs/kane-cli-remote-execution/), which needs no mobile tooling on your machine. ::: ## Update @@ -307,8 +307,22 @@ Kane CLI requires Node.js 18+. Check your version and upgrade if needed: node --version ``` +## Plugins + +Some capabilities ship as plugins that Kane CLI installs into a versioned local layout under `~/.testmuai/kaneai/plugins/`. The one you are most likely to need is `remote-execution`, which owns the HyperExecute binary behind [`kane-cli testrun run --remote`](/support/docs/kane-cli-remote-execution/): + +<VerifiedTag value="Verified" /> + +```bash +kane-cli plugin install remote-execution # install (add --version <v> to pin one) +kane-cli plugin list # what is installed, with versions +kane-cli plugin doctor remote-execution # readiness: installed, binary present, logged in +kane-cli plugin remove remote-execution +``` + ## Next Step [Quick Start](/support/docs/kane-cli-quickstart/): Authenticate and run your first test. - [Authentication](/support/docs/kane-cli-authentication/) +- [Remote Runs](/support/docs/kane-cli-remote-execution/) diff --git a/docs/kane-cli-mobile.md b/docs/kane-cli-mobile.md index 9488c86c3..2354f9a9f 100644 --- a/docs/kane-cli-mobile.md +++ b/docs/kane-cli-mobile.md @@ -2,7 +2,7 @@ id: kane-cli-mobile title: Mobile Testing with Kane CLI sidebar_label: Mobile Testing -description: Run Kane CLI tests against local mobile virtual devices. Set up the iOS Simulator or the Android Emulator, then drive a native app on macOS Apple Silicon. +description: Run Kane CLI tests on mobile virtual devices, on your own machine or on the cloud grid. Set up the iOS Simulator or the Android Emulator and drive a native app. keywords: - kane cli mobile - kane cli emulator @@ -18,6 +18,9 @@ keywords: - arm64-v8a system image - avdmanager - kane cli doctor + - kane cli devices list + - kane cli remote runs + - hyperexecute mobile - kaneai - testmu ai url: https://www.testmuai.com/support/docs/kane-cli-mobile/ @@ -149,14 +152,14 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "This is the same for both platforms", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli login\nkane-cli doctor --install" + "text": "kane-cli login\nkane-cli doctor --target simulator --install # or --target emulator" }, { "@type": "SoftwareSourceCode", "name": "Confirm Kane CLI sees a ready toolchain and, optionally, the devices on your machine", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli doctor # required checks, each with a fix if it fails\nkane-cli doctor --targets # also list the simulators and emulators Kane CLI can run against" + "text": "kane-cli doctor --target simulator # or --target emulator\nkane-cli devices list --target simulator # the devices Kane CLI can run against" }, { "@type": "SoftwareSourceCode", @@ -176,7 +179,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "@context": "https://schema.org", "@type": "HowTo", "name": "Setup", - "description": "Run Kane CLI tests against local mobile virtual devices. Set up the iOS Simulator or the Android Emulator, then drive a native app on macOS Apple Silicon.", + "description": "Run Kane CLI tests on mobile virtual devices, on your own machine or on the cloud grid. Set up the iOS Simulator or the Android Emulator and drive a native app.", "step": [ { "@type": "HowToStep", @@ -189,14 +192,14 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "@type": "HowToStep", "position": 2, "name": "Step 2: Install the Kane CLI Test Tooling", - "text": "Sign in and let Kane CLI install the tooling it manages. This is the same for both platforms: You do not need to boot a simulator, boot an emulator, or run adb yourself. Kane CLI discovers the device, boots it, installs your app, and runs the test.", + "text": "Sign in and let Kane CLI install the tooling it manages for the target you set up, emulator or simulator. You do not need to boot a simulator, boot an emulator, or run adb yourself. Kane CLI discovers the device, boots it, installs your app, and runs the test.", "url": "https://www.testmuai.com/support/docs/kane-cli-mobile/#step-2-install-the-kane-cli-test-tooling" }, { "@type": "HowToStep", "position": 3, "name": "Step 3: Ready Check", - "text": "Confirm Kane CLI sees a ready toolchain and, optionally, the devices on your machine: When the checks for your platform pass, setup is complete. On Android, confirm your AVD is listed by --targets.", + "text": "Confirm Kane CLI sees a ready toolchain and, optionally, the devices on your machine: When the checks for your platform pass, setup is complete. On Android, confirm your AVD is listed by kane-cli devices list --target emulator.", "url": "https://www.testmuai.com/support/docs/kane-cli-mobile/#step-3-ready-check" } ] @@ -204,10 +207,17 @@ import VerifiedTag from '@site/src/component/verifiedTag'; ]) }} /> -Kane CLI can run tests against local mobile virtual devices: Apple's **iOS Simulator** and Google's **Android Emulator**. You author and run mobile tests the same way you already do for the browser. The differences are that a mobile test runs against an **app you provide**, and that the target device is a simulator or emulator on your machine. +Kane CLI can run tests against mobile virtual devices: Apple's **iOS Simulator** and Google's **Android Emulator**. You author and run mobile tests the same way you already do for the browser. The differences are that a mobile test runs against an **app you provide**, and that the target device is a simulator or emulator instead of a browser. + +There are two places that device can live: + +| | Where the device runs | What the machine needs | How to run | +|---|---|---|---| +| **Local** | A simulator or emulator on your machine | **macOS on Apple Silicon (arm64)** with Xcode, Android Studio, or both | `kane-cli run … --target emulator\|simulator`, `kane-cli testmd run`, `kane-cli testrun run` | +| **Cloud grid** | A virtual device on a HyperExecute macOS host | **Any machine**: Linux, Windows, or an Intel or Apple Silicon Mac, with no mobile tooling. Your <BrandName /> plan must include HyperExecute with macOS runners | `kane-cli testrun run … --remote`, see [Remote Runs](/support/docs/kane-cli-remote-execution/) | :::note -This release supports **macOS on Apple Silicon (arm64) only**. Mobile testing is not yet available on Intel Macs, Linux, or Windows. Everything below assumes a mac-arm64 host. +Local mobile runs require **macOS on Apple Silicon (arm64)**. Intel Macs, Linux, and Windows cannot boot the simulator or emulator locally. On those machines, run mobile suites on the cloud grid with `--remote`. The setup steps below cover the local path. ::: ## What Mobile Means Here @@ -220,11 +230,11 @@ Pointing a mobile run at a website is not supported yet. Mobile runs target a na **Two targets.** `emulator` is a virtual Android device and `simulator` is a virtual iOS device. The default target stays **desktop**, the browser, so nothing changes for your existing web runs. -## Why a Single Architecture +## Why a Single Architecture for Local Runs Apple Silicon runs both mobile stacks natively. The iOS Simulator is a first-class Apple target, and Android ships `arm64-v8a` emulator images that run on the Mac's built-in hypervisor with hardware acceleration. -Standardising on one host architecture for the first release keeps setup predictable and runs fast, with no cross-architecture translation in the path. Support for other hosts will follow in a later release. +Standardising on one host architecture keeps local setup predictable and runs fast, with no cross-architecture translation in the path. Other machines reach the same devices through the cloud grid, where the grid's macOS runners do the booting. ## How Setup Works @@ -232,9 +242,9 @@ There are two halves, and Kane CLI owns the second. **1. You provide the virtual device.** Apple's and Google's own tooling, Xcode or Android Studio, supplies the simulator or emulator. These are the same tools Apple and Google already ship for building simulators and emulators. Kane CLI does not ship an iOS runtime, an Android SDK, an emulator, or a system image. -**2. Kane CLI installs its own test tooling and drives the device.** After a one-time `kane-cli doctor --install`, covered in [Setup](#setup) below, Kane CLI discovers the device, boots it, installs your app, and runs the test. You do not boot the simulator or emulator by hand. +**2. Kane CLI installs its own test tooling and drives the device.** After a one-time `kane-cli doctor --target emulator|simulator --install`, covered in [Setup](#setup) below, Kane CLI discovers the device, boots it, installs your app, and runs the test. You do not boot the simulator or emulator by hand. -Run `kane-cli doctor` at any time to check what is ready and what is missing. It prints one line per required check, each with a fix. +Run `kane-cli doctor --target emulator|simulator` at any time to check what is ready and what is missing. It prints one line per required check, each with a fix. `kane-cli devices list --target emulator|simulator` lists the devices Kane CLI can run against. ## Prerequisites @@ -245,10 +255,14 @@ Run `kane-cli doctor` at any time to check what is ready and what is missing. It An uploaded app id is `APP` followed by six or more digits. -Both targets require macOS on Apple Silicon and a one-time `kane-cli doctor --install`. +Both targets require macOS on Apple Silicon and a one-time `kane-cli doctor --target emulator|simulator --install`. Set up only the platform you intend to test, or both if you test on both. None of this is needed for `--remote` runs. ## Setup +:::tip +Prefer not to set up a device on your machine? `kane-cli testrun run … --remote` runs the same mobile tests on a virtual device on a HyperExecute macOS host, from any machine and with none of the steps below. See [Remote Runs](/support/docs/kane-cli-remote-execution/). +::: + ### Step 1: Prepare the Virtual Device Follow the tab for the platform you intend to test. Set up both if you test on both. @@ -328,13 +342,13 @@ If your SDK is at the default path, skip this step. ### Step 2: Install the Kane CLI Test Tooling -Sign in and let Kane CLI install the tooling it manages. This is the same for both platforms: +Sign in and let Kane CLI install the tooling it manages for the target you set up: <VerifiedTag value="Verified" /> ```bash kane-cli login -kane-cli doctor --install +kane-cli doctor --target simulator --install # or --target emulator ``` You do not need to boot a simulator, boot an emulator, or run `adb` yourself. Kane CLI discovers the device, boots it, installs your app, and runs the test. @@ -346,11 +360,11 @@ Confirm Kane CLI sees a ready toolchain and, optionally, the devices on your mac <VerifiedTag value="Verified" /> ```bash -kane-cli doctor # required checks, each with a fix if it fails -kane-cli doctor --targets # also list the simulators and emulators Kane CLI can run against +kane-cli doctor --target simulator # or --target emulator +kane-cli devices list --target simulator # the devices Kane CLI can run against ``` -When the checks for your platform pass, setup is complete. On Android, confirm your AVD is listed by `--targets`. +When the checks for your platform pass, setup is complete. On Android, confirm your AVD is listed by `kane-cli devices list --target emulator`. Address a device on a run with `--device-name "<name>" --os-version <v>`, as the list prints them. ## Running a Mobile Test @@ -361,20 +375,44 @@ Once a target is set up, point a run at it: ```bash # one-off, from the command line kane-cli run "Sign in and open the account tab" --target simulator --app ./builds/MyApp.zip -kane-cli run "Add the first item to the cart" --target emulator --app ./builds/app-debug.apk # or set a default target once, then just run kane-cli config set-target emulator kane-cli run "Add the first item to the cart" --app ./builds/app-debug.apk + +# a saved test, or a whole folder of them +kane-cli testmd run tests/checkout_test.md +kane-cli testrun run tests/app/ --device-name "Pixel 7 API 35" --os-version 15 ``` -`--app` is required for every mobile run. The simulator target accepts a `.zip` build, the emulator target accepts an `.apk` build, and both accept an uploaded app id, `APP` followed by six or more digits. +`--app` is required for every mobile run. The simulator target accepts a `.zip` build, the emulator target accepts an `.apk` build, and both accept an uploaded app id, `APP` followed by six or more digits. `kane-cli apps list --target emulator|simulator` lists the uploaded builds your account can use. -You also need a device. In the interactive TUI, leaving `--device` off opens a one-time picker and saves your choice. A non-interactive run, such as one in CI, needs a device already set with `--device` or `kane-cli config set-device`, or the run exits and prints the fix. On the `desktop` target, `--device` and `--app` are ignored. +You also need a device. Pick it with `--device-name` and `--os-version`, as `kane-cli devices list --target emulator|simulator` prints them: a name needs a version, and a version on its own matches any device running it. In the TUI or an interactive terminal, leaving the device flags off opens a one-time picker and saves your choice. A non-interactive run, such as one in CI, needs a device already set with the flags or with `kane-cli config set-device-name` and `kane-cli config set-os-version`, or the run exits and prints the fix. On the `desktop` target, the device flags and `--app` are ignored. In the interactive TUI, switch targets with `/mobile` and `/desktop`, and run `/doctor` to check mobile tooling and devices. -For the full flag list and the app formats each target accepts, see the [CLI Reference](/support/docs/kane-cli-cli-reference/#kane-cli-run). To save a default target, device, and app instead of passing flags every time, see [Configuration](/support/docs/kane-cli-configuration/#mobile-target). To run a mobile test from a file, see [Test.md](/support/docs/kane-cli-testmd/#mobile-target). +For the full flag list and the app formats each target accepts, see the [CLI Reference](/support/docs/kane-cli-cli-reference/#kane-cli-run). To save a default target, device, and app instead of passing flags every time, see [Configuration](/support/docs/kane-cli-configuration/#mobile-target). To run a mobile test from a file, see [Test.md](/support/docs/kane-cli-testmd/#mobile-target), and to run a folder of them, see [Batch Runs](/support/docs/kane-cli-testrun/#mobile-members). + +## Running a Mobile Suite on the Cloud Grid + +`kane-cli testrun run --remote` sends your mobile `_test.md` files to <BrandName /> HyperExecute, which boots a virtual device on a macOS host, installs the app, runs the suite, and returns the recordings and evidence pack to your project. Anyone on the team can author and run mobile tests this way, from any operating system, with none of the local setup above. + +<VerifiedTag value="Verified" /> + +```bash +kane-cli plugin install remote-execution # once +kane-cli devices list --target emulator --remote # what the grid can provision +kane-cli testrun run tests/app/ --remote --device-name "Pixel 7" --os-version 14 --dry-run +kane-cli testrun run tests/app/ --remote --device-name "Pixel 7" --os-version 14 +``` + +Three things differ from a run on your own machine: + +- **The device comes from the grid catalog.** List it with `kane-cli devices list --target emulator|simulator --remote`, not from the AVDs or simulators on your machine. +- **One job runs one platform.** Emulator members run on one Android version, and simulator members run on one HyperExecute pool. +- **A local build is uploaded** from your machine before dispatch and handed to the grid as an `APP…` id. + +The prerequisites, the app rules, and what one job can hold are in [Remote Runs](/support/docs/kane-cli-remote-execution/). ## Evidence for a Mobile Run @@ -394,6 +432,8 @@ The result summary records the **device** in the run environment, for example th ## Next Steps +- [Remote Runs](/support/docs/kane-cli-remote-execution/) to run mobile suites from any machine +- [Batch Runs](/support/docs/kane-cli-testrun/) to run a folder of mobile tests as one execution - [CLI Reference](/support/docs/kane-cli-cli-reference/) for the full flag and command list - [Configuration](/support/docs/kane-cli-configuration/) to save a default target, device, and app - [Troubleshooting](/support/docs/kane-cli-troubleshooting/) for wider setup and run problems diff --git a/docs/kane-cli-modes.md b/docs/kane-cli-modes.md index ed52f670c..a5254ea6b 100644 --- a/docs/kane-cli-modes.md +++ b/docs/kane-cli-modes.md @@ -310,7 +310,6 @@ The customer-facing flags accepted by `kane-cli run`: | `--session-context <json>` | Prior runs context JSON. | None | | `--username <user>` | Basic auth username (skip OAuth). | None | | `--access-key <key>` | Basic auth access key (skip OAuth). | None | -| `--env <name>` | Environment (`prod`). | Active profile's env | | `--mode <name>` | Run mode: `action` (strict) or `testing` (lenient). | Config value, otherwise `testing` | | `--bug-detection <mode>` | Detect product bugs while authoring: `off`, `stop` (halt the run on a confirmed bug), or `continue` (record it and keep going). Overrides `config set-bug-detection`. See [Configuration](/support/docs/kane-cli-configuration/). | Config value, otherwise `off` | | `--agent` | Plain NDJSON output, no colors or UI. | Off | @@ -319,11 +318,21 @@ The customer-facing flags accepted by `kane-cli run`: | `--skip-code-validation` | Skip post-codegen worker-side validation. | On | | `--no-skip-code-validation` | Force post-codegen worker-side validation. | Off | +### Unresolved variables + +Every `{{name}}` in the objective must have a value before the run starts. A name with no value stops the run there, with no browser and no session, exit code `2`, and a receipt naming each variable and what it needs. There is no flag to bypass it: fill the value or remove the reference. With `--agent`, the refusal is a single typed event: + +```json +{"type":"error","code":"unresolved_variables","message":"2 variable(s) have no value — nothing was dispatched","suggested_file":".testmuai/variables/variables.json","variables":[{"name":"checkout_url","reason":"not_declared","used_by":[{"file":"objective","step":1}]},{"name":"login_password","reason":"value_missing","file":".testmuai/variables/variables.json","used_by":[{"file":"objective","step":1}]}]} +``` + +`reason` is `value_missing` (the key exists in `file`, with no value) or `not_declared` (the key is in no file, and `suggested_file` is where to add it). Names an earlier step stores, and the `{{smart.*}}`, `{{environment.*}}`, `{{secrets.*}}` and `{{totp.*}}` namespaces, are never checked. + For variables and context file behavior, see [Variables and context](/support/docs/kane-cli-variables-and-context/). For code export and the run mode toggle, see [Configuration](/support/docs/kane-cli-configuration/). ### Mobile runs -By default a run targets the **desktop** browser (Chrome), so every example above is unchanged. On macOS Apple Silicon you can instead point a run at a virtual mobile device: an `emulator` (a virtual Android device) or a `simulator` (a virtual iOS device). Every mobile run needs an app under test. +By default a run targets the **desktop** browser (Chrome), so every example above is unchanged. On macOS Apple Silicon you can instead point a run at a virtual mobile device on this machine: an `emulator` (a virtual Android device) or a `simulator` (a virtual iOS device). Every mobile run needs an app under test. From any other machine, run a saved mobile suite on the cloud grid with [`testrun run --remote`](/support/docs/kane-cli-remote-execution/). <VerifiedTag value="Verified" /> @@ -341,12 +350,12 @@ kane-cli run "Sign in and open the account tab" --target simulator --app ./build The mobile run flags: - `--target desktop|emulator|simulator`: which target to run against. Defaults to the saved session target, otherwise `desktop`. -- `--device <id>`: pick a device by name, serial, `ip:port`, or udid. In the TUI/TTY, omitting it opens a one-time picker and the choice is saved; in non-interactive runs a device must already be set (via `--device` or `kane-cli config set-device`) or the run exits with the fix spelled out. -- `--app <path|APPid>`: the app under test, required for every mobile run. Pass a build (emulator: `.apk`, simulator: `.zip`) or an uploaded app id (`APP` followed by six or more digits). On the `desktop` target, `--device` and `--app` are ignored. +- `--device-name <name>` and `--os-version <version>`: pick a device as `kane-cli devices list --target emulator|simulator` prints it. A name needs a version, and a version on its own matches any device running it. In the TUI or an interactive terminal, omitting them opens a one-time picker and the choice is saved. In non-interactive runs a device must already be set, with the flags or with `kane-cli config set-device-name` and `kane-cli config set-os-version`, or the run exits with the fix spelled out. +- `--app <path|APPid>`: the app under test, required for every mobile run. Pass a build (emulator: `.apk`, simulator: `.zip`) or an uploaded app id (`APP` followed by six or more digits), and `kane-cli apps list --target <kind>` lists yours. On the `desktop` target, the device flags and `--app` are ignored. In the interactive TUI, a first run offers a Desktop / Emulator / Simulator chooser, and you can switch targets at any time with `/mobile` and `/desktop`. Run `/doctor` to check mobile tooling and devices. -For setup (Xcode or Android Studio, `kane-cli login`, and `kane-cli doctor --install`) and the app formats each target accepts, see [Mobile testing](/support/docs/kane-cli-mobile/). +For setup (Xcode or Android Studio, `kane-cli login`, and `kane-cli doctor --target emulator|simulator --install`) and the app formats each target accepts, see [Mobile testing](/support/docs/kane-cli-mobile/). ### Output Streams diff --git a/docs/kane-cli-remote-execution.md b/docs/kane-cli-remote-execution.md new file mode 100644 index 000000000..eeac1c00d --- /dev/null +++ b/docs/kane-cli-remote-execution.md @@ -0,0 +1,239 @@ +--- +id: kane-cli-remote-execution +title: Remote runs on the cloud grid +sidebar_label: Remote Runs +description: "Dispatch a suite of _test.md files to HyperExecute with kane-cli testrun run --remote: grid browsers for web suites, grid emulators and simulators for mobile suites, from any machine." +keywords: + - kane cli remote + - kane cli testrun remote + - hyperexecute + - cloud grid + - grid emulator + - grid simulator + - remote execution plugin + - testmu ai +url: https://www.testmuai.com/support/docs/kane-cli-remote-execution/ +site_name: TestMu AI +slug: kane-cli-remote-execution/ +canonical: https://www.testmuai.com/support/docs/kane-cli-remote-execution/ +--- +import BrandName, { BRAND_URL } from '@site/src/component/BrandName'; +import VerifiedTag from '@site/src/component/verifiedTag'; + + +<script type="application/ld+json" + dangerouslySetInnerHTML={{ __html: JSON.stringify({ + "@context": "https://schema.org", + "@type": "BreadcrumbList", + "itemListElement": [{ + "@type": "ListItem", + "position": 1, + "name": "Home", + "item": "https://www.testmuai.com" + },{ + "@type": "ListItem", + "position": 2, + "name": "Support", + "item": "https://www.testmuai.com/support/docs/" + },{ + "@type": "ListItem", + "position": 3, + "name": "Remote runs on the cloud grid", + "item": "https://www.testmuai.com/support/docs/kane-cli-remote-execution/" + }] + }) }} +></script> + +`kane-cli testrun run --remote` dispatches a suite of `_test.md` files to <BrandName /> **HyperExecute** instead of running it on your machine. The grid provisions the runtime, runs every member, and streams the job back to your terminal: one job, one exit code, one sealed [evidence pack](/support/docs/kane-cli-evidence/), and the same recordings and evidence you get from a run on your own machine. + +Remote runs cover both kinds of test: + +- **Web suites** run on **Chrome on a HyperExecute macOS runner**, with no Chrome on your machine or CI runner, headless by construction, and `--parallel N` spreads the members across N grid runners. +- **Mobile suites** (`target: emulator` or `target: simulator`) run on a **virtual Android emulator or iOS simulator on a HyperExecute macOS host**, so you can author and run mobile tests **from any machine**: Linux, Windows, Intel Macs, or a Mac without Xcode or Android Studio. The [macOS Apple Silicon requirement](/support/docs/kane-cli-mobile/) applies only to mobile runs on your own machine. + +<VerifiedTag value="Verified" /> + +```bash +kane-cli plugin install remote-execution # once +kane-cli testrun run --tags smoke --remote --parallel 4 # a web suite on 4 grid runners +kane-cli testrun run tests/app/ --remote --device-name "Pixel 7" --os-version 14 # an Android suite +kane-cli testrun run tests/ios/ --remote --device-name "iPhone 15" --os-version 17.5 +``` + +:::note +`--remote` is not the same as `--ws-endpoint` or `--cdp-endpoint`. Those attach a **remote browser** to a run that still executes on your machine (`kane-cli run`, `kane-cli testmd run`). `--remote` moves the **whole suite** to the grid: kane-cli itself runs there, and nothing but Node and the plugin is needed locally. +::: + +## Prerequisites + +| You need | Why | How to check | +|---|---|---| +| A <BrandName /> plan that includes **HyperExecute**, with **macOS runner concurrency** (web and mobile members both run on macOS runners) | Every remote run is a HyperExecute job | Ask your account owner, or open the HyperExecute dashboard for your organisation | +| The `remote-execution` plugin | It owns the HyperExecute binary kane-cli dispatches with | `kane-cli plugin install remote-execution`, then `kane-cli plugin doctor remote-execution` | +| A <BrandName /> **username and access key** | HyperExecute authenticates with basic auth. An OAuth profile is exchanged for them automatically; otherwise pass `--username` and `--access-key` | `kane-cli whoami` | +| A project directory that **contains the tests** | The current directory is zipped and shipped as the job payload | Run from the repository root, or any parent of the tests | +| Recordings **not gitignored** | The payload respects `.gitignore`, and an ignored `output-<stem>/` never reaches the grid. Builds are not payload, see [The app under test on the grid](#the-app-under-test-on-the-grid) | `--dry-run` reports `gitignored_inputs`; un-ignore with, for example, `!output-*/` | + +The project and folder the run uploads to are the ones configured on your profile (`kane-cli config project` and `kane-cli config folder`); they are passed to the grid's login. + +## How a remote run works + +1. **Preflight, then remote preflight.** The normal `testrun` plan is built (one organisation, one project, see [Preflight](/support/docs/kane-cli-testrun/#preflight)); then kane-cli checks the selection can be **one grid job** ([What one job can hold](#what-one-job-can-hold)) and, for mobile, resolves the device against the grid catalog. Anything wrong stops here, nothing is dispatched, exit `2`. +2. **Payload.** The current directory is zipped (respecting `.gitignore`) and uploaded with a generated job definition. The dispatch leaves `.hyperexecute/`, `hyperexecute-cli.log` and `.updatedhyperexecute.yaml` in that directory: add them to `.gitignore`, they are not inputs. +3. **On the grid** (a macOS runner): a pre-step installs kane-cli and logs in with your credentials and project; for mobile it installs the device tooling and boots the device. Then **each member runs as its own `kane-cli testmd run`**, headless, exactly as it would on your machine. A member with recordings replays them, a member without authors on the grid. +4. **Back to you.** The job streams progress and the dashboard link to your terminal. When it ends, the members' `output-<stem>/` recordings and the sealed evidence pack are downloaded into your project, the pack is published to Test Manager, and the suite summary and exit code are the same as a local `testrun`. + +Allow a few minutes on top of the tests' own time: in practice about 15 seconds of setup for a web job, and a minute or more for a mobile job (device boot, app install). A wall-clock timeout of 10 minutes is a safe starting point in CI. + +## Dispatching a run + +`--remote` takes the normal `testrun` selection (paths, `--match`, `--tags`). + +```bash +kane-cli testrun run --tags smoke --remote --dry-run # both preflights, nothing dispatched +kane-cli testrun run --tags smoke --remote --parallel 4 +``` + +`--dry-run` runs both preflights and resolves any device **without creating a job**. Use it before every new selection; it costs nothing. + +A real run prints the job id and dashboard link, then tracks the job until it completes: + +``` +job 24fc58b2-… dispatched → https://hyperexecute.lambdatest.com/hyperexecute/task?jobId=24fc58b2-… +``` + +| Flag | On the grid | +|---|---| +| `--parallel <n>` | Becomes the job's concurrency: the members are auto-split across `n` grid runners, each running its share one member at a time. Device suites parallelize the same way, and every task has its own VM and device | +| `--headless` | Not needed, every member runs headless on the grid | +| `--on-failure`, `--name`, `--bug-detection`, `--author`, `--no-adaptive-heal` | Forwarded to the members on the grid | +| `--username`, `--access-key` | Used for the grid login and the Test Manager upload | + +## Web suites on the grid + +A web suite needs nothing beyond the prerequisites: the grid runner has Chrome, kane-cli finds it and runs each member headless. Use it when the runner cannot have Chrome, when you want the suite off your laptop, or when you want more parallelism than one machine gives you. + +```bash +kane-cli plugin install remote-execution +kane-cli testrun run tests/web/ --remote --parallel 4 --on-failure fail-fast +``` + +What you see back is a normal `testrun` summary; the only extra lines are the dispatch and the job link. A member that authors on the grid comes back with its `output-<stem>/` recordings, so the next run, on the grid or on your machine, replays them. Commit those recordings as you would after a local run. + +A web selection cannot share a job with device members (`mobile_remote_mixed`). Run the two suites separately. + +## Choosing a grid device + +Remote devices come from the **grid catalog**, not from the AVDs or simulators on your machine. List what the grid can provision: + +```bash +kane-cli devices list --target emulator --remote # Android emulators +kane-cli devices list --target simulator --remote # iOS simulators +kane-cli devices list --target simulator --remote --os-version 17.5 # only that OS version +``` + +Each row is a device **name** plus the **OS versions** it ships with. Address one with both: + +| Flag or key | Purpose | +|---|---| +| `--device-name "<name>"` | The name exactly as the catalog prints it (`"Pixel 7"`, `"iPhone 15"`). Validated before dispatch | +| `--os-version <v>` | The OS version (`14`, `17.5`). On its own, it means any catalog device running that version | +| `device_name:` or `os_version:` in a `_test.md` | Per-test defaults, used when the flags are absent. See [Mobile target](/support/docs/kane-cli-testmd/#mobile-target) | + +If you pass neither, kane-cli picks a catalog default for the platform and prints it on the device line. Read it before relying on it. + +The two platforms bind the device differently: + +- **Emulator**: the job allocates **one device for all its members**, so they must agree on one Android version (`mobile_os_version_split` otherwise, or force one with `--os-version`). An AVD name from your own machine is ignored in a member (you get a `device_name_ignored` note), because a remote job's device is named by the catalog. +- **Simulator**: each task boots a simulator inside its own VM, so **each member binds its own `device_name:` and `os_version:`** (validated against the catalog). `--device-name` and `--os-version`, when passed, apply to every member. Members may ask for different iOS versions as long as their iOS majors map to one HyperExecute pool, which the catalog decides (today 17 and 18 share one, 26 is another); otherwise `mobile_pool_split`. + +## The app under test on the grid + +The grid machine has to *obtain* the app. A build never rides the payload, it reaches the grid by id: + +| `app:` in the test | What happens | +|---|---| +| A local build, `.apk` for `emulator` or a `.zip` of the `.app` for `simulator`, anywhere on disk | **Uploaded from your machine at preflight**, once per distinct file (a per-machine cache skips a build your account already has), and handed to the grid as `--app <id>`. It may be gitignored or outside the project. `--dry-run` uploads nothing | +| An uploaded `APP…` id | Used as is, and the grid downloads it | + +Each member gets its own id, so a run may hold members that name different builds. A `.ipa` is refused up front: it is a device build, and the emulator and simulator upload does not take it. Every upload is reported (`app: <file> → APP… (uploaded)`, or the `remote_app` event for agents). + +`kane-cli apps list --target emulator|simulator` shows the uploaded builds your account can use, and the **APP ID** column is what `app:` takes. There is no upload subcommand: any run with a local build, on the grid or on your machine, uploads it and prints the `APP…` id. Uploads belong to an organisation, so `apps list` for the current profile is the authority on which ids a run can use. + +## What one job can hold + +One remote run is one HyperExecute job, which allocates **one kind of runtime**. The remote preflight refuses a selection that needs more than one, and tells you how to split it (`--match` or `--tags`): + +| Reason | Meaning | Fix | +|---|---|---| +| `mobile_remote_mixed` | Web and device tests in one selection | Two runs: one for the device tests, one for the rest | +| `mobile_remote_mixed_platform` | Emulator and simulator tests in one selection | Two runs, one per platform | +| `mobile_os_version_split` | Emulator tests asking for different Android versions | One run per version, or `--os-version` to force one | +| `mobile_pool_split` | Simulator tests whose iOS versions need different HyperExecute pools | One run per pool, or `--os-version` to force one | +| `mobile_remote_unsupported` | A device target the grid cannot provide | Run it on your own machine, or deselect it | +| `mobile_app_missing` | A device test names a local build that is not on this machine | Fix the path, or use an `APP…` id | +| `mobile_app_not_uploadable` | The build is not one the cloud takes (a `.ipa`, or the wrong extension for the platform) | `.apk` for emulator, a `.zip` of the `.app` for simulator, or an `APP…` id | +| `mobile_app_upload_failed` | Uploading the build from your machine failed | Fix the upload (network, auth), or use an `APP…` id | +| `member_outside_payload` | A test lives outside the dispatched directory | Run from a directory that contains it | +| `gitignored_inputs` | Required recordings are gitignored | Un-ignore them (for example `!output-*/`) or commit them | +| `on_grid` | Already running on a HyperExecute grid | `--remote` cannot re-dispatch from inside a job | + +Every reason arrives with the offending paths, both in the terminal and as a `remote_error` event for agents. + +## What comes back + +- **Recordings**: authored members' `output-<stem>/` directories land in your project exactly as a run on your machine would leave them, so the next run replays from cache. +- **Evidence**: one sealed pack for the suite in `.testmuai/evidence/`, published to your project's execution history in Test Manager. +- **Job logs**: the per-member session logs under `~/.testmuai/kaneai/sessions/remote/<job-id>/`, and the full stage logs on the HyperExecute dashboard at the printed job link. +- **Exit code**: the same as a local `testrun`. `0` all passed, `1` a member failed or broke, `2` preflight, auth or usage (nothing dispatched), `3` cancelled. + +## When a remote run fails + +- **A member failed or broke** (exit `1`): read it like any other failure. `output-<stem>/Result.md` names the failing step and reason, and the evidence pack has the screenshots and logs. See [Debugging with a pack](/support/docs/kane-cli-troubleshooting/#debugging-a-failed-run-with-its-evidence-pack). +- **A member is `broken` with no steps** and nothing was published: the grid-side kane-cli refused before launching. Open the job link and read the scenario stage log. For mobile, the usual cause is an `APP…` id that belongs to a different organisation than the account running the job. +- **Nothing was dispatched** (exit `2`): the printed reason is one of the preflight codes above, or `kane-cli plugin doctor remote-execution` shows what is missing (plugin, binary, login). + +## In CI + +A remote run needs no Chrome, Xcode or Android Studio on the runner, only Node and the plugin: + +```bash +npm install -g @testmuai/kane-cli +kane-cli plugin install remote-execution + +# a web suite +kane-cli testrun run tests/web/ --remote --parallel 4 \ + --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY" --on-failure fail-fast + +# a mobile suite +kane-cli testrun run tests/app/ --remote \ + --device-name "Pixel 7" --os-version 14 \ + --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY" --on-failure fail-fast +``` + +Archive `.testmuai/evidence/*.evidence` as the build artifact. More pipeline shapes are in [CI/CD](/support/docs/kane-cli-cicd/). + +## For agents: NDJSON events + +In agent or non-TTY mode a remote run adds typed events around the normal `testrun_*` stream (see [For agents](/support/docs/kane-cli-testrun/#for-agents-ndjson-events)): + +| `type` | Payload | Notes | +|---|---|---| +| `remote_start` | `backend`, `env` | Dispatch begins | +| `remote_device` | `platform`, `slug`, `name`, `os_version`, `avd_id?`, `pool?` | The resolved grid device, mobile only. A web run has no device line | +| `remote_device_hint` | `reason`, `detail` | `device_name_ignored` (emulator only, a local AVD name was dropped) or `catalog_stale` | +| `remote_app` | `path`, `app_id`, `source` | One per distinct local build uploaded from your machine, mobile only. `source` is `uploaded`, `cache` (already uploaded by this machine) or `dry-run` (`app_id` empty, nothing sent) | +| `remote_dispatched` | `job_id`, `job_url` | The HyperExecute job exists, and the link opens the dashboard | +| `remote_error` | `code`, `detail` | Remote preflight refused the selection (codes above), followed by `testrun_done` and exit `2` | +| `remote_exec_sync`, `remote_coverage` | `status`, `reason`, `detail?` | Informational: assurance graph sync and coverage, skipped when the project has no `.context` store | +| `remote_done` | `status`, `exit`, `job_id`, `sessions_path` | Terminal for the remote wrapper, and follows `testrun_done` | + +`testrun_summary` also carries a `remote` object (`backend`, `jobId`, `jobUrl`, `sessionsPath`). + +## Next Steps + +- [Batch Runs (testrun)](/support/docs/kane-cli-testrun/) for selection, preflight, flags and exit codes +- [Mobile Testing](/support/docs/kane-cli-mobile/) for setup on your own machine, or skip it with `--remote` +- [Writing test.md files](/support/docs/kane-cli-testmd/#mobile-target) for `target:`, `app:`, `device_name:` and `os_version:` +- [Evidence Packs](/support/docs/kane-cli-evidence/) for what comes back and how to view it +- [CI/CD](/support/docs/kane-cli-cicd/) for pipeline patterns, including runners with no Chrome diff --git a/docs/kane-cli-testmd-running.md b/docs/kane-cli-testmd-running.md index 6e043670d..9cc44fbe6 100644 --- a/docs/kane-cli-testmd-running.md +++ b/docs/kane-cli-testmd-running.md @@ -280,6 +280,8 @@ Every flag accepted by `kane-cli testmd run`: Most flags have a frontmatter counterpart with the same name (with underscores). Where both are set, the CLI flag wins — except for `variables`, which the file owns; see [Writing test.md files](/support/docs/kane-cli-testmd/#variables). +Before any step runs, every `{{name}}` the authored steps reference must have a value, from the file's own `variables:` frontmatter, a variable file, `--variables-file`, or `--variables`. A name with no value stops the run before the browser starts, with exit `2` and a receipt naming the variable, the file waiting for its value, and the step. Replayed steps are not checked. See [Before a run](/support/docs/kane-cli-variables-and-context/#before-a-run-unresolved-variables). + ## How a run works A run proceeds in four phases: @@ -472,6 +474,8 @@ kane-cli testmd run ./tests/checkout_test.md \ In a non-interactive run (stdin is not a TTY), there is no one to answer an interactive `ask_user` prompt, so kane-cli disables it: a step that would otherwise wait for input fails cleanly instead of blocking forever. Write test steps that do not depend on mid-run prompts when running in CI. +If the runner cannot have Chrome at all, run the tests as a suite on the cloud grid instead. `kane-cli testrun run tests/ --remote` executes every `testmd run` on a HyperExecute macOS runner and returns the recordings and the evidence pack. See [Remote Runs](/support/docs/kane-cli-remote-execution/). + Capture exit code in a shell script: <VerifiedTag value="Verified" /> @@ -552,7 +556,6 @@ kane-cli testmd sync ./tests/checkout_test.md | Flag | Description | |---|---| -| `--env <name>` | Environment (`prod` or `stage`) | | `--username <user>` | Basic-auth username (skips OAuth) | | `--access-key <key>` | Basic-auth access key | diff --git a/docs/kane-cli-testmd.md b/docs/kane-cli-testmd.md index 432d87929..e1b2b7bdf 100644 --- a/docs/kane-cli-testmd.md +++ b/docs/kane-cli-testmd.md @@ -127,7 +127,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "Mobile Target", "codeSampleType": "code snippet", "programmingLanguage": "YAML", - "text": "---\ntarget: emulator # emulator (Android) | simulator (iOS)\napp: ./builds/app-debug.apk\nno_reset: false # optional\n---" + "text": "---\ntarget: emulator # emulator (Android) | simulator (iOS)\napp: ./builds/app-debug.apk # or an APP… id from kane-cli apps list --target emulator\ndevice_name: Pixel 7 # optional default device, with os_version\nos_version: \"14\" # optional, overridden by --device-name and --os-version\nno_reset: false # optional\n---" }, { "@type": "SoftwareSourceCode", @@ -307,9 +307,11 @@ headless: true | `code_export` | root + step | Generate Playwright code after the run | | `code_language` | root + step | `python` or `javascript` for code export | | `global_context` / `local_context` | root + step | Inline Markdown or file path for agent context | -| `target` | root | Where the test runs: a browser transport (`chrome`, the default, `cdp`, or `ws`) or a mobile target (`emulator` or `simulator`, macOS Apple Silicon). See [Mobile Target](#mobile-target). | +| `target` | root | Where the test runs: a browser transport (`chrome`, the default, `cdp`, or `ws`) or a mobile target (`emulator` or `simulator`, a virtual Android or iOS device, on macOS Apple Silicon or on the cloud grid with `testrun run --remote`). See [Mobile Target](#mobile-target). | | `app` | root | Mobile only. The app under test: a build path (emulator `.apk`, simulator `.zip`) or an uploaded `APP…` id. Required with a mobile target, rejected with a browser target. | | `no_reset` | root | Mobile only. Keep the app's existing state between runs instead of resetting it. | +| `device_name` | root | Mobile only. The default device for this test, as `kane-cli devices list --target <kind>` prints it (add `--remote` for the grid catalog). Overridden by `--device-name`. | +| `os_version` | root | Mobile only. The device's OS version (`14`, `17.5`). Overridden by `--os-version`. Required alongside `device_name`. | | `chrome_profile` | root only | Named Chrome profile under `~/.testmuai/kaneai/chrome-profiles/`. | | `cdp_endpoint` | root only | Reuse an external Chrome over CDP. | | `ws_endpoint` | root only | LambdaTest / Playwright WebSocket endpoint. | @@ -317,14 +319,16 @@ headless: true ### Mobile Target -On macOS Apple Silicon, `target:` also accepts the two mobile values, `emulator` for a virtual Android device and `simulator` for a virtual iOS device, with the app under test as its own root key: +`target:` also accepts the two mobile values, `emulator` for a virtual Android device and `simulator` for a virtual iOS device, with the app under test as its own root key: <VerifiedTag value="Verified" /> ```yaml --- target: emulator # emulator (Android) | simulator (iOS) -app: ./builds/app-debug.apk +app: ./builds/app-debug.apk # or an APP… id from `kane-cli apps list --target emulator` +device_name: Pixel 7 # optional default device, with os_version +os_version: "14" # optional, overridden by --device-name and --os-version no_reset: false # optional --- ``` @@ -332,12 +336,13 @@ no_reset: false # optional - **`target`**: `emulator` runs on an Android emulator, `simulator` on an iOS simulator. The platform never appears separately, the target implies it. - **`app`**: the app under test, required with a mobile target and rejected with a browser one. A build path (emulator `.apk`, simulator `.zip`) or an uploaded app id, `APP` followed by six or more digits. On-device package ids are not accepted. - **`no_reset`**: optional. Keep the app's existing state between runs instead of resetting it. +- **`device_name`** and **`os_version`**: optional per-test defaults for the device, in the vocabulary of `kane-cli devices list --target <kind>` for a device on this machine, or `kane-cli devices list --target <kind> --remote` for the grid catalog. The run flags `--device-name` and `--os-version` override them, and a name needs a version. :::warning The nested form, `target: {platform, app}`, is not accepted. The parser refuses it and spells out the flat shape above. ::: -Mobile tests run with `kane-cli testmd run`. A batch run does not support mobile members: a `_test.md` with a mobile target is rejected up front, before the suite runs. Setup is covered in [Mobile Testing](/support/docs/kane-cli-mobile/). +Mobile tests run with `kane-cli testmd run`, and a folder of them runs in one execution with [`testrun`](/support/docs/kane-cli-testrun/#mobile-members). On this machine that needs macOS Apple Silicon, and setup is covered in [Mobile Testing](/support/docs/kane-cli-mobile/). With [`testrun run --remote`](/support/docs/kane-cli-remote-execution/), the suite runs on a grid emulator or simulator from any machine. ### Root-only vs root-or-per-step diff --git a/docs/kane-cli-testrun.md b/docs/kane-cli-testrun.md index 7ba8f899c..e95c52c46 100644 --- a/docs/kane-cli-testrun.md +++ b/docs/kane-cli-testrun.md @@ -175,6 +175,7 @@ A member can fail preflight for these reasons: |---|---|---| | `org_mismatch` | Belongs to a different organisation than the rest | Check with `kane-cli testmd status <path>` | | `project_mismatch` | Belongs to a different project than the rest | Check with `kane-cli testmd status <path>`; run project-by-project | +| `unresolved_variables` | An authored step references a `{{name}}` that has no value in any variable file or in the member's own `variables:` frontmatter | Fill the value in `.testmuai/variables/*.json` (the receipt names the file) or remove the reference. `testrun run` has no `--variables` flag | If any member fails preflight, the plan is invalid and **nothing runs** (exit `2`). The offenders print to stderr: @@ -186,7 +187,36 @@ error: plan invalid — 2 offending test(s): tests/other_project_test.md: project_mismatch ``` -> **Mobile is not supported in a batch run.** A `_test.md` with a mobile [`target:`](/support/docs/kane-cli-testmd/#mobile-target) (`emulator` / `simulator`) is rejected up front, before the suite runs. Run mobile tests one at a time with `kane-cli testmd run <path>`. +Variable offenders get the full receipt instead of a one-line code: every unresolved name across the members, each with the test files and steps that use it. + +``` +✗ 2 variables have no value — nothing was dispatched + + Not in any variables file + other_key b_test.md step 1 + shared_url a_test.md step 1 · b_test.md step 1 + + Add them to .testmuai/variables/variables.json + + If {{name}} is literal page text, write \{{name}} to keep it as-is. + Fill the values and run again. +``` + +In agent mode (stdin is not a TTY) the same information arrives as one `error` event with `code: "unresolved_variables"` right after `testrun_plan`. See [Modes of Operation](/support/docs/kane-cli-modes/#unresolved-variables) for the shape. + +## Mobile members + +A `_test.md` with a mobile [`target:`](/support/docs/kane-cli-testmd/#mobile-target) (`emulator` or `simulator`) is a normal member: + +- **On this machine**, the suite drives the emulators and simulators installed here, so the host must be macOS Apple Silicon with the [mobile setup](/support/docs/kane-cli-mobile/#setup) done. Pick the device with `--device-name` and `--os-version` as `kane-cli devices list --target emulator|simulator` prints it, or set `device_name:` and `os_version:` in the file. +- **On the cloud grid** (`--remote`), the suite runs on a virtual device on a HyperExecute macOS host, so it works from any machine: Linux, Windows, or a Mac with no Xcode or Android Studio. Pick the device from `kane-cli devices list --target emulator|simulator --remote`. One grid job runs one platform, emulator members on one Android version and simulator members on one HyperExecute pool, and a member's local build is uploaded from your machine before dispatch and handed to the grid as an `APP…` id. Everything else is in [Remote Runs](/support/docs/kane-cli-remote-execution/). + +<VerifiedTag value="Verified" /> + +```bash +kane-cli testrun run tests/app/ --device-name "Pixel 7 API 35" --os-version 15 # devices on this machine +kane-cli testrun run tests/app/ --remote --device-name "Pixel 7" --os-version 14 # the cloud grid +``` ## Running @@ -202,7 +232,9 @@ error: plan invalid — 2 offending test(s): | `--retry-count <n>` | Max replay restart attempts before a full re-author | `3` | | `--bug-detection <mode>` | `off` \| `stop` \| `continue` — see [Configuration](/support/docs/kane-cli-configuration/#bug-detection) | config value | | `--headless` | Run Chrome without a visible window | off | -| `--env <name>` | Environment (`prod` or `stage`) | active env | +| `--remote [backend]` | Dispatch the suite to the cloud grid instead of Chrome or devices on this machine (default backend: `hyper`). Needs `kane-cli plugin install remote-execution`. See [Remote Runs](/support/docs/kane-cli-remote-execution/) | off | +| `--device-name <name>` | Device for the suite's mobile members: as `kane-cli devices list --target <kind>` prints it locally, or a grid catalog device with `--remote` | member's `device_name:` | +| `--os-version <version>` | OS version for the mobile members (`14`, `17.5`). On its own, it matches any device running it | member's `os_version:` | | `--username <user>` / `--access-key <key>` | Basic auth (skips OAuth) | — | Each worker gets its **own isolated Chrome** with a fresh temporary profile, so parallel members never share cookies, logins, or tabs — and never fight over your real browser profile. diff --git a/docs/kane-cli-troubleshooting.md b/docs/kane-cli-troubleshooting.md index e85965847..e591957ca 100644 --- a/docs/kane-cli-troubleshooting.md +++ b/docs/kane-cli-troubleshooting.md @@ -251,7 +251,7 @@ import VerifiedTag from '@site/src/component/verifiedTag'; "name": "Start every mobile problem with doctor, which prints one line per required check, each with a fix", "codeSampleType": "code snippet", "programmingLanguage": "Shell", - "text": "kane-cli doctor # required checks, each with a fix if it fails\nkane-cli doctor --install # install the test tooling Kane CLI manages\nkane-cli doctor --targets # list the emulators and simulators available" + "text": "# iOS Simulator\nkane-cli doctor --target simulator # required checks, each with a fix if it fails\nkane-cli doctor --target simulator --install # install the test tooling Kane CLI manages\nkane-cli devices list --target simulator # the simulators Kane CLI can run against\n\n# Android Emulator\nkane-cli doctor --target emulator\nkane-cli doctor --target emulator --install\nkane-cli devices list --target emulator" } ], "dateModified": "2026-09-07T15:18:23+05:30" @@ -495,6 +495,14 @@ If your environment also breaks `kane-cli login`, apply the Node-side fix in the ### "Variables not resolving": `{{key}}` appears literally +A run **refuses to start** when an authored step references a `{{name}}` that has no value. You get a receipt naming each variable, the file that is waiting for its value (or `Not in any variables file`), and the step that uses it, with exit code `2` and nothing dispatched. Read the receipt first: it tells you whether to fill an existing key or add a new one, and which file. See [Before a run](/support/docs/kane-cli-variables-and-context/#before-a-run-unresolved-variables). + +If a `{{my_var}}` placeholder appears **literally** in a browser action, one of three things is true: + +- **The step is a replay.** Replayed steps resolve from their tape and from Test Manager, and a missing value there produces a warning line rather than a refusal. Fill the value and run again. +- **The reference is escaped.** `\{{my_var}}` is typed as is on purpose, for pages where the braces are real text. +- **The variable file is not being loaded at all.** Check the three points below. + **Cause:** Variable file not loaded, wrong JSON format, or wrong variable key name. **Fix:** @@ -663,14 +671,22 @@ Two other triggers: npm configured to skip optional dependencies (`npm config ge ## Mobile Issues -Mobile testing is supported on **macOS Apple Silicon (arm64) only**. Start every mobile problem with `doctor`, which prints one line per required check, each with a fix: +Mobile runs on your own machine are supported on **macOS Apple Silicon (arm64) only**. On other machines, run mobile suites on the cloud grid with `kane-cli testrun run --remote`, see [Running a Mobile Suite on the Cloud Grid](/support/docs/kane-cli-mobile/#running-a-mobile-suite-on-the-cloud-grid). + +Start every local mobile problem with `doctor`, which prints one line per required check, each with a fix. `doctor` checks one target at a time, so pass `--target`: <VerifiedTag value="Verified" /> ```bash -kane-cli doctor # required checks, each with a fix if it fails -kane-cli doctor --install # install the test tooling Kane CLI manages -kane-cli doctor --targets # list the emulators and simulators available +# iOS Simulator +kane-cli doctor --target simulator # required checks, each with a fix if it fails +kane-cli doctor --target simulator --install # install the test tooling Kane CLI manages +kane-cli devices list --target simulator # the simulators Kane CLI can run against + +# Android Emulator +kane-cli doctor --target emulator +kane-cli doctor --target emulator --install +kane-cli devices list --target emulator ``` The common setup failures for each platform, and their fixes, are listed on the mobile testing page: @@ -700,6 +716,16 @@ The full walkthrough is in [Evidence packs → Debugging a failed run from its p One more debugging aid for batch runs: `testrun` members normally run silently — set `KANE_TESTRUN_MEMBER_DEBUG=1` to route their per-member output to stderr (prefixed `[member]`). +## A `--remote` run refused, failed, or came back empty + +Three different situations, told apart by the exit code and what came back: + +- **Exit `2` and no job link.** The remote preflight refused the selection before anything was dispatched. The reason is printed with the offending paths: the plugin is missing (`kane-cli plugin install remote-execution`, then `kane-cli plugin doctor remote-execution`), the selection mixes web and device tests or two mobile platforms (run them as two suites), a device test names a build that is not on this machine or that the cloud cannot take (`mobile_app_missing`, `mobile_app_not_uploadable`), the build's upload from your machine failed (`mobile_app_upload_failed`), or required recordings are gitignored (`gitignored_inputs`, un-ignore with `!output-*/`). `--dry-run` reproduces the check without creating a job. +- **Exit `1` with recordings and a pack.** The job ran and a member failed. Debug it like a failure on your own machine: `output-<stem>/Result.md` names the step and reason, and the pack has the screenshots and logs (next section). +- **A member reported `broken` with no steps, nothing published.** The grid-side Kane CLI refused before launching. Open the printed job link and read the scenario stage's log. For mobile members the usual cause is an `APP…` id that belongs to a different organisation than the account running the job, and `kane-cli apps list --target <kind>` for the active profile is the authority. The members' session logs are also under `~/.testmuai/kaneai/sessions/remote/<job-id>/`. + +Full reference: [Remote Runs](/support/docs/kane-cli-remote-execution/). + --- ## testrun says "plan invalid" or skips members @@ -715,6 +741,68 @@ One more debugging aid for batch runs: `testrun` members normally run silently Use `kane-cli testrun run --dry-run …` to see the full plan and every offender without executing anything. See [Batch runs with testrun](/support/docs/kane-cli-testrun/#preflight). +## Context sync: Git not found or too old + +A GitHub location (`kane-cli context sync add`, `kane-cli context clone`, or `kane-cli context sync setup`) needs Git **2.31 or newer** on the machine that runs Kane CLI. Without it the command refuses before touching anything: + +``` +$ kane-cli context sync add team git@github.com:example-org/team-context.git +checking team: read access and safe publishing… +error: This location needs Git 2.31 or newer. +next: Install or update Git, then run setup again: https://git-scm.com/downloads +``` + +Install Git from the link, or update it (`brew install git`, `apt-get install git`, or the Windows installer), open a new terminal so `git --version` reports 2.31 or newer, and run the same command again. On a CI runner, add a Git install step before Kane CLI. Folder and S3-compatible locations do not need Git at all, see [Sharing the context graph](/support/docs/kane-cli-assurance-sharing/#locations). + +## Context sync: a location cannot be reached or refuses you + +Two different refusals, told apart by the message: + +- **Nothing answered.** The folder does not exist and cannot be created, the host is down, or no repository answered at that address, and a private repository you have not been invited to looks missing: + + ``` + error: nothing answered at /no/such/parent/team-context: that folder does not exist and cannot be created + next: check the path, then run the command again + ``` + + Check the address (`kane-cli context sync list` shows what the store has), that the drive is mounted, and that the repository exists and is shared with your account. + +- **The location answered and refused you.** Keys, a key pair, or an account without read permission: + + ``` + error: the bucket at https://team-context.s3.eu-west-1.amazonaws.com did not accept origin's access keys, even for reading + next: ask the owner of the bucket for access, or bind it again with the right keys: kane-cli context sync add origin <descriptor> --credential-env <VAR> + ``` + + `<descriptor>` in that line is the address of the location. Ask the owner for access, or bind the location again under the same name with the right keys: `kane-cli context sync add` on an existing name replaces its keys, and a pair the location refuses never replaces one that worked. For a GitHub location over HTTPS in CI, check that `KANE_SYNC_GIT_TOKEN` grants Contents read and write on *that* repository. + +- **Over SSH.** Kane CLI never answers an SSH prompt. A host key this computer has not accepted yet refuses with `this computer has not accepted github.com's SSH host key yet`: run `ssh -T git@github.com` once in a terminal and answer yes. A key that is not loaded, or not on the account, refuses with `github.com did not accept your SSH key`: load it with `ssh-add`, or ask the repository owner to grant your account access. Over HTTPS without a stored login the message is `sign in is needed, or your account has no access to this repository on github.com`: sign in through Git with `gh auth login --hostname github.com --git-protocol https --web`, then `gh auth setup-git --hostname github.com`, and run the command again. + +- **The connection check could not finish.** `The server did not complete the concurrent connection check. Its scratch-ref policy may differ from the storage branch.` (`SYNC_PROBE_INCONCLUSIVE`) means the concurrent connection check did not finish, and repository rules that block the scratch reference the check pushes under `refs/kane/probe/` are the usual cause. Nothing was bound: ask the repository owner to check that the reference is allowed, then run the command again. + +A location you can read but not write is not a refusal: it binds as download-only (`read-only: this location can be cloned and pulled, never pushed`). `kane-cli context push` to it refuses with `SYNC_READ_ONLY`, and so does `kane-cli context sync`, after its pull has already landed, so its exit `2` does not mean nothing happened. Take records from such a location with `kane-cli context pull`. The two refusals above change nothing in your store. + +## Context sync: "a rebase is open" and every change is refused + +``` +error: a rebase (2026-09-14T10-13-45-679Z-reset) is open with 2 decisions unresolved; this store takes no other change until it is finished — run kane-cli context sync or kane-cli context pull to continue, or kane-cli context sync doctor --abort to close it +next: run kane-cli context sync or kane-cli context pull to continue, or kane-cli context sync doctor --abort to close it +``` + +A `kane-cli context pull origin --rebase` stopped on decisions you have not answered yet, or was interrupted, and until it is finished the store takes writes from the rebase only, exactly like git mid-rebase. `kane-cli context extract`, `kane-cli design tests`, `kane-cli maintain reconcile`, `kane-cli context ingest`, `kane-cli context review`, `kane-cli context name`, `kane-cli context retire` and `kane-cli context revert` refuse with these two lines. `kane-cli context push` refuses with the same code (`SYNC_REBASE_PENDING`) and the same `next:` line, its reason naming the rebase and saying nothing was pushed. A test run does not refuse, and nothing is lost: its results wait to the side and land on the first run after the rebase closes. + +1. See what is open: `kane-cli context sync status origin` lists each decision on one line, and `--show <n>` prints one saved record in full. +2. Answer: `kane-cli context sync origin` walks the cards on a terminal. Headless, use `kane-cli context sync origin --answer <id>=keep-theirs` (or `apply-mine`), one flag per decision. `keep-theirs` writes nothing, but it is a choice, not a default: the local change stays in the backup unapplied, and a later record that was built on it is looked at again. +3. Or close it: `kane-cli context sync doctor --abort` keeps what was already reapplied and leaves the unanswered records in the backup. A rebase interrupted before it imported origin's records is undone by the same command, and the store is put back as it was. Once the import has landed it can only be finished (`SYNC_RESET_IMPORTED`): run `kane-cli context sync` or `kane-cli context pull` to finish it, then close it if you still want to. `kane-cli context sync doctor --export <dir>` rebuilds the pre-rebase store beside, as its own store. + +:::warning +Never delete files under `.context/` to get past the refusal. See [When two people changed the same thing](/support/docs/kane-cli-assurance-sharing/#rebase). +::: + +## Context sync: publication could not be confirmed + +A `kane-cli context push` to a GitHub location can lose the server's answer after the upload, through a dropped connection or a proxy timeout. Kane CLI then refuses with `SYNC_PUBLICATION_UNKNOWN` (exit `2`) rather than guess, because the batch may have landed. Your store is unchanged and nothing needs to be redone. Check connectivity and run the **same** command again: it reads the location first and reconciles what actually landed, so a batch that arrived is recognised as already there and never published twice, and a batch that did not is published now. Only after that should anything else run. If a proxy rejects large uploads, `KANE_SYNC_GIT_HTTP_POST_BUFFER=33554432` raises Git's upload buffer for that command, and a very slow link gets more time with `KANE_SYNC_GIT_TRANSFER_TIMEOUT_SECONDS`. See [Context sync environment variables](/support/docs/kane-cli-configuration/#context-sync-environment-variables). + --- ## Filing a Bug Report diff --git a/docs/kane-cli-variables-and-context.md b/docs/kane-cli-variables-and-context.md index 0860221a8..41cd6ef67 100644 --- a/docs/kane-cli-variables-and-context.md +++ b/docs/kane-cli-variables-and-context.md @@ -198,7 +198,7 @@ Variables are JSON objects keyed by name. Each entry describes a single variable | Field | Required | Type | Default | Description | |-------|----------|------|---------|-------------| -| `value` | Yes | string | None | The variable's value. Entries without `value` are ignored. | +| `value` | Yes | string | None | The variable's value. A number is accepted and loaded as its string (`8080` becomes `"8080"`), while a boolean, object or array is not a value. Entries without `value` are ignored. | | `secret` | No | boolean | `false` | When `true`, the value is masked in logs and routed to the <BrandName /> secrets store instead of being synced as plain Test Manager variables. | | `syntax` | no | string | `{{<name>}}` | Custom placeholder syntax. Defaults to the double-brace form using the variable name. | @@ -315,6 +315,44 @@ Mark a variable as secret by setting `"secret": true`: Secret values are masked in displayed output and logs, and are routed to the <BrandName /> secrets store instead of being synced to Test Manager as plain variables. Use this for credentials, tokens, and anything else that should not appear in shareable artifacts. +### Variables declared by `kane-cli design tests` + +When a design session needs a value nobody has, such as a start URL or a test account, it references it as `{{name}}` and declares it. Before choosing a name, design reads every `*.json` in both variable directories and reuses a name you already have rather than inventing a new one. The agent sees only which names exist and whether each has a value, never the values themselves. + +For each name it declares, Kane CLI writes an **empty stub** into `.testmuai/variables/assurance.json`, never a value, and tells you during the round (`⚒ declared 2 new variables — login_email · login_password (values needed)`) and again at the end (`2 variable(s) need values — see .testmuai/variables/assurance.json`): + +```json +{ + "login_email": { "value": "", "secret": false, "description": "the account the login tests sign in with" } +} +``` + +Fill `value`. Kane CLI never overwrites a key that already exists in any of your variable files, so a stub cannot replace a value you set. A value you type in chat stays a literal in the step, and a variable is created only when the value is unknown. + +### Before a run: unresolved variables + +`kane-cli run`, `kane-cli testmd run` and `kane-cli testrun run` check every `{{name}}` an authored step references **before anything starts**, with no browser and no session. A name with no value stops the run there, with exit code `2`, and the receipt says what each one needs: + +``` +✗ 3 variables have no value — nothing was dispatched + + Waiting for a value in .testmuai/variables/assurance.json + storefront_sign_in_url step 1 + registered_customer_email step 2 + + Not in any variables file + checkout_url step 4 + + Add it to .testmuai/variables/assurance.json + + If {{name}} is literal page text, write \{{name}} to keep it as-is. + Fill the values and run again. +``` + +The pool file is named once per group. A test filename appears only when it is not the file you named: an `@import`ed unit and a testrun member both keep theirs, and a `kane-cli run` objective shows no location at all. A name that is in no file gets `Add it to <file>` when a pool file exists, or the JSON to create when there is none yet. + +There is no flag to bypass the check: fill the value or remove the reference. Never checked are `{{smart.*}}`, `{{environment.*}}`, `{{secrets.*}}` and `{{totp.*}}` (resolved at run time), a name an earlier step stored (`store the price as 'price'`), a test's own frontmatter `variables:`, and replayed steps, which resolve from their tape and from Test Manager, where a replay with a missing value gets a warning line and never a refusal. `${x}` is not a variable reference on this path. In agent mode the refusal is one typed `error` event with `code: "unresolved_variables"`, see [Modes of Operation](/support/docs/kane-cli-modes/#unresolved-variables). + :::warning Do not commit credential files to version control. Add `.testmuai/variables/` to your `.gitignore`, or use environment variable substitution in CI/CD. ::: diff --git a/sidebars.js b/sidebars.js index 9e75d1e85..5f896a5cf 100644 --- a/sidebars.js +++ b/sidebars.js @@ -4652,9 +4652,20 @@ module.exports = { ], }, { - type: "doc", + type: "category", + collapsed: true, label: "Batch Runs (testrun)", - id: "kane-cli-testrun", + link: { + type: "doc", + id: "kane-cli-testrun", + }, + items: [ + { + type: "doc", + label: "Remote Runs", + id: "kane-cli-remote-execution", + }, + ], }, ], }, @@ -4736,6 +4747,11 @@ module.exports = { label: "Maintaining the Suite", id: "kane-cli-assurance-maintain", }, + { + type: "doc", + label: "Sharing with Your Team", + id: "kane-cli-assurance-sharing", + }, { type: "doc", label: "Agents & CI",