From ed510886333b49ecd68a3aadbb4c59d009da2af9 Mon Sep 17 00:00:00 2001 From: Dunqing Date: Thu, 22 Oct 2026 01:49:53 +0800 Subject: [PATCH 1/2] docs(e2e): describe argv steps in the snapshot test README The README showed string steps ("vp run build") and a table form with `command`, and said steps run through /bin/sh. `Step` in main.rs is an argv array or a table with `argv`, a string step makes the test binary panic while loading fixtures, and steps start without a shell in a cleared environment. Update the examples and notes to match: the table and case keys, how `vt`/`vtt` are found, the full `write-key` list, `vtt pipe-stdin` instead of shell pipes, the step timeout, and how new snapshots are written and accepted. --- crates/vt_bin/tests/e2e_snapshots/README.md | 33 +++++++++++---------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/crates/vt_bin/tests/e2e_snapshots/README.md b/crates/vt_bin/tests/e2e_snapshots/README.md index 7814b283e..469907e87 100644 --- a/crates/vt_bin/tests/e2e_snapshots/README.md +++ b/crates/vt_bin/tests/e2e_snapshots/README.md @@ -11,43 +11,46 @@ End-to-end tests that execute the `vp` binary and verify its output. ## How it works -Each fixture in `fixtures/` is a self-contained workspace. Tests are defined in `snapshots.toml`: +Each fixture in `fixtures/` is a self-contained workspace. Tests are defined in `snapshots.toml`. A step is the argv of one command, as an array: ```toml [[e2e]] -name = "descriptive test name" +name = "descriptive_test_name" steps = [ - "vp run build", - "vp run build", # second run to test caching + ["vt", "run", "build"], + ["vt", "run", "build"], # second run to test caching ] ``` -Steps also support an object form with interactions: +Steps also support a table form with more options: ```toml [[e2e]] -name = "interactive step" +name = "interactive_step" steps = [ - { command = "vp interact", interactions = [{ expect-milestone = "ready" }, { write = "hello" }, { write-line = "world" }, { write-key = "up" }, { write-key = "down" }, { write-key = "enter" }] }, - "echo -n | node check-stdin.js", + { argv = ["vt", "run", "dev"], comment = "cache miss", envs = [["MY_ENV", "1"]], interactions = [{ expect-milestone = "ready" }, { write = "hello" }, { write-line = "world" }, { write-key = "up" }, { write-key = "down" }, { write-key = "enter" }] }, + ["vtt", "pipe-stdin", "hello", "--", "vt", "run", "read-stdin"], ] ``` Notes: -- String steps are shorthand for `{ command = "..." }`. -- `write-key` accepts `up`, `down`, and `enter`. +- An array step is shorthand for `{ argv = [...] }`. A string step such as `"vt run build"` doesn't parse, and the test binary panics while loading the fixtures, before any case runs. +- A table step takes `argv` (required), `cwd` (relative to the fixture root, defaulting to the case's `cwd`), `comment` (shown under the step's heading in the snapshot), `envs` (`[name, value]` pairs), `interactions`, and `formatted-snapshot` (keep colors and styles in the snapshot). Other keys are an error. +- A case takes `name` (ASCII letters, digits, and `_` only; it also names the snapshot file), `steps`, and optionally `comment`, `cwd`, `cfg` (a Rust cfg expression such as `'target_os = "linux"'`; the case runs only where it matches), and `ignore` (the case runs only with `-- --ignored`). +- `vt` and `vtt` are the binaries built for the test. Other programs are looked up in `PATH`, with `packages/tools/node_modules/.bin` (when installed) ahead of the inherited entries. +- `write-key` accepts `up`, `down`, `enter`, `escape`, `backspace`, and `ctrl-c`. - Snapshots include every interaction line, and each `expect-milestone` records the screen at that point. -- For stdin pipe scenarios, write the step command with shell piping, for example: `echo -n | command`. +- Steps don't run in a shell, so pipes, `&&`, and redirections aren't available. To pipe data to a command's stdin, use `vtt pipe-stdin -- [args...]`. The test runner: 1. Copies the fixture to a temp directory -2. Executes each step using `/bin/sh` (Unix) or `bash` (Windows) -3. Runs each step in PTY mode (`TestTerminal`) +2. Starts each step's `argv` directly, without a shell, in an environment holding only `PATH`, `TERM`, and the step's `envs` (plus `PLAYWRIGHT_HOST_PLATFORM_OVERRIDE` when set, and a few system variables on Windows) +3. Runs each step in PTY mode (`TestTerminal`), with a timeout of 20 seconds (60 seconds on Windows and x86_64 macOS) 4. Applies configured interactions in order for PTY steps 5. Captures output and exit codes -6. Compares against snapshot in `fixtures//snapshots/` +6. Compares against the snapshot `fixtures//snapshots/.md` ## Adding a new test @@ -55,4 +58,4 @@ The test runner: 2. Add `package.json` (and `pnpm-workspace.yaml` for monorepos) 3. Add `snapshots.toml` with test cases 4. Run `cargo test -p vt_bin --test e2e_snapshots` -5. Review and accept new snapshots +5. Review the new snapshots, written next to the expected ones as `.md.new`, and accept them by re-running with `UPDATE_SNAPSHOTS=1` From 33772a1b356c8da2b5b4037b0cf829b4f64c4a65 Mon Sep 17 00:00:00 2001 From: Dunqing Date: Thu, 22 Oct 2026 11:04:09 +0800 Subject: [PATCH 2/2] docs(e2e): say the tests run `vt`, and where a step comment appears The README said the tests execute `vp`; they run the `vt` binary built for the test. The `comment` field of a step was documented as appended to the command line, but the snapshot shows it under the step heading. --- crates/vt_bin/tests/e2e_snapshots/README.md | 2 +- crates/vt_bin/tests/e2e_snapshots/main.rs | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/crates/vt_bin/tests/e2e_snapshots/README.md b/crates/vt_bin/tests/e2e_snapshots/README.md index 469907e87..962030e6d 100644 --- a/crates/vt_bin/tests/e2e_snapshots/README.md +++ b/crates/vt_bin/tests/e2e_snapshots/README.md @@ -1,6 +1,6 @@ # E2E Snapshot Tests -End-to-end tests that execute the `vp` binary and verify its output. +End-to-end tests that execute the `vt` binary and verify its output. ## When to add tests here diff --git a/crates/vt_bin/tests/e2e_snapshots/main.rs b/crates/vt_bin/tests/e2e_snapshots/main.rs index 0e4d1c819..a9f454d03 100644 --- a/crates/vt_bin/tests/e2e_snapshots/main.rs +++ b/crates/vt_bin/tests/e2e_snapshots/main.rs @@ -46,7 +46,7 @@ struct StepConfig { /// Defaults to the case-level `cwd`. #[serde(default)] cwd: Option, - /// Appended as `# comment` in the snapshot display line. + /// Shown under the step's heading in the snapshot. #[serde(default)] comment: Option, /// Extra environment variables set for this step.