Skip to content

test(evals): exercise the workspace V4 CLI - #2838

Draft
shrey150 wants to merge 10 commits into
agent/browse-v4-5-remove-xpathfrom
agent/browse-v4-6-release
Draft

test(evals): exercise the workspace V4 CLI#2838
shrey150 wants to merge 10 commits into
agent/browse-v4-5-remove-xpathfrom
agent/browse-v4-6-release

Conversation

@shrey150

@shrey150 shrey150 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Make workspace evals and packaging consume the V4 Browse CLI proven in #2835.

  • Point Stagehand evals at the workspace Browse package instead of published Browse 0.9.5.
  • Update native and external-agent adapters to the current Browse command grammar.
  • Keep eval session names within Unix socket limits, restrict the wrapper to driver commands, and clean daemons up on abort/signals.
  • Add command-mapping and real wrapper-process coverage.

The Browse migration changeset now belongs to #2835, the first complete supported-parity head; this PR has no changeset in its own diff. The unrelated eval CLI --verbose changes from the old draft are intentionally excluded.

Stack (#2872)

  1. chore(cli): import Browse V3 baseline #2833 — exact Browse V3 baseline import
  2. refactor(cli): migrate Browse runtime and commands to Stagehand V4 #2834 — Stagehand V4 runtime and standard command parity
  3. feat(cli): restore cursor overlay through page.evaluate #2869 — CLI-owned cursor overlay
  4. fix(cli): restore V3 network capture through a CDP sidecar #2849 — CLI-private CDP sidecar; V3 network parity
  5. refactor(cli): remove coordinate XPath return flag #2835 — remove --return-xpath; supported V3 parity/release checkpoint
  6. test(evals): exercise the workspace V4 CLI #2838 — eval and packaging integration
  7. fix(cli): persist context names in Browserbase #2839 — managed Context names (fast-follow)
  8. refactor(cli): consume shared Functions core #2701 — shared Functions core consumer (fast-follow)

Review boundary

This is a consumer/integration layer, not another runtime-parity layer. Its diff is limited to eval adapters, their tests, workspace dependency wiring, and the lockfile.

E2E verification

Flow Observed result
Isolated packed Stagehand + Browse consumer Browse resolved the packed Stagehand package from isolated node_modules; install audit reported 0 vulnerabilities
Installed node_modules/.bin/browse in real Chromium Open/snapshot/fill/focus/press/bare type/ref click/DOM reads passed; final output Submitted: TypedByBareCommand
Installed cursor + coordinate mouse Cursor DOM moved to 211px, 233px, with SVG present
Real BrowseCliTool subprocess Input AdapterBareType produced Submitted: AdapterBareType; daemon cleanup succeeded
Integration proof-commit typecheck Passed on 5a9ab832b2 after a clean frozen install and dependency-closure build
Integration proof-commit eval suite 58 files / 446 tests passed
Integration proof-commit eval build ESM and CLI builds passed
Changeset status Valid; Browse minor bump inherited from #2835, no migration changeset owned here

The packed consumer and wrapper-process smokes use real processes and Chromium rather than mocked command handlers.

@changeset-bot

changeset-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b3a3b41

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant