Phantom

A local, agent-first browser. Phantom launches a real browser — a bundled ungoogled-chromium, with Google's web-service integration stripped out — compiles each page's accessibility tree into a compact SST (Semantic State Tree) that any LLM can read cheaply and act on precisely, and exposes a small set of verbs over MCP (Model Context Protocol, stdio). Point Claude, GPT, Grok — any MCP-capable model — at it, and the model drives a browser that already knows how to log in, stay stealthy, and hand control back to you when a decision is yours.

It is not a Chromium fork and not a headless-automation wrapper bolted onto a custom UI. It is the normal engine, driven through the DevTools Protocol, with three things layered on top: a perception format built for models (the SST), an identity layer that runs your password manager in place, and a stealth layer so bot-management doesn't wall the agent off.

Priorities

1. Agent-first, LLM-agnostic. The browser is a fixed local tool; the model is a swappable remote brain that connects over MCP. No provider lock-in, no phone-home.
2. Stealth. Sites shouldn't be able to tell an agent is driving. Clean automation fingerprint, and human-like input engaged automatically on pages that run bot-management.
3. Co-pilot for humans. A person can watch and take over — literally the same window, with PHANTOM_COPILOT=1.

The one fixed guardrail: money is human-authorized. The agent can prepare a transfer or a purchase, but a person confirms it. Everything else leans on the design tenet surface, don't impose — Phantom shows you the choice rather than making it for you.

Architecture

┌────────────┐   spawn + Job Object    ┌──────────┐
│ phantom-   │────────────────────────▶│ Chromium │   real engine, throwaway
│ pool       │   DevToolsActivePort    └────┬─────┘   or durable named profile
└────────────┘                              │ ws (CDP)
┌────────────┐  JSON-RPC mux, sessions ┌────▼─────┐
│ phantom-   │◀────────────────────────│ phantom- │   thin CDP client +
│ sst        │  AX tree + DOMSnapshot  │ cdp      │   input humanizer
└────┬───────┘                         └────┬─────┘
     │ SST + macro-ids                      │ Storage.setCookies
┌────▼───────┐   MCP over stdio (rmcp) ┌────▼─────┐
│ phantom-   │◀──────────────────────  │ phantom- │   Firefox cookie import
│ ipc        │   navigate/click/type/… │ vault    │
└────────────┘        ▲                └──────────┘
                      │ MCP
                  LLM host (Claude Desktop, …)

Crates

crate	milestone	what it does
phantom-pool	M1	find + launch Chromium (--remote-debugging-port=0), Windows Job Object kill-on-close, DevTools ws discovery, ephemeral and durable named profiles, teardown/sweep
phantom-cdp	M1 / M5	thin CDP client (one ws, calls muxed by id, per-tab flat sessions, event broadcast); Tab helpers (navigate, eval, input, quads, screenshots); the humanizer (Bézier mouse + typing cadence) and stealth UA/Client-Hints
phantom-sst	M2	Accessibility.getFullAXTree + DOMSnapshot.captureSnapshot → pruned SST; monotonic macro-ids on interactive nodes; node/child/depth budgets with expandable … markers; screenshot-fallback flag for canvas/embeds
phantom-vault	M4	import cookies from the user's other browsers (Firefox first — plaintext moz_cookies) and inject them over CDP so Phantom starts already-logged-in
phantom-ipc	M3 / M4	the verb core + MCP server (rmcp v2, stdio); the phantom setup/import CLI; password-manager autofill; wires the humanizer dial

The contract

Every verb returns one envelope — success or failure, always carrying the fresh SST so the model can re-orient in a single step:

{
  "ok": true,
  "message": "what happened, in plain language",
  "page": { "url": "…", "title": "…", "viewport": {…} },
  "sst_unchanged": true,                  // no tree follows: the one you have is current
  "sst_omitted": "large",                 // no tree follows: a summary does instead
  "error": { "kind": "…", "hint": "…" }   // only on failure — WITH the current tree
}

The tree follows in a second text block, as an indented outline — one line per node, indentation for nesting:

page The Internet
  h2 Login Page
  form
    Username
    textbox #1 Username = ada +focused
    button #3 Login
  link #4 Elemental Selenium > http://elementalselenium.com/

A line with no role word is prose. #12 is the macro-id you act on. > is where a link goes, = is what a field holds, +flag is state, !verb marks an action the role would not have implied. h1…h6 are headings.

It is JSON no longer because JSON was mostly punctuation: a trimmed tree and this outline are the same size in bytes, but the JSON spent four fifths of its characters on braces, quoted keys and commas — and punctuation is what tokenizes worst. Nothing parses this back; it is written, sent, and read.

Errors are recoverable data, not thrown exceptions. Kinds: stale_id · not_actionable · no_change · not_found · timeout · navigation_failed.

A macro-id stays valid for as long as its element is on the page — through as many photos as you like. Ids only ever climb, so one whose element is gone is detected as stale rather than silently re-targeting something else.

Because ids survive, so does the tree: when a verb finds the page byte-for-byte what it already sent, it returns "sst_unchanged": true and no sst at all, rather than restating a page nobody touched.

Between "identical" and "here is the whole page again" sits the case that happens most and cost the most: a page that changed a little. Acting on a page nearly always does. There, the envelope says "sst_delta": true and carries the difference against the tree you were last sent — which lines moved, what they said, what they say now, and a count of everything that did not. On a comment thread, reporting a filled-in field went from 24,518 bytes to 468.

A delta is only ever computed against a tree the agent was actually handed, and a run of them ends on its own after a handful, because a host may compact that baseline away and nothing tells us when. read_page always answers in full, so a reader who has lost the thread has somewhere to go.

Going back is the same idea taken further. A history navigation builds a new document, so the page is renumbered from scratch and used to cost the whole tree again — 11,108 bytes to learn that nothing had changed, and every id you were holding silently wrong. If the tree is the same tree apart from the numbering, the old numbers go back on: the page is then byte-identical to the one you have, it says so, and your ids still point where they did. 206 bytes. One line different anywhere and it is simply sent, because re-using numbering on a page that moved on would aim a held id at a different element.

A tree can also be left out for being large. Clicking a link used to deliver the whole destination — 25 KB of comments in answer to "open this discussion" — before the caller could say whether it wanted the contents or merely the fact of having arrived. Past a size budget the envelope says "sst_omitted": "large" and carries a summary instead: where you landed, how much is actionable, the headings, and the three ways to ask for more. About 90 tokens against 6,000. The budget is larger for verbs whose purpose was to show you a page (navigate, go_back, scroll, the tab verbs) than for verbs asked to do something, where the page is a side effect.

read_page does neither. It always answers in full — it is the way back for a caller that has lost the thread, and an escape hatch with a size limit is not an escape hatch. Ask it narrowly and it is cheap: read_page(query="…") returns just what matches and its id, 60 bytes against 9,000 for reading a page to find one control you could already name.

The verbs: navigate · read_page · click · type · hover · scroll · press · select · screenshot · go_back · wait_for · autofill_login · list_tabs · switch_tab · open_tab · close_tab.

Verbs act on one tab — the active one. Once a second tab exists the envelope grows a tabs array, so a link that opened its own tab is visible immediately; each tab keeps its own id registry, so an id from elsewhere is refused by name rather than acted on.

Full verb-by-verb reference: docs/TOOLS.md.

Quick start

cargo build --workspace

# 1. Run the MCP server on stdio (this is what an LLM host launches):
cargo run -p phantom-ipc --bin phantom

# 2. One-time: install your password manager into a durable profile (headed):
PHANTOM_PROFILE=main PHANTOM_MANAGER="proton pass" \
  cargo run -p phantom-ipc --bin phantom -- setup

# 3. Optional: import your Firefox logins (cookies) into that profile:
PHANTOM_PROFILE=main cargo run -p phantom-ipc --bin phantom -- import

# Or just open a browser to drive yourself (co-pilot; uses the same profile):
PHANTOM_PROFILE=main cargo run -p phantom-ipc --bin phantom -- open

On Windows PowerShell, set env vars first: $env:PHANTOM_PROFILE = "main" then run the command (the VAR=value cmd form is bash-only).

Connecting the server to an MCP host (e.g. Claude Desktop): see docs/MCP.md. phantom connect does it without hand-editing — Claude Desktop by default, phantom connect --list for the others it knows (Claude Code, Cursor, VS Code, Cline, Windsurf) and --print for anything else.

The bundled browser is a portable tree, so nothing updates it on its own: phantom update-engine fetches the current ungoogled-chromium release (--check just says whether one is waiting). It refuses while that browser is running, and puts the old tree back if the swap fails.

Co-pilot (M7)

By default the agent drives its own browser: its own window, its own copy of your profile. That works, and it means you cannot see what it is doing without going and looking, and that the copy has to be kept in step with the original.

PHANTOM_COPILOT=1 makes it join your window instead.

"env": {
  "PHANTOM_COPILOT": "1",        // share the window; "1" ⇒ the PHANTOM_PROFILE below
  "PHANTOM_PROFILE": "main"
}

Then phantom open (or the Desktop shortcut) is the browser you both use. The agent finds it through the profile's DevToolsActivePort — Chromium writes it, the profile path is deterministic, so there is no IPC, no lockfile and no state of Phantom's to go stale. It adopts the tabs already open and starts on the one you are actually looking at, which it works out by asking each tab whether document.visibilityState says it is visible.

Three properties worth knowing before you turn it on:

• It acts as you. Your session, your logins, your cookies. That is the point of sharing a window, and it is why this is opt-in rather than the default.
• It will not close your browser. A window the agent joined is one it borrowed; shutting the agent down leaves it exactly where it was.
• If nothing is running, it opens one — the same window phantom open would have — and then disowns it. Co-pilot mode should never be the reason a browser is missing, and never the reason one disappears.

PHANTOM_COPILOT=<name> shares a specific profile instead. Unset, 0, or off keeps the old behaviour.

The CLI is kept running between messages. A process per message spent about two seconds before the model did anything — Node starting, Claude Code loading its config, MCP connecting, the session resuming — so messages go down its stdin instead and that is paid once. Measured over four turns: 7.6s a turn throwing it away, 6.0s keeping it. Stopping a turn still kills it, which is what makes the stop button mean anything, and the next message starts a fresh one and resumes.

Every turn says where its time went — the total, how much was the model, and how much was startup on the turns that had any. A slow model and a slow tool look identical while you wait, and that is not a question anyone should have to answer by guessing.

Where the co-pilot lives

Two homes, one co-pilot. Click the Phantom toolbar icon and it moves into Chrome's side panel — browser furniture rather than page content. That is the difference between a panel that behaves and one that argues with every site it lands on:

	in the page	in the side panel
covers the page	not if it can help it — it sets a margin	never; the browser lays it out
resize	a drag handle we implement	the browser's own edge
hide and unhide	a button and a tab we implement	the toolbar icon
a site's fixed headers	moved out of the way, one by one	not our problem
following a link	rebuilt from scratch, transcript replayed	untouched
strict CSP, Trusted Types	worked around	irrelevant

The panel's document is deliberately empty: Phantom builds the same co-pilot into it over the same channel, so it is the same UI with the furniture around it removed. While the panel is open the bars in the pages stand down — hidden, and their margin handed back — and they return the moment it closes.

Opening it needs one click, and always will: a side panel opens only on a real user gesture, which is not something an extension can give itself or Phantom can send over CDP. Chrome remembers the choice per window afterwards.

It also makes phantom sync-agent unnecessary. That command exists because two Chromium instances cannot share one user-data-dir — so with a window open on main, the agent had to run on a copy of it, and the copy had to be kept in step with every new login. The co-pilot does not launch a second browser, so there is no copy and nothing to keep in step. sync-agent stays for the non-co-pilot path, which is still the default.

Identity (M4)

Phantom runs your credentials in place — it never extracts or decrypts a vault.

• Durable profiles. PHANTOM_PROFILE=<name> gives a persistent profile dir under %LOCALAPPDATA%\phantom\profiles\<name> that survives restarts, so an installed password-manager extension and its logins stick. Unset ⇒ an ephemeral throwaway.
• Password manager. phantom setup opens the durable profile headed so you install and sign into your manager once (PHANTOM_MANAGER, seeded with Proton Pass; Bitwarden / 1Password / Dashlane / Google also recognized). The autofill_login verb then drives the manager's own inline picker.
• Firefox cookie import. phantom import reads Firefox's plaintext cookies and injects them, so Phantom starts logged in wherever Firefox is. (Chrome/Edge cookies are v20 app-bound-encrypted and stay off.)

Stealth (M5)

The automation fingerprint is scrubbed, and human-like input engages automatically only where it matters:

• No navigator.webdriver — hidden by a script injected before every page's scripts (CDP addScriptToEvaluateOnNewDocument), not the launch flag --disable-blink-features=AutomationControlled (that flag trips Chrome's "unsupported command-line flag" warning bar). phantom open carries a debug port too — the bundled browser only honours --load-extension with one, and the co-pilot needs it to drive the same window — so the same injection covers it.
• Consistent identity headless — the HeadlessChrome UA token is rewritten to Chrome, with a matching Client-Hints userAgentMetadata so navigator.userAgentData stays populated and consistent (no empty-brands tell).
• The humanizer — Bézier mouse paths (ease-in-out, curvature, tremor, overshoot) and per-keystroke typing cadence, riding the same trusted CDP input the plain verbs use.
• The auto dial — PHANTOM_HUMANIZE = off | on | auto (default). In auto, the humanizer engages only on pages that appear to care, so ordinary pages stay fast.

The point is not to defeat anything. It is that an agent driving a browser should not be walled off from ordinary sites for looking like a robot, and that synthetic input arriving in zero milliseconds is the tell that does it.

Which tells were measured, which patches were deliberately not applied, and where the approach stops: docs/STEALTH.md.

Environment

var	effect
PHANTOM_BROWSER	path to a chrome.exe/msedge.exe; else auto-detected
PHANTOM_HEADLESS	1/true ⇒ --headless=new
PHANTOM_PROFILE	durable profile name; unset ⇒ ephemeral throwaway
PHANTOM_MANAGER	password manager for setup/autofill_login (default: Proton Pass)
PHANTOM_LOAD_EXTENSIONS	unpacked extension dirs (;/, separated) to load at launch
PHANTOM_HUMANIZE	off | on | auto (default auto)
PHANTOM_FIREFOX_PROFILE	override the Firefox profile dir for import
PHANTOM_TREE_BUDGET	bytes of tree an acting verb hands over unasked (default 16 KB shown / 6 KB acted)
PHANTOM_MAX_NODES	nodes kept in a full read before the tail becomes expandable stubs (default 1600)
PHANTOM_BRAIN	which brain the co-pilot bar talks to: cli (default) | off. api and openai-compat are recognised but not built — they say so and fall back to cli
PHANTOM_COPILOT	join the window already open on PHANTOM_PROFILE instead of launching one
PHANTOM_PERMISSIONS	open | normal (default) | strict — how hard consequential actions are held. No setting opens the money tier
PHANTOM_GRANT_FILE	where phantom authorize writes its single-use grant
PHANTOM_ENGINE_DIR	where the bundled browser lives, for update-engine — an install with its own layout, or a throwaway tree to test against
PHANTOM_CLAUDE_CONFIG	the config file connect writes, instead of the host's own
PHANTOM_LOG	tracing filter (logs go to stderr; stdout is the MCP protocol)

Testing

cargo test --workspace                       # hermetic unit tests (no browser)
cargo test -p phantom-ipc --test live -- --ignored   # live browser integration

Hermetic tests cover the SST compiler, verb plumbing, the humanizer (dial, keymap, RNG, detection helpers), profile paths, and cookie mapping. The #[ignore]d live tests drive a real headless Chromium; each takes an ephemeral profile of its own, so they neither collide nor need cleaning up between runs. The one check that needs a durable profile with a saved manager login says so and skips without it, rather than failing for want of its precondition.

Standalone harnesses live in each crate's examples/. The ones that earn their keep are the diagnostic pair — inputcheck (times each input event against a fresh browser: separates "the click missed" from "the browser never answered") and clickcheck / hoverdiff (reproduce a verb against a real page and check the DOM, since the tree is Phantom's opinion and sometimes the question is whether the opinion is right) — plus wire (--walk prices a real path, --cut reports what a node budget buys, --repeat prices repeated no-op actions), stealth_probe, humanize_check and hardening. manifest_capture reports the .m3u8/.mpd a page's player fetches together with the headers needed to fetch it from outside the browser, and replays the request to prove they work; it refuses to report an encrypted stream rather than hand back a url that fails later.

Roadmap

Done: M1 process/CDP spine · M2 SST compiler · M3 MCP verb server · M4 identity (profiles, in-place manager, Firefox import) · M5 stealth (flags, UA/Client-Hints, humanizer, auto bot-management detection) · M6 pointer & keyboard verbs (hover, scroll, press, select) and multi-tab (list_tabs, switch_tab, open_tab, close_tab, popup adoption).

M7 the co-pilot: the agent joins the window the human already has open (PHANTOM_COPILOT) instead of driving a private clone of their profile, and that window carries a bar you can talk to — a side panel injected over CDP into every tab, wired to a brain behind a swappable seam (crates/phantom-brain). phantom bar starts it; double-clicking phantom.exe does the same, since bare phantom decides what to be from whether its stdin is a console or a pipe.

The panel pushes the page aside rather than covering it, and nudges the position: fixed furniture that a margin cannot move — a site's sticky header would otherwise sit underneath it. That part is a mitigation, not a cure: it moves what overlaps (measured: three elements on GitHub, two on Wikipedia, none on Hacker News) and cannot be made general from inside the page, because the only CSS lever that repositions fixed descendants — a transform on <html> — also stops them being fixed, so a sticky header would scroll away. Panel-as-browser-furniture, via an extension side panel, is the real answer.

Token efficiency is its own line of work: the tree a page compiles to is shaped by what the model actually needs to act, and several plausible compression ideas were measured and rejected for costing more than they saved.

Permission tiers are in the surface: reads are free, ordinary interaction is free, an action whose target reads as submit / send / publish / delete is announced on the envelope, and one that reads as a payment is refused until a human runs phantom authorize — a single-use, minutes-long grant. The gate sits in PhantomCore, so every caller has it and not only the MCP host. It is a guardrail against the agent's own mistakes and against instructions injected through a page, and it says so at the point of use rather than pretending to be a security boundary: docs/PERMISSIONS.md.

Deliberately deferred:

• Payments rails — the lighter purchase tier below the transfer hard-gate.

Design notes

• Thin custom CDP client (untyped serde_json), not a generated protocol binding — Phantom touches a handful of CDP domains and owns its own schema.
• Job Object kill-on-close ties the browser's lifetime to ours; a crashed or closed agent never leaks a browser tree.
• Macro-ids map to Chrome backendNodeIds and re-resolve live quads on every action, so a click stays correct even if the page moved since the photo.