Skip to content

simulate: fail fast and clearly for Node agents that can't run simulations - #1048

Open
u9g wants to merge 3 commits into
mainfrom
jl/sim-node-agent-floor
Open

u9g wants to merge 3 commits into
mainfrom
jl/sim-node-agent-floor

Conversation

@u9g

@u9g u9g commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Two ways lk agent simulate dead-ends Node agent users, with no hint at the cause.

1. @livekit/agents 1.0.0–1.4.9 pass the version check, then die. The CLI's Node floor is 1.0.0. Simulate, though, starts the agent with --simulation and LIVEKIT_AGENT_NAME_OVERRIDE, and both first shipped in 1.4.11 (1.4.10 was never published). On 1.4.9 the user gets the agent exited before registering. Agent output: error: unknown option '--simulation'. Projects created from agent-starter-node before 2026-06-30 pin ^1.4.7, often held there by a lockfile. Simulate now checks for 1.4.11 and says @livekit/agents version 1.4.9 is too old, please upgrade to 1.4.11 or newer. The shared 1.0.0 floor for dev/start/console is unchanged, since those commands don't use either flag.

2. A Node agent that crashes in entry() shows only "No agent joined room". Broken-agent detection matched only Python log lines. A throwing Node entrypoint left every job to time out at 30s, and the real error stayed in the temp agent log. The 3-scenario starter can never reach the more-than-3 "not joined" threshold, so the run always played out in full. Adding agents-js's error in entry function (job_lifecycle.ts) to the markers lets the existing path cancel the run and print the error.

Repro (starter Node agent, CI=1 lk agent simulate text --scenarios scenarios.yaml --yes):

  • @livekit/agents 1.4.9: main prints unknown option '--simulation'. This branch prints the upgrade message.
  • throw new Error("boom: missing API key") at the top of entry(): main took 73–92s, with 3 of 3 jobs failing on No agent joined room. This branch cancelled within 9–18s and printed error in entry function: boom: missing API key.

The broken-agent path still exits 0 in CI on main. That's fixed separately in #1047.

Commits: a pure refactor that makes the version check take the floor as a parameter, then one commit per fix. go test ./cmd/lk/ passes.

u9g added 3 commits October 15, 2026 19:31
Earlier releases pass the 1.0.0 floor but reject --simulation, so the
agent exits before registering with "unknown option '--simulation'".
The fatal markers only matched Python log lines, so a Node agent whose
entrypoint threw surfaced only as "No agent joined room" timeouts.
@u9g

u9g commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Before/after on a Node agent. With @livekit/agents 1.4.9, main dies on unknown option '--simulation' and this branch asks the user to upgrade to 1.4.11. When entry() throws, main spends 73s on three No agent joined room timeouts; this branch cancels in 9s and shows the agent's real error.

node-sim.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant