Skip to content

feat(simulate): generate a scenario from a recorded session - #1049

Open
u9g wants to merge 2 commits into
mainfrom
jason/sessions-to-scenarios
Open

u9g wants to merge 2 commits into
mainfrom
jason/sessions-to-scenarios

Conversation

@u9g

@u9g u9g commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Adds lk agent simulate scenario generate --from-session SESSION_ID, which turns a recorded session into a scenarios.yaml document that lk agent simulate --scenarios can run.

lk agent simulate scenario generate --from-session RM_xxx > scenarios.yaml
lk agent simulate text --scenarios scenarios.yaml
  • The backend reads the session's chat_history recording, so the session must have been recorded (project recording setting on, or session.start(record=True)).
  • Works with API keys (AgentSimulation.CreateScenarioFromSession via server-sdk-go) and under --experimental-auth (Public API POST /v0/projects/{projectId}/scenarios:generateFromSession).
  • --json prints the scenario as the API returns it.
  • Separate commit: generated scenarios YAML no longer emits name: "", tags: {} and userdata: {} when they are empty. This also applies to the scenarios file saved after a generated-scenarios run.

Tested end to end against a production project: generated a scenario from a recorded session, then ran it with lk agent simulate text --scenarios (1 passed).

Known issue outside this PR: production public-api-server is still on v2026-10-13-1825 (595078c), where some pods serve simulations in the stubbed grpc mode, so every Public API simulation endpoint (including simulate list/get) currently returns 501 service operation not implemented yet under --experimental-auth. The API-key path is unaffected. Deploying public-api-server main (≥ e43438e) fixes it.

u9g added 2 commits October 15, 2026 22:22
Derives a scenario from a recorded session's chat history and prints it as a
scenarios.yaml document for `lk agent simulate --scenarios`. Works with API
keys (AgentSimulation.CreateScenarioFromSession) and under --experimental-auth
(Public API scenarios:generateFromSession).
@u9g

u9g commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Demo: lk agent simulate scenario generate --from-session turns a recorded production session into a scenarios.yaml, which then runs as-is with lk agent simulate text --scenarios (1 passed).

scenario-generate.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant