Repository navigation
feat(episode): record episodes in the run index - #457
Open
polakamtejas wants to merge 3 commits into
Open
polakamtejas wants to merge 3 commits into
polakamtejas wants to merge 3 commits into
Conversation
An episode called Task.run directly, so the run index never saw it and the explorer's run lists, which read only the index, never showed it: training episodes were reachable only by instance id. An episode now records itself as a runner records the runs it accepts (runner `episode`): RUNNING before Task.run, then COMPLETED, FAILED with the error, or CANCELED with the abandon reason once its teardown is done. `EpisodeSpec.start(run_group_id=...)` files it in a run group, so a trainer can show a group of rollouts together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g it from the explorer, and keep a failed record write from leaking - An episode's run index status now follows `run_outcome`, the rule every runner uses: a step that fails without failing the task fails the run, rather than leaving it COMPLETED. Abandoned or cancelled stays CANCELED. - The explorer's cancel route refuses, with 409 and where to stop it, a run another runner records: the configured runner can't cancel an episode a trainer plays, so asking it silently did nothing. The UI shows the reason. - The run record is written inside the try whose finally takes the episode out of `_live`, so a write that fails is the run's error and leaves nothing behind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
polakamtejas
force-pushed
the
feat/episode-run-index
branch
from
November 6, 2026 16:32
172e5f1 to
728000f
Compare
The explorer reads a run's seed from `overrides.metadata.seed`, so an episode's run group listed `seed: null` for a seeded episode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An episode now records itself in the run index, as a runner records the runs it accepts:
Task.run: aRunRecordwith runnerepisode, statusRUNNING, the task, its version and the instance id. It's written before the first await, sostart()still returns with the run listed.mark_terminalwithCOMPLETED,FAILEDand the error, orCANCELEDand the abandon reason.EpisodeSpec.start(run_group_id=...): files the run in that run group (overrides.metadata.run_group_id), the key the explorer groups a task's runs by. Without it, each run is its own group.Why
The explorer's run lists read only the run index (
agent_env.runner.store). An episode calledTask.rundirectly, so its runs never appeared there: training episodes were reachable only by instance id. That holds for every trainer that plays through the Episode API, the OpenEnv-based helpers included. With a run group, a trainer can show a group of rollouts together, with their mean score.The terminal status is written after teardown, as the cancelled record already is. A scored episode still answers
result(teardown=False)at once (#393), and its run turnsCOMPLETEDonce its teardown is done.Known limit
An episode in a trainer process that dies stays
RUNNINGin the index.LocalRunnermarks its own interrupted runs failed on restart, and nothing does that for episodes yet.Tests
run_test.py::test_an_episode_is_in_the_run_index_running_then_ended_in_the_group_it_was_started_in. It checks running with its group, then completed, cancelled with the reason, and failed with the error.tst/unit/task,runner,explorer,openenvandtrainingpass, 532 of 532, on five runs. One run right afteruv synchad a single failure that didn't recur, likely a timing flake; the log didn't keep which test.run_group_id.🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; the missing-seed finding is fixed and no new actionable issues remain.Summary
Episodes now appear in the run index, with optional run groups and a final status written after teardown.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A["EpisodeSpec.start"] --> B["Save RUNNING record and metadata"] B --> C["Task.run"] C --> D["Release proxies and tear down"] D --> E["Save final run status"] B --> F["Explorer run groups"] E --> FReviews (3) · Last reviewed commit: "fix(episode): keep an episode's seed in ..." · Reviewed by Greptile