Skip to content

fix(harbor): pi rollouts that hit a step or context limit fail reconciliation - #1561

Open
agokrani wants to merge 1 commit into
openenv:mainfrom
agokrani:fix/pi-budget-stop-reconcile
Open

agokrani wants to merge 1 commit into
openenv:mainfrom
agokrani:fix/pi-budget-stop-reconcile

Conversation

@agokrani

@agokrani agokrani commented Nov 5, 2026 •

Copy link
Copy Markdown

Summary

_pi_session_as_atif keeps only token counts, but reconcile recognises the proxy's budget
stop by its text. So every pi rollout that hits agent_step_limit or the context limit fails
reconciliation with turn_mismatch and cannot be trained on. This keeps each reply's text and
tool calls in the pi trace, so the existing stop rule applies.

Type of Change

  • Bug fix

Details

When a rollout hits its limit, the capture proxy answers the next request itself with
"Step budget exhausted; stopping." pi logs that as an ordinary assistant record (zero tokens,
responseId: "budget-…"). reconcile is meant to discount it (proxy_budget_stops), but it
matches on step["message"], which the pi reader never set. The pi trace then has one more
reply than the capture, and the rollout fails.

Alignment Checklist

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run lint and tests and addressed all issues

RFC Status

  • Not required (bug fix)

Test Plan

Two tests in tests/envs/test_harbor_reconcile.py write a pi session log and run load_trace

  • reconcile on it:
  • test_pi_rollout_ended_by_the_proxy_budget_stop_reconciles: fails on main, passes here.
  • test_pi_zero_token_reply_that_is_not_a_budget_stop_still_fails: an unexplained zero-token
    reply is still rejected.

PYTHONPATH=src:envs uv run pytest tests/envs/test_harbor*.py: 562 passed, 27 skipped, 1 failed.
The failure, test_start_refuses_a_port_held_by_a_non_capture_listener, also fails on an
unmodified main on macOS. ruff format, ruff check and usort pass on the changed files.


Note

Low Risk
Localized adapter fix for pi trace loading with regression tests; no change to core reconcile rules beyond supplying fields pi traces lacked.

Overview
pi session → ATIF fallback used to emit only metrics.completion_tokens, so reconcile could not recognize the proxy’s zero-token budget-stop reply (it matches on step["message"]). Limit-hit pi rollouts then looked like an extra assistant turn and failed with turn_mismatch, blocking training.

_pi_session_as_atif now maps assistant text and tool calls from pi’s message.content (including string content), alongside token counts. The existing proxy_budget_stops path applies unchanged.

Tests build synthetic pi *.jsonl logs via load_trace and assert budget-stop rollouts reconcile (with proxy_budget_stops) while other zero-token replies still fail.

Reviewed by Cursor Bugbot for commit 8fd35e3. Bugbot is set up for automated code reviews on this repo. Configure here.

…concile

When a rollout hits its step or context limit, the capture proxy answers
pi with "Step budget exhausted; stopping." instead of calling the model.
pi logs that reply like any other. reconcile() already discounts it, but
recognises it by its text, and _pi_session_as_atif kept only token
counts. Every pi rollout that hit a limit therefore had one more reply
than the proxy captured, failed with turn_mismatch, and could not be
trained on.

Keep each reply's text and tool calls in the ATIF steps the pi reader
returns, so the existing stop rule applies. The rule itself is
unchanged: it still requires the proxy's own stop counter, zero tokens
and no tool calls.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant