fix(sessions): declare the withheld interrupted write as a held pending Session write - #4828
fix(sessions): declare the withheld interrupted write as a held pending Session write#4828dixso wants to merge 29 commits into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6e52216fa5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3730311a98
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ba9fefa94
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d93e3bf76a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c6e6470a2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b09d8d14a2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
I don’t think (type, call_id) is collision-free across the Session. Custom model providers can reuse a call ID on a later turn; if an older matching call is still in this tail window, present suppresses the current deferred call and its output can again be persisted without its call. Could this key include a response/turn identity, or otherwise scope the match to the current response instead of treating call_id as globally unique?
|
@sylvesterkaczmarek Reproduced before answering, and it fails exactly as you describe. With
Scoping the match to the current response is the right direction, but I couldn't find a reliable way to do that from Session history alone. A Session persists a flat sequence of I also tested the narrower alternative of matching the entire converted prefix as an ordered block instead of matching individual items. That fixes the collision case, but breaks partial writes: if an earlier attempt persisted the calls but failed before persisting the output, the full prefix no longer matches and the calls are appended again. That seems to be the recurring signal from the edge cases on this PR: we're trying to answer "was this batch already written?" from Session history, but Session history doesn't contain enough provenance to answer that reliably. So I think the cleaner direction is to stop inferring it.
I prototyped changing the deferred park so that it records the withheld batch as the pending session write rather than dropping it and reconstructing it later. On resume, we then reconcile a declared batch instead of guessing from history. That removes the id-reuse, partial-write, detached-resume and cancellation cases I was able to construct, and actually deletes a fair amount of the reconciliation logic added by this PR. There are two semantics I don't want to choose on behalf of the maintainers, though:
Full write-up and reproducer are in #4827. I can push the prototype to this PR, open it separately, or hand the approach over if you'd rather own that shape. If you'd prefer to keep #4828 narrow and land the deeper persistence change separately, I can also just adjust the key here. |
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
I think the non-streaming loop still has the deferred-prefix loss case. On NextStepRunAgain, it clears deferred_session_prefix even when turn_session_items is empty and therefore nothing was persisted; the streamed loop now guards that case. Could the non-streaming path keep the deferred prefix until at least one resolved turn item is actually saved?
|
@sylvesterkaczmarek I reproduced this before answering, and the result is clearer than I expected. I built the empty-resolved-turn case end to end: park a gated call, approve it, then resolve into a turn whose session items are emptied by a handoff I then traced the non-streaming loop for that exact run. The I also couldn't construct a Finally, as a mutation test, I deleted the clear entirely and re-ran the eight reproducers from this PR plus the full suite. The outcomes were byte-identical. The variable only lives for a single resume pass: that branch is entered while So my earlier comment on that line ("later turns of this run must not re-send it") was incorrect. The loss your review points out is real, but it isn't reachable through that clear, and retaining the variable can't bridge it. Within the pass there is no later consumer; across runs, the only carrier is reconciliation against Session history, which your collision finding already showed can't be made reliable. The one place the batch survives all of this is the serialized checkpoint. That's the What I did push is a regression test ( I left the clear itself alone. Deleting it changes nothing measurable, and changing dead code as if it fixed this would be misleading. If a maintainer weighs in on #4827, I'll finish the declarative version, which makes this whole family of cases unreachable. |
seratch
left a comment
There was a problem hiding this comment.
The ordinary orphan-output case is real, but the current prefix-inference approach still has supported resume failures: the lookup happens after approved tool execution, the streamed lookup drops the context wrapper, legacy no-argument Sessions fail, and finalization/reconnect can duplicate or lose the batch. I recommend redesigning around the deferred response's durable ownership instead of adding another inference branch. Reusing pending-write recovery must also preserve the output-guardrail persistence gate; eagerly writing the held prefix before that gate is not a safe replacement.
…val resume continues the run With output guardrails and a non-default tool_use_behavior, _should_defer_interrupted_session_items defers the interrupted turn's session items at interruption time. When the approval resume resolves into next_step_run_again (or a handoff), the resume-side write only carried the resolved turn's new items - the tool output - and no later write recovered the deferred function_call. The Session ended up with a function_call_output whose call was never persisted, and the Responses API rejects every later run over that Session with 'No tool call found for function call output'. Persist the deferred prefix (the current response's session items, located via the resumed response boundary) ahead of the resolved turn's items once the resume commits to continuing the run, in both the streamed and non-streamed paths. The final-output path is untouched: its persistence already reconstructs the full current response. A resume that interrupts again keeps deferring.
Codex review: the selection lived twice, once per resume path, and AGENTS.md wants runtime logic under run_internal. It now lives next to the gate that governs it (_deferred_interrupted_session_prefix in blocked_output.py) and both paths call it.
Codex review: re-evaluating _should_defer_interrupted_session_items against the live configuration at resume time loses the deferred function_call again when the caller resumes with tool_use_behavior='run_llm_again' (reproduced before changing anything). A non-deferred interruption write bumps _current_turn_persisted_item_count, so persisted_count == 0 identifies the deferred park on its own — the helper now keys on checkpoint state only, which also keeps the prefix empty (no double write) when the interruption-time write actually ran. Two tests: the behavior-change resume, and the non-deferred park not being written twice (mutation-checked: dropping the persisted-count guard turns it red).
Codex review: a resume can interrupt again (partial approval of a multi-approval response). If the gate no longer defers, that re-interruption write is the deferred prefix's last chance — it bumps the persisted count, so writing only the approved tool's output there orphaned BOTH parked calls for every later resume (reproduced: 2 orphans before this change). The streamed re-interruption branch now prepends the prefix exactly as the non-streaming path already did; a gate that still defers keeps deferring, and the still-deferring variant recovers everything at final output (verified). Regression test proven red against the previous commit.
…every resume exit Three defects in the previous commits, each reproduced before changing anything: 1. A resume whose approved tool IS terminal ends in final output, and _final_turn_items_for_persistence only rebuilds the current response when the agent has output guardrails — a resume may legitimately run without the ones the park had, and the parked function_call was dropped again (both runners). The prefix now rides that exit too. 2. persisted_count can legitimately lie: the resumed-safety validator resets it to zero for a DETACHED resume, and that reset outlives the run, so a later resume reconnecting the original Session rewrote items it already held (duplicate function_calls, measured). The prefix is now CONFIRMED against the Session's own tail using the existing fingerprint helpers, so the write is idempotent by construction and a detached resume degrades to writing nothing. 3. An empty resolved turn (a handoff input_filter can drop every item) must not strand the prefix on its own: a call written without its output poisons the Session exactly as the orphaned output does. It keeps deferring instead, and both runners now agree on that. The helper moves to session_persistence, where the Session read and the fingerprint helpers already live, and becomes async. Five new tests, the three new ones proven red against the previous commit.
… identity Codex review: filtering the prefix against an unordered set of content fingerprints drops an item that merely LOOKS like one already there. An assistant preamble repeats verbatim across turns, so a tail holding an identical preamble from an EARLIER turn made the current one vanish while its calls were still appended — a legitimate occurrence lost from history (reproduced before changing anything). Matching the whole prefix as an ordered block was the obvious answer and is wrong too: a partially written response (calls persisted by an earlier attempt, output not yet) then matches nothing and duplicates the calls. Measured, both ways. So suppression is now keyed on identity that cannot collide — (type, call_id), unique per turn — and anything without one is kept unconditionally. A partially written response contributes exactly its missing half; nothing is ever dropped for looking familiar. Regression test proven red against the previous commit.
…efix Codex review: not every item family names its id 'call_id'. A hosted MCP approval request identifies itself with 'id' and its response points back with 'approval_request_id', so _identity_key returned None for both (measured) and a partially written response would append requests the Session already holds — duplicate request ids corrupt the history the next model call reads. The request identity is read through get_hosted_mcp_approval_request_identity, the repository's canonical helper, rather than a local rule. Request and response keep DISTINCT identities (same id, different type), so persisting one never suppresses the other. Items with no collision-free id still return None and are therefore never suppressed. Regression test proven red against the previous commit.
…nner Review follow-up. The empty-turn shape (a handoff input_filter drops every item of the resolved turn) must not write the deferred prefix on its own: a call with no output poisons the Session exactly as the orphaned output does. Both runners must also agree item for item, because a divergence here is how a dangling-call regression first shows up. Proven red against 9ba9fef, where the streamed path wrote both calls dangling (call_PARKED, call_HANDOFF) with no outputs. Also measured for the review discussion, not encoded in the test because they are properties of dead code: the RunAgain clear at run.py never executes in this shape (the emptied turn resolves into a handoff), RunAgain with empty session items is not constructible (approve and reject both yield an output item), and deleting the clear outright changes nothing across all reproducers and the full suite. The batch loss itself is real in BOTH runners and is the same root as the call-id collision finding: only the serialized checkpoint can carry it (openai#4827).
…ng Session write The output-guardrail persistence gate withholds the interrupted turn's session write at park time. The previous approach reconstructed that batch on resume by reconciling the checkpoint against the Session's history, which broke context wrapper propagation, legacy sessions, and detached reconnects, and could not prove what the park had withheld. The park now registers the withheld batch on the existing RunState._pending_session_write slot with a held marker. Registering is not writing: the Session is only touched at a gate-legal exit of a later resume, where the batch lands ahead of the resolved turn's items in one ordered append and inherits the digest-based crash recovery. A run-again checkpoint settles at entry, a detached exit folds the resolved items into the standing batch, an emptied resolved turn discards it, and the blocked-output redaction never sees it raw. The history-reconciliation machinery is deleted.
…ized contract The acceptance battery drives park, approve, reject, re-park, detached carry, tripwire, guardrail crash, emptied turn, legacy and context-aware sessions, and a failed settle recovered on the next resume, each through both runners and a serialized checkpoint. The resume-path suite pins the held marker's validation and that a checkpoint without the marker keeps its released eager-settle meaning.
…fer signature The non-streamed runner always builds a RunState for a fresh run, so the interruption result reads the held record from the state itself; the extra carrier parameter could never be exercised.
2312428 to
47bfebe
Compare
There was a problem hiding this comment.
💡 Codex Review
openai-agents-python/src/agents/run_internal/session_persistence.py
Lines 795 to 799 in fe5790e
When a streamed approval resume resolves directly to NextStepFinalOutput, _save_resumed_stream_items has already removed the held batch via take_held_session_write, but this condition excludes the final step from resumed_write_state. A Session append failure or lost acknowledgement therefore leaves no pending write in the checkpoint; retrying skips the completed tool while its call/output exchange remains absent from the Session. Route final settlement through the pending-write recovery path before clearing the held record.
AGENTS.md reference: AGENTS.md:L104-L104
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…pprovals, and completions Four holes the adversarial pass over the final diff surfaced, each reproduced in both runners before fixing. A fresh park during a detached resume now folds the new call into the standing held batch instead of losing it. The pairing guard exempts calls whose approvals are still open on the current step: their outputs are missing because they have not run yet, not because a filter removed them, so they settle like a non-deferred park writes a call before its output. The entry settle applies the same pairing contract against the batch alone. And a detached completion discards the batch at the fresh final exit too, so a completed run's checkpoint stays loadable and the runners agree. The streamed resume test double now forwards the settling batch.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f007612498
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…ettles, and restore Conversations sanitization at entry Three findings from the third automated review round. The sandbox runtime defers the pre-turn input save, so a deferred park used to fold the Session's only copy of the accepted input into the held batch, where a tripwire discard would take it along: the deferred arm now persists any unsaved input exactly as the non-deferred arm does, and the batch carries only the withheld response. The final-output settle now registers the claimed batch before appending, so a crash inside that append fails closed with the batch recorded instead of silently losing it. And the attached entry settle re-applies the Conversations-specific sanitization a detached extension could not, restoring the backend invariant before the direct append.
…d pair every approval family Two more findings from the same review round. With output guardrails the final sweep rebuilds the whole current response, held batch included, and the item deduplication cannot key the batch's unkeyed companions, so an assistant preamble landed twice: when the final items already carry every held request the batch is redundant and is dropped, in both runners. And the pairing guard now speaks every supported approval identity, hosted MCP requests and responses included, instead of recognizing only the function-call pair.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b470baec5c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| extend_held_session_write( | ||
| run_state, | ||
| run_items=turn_session_items, | ||
| reasoning_item_id_policy=( |
There was a problem hiding this comment.
Checkpoint held outputs at the commit boundary
When an approved tool finishes but a supported post-output callback such as custom_data_extractor or on_tool_end raises, tool_output_committer has already recorded the output and marked the invocation complete, but resolve_interrupted_turn() exits before this later held-batch extension runs. Retrying skips the completed invocation, treats its output as an existing pre-step item, and produces no new session items, so the empty-turn branch discards the still call-only held batch and the Session permanently misses both the call and output. Fresh evidence beyond the earlier pre-execution reconciliation thread is the failure window after the output commit but before this extension; fold committed outputs into the held record at the commit boundary.
AGENTS.md reference: AGENTS.md:L104-L104
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed reachable and left as a declared limitation, with the reasoning. I reproduced it: an on_tool_end callback that raises after the approved tool committed its output leaves the run at a point where a retry skips the completed invocation, the resolved turn is empty, and the call-only held batch is discarded, losing both call and output.
The correct fix is the one you name: fold the committed output into the held record at the commit boundary, inside tool_output_committer. That is outside this PR's surface (the committer and resolve_interrupted_turn are not touched here), and doing it safely means moving the batch extension from the turn exit into the commit path, which changes the commit ordering for every resumed tool, not just the deferred one. I would rather land that as its own change with its own tests than graft it onto this PR's settle logic, where I cannot prove it crash-safe against the other commit-boundary callers.
This is the same class as the final-settle crash window already documented in the description: a user callback failing inside the commit-to-extension gap. I have added it to the Known limitations so it is explicit rather than silent. Happy to open the follow-up if you agree with the placement.
There was a problem hiding this comment.
Fixed rather than left as the declared limitation. The resumed turn's output committer now folds the committed output into the held batch as it commits it, so a post-output callback that raises (custom_data_extractor, on_tool_end) cannot leave a retry that skips the completed invocation and drops the executed call and its result. Reproduced before the fix (the batch held the call alone and the pair vanished from history) and pinned afterwards by test_a_post_output_callback_failure_keeps_the_executed_output in both runners, mutation-proven red.
…counting rules Four more findings from the same review round. The held pairing guard now delegates to the canonical drop_orphan_function_calls, so every tool-call family in _TOOL_CALL_TO_OUTPUT_TYPE pairs (shell and apply-patch included) and a reasoning item riding before a dropped call is pruned with it, as the Responses API requires. A Conversations-backed registration forces the reasoning-id policy to None like the normal save, so a server-identified reasoning item stays persistable. And settled held items count toward the turn's persisted count, so a later gate-enabled resume fails fast on the persisted-items refusal instead of re-appending the stored calls.
There was a problem hiding this comment.
💡 Codex Review
openai-agents-python/src/agents/run_internal/session_persistence.py
Lines 703 to 710 in 90868a8
When a held multi-approval checkpoint is resumed with the persistence gate off but without a new approval decision, the approval placeholders trigger settlement but convert to zero new_items, while the held calls are passed through original_input; consequently saved_run_items_count is zero and this pending record also stores a zero persisted count. If the append fails or loses its acknowledgement, resume_pending_session_write() reconciles the held calls but restores that zero count, so a later gate-enabled resume can pass the output-guardrail safety check and append those calls again. Fresh evidence beyond the prior successful-settle counting thread is that save_resumed_turn_items() adds len(held_input) only after success, leaving this failure-recovery metadata uncorrected.
AGENTS.md reference: AGENTS.md:L104-L104
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… cover the held batch The guardrail rebuild deduplicates the held batch out of the append to avoid doubling its unkeyed companions, but the append still lands the approved call and output, so the recovery registration must stay armed. Arming now keys off whether a held batch was claimed at all, captured before the dedup empties the payload, in both the resumed-turn helper and the zero-count final save; a crash inside the append leaves the batch recorded to reconcile on retry instead of silently losing it.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 821afdc3f7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
I don't think the current head is ready to approve yet. There are still unresolved correctness/compatibility issues on the current diff: the new pending_session_write.held serialization needs schema-version treatment consistent with the released reader contract; held-write reattachment bypasses the Responses compaction bookkeeping used by canonical persistence; and the streaming max-turn-handler terminal path can retain a held write after reporting completion. The post-tool callback failure window documented in the open thread is also a real durability gap at the commit boundary. Please resolve the remaining current-head threads, or narrow/document the compatibility contract sufficiently, before re-requesting review.
…ttle it through the canonical path, and close the terminal and commit-boundary gaps Recovered work addressing the four blocking review points: - The held variant adds keys the released 1.17 reader rejects by exact key set, so it now has its own schema version. 1.17 keeps its four-key form and its original summary; held and response_id are gated to 1.18, with corpus fixtures, sources, README and the version-boundary test updated. - The entry settle no longer appends behind the canonical persistence path: it goes through save_result_to_session like every other settle, inheriting the Conversations sanitization, the ordered dedup, the pending-write registration and the compaction bookkeeping for the response the batch belongs to, which the park now records. - A max-turn handler ends the run, so both runners discard a held batch there. - The resumed turn's output committer folds a committed tool output into the held batch, so a post-output callback that raises cannot leave a retry that skips the completed invocation and drops the executed call and its result.
…ool output The consolidated settle hands the held batch to the canonical path through the original_input slot, but the deferral decision only inspected new_items, so a batch containing the approved tool's output reported no local tool output and compacted the very response whose output had just landed. The decision now asks whether the append persists a local tool output at all, whichever slot carried it, and the batch records the store setting of the turn it was withheld in so the deferral resolves the same compaction mode the ordinary path would.
…e held-only keys The corpus entries claimed a 1.18 writer for a commit that emits 1.17 and has no response_id, and the generator had no 1.18 scenario, so regenerating the corpus would have dropped them. Both fixtures are now what the recorded 1.17 writer emits with only the schema label changed, the generator carries the matching scenarios, and the README says the same thing. The reader also refuses response_id and store on an ordinary pending write, where they describe nothing, and the schema rationale no longer claims 1.17 shipped in a release: its readers are on main, which is reason enough not to rewrite what they already emit.
A handoff input_filter can drop every resolved item, but by then the approved tool has run and its output was folded into the held batch. Emptiness of the turn was the wrong predicate: pairing is. The executed call and output now settle through the canonical path and only the unpaired requests drop, so the Session keeps the only record that the tool ran and a later run does not re-issue its side effect. Three defects the same review surfaced go with it: - The compaction deferral read the whole ``original_input`` slot, which carries the caller's own input on an ordinary save. Only a settling batch reads it now. - The settled count added the batch's raw length, overcounting whatever the dedup dropped; the append reports what it actually wrote, and that count slices a later save of the same turn. - The max-turns discard ran before ``validate_handler_final_output``, so a wrongly typed handler output lost the batch that the streamed runner keeps. Each is pinned by a test proven red against the previous behaviour.
|
Thanks, that was a precise list. All four are addressed on the current head, each with a regression test proven red against the previous commit. Schema. You and Codex were right and my reasoning was wrong: I leaned on 1.17 being unreleased, but the 1.17 reader validates the pending write by exact key set, so a checkpoint written under that label with the extra keys is not loadable by a 1.17 reader, and my own regression called the four-key form released behaviour. Compaction bookkeeping. Fixed at the root rather than patched: the entry settle no longer appends behind the canonical path, it goes through Max-turn terminal path. Both runners now discard a still-standing held batch when a max-turn handler ends the run, so the finished run's checkpoint stays loadable and the streaming result no longer reports terminal output while carrying a resumable pending write the non-streaming result had already dropped. Post-tool callback window. Fixed rather than documented: the resumed turn's output committer folds the committed output into the held batch as it commits it, so a callback that raises afterwards cannot leave a retry that skips the completed invocation and drops the executed call and its result. One more round, self-inflicted. Reviewing my own diff afterwards turned up four defects it had introduced, all fixed on this head with a test proven red against the previous commit:
Verification on this head: full suite green (9471 passed) except one pre-existing sandbox failure that also fails here with this branch stashed, |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14c8315506
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| validated_output = validate_handler_final_output(agent, output) | ||
| # Only past the validation does the handler actually end the run; discarding above | ||
| # it would throw the batch away on a rejection the streamed runner survives. | ||
| take_held_session_write(run_state) |
There was a problem hiding this comment.
Preserve held writes until max-turn finalization succeeds
When a detached approval resume carries a held batch into a max-turn handler and a final-output hook or output guardrail raises, this consumes the declaration before either fallible stage runs. No terminal result is produced, but a retry or reattachment can no longer settle the already-executed tool's call/output into the original Session. Clear the held write only after successful finalization, while handling an intentional tripwire disposition separately.
AGENTS.md reference: AGENTS.md:L104-L105
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed, and the fix is the class rather than this site. Confirmed first: with a final-output hook that raises, the batch was gone and the run produced no terminal result, so a retry or reattachment could no longer settle the executed tool's call and output.
The rule now is that choosing a terminal step is not ending the run. finalize_max_turns_handler_output disposes of the batch only after the validation, the hooks and the guardrails have all completed, and an OutputGuardrailTripwireTriggered disposes of it separately as the decided blocked outcome it is, exactly as you suggested.
I inventoried the nine sites that consume the batch instead of fixing only the two you flagged. Four carried this ordering: this one, the detached final output in both runners, and the detached final output on the resumed streamed loop, which nothing had flagged and which is the very path this PR exists for. All four now dispose after finalization. The remaining five follow an outcome that is already decided (an input-guardrail trip, a blocked-output redaction) and are unchanged on purpose.
Pinned by test_a_failed_max_turns_finalization_keeps_the_held_record, mutation-proven red.
| # A detached final output has no Session to settle against | ||
| # and the run ends here, so the batch is discarded rather | ||
| # than left to invalidate the completed run's checkpoint. | ||
| take_held_session_write(run_state) |
There was a problem hiding this comment.
Delay detached write disposal until final output is finalized
When a detached resumed run reaches NextStepFinalOutput, this clears the held call/output before _finalize_streamed_final_output() runs its hooks and output guardrails. If either raises, stream_events() reports failure rather than completion, yet the live checkpoint has lost the only batch that a later attached retry could persist; the non-streaming detached-final branch has the same ordering. Dispose the batch only after successful completion or an explicit blocked-output outcome.
AGENTS.md reference: AGENTS.md:L104-L105
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in both runners, and reproduced before answering: with an output guardrail that raises, the batch was gone on the streamed detached completion and on the non-streaming one, exactly as you describe.
Both now dispose of the batch after _finalize_streamed_final_output (and after the guardrails and the final save on the non-streaming side) rather than before. A fourth site had the same ordering and nothing had flagged it, the detached final output on the resumed streamed loop, which is the path this PR exists for; it is fixed with the rest.
One honest note on evidence. Three of the four are pinned by tests proven red against the previous ordering. The fourth, the main streaming loop's detached completion, is aligned for consistency but I could not build a scenario that distinguishes it: measured, that site only executes on the success path, because a failing finalization leaves through the resumed loop's site instead. I am not claiming a test covers it.
Pinned by test_a_failed_detached_completion_keeps_the_held_record, parameterized over both runners and over a final output on the resumed turn and on a later turn, because the first shape alone left the non-streaming site unmutated.
| "store": (pending.get("store") if pending is not None else None) | ||
| if (pending is not None and pending.get("store") is not None) | ||
| else store, |
There was a problem hiding this comment.
Preserve a held null store setting across re-parks
When the original parked response used the ordinary store=None setting and a partial-approval resume re-interrupts under store=False or store=True, this truthiness-based selection overwrites the park-time value even though response_id still identifies the original response. A later OpenAIResponsesCompactionSession settle therefore resolves compaction mode using the resume's storage setting—for example forcing input-mode compaction for an originally default-stored response. Preserve the existing key even when its value is None; only use the new value when the pending record lacks the key.
AGENTS.md reference: AGENTS.md:L102-L102
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Correct, and this one is worth recording because I had ruled it out for the wrong reason. I checked this merge earlier by varying response_id only, found that the pending record was always absent at the re-park in the scenarios I ran, and concluded the branch was unreachable. Varying store is the case I did not run, and it reaches it exactly as you describe.
Measured on all four combinations: a park under store=None was overwritten by a re-interruption's False or True, while False and True parks were preserved. So the settle would resolve the original response's compaction mode from the resuming turn's storage setting.
Presence of the key decides now, not its truthiness, and response_id follows the same rule for the same reason: the batch belongs to the parked response, so both of that response's settle inputs come from the park. Pinned by test_a_re_park_keeps_the_storage_setting_the_response_was_produced_under across None, False and True, mutation-proven red on the None case, which is the only one the old code got wrong.
…its last step is chosen Choosing a terminal step is not the same as ending the run. Validation, the final-output hooks, the output guardrails and the final save all run after that choice, any of them can raise, and a run that raises may still be retried or reattached with the approved tool's call and output reachable only through the held batch. Consuming the batch at the choice threw it away on every one of those failures. Four sites carried that ordering, and they are the whole class: the max-turn handler finalization, the detached final output in both runners, and the detached final output on the resumed streamed loop. Each now disposes of the batch once finalization has completed, with a tripwire handled separately as the decided blocked outcome it is. The five remaining disposal sites are deliberate ones that follow an outcome already decided, and they are unchanged. Also: a re-interruption no longer overwrites the storage setting the parked response was produced under. Presence of the key decides, not its truthiness, so an ordinary ``store=None`` park keeps its own setting and the settle resolves that response's compaction mode from the right turn. ``response_id`` follows the same rule for the same reason. Tests pin the failure of each finalization stage in both runners, and the park storage settings across None, False and True.
There was a problem hiding this comment.
💡 Codex Review
When a partial-approval turn settles a held call/output batch into an OpenAIResponsesCompactionSession, this local-output branch returns only saved_run_items_count, omitting the settled_batch_items that were appended. The current persisted count can therefore remain zero; if the caller later re-enables the output-guardrail gate and approves the remaining call, the resumed-safety check permits the final sweep to append the already-stored calls again, corrupting Session history. Fresh evidence beyond the earlier count thread is that this compaction-only return bypasses the corrected common return on line 790. Return the combined count here as well.
AGENTS.md reference: AGENTS.md:L102-L102
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| # final save above can raise, and a run that raises may | ||
| # still be retried or reattached, with the executed | ||
| # tool's call and output reachable only through it. | ||
| take_held_session_write(run_state) |
There was a problem hiding this comment.
Preserve a resumable step when detached finalization fails
When a detached approval resume reaches NextStepFinalOutput and a final-output hook or output guardrail raises, delaying this disposal retains the held batch but leaves _current_step as NextStepFinalOutput. That step serializes as null, after which RunState.from_json() rejects the held pending write because it only accepts one alongside NextStepRunAgain or NextStepInterruption; even a live reattachment does not settle it because the held-entry path only settles NextStepRunAgain. Fresh evidence beyond the earlier disposal-order thread is that the retained state is therefore not actually retryable or reattachable. Restore a supported resumable step on failed finalization, or serialize and consume the terminal step explicitly.
AGENTS.md reference: AGENTS.md:L104-L104
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Thank you for pushing on the retry semantics here: it made me verify a property I had only reasoned about, which is worth it regardless of the outcome. I tried to reproduce this before changing anything, and I could not make the premise hold on this head. Measured in both runners, with both failure shapes (an output guardrail that raises, and an on_agent_end hook that raises) on a detached approval resume:
_current_stepafter the failure isNextStepRunAgain, notNextStepFinalOutput: the resumed loop resets the step to run-again when it re-enters the turn, and a failed finalization never reaches the_current_step = Noneassignment.- The serialized checkpoint carries
{"type": "next_step_run_again"}andRunState.from_json()loads it: the held validator accepts a run-again step. - The reattachment then settles the pair: reattaching the loaded state against the original Session wrote the parked
function_calland its output at entry, which is precisely the held-entry rule (a run-again checkpoint only exists once the parked response's outputs went back to the model, so the gate has expired by construction).
So the retained state is both loadable and settleable, in both runners. If there is a path that reaches this disposal with a step that serializes as null, I would genuinely like to see it and will fix it, but I could not construct one.
|
|
||
| # Serialized item types that represent a locally produced tool output, i.e. the output | ||
| # kinds of the canonical call-to-output map. | ||
| _LOCAL_TOOL_OUTPUT_TYPES = frozenset(_TOOL_CALL_TO_OUTPUT_TYPE.values()) |
There was a problem hiding this comment.
Defer compaction for held MCP approval responses
When a held hosted-MCP approval settles into an OpenAIResponsesCompactionSession, mcp_approval_response is absent from this set because it is not part of _TOOL_CALL_TO_OUTPUT_TYPE. The settle consequently runs compaction immediately instead of deferring it; in previous_response_id mode the compaction request uses only the parked response ID and then replaces the underlying Session, dropping the locally appended approval response before the next model request can consume it, so the approved hosted call may not proceed or may be requested again. Fresh evidence beyond the earlier held-compaction thread is that the new classification covers mapped tool outputs but omits the separately supported MCP approval pair. Include the MCP approval response in the local continuation classification.
AGENTS.md reference: AGENTS.md:L102-L102
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed by probe before answering: a held MCP approval pair settling into a compaction-aware session ran run_compaction immediately (deferred=[], compacted=[resp]), exactly as you describe.
Fixed in both carriers rather than only the settled dict, because the classification has two consumption paths and fixing one would leave them inconsistent: the constant is now _LOCAL_CONTINUATION_OUTPUT_TYPES (the canonical call-to-output kinds plus mcp_approval_response, with the comment stating why the approval response is the locally produced half of its pair), and the run-item check alongside it now recognizes MCPApprovalResponseItem, which the non-deferred resume commits through new_items with the same association requirement. Both pinned (test_a_held_mcp_approval_pair_defers_compaction_when_it_settles, test_an_ordinary_mcp_approval_response_defers_compaction_too), each mutation-proven red against its own carrier.
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
Thanks for requesting my review. I reviewed the current e4ce522 head against the held-write ownership, recovery, schema-version and compaction paths.
One blocker remains in save_result_to_session(): settled_batch_items is computed, but the has_local_tool_outputs compaction branch returns early with only saved_run_items_count. On a held partial-approval settle, the held call/output batch can therefore be appended successfully while the turn's persisted count remains zero. If the caller later re-enables the output-guardrail gate and approves the remaining call, the resumed-safety check can treat the turn as having persisted nothing and allow the already-stored held calls to be appended again.
Please return the combined settled count from that branch as well, and pin it with a compaction-session regression covering partial approval, held settlement, gate re-enable and the remaining approval. The other durability and schema issues I had raised on earlier revisions look addressed on this head.
I would not clear this head until that compaction-count path is fixed.
…n path Two defects in how a settling held batch meets a compaction-aware session: - The compaction-deferral branch returned the run-item count alone, and it is the branch every held settle with outputs takes on such a backend, so exactly the sessions that defer were the ones whose settled turns undercounted. The count gates the resumed-safety refusal and slices later saves of the same turn, so it must equal what the append wrote. The branch now returns the combined count. - The local-continuation classification knew the mapped tool outputs but not the hosted MCP approval response, which is the locally produced half of its approval pair and must stay associated with the response chain that carried the request. Compacting that response before the model consumes the approval drops it in previous_response_id mode. The constant is now _LOCAL_CONTINUATION_OUTPUT_TYPES and covers both carriers: the settled dict and the MCPApprovalResponseItem the non-deferred resume commits, because classifying one and not the other would defer or compact the same response depending on which path persisted it. The four-stage scenario behind the count (partial approval, held settlement with a lapsed gate, gate re-enable, remaining approval) is pinned end to end: it must end in the documented fail-fast refusal with nothing duplicated. Each fix is also pinned at the unit boundary and proven red by mutation.
|
Confirmed, and thank you: this was the sharpest catch of the round. The deferral branch is the one every held settle with outputs takes on a compaction-aware backend, and it returned On the four-stage regression you asked for: I built it (partial approval, held settlement with the gate lapsed on a compaction session, gate re-enable, remaining approval) and measured an honest wrinkle. In every reachable shape I tried, the undercounted value was 1 rather than 0, because the resolved turn always contributes at least the committed output as a surviving run item, so the resumed-safety check refused the gated resume on both the broken and the fixed head. Reaching the count==0 doom seems to require an emptied re-interruption settle, and approvals cannot empty a resolved turn today (only a handoff filter can, and a handoff leaves the interruption state). So the pinned end-to-end, |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b6a6636047
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| extend_held_session_write( | ||
| run_state, | ||
| run_items=[item], | ||
| reasoning_item_id_policy=run_state._reasoning_item_id_policy, | ||
| ) |
There was a problem hiding this comment.
Exclude checkpointed outputs removed by handoff filters
When an approved call is followed by a handoff whose input_filter removes that call/output, this commit-boundary extension adds the output to the held batch before the filter runs. _held_items_safe_to_settle() subsequently treats outputs already in held_items as pairing evidence, so it retains and persists both the call and output even though the filter removed them; later runs over the Session therefore see tool data that filters such as remove_all_tools deliberately excluded. Fresh evidence after the earlier filtered-handoff fix is that this pre-filter checkpoint now defeats the post-filter pairing check; committed outputs need recovery ownership without being treated as proof that they survived the handoff filter.
AGENTS.md reference: AGENTS.md:L102-L102
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I measured the baseline before deciding whether this is a defect, because the claim rests on the handoff filter owning what the Session keeps, and that contract has to come from the non-deferred path. It does not hold there. With the gate off (no output guardrails, classic eager persistence), both runners on this head persist the parked function_calls at park time, before any filter can run; when the input_filter then drops the resolved items, the Session is left with dangling calls and no outputs, in both the emptied-turn and the drop-one-output shapes. So the already supported path does not honor the filter's exclusion either: it keeps the calls and breaks the pairing, which is precisely the corruption class #4827 is about.
Given that baseline, the held path's choice is the only one of the three reachable states that leaves a valid Session: baseline persists the filtered calls without their outputs, discarding the batch loses the only record of a tool that already ran, and settling the executed pair keeps history consistent. The filter still fully controls what the target agent receives as input; what it never controlled upstream is Session persistence of the parked call.
Both shapes are pinned (test_an_emptied_resolved_turn_settles_the_paired_part_of_the_held_batch, test_a_filter_that_drops_one_output_takes_its_held_call_with_it), asserting pairing symmetry: nothing dangling, nothing orphaned, unpaired requests dropped. If there is an established caller-visible contract that a handoff filter excludes tool data from the Session, I could not find it in the code or the references, and the baseline actively contradicts it.
| run_state._pending_session_write = None | ||
| return list(pending["items"]) |
There was a problem hiding this comment.
Retain parked compaction metadata when claiming the batch
When a response parks under one store setting and the caller changes model settings before approving it, this claim clears the pending record and returns only its items, discarding the recorded response_id and store. Every exit-based settle then passes the resume's current store_setting, so an OpenAIResponsesCompactionSession can select the wrong compaction mode for the parked response. Fresh evidence after the earlier null-store re-park fix is that preserving the keys in the record is ineffective on these normal terminal, handoff, and re-interruption exits because take_held_session_write() never carries them to the save.
AGENTS.md reference: AGENTS.md:L102-L102
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I checked this against what the compaction session actually anchors on, and I believe the recorded keys are consulted at the only settle where they are the right anchor.
_defer_compaction(response_id, store) resolves the mode for the response it is handed, and run_compaction anchors on the session's latest tracked response. At every exit-based settle a newer model response exists by definition of the exit, its outputs land in the same append as the batch, and the deferral correctly names that response with the store it was actually produced under; the parked response is no longer the frontier there and is never independently compacted afterwards. The recorded response_id and store exist for the entry settle, where the parked response still is the frontier (a run-again checkpoint reattaching with nothing newer), and that settle reads both from the record.
So the record's keys are not dead on the exits; they are scoped to the settle whose anchor they describe. If there is a sequence where the parked response becomes the compaction anchor again after a newer response has landed, that would change my analysis and I would plumb the keys through take_held_session_write(), but I could not construct one against OpenAIResponsesCompactionSession.
| return | ||
| defer_interrupted_session_write( | ||
| run_state, | ||
| None, |
There was a problem hiding this comment.
Preserve reasoning IDs on detached Conversation re-parks
When a held write originated from an OpenAIConversationsSession, a detached resume can reach another model response containing an ID-only reasoning item and a new approval request. This extension calls defer_interrupted_session_write() with session=None, so reasoning_item_id_policy="omit" strips the server ID because the helper can no longer recognize the Conversation backend; reattachment then classifies that reasoning item as unpersistable and drops it, producing incomplete Conversation history. Fresh evidence beyond the earlier attached-registration fix is the supported detached fresh-park path, where the original backend identity is available only through the standing held record and is not consulted.
AGENTS.md reference: AGENTS.md:L102-L102
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed by probe before answering: a detached fold with the resuming run's omit policy stripped a server reasoning id from a Conversations-origin batch (rs_... became id-less), and the reattach would then drop the item as unpersistable, exactly as you describe.
Fixed with record ownership rather than backend sniffing: the park now records the conversion policy it actually used (None for a Conversations backend, the run's policy otherwise) on the held record, and a detached fold converts under the record's policy instead of the caller's, so an id that must survive is never stripped at a point where nothing can restore it. The key is gated and validated with the other held keys under 1.18, refused on ordinary pending writes and on unknown values, and an absent key falls back to the caller's policy.
Pinned by test_a_detached_re_park_folds_under_the_batch_registration_policy and test_the_park_records_the_conversion_policy_it_used, plus the policy-shape rejection case; each guard mutation-proven red.
There was a problem hiding this comment.
💡 Codex Security Review
Here are some automated security review suggestions for this pull request.
Reviewed commit: b6a6636047
ℹ️ About Codex security reviews in GitHub
This is an experimental Codex feature. Security reviews are triggered when:
- You comment "@codex security review"
- A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review
Once complete, Codex will leave suggestions, or a comment if no findings are found.
| non-deferred park persists a call before its output exists. | ||
| """ | ||
| pending_call_ids = pending_call_ids or set() | ||
| working: list[TResponseInputItem] = list(held_items) |
There was a problem hiding this comment.
Security: Honor handoff filters when settling held tool data
When an output-guarded, approval-required turn also hands off, and the filter removes a complete call/output pair by clearing new_items, this helper still seeds pairing from held_items; the commit-boundary hook has already added the tool output, so the pair survives and is written to Session. A later run reloads that Session and exposes tool data the filter withheld to its model. This matters when filters separate a secret-bearing tool from a lower-trust agent/provider; approval authorizes the tool action, not this disclosure. Fresh evidence beyond the earlier orphan-pair threads is the checked-in empty-filter test explicitly asserting that the removed pair is persisted. Derive settlement from filtered session_step_items, retaining held data only for retry recovery.
Dismiss this finding: Reply with @codex security dismiss <reason> [context]. Codex will resolve this conversation automatically; GitHub may require a page refresh to show the result.
Valid reasons: false-positive, duplicate, out-of-scope, compensating-control, risk-accepted, or other. Example: @codex security dismiss duplicate Already flagged by another review
What each reason means
false-positive— Not a vulnerabilityduplicate— Already tracked elsewhereout-of-scope— Outside this review's scopecompensating-control— Mitigated by another controlrisk-accepted— Risk intentionally acceptedother— Another reason; context required
Useful? React with 👍 / 👎.
A detached re-park cannot see the Session backend, so it folded new items under the resuming run's own reasoning-id policy. For a Conversations-origin batch that strips the server id at the one point where nothing can restore it, and the reattach then drops the reasoning item as unpersistable. The park now records the conversion policy it actually used (None for a Conversations backend, the run's policy otherwise) on the held record, and a fold converts under the record's policy instead of the caller's. The key is gated and validated with the other held keys under the unreleased 1.18 schema, refused on ordinary pending writes and on unknown values; an absent key falls back to the caller's policy. Pinned at the fold and at the park, plus the validator rejection, each guard proven red by mutation.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01f6e158e1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if run_state is None or run_state._pending_session_write is None: | ||
| return |
There was a problem hiding this comment.
Preserve ownership after a partial held settle
When a two-call deferred checkpoint is partially resumed against its Session with the persistence gate disabled, the settle writes both calls plus the first output and clears the held record while the second approval remains. If that remaining approval is then resumed without a Session and stopped with after_turn cancellation, this early return drops its newly committed output because _pending_session_write is now None; reattaching can continue, but the original Session permanently contains the executed second call without its output, so later runs reload incomplete history and may repeat the side effect. Fresh evidence beyond the prior detached-cancellation thread is the intermediate attached partial settle that consumes the record before the detached resume; retain the Session ownership record until every pending approval is resolved, or create a recoverable write for this detached output.
AGENTS.md reference: AGENTS.md:L104-L104
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I reproduced your chain end to end before answering, and it does reach the state you describe: after the attached partial settle consumes the record, a detached resume of the remaining approval with after_turn cancellation leaves the second call in the Session without its executed output.
Then I ran the decisive control: the same shape on this PR's base commit (f1ffb3db), with no gate, no deferral and no held machinery anywhere — an eager park, a detached approval resume, after_turn cancellation, reattach. The base drops the executed output identically (call_PARKED dangling, probe output calls=['call_LOOKUP','call_PARKED'] outs=['call_LOOKUP']). A detached resume's committed output with no standing record has nowhere durable to live upstream today; that is the gap, and it predates this PR. What this PR changes is that the record-standing detached carry now works (the fold plus the entry settle), which narrows the gap rather than widening it; the partial-settle route you found is another way into the preexisting no-record state, not a new loss path.
Closing it for real needs the checkpoint to carry the Session identity independently of the pending-write record, so a detached commit can register a recoverable write from scratch. That is new machinery with its own schema and trust-boundary questions, and this PR is deliberately scoped to the withheld-batch ownership the maintainers asked for, so I would rather file it as a follow-up issue with both reproductions attached than grow this change set further. Happy to do that if the maintainers agree.
This pull request fixes #4827 by giving the interruption park durable ownership of the
session batch it withholds, replacing the history-reconciliation approach that the
previous revision of this PR used and that review rejected.
The bug
_should_defer_interrupted_session_itemswithholds the interrupted turn's sessionwrite when the agent has output guardrails and a non-default
tool_use_behavior. Thepark leaves no record of what it withheld, so no later code can prove it happened.
When the approval resume resolves into
next_step_run_again, only the resolved turn'sitems are written: the Session ends up holding a
function_call_outputwhosefunction_callwas never persisted, and the Responses API rejects every later runover that Session with
No tool call found for function call output. Theconversation is permanently dead. This is reproducible with
ScriptedModelalone(see
tests/test_deferred_interrupted_session_write.py, red on main).The fix: registering is not writing
RunState._pending_session_writealready is the durable single-slot primitive for"one canonical resumed append awaiting acknowledgement", with digest-based crash
recovery. The park now registers the withheld batch on that slot with a
heldmarker:
persistence gate is preserved exactly: nothing reaches the Session until the gate
stops applying to the parked response.
turn's items in one ordered
save_result_to_sessionappend (call before output byconstruction) and re-registers as the one ordinary pending write, inheriting the
existing digest reconciliation for a crash mid-append.
after_turncancellation of a detached resumeleaves behind) settles at resume entry: the parked response's outputs already went
back to the model, which only happens after the gate stopped applying, and the
run-again turn's saves never arm
resumed_write_state, so entry is the only settlepoint that checkpoint will ever reach.
resume settles call and output together.
input_filtercan drop every item) settlesthe pairs the batch already holds and drops the rest: by then the approved tool has
run and its output is in the batch, so emptiness of the turn is the wrong predicate
and pairing is the right one, exactly as at every other settle.
the declaration once the blocked outcome is decided, and persists only the
sanitized rebuild.
The history-reconciliation machinery from the previous revision
(
deferred_interrupted_session_prefix,_identity_key,_PREFIX_MATCH_LOOKBACK) isdeleted.
How this addresses each review point
lookup anymore. The batch is on the checkpoint before the resume starts.
every settle flows through the existing save and settle machinery, whose call
sites carry
wrapper=context_wrapper. Pinned bytest_settle_reaches_a_context_aware_session_through_the_wrapper.limit=read is deleted withthe lookup. The settle inherits the pre-existing
limit=reads ofresume_pending_session_write; a companion change makes_session_get_itemsprobe the signature and fall back to a full read with the released latest-N
semantics applied locally. Pinned by
test_a_session_without_optional_kwargs_survives_a_deferred_resume.survives serialization and the step flip to run-again, and every exit either
settles, extends, or discards it deliberately.
to_state()on a completed runnever carries a stale record. Pinned by
test_after_turn_cancel_keeps_the_held_batch_for_the_next_attachand thestate._pending_session_write is Noneassertions.Serialized-state note (per
.agents/references/runstate-schema.md)The held variant adds keys the 1.17 reader rejects by exact key set, and 1.17 readers
are already on main, so writing it under that label would emit checkpoints they cannot
load. It therefore has its own schema version:
CURRENT_SCHEMA_VERSIONis now1.18, 1.17 keeps the four-key form and the original summary it shipped with, and the
held keys are gated to 1.18 while 1.17 payloads still restore and settle eagerly as
before. Corpus fixtures (
minimal/v1_18.json,features/v1_18_held_pending_session_write.json),sources.json, the corpus README and the supported-version boundary test are updated.Pinned by
test_pending_session_write_without_the_held_key_keeps_its_meaning(the1.17 form still loads and settles) and by the
held-under-1-17case oftest_pending_session_write_rejects_invalid_serialized_checkpoint(the held variantunder the older label is refused).
Contract-surface inventory for
heldpending_session_write.helddefer_interrupted_session_write)from_jsonvalidation, resume-entry settle rules, exit settles (take_held_session_write), detached folds (extend_held_session_write),to_jsonand both checkpoint copies (opaque deepcopy), the non-streamed result bridgefalsekeeps the released meaning: eager settle at resume entryheldwith recordedbeforedigests rejectedAwait-boundary inventory for the settle
from the slot (
take_held_session_writefrees the slot first). Suspension point:the composed
save_result_to_sessionappend. While suspended: the write isregistered as the one ordinary pending write with
beforedigests, so a crash orcancellation mid-append is recovered by the existing committed-versus-unchanged
reconciliation on the next resume (pinned by
test_a_failed_settle_of_the_held_batch_is_recovered_on_the_next_resume, 16interleavings across both runners and serialization).
itself; a second batch while one is unresolved fails fast with the existing
UserError.copies of the same checkpoint have no cross-object interlock
(
resume_pending_session_writedocuments that the application must serializeaccess), so the held variant inherits exactly the same contract.
Decisions taken with a stated default
1.17 in place, because the 1.17 reader validates the pending write by exact key set
and would reject a checkpoint written under its own label.
discarding it wholesale. Discarding was defensible while the batch could only hold
the parked call; once the resumed turn's committer folds the executed output in, a
wholesale discard loses the Session's only record that the tool ran and lets the
next run re-issue its side effect.
checkpoint shape only exists after the parked response's outputs went back to the
model and no later settle point exists on that path.
get_itemssignature probe is a companion fix in this PRbecause the settle machinery it protects predates this change; without it a
pre-
limitstructural Session hard-fails any pending-write recovery.existing same-session error, before the approved tool can execute; a detached
resume still rides.
input_filterthat drops one resolved output takes exactly that call with it,while calls whose approvals are still open settle unpaired on purpose: their
outputs have not run yet, matching how a non-deferred park persists a call before
its output. The entry settle applies the same contract against the batch alone.
standing held batch (the declaration carries the session identity), and a detached
completion discards the batch at the final exit so a completed run's checkpoint
stays loadable in both runners.
input. The gate withholds model output, not the user's accepted input, so a
deferred park persists any still-unsaved input (the sandbox runtime defers the
pre-turn save) exactly as the non-deferred arm does, and a tripwire discard can
never take the Session's only copy of the input with it.
its append, regardless of the current step, so a crash inside the settle fails
closed with the batch recorded. The resulting mid-settle checkpoint is rejected
on load on purpose: the run ended mid-settle, and failing closed beats replaying
an approved side effect as if nothing happened.
sanitization before the direct append, restoring the invariant for items a
detached extension added without a Session in hand.
is dropped as redundant at the shared settle choke point, because the item
deduplication cannot key its unkeyed companions (an assistant preamble, an id-less
reasoning item) and feeding it again would duplicate them.
compaction. The batch reaches the canonical path through the
original_inputslot,so a decision that only inspected
new_itemswould compact the very response whoseoutput had just landed; the record also carries the store setting of the turn it was
withheld in, so the deferral resolves the mode the ordinary path would have.
response_idandstoreare refused on an ordinary pending write, wherethey describe nothing, and the 1.18 corpus entries are what the recorded 1.17 writer
emits with only the schema label changed, with matching generator scenarios so the
corpus stays reproducible.
drop_orphan_function_calls,so every tool-call family in
_TOOL_CALL_TO_OUTPUT_TYPEpairs (shell and apply-patchincluded) and a reasoning item riding before a dropped call is pruned with it; hosted
MCP approvals pair alongside through the canonical request identity.
Nonelike the normal save, and settled held items count toward the turn's persistedcount so a later gate-enabled resume fails fast instead of re-appending.
output-commit boundary, so a post-output callback that raises cannot leave a retry
that skips the completed invocation and drops the executed call and its result.
discarded in both runners; the finished run's checkpoint stays loadable and the two
runners report the same terminal state. The discard sits below
validate_handler_final_output, because only past that validation does the handleractually end the run.
original_inputslot only on thecalls that settle a batch through it. That slot carries the caller's own input on
an ordinary save, and an earlier tool output sitting there says nothing about
whether this response produced one.
the batch's raw length. The resolved turn re-delivers the outputs the batch already
folded in, they dedup away, and the count slices a later save of the same turn. The
compaction-deferral branch returns the same combined count: it is the branch every
held settle with outputs takes on a compaction-aware backend, so returning the
run-item count alone there undercounted exactly the sessions that defer.
response in both of its carriers (the settled dict and the run item the
non-deferred resume commits). The approval response is the locally produced half
of its pair and must stay associated with the response chain that carried the
request; classifying one carrier and not the other would defer or compact the same
response depending on which path persisted it.
(
reasoning_item_id_policy, gated and validated with the other held keys). Adetached fold cannot see the Session backend, so converting under the resuming
run's own policy could strip a Conversations server reasoning id at the one point
where nothing can restore it; the record carries the policy the park actually
used, and folds convert under it.
response, not the parked one. At every exit a newer response is the session's
compaction frontier and the batch lands inside its append; the recorded
response_idandstoreare for the entry settle, the one place where theparked response still is the frontier.
Test plan
tests/test_deferred_interrupted_session_write.py: park, approve, reject,re-park, detached carry with
after_turncancellation, tripwire (with a preamblepinning that the redaction's drops are not resurrected), guardrail crash, emptied
turn, context-aware and legacy sessions, single ordered write, and failed-settle
recovery; each scenario runs both runners and a serialized checkpoint round trip.
tests/test_run_impl_resume_paths.py:heldvalidation (non-bool, held withbefore) and the no-marker compatibility pin.extend, each discard, the pairing guard and its pending-approval exemption, the
session-identity check, the bridge, the slot release, the signature probe, the
emptied-turn settle, the compaction predicate, the settled count and the max-turn
discard ordering, every one proven to turn at least one test red.
make format,make lint,make typecheck,make testsvia.agents/skills/code-change-verification/scripts/run.sh.