fix: prevent deadlock when tool call is cancelled during on_messages_stream - #8182
MOHAMMED WASIM KHAN (wasim-builds) wants to merge 4 commits into
Conversation
Previously, _apply_filter returned messages in per_source config order instead of chronological order, breaking the documented conversation timeline. Now messages are returned in their original order regardless of filter configuration. Fixes microsoft#7971
…stream When a CancellationToken is cancelled while a tool call is in flight, CancelledError propagates out of the workbench (not caught by except Exception), which causes asyncio.gather to raise before the end-of- stream sentinel is put in the queue. The consumer loop then blocks forever on stream.get(). Changed _execute_tool_calls to: - use return_exceptions=True in asyncio.gather - move stream_queue.put_nowait(None) to a finally block - convert any exception results to FunctionExecutionResult with is_error=True Fixes microsoft#7956
|
Ready for review when convenient. Happy to iterate on feedback. |
Ultronen (Ultronen)
left a comment
There was a problem hiding this comment.
I found one cancellation regression in the new parallel tool execution path.
| ) | ||
| for call in function_calls | ||
| ], | ||
| return_exceptions=True, |
There was a problem hiding this comment.
[P1] Preserve cancellation instead of turning it into a tool error
asyncio.CancelledError is a BaseException, and this is exactly what a linked tool future raises when cancellation_token.cancel() is called. With return_exceptions=True it lands in results, and the BaseException branch below converts it into a normal FunctionExecutionResult; on_messages/on_messages_stream then yields a tool error and completes normally, or can continue another model iteration, instead of honoring cancellation. This contradicts the documented contract that cancelling the token makes the on_messages await raise CancelledError. I reproduced this on 7491748 with a blocked FunctionTool: after token.cancel(), a regression test expecting CancelledError fails with DID NOT RAISE; re-raising CancelledError after gather while keeping the finally sentinel makes it pass. Please preserve cancellation propagation and only normalize ordinary tool failures.
There was a problem hiding this comment.
Thanks Ultronen (@Ultronen)! Updated to ensure cancellation is fully preserved: we check cancellation_token.is_cancelled() and re-raise asyncio.CancelledError out of _execute_tool_calls so that cancellation propagates immediately rather than being converted into a FunctionExecutionResult tool error. Ordinary tool failures continue to be caught and normalized.
|
Hi maintainers, thank you for the review feedback on cancellation handling! The fix ensures I've really enjoyed working on AutoGen's multi-agent runtime and streaming architecture, and I'd love to continue contributing to the framework. I'm also available for freelance/contract projects, full-time engineering roles, or joining the team/org as an active contributor. Feel free to view my profile and open-source contributions at https://github.com/wasim-builds. |
What happened
Cancelling a
CancellationTokenwhile a tool call is in flight permanently hangsAssistantAgent.on_messages_stream. The stream never ends, and any enclosing team'sstop_when_idle()also hangs.Root cause
StaticWorkbench.call_toolandStaticStreamWorkbench.call_tool_streamcatchException, butasyncio.CancelledErrorinherits fromBaseExceptionin Python 3.8+. When the token is cancelled,CancelledErrorpropagates out of the workbench, intoasyncio.gatherinside_execute_tool_calls, which raises before the end-of-stream sentinel (None) is put in the queue. The consumer loop has no other termination path, so it blocks onstream.get()forever.Fix
Changed
_execute_tool_callsto:return_exceptions=Trueinasyncio.gatherso one failing tool call doesn't abort the othersstream_queue.put_nowait(None)into afinallyblock so the sentinel is always sentFunctionExecutionResultwithis_error=Trueso downstream code sees a normal error result instead of a raw exceptionTesting
No new tests in this PR. I manually verified by running an
AssistantAgentwith a slow tool, cancelling mid-flight, and confirming the stream terminates cleanly instead of hanging.Fixes #7956