Repository navigation
fix(pydantic-ai): don't log deferred tool calls as tool span errors - #969
Closed
Ethan Leifer (ethanleifer) wants to merge 1 commit into
Closed
Ethan Leifer (ethanleifer) wants to merge 1 commit into
Ethan Leifer (ethanleifer) wants to merge 1 commit into
Conversation
pydantic_ai tools raise CallDeferred or ApprovalRequired to hand a call back to the caller for external execution or approval. The tool span recorded these exceptions as errors, so every deferred call looked like a failed tool in traces. Log these deferrals as the span output and re-raise them after the span closes. Other exceptions are still recorded as errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Member
|
opened #975 to supercede this PR, closing this PR out |
Abhijeet Prasad (AbhiPrasad)
added a commit
that referenced
this pull request
Oct 19, 2026
Fixes https://linear.app/braintrustdata/issue/SDK-581 Supersedes #969 (thanks to @ethanleifer for the PR) ## AI Summary Pydantic AI tools that raise `CallDeferred` or `ApprovalRequired` were logged as **failed** tool spans, even though these exceptions are Pydantic AI's normal way to defer a call or request human approval. Real failures show up as errors on the tool span, so deferred calls were tripping monitoring and alerting. ## How this differs from #969 | | #969 | This PR | | --- | --- | --- | | Where the deferral is recorded | `output={"deferred": True, "reason": "<ExceptionName>"}` | `metadata.deferred` (+ `deferred_metadata`) | | Fallback for Pydantic AI without these exceptions | `ImportError` fallback | not needed: the integration's `min_version` is `1.10.0`, which has both | | Reference to the re-raised exception | kept | cleared after re-raise, to avoid an exception → traceback → frame reference cycle | 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pydantic_ai tools can raise
CallDeferredorApprovalRequiredto hand a call back to the caller for external execution or human approval. These exceptions are control flow, and the agent run ends normally withDeferredToolRequests._trace_tool_executionopens the tool span as awithblock, so these exceptions leave the span and get recorded as its error. Every deferred or approval-gated tool call therefore shows up as a failed tool in traces. We hit this in production on a tool that defers to the UI for user confirmation.Fix
CallDeferredandApprovalRequiredand log{"deferred": true, "reason": "<exception name>"}as the span output, along with the usual timing metrics.ImportErrorguard, because older pydantic_ai versions (for example 0.1.9 in the wrap-openai matrix) don't define them.Testing
Added tests to
test_pydantic_ai_integration.py:test_deferred_tool_call_is_not_logged_as_error, parametrized overCallDeferredandApprovalRequired. It checks the run returnsDeferredToolRequestsand that the tool span has no error and a deferred output. Both cases fail onmainand pass with this change.test_tool_exception_is_logged_as_errorchecks that aRuntimeErrorfrom a tool is still recorded on the span.These tests drive the agent with pydantic_ai's
FunctionModelinstead of a VCR cassette. The behaviour under test comes from the local tool raising an exception, not from the shape of a provider response.Sessions run locally, all passing:
test_pydantic_ai_integration(latest)and(1.10.0)test_pydantic_ai_wrap_openai(latest),(1.0.1)and(0.1.9)test_pydantic_ai_logfire(latest)pylintand pre-commit on the changed files🤖 Generated with Claude Code