Skip to content

fix(agents): send Gemini thought signatures back through the chat-completions bridge - #409

Merged
cikowski merged 1 commit into
mainfrom
mateusz/gemini-thought-signature
Oct 30, 2026
Merged

cikowski merged 1 commit into
mainfrom
mateusz/gemini-thought-signature

Conversation

@cikowski

@cikowski cikowski commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

The Gemini agent's chat-completions bridge, used for provider-prefixed models (vertex_ai/...), reads each tool call's thought signature off the LiteLLM response and hands it to the CLI as thoughtSignature, but drops it when it turns the CLI's history back into OpenAI messages. Gemini 3 and newer require the signature on the next request, so Vertex answers Function call is missing a thought_signature in functionCall parts and the turn fails after its first tool call. LiteLLM hides this only for model names its _is_gemini_3_or_newer recognizes, where it fills in a placeholder signature; for any other Gemini 3+ name the call goes out unsigned and fails every time.

This change carries each part's thoughtSignature back as the tool call's provider_specific_fields.thought_signature, the field LiteLLM reads first on input. Because every signature now goes back to Vertex, the response side also stops copying a message-level signature onto every call of a parallel batch: Gemini signs only the first call, and LiteLLM's Vertex transformation notes that Gemini 3+ bills each replayed copy as the previous turn's reasoning tokens (litellm/llms/vertex_ai/gemini/transformation.py).

Rollout

  • Changes only the request the bridge sends upstream, and which parallel call carries a message-level signature in the response it hands the CLI. A history with no signatures produces the same request as before. The native /gemini passthrough (bare model names) is untouched.
  • Ships in the next agentenv-gemini image (src-<hex> changes with this folder). Anyone running a registered copy of the agent needs to register the new image to pick it up. Until then, a bare model name avoids the bridge.
  • Revert: revert this commit. No data or config migration.

Validation

Head c6bfb22 (same tree as the reviewed 5cb0252).

Done when Target Status Evidence Limits
Every tool call's signature reaches the next upstream request, and calls without one stay unsigned the /litellm-gemini proxy, two tool turns end to end MET test_each_tool_calls_thought_signature_survives_the_round_trip fails on main (no signatures sent), passes on head upstream LiteLLM is a mock transport
A message-level signature goes only to the first parallel call the proxy's response to the CLI MET test_a_message_level_signature_goes_only_to_the_first_parallel_call fails on main, passes on head mock transport
Vertex accepts the round-tripped signatures a Gemini 3+ model through a live LiteLLM proxy MET 6-turn tool loop through the bridge functions (1, 3 parallel, 1, 2 parallel, 1 calls, then a final answer): every request 200 on head that proxy fills placeholder signatures, so main passes there too: shows acceptance, not the fix
The 400 no longer happens on a model LiteLLM does not recognize live run UNPROVEN needs the released image
  • pytest tst/unit/agents/gemini: 146 passed, 6 skipped. Mutants that drop the signature, attach it unconditionally, or keep only the first turn's fail the suite.
  • make unit-test: 11896 passed, 66 skipped, 2 failed, neither in code this PR touches: tst/unit/agents/hermes/test_hermes_bridge.py::test_the_event_pipe_is_not_left_open_in_the_server fails the same way on a clean origin/main locally (macOS), and tst/unit/cli/submit_runs_test.py::test_a_first_ctrl_c_lets_the_runner_finish_stopping passes 3 of 3 alone and passed in an earlier full run (timing under the parallel run).
  • ruff check .: clean. Not run: integration tiers (nothing outside the Gemini agent folder changed).

Not in this PR

  • Thought signatures on text parts are still dropped in both directions. Gemini treats them as optional, so it costs quality rather than a 400.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

The Gemini bridge now sends tool-call thought signatures back to LiteLLM.

  • Tool-call signatures travel back to LiteLLM with their calls.
  • Parallel replies keep a message-level signature on the first call.

No actionable issues were found. Tests were inspected, not run during this review.

Acknowledged scope: cikowski states that signatures on text parts remain dropped because they are optional and outside this fix.

Diagram

sequenceDiagram
    participant CLI as Gemini CLI
    participant Bridge as Gemini bridge
    participant LiteLLM
    LiteLLM-->>Bridge: Tool calls with thought signatures
    Bridge-->>CLI: Function-call parts with thoughtSignature
    CLI->>Bridge: History containing those parts
    Bridge->>LiteLLM: Tool calls with provider_specific_fields.thought_signature
Loading

Reviews (1) · Last reviewed commit: "fix(agents): send Gemini thought signatu..." · Reviewed by Greptile

…pletions bridge

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cikowski
cikowski requested a review from a team as a code owner October 30, 2026 17:04
@cikowski
cikowski merged commit f47e70e into main Oct 30, 2026
41 of 44 checks passed
@cikowski
cikowski deleted the mateusz/gemini-thought-signature branch October 30, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants