Skip to content

feat(core): send mid-conversation tool changes in place - #64381

Open
rekram1-node wants to merge 5 commits into
devfrom
core-tool-changes
Open

rekram1-node wants to merge 5 commits into
devfrom
core-tool-changes

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

None.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

When the tools offered to the model change mid-session (an MCP server connects or drops, a plugin tool appears, an agent loses a permission), core now sends the change in place on models that support it, instead of rebuilding the top-level tool list. The top of the prompt stays the same, so the prompt cache still hits. It builds on Message.tools({ add, remove }) from #64166.

How it records and replays changes. Before each step, the runner compares the tools it is about to offer (after context hooks and freeform filtering, the same definitions core already sends) with the tools the model has been told about. Tools are compared by name and canonical JSON, so reordering alone is not a change. A difference is recorded once, as one durable fact, before the request goes out:

session.tools.updated { add: [full definitions], remove: [names], model? }
  → projected as a hidden system message: { type: "system", text: "", tools: { add, remove, model? } }

With model set, the message is a baseline: add is the whole set. A change has no model. Requests are built from the history:

history:  user · [baseline A,B] · assistant · user · [+C −B] · assistant · user
request:  tools: [A, B]              ← latest baseline for this model, never changes
          messages: user, assistant, user, Message.tools({ add:[C], remove:[B] }), assistant, user

The step that records a change sends the marker at the same place, after the history and before any assistant prefill. The next step replays it from the stored message, so both requests carry the same bytes. The planner in @opencode/ai then sends it in place (Anthropic tool_addition/tool_removal, OpenAI Responses additional_tools) or folds it.

I kept this in the message history instead of adding a state table. History windowing already handles every case where the baseline has to change:

  • Session start: no baseline yet, so the first step records one.
  • Compaction: the window starts at the checkpoint and leaves the old baseline behind, so the next step declares the current tools again. Provider checkpoints don't keep tools added later, so this is required.
  • Model or provider change: the latest baseline names another model, so a new one is recorded. Variants share the baseline.
  • Revert or fork: only changes before the cut survive. The surviving prefix is what the provider already saw, so the next step just records the remaining difference. Reverting past the baseline starts over.

The current tools still execute as before: a tool added mid-conversation runs, and a removed one is refused, because executeTool uses the tools offered now, not the request's top-level list.

Capability pass-through. models.dev publishes api.<protocol>.capabilities.tools.updates per API (anomalyco/models.dev#13856, live). I decode it for the API the model resolves to, drop unknown values, and treat the old boolean form in the bundled snapshot as none. It lands on compatibility.supportsToolUpdates, which the model resolver maps to the AI model's capabilities.tools.updates, the same path as supportsEffortUpdates. Config overrides it the same way (compatibility: { supportsToolUpdates: [] } turns it off).

No change for models without tools.updates. Core records only when the model's tools.updates overlaps what its route can encode. Other models record nothing, get no markers, and send the current tool list in today's order. A session that switches to such a model sends its plain current tools.

Public surface (additive, optional):

  • Model.Compatibility.supportsToolUpdates?: ("add" | "remove" | "replace")[] (model-level only, not on provider compatibility)
  • Session.Message.System.tools?: { add: Tool.Definition[], remove: string[], model?: Model.Ref }, plus the new Tool.Definition, Tool.Changes and Model.ToolUpdate schemas
  • session.tools.updated is an internal durable event, like session.usage.recorded: it is not in the public event manifest. It shows up only in the experimental session log type.

Client types are regenerated. The solid client store skips system messages with tools, so TUI, app and desktop transcripts never show them. Export and import keep them, and sanitized export redacts their descriptions and schemas.

Trade-off: each baseline stores the full tool definitions once (about 14 to 15 tools here, more with MCP servers), at session start and after each compaction or model switch. Clients receive these rows in message pages and drop them.

Ships with: https://github.com/anomalyco/opencode-console/pull/7776 makes the Zen gateway's free-tier and agent-tool checks see tools declared in additional_tools. It will ship well before a release with this change, so nothing here waits for it.

How did you verify your code works?

Live, through a real serve process driven by the client (scripts in my notes). The run: two warm-up prompts, add an MCP server mid-session (experimental.mcp.add), ask for its tool, compact, ask again, remove the server, ask again. Values are cache read tokens for the first request after each event:

B (control) after add after compaction after remove
claude-sonnet-5-5 11410 11425, calls the tool 0 (new prompt), calls the tool 12007, says no tool
claude-opus-5-5 11398 11413, calls 0, calls 11940, no tool
gpt-6-astra 7271 7288, calls 0, calls 7551, no tool
claude-sonnet-5 (no updates) 11470 0, calls 11482, calls 11390, no tool
deepseek-flash (Chat, no updates) 7680 6912, calls 7680, calls calls the removed tool, refused, then no tool

On capable models the request after a change reads the whole previous prompt from cache. claude-sonnet-5 shows today's behavior (cache miss on the add), and nothing is recorded for the two models without the capability.

Tests:

  • packages/core/test/session-tool-changes.test.ts: diffing (add, remove, redefine, re-add after remove, reorder only), epochs and replay.
  • packages/core/test/session-runner.test.ts (tool changes): unchanged requests without the capability; the in-place marker and identical replay, also compiled to the Anthropic wire; running an added tool and refusing a removed one; baseline after compaction, model switch (not variant), revert; switching to a model without the capability and back; fork.
  • packages/core/test/models.test.ts: models.dev decoding, unknown values, old booleans, config override.
  • packages/client/test/solid-data.test.ts: the store leaves tool changes out of transcripts.
  • The existing recorded and wire tests pass unchanged.
  • bun typecheck in core, schema and client, and bun run check at the root. Core, schema, client, server and sdk suites pass, except tests that also fail on dev locally: core shell tool and parity tests, and server experimental artifacts, automations and media_blob tests.

Visual

The Anthropic request after an MCP server swaps lookup for fresh (from the runner test, compiled to the wire):

 tools: [defect, echo, lookup, storefail]      ← dev sends [defect, echo, fresh, storefail] here, a cache miss
 messages:
   user  "First"
   user  "Second"
+  system [tool_addition fresh, tool_removal lookup]

Screenshots / recordings

No UI change. Recorded tool changes are hidden from transcripts.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Added by the opencode bot (AI-generated).

Public API: adds optional Model.Compatibility.supportsToolUpdates, Session.Message.System.tools, Tool.Definition/Tool.Changes schemas and the durable session.tools.updated event (backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work as pushed. The head commit has only packages/core/src/session/tool-changes.ts and its test. The rest of the work the description covers isn't in the diff: the Tool.Definition/Tool.Changes schemas, SessionMessage.System.tools, the session.tools.updated event and projection, the runner wiring, supportsToolUpdates decoding, the client store filter and the regenerated types, as well as the runner, models and solid-data tests. Maybe those commits weren't pushed, or the branch was cut from a partial state.

What I ran in packages/core:

  • bun run typecheck:tsc fails in both new files: Tool.Definition and Tool.Changes aren't exported from @opencode/schema/tool, and tools doesn't exist on SessionMessage.System.
  • bun test test/session-tool-changes.test.ts: 5 of 8 tests fail. SessionMessage.System.make drops the tools field, so epoch() never finds a baseline (start is undefined instead of 3).

Nothing imports SessionToolChanges yet, so as pushed the module is unused code that breaks the type check.

Public API: none as pushed (one new internal core module). The additions in the description (Model.Compatibility.supportsToolUpdates, Session.Message.System.tools, Tool.Definition/Tool.Changes) are all optional fields and would be backwards compatible, but none of them are in this diff.

Could you push the rest of the branch? I'll review the full change once it's there.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier point is addressed: the branch now has the full change, and it type-checks and passes the tests. I found nothing that should block a merge.

What I ran:

  • tsc -b in packages/core passes.
  • Tests pass: the core session-tool-changes, session-runner, models and model-resolver suites; client solid-data; and the schema suite.

I couldn't check the cache gain live, because there are no provider credentials here and the simulated model has no tools.updates. The runner tests, which compile the request to the Anthropic wire format, cover the in-place marker, identical replay, executing an added tool, refusing a removed one, and resetting the baseline after compaction, a model switch or a revert. From reading the code: retries reload the history, so a change isn't recorded twice, and every client transcript (TUI, app, context tab) reads through the solid store, which filters these messages.

Public API: additions only, all backwards compatible:

  • Model.Compatibility.supportsToolUpdates?: ("add" | "remove" | "replace")[] (model level only, not on Provider.Compatibility)
  • Session.Message.System.tools?: { add, remove, model? }
  • new Tool.Definition, Tool.Changes and Model.ToolUpdate schemas
  • a durable session.tools.updated event, which appears only in the experimental session log output type

Existing configs, clients and plugins are unaffected.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only new commit regenerates the OpenAPI document. bun run check:generated passes, so it matches the generator, and its additions match the schema changes already reviewed. No new concerns.

Public API: the OpenAPI spec now includes the additive, optional pieces: Model.Compatibility.supportsToolUpdates, Model.ToolUpdate, Session.Message.System.tools, Tool.Definition and Tool.Freeform. The session.tools.updated event appears only under Session.Event.Durable, next to session.usage.recorded. All of this is backwards compatible.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only new commit (after the rebase) looks sound, and there are no new concerns. Hiding tool-change rows now happens in SessionStore.messageRows, so every message listing (messages, messagesWithRefs, including cursor paging) leaves them out. Only export asks for them, through toolChanges: true. The runner still sees the tool baselines because history.ts uses its own query. The new store test covers the listing, the cursor and the export option.

Checked with tsc -b in packages/core and the session-store, session-tool-changes, session-runner, models and model-resolver tests (all pass). I still haven't checked the cache gain live; there are no provider credentials here.

Public API: unchanged from my last review (all additions are optional and backwards compatible). One behaviour change in this commit: system messages that hold tool changes no longer show up in the server's message listings, so SDK and plugin clients never receive them. toolChanges is an internal store option and isn't exposed over HTTP.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two new commits look sound, and there are no new concerns.

  • Output schemas: changing a tool's output schema no longer counts as a tool change. Nothing in packages/ai's protocols, providers or routes sends outputSchema to the provider, so this doesn't change what the model sees.
  • Crash recovery: the max-steps prefill is now a separate prefill input, and the tool change goes before it only. A step cut short by a crash stays history, so the change sits after it, where it replays on the next step. The new runner test covers this case.

Checked with tsc -b in packages/core, the session-tool-changes, session-runner, session-store and session-generate tests (all pass), and check:generated in packages/protocol. I still haven't checked the cache gain live; there are no provider credentials here.

Public API: same as my last review, all additions optional and backwards compatible. Tool.Definition (new in this PR) no longer has outputSchema, and the OpenAPI spec and client types are regenerated to match.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant