Skip to content

feat(plugin): add experimental model HTTP hooks - #62820

Open
rekram1-node wants to merge 9 commits into
devfrom
model-http-hooks
Open

rekram1-node wants to merge 9 commits into
devfrom
model-http-hooks

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Plugin HTTP hooks only run for Session requests. generate text, image, decision and video calls and voice session setup send provider requests that no hook can see or change (#62096). Anything that has to touch every model request, like the provider client certificates in #61832, has no single place to do it.

Change

Adds ctx.model.hook("experimental.http.request" | "experimental.http.response", ...). It runs for every HTTP request to a model provider, whichever Session or API makes it.

who calls             kind                                     session
Session step       →  primary / compaction / title / generate  { id, agent }
generate.text      →  text                                     undefined   NEW
generate.image     →  image                                    undefined   NEW
generate.decision  →  decision                                 undefined   NEW
generate.video     →  video                                    undefined   NEW
voice session      →  voice                                    undefined   NEW
type ModelRequestScope = { model } & (
  | { kind: "primary" | "compaction" | "title" | "generate"; session: { id, agent } }
  | { kind: "text" | "image" | "decision" | "video" | "voice"; session?: undefined }
)
// experimental.http.request:  ModelRequestScope & { request }
// experimental.http.response: ModelRequestScope & { request, response }

session is set for a Session's own model requests, so checking it narrows kind. A tool that calls Generate from inside a Session (like the built-in image tool) makes a Generate call with no session.

The session http.request / http.response hooks are unchanged and still only see Session requests. The docs recommend the model hooks. For a Session request, both run in one middleware, with the model hooks outside:

model request hook → session request hook → send → session response hook → model response hook

How it's wired:

  • core/src/model-http.ts builds the middleware. It's moved out of session/model-request.ts, which now calls it and passes it per call as before.
  • Generate (text, image, decision, video start/status/result) and Voice.session wrap the call in ModelHttp.scoped, which uses the AI package's RequestExecutor.withMiddleware from feat(ai): scope HTTP middleware to any call, for every modality #64152. That covers every request of the call, media polls included. A result downloaded from the provider's own host keeps the hook; one from a CDN doesn't.
  • Fix: the AI SDK route sends through its own fetch, so it ignored scoped middleware. It now applies it, outside the per-call one.
  • PluginHooks only gains the model domain.

The hooks run on each request sent (retries too), after the provider's auth headers are set, and a hook that fails fails the request. The docs say so.

generate calls through the HTTP API run in the global config location, so only globally configured plugins see them. That's noted in the docs.

Visual

<svg xmlns="http://www.w3.org/2000/svg" width="240" height="72" viewBox="0 0 240 72" fill="none" stroke="currentColor" font-family="system-ui,sans-serif" font-size="9">
  <style>
    .dot{animation:go 2.4s linear infinite}
    .d2{animation-delay:.8s}.d3{animation-delay:1.6s}
    .ring{animation:pulse 2.4s ease-in-out infinite}
    @keyframes go{0%{transform:translateX(0);opacity:0}10%{opacity:1}90%{opacity:1}100%{transform:translateX(150px);opacity:0}}
    @keyframes pulse{0%,100%{opacity:.35}50%{opacity:1}}
    @media (prefers-reduced-motion:reduce){.dot,.ring{animation:none}}
  </style>
  <g stroke="none" fill="currentColor">
    <text x="4" y="18" opacity=".8">session</text>
    <text x="4" y="40" opacity=".8">text</text>
    <text x="4" y="62" opacity=".8">image</text>
    <text x="196" y="40" opacity=".8">provider</text>
  </g>
  <path d="M42 15H120M42 37H120M42 59H120" opacity=".3"/>
  <path d="M120 15Q130 37 140 37M120 59Q130 37 140 37M120 37H140" opacity=".3"/>
  <rect class="ring" x="140" y="27" width="30" height="20" rx="5"/>
  <text x="146" y="40" stroke="none" fill="currentColor">hook</text>
  <path d="M170 37H192" opacity=".5"/>
  <g stroke="none" fill="currentColor">
    <circle class="dot" cx="42" cy="15" r="2.5"/>
    <circle class="dot d2" cx="42" cy="37" r="2.5"/>
    <circle class="dot d3" cx="42" cy="59" r="2.5"/>
  </g>
</svg>

No feature flag: the new hooks are opt-in and named experimental.*, and the session hooks run for the same requests as before.

Built-in provider plugins

Azure (Entra tokens), GitHub Copilot (auth and request metadata headers), Snowflake Cortex (env token, "conversation complete" handling) and OpenCode (the Console 401/403 explanation) now register model HTTP hooks instead of session ones. Before, their hooks skipped generate calls, so for example an Azure Entra model used through generate.text or the image tool was sent without its token.

Out of scope

  • Model-level WebSocket hooks. The session experimental.ws.* hooks are unchanged.
  • model.request (baseURL and headers) and the context / compaction / generate hook split.

Verification

  • bun run check: passes.
  • packages/core: new test in generate.test.ts with the real hook registry: Generate.image (a URL-image model), Generate.decisions, and a video start and status poll each run the request and response hooks with the right kind, the provider requests carry the hook's header, and the image download doesn't. It fails if the video wiring is removed. aisdk.test.ts checks scoped middleware reaches an AI SDK request, outside the per-call one (fails without the fix). Another new test in session-model-request-hooks.test.ts registers a model hook and a session hook. A text call runs only the model hook, a primary call runs both, and the session hook sees the model hook's header. A third test triggers the Azure hook with no Session and checks the Entra token is still applied (it fails before this change). The generate, session-hook, native-compaction, warming and provider plugin tests pass. packages/ai tests pass.
  • Ran opencode serve with a globally configured probe plugin that logs both hook types and adds an x-probe header:
    • generate/text (openai/gpt-5.4-nano), generate/image (openai/gpt-image-2), generate/decisions (opencode/jev-1.13-free) and voice/session (openai/gpt-realtime-2.1): the model hooks fired with the right kind and no session, and the provider got the replaced request. Video wasn't run live, since starting a job is paid.
    • A Session prompt on anthropic/claude-haiku-5-5: primary and title ran the model and session hooks with the session ID set.

Added by the opencode bot (AI-generated).

Related: #62096

Public API: adds experimental ctx.model.hook("experimental.http.request" | "experimental.http.response") covering Session, generate (text/image/decision/video) and voice requests (backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change works as described. In a temporary copy of generate.test.ts, I swapped in a hooks mock that records events and rewrites requests. generate.text and generate.image ran the model request and response hooks with the right kind, the provider received the rewritten request, and the URL-image download stayed unhooked. The decision tests passed with the same mock, but I didn't check their events. The existing generate and session-hook tests pass, and tsc is clean for core and plugin.

Public API: this adds to the plugin API and is backwards compatible.

  • New: ctx.model.hook("experimental.http.request" | "experimental.http.response", callback, { providerID? }) in both the Effect and Promise plugin APIs.
  • New exported types: ModelRequestKind, ModelRequestScope, ModelHttpRequest, ModelHttpResponse and ModelRequestHooks.
  • Unchanged: the session http.request / http.response hooks and their order. Model hooks wrap them.
  • Nothing is removed or renamed. The only break is for code that implements Plugin.Context["model"] itself, such as test hosts: it now needs a hook member.

One gap: no test exercises the new hooks (see the inline comment).

Related: #62096


const resolver = ModelResolver.layer.pipe(Layer.provide(Layer.mergeAll(providers, models, integrations, npm, aisdk)))
const generateDeps = Layer.mergeAll(models, RequestExecutor.fetchLayer)
const hooks = Layer.mock(PluginHooks.Service, { has: () => Effect.succeed(false) })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every Generate test now runs with has: () => false, so no test exercises the new model hooks: not the per-call image/decision client wiring, and not the model → session ordering in ModelHttp.middleware. Could you add one small test? For example, a hooks mock that reports model hooks as present and changes a header in experimental.http.request, then a check that generate.image (or generate.text) sends that header and passes the right kind. A small check of the model-around-session order in session-model-request-hooks.test.ts would also help.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your new commits address my earlier point: the new test in session-model-request-hooks.test.ts checks that a text call runs only the model hook, that a primary call runs both hooks with the session hook seeing the model hook's header, and that the provider gets the replaced request. The generate, session-hook, warming and Azure/OpenCode/Copilot provider plugin tests pass (151), and tsc is clean for core and plugin. The only remaining session http.* uses are registrations in the built-in provider plugins, and nothing triggers the old event shape, so remapping them to the model hooks can't silently skip a caller.

Public API (backwards compatible):

  • Added: ctx.model.hook("experimental.http.request" | "experimental.http.response") in the Effect and Promise plugin APIs, with session?: { id, agent } on the event.
  • Added types: ModelRequestKind, ModelRequestSession, ModelRequestScope, ModelHttpRequest, ModelHttpResponse, ModelRequestHooks.
  • Deprecated: the session http.request / http.response hooks. Their event shape is unchanged and they still run only for Session requests.
  • Changed: when several plugins register HTTP hooks, they now run in plugin order rather than model hooks first and then session hooks.
  • Code that implements Plugin.Context["model"] itself now needs a hook member.

No new concerns.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your latest commit addresses my remaining point: the new generate.test.ts case checks that image and decision calls each run one request and one response model hook with the right kind, that the provider requests carry the hook's header and that the URL-image download doesn't. When no hook is registered, image and decision calls are back on the shared clients, and has/trigger no longer accept the session http.* names. The generate, session-hook, warming and provider plugin tests pass (151), and tsc is clean for core and plugin.

Public API: the same as in my previous review, minus ModelRequestKind, which was only added earlier in this pull request. It's backwards compatible.

No new concerns.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit is sound. The built-in Azure, Copilot, Snowflake Cortex and OpenCode plugins now use model HTTP hooks with the same providerID filters, and none of the moved callbacks read session-only fields, so generate calls now get the same auth and response handling as Sessions. The new Azure assertion covers a Session-less request. The provider, warming, session-hook and generate tests pass (158), and tsc is clean for core and plugin.

My earlier test point was already addressed.

Public API (backwards compatible):

  • Added: ctx.model.hook("experimental.http.request" | "experimental.http.response") in the Effect and Promise plugin APIs.
  • Unchanged for existing plugins: the session http.* hooks, which are no longer marked deprecated, keep their event shape and still run only for Session requests.
  • Changed: when several plugins register HTTP hooks, they run in plugin order.

No new concerns.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit is sound. Session http.* hooks are back in their own list, and ModelHttp.middleware runs them inside the model hooks only for Session requests. The model/session hook test still covers that order and the Session-only filter. The provider, warming, session-hook and generate tests pass (158), and tsc is clean for core and plugin.

My earlier test point was already addressed.

Public API (backwards compatible):

  • Added: ctx.model.hook("experimental.http.request" | "experimental.http.response") in the Effect and Promise plugin APIs.
  • Unchanged: the session http.* hooks keep their event shape and still run only for Session requests.
  • Order: for Session requests, model hooks always wrap session hooks: model request → session request → send → session response → model response.

No new concerns.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit is sound. Image and decision calls now pass the model hooks per call ({ http } on the shared clients, as llm.generate already did) rather than building a fresh hooked client, and queued image polling goes through the same wrapped executor. The image/decision hook test now uses the real hook registry. The core tests I ran (67) and the full packages/ai suite (2093) pass, and tsc is clean for core, ai and plugin.

My earlier test point was already addressed.

Public API (backwards compatible):

  • Plugin API: unchanged from my previous review (adds the experimental ctx.model.hook("experimental.http.*"); the session http.* hooks are unchanged).
  • New in @opencode/ai: RequestExecutor.withHttp. The image/media client's generate, stream, start and resume take an optional http middleware, and DecisionClient.generate takes an optional { http }. Existing callers don't need to change.

No new concerns.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only new commit merges dev, and the pull request's own changes are the same as before (only line offsets differ). The hook, generate, Azure and warming tests pass (70), and tsc is clean for core, ai and plugin. I didn't check how the newly merged client-certificate change (#61832) interacts with the hooked generate calls.

My earlier test point was already addressed.

Public API: unchanged since my previous review, and backwards compatible.

No new concerns.

# Conflicts:
#	packages/ai/src/media-client.ts
#	packages/ai/src/route/executor.ts
#	packages/core/src/generate.ts

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merge moves Generate (text, image, decision and video) and Voice.session onto ModelHttp.scoped, built on the scoped middleware from #64152, and the new wiring looks sound. Hook-made requests aren't wrapped again: the executor clears the scoped middleware, and so does the new code in aisdk.ts. Retries sit outside the scope, so each retry runs the hooks. AI SDK routes now apply scoped middleware outside the per-call one. The hook, generate, aisdk, voice, warming and provider plugin tests pass (211), and tsc is clean for core and plugin.

My earlier test point was already addressed.

Public API (backwards compatible):

  • Plugin API: adds the experimental ctx.model.hook("experimental.http.request" | "experimental.http.response"), with kind now including video and voice.
  • Unchanged: the session http.* hooks keep their event shape and still run only for Session requests.
  • The per-call http options on the @opencode/ai image and decision clients are no longer part of this pull request.

No new concerns.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant