Repository navigation
feat(plugin): add experimental model HTTP hooks - #62820
rekram1-node wants to merge 9 commits into
Conversation
There was a problem hiding this comment.
The change works as described. In a temporary copy of generate.test.ts, I swapped in a hooks mock that records events and rewrites requests. generate.text and generate.image ran the model request and response hooks with the right kind, the provider received the rewritten request, and the URL-image download stayed unhooked. The decision tests passed with the same mock, but I didn't check their events. The existing generate and session-hook tests pass, and tsc is clean for core and plugin.
Public API: this adds to the plugin API and is backwards compatible.
- New:
ctx.model.hook("experimental.http.request" | "experimental.http.response", callback, { providerID? })in both the Effect and Promise plugin APIs. - New exported types:
ModelRequestKind,ModelRequestScope,ModelHttpRequest,ModelHttpResponseandModelRequestHooks. - Unchanged: the session
http.request/http.responsehooks and their order. Model hooks wrap them. - Nothing is removed or renamed. The only break is for code that implements
Plugin.Context["model"]itself, such as test hosts: it now needs ahookmember.
One gap: no test exercises the new hooks (see the inline comment).
Related: #62096
|
|
||
| const resolver = ModelResolver.layer.pipe(Layer.provide(Layer.mergeAll(providers, models, integrations, npm, aisdk))) | ||
| const generateDeps = Layer.mergeAll(models, RequestExecutor.fetchLayer) | ||
| const hooks = Layer.mock(PluginHooks.Service, { has: () => Effect.succeed(false) }) |
There was a problem hiding this comment.
Every Generate test now runs with has: () => false, so no test exercises the new model hooks: not the per-call image/decision client wiring, and not the model → session ordering in ModelHttp.middleware. Could you add one small test? For example, a hooks mock that reports model hooks as present and changes a header in experimental.http.request, then a check that generate.image (or generate.text) sends that header and passes the right kind. A small check of the model-around-session order in session-model-request-hooks.test.ts would also help.
9ba58da to
704df14
Compare
There was a problem hiding this comment.
Your new commits address my earlier point: the new test in session-model-request-hooks.test.ts checks that a text call runs only the model hook, that a primary call runs both hooks with the session hook seeing the model hook's header, and that the provider gets the replaced request. The generate, session-hook, warming and Azure/OpenCode/Copilot provider plugin tests pass (151), and tsc is clean for core and plugin. The only remaining session http.* uses are registrations in the built-in provider plugins, and nothing triggers the old event shape, so remapping them to the model hooks can't silently skip a caller.
Public API (backwards compatible):
- Added:
ctx.model.hook("experimental.http.request" | "experimental.http.response")in the Effect and Promise plugin APIs, withsession?: { id, agent }on the event. - Added types:
ModelRequestKind,ModelRequestSession,ModelRequestScope,ModelHttpRequest,ModelHttpResponse,ModelRequestHooks. - Deprecated: the session
http.request/http.responsehooks. Their event shape is unchanged and they still run only for Session requests. - Changed: when several plugins register HTTP hooks, they now run in plugin order rather than model hooks first and then session hooks.
- Code that implements
Plugin.Context["model"]itself now needs ahookmember.
No new concerns.
There was a problem hiding this comment.
Your latest commit addresses my remaining point: the new generate.test.ts case checks that image and decision calls each run one request and one response model hook with the right kind, that the provider requests carry the hook's header and that the URL-image download doesn't. When no hook is registered, image and decision calls are back on the shared clients, and has/trigger no longer accept the session http.* names. The generate, session-hook, warming and provider plugin tests pass (151), and tsc is clean for core and plugin.
Public API: the same as in my previous review, minus ModelRequestKind, which was only added earlier in this pull request. It's backwards compatible.
No new concerns.
There was a problem hiding this comment.
The new commit is sound. The built-in Azure, Copilot, Snowflake Cortex and OpenCode plugins now use model HTTP hooks with the same providerID filters, and none of the moved callbacks read session-only fields, so generate calls now get the same auth and response handling as Sessions. The new Azure assertion covers a Session-less request. The provider, warming, session-hook and generate tests pass (158), and tsc is clean for core and plugin.
My earlier test point was already addressed.
Public API (backwards compatible):
- Added:
ctx.model.hook("experimental.http.request" | "experimental.http.response")in the Effect and Promise plugin APIs. - Unchanged for existing plugins: the session
http.*hooks, which are no longer marked deprecated, keep their event shape and still run only for Session requests. - Changed: when several plugins register HTTP hooks, they run in plugin order.
No new concerns.
There was a problem hiding this comment.
The new commit is sound. Session http.* hooks are back in their own list, and ModelHttp.middleware runs them inside the model hooks only for Session requests. The model/session hook test still covers that order and the Session-only filter. The provider, warming, session-hook and generate tests pass (158), and tsc is clean for core and plugin.
My earlier test point was already addressed.
Public API (backwards compatible):
- Added:
ctx.model.hook("experimental.http.request" | "experimental.http.response")in the Effect and Promise plugin APIs. - Unchanged: the session
http.*hooks keep their event shape and still run only for Session requests. - Order: for Session requests, model hooks always wrap session hooks: model request → session request → send → session response → model response.
No new concerns.
There was a problem hiding this comment.
The new commit is sound. Image and decision calls now pass the model hooks per call ({ http } on the shared clients, as llm.generate already did) rather than building a fresh hooked client, and queued image polling goes through the same wrapped executor. The image/decision hook test now uses the real hook registry. The core tests I ran (67) and the full packages/ai suite (2093) pass, and tsc is clean for core, ai and plugin.
My earlier test point was already addressed.
Public API (backwards compatible):
- Plugin API: unchanged from my previous review (adds the experimental
ctx.model.hook("experimental.http.*"); the sessionhttp.*hooks are unchanged). - New in
@opencode/ai:RequestExecutor.withHttp. The image/media client'sgenerate,stream,startandresumetake an optionalhttpmiddleware, andDecisionClient.generatetakes an optional{ http }. Existing callers don't need to change.
No new concerns.
There was a problem hiding this comment.
The only new commit merges dev, and the pull request's own changes are the same as before (only line offsets differ). The hook, generate, Azure and warming tests pass (70), and tsc is clean for core, ai and plugin. I didn't check how the newly merged client-certificate change (#61832) interacts with the hooked generate calls.
My earlier test point was already addressed.
Public API: unchanged since my previous review, and backwards compatible.
No new concerns.
# Conflicts: # packages/ai/src/media-client.ts # packages/ai/src/route/executor.ts # packages/core/src/generate.ts
There was a problem hiding this comment.
The merge moves Generate (text, image, decision and video) and Voice.session onto ModelHttp.scoped, built on the scoped middleware from #64152, and the new wiring looks sound. Hook-made requests aren't wrapped again: the executor clears the scoped middleware, and so does the new code in aisdk.ts. Retries sit outside the scope, so each retry runs the hooks. AI SDK routes now apply scoped middleware outside the per-call one. The hook, generate, aisdk, voice, warming and provider plugin tests pass (211), and tsc is clean for core and plugin.
My earlier test point was already addressed.
Public API (backwards compatible):
- Plugin API: adds the experimental
ctx.model.hook("experimental.http.request" | "experimental.http.response"), withkindnow includingvideoandvoice. - Unchanged: the session
http.*hooks keep their event shape and still run only for Session requests. - The per-call
httpoptions on the@opencode/aiimage and decision clients are no longer part of this pull request.
No new concerns.
Problem
Plugin HTTP hooks only run for Session requests.
generatetext, image, decision and video calls and voice session setup send provider requests that no hook can see or change (#62096). Anything that has to touch every model request, like the provider client certificates in #61832, has no single place to do it.Change
Adds
ctx.model.hook("experimental.http.request" | "experimental.http.response", ...). It runs for every HTTP request to a model provider, whichever Session or API makes it.sessionis set for a Session's own model requests, so checking it narrowskind. A tool that calls Generate from inside a Session (like the built-in image tool) makes a Generate call with nosession.The session
http.request/http.responsehooks are unchanged and still only see Session requests. The docs recommend the model hooks. For a Session request, both run in one middleware, with the model hooks outside:How it's wired:
core/src/model-http.tsbuilds the middleware. It's moved out ofsession/model-request.ts, which now calls it and passes it per call as before.Generate(text, image, decision, video start/status/result) andVoice.sessionwrap the call inModelHttp.scoped, which uses the AI package'sRequestExecutor.withMiddlewarefrom feat(ai): scope HTTP middleware to any call, for every modality #64152. That covers every request of the call, media polls included. A result downloaded from the provider's own host keeps the hook; one from a CDN doesn't.PluginHooksonly gains themodeldomain.The hooks run on each request sent (retries too), after the provider's auth headers are set, and a hook that fails fails the request. The docs say so.
generatecalls through the HTTP API run in the global config location, so only globally configured plugins see them. That's noted in the docs.Visual
No feature flag: the new hooks are opt-in and named
experimental.*, and the session hooks run for the same requests as before.Built-in provider plugins
Azure (Entra tokens), GitHub Copilot (auth and request metadata headers), Snowflake Cortex (env token, "conversation complete" handling) and OpenCode (the Console 401/403 explanation) now register model HTTP hooks instead of session ones. Before, their hooks skipped
generatecalls, so for example an Azure Entra model used throughgenerate.textor the image tool was sent without its token.Out of scope
experimental.ws.*hooks are unchanged.model.request(baseURL and headers) and thecontext/compaction/generatehook split.Verification
bun run check: passes.packages/core: new test ingenerate.test.tswith the real hook registry:Generate.image(a URL-image model),Generate.decisions, and a video start and status poll each run the request and response hooks with the rightkind, the provider requests carry the hook's header, and the image download doesn't. It fails if the video wiring is removed.aisdk.test.tschecks scoped middleware reaches an AI SDK request, outside the per-call one (fails without the fix). Another new test insession-model-request-hooks.test.tsregisters a model hook and a session hook. Atextcall runs only the model hook, aprimarycall runs both, and the session hook sees the model hook's header. A third test triggers the Azure hook with no Session and checks the Entra token is still applied (it fails before this change). The generate, session-hook, native-compaction, warming and provider plugin tests pass.packages/aitests pass.opencode servewith a globally configured probe plugin that logs both hook types and adds anx-probeheader:generate/text(openai/gpt-5.4-nano),generate/image(openai/gpt-image-2),generate/decisions(opencode/jev-1.13-free) andvoice/session(openai/gpt-realtime-2.1): the model hooks fired with the rightkindand no session, and the provider got the replaced request. Video wasn't run live, since starting a job is paid.primaryandtitleran the model and session hooks with the session ID set.Added by the opencode bot (AI-generated).
Related: #62096
Public API: adds experimental
ctx.model.hook("experimental.http.request" | "experimental.http.response")covering Session, generate (text/image/decision/video) and voice requests (backwards compatible)