automations: capture client customizations for offline runs - #338990
Merged
Connor Peet (connor4312) merged 10 commits intoOct 1, 2026
Merged
Conversation
Syncs microsoft/agent-host-protocol#474 (rebased onto c02ad7ef) which adds AutomationSessionTemplate.customizations, AutomationEntry.customizations and the automations.customizations capability. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Captures client plugins from an automation's session template into immutable host-owned directories on create/retarget, and seeds run sessions with them through a static active client, so every provider loads them through its existing client-plugin path. Unreferenced copies are collected at startup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the host advertises automations.customizations, send the target scope's enabled plugins on create and retarget, and remap a selected bundled custom agent to its synced URI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Connor Peet (connor4312)
force-pushed
the
connor4312/automation-customizations-v2
branch
from
September 30, 2026 21:04
9713828 to
6820753
Compare
Copilot started reviewing on behalf of
Connor Peet (connor4312)
September 30, 2026 21:04
View session
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Captured plugins are mutable through resource APIs, can collide across clients, and leak staging data after failures.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Enables automations to capture client plugins so scheduled and manual runs can use them without a connected client.
Changes:
- Adds the AHP customization capability and automation fields.
- Captures enabled plugins during creation or retargeting.
- Runs automations with host-owned plugin copies and adds coverage.
| File | Description |
|---|---|
src/vs/workbench/contrib/chat/test/browser/agentSessions/syncedCustomizationBundler.test.ts |
Tests bundled URI lookup. |
src/vs/workbench/contrib/chat/test/browser/agentSessions/agentHostUntitledProvisionalSessionService.test.ts |
Updates scope mock. |
src/vs/workbench/contrib/chat/test/browser/agentSessions/agentHostChatContribution.test.ts |
Updates scope mock. |
src/vs/workbench/contrib/chat/browser/agentSessions/agentHost/syncedCustomizationBundler.ts |
Maps source URIs to bundled URIs. |
src/vs/workbench/contrib/chat/browser/agentSessions/agentHost/agentHostActiveClientService.ts |
Exposes bundled URI lookup. |
src/vs/sessions/contrib/providers/remoteAgentHost/test/browser/remoteAgentHostSessionsProvider.test.ts |
Updates remote scope mock. |
src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts |
Updates local scope mocks. |
src/vs/sessions/contrib/providers/agentHost/test/browser/agentHostAutomationStore.test.ts |
Tests client capture behavior. |
src/vs/sessions/contrib/providers/agentHost/browser/agentHostAutomationStore.ts |
Captures plugins when saving automations. |
src/vs/sessions/AUTOMATIONS.md |
Documents captured customizations. |
src/vs/platform/agentHost/test/node/copilotAgent.test.ts |
Updates plugin-manager fake. |
src/vs/platform/agentHost/test/node/codex/codexPrewarmEviction.test.ts |
Updates plugin-manager stub. |
src/vs/platform/agentHost/test/node/codex/codexCreateChat.test.ts |
Updates plugin-manager stub. |
src/vs/platform/agentHost/test/node/claudeAgent.test.ts |
Updates plugin-manager fake. |
src/vs/platform/agentHost/test/node/claudeAgent.integrationTest.ts |
Updates integration stubs. |
src/vs/platform/agentHost/test/node/agentServiceTestUtils.ts |
Registers required host services. |
src/vs/platform/agentHost/test/node/agentPluginManager.test.ts |
Tests host-owned plugin loading. |
src/vs/platform/agentHost/test/node/agentHostAutomationService.test.ts |
Tests capture, execution, and cleanup. |
src/vs/platform/agentHost/test/node/agentHostAutomationCustomizations.test.ts |
Tests the capture module. |
src/vs/platform/agentHost/node/agentService.ts |
Passes client identity and captured plugins. |
src/vs/platform/agentHost/node/agentPluginManager.ts |
Loads host-owned directories in place. |
src/vs/platform/agentHost/node/agentHostAutomationService.ts |
Integrates captures into automation lifecycle. |
src/vs/platform/agentHost/node/agentHostAutomationCustomizations.ts |
Implements capture, mapping, and startup GC. |
src/vs/platform/agentHost/common/state/protocol/common/commands.ts |
Adds the capability contract. |
src/vs/platform/agentHost/common/state/protocol/channels-automation/state.ts |
Adds customization state fields. |
src/vs/platform/agentHost/common/state/protocol/channels-automation/actions.ts |
Documents capture action semantics. |
src/vs/platform/agentHost/common/state/protocol/.ahp-version |
Updates the synced AHP revision. |
src/vs/platform/agentHost/common/agentPluginManager.ts |
Exposes the host plugin path. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
Screenshot ChangesBase: Changed (7)
|
| Before | After |
|---|---|
 |
 |
sessions/accountMenu/WeeklyLimitOnly/Light
| Before | After |
|---|---|
 |
 |
sessions/accountMenu/FiveHourLimitOnly/Light
| Before | After |
|---|---|
 |
 |
sessions/accountMenu/UsageUnavailable/Light
| Before | After |
|---|---|
 |
 |
sessions/accountMenu/LimitsWithoutReset/Light
| Before | After |
|---|---|
 |
 |
sessions/accountMenu/FiveHourLimitHover/Light
| Before | After |
|---|---|
 |
 |
chat/aiCustomizations/aiCustomizationManagementEditor/DiscoverPluginsLoadingMore/Light
| Before | After |
|---|---|
 |
 |
2 insignificant change(s) omitted (≤20 px, Δ≤2). See CI logs for details.
Copilot started reviewing on behalf of
Connor Peet (connor4312)
September 30, 2026 21:21
View session
Advertise the customizations capability in the automation conformance snapshot, and back the test plugin manager with an in-memory provider so startup cleanup doesn't warn when callers pass a bare file service. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collect copies no automation references after each create, update, and removal, keeping copies used by run sessions in this process. When retargeting, map a selected agent saved from the previous target's bundle into the new bundle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Connor Peet (connor4312)
marked this pull request as ready for review
October 1, 2026 00:10
Contributor
📬 CODENOTIFYThe following users are being notified based on files changed in this PR: Ulugbek Abdullaev (@ulugbekna)Matched files:
Ben Villalobos (@benvillalobos)Matched files:
|
roblourens
previously approved these changes
Oct 1, 2026
When an MCP server in an automation run session requires authentication, the host asks connected VS Code clients via vscode/requestMcpAuthentication. Clients reuse an existing allowed sign-in without prompting and push the token through the regular authenticate command, which unblocks the run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…-customizations-v2 # Conflicts: # src/vs/platform/agentHost/common/state/protocol/.ahp-version
roblourens
previously approved these changes
Oct 1, 2026
Member
|
Only concern is that saving plugins could lead to slow feeling UX is a remote if network is slow
If the stop button does nothing and shows no UX (spinner or something) we may need to smooth that out. Otherwise lgtm |
Member
Author
|
Thanks for the review! Runs with a concurrent stop should generally work barring the entire connection being bogged down. That is a thing that has come up a few times and I'm probably going to make a protocol-level change to help resolve that (adding SSH-like receive windows for backpressure per-channel). But I will do a followup UI PR to include a 'saving' state for automations. |
Connor Peet (connor4312)
enabled auto-merge
October 1, 2026 17:07
…ests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…-customizations-v2 # Conflicts: # src/vs/platform/agentHost/common/state/protocol/.ahp-version
roblourens
approved these changes
Oct 1, 2026
Connor Peet (connor4312)
deleted the
connor4312/automation-customizations-v2
branch
October 1, 2026 17:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lets scheduled and manual automation runs use the client's plugins without the client being connected. This is a simpler replacement for #338777 and depends on microsoft/agent-host-protocol#474.
How it works
AutomationSessionTemplate.customizations,AutomationEntry.customizations, and theautomations.customizationscapability.AgentHostAutomationCustomizationsmodule copies new or changed entries into immutable directories underagentPlugins/.host/automations/and reuses copies whoseid/uri/nonceare unchanged. Any capture failure rejects the mutation.vscode.automationactive client whose plugins point at the copies. Every provider (Copilot, Claude, Codex) already loads active-client plugins before the first turn, so there are no provider, enablement-gate, or session-config changes. The plugin manager uses.hostdirectories in place.Compared with #338777, this drops plugin-cache holders and leases, per-run snapshots, session-deletion hooks, reattaching plugins on restore, provider-specific captured-plugin branches, and client bundle snapshots. Production code is about +320/−54 instead of +1,511/−266.
MCP authentication in runs
Until the host can own MCP authentication, run sessions borrow existing sign-ins from connected windows. When an MCP server in a run session requires authentication, the host sends a VS Code-specific
vscode/requestMcpAuthenticationrequest to connected clients one at a time. A client never prompts: it reuses a sign-in the user already allowed for that server, pushes the token through the regularauthenticatecommand, and reports success. The existing token fan-out then unblocks the run, so there are no provider changes. Each challenge is requested once per session and server; if no client can supply a token, the run continues without that server.Known limitations
Validation
npm run typecheck-client: passed./scripts/test.sh --grep 'AgentHostAutomationCustomizations|AgentHostAutomationService|AgentPluginManager|AgentHostAutomationStore|SyncedCustomizationBundler': 160 passing./scripts/test.sh --runGlob '**/agentHost/test/node/**/*.test.js': 7,862 passing, 72 pending./scripts/test-integration.sh --run src/vs/platform/agentHost/test/node/e2e/conformance/agentHostConformance.integrationTest.ts: automation capability test passes; one changeset-discard test times out locally and fails identically onmaingit diff --check: cleanworkiq-me-GetMyDetails. Servers without a prior sign-in stayed unauthenticated as intended.