Skip to content

[cherry-pick] automations: fix: limit New badge to returning Agents users - #334944

Merged
Ulugbek Abdullaev (ulugbekna) merged 2 commits into
release/1.137from
cherry-pick/334898
Sep 7, 2026
Merged

[cherry-pick] automations: fix: limit New badge to returning Agents users#334944
Ulugbek Abdullaev (ulugbekna) merged 2 commits into
release/1.137from
cherry-pick/334898

Conversation

@vs-code-engineering

Copy link
Copy Markdown
Contributor

Cherry-pick of #334898 from main.

Summary

Restrict the Automations New badge to returning Agents-window users with no prior Automation discovery, and stop treating loading or failed catalogues as proof that Automations are unused.

Follow-up to #334091. The badge designs and renderer already exist; this changes eligibility, lifecycle handling, and related tests.

Rebased onto 70a1d06d93f6a90d3cae4bf84754b096d818ef40. The rebase reuses the canonical catalogueState API introduced by #334836 instead of retaining a parallel discovery contract. The PR now changes 15 files rather than 24.

Why

The previous badge sampled the seen bit, definitions, and runs once, synchronously. If all were empty, it resolved the style and displayed New.

That causes two problems:

  • A first-time Agents-window user sees a promotion even though the whole interface is new.
  • Agent Host providers register and hydrate asynchronously. Empty arrays can mean "not loaded yet" or "unavailable", and the one-shot check deliberately ignored later definitions/runs. Unreadable legacy data can also produce an empty projection without establishing non-use.

Implementation

Returning-window detection without storage infrastructure changes

SessionsWindowUsageService snapshots whether the existing application-scoped, machine-targeted agentSessions.telemetry.summary.appLaunchCount was nonzero before recording the current open.

The first window therefore retains hadPriorWindowOpen = false even after it stores count 1. Later windows, including users whose count was written by older builds, are recognized without relying on chat/session history.

The usage service owns the increment; SessionsLifecycleTracker receives the captured count through SessionsTelemetryContribution rather than incrementing it a second time. The stored format and telemetry event schemas are unchanged, although the increment now occurs when the usage service is instantiated rather than specifically when the telemetry tracker is constructed.

This uses ordinary application storage. No shared storage API, IPC, IndexedDB, or base/platform changes are included. Two simultaneous first opens can both read zero and remain unbadged; that conservative false negative is acceptable for a discovery cue.

Reuse upstream catalogue completeness

The badge reads the existing IAutomationService.catalogueState and accepts only ready. loading, unavailable, and error all suppress it.

Upstream already provides:

  • Aggregation over registered provider stores and the global legacy store, with precedence error > loading > unavailable > ready.
  • A loading guard until initial provider contributions settle, plus reactive updates for provider registration and catalogue changes.
  • Observable legacy-ledger readability and Agent Host subscription health, without conflating those with migration authority.
  • Conservative connection-state mapping: unsupported, disabled, and disconnected hosts remain unavailable unless their legacy storage reports an error.

The shared contract, aggregate and legacy implementations, Agent Host store implementations, provider interface, and architecture specification are unchanged from upstream. The former optional initialDiscoveryState API and its duplicated implementation are no longer part of this PR.

The local provider still needs the lifecycle fix: clear its connection listeners and Automation connection on host exit, then use its existing start binding to reconnect. This prevents a stale ready catalogue from surviving local host shutdown.

Migrated definitions and execution remain Agent Host-owned. Legacy ledgers remain compatibility sources; no schema, scheduling, permission, or session-template change is introduced.

Conservative badge lifecycle

For an unseen returning user, the badge waits until LifecyclePhase.Eventually, then evaluates the current aggregate once. It shows only when the catalogue is ready and definitions/runs are empty. It does not await a retrying migration promise.

Scenario Behavior
First Agents window Hidden; no direct badge-style treatment request.
Returning user, ready and empty catalogue Show the resolved style.
Existing definition/run or Automations view activation Persist seen and hide.
Loading, unavailable, or errored catalogue at the startup decision Suppress for that badge-state instance without recording seen.
Catalogue regresses after eligibility Suppress; recovery does not automatically reveal it again.
Definition/run appears later, including through tools Retire the badge and persist seen.
Developer reset Explicitly preview despite prior-use/readiness/evidence gates, until Automations is activated.

The startup phase is an opportunity for initial providers to populate, not a guarantee that every future provider has registered. The aggregate includes upstream's initial-provider-settlement guard; later data/readiness changes remain observable.

The existing hidden style setting, ExP styles, and accessible row labels are preserved. Style refreshes cannot bypass eligibility, and disposed states do not resume presentation after async work. The evidence observer is cleared on retirement.

Scope and tradeoffs

  • No new telemetry event or user-visible setting.
  • No badge CSS, renderer redesign, or screenshot-baseline change.
  • A slow, unsupported, unavailable, or errored source can hide the badge even for a user with no Automations; silence is preferred to a misleading promotion.
  • Prior-use evidence is local application state, not account-wide adoption tracking. Historical view-only use before the seen bit existed cannot be reconstructed exactly.
  • Startup suppression and developer preview are in-memory badge state; normal seen dismissal is persisted.

Validation

After rebasing, using fresh transpiled output:

  • 290 focused unit tests passed across window usage, lifecycle tracker, telemetry contribution, badge, Sessions list, Agent Host Automation store, aggregate Automation service, and legacy Automation service, plus the targeted local host exit/restart regression.
  • npm run typecheck-client passed.
  • Targeted ESLint passed for all 15 changed TypeScript files; the final lifecycle-fixture adjustment was rechecked.
  • git diff --check passed.
  • Verified zero diff from the new base for base/platform code, workbench storage services, generic fixture utilities, shared Automation contracts, aggregate/legacy implementations, Agent Host stores, and the architecture specification.
  • Reviewed the rebase range-diff to confirm the prior-window snapshot, telemetry integration, and local-host cleanup were preserved.

The badge regressions cover all three non-ready catalogue states both at startup and after presentation, including recovery remaining quiet. The local-host regression starts with a completed catalogue and checks ready -> unavailable -> ready across exit/restart.

The earlier review concerns remain covered: unsupported hosts never imply a ready catalogue, and upstream now directly tests provider registration, live aggregate updates, startup settlement, and the full precedence matrix in both provider orders. This PR also checks aggregate error for a future-schema global legacy ledger.

Earlier validation exercised all 15 badge variants across Dark, Light, and Dark High Contrast. Visual fixtures were not rerun after this rebase; no visual styles or screenshot baselines changed.

To rerun the focused tests:

npm run transpile-client
./scripts/test.sh \
  --run src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts \
  --run src/vs/sessions/contrib/sessions/test/browser/sessionsLifecycleTracker.test.ts \
  --run src/vs/sessions/contrib/sessions/test/browser/sessionsTelemetry.contribution.test.ts \
  --run src/vs/sessions/contrib/sessions/test/browser/automationsNewBadge.test.ts \
  --run src/vs/sessions/contrib/sessions/test/browser/sessionsList.test.ts \
  --run src/vs/sessions/contrib/providers/agentHost/test/browser/agentHostAutomationStore.test.ts \
  --run src/vs/sessions/contrib/automations/test/browser/providerAutomationService.test.ts \
  --run src/vs/sessions/contrib/automations/test/browser/automationService.test.ts \
  --run src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts \
  --grep '^(AutomationsNewBadgeState|SessionsWindowUsageService|SessionsLifecycleTracker|SessionsTelemetryContribution|Sessions - SessionsList|ProviderAutomationService|AutomationService|AgentHostAutomationStore|LocalAgentHostSessionsProvider Automation catalogue state follows)' \
  --reporter dot

For visual iteration in a development Agents window, Developer: Reset Automations New Badge force-previews the current sessions.automations.newBadgeStyle (accent, soft, or outline). Opening Automations dismisses it. This previews presentation, not production returning-user eligibility.

Copilot AI balanced review requested due to automatic review settings September 7, 2026 15:59
@vs-code-engineering vs-code-engineering Bot added the cherry-pick-artifact Auto-generated cherry-pick PR label Sep 7, 2026
@vs-code-engineering

Copy link
Copy Markdown
Contributor Author

📬 CODENOTIFY

The following users are being notified based on files changed in this PR:

Sandeep Somavarapu (@sandy081)

Matched files:

  • src/vs/sessions/services/sessions/browser/sessionsWindowUsageService.ts
  • src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts

Ladislau Szomoru (@lszomoru)

Matched files:

  • src/vs/sessions/services/sessions/browser/sessionsWindowUsageService.ts
  • src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused lifecycle changes are internally consistent and comprehensively covered by regression tests.

Pull request overview

Restricts the Automations New badge to eligible returning Agents-window users and prevents incomplete catalogues from triggering it.

Changes:

  • Tracks prior Agents-window usage without double-counting telemetry launches.
  • Gates and retires the badge using lifecycle, catalogue readiness, and Automation evidence.
  • Clears and restores local Agent Host Automation state across disconnects.
File summaries
File Description
src/vs/workbench/test/browser/componentFixtures/sessions/sessionsList.fixture.ts Updates badge fixture dependencies and preview setup.
src/vs/sessions/sessions.common.main.ts Registers the window-usage service.
src/vs/sessions/services/sessions/test/browser/sessionsWindowUsageService.test.ts Tests prior-window detection and counting.
src/vs/sessions/services/sessions/browser/sessionsWindowUsageService.ts Adds shared Agents-window usage tracking.
src/vs/sessions/contrib/sessions/test/browser/sessionsTelemetry.contribution.test.ts Supplies usage state to telemetry tests.
src/vs/sessions/contrib/sessions/test/browser/sessionsListTestUtils.ts Adds the usage-service test stub.
src/vs/sessions/contrib/sessions/test/browser/sessionsList.test.ts Updates accessible-label badge coverage.
src/vs/sessions/contrib/sessions/test/browser/sessionsLifecycleTracker.test.ts Adapts lifecycle tests to captured launch counts.
src/vs/sessions/contrib/sessions/test/browser/automationsNewBadge.test.ts Covers eligibility, suppression, and preview behavior.
src/vs/sessions/contrib/sessions/browser/sessionsTelemetry.contribution.ts Passes the shared launch count into telemetry tracking.
src/vs/sessions/contrib/sessions/browser/sessionsLifecycleTracker.ts Stops independently incrementing launch counts.
src/vs/sessions/contrib/sessions/browser/automationsNewBadge.ts Implements conservative badge eligibility and lifecycle handling.
src/vs/sessions/contrib/providers/agentHost/test/browser/localAgentHostSessionsProvider.test.ts Tests catalogue state across host restart.
src/vs/sessions/contrib/providers/agentHost/browser/localAgentHostSessionsProvider.ts Clears Automation bindings when the host exits.
src/vs/sessions/contrib/automations/test/browser/providerAutomationService.test.ts Verifies future-ledger catalogue errors.
Review details
  • Files reviewed: 15/15 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

The badge cherry-pick calls catalogueState, but release/1.137 does not yet contain the catalogue API from #334836. Backport the required contract, aggregate readiness, legacy readability, and Agent Host lifecycle support without importing the templates UI. Include focused catalogue and migration regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7e75f732-f18f-42a7-b1e2-a034c03596eb
@ulugbekna

Copy link
Copy Markdown
Contributor

The CI failures were caused by a missing release-branch prerequisite: the badge cherry-pick uses catalogueState, which was added on main by #334836 but is not yet in release/1.137. Fixed in d83f4e1 by backporting only the catalogue contract, provider aggregation/startup readiness, legacy readability, and Agent Host lifecycle support, plus focused regressions. This does not import the templates UI or modify generic storage infrastructure. The separate full backport #334865 remains independent.

Validation on the release checkout: client typecheck and targeted lint/hygiene passed; 518 Electron tests passed, and 388 tests passed in each of Chromium and WebKit (11 existing pending tests per run). The two original CI failures now pass. Waiting for fresh CI before merging.

@ulugbekna

Copy link
Copy Markdown
Contributor

All CI checks on d83f4e1 are now green, including Windows: 29 passing checks and no failing or queued test jobs. The compile and catalogue test failures are resolved. Auto-merge is enabled; the only remaining gate is the required collaborator approvals.

@ulugbekna Ulugbek Abdullaev (ulugbekna) added this to the 1.137.0 milestone Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-pick-artifact Auto-generated cherry-pick PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants