Repository navigation
feat(telemetry): add the Copilot CLI telemetry adapter and guard Copilot hooks in VS Code (3/4, GRO-305) - #28
Conversation
…pter Every Copilot hook command now checks that its script exists, so VS Code, which reads the same hooks.json without setting PLUGIN_ROOT, exits 0 with no output. The extra event names and adapter name are inactive while TELEMETRY_ENABLED is false. Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Port PR #13 Copilot mapping into hooks/telemetry-adapters/copilot-cli.mjs with fixture-driven captureTelemetry tests. Telemetry remains off; the checked-in Copilot hooks.json is unchanged. Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Match PR #13 field selection in normalize, filter telemetry rows in the enabled-manifest wiring test, and split normalize tests by behavior. Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
# Conflicts: # test/hooks.test.mjs Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
teallarson
left a comment
teallarson
left a comment
There was a problem hiding this comment.
Independent review of the current telemetry-OFF draft. Findings below remain valid at this exact head; no implementation, approval, merge or collection enablement is implied.
— Strider 🐦⬛, Teal's agent · approved by Teal
Both the shell and PowerShell hard-OFF tests now set ARCADE_PLUGIN_TELEMETRY_HOST to the loopback capture server, use a prompt that would produce an event if the gate were open, and finish with a positive control that sends a synthetic event through the same environment and requires the server to receive it. Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
|
@cursor review — Strider 🐦⬛, Teal's agent · approved by Teal |
…y tests An inherited DO_NOT_TRACK, COPILOT_OFFLINE or ARCADE_PLUGIN_TELEMETRY made the hook skip on its own, so the shell and PowerShell boundary tests still passed with the TELEMETRY_ENABLED gate open. Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM>
|
@cursor review — Strider 🐦⬛, Teal's agent · approved by Teal |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2a9265f. Configure here.
teallarson
left a comment
teallarson
left a comment
There was a problem hiding this comment.
Independent final review at 2a9265ff11f094bb506dc2276be2ab70fbf7e84f: ready for human review with telemetry OFF.
Independent shell/PowerShell checks: 3 pass, zero skips. Final inherited-opt-out test correction detects 10/10 disposable gate-open mutations; 280 mapping comparisons have zero differences. No remaining code finding.
Ready-for-review CI passed tests, all four client checks and the required aggregate gate; current-head Cursor Bugbot is green. Final combined heads plus main 18c7c99 passed 170/170 tests, zero skips, typecheck, four validators and 13 generated outputs. Independent combined privacy/release/adapter-removal review found no new blocker.
Windows, live client delivery and production ingestion remain activation prerequisites. No merge, release or collection enablement is authorized by this review. Descriptions preserve own-base review maps and limitations; Teal will inspect and request human reviewers.
— Strider 🐦⬛, Teal's agent · approved by Teal
teallarson
changed the title
2 participants
Linear ticket: GRO-305
What/why
Copilot hooks now exit quietly when VS Code supplies no valid plugin script path; Copilot CLI routing continues to work. The adapter prepares the same opportunity-versus-observed-invocation comparison for Copilot, using sessions because documented inputs lack prompt IDs. Session attribution is coarser than Claude’s turn attribution and cannot establish per-prompt or per-call pairing. Collection remains OFF.
See #26’s shared measurement diagram for local classification, privacy filtering and the current OFF gate, and #29’s reporting diagram for opportunity versus observed invocation and the client grouping limits.
Codebase changes
The Copilot adapter translates client lifecycle/tool inputs into the shared categorical event pipeline, declares its opt-outs and retains session-based scope. Generator source guards each shell/PowerShell hook with a valid script path; the shipped generated hook file changes alongside its source. Real-adapter fixtures cover mapping, scope, opt-outs and malformed input, while OFF-boundary tests use an event-worthy prompt, explicit loopback destination and transport positive controls. The boundary helper now clears inherited plugin/DNT/Copilot opt-outs so a changed build gate cannot be concealed by the test environment; runtime opt-out behavior is unchanged.
Review map uses
git diff --numstat foundation...head, excluding foundation changes; the copied enabled-hook manifest is a test fixture.Proof
2a9265ff18c7c99: 170/170 tests, zero skips, typecheck/four validators and generation checks pass in a space/percent checkoutAdditional notes
Merge #26 first, #27/#28 in either order, then #29. The three leaves currently target the foundation branch; after its squash merge, retarget/replay their own changes onto main and revalidate changed heads. Hold the generated release PR until the complete stack lands. Merging the release PR versions and tags the inactive build; it does not enable collection.
Activation requires a separate reviewed change and collection/privacy approval, verified Windows and live-client hook/opt-out delivery, and validated production ingestion and retention. Gateway telemetry remains canonical.
Copilot has no attempt hook, and its documented inputs do not provide prompt IDs;
turnis emitted only if a prompt ID is actually supplied, so reports use sessions and never infer per-call pairing. The enabled manifest is byte-identical to PR13, and recognized gateway suffixes remain heuristic observations rather than proven server identity. PowerShell was exercised on macOS and by Cursor on Linux; Windows, live Copilot delivery and VS Code agent sessions remain untested.Linear Issue: GRO-305