Skip to content

feat(feedback): add volcano feedback command for agent-reported issues - #319

Open
tkkhq wants to merge 5 commits into
feat/agent-detectionfrom
feat/agent-feedback
Open

tkkhq wants to merge 5 commits into
feat/agent-detectionfrom
feat/agent-feedback

Conversation

@tkkhq

@tkkhq tkkhq commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Tracking

Why

  • agents hit Volcano problems (CLI bugs, skills that disagree with the CLI, missing commands) and work around them silently; telemetry shows that a command failed, but not what the agent expected, what it tried, or how it got around it

Summary

  • add volcano feedback with --category (cli_bug, docs_mismatch, missing_capability, platform_error, confusing_ux, instruction_conflict), --summary, --details, repeatable --command, --expected, --actual, --workaround, and --dry-run; required fields and length limits are checked on the text as written, so an agent can shorten and rerun
  • add --input: the report as one JSON object with keys category, summary, details, commands, expected, actual, workaround, given inline (a value starting with {), as a file path, or as - for stdin; unknown keys and trailing data are errors, and --input cannot be combined with the field flags; validation messages name the field without a -- prefix so they read right in both modes
  • add internal/feedback: the POST /cli/feedback wire contract (Report), validation, and a synchronous send with a 10s timeout and the saved login token when present; the CLI generates feedback_id, so it prints the ID without reading the response
  • attach the machine and agent fields a usage event carries (device ID, CLI version, OS, arch, agent, agent source, plugin installed and version, project ID) through a new telemetry.Recorder.Client(); the server joins a report to the same device's usage events around its timestamp to find the failing command, so the CLI keeps no run history
  • feedback follows the telemetry opt-out: when telemetry is off, or before the first-run notice, the command prints Feedback not sent and exits 0 so an agent does not retry
  • reuse main's recorder through feedbackcmd.SetRecorder, so the process still reads the telemetry setting once
  • add telemetry.ScrubText: the error-message scrubbing (home dir to ~, ANSI and control characters removed, Unicode spaces normalized, secrets redacted) applied per line, keeping line breaks; every text field goes through it
  • always post to the cloud API URL (config.Load), never a local-mode server
  • name feedback in the first-run telemetry notice, volcano telemetry --help, and docs/telemetry.md, since turning telemetry off also turns feedback off
  • add docs/feedback.md and link it from docs/README.md
  • telemetry package changes are additive except the first-run notice text; existing telemetry tests are unchanged
  • defer rate caps and dedupe
  • needs a POST /cli/feedback endpoint in volcano-hosting, not built yet; until it ships the send returns HTTP 404, the command exits 1, and the skill tells agents to retry once and drop it
  • merge order: Kong/volcano-hosting#1919, then feat: CLI usage telemetry with coding agent detection #318, then a POST /cli/feedback endpoint in volcano-hosting, then feat(feedback): add volcano feedback command for agent-reported issues #319 in a release, then feat(feedback): add volcano-feedback skill for reporting Volcano problems volcano-skills#63; the skills sync to the plugin as soon as they merge, so merging chore(main): release 0.1.2 #63 before the endpoint is live would send agents into a 404 and a retry for every note
  • free-text reports add to the GDPR/ePrivacy questions feat: CLI usage telemetry with coding agent detection #318 lists for legal: an agent can paste text that redaction does not catch, so the skill forbids file contents, env values, app data, and customer names

Verification

  • make lint
  • go test ./...
  • ran the built binary with a fresh HOME against a local sink: the first run printed the telemetry notice and Feedback not sent (exit 0) and posted nothing; the next run posted one report to /cli/feedback with the device ID matching the telemetry event, the home dir replaced with ~, and token=abc123 redacted; --dry-run printed the report and posted nothing; after volcano telemetry disable it printed Feedback not sent (exit 0); a bad category and missing --details exited 1 with both errors
  • ran the built binary with --input - and a heredoc JSON report against the local sink: it posted the report with line breaks kept and token=abc redacted; --input - with --summary exited 1
  • ran the built binary with --input given inline, as a file, and on stdin with --dry-run: each printed the report; the file form kept a single quote in the error text

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants