Skip to content

Add beam feedback - #102

Open
mernit wants to merge 2 commits into
masterfrom
eli/agent-feedback
Open

mernit wants to merge 2 commits into
masterfrom
eli/agent-feedback

Conversation

@mernit

@mernit mernit commented Oct 31, 2026 •

Copy link
Copy Markdown

Coding agents, and people, can now tell us when Beam blocks or misleads them:

beam feedback -m "deploy ignored --port" --kind bug --blocked \
  --command "beam deploy app.py:handler --port 8080" --error "port 8000 is not exposed"

The command posts to the selected context's account API at {auth_url}/cli/feedback. That's https://api.beam.cloud/v2/oauth/cli/feedback in production, and staging or local with --context. It sends the workspace token, as beam setup does for credit checkout. internal-api (beam-cloud/internal-api#343) turns each report into a PostHog event. Nothing here or in beta9 stores feedback.

  • Options: -m/--message is required, and is prompted for in a terminal. The rest are --kind (bug, error_message, missing_capability, docs, performance, praise, other), --blocked, --command, --error, --details, --suggest, --model, --format json and --context.
  • Attribution: the request carries beta9's caller headers (x-beta9-caller, x-beta9-agent-session), so reports from one agent run group together. Once beam-cloud/beta9#2087 ships, the caller also names the agent, e.g. cli/0.1.293 (claude-code).
  • Reply: any 2xx prints "Feedback sent. Thanks.", or {"sent": true} with --format json.
  • Failures carry codes agents can branch on with BETA9_JSON=1:
    • INPUT_REQUIRED
    • INVALID_ARGS, which names the flag to fix, e.g. --message: At most 300 characters; put the rest in details.
    • NOT_AUTHENTICATED and UNAVAILABLE
  • Opt-out: BEAM_NO_FEEDBACK=1 makes the command refuse with FEEDBACK_DISABLED.
  • Agent skill: feedback.SKILL_SECTION tells agents when to report (Beam's fault, not the user's), what to include, and to carry on afterwards.
    • beam setup agent appends it to the use-beam skill through SDKSettings.skill_sections (beam-cloud/beta9#2092).
    • It's assigned after the settings are built, so beta9 releases without that field just ignore it. It takes effect once beam-client's beta9 dependency includes #2092.

Release order

Release this after internal-api#343 is deployed; until then the endpoint returns 404. There's no version bump here.

Testing

tests/cli/test_feedback.py has 18 tests, passing with beta9 0.1.291 (the locked version). They cover:

  • the request URL, headers and body
  • text and JSON output
  • the opt-out and a missing summary
  • validation, auth, server and network failures
  • contexts without an account API
  • the skill section and its wiring
  • beam feedback --help through the real CLI

With #2092's beta9 on PYTHONPATH, the use-beam skill installed with these settings ends with the rendered Feedback section.

I also ran it end to end against internal-api#343 running locally on port 8008, using --context local:

  • The report arrived as one feedback.submitted event with every field, plus the caller and session headers.
  • --format json printed {"sent": true}.
  • A 301-character summary failed, naming --message.

mernit and others added 2 commits October 31, 2026 12:52
Coding agents (and people) can report what blocked or misled them:

    beam feedback -m "deploy ignored --port" --kind bug --blocked

The report goes to the account API of the selected context
({auth_url}/cli/feedback, as `beam setup` does for checkout) with the
workspace token and the CLI's caller and agent-session headers. Failures
carry codes agents can branch on: INPUT_REQUIRED, INVALID_ARGS (naming the
flag to fix), NOT_AUTHENTICATED, RATE_LIMITED, UNAVAILABLE and
FEEDBACK_DISABLED. BEAM_NO_FEEDBACK=1 turns the command off.

Co-authored-by: Cursor <cursoragent@cursor.com>
beam setup agent appends a Feedback section to the use-beam skill once beam-client depends on a beta9 with SDKSettings.skill_sections (beam-cloud/beta9#2092). Older beta9 releases ignore it.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mernit
mernit force-pushed the eli/agent-feedback branch from c93a100 to 8561001 Compare October 31, 2026 16:52
@mernit
mernit requested a review from luke-lombardi November 1, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant