Skip to content

Run vulnerability remediation fallback agent with Pi and no write credentials - #31

Merged
rgarcia merged 3 commits into
mainfrom
hypeship/vuln-remediation-pi
Nov 9, 2026
Merged

rgarcia merged 3 commits into
mainfrom
hypeship/vuln-remediation-pi

Conversation

@rgarcia

@rgarcia rgarcia commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Move the fallback agent in the reusable vuln-remediation.yml workflow from Cursor CLI to Pi, and run it in a job that references no secrets except the model API keys.

Before this change, the fix job minted an unscoped GitHub App token, persisted it in the checkout, and wrote it into a global git config url.<token>.insteadOf before the agent ran. The agent also got SOCKET_SECURITY_API_KEY. In practice the agent could push to any branch of the caller repo.

Revoking a minted token before the agent runs is not enough. Every secret a job references is delivered to the runner at job start and stays in Runner.Worker memory for the whole job. Hosted runners have passwordless sudo, so an agent in that job could read the app private key from memory and mint its own token. The job that runs Pi therefore references only ANTHROPIC_API_KEY and OPENAI_API_KEY.

Jobs

scan and triage are unchanged apart from SHA pinning. After them:

Job Secrets What it does
fix ADMIN_APP_* (read token), SOCKET_API_TOKEN Runs bun install and socket fix with a read-only token scoped to the caller repo. The token is passed through per-step GIT_CONFIG_* env vars and never written to global git config. Uploads Socket's diff as a patch. If Socket left no diff, it runs go mod download for the Go modules named in the fixes and uploads only $GOMODCACHE/cache/download/github.com/kernel. Never runs the agent.
agent ANTHROPIC_API_KEY, OPENAI_API_KEY Runs only when Socket left no diff. Checks out the recorded base SHA and restores the private-module subtree. Disables the setup-go/setup-bun caches so the agent can't poison caches that other workflows restore. Runs Pi, then uploads tracked changes, committed or not, against the base as a patch.
verify SOCKET_API_TOKEN Runs on a fresh runner. Applies the Socket or agent patch to the base SHA and checks it against its file list. Then runs validate-diff, the post-fix Socket plan, confirm, and summarize-fix.
pr ADMIN_APP_* Re-applies the patch on a fresh checkout and checks the file list. Re-runs validate-diff (allowlist) and rejects anything other than regular non-executable files. Commits, then mints a contents/pull-requests write token. Pushes with --force-with-lease against the branch SHA recorded before the agent ran, and creates or updates the PR as before. Never runs repo code, package managers, Socket, or Pi.

The workflow sets permissions: contents: read, so the caller's GITHUB_TOKEN is read-only in every job. All actions are pinned to SHAs.

Agent invocation

  • agent -p --model $CURSOR_PREFERRED_MODEL ... becomes pi --print --no-session --approve --model "$PI_PREFERRED_MODEL" --thinking max @prompt. This is the same setup as the other Pi workflows: Node 22, npm install -g --ignore-scripts @earendil-works/pi-coding-agent, PI_SKIP_VERSION_CHECK=1, and PI_TELEMETRY=0.
  • The prompt is still fetched from .github/workflows/vuln-remediation/fix-prompt.md at security-workflows-ref, rendered with envsubst into $RUNNER_TEMP, and passed with @file.
  • The Cursor install and Max Mode steps are removed. No CURSOR_* reference is left in this workflow.
  • The agent step timeout is 20 minutes. The old 10-minute step covered both Socket fix and the agent.
  • Prompt changes: the credentials sentence now says the agent has no GitHub credentials and explains what gets published. It also says private github.com/kernel/* modules not already in the module cache can't be downloaded. triage-prompt.md is not used by this workflow and is untouched.

Callers

All 21 caller repos use secrets: inherit, and none has private git dependencies in package.json. No caller changes are needed.

Validation

  • actionlint passes on both changed workflows. With shellcheck enabled, the only finding is a pre-existing SC2086 in the untouched scan job.
  • python3 -m unittest scripts.test_vuln_remediation: 17 tests pass.
  • New scripts/test_vuln_remediation_workflow.py (17 tests) runs in the self-test tests job.
    • Workflow checks: the job that runs pi is the only agent job, and its secrets.* set is exactly the two model keys. No other job runs Pi. fix, verify, and pr each reference only the secrets listed in the table above. Actions are pinned, checkouts don't persist credentials, agent-job caches are off, and the publish job mints its token last.
    • The real snapshot steps from fix and agent are executed against temp git repos.
    • The real verify and pr gate scripts are executed on fresh clones. They reject source edits (validate-diff allowlist exit) and executable files. The publish mode check rejects a symlinked lockfile that validate-diff lets through. Both reject a patch that doesn't match its file list, and a valid lockfile change is committed on security/vuln-remediation.
  • Mutation checks: each of these makes the suite fail:
    • putting SOCKET_API_TOKEN or ADMIN_APP_PRIVATE_KEY back into the Pi job;
    • adding an app-token mint to the Pi job;
    • re-enabling the Go cache there;
    • dropping the soft reset to base;
    • dropping the publish file-list check, mode check, or allowlist;
    • removing the read token's read-only scope.
  • Go module handoff: I ran the real fix download step on a fixture that requires github.com/kernel/hypeman-go, then the real agent restore step into an empty module cache. go build succeeded with all github.com git access redirected to an invalid host. Without the restored subtree, the same build fails.
  • Local Pi smoke run with the exact agent step script, openai/gpt-5.6-sol, and a fresh Pi config dir, against an npm fixture with minimist@1.2.5. Pi bumped it to 1.2.6 in about 40 seconds without committing or staging. The snapshot, validate-diff, confirm, and publish apply/commit steps then passed on its output.
  • Live workflow_dispatch of vuln-remediation-self-test.yml with run-live-remediation=true on the latest commit (run 42125023082) succeeded. scan, triage, and fix passed. Socket found no fixable alerts in this repo, so agent, verify, and pr were skipped. Pi has not run on GitHub Actions yet. The first scheduled caller run where Socket can't fix an alert will be the first live Pi run.

Risks and not verified

  • I couldn't check org secret/variable visibility for all 21 callers with my permissions. kernel/kernel-go-sdk, one of the callers, already runs Pi with both keys and PI_PREFERRED_MODEL set.
  • The read token keeps its original scope, the caller repo only. Private modules in other kernel repos (for example kernel-images-private, which kernel/kernel requires) still can't be downloaded. That was already true before this change. Widening it to owner: kernel with contents read would make them work and would still keep the token out of the agent job.
  • The post-fix Socket plan in verify runs without a GitHub token, the same as the triage plan. If it fails, confirmation fails closed and no PR is opened.
  • The agent job still holds the runner's ACTIONS_RUNTIME_TOKEN (artifacts and cache) and a read-only GITHUB_TOKEN, as every job does. It could upload a stray artifact, which makes verify's upload fail closed. It could also write Actions cache entries directly; disabling the setup-action caches removes only the default path for that.

Follow-ups

  • The README "Required secrets/variables" section still lists CURSOR_API_KEY / CURSOR_PREFERRED_MODEL. I left it alone because other workflow migrations in this repo touch the same lines.

@rgarcia

rgarcia commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Live workflow_dispatch of vuln-remediation-self-test.yml on this branch with run-live-remediation=true: run 42112262317 succeeded. scan, triage, and fix passed. Socket found no fixable alerts in this repo, so fix skipped its remediation steps and pr was skipped. Pi did not run in this live run; the Pi path was exercised only by the local smoke run described in the PR body.

@rgarcia

rgarcia commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Picked ulziibay-kernel to review: touched the changed code in workflows; 24 pending

@rgarcia

rgarcia commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Pushed 5e4d780: the Pi job now references only the model keys.

Revoking the minted token before Pi ran wasn't enough. Every secret a job references sits in Runner.Worker memory for the whole job, and the old fix job referenced ADMIN_APP_* and SOCKET_API_TOKEN alongside Pi. The old fix job is now four jobs:

  • fix (ADMIN_APP_* read token, SOCKET_API_TOKEN): bun install and socket fix. Uploads Socket's patch. If Socket left no diff, it uploads only $GOMODCACHE/cache/download/github.com/kernel for the Go modules named in the fixes. No agent.
  • agent (ANTHROPIC_API_KEY, OPENAI_API_KEY only): base-SHA checkout, restored module subtree, setup-go/setup-bun caches off, Pi, then uploads the patch.
  • verify (SOCKET_API_TOKEN): fresh runner. Applies the patch and runs validate-diff, the post-fix Socket plan, and confirm.
  • pr (ADMIN_APP_*): fresh runner. Re-applies the patch, runs the allowlist and file-mode gate, commits, then mints the write token and pushes. No code execution.

Tests (scripts/test_vuln_remediation_workflow.py, 17 tests):

  • The Pi job's secrets.* set must be exactly the two model keys. Putting SOCKET_API_TOKEN or ADMIN_APP_PRIVATE_KEY back makes the suite fail; I checked by editing the workflow, and a test also does this in memory.
  • The real verify and pr gate scripts run on fresh clones. They reject source edits through the validate-diff allowlist exit and reject executable files. The publish mode check catches a symlinked lockfile that validate-diff passes.
  • I also checked the Go subtree handoff end to end: build succeeds offline with the subtree and fails without it.

Live self-test on 5e4d780 (run 42125023082) succeeded. Nothing was fixable, so agent/verify/pr were skipped. The PR description has the full details.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5e4d780. Configure here.

Comment thread .github/workflows/vuln-remediation.yml Resolved
@rgarcia
rgarcia merged commit 8073637 into main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant