Move Semgrep triage from Claude Code to Pi

.github/workflows/semgrep-tests.yml

@@ -0,0 +1,29 @@
+name: Semgrep Tests
+
+on:
+  pull_request:
+    paths:
+      - .github/workflows/semgrep.yml
+      - .github/workflows/semgrep-tests.yml
+      - scripts/semgrep_triage.py
+      - scripts/test_semgrep_triage.py
+  push:
+    branches: [main]
+    paths:
+      - .github/workflows/semgrep.yml
+      - .github/workflows/semgrep-tests.yml
+      - scripts/semgrep_triage.py
+      - scripts/test_semgrep_triage.py
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - run: |
+          python3 -m venv /tmp/semgrep-tests
+          /tmp/semgrep-tests/bin/pip install PyYAML==6.0.3
+          /tmp/semgrep-tests/bin/python -m unittest discover -s scripts -p test_semgrep_triage.py

.github/workflows/semgrep-triage-prompt.md

@@ -1,32 +1,27 @@
 You are a security analyst triaging Semgrep static analysis findings for a pull request.
 
-The GitHub CLI is available as `gh` and authenticated via GH_TOKEN. You can comment on pull requests but must not create or edit PRs directly.
+You have read-only file tools and no shell or GitHub access. Do not post anything yourself:
+your final response is the result, and the workflow posts it to the PR.
 
 # Context
 
-- Repo: ${GITHUB_REPOSITORY}
+- Repo: ${GITHUB_REPOSITORY} (the PR checkout is the current directory)
 - Codebase: ${CODEBASE_DESCRIPTION}
 
 # Task
 
-1. Get the PR number and commit SHA:
-   ```
-   PR_NUMBER=$(gh pr view --json number --jq '.number')
-   COMMIT_SHA=$(git rev-parse HEAD)
-   ```
+1. Read semgrep-results.json
 
-2. Read semgrep-results.json
+2. For each finding, read the affected source file and surrounding code to understand context
 
-3. For each finding, read the affected source file and surrounding code to understand context
-
-4. Evaluate each finding:
+3. Evaluate each finding:
    - Is this a true positive with real security impact?
    - Is the vulnerable code path reachable from user input?
    - Are there existing sanitizations or mitigations nearby?
 
-5. If NO valuable findings: output "No actionable security findings." and exit without posting a comment
+4. If NO valuable findings: respond with `{"comments": [], "omitted": 0}`
 
-6. If valuable findings exist: post inline review comments on the PR
+5. If valuable findings exist: respond with inline review comments for them (see Output)
 
 # Triage Rules
 
@@ -43,20 +38,22 @@ Only report findings that are:
 - True positives in production code paths
 - Not already mitigated by upstream sanitization
 
-# Comment Format (only if valuable findings exist)
+# Output
 
-Post inline review comments on the exact lines where issues were found using the GitHub API:
+Respond with only this JSON object, with no other text:
 
 ```
-gh api repos/${GITHUB_REPOSITORY}/pulls/$PR_NUMBER/reviews \
-  --method POST \
-  -f commit_id=$COMMIT_SHA \
-  -f event=COMMENT \
-  -f 'comments[][path]=<file>' \
-  -F 'comments[][line]=<line>' \
-  -f 'comments[][body]=<message>'
+{
+  "comments": [
+    {"path": "<file>", "line": <line>, "body": "<message>"}
+  ],
+  "omitted": <number of actionable findings left out>
+}
 ```
 
+`path` is the file path as it appears in semgrep-results.json, and `line` is the flagged line
+in the PR version of the file. List comments highest severity first.
+
 Each inline comment body should include:
 1. What the issue is and why it is a true positive
 2. Recommended fix with a code snippet if applicable
@@ -67,28 +64,31 @@ Each inline comment body should include:
    - Add path to .semgrepignore to exclude the file entirely
    - Both options should be mentioned with the exact rule ID from the finding
 
-# Deduplication (strictly enforced)
+# Commentable Lines
 
-Before posting anything, fetch ALL existing inline review comments on this PR:
+Review comments can only be placed on lines inside the PR diff. ${TRIAGE_DIR}/commentable-lines.json
+maps each changed file to the `[first, last]` line ranges of its diff hunks. Do not comment on a
+line outside those ranges.
 
-```
-gh api repos/${GITHUB_REPOSITORY}/pulls/$PR_NUMBER/comments
-```
+# Deduplication (strictly enforced)
+
+${TRIAGE_DIR}/existing-comments.json lists ALL existing inline review comments on this PR, with
+their file, line, original line, and author.
 
 For each Semgrep finding, check whether ANY existing comment (from any author --
 github-actions[bot], cursor[bot], vercel[bot], socket-security[bot], or any human)
 already covers the same file and line. If a comment already exists on that line,
-skip it entirely -- do NOT post a duplicate regardless of whether the existing
+skip it entirely -- do NOT include a duplicate regardless of whether the existing
 comment is from us or another bot.
 
-Only post a comment for a finding if that exact file+line has zero existing comments.
+Only include a comment for a finding if that exact file+line has zero existing comments.
 
 # Constraints
 
-- Do NOT post any comments if there are no actionable findings
-- Do NOT post if an existing comment already covers the finding (any author)
-- Post inline review comments, not top-level PR comments
-- Post all new findings in a single review API call (batch the comments[] array)
-- Post at most 5 inline comments -- if more than 5 actionable findings remain after
-  triage, post only the 5 highest severity ones and note how many were omitted
-- Do not create or edit PRs directly
+- Do NOT include any comments if there are no actionable findings
+- Do NOT include a comment if an existing comment already covers the finding (any author)
+- Comments are inline review comments on the flagged line, never top-level PR comments
+- All new findings are posted in a single review
+- Include at most 5 comments -- if more than 5 actionable findings remain after
+  triage, include only the 5 highest severity ones and set `omitted` to how many were left out
+- Do not create or edit files

.github/workflows/semgrep.yml

@@ -91,24 +91,77 @@ jobs:
     timeout-minutes: 15
     needs: scan
     if: needs.scan.outputs.findings_count != '0' && fromJson(needs.scan.outputs.findings_count) <= 20
+    # The agent gets read-only file tools and no GitHub token. Runner steps gather the PR's
+    # existing comments before it runs and post its validated result as one review after.
+    env:
+      REPO: ${{ github.repository }}
+      NUMBER: ${{ github.event.pull_request.number }}
+      HEAD_SHA: ${{ github.event.pull_request.head.sha }}
     steps:
+      # Semgrep scanned the merge commit, but review comments go on the PR head. Depth 2 fetches
+      # both, so the findings can be moved onto the head before the agent reads it.
       - name: Checkout repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+          persist-credentials: false
 
       - name: Download results
         uses: actions/download-artifact@v4
         with:
           name: semgrep-results
 
-      - name: Install Claude Code CLI
+      - name: Fetch triage prompt and script
+        run: |
+          echo "TRIAGE_DIR=$RUNNER_TEMP/semgrep-triage" >> "$GITHUB_ENV"
+          base=https://raw.githubusercontent.com/kernel/security-workflows/main
+          curl -fsSL "$base/.github/workflows/semgrep-triage-prompt.md" -o "$RUNNER_TEMP/semgrep-triage-prompt.md"
+          curl -fsSL "$base/scripts/semgrep_triage.py" -o "$RUNNER_TEMP/semgrep_triage.py"
+
+      - name: Gather PR comments
+        env:
+          GH_TOKEN: ${{ github.token }}
         run: |
-          curl -fsSL https://claude.ai/install.sh | bash
-          echo "$HOME/.local/bin" >> $GITHUB_PATH
+          python3 "$RUNNER_TEMP/semgrep_triage.py" gather --repo "$REPO" --number "$NUMBER" \
+            --head "$HEAD_SHA" --dir "$TRIAGE_DIR" --results semgrep-results.json
+          git checkout -q --detach "$HEAD_SHA"
+
+      - name: Setup Node
+        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version: '22'
+
+      - name: Install Pi
+        env:
+          PI_TELEMETRY: '0'
+        run: |
+          npm install -g --ignore-scripts @earendil-works/pi-coding-agent
+          pi --version
 
       - name: Triage findings
         env:
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+          PI_PREFERRED_MODEL: ${{ vars.PI_PREFERRED_MODEL }}
+          PI_SKIP_VERSION_CHECK: '1'
+          PI_TELEMETRY: '0'
+          CODEBASE_DESCRIPTION: ${{ inputs.codebase-description }}
+        run: |
+          : "${PI_PREFERRED_MODEL:?Set the PI_PREFERRED_MODEL organization variable}"
+          # shellcheck disable=SC2016 # envsubst takes the variable names literally
+          envsubst '${GITHUB_REPOSITORY} ${CODEBASE_DESCRIPTION} ${TRIAGE_DIR}' \
+            < "$RUNNER_TEMP/semgrep-triage-prompt.md" > "$TRIAGE_DIR/prompt.md"
+          # The checkout is PR content: --no-approve keeps its .pi resources from loading and
+          # --no-context-files keeps its AGENTS.md and CLAUDE.md out of the instructions.
+          pi --print --no-session --no-approve --no-context-files \
+            --tools read,grep,find,ls \
+            --model "$PI_PREFERRED_MODEL" --thinking max \
+            @"$TRIAGE_DIR/prompt.md" > "$TRIAGE_DIR/agent-output.txt"
+          cat "$TRIAGE_DIR/agent-output.txt"
+
+      - name: Post review
+        env:
+          GH_TOKEN: ${{ github.token }}
         run: |
-          export CODEBASE_DESCRIPTION="${{ inputs.codebase-description }}"
-          curl -fsSL https://raw.githubusercontent.com/kernel/security-workflows/main/.github/workflows/semgrep-triage-prompt.md | envsubst '${GITHUB_REPOSITORY} ${CODEBASE_DESCRIPTION}' | claude -p --model ${{ vars.CLAUDE_CODE_PREFERRED_MODEL }} --dangerously-skip-permissions --output-format text --verbose
+          python3 "$RUNNER_TEMP/semgrep_triage.py" post --repo "$REPO" --number "$NUMBER" \
+            --head "$HEAD_SHA" --dir "$TRIAGE_DIR" --results semgrep-results.json

README.md

@@ -131,7 +131,8 @@ projectIgnorePaths:
 
 Consumer repos need these secrets (set at org or repo level):
 
-- `ANTHROPIC_API_KEY` — for the `semgrep.yml` triage agent and the `pr-risk.yml` classifier (Claude Code)
+- `ANTHROPIC_API_KEY` — for the `semgrep.yml` triage agent (Pi) and the `pr-risk.yml` classifier (Claude Code)
+- `OPENAI_API_KEY` — for the `semgrep.yml` triage agent (Pi), when `PI_PREFERRED_MODEL` is an OpenAI model
 - `CURSOR_API_KEY` — for the other fix/remediation agents (Cursor)
 - `ADMIN_APP_ID` + `ADMIN_APP_PRIVATE_KEY` — GitHub App for write access
 - `SOCKET_API_TOKEN` — Socket.dev API token
@@ -140,5 +141,6 @@ Consumer repos need these secrets (set at org or repo level):
 
 Consumer repos need these variables (set at org or repo level):
 
-- `CLAUDE_CODE_PREFERRED_MODEL` — model for the `semgrep.yml` triage agent and the `pr-risk.yml` classifier (Claude Code)
+- `PI_PREFERRED_MODEL` — model for the `semgrep.yml` triage agent (Pi)
+- `CLAUDE_CODE_PREFERRED_MODEL` — model for the `pr-risk.yml` classifier (Claude Code)
 - `CURSOR_PREFERRED_MODEL` — model for the other Cursor agent invocations