Skip to content

Run the PR risk classifier on federated model credentials - #36

Open
rgarcia wants to merge 2 commits into
mainfrom
hypeship/pr-risk-workload-identity
Open

rgarcia wants to merge 2 commits into
mainfrom
hypeship/pr-risk-workload-identity

Conversation

@rgarcia

@rgarcia rgarcia commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The reusable PR risk workflow no longer needs ANTHROPIC_API_KEY or OPENAI_API_KEY. The classify job exchanges its GitHub OIDC token for short-lived provider credentials through workload identity federation, so no long-lived model key is stored in or passed between repos.

  • New composite action, kernel/security-workflows/.github/actions/llm-credentials. It takes model (provider/id) and the six federation IDs. It fetches credentials once, then refreshes them every minute in the background for the rest of the job. GitHub OIDC tokens last about five minutes, and each provider credential expires no later than the token it was exchanged for.
    • anthropic: writes a fresh GitHub token to a file and exports ANTHROPIC_FEDERATION_RULE_ID, ANTHROPIC_ORGANIZATION_ID, ANTHROPIC_SERVICE_ACCOUNT_ID and ANTHROPIC_IDENTITY_TOKEN_FILE to later steps. The Anthropic SDK inside pi does the exchange itself and rereads the file before each refresh.
    • openai: exchanges the token at auth.openai.com/oauth/token and writes the access token to pi's auth.json, which pi rereads whenever the file changes.
    • A failed refresh never overwrites a working credential.
    • Other workflows that run pi (semgrep, container-remediation, and pi workflows in other repos) can adopt it with one step. The README has a usage example.
  • pr-risk.yml:
    • The classify job now requests id-token: write.
    • A new "Model credentials" step uses the action, passing the federation IDs from org variables. These are not secrets.
    • The Classify step no longer receives API keys.
  • The two key secrets are still declared but ignored, so existing callers that still pass them keep working.
  • README and tests updated. The new test_llm_credentials.py runs in PR Risk Tests.

Notes for review

  • The action can't read vars itself. GitHub rejects ${{ vars.* }} in action metadata (Unrecognized named-value: 'vars'), so callers pass the six IDs as inputs. Callers of pr-risk pass nothing; pr-risk.yml passes them once. I didn't hardcode the IDs because this repo is public, and they would then live in two places.
  • pr-risk uses the action from @main. uses: can't take an expression, so the action doesn't follow the security-workflows-ref input. The classifier script and prompt still do. Within this PR that's fine, because the action and the change to pr-risk.yml merge together.

Merge order

A reusable workflow job can't request more permissions than its caller grants. If this merges first, pr-risk fails to start in every caller. Each caller needs id-token: write first:

Callers can keep passing the old secrets until they're cleaned up.

Testing

  • unittest for test_pr_risk.py (18) and test_llm_credentials.py (7) pass. The new tests run the action's own shell code against a fake credential script. shellcheck and actionlint are clean.
  • Live, full pr-risk run with no API key passed: this code ran as a reusable workflow on a throwaway PR, with only the action ref pointed at the test branch. The classifier posted its verdict, and every step succeeded.
  • Live action check from another workflow: the action, then a 400-second sleep in a later step, then a pi call with no key in its env. Both openai/gpt-6-luna and anthropic/claude-haiku-5-5 answered after the original credentials had expired. The credential files had been rewritten about six minutes after start, and the Anthropic settings reached the later step.

Reviewed with sol-loop: 2 rounds with openai-codex/gpt-6.1-sol on extra-high reasoning. Round 1, before the action, was approved. Round 2 found one issue in the README example: the pi step didn't set PI_PREFERRED_MODEL. I fixed it with a README-only change, which wasn't re-reviewed.

@rgarcia

rgarcia commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Picked tnsardesai to review these related PRs (#36, kernel/kernel#5359, kernel/hypeman#550, kernel/infra#2088, kernel/kernel-images-private#584): touched the changed code in workflows; 4 pending

@rgarcia
rgarcia requested a review from JonnyShuali November 9, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant