Skip to content

fix: share one exchanged WIF credential across spawned Claude processes#1407

Open
KeisukeYamashita wants to merge 2 commits into
anthropics:mainfrom
KeisukeYamashita:fix/wif-shared-credentials-cache
Open

fix: share one exchanged WIF credential across spawned Claude processes#1407
KeisukeYamashita wants to merge 2 commits into
anthropics:mainfrom
KeisukeYamashita:fix/wif-shared-credentials-cache

Conversation

@KeisukeYamashita

Copy link
Copy Markdown

Fixes #1406

Problem

GitHub OIDC tokens are single-use at the Anthropic token-exchange endpoint (the same jti cannot be exchanged twice). When the plugins / plugin_marketplaces inputs are configured, the action spawns several short-lived claude processes per job (claude plugin marketplace add, one claude plugin install per plugin, then the main query). Each process resolved workload identity federation from the bare env vars and exchanged the same identity-token file independently: the first exchange succeeded and every later process got 401 (jti_reused), which the main query retried for ~3 minutes before failing the job. Details, eBPF process traces, and request-ids in #1406.

Fix

The SDK only enables its on-disk credentials cache (<config_dir>/credentials/<profile>.json, shared across processes) when federation is loaded from a profile config file, not from bare env vars. setupWorkloadIdentity() now additionally writes a profile pointing at the identity-token file and selects it via ANTHROPIC_CONFIG_DIR + ANTHROPIC_PROFILE, so the first process exchanges once and every other process reuses the cached access token.

  • The profile lives in an action-owned dir under RUNNER_TEMP (0700 dir / 0600 file), next to the identity token.
  • ANTHROPIC_IDENTITY_TOKEN_FILE and the federation env vars are kept as a fallback for CLIs that predate profile support — older CLIs degrade to today's behavior, never worse.
  • The existing 4-minute identity-token refresh is unchanged; after the cached access token expires, the next exchange picks up the refreshed (new-jti) token from the same file path recorded in the profile.

Verification

  • bun test: 702/702 pass (includes 2 new unit tests for the profile file), bun run typecheck, bun run format:check all green.
  • Local reproduction against a mock exchange endpoint that enforces single-use jti: two sequential claude processes sharing one identity token — env-var path: second process retries jti_reused for ~3 minutes and dies (matches production failures exactly); profile path: second process reuses the cached credential with zero additional exchanges and succeeds. Verified with both Claude Code 2.1.167 and 2.1.173.
  • On a GitHub-hosted runner with a real federation rule: the exact configuration that previously failed on 100% of runs (two trailofbits plugins + WIF, on both v1.0.139 and v1.0.144) succeeds with this branch — is_error: false, main query completes in ~6s. An eBPF trace shows the same five claude processes contacting api.anthropic.com, now with a single token exchange.

Notes / limitations

  • A cold-start race (two processes exchanging before the first cache write) is still theoretically possible, but the action spawns its subprocesses sequentially, so in practice the first one exchanges and the rest reuse. A complete fix (serializing the exchange / locking the credentials file) would belong in the CLI/SDK.
  • If a user has pre-set ANTHROPIC_CONFIG_DIR / ANTHROPIC_PROFILE, they are overridden while federation inputs are configured (federation inputs are an explicit opt-in, and previously the env-var path was similarly authoritative).

GitHub OIDC tokens are single-use at the Anthropic token-exchange
endpoint (the same jti cannot be exchanged twice). With plugins
configured, the action spawns several short-lived claude processes
(plugin marketplace add, one plugin install per plugin, then the main
query). Each resolved federation from bare env vars and exchanged the
same identity-token file independently: the first exchange succeeded
and every later process got 401 (jti_reused), which the main query
retried for ~3 minutes before failing the job.

The SDK only enables its on-disk credentials cache when federation is
loaded from a profile config file, not from bare env vars. Write a
profile pointing at the identity-token file and select it via
ANTHROPIC_CONFIG_DIR / ANTHROPIC_PROFILE so the first process exchanges
once and the rest reuse the cached access token. The env vars are kept
as a fallback for CLIs that predate profile support.

@ashwin-ant ashwin-ant left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find, and the diagnosis holds up. buildProvider in @anthropic-ai/sdk/lib/credentials/credential-chain.mjs only wraps the exchange in cachedExchangeProvider when a credentials_path is set, and that only happens for file-backed configs. So env vars can't share a token across processes and a profile can. Your profile also matches AnthropicConfig exactly.

Two changes before merge.

The cache path needs a config discriminator. configDir is a constant under RUNNER_TEMP, which is per-job, but the federation rule / org / workspace inputs are per-step. cachedExchangeProvider reuses a cached token based on expires_at alone, with no fingerprint of the config that minted it, and the workspace is bound at mint time. So two steps in one job with different workspaces will silently reuse the first step's token: wrong scope, wrong billing, no error. Today's uncached path exchanges fresh each step and gets this right, so it's a regression. The SDK comment right above the cache says this is exactly why env-only configs skip it. Hashing federation_rule_id + organization_id + workspace_id + base_url into the config dir name fixes it.

Don't clobber an operator-set ANTHROPIC_PROFILE / ANTHROPIC_CONFIG_DIR. You flag this under limitations, but it's free to fix rather than a tradeoff: if ANTHROPIC_PROFILE is already set, the disk cache is already on, so just core.warning() and skip writeFederationProfile. Same shape as the ANTHROPIC_API_KEY guard above it. Worth a test.

Smaller stuff:

  • workspace_id, base_url, and service_account_id look redundant. loadConfig gap-fills them from env for file-backed profiles, and the resolved base URL prefers the env var anyway. { version, authentication: { type: "oidc_federation" } } should do it.
  • stop() could rmSync the token dir, so the exchanged token doesn't outlive the step.
  • Worth a docstring line that this depends on the plugin subprocesses spawning sequentially. cachedExchangeProvider isn't cross-process serialized, and its "exchanges are idempotent" assumption is what single-use jti breaks. Parallelize that loop later and the bug comes back.

One question. Single-use jti is server side and I can't confirm it from the client. Your mock assumes it and the Console events point that way, but let me check with the token exchange owners before we call this a correctness fix. If reuse is allowed it's still worth landing, just as a latency fix, and the description should say so.

Address review feedback on the shared-credentials-cache fix:

- Embed a fingerprint of the federation inputs (rule, org, service
  account, workspace, base URL, scope) in the config dir name. The SDK
  cache reuses a token on expires_at alone and RUNNER_TEMP is per-job,
  so a later step with different federation inputs would silently reuse
  the first step's token. service_account_id and scope are included
  beyond the reviewed list because both are sent in the exchange
  request body and change which credential is minted.
- Skip the action-managed profile with a warning when the operator has
  already set ANTHROPIC_CONFIG_DIR or ANTHROPIC_PROFILE.
- Shrink the profile to the minimal file-backed form; the CLI's bundled
  SDK gap-fills the federation fields from the env vars the action
  already exports (verified against the pinned 2.1.173 binary).
- Remove the token dir in stop() so the identity token and the cached
  exchanged credential don't outlive the step.
- Document that cache sharing relies on the plugin subprocesses
  spawning sequentially.
@KeisukeYamashita

Copy link
Copy Markdown
Author

@ashwin-ant Hi, thank you for your kind review.

All four changes are in: 34d3228

Config discriminator. The config dir is now config-<sha256[:16]>, derived from the federation inputs. I added two fields beyond the ones you listed: service_account_id and scope. Both are sent in the exchange request body (oidc-federation.mjs builds { grant_type, assertion, federation_rule_id, organization_id, service_account_id }, and loadConfig gap-fills authentication.scope), so two steps differing only in service account would hit the same silent-reuse bug as the workspace case does. anthropic_oidc_audience is deliberately excluded—it changes which JWT is minted, not the scope of the exchanged token.

Operator-set ANTHROPIC_PROFILE / ANTHROPIC_CONFIG_DIR. If either is set, the action now warns and skips writeFederationProfile entirely, following the same pattern as the ANTHROPIC_API_KEY guard. ANTHROPIC_IDENTITY_TOKEN_FILE is still exported so the operator’s profile—or the environment fallback—can consume the token file. Tests were added for both environment variables.

Minimal profile. It is now exactly:

{
  "version": "1.0",
  "authentication": {
    "type": "oidc_federation"
  }
}

Before shrinking it, I verified that the gap-fill exists in the CLI version pinned by this action (2.1.173), not just in a current SDK. The bundled loadConfig in the @anthropic-ai/claude-code-darwin-arm64@2.1.173 binary references ANTHROPIC_ORGANIZATION_ID, ANTHROPIC_IDENTITY_TOKEN_FILE, ANTHROPIC_FEDERATION_RULE_ID, ANTHROPIC_WORKSPACE_ID, ANTHROPIC_BASE_URL, ANTHROPIC_SERVICE_ACCOUNT_ID, and ANTHROPIC_SCOPE in its file-backed branch.

stop() cleanup. stop() now uses rmSync to remove the entire token directory—the identity token, profile, and SDK-written credentials cache—so nothing outlives the step. This also means that an unchanged config re-exchanges on the next step rather than reusing the cached credentials. The fingerprint still matters for the crash path where stop() never runs.

Docstring. I added the sequential-spawn dependency note to writeFederationProfile: the plugin loop in install-plugins.ts uses for...await, the SDK cache is not serialized across processes, and parallelizing the installs would reintroduce jti_reused.

On the jti question: agreed, that is for you to confirm with the token exchange owners. If reuse turns out to be allowed, I’ll reframe the description as a latency fix.

bun test passes with 705/705 tests. bun run typecheck and bun run format:check are also clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WIF federation fails with 401 (jti_reused) when plugins are configured — each spawned claude process re-exchanges the single-use OIDC token

2 participants