feat(ai-gateway): add regional latency benchmark

[dtice25]

Summary

Adds a regional AI gateway latency benchmark where each runner location is a dimension. Each invocation runs one region, tags every record with region, and multiple invocations can share the same --run-key to appear as one platform run.

When a --run-key is used, the benchmark suffixes each provider participant slug with the runner region (<provider>-<region>) so sibling regional runners register distinct participants. The original provider name is preserved in a new displayName field so legacy JSON and SVG output stay readable.

What changed

• benchmarks/ai-gateway/ai-gateway-regional.bench.ts (new): single-region-per-run benchmark that reads BENCH_REGION or --ai-gateway-region. Defaults to 5 cold + 5 warm iterations and honors --iterations / --ai-gateway-iterations-cold / --ai-gateway-iterations-warm.
• benchmarks/ai-gateway/shared-task.ts: resolveAIGatewayRegionalPhases builds <region>:cold / <region>:warm phases, makeAIGatewayTask tags records with region, and parsePhase extracts region/mode from the phase string.
• benchmarks/ai-gateway/types.ts: removed the unused regionalEndpoints field; added optional displayName to AIGatewayProviderConfig for clean legacy/SVG output when participant slugs carry extra qualifiers.
• benchmarks/ai-gateway/legacy-results.ts: emits provider from displayName (falling back to the participant slug).
• package.json: added bench:ai-gateway-regional scripts.
• .github/workflows/ai-gateway-regional-benchmarks.yml (new): weekly/ manually-triggered matrix that runs one job per region with the same --run-key. The runs-on value currently uses namespace-profile-default with vault grants as a placeholder; region-specific Namespace runner profiles/labels must be configured for actual regional scheduling. All uses: actions are pinned to commit SHAs, pnpm install --frozen-lockfile is used for all triggers, the pnpm cache is disabled, and unused permissions are removed.

Usage

BENCH_REGION=us-east-1 pnpm bench:ai-gateway-regional
# or join four regional runners into one platform run
BENCH_REGION=eu-west-1 npx tsx packages/benchsdk-runner/dist/bin.js run \
  benchmarks/ai-gateway/ai-gateway-regional.bench.ts --run-key regional-ai-gateway-123

Link to Devin session: https://app.devin.ai/sessions/e421351b95ad4ddba4959a7c2aa41d92
Open in Devin Desktop: https://app.devin.ai/desktop/session/e421351b95ad4ddba4959a7c2aa41d92?variant=devin
Requested by: @dtice25

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[open-cla]

Contributor License Agreement

All contributors are covered by a CLA.

[devin-ai-integration]

Fixed: resolveAIGatewayRegionalPhases now receives [...process.argv.slice(2), '--ai-gateway-regions', region], so --iterations, --ai-gateway-iterations-cold, and --ai-gateway-iterations-warm are parsed correctly. Verified with tsx -e that us-east-1:cold gets 2 iterations and us-east-1:warm gets 1 when those flags are passed. Pushed in 0f41d12.

[devin-ai-integration]

Fixed: resolveAIGatewayRegionalPhases now throws when the generic --iterations flag is <= 0, and the benchmark throws when no phases remain. Per-phase --ai-gateway-iterations-* 0 still disables just that phase. Verified: --ai-gateway-iterations-cold 0 yields only warm phases, --iterations 0 errors, and --ai-gateway-iterations-cold 2 --ai-gateway-iterations-warm 1 yields 2 cold + 1 warm. Pushed in 0f341bd.

[devin-ai-integration]

Fixed: added the Namespace vault object list/describe grants to the runs-on value so load-vault-secrets.sh can retrieve credentials in each regional leg.

[devin-ai-integration]

Fixed: when --run-key is used the benchmark suffixes each provider participant slug with the runner region (<provider>-<region>) and stores the original provider name in the new displayName field. Sibling regional runners now register distinct participant identities while data.region still drives scoring.groupBy. Without --run-key (e.g. per-provider local runs) the original slugs are preserved.

[devin-ai-integration]

Fixed: updated the load-vault-secrets.sh regex to load the full set of gateway credential env vars used by benchmarks/ai-gateway/providers.ts, matching the Anthropic-family leg of the existing AI gateway workflow.

[devin-ai-integration]

Fixed: removed the schedule-only pnpm update --ignore-scripts branch; all workflow triggers now use pnpm install --frozen-lockfile --ignore-scripts so scheduled runs don't pull unreviewed dependency updates in a secret-bearing job.

[devin-ai-integration]

Fixed: pinned actions/checkout, pnpm/action-setup, actions/setup-node, namespacelabs/nscloud-setup, and actions/upload-artifact to verified full commit SHAs with the version tag in a comment.

[devin-ai-integration]

Fixed: removed cache: 'pnpm' from actions/setup-node so the privileged benchmark job does not restore a shared dependency cache before loading secrets.

[devin-ai-integration]

Fixed: removed the unused pull-requests: write permission from the workflow.

[devin-ai-integration]

Fixed: added RAMP_ROUTER_API_KEY to the load-vault-secrets.sh regex so the regional workflow loads Ramp credentials alongside the other gateway keys.