Skip to content

Model routing: route Update Astro, Changeset Generator and Sighthound Security Scan (Codex on Copilot) instead of pinning GPT-5.6 Sol and GPT-6 Sol #74865

Open

Description

@rabeckett

Estimated saving: about 450-650 AIC/week. These three Codex workflows already use the Copilot provider but pin a large GPT model. Route them with economy, like Auto-Triage Issues, Ponytail Reviewer and Daily Go Test Parallelizer (#74199, #74201, #74586).

Workflow Output Now Mode Agent runs/wk AIC/run now Est. saving/wk
update-astro PR (Astro and npm upgrade for the docs) copilot/gpt-5.6-sol economy ~2 ~200 250-320
changeset changeset file pushed to the PR copilot/gpt-5.6-sol economy 3-6 ~52 130-260
sighthound-security-scan issue (top Sighthound findings) copilot/gpt-6-sol economy 5 ~17 50-60

AIC/run includes threat detection.

Why

  • Codex routing on Copilot works. On AWF v0.29.6, Ponytail Reviewer succeeded on 39 of 40 runs (the one failure was a PR-branch run that never reached routing), Auto-Triage Issues 3 of 3 and Daily Go Test Parallelizer 9 of 9. None of these three workflows uses sub-agents, so the open Codex sub-agent bugs (Codex sub-agent endpoint checks reject OpenAI responses metadata as catalog unavailable #71157, Codex startup rejects advertised Responses support because endpoint enums are compared with URL paths #72094) don't apply.
  • Routed runs cost much less. Auto-Triage Issues went from 79 to 20 AIC per run (GPT-6 Astra before), Ponytail Reviewer from 15 to 4 (Terra), and Daily Go Test Parallelizer, which also writes code and opens PRs, from 55 to 11-33 on economy (Terra). Terminal Stylist went from 106 to 35 after leaving its GPT-5.6 Sol pin.
  • economy for all three. Update Astro's PR has to pass the docs build and is reviewed. Changeset Generator writes one small changeset file. Sighthound finds the violations itself, and the agent only runs when there are actionable findings and summarises the top ones in an issue. For this work, economy picks GPT-5.6 Luna or GPT-6 Luna (high to max); balanced picks the same Luna for most of these labels.
  • Detection gets much cheaper. Threat detection inherits today's pin: about 40 AIC per run for Update Astro, 27 for Changeset Generator (as much as its agent) and 5 for Sighthound. Without a pin it uses the detection model, about 1 AIC, as on Ponytail Reviewer and Auto-Triage Issues.
  • No model restrictions. Only a hard cross-system label picks gpt-5.6-sol:max, which could cost more than today. None of the three works across systems.

Change

In .github/workflows/update-astro.md, changeset.md and sighthound-security-scan.md, delete the top-level model: line and add model-routing to the existing engine block:

engine:
  id: codex
  model-provider: github
  model-routing:
    goal: cost
    mode: economy

Add copilot-requests: write to permissions in update-astro.md and changeset.md; sighthound-security-scan.md already has it. Keep the prompts, steps and safe outputs. Don't add model restrictions.

Run gh aw compile update-astro changeset sighthound-security-scan. On main (fdfc5d5a0d) these compile with no new warnings: routing is enabled with GPT candidates on Copilot, Update Astro and Changeset Generator move from the COPILOT_GITHUB_TOKEN secret to github.token, and detection uses the detection model.

Check

  • Audit the first routed runs with gh aw audit <run>: expect a Luna pick, and compare AIC per run with the table.
  • A gpt-5.6-sol:max pick means the run was labelled hard cross-system work. If Sighthound keeps getting it and costs more than about 17 AIC per run, restore its copilot/gpt-6-sol pin.
  • Check that Update Astro's next PRs still pass the docs build.

Estimates use gh-aw usage from 19 October to 3 November and AWF v0.29.6's routing tables; the routed cost per run is estimated from the routed workflows above.

Activity

  1. locked and limited conversation to collaborators on Nov 4, 2026
  2. unlocked this conversation on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions