docs(stage-router): correct the decision flow and rewrite threshold tuning

[sabhatinas]

Docs only. No code change.

Decision flow was wrong

• The old diagram gated on confidence >= threshold and jumped straight to "signals pick capable/efficient", which hid both hard rules and the sign check. It also implied the score is the confidence.
• New diagram shows the real cascade: critical-error/compaction override → tests-passed de-escalation → scorer → threshold → sign → classifier/fall-open.

Show where the score comes from

Added the actual formula and what it implies, rather than asserting "~0.46":

raw   = 0.10 × (severity/0.7 + spinning + exploring − production_intensity)
score = tanh(5.0 × raw)        → [-1, +1],  confidence = |score|

with a table of what 1 / 1.5 / 2 / 3 corroborating signals actually score.

Threshold tuning rewritten

• Replaced the 0.0 / 0.5 / 0.7 / 1.0 recommendation table — it read as a menu of portable settings, which they are not.
• Added number lines showing the three bands and who owns the middle: efficient_first sends [-1, t) to efficient and [t, +1] to capable; capable_first sends [-1, -t] to efficient and (-t, +1] to capable.
• Added a table converting t into "maxed signals needed to leave the default tier".
• Changed the starting recommendation to 0.3 and said plainly that swapping either model changes the trajectories, so the score distribution moves and the same t buys a different split. Recalibrate per tier pair.

Calibration rewritten

• Dropped the probe-set stratification table and the RESCUE/LOSS/SAFE/HARD quadrant construction. It demanded two matched runs and a 20-task stratified sample before you could pick a number, which is more ceremony than picking a threshold needs.
• Now: replay any run with real tool traffic through the switchyard-stage-router-scorer skill, histogram the scores, put the cut line where the mass tells you to, sweep, and check escalations land where the efficient tier actually fails.
• Added an ASCII histogram showing the threshold as a cut line on the distribution.
• Added a section on recent_turn_window — short windows react fast and flap, long windows need sustained trouble and react late — and noted it is not independent of t.
• Replaced "derived from SWE-Bench Pro Python-75" with the honest version: calibrated on Terminal-Bench 2.1 and more recently SWE-Bench Pro, which is why you should re-derive rather than adopt.

Route configuration

Replaced the single-provider OpenRouter gpt-4o/4o-mini example with the two-provider shape we actually benchmark: Anthropic Messages capable tier, OpenAI Chat efficient tier, per-target extra_body, handoff notes, and the --routing-log-file flag.

Verified with mkdocs build --strict.

Summary by CodeRabbit

• Documentation
  • Updated routing guidance to explain signed scores, routing bands, thresholds, and hard overrides.
  • Added score-calibration recommendations using replayed runs and configurable thresholds.
  • Refreshed configuration examples with provider-specific settings, handoff notes, and per-target options.
  • Added startup guidance for routing-decision logging and route ID alignment.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-288/
Built to branch gh-pages at 2026-08-06 23:44 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[coderabbitai]

Walkthrough

The routing documentation now defines signed confidence scoring, capable and efficient routing bands, hard overrides, replay-based calibration, provider-specific configuration, handoff behavior, and startup logging requirements.

Changes

Stage router routing

Layer / File(s)	Summary
Scoring and decision flow docs/routing_algorithms/stage_router_routing.md	Documents the signed score, bounded confidence model, corroboration, hard overrides, classifier path, and picker fallbacks.
Threshold bands and calibration docs/routing_algorithms/stage_router_routing.md	Defines picker-specific threshold bands and replay-based score-distribution and window calibration.
Provider configuration and startup guidance docs/routing_algorithms/stage_router_routing.md	Updates provider targets, per-target options, handoff settings, routing-decision logging, and route ID requirements.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit tuning routes tonight,
With signed scores hopping left and right.
Thresholds sweep, providers flow,
Logs record each route we know.
Efficient hops make docs just right!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the documentation scope and the two main changes: correcting the decision flow and rewriting threshold-tuning guidance.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/routing_algorithms/stage_router_routing.md`:
- Around line 68-76: Update the Mermaid flowchart’s de-escalation decision to
require tests_passed, a recent write or edit, and severity <= 0.0 before
selecting the efficient tier. Update the classifier path so an invalid or absent
classifier tier falls through to the picker’s default fall_open tier, matching
the runtime behavior in the stage-router picker.
- Around line 200-204: Update the “Caveat on efficient outcomes” section to
remove the claim that stage-router efficient performance is always at least as
good as standalone efficient performance or constitutes a conservative lower
bound. State that inherited conversation history may help or hurt, and treat the
two runs as non-equivalent baselines unless matched benchmark results establish
a lower bound.
- Around line 258-263: Update the routing-log example in the documentation to
use a path writable by the server user, or explicitly document the required
ownership and permissions for /var/lib/switchyard. In the request example, set
the model value to "switchyard" so it matches routes.stage.id and selects the
intended route.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e43c89fd-971a-4c60-ae0e-1ece77160281

📥 Commits

Reviewing files that changed from the base of the PR and between 6e63ad9 and e87b43c.

📒 Files selected for processing (1)

• docs/routing_algorithms/stage_router_routing.md