fix(pull): close the scheduler and breaker gaps on pull pilots (#2514)

[obasilakis]

Summary

Closes the remaining gaps from #2514, plus a breaker gap found while writing its test.

• Scheduler dispatch-error path (_dispatch_and_record_outcome, issue §3). When the dispatch call failed after the backend had already queued the row, the scheduler read queued as finished: it published schedule_execution_completed(status=queued), never polled, and lost the real terminal and retry/validation. Its FAILED write was also unconditional, so it could land on a row a pull worker had claimed and publish a false failure that made the run retry-eligible. The FAILED write is now a CAS on status = running AND claim_token IS NULL. A row it cannot fail is polled to its real terminal through the same background poll the accepted path uses.
• Breaker on the pull path (issue §4). Test added: an open breaker refuses the turn with CircuitOpen before the persistent-queue branch, so nothing is enqueued.
• Half-open probe on a pilot. The probe took a push slot, running the turn outside the pilot's worker pool (fix(pull): make the pilot flag a true either/or + soak measurement set (#1766) #1982). It is now enqueued. The pull sink did not record any breaker verdict, so a pilot's breaker never tripped on pulled auth failures and could never close on a pulled probe. It now records the verdict on its CAS-won branch: success records a success, auth counts, any other failure records nothing.

Issue §1 (schedule context on pulled cron turns) is already on dev via #3114. §2 (retry stacking, poison-parked rows) was closed by #2845 / #2980.

Changes

• src/scheduler/service.py: CAS-guarded failure write, poll fallback, _spawn_poll helper shared with the accepted path.
• src/scheduler/database.py: update_execution_status(require_unclaimed=True) adds AND claim_token IS NULL.
• src/backend/services/capacity_manager.py: pull_exclusive computed before the breaker gate; a pilot's probe skips the slot path and is enqueued.
• src/backend/services/pull_coordination_service.py: _spawn_breaker_verdict on the CAS-won branch of apply_task_result.
• Docs: docs/memory/architecture/execution.md, docs/memory/feature-flows/dispatch-circuit-breaker.md.

Test Plan

• New tests: cd tests && pytest unit/test_2514_scheduler_pull_gaps.py -v. 15 pass. They were red on the old code for the expected reasons (no poll / claimed row written failed / probe admitted / no verdict recorded).
• Related suites (feat(pull): let scheduled work reach the durable queue — Option 2 deferred from #2048 #2391, bug(subscriptions): a pull-dispatched terminal never triggers SUB-003 — the switch hook is missing on apply_task_result #2643, Production soak-test of pull/work-stealing coordination (#1081) before default-on #1766, feat: per-agent dispatch circuit breaker to prevent backlog poisoning (RELIABILITY-007) #526): 102 pass after rebase onto dev.
• Full unit suite, branch and unmodified dev at the same base, run side by side. Branch: 30 failed / 2 errors. dev: 36 failed / same 2 errors. Every branch failure also fails on dev.
• Live, local stack (Postgres, real backend, gtm-synthesizer as a real pull pilot). The scheduler's real handler was run with the backend called for real and its reply discarded, to simulate a dispatch timeout:

Case	Old code	This branch
Row already queued	completion published as queued, no poll	poll started, real success reported
Row claimed by the worker	written failed, failure event published (worker later overwrote to success)	row untouched, poll reported success
Push agent, unclaimed running row	—	failed + failure event, unchanged

• Live breaker check: branch backend code against live Postgres + Redis, breaker enabled for one pilot, pilot container paused so the harness acted as its worker:

Step	Result
3 auth failures, cooldown lapsed	breaker open
Probe dispatched on the pilot	QUEUED, push slots 0 → 0
Pulled success reported	breaker closed, failures 0
Pulled auth failure	failures 1
Pulled timeout failure	failures unchanged

Fixes #2514

🤖 Generated with Claude Code

[vybe]

merge-train: batch validated on train/20261005-1214 (#3228, all gates green)