Skip to content

fix: record /daily runs before scanning and pause on interrupted runs - #163

Merged
elfrost merged 1 commit into
mainfrom
fix/daily-interrupted-run-guard
Oct 22, 2026
Merged

elfrost merged 1 commit into
mainfrom
fix/daily-interrupted-run-guard

Conversation

@elfrost

@elfrost elfrost commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Why

Between 2026-09-25 and 2026-10-03 seven /daily runs scanned a repo and then ended in curation (a model safety stop; the headless logs end with API Error: ... safeguards flagged this session ... Details: [cyber]) before Phase 6 wrote any state. last_run stayed 2026-09-28, the one-scan-per-day rule never fired, and every morning spent a fresh scan: seven scans lost, nothing published, nobody told for a week. The pipeline has been paused (.daily-paused) since 2026-10-04.

What

  • scanner/run_orphan_check.py (new CLI, exit 2 on any orphan, same convention as run_verdict_check.py) finds runs_recent entries left in-progress by an earlier day, and scan output newer than last_run with no post and no run record. --write-pause creates .daily-paused naming them (and never overwrites one); --acknowledge closes them after review (atomic write, dated .bak-<date> backup of the state file).
  • /daily (Claude command and Codex mirror): the run record is opened as soon as Phase 2 picks a target and closed in Phase 6; Phase 0 runs the check and stops on an orphan. Guardrail 10 documents the rule.
  • CLAUDE.md, AGENTS.md and ROADMAP.md updated.

Not in this PR, because it lives outside the repo: the scheduled wrapper run-daily.ps1 now exits 1 when Claude fails and 3 when the run ended with .daily-paused present. It used to always exit 0, so Task Scheduler showed success for dead runs.

Test plan

  • pytest tests/: 441 passed (25 new, in tests/test_orphan_check.py)
  • ruff check and black --check clean
  • Mutation check: 13 realistic bugs injected into the script one at a time, every one caught by the tests
  • Run against the real state file: no false positive. Replayed on a copy with last_run reset to 2026-09-28, it flags the five scans written after the last good run
  • First real /daily run after merge, watched

Limits

The safety stop itself is platform-side. This makes a dead run visible the next morning and bounds the loss to one scan; it does not prevent the stop.

🤖 Generated with Claude Code

Between 2026-09-25 and 2026-10-03 seven /daily runs scanned a repo and then
ended in curation (a model safety stop) before Phase 6. State was only
written there, so last_run stayed stale, the one-scan-per-day rule never
fired and every morning spent a fresh scan: seven scans lost, nothing
published, nobody told for a week.

- scanner/run_orphan_check.py: finds runs_recent entries left in-progress
  by an earlier day, and scan output newer than last_run with no post and no
  run record. Exit 2 on any orphan; --write-pause creates .daily-paused,
  --acknowledge closes them after review (dated state backup, atomic write)
- daily command and Codex mirror: open the run record when Phase 2 picks a
  target, close it in Phase 6, run the check in Phase 0
- tests: 25 cases; 13 realistic mutations of the check are all caught
- docs: CLAUDE.md, AGENTS.md, ROADMAP.md

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T20:56:52.680257Z 3297c09 PR opened
🔒 Security Review ✅ Completed 2026-10-13T20:56:06.720254Z 3297c09 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3297c09292

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

that has neither a post nor a run record.
"""
today_day = date.fromisoformat(today)
entries = [entry for entry in state.get("runs_recent") or [] if isinstance(entry, dict)]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject a non-list runs_recent value

If the manually maintained state contains runs_recent as an object or string rather than a list, this comprehension silently discards the malformed history and reports no open record. The fallback may also miss that run because opening a record sets last_run to the scan date and line 150 ignores output written on that same date, allowing the next autonomous scan to proceed without detecting the interruption. Validate the field type and fail closed instead of filtering it away.

AGENTS.md reference: AGENTS.md:L211-L216

Useful? React with 👍 / 👎.


## State
`reports/.daily_state.json` (gitignored): `{"last_run":"YYYY-MM-DD","last_slug":"...","runs":[...]}`. If `last_run == today`, rate-limit to status-only.
`reports/.daily_state.json` (gitignored): `{"last_run":"YYYY-MM-DD","last_slug":"...","runs":[...]}`. If `last_run == today`, rate-limit to status-only. The run record is opened as soon as Phase 2 picks a target and closed in Phase 6 (same `runs_recent` entry, real outcome; `halted: <reason>` on any early stop).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Show runs_recent in the Codex state example

The Codex /daily state example still models run history under runs, although guardrail 8 calls that key vestigial and run_orphan_check.py reads only runs_recent. An agent following this JSON example can open and close the wrong list; since it also sets last_run, the same-day report-directory fallback will not detect the interrupted run later. Change the example key to runs_recent so the Codex and Claude workflows describe the same schema.

AGENTS.md reference: AGENTS.md:L3-L4

Useful? React with 👍 / 👎.

Comment on lines +187 to +190
for orphan in orphans:
if orphan.kind == UNRECORDED_SCAN:
entries.append({"date": today, "slug": orphan.slug, "outcome": note})
changed += 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the scan date when acknowledging output

When an unrecorded scan created today is acknowledged, this appends a record dated today but leaves last_run unchanged. Phase 0 rate-limits exclusively through last_run, so after the user deletes the pause file and resumes on the same day, the pipeline treats the day as unused and can scan a second repository. Preserve the detected output date and advance last_run for a current-day acknowledged scan, without charging historical acknowledgments to today.

AGENTS.md reference: AGENTS.md:L249-L249

Useful? React with 👍 / 👎.

Comment on lines 38 to +39
## Phase 2 — Candidate discovery + pre-check
`gh search repos --language=python --stars=1500..6000 --sort=updated` filtered to AI/agent/LLM/RAG/inference. Drop already-scanned slugs. Responsiveness check (recent closed issues + merged PRs from ≥2 contributors). Strict-norm detection → publication mode. Pick ONE; record why.
`gh search repos --language=python --stars=1500..6000 --sort=updated` filtered to AI/agent/LLM/RAG/inference. Drop already-scanned slugs. Responsiveness check (recent closed issues + merged PRs from ≥2 contributors). Strict-norm detection → publication mode. Pick ONE; record why. Then open the run record before Phase 3.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip record creation during Codex dry runs

This instruction unconditionally opens a run record after target selection, including in --dry-run, while the parallel Claude workflow explicitly says that dry runs open no record. Because opening the record also sets last_run = today, a Codex dry run consumes the daily quota and causes a later real invocation that day to downgrade to status-only. Add the same --dry-run exemption here to preserve runtime parity and dry-run semantics.

AGENTS.md reference: AGENTS.md:L3-L4

Useful? React with 👍 / 👎.

@elfrost
elfrost merged commit 30d79cb into main Oct 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant