diff --git a/AGENTS.md b/AGENTS.md index a59e2e8..0ba2d1d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,3 +13,10 @@ Project harness follows `jammyfu/CoderSkills` → `skills/continuous-project-loo - The "hottest new repos (7 days)" ranking is discovered by `tools/newcomers.py` from the search API; it must never repeat a project from another ranking (see `docs/NEW_ARRIVALS.md`). - Categories (AI primary + others) are defined in `data/categories.json` and documented in `docs/CATEGORIES.md`. Newcomers are per-category; see `docs/NEW_ARRIVALS.md`. + +## Harness rules (from `jammyfu/CoderSkills`: `skills/continuous-project-loop`, `skills/multi-project-program-management`) + +- Each round: Assess (read this file, `CURRENT_PLAN.md`, `TODO_BACKLOG.md`, `WORKLOG.md`) -> Plan (one Goal, 3-7 verifiable tasks, Out of Scope, Next Candidates; check with `python3 tools/next_plan.py --check`) -> Implement one minimal closed loop -> Verify (`python3 tools/verify.py`) -> Record (`CURRENT_PLAN.md`, `WORKLOG.md`, `TODO_BACKLOG.md`, `DECISIONS.md`, `CHANGELOG.md` when users can see the change) -> Continue (write the next step into the files, not only into the reply). +- Dirty worktree: if a run starts with uncommitted changes, finish or park them first (`verify` -> PR); do not stack new work on top. `tools/local_daily.sh` refuses to run on a dirty tree or off `main`. +- Verification must not depend on the shell PATH: `tools/verify.py` and `tools/local_daily.sh` use explicit interpreters, no `shell=True` (guarded by `tests/test_harness.py`). +- Do not edit the plan files to claim a result that `tools/verify.py` has not confirmed. diff --git a/CURRENT_PLAN.md b/CURRENT_PLAN.md index 343265b..c65ceb2 100644 --- a/CURRENT_PLAN.md +++ b/CURRENT_PLAN.md @@ -1,16 +1,39 @@ -# CURRENT_PLAN +# CURRENT_PLAN.md + +## Plan ID + +PLAN-001 ## Goal -Rebrand `ai-growth-atlas` → `github-risers` with multi-category rankings (AI primary). Open PR only; rename repo after CI green; do not merge. +Align `github-risers` with the CoderSkills harness (`continuous-project-loop`, `multi-project-program-management`) without touching rankings, translations, GEO files or the slim README. + +## In Scope + +- Plan files and governance records that are accurate and rollable (`CURRENT_PLAN.md`, `MASTER_PLAN.md`, `TODO_BACKLOG.md`, `docs/project-governance/*`) +- The loop rules written into repo files, not only into chat (`AGENTS.md`, `docs/AUTOMATION_COMMANDS.md`, `docs/LONG_RUNNING_AUTONOMY.md`) +- `tools/next_plan.py --check` plus regression tests for the verify and local-daily entry points (`tests/test_harness.py`) + +## Out of Scope + +- Ranking logic, categories, the ⚠ suspected-crack rules, `data/`, `latest/`, `rankings/`, generated READMEs +- The translation library, glossary, GEO files (`docs/index.html`, `llms.txt`, `CITATION.cff`, FAQ, FACTS, social preview) +- Any new telemetry, any data not coming from the GitHub API + +## Tasks + +- [x] Audit the repo against `skills/continuous-project-loop/SKILL.md` and `skills/multi-project-program-management/SKILL.md`; list the gaps in `docs/project-governance/DECISIONS.md` +- [x] Replace the stale plan, master plan, backlog, acceptance, worklog, decisions and changelog with the current state +- [x] Put the dirty-worktree rule, the record order and the verify hardening rule into `AGENTS.md` and `docs/` +- [x] Add `python3 tools/next_plan.py --check` (planning rules) and `tests/test_harness.py` (required files, plan shape, rolling, verify and local_daily hardening) +- [ ] Owner reviews the PR; merge only with approval -## Done this round +## Verification -- Categories: ai, devtools, security, web, infra, design, data -- Per-category newcomers + week/month/year/total -- Rename strings, docs, launchd, User-Agent, llms.txt -- Real API run + verify green +- `python3 tools/verify.py` +- `python3 tools/next_plan.py --check` -## Next +## Next Candidates -- CI green on PR → `gh repo rename github-risers` +- Thin Developer Tools / Design / Data new-repo boards: see `TODO_BACKLOG.md` +- Acceleration column (this week vs the week before) diff --git a/MASTER_PLAN.md b/MASTER_PLAN.md index 9487ff5..73fb38f 100644 --- a/MASTER_PLAN.md +++ b/MASTER_PLAN.md @@ -2,32 +2,38 @@ ## Goal -A trustworthy, daily-refreshed ranking of fast-rising AI projects, readable in five languages. +A trustworthy ranking of fast-rising GitHub repositories by stars gained per day, grouped by category (AI first), refreshed twice a day from the GitHub API and readable in five languages. ## Current phase -Phase 2: growth rankings, i18n, daily automation (done); Phase 2b: top-20 rankings, 20+ projects, dual-midnight schedule. +Phase 4: harness and upkeep (Phases 1-3 are done and merged). ## Phases ### Phase 1: Structure (done) -- Problem-based categories, project list, short descriptions +- Project list, short descriptions, five languages -### Phase 2: Growth data and automation -- stars/day for 7/30/365 days from the GitHub API -- Five-language READMEs, star-history curves -- Daily update (Actions + local fallback) +### Phase 2: Growth data and automation (done) +- stars/day for 7/30/365 days from `stargazers/history` +- Daily update (Actions + local fallback) at Beijing 00:00 and Los Angeles 00:00, per-slot history, `latest/` +- Top-20 boards, 7-day newcomers board -### Phase 3: Discovery -- Automatically surface candidate projects from GitHub search/trending-style API queries, owner approves additions -- Acceleration (change in stars/day) column once enough snapshots exist +### Phase 3: Categories, i18n pipeline, discovery (done) +- Rebrand to `github-risers`; categories ai (primary), devtools, security, web, infra, design, data; 70 tracked projects +- Per-category newcomers by topic only; suspected cracked/pirated repos flagged, ranked last, never leaders +- Translation library + glossary, slim README first screen, GEO (Pages, `llms.txt`, `CITATION.cff`, FAQ, FACTS) -## Definition of done for Phase 2 +### Phase 4: Harness and upkeep (now) +- Plan and governance files follow `jammyfu/CoderSkills` (`continuous-project-loop`, `multi-project-program-management`) +- Thin boards (Developer Tools quality, Design, Data) and the acceleration column: see `TODO_BACKLOG.md` + +## Definition of done for a phase - `python3 tools/verify.py` passes; data in `data/stats.json` came from a real run -- A daily update has run unattended at least once (Actions or local), at both the Beijing and Los Angeles slots +- A daily update has run unattended at both the Beijing and Los Angeles slots ## Questions for next review -- Should the year ranking use repo age as divisor for young repos (current behaviour, flagged with †)? +- Should the year ranking keep dividing young repos by their age (current behaviour, flagged with †)? - Which projects should be added or retired? +- Is a quality signal (beyond topics and the ⚠ rules) wanted for the Developer Tools new-repo board? diff --git a/TODO_BACKLOG.md b/TODO_BACKLOG.md index 4e78d42..a2fff36 100644 --- a/TODO_BACKLOG.md +++ b/TODO_BACKLOG.md @@ -2,18 +2,21 @@ ## Now -- [ ] Confirm the first scheduled `daily-update` run after merge (check Actions tab or `gh run list`) +- [ ] Confirm the first unattended cron runs after the merge: `cn-0000` (Beijing) and `pt-0000` (Los Angeles) folders appear under `rankings//` (`gh run list --workflow daily-update.yml`) +- [ ] Developer Tools new-repo board: 15 of 20 entries were flagged ⚠ in the last run because star-farmed repos use `cli` / `developer-tools`; decide whether to add a quality signal (owner call) +- [ ] Design and Data new-repo boards are thin (0-2 entries): widen with specific topics only (no generic ones; `tests/test_spam.py` guards that) ## Next - [ ] Acceleration column (stars/day this week vs. previous week) from `data/stars.json` snapshots -- [ ] Add more projects (needs five-language `desc`) +- [ ] Add more tracked projects (needs five-language `desc`) ## Later -- [ ] Candidate discovery via GitHub search API with owner approval +- [ ] Candidate discovery for tracked projects with owner approval ## Risks / Blockers -- [ ] GitHub may change or rate-limit `stargazers/history` (documented max 30 weeks/page, page ≤ 100) -- [ ] If the repo becomes private, Actions fails on the account billing problem; use `tools/local_daily.sh` +- [ ] GitHub may change or rate-limit `stargazers/history` (documented max 30 weeks/page, page <= 100) +- [ ] If Actions cannot run (billing, quota), use `tools/local_daily.sh` +- [ ] The search API is index-based and can lag by minutes; thin categories can be empty in one run diff --git a/docs/AUTOMATION_COMMANDS.md b/docs/AUTOMATION_COMMANDS.md index 01c6147..62bba9c 100644 --- a/docs/AUTOMATION_COMMANDS.md +++ b/docs/AUTOMATION_COMMANDS.md @@ -13,7 +13,8 @@ python3 tools/verify.py ## Roll to the next plan ```bash -python3 tools/next_plan.py +python3 tools/next_plan.py # roll a finished plan forward (draft: edit it by hand) +python3 tools/next_plan.py --check # validate CURRENT_PLAN.md: one Goal, 3-7 tasks, Out of Scope, Next Candidates ``` ## Refresh star data (GitHub API) @@ -31,8 +32,8 @@ tools/local_daily.sh --no-push # full local daily cycle, commit but do 1. Read `CURRENT_PLAN.md`, `MASTER_PLAN.md`, `TODO_BACKLOG.md`, `docs/project-governance/WORKLOG.md` 2. Finish the smallest closed loop 3. `python3 tools/verify.py` -4. Update the record files -5. If the plan is complete, `python3 tools/next_plan.py` +4. Update the record files (`CURRENT_PLAN.md`, `TODO_BACKLOG.md`, `WORKLOG.md`, `DECISIONS.md`, `CHANGELOG.md`) +5. If the plan is complete, `python3 tools/next_plan.py`, edit the draft, `python3 tools/next_plan.py --check` `update_stars.py` also re-discovers the 7-day new-repo ranking (about 2 minutes; rules in [NEW_ARRIVALS.md](NEW_ARRIVALS.md)). diff --git a/docs/LONG_RUNNING_AUTONOMY.md b/docs/LONG_RUNNING_AUTONOMY.md index 6ff79d8..7b69335 100644 --- a/docs/LONG_RUNNING_AUTONOMY.md +++ b/docs/LONG_RUNNING_AUTONOMY.md @@ -11,8 +11,8 @@ For scheduled and long-running agents working on this repo. 1. Read `AGENTS.md`, `PROJECT_BRIEF.md`, `MASTER_PLAN.md`, `CURRENT_PLAN.md` 2. Finish the current minimal closed loop 3. Run `python3 tools/verify.py` -4. Update `docs/project-governance/WORKLOG.md`, `DECISIONS.md`, `CHANGELOG.md` -5. If the plan is complete, run `python3 tools/next_plan.py` and hand-edit the draft +4. Record: `CURRENT_PLAN.md`, `TODO_BACKLOG.md`, `docs/project-governance/WORKLOG.md`, `DECISIONS.md`; `CHANGELOG.md` only when users can see the change +5. If the plan is complete, run `python3 tools/next_plan.py`, hand-edit the draft, then `python3 tools/next_plan.py --check` ## Rules @@ -21,3 +21,10 @@ For scheduled and long-running agents working on this repo. - Zero telemetry - Open pull requests; do not merge to `main` without the owner's approval - Do not claim done without a green `tools/verify.py` + +## Dirty worktree rule + +- If a wake-up starts with uncommitted changes, deal with them first: if they already form a closed loop, `verify` and open a PR; if not, finish or park them in `WORKLOG.md` and stop. Do not stack a new task on top. +- The only automated writer of `data/`, `latest/`, `rankings/` and `README*.md` is the daily job; `tools/local_daily.sh` refuses to run on a dirty tree or off `main` and uses its own PATH and `python3` path. +- `tools/verify.py` runs with `sys.executable`, never `shell=True` and never a bare `python`/`pytest` (`tests/test_harness.py`). +- The daily data run (Actions `schedule` / `workflow_dispatch`, or `tools/local_daily.sh` with `RISERS_DAILY_RUN=1`) skips the doc-state checks in `tests/test_harness.py`, so a stale plan file can never block a data update; CI runs them on every pull request. diff --git a/docs/project-governance/ACCEPTANCE.md b/docs/project-governance/ACCEPTANCE.md index a11e552..19acb08 100644 --- a/docs/project-governance/ACCEPTANCE.md +++ b/docs/project-governance/ACCEPTANCE.md @@ -9,3 +9,17 @@ - No analytics/tracking script anywhere (`tests/test_repo.py::ZeroTelemetryTests`) - `daily-update.yml` and `tools/local_daily.sh` exist; `docs/DAILY_UPDATE.md` states whether Actions can run and why - `python3 tools/verify.py` passes + +## Phase 3: categories and newcomers + +- Categories in `data/categories.json` (ai primary + 6 others), each with newcomers / week / month / year / total, total last +- New repos are found by topic only and must carry a category topic; no generic topics (`tests/test_spam.py`) +- Suspected cracked / pirated repos are listed after every normal repo, marked ⚠, and never appear in "Category leaders" +- Every scheduled entry (both Beijing and Los Angeles crons, `workflow_dispatch`, local launchd) runs `tools/update_stars.py` (`tests/test_spam.py::ScheduledPathTests`) +- README is a first screen (top 10 per board); the full top 20 is in `latest/`; GEO files exist and are linked + +## Phase 4: harness + +- `CURRENT_PLAN.md` passes `python3 tools/next_plan.py --check` (one Plan ID, one Goal, 3-7 tasks, Out of Scope, Verification, Next Candidates) +- `tools/verify.py` and `tools/local_daily.sh` do not depend on the shell PATH or `shell=True` (`tests/test_harness.py`) +- The harness files from `continuous-project-loop` exist and `AGENTS.md` names the two CoderSkills skills diff --git a/docs/project-governance/CHANGELOG.md b/docs/project-governance/CHANGELOG.md index 414c93c..9ab229e 100644 --- a/docs/project-governance/CHANGELOG.md +++ b/docs/project-governance/CHANGELOG.md @@ -25,3 +25,12 @@ - `data/glossary.json`: fixed terms for common words; only the terms present in a batch go into the prompt. Polish sends only un-polished entries and logs how many were skipped. - READMEs in all five languages are now a first screen: headline, one sentence, update stamp, the category leaders table (rows link to the category boards), then the category boards. Intro, schedule, "how it is produced", categories table and contributing moved to generated `docs/HOW_IT_WORKS*.md`; the long new-repo explanation moved there too. - README first screen reworked (generator only): first line `GitHub 涨星榜 / GitHub Risers` (English: `GitHub Risers: Ranked by Stars Gained per Day`), header image `.github/social-preview.png`, three badges (MIT, CI, last updated), website link, three Q&A subheads linking `docs/FAQ.md`, a star prompt, a "how to cite" line (`CITATION.cff`) and a link to `docs/FACTS.md`. Each board shows the top 10 in the README; the full top 20 stays in `latest/`. Those linked files come from the GEO pull request (#9); the generator only links them, and a test checks the FAQ anchors. + +## 2026-10-30 (categories) + +- Rebranded to GitHub Risers with categories (AI first, then developer tools, security, web, infrastructure, design, data), each with new-repo, week, month, year and total boards. +- New-repo discovery is topic-only; suspected cracked/pirated repos are marked ⚠, ranked last and never shown as category leaders. + +## 2026-10-30 (harness) + +- Plan, backlog and governance files brought up to date; `python3 tools/next_plan.py --check`; harness regression tests. No change to rankings or generated pages. diff --git a/docs/project-governance/DECISIONS.md b/docs/project-governance/DECISIONS.md index 70dd7d6..18ce2c0 100644 --- a/docs/project-governance/DECISIONS.md +++ b/docs/project-governance/DECISIONS.md @@ -38,3 +38,25 @@ - Hotness = stars / max(age days, 1); descriptions are the repo's own GitHub text (not translated). - Dedup excludes tracked projects and all displayed top-20 repos, so no project is in two rankings; a repo must be added to `data/projects.json` by the owner to be tracked. + +## 2026-10-30 (categories) + +### Newcomers are found by topic only + +- Keyword and language searches pulled in unrelated and star-farmed repos. A repo must carry one of the category's topics; generic topics are not used. + +### Suspected cracked / pirated repos: keep, flag, rank last + +- Owner decision: do not exclude them. They are marked ⚠, always rank after every normal repo and never lead a category or the leaders table. Rules in `tools/spam.py`, documented in `docs/CATEGORIES.md`. + +### One code path for every scheduled run + +- Beijing cron, both Los Angeles crons, `workflow_dispatch` and the local launchd jobs all end in `tools/update_stars.py --slot `, so the rules above apply to every run. + +## 2026-10-30 (CoderSkills harness) + +### Follow `continuous-project-loop` and `multi-project-program-management`; other CoderSkills skills do not apply + +- Applied: standard file set, one plan entry, record order, verify hardening rule, dirty-worktree rule written into repo files, planning rules (one Goal, 3-7 tasks, Out of Scope, Next Candidates). +- Not adopted: `sync_or_queue.py` / LaunchAgent safe-sync tooling (this repo changes through pull requests; the only automated writer is the daily job), `docs/architecture/*ROADMAP*.md` (not a platform-core project), `git-autosync-signal` (supports only `dramaforge` and `clippipeline`). +- `readme-tech-visualizer`: audit only. The README stays the slim first screen; the header image belongs to the GEO work, so no hero or tech-badge block is generated. diff --git a/docs/project-governance/WORKLOG.md b/docs/project-governance/WORKLOG.md index 6987c36..23c7f2c 100644 --- a/docs/project-governance/WORKLOG.md +++ b/docs/project-governance/WORKLOG.md @@ -35,3 +35,20 @@ - Replaced AI-only subcategories with top-level categories: ai (primary), devtools, security, web, infra, design, data. - Each category has newcomers / week / month / year / total (top 20); newcomers deduped within category and AI-first across categories. - Tracked projects expanded to 70; real API run stored as rankings/2026-10-30/manual-0750. + +## 2026-10-30 — categories, topic-only discovery, suspected-crack rules (#6 follow-up) + +- Newcomers: topic-only queries, repos must carry a category topic, generic topics removed; suspected cracked/pirated repos are kept, marked ⚠, ranked last and never leaders (`tools/spam.py`). +- Real re-run, then `daily-update` triggered by `workflow_dispatch` after merge: green, committed `chore(data)`. +- Verification: `python3 tools/verify.py` green (93 tests at the time). + +## 2026-10-30 — i18n pipeline, GEO, slim README (#7-#10) + +- Translation library + glossary, local polish with `agy`, CI translates only; GEO Pages and `llms.txt`/`CITATION.cff`/FAQ/FACTS; README cut to a first screen. +- Verification: `python3 tools/verify.py` green (159 tests on `41a555e`). + +## 2026-10-30 — CoderSkills harness alignment (PLAN-001) + +- Read `jammyfu/CoderSkills` (`d37305a`). Applicable: `skills/continuous-project-loop`, `skills/multi-project-program-management`; partly: `skills/readme-tech-visualizer` (audit only, README first screen already free of runtime tables, timestamps-as-noise and "Generated by"). Not applicable: `git-autosync-signal` (only `dramaforge`/`clippipeline`), `skill-lab-builder`, content and media skills. +- Gaps found: stale `CURRENT_PLAN.md`/`MASTER_PLAN.md`/`TODO_BACKLOG.md`/governance files; dirty-worktree and record-order rules only in chat; no check of the plan shape; no regression test that `verify.py` and `local_daily.sh` avoid PATH/`shell=True`. +- Added `tools/next_plan.py --check`, `tests/test_harness.py`, and wrote the rules into `AGENTS.md` and `docs/`. diff --git a/tests/test_harness.py b/tests/test_harness.py new file mode 100644 index 0000000..68f2044 --- /dev/null +++ b/tests/test_harness.py @@ -0,0 +1,188 @@ +"""Harness regression tests (jammyfu/CoderSkills: continuous-project-loop, multi-project-program-management). + +Guards the loop files, the plan shape, plan rolling, and that verify / local-daily do not depend on PATH. +""" +import os +import re +import shutil +import sys +import tempfile +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "tools")) +import next_plan # noqa: E402 +import verify # noqa: E402 + +HARNESS_FILES = [ + "AGENTS.md", "PROJECT_BRIEF.md", "MASTER_PLAN.md", "CURRENT_PLAN.md", "TODO_BACKLOG.md", + "docs/project-governance/WORKLOG.md", "docs/project-governance/DECISIONS.md", + "docs/project-governance/ACCEPTANCE.md", "docs/project-governance/CHANGELOG.md", + "docs/AUTOMATION_COMMANDS.md", "docs/LONG_RUNNING_AUTONOMY.md", "tools/verify.py", "tools/next_plan.py", +] +GOOD_PLAN = """# CURRENT_PLAN.md + +## Plan ID + +PLAN-007 + +## Goal + +One clear goal. + +## In Scope + +- a + +## Out of Scope + +- b + +## Tasks + +- [x] one +- [x] two +- [x] three + +## Verification + +- `python3 tools/verify.py` + +## Next Candidates + +- c +""" + + +# The daily job (Actions schedule / workflow_dispatch, or tools/local_daily.sh) must never be blocked by a +# documentation state; the doc checks still run in CI on every pull request and push. +IN_DAILY_RUN = os.environ.get("GITHUB_EVENT_NAME") in ("schedule", "workflow_dispatch") or bool(os.environ.get("RISERS_DAILY_RUN")) +doc_checks = unittest.skipIf(IN_DAILY_RUN, "doc-state checks are skipped inside the daily data run") + + +@doc_checks +class HarnessFileTests(unittest.TestCase): + def test_standard_files_exist(self): + for f in HARNESS_FILES: + self.assertTrue((ROOT / f).is_file(), f) + + def test_agents_names_both_coderskills_skills_and_the_loop_rules(self): + t = (ROOT / "AGENTS.md").read_text("utf-8") + self.assertIn("skills/continuous-project-loop", t) + self.assertIn("skills/multi-project-program-management", t) + for rule in ("Dirty worktree", "python3 tools/verify.py", "python3 tools/next_plan.py --check", "pull requests"): + self.assertIn(rule, t) + + def test_docs_carry_the_dirty_worktree_and_record_rules(self): + auto = (ROOT / "docs/LONG_RUNNING_AUTONOMY.md").read_text("utf-8") + self.assertIn("Dirty worktree rule", auto) + self.assertIn("TODO_BACKLOG.md", auto) + cmds = (ROOT / "docs/AUTOMATION_COMMANDS.md").read_text("utf-8") + self.assertIn("next_plan.py --check", cmds) + + def test_governance_files_are_not_stale_stubs(self): + # the plan files must describe github-risers, not the pre-rebrand AI-only project + for f in ("MASTER_PLAN.md", "CURRENT_PLAN.md", "TODO_BACKLOG.md"): + self.assertNotIn("ai-growth-atlas", (ROOT / f).read_text("utf-8"), f) + self.assertIn("categor", (ROOT / "MASTER_PLAN.md").read_text("utf-8").lower()) + + +class PlanTests(unittest.TestCase): + @doc_checks + def test_current_plan_follows_the_planning_rules(self): + self.assertEqual(next_plan.check_plan((ROOT / "CURRENT_PLAN.md").read_text("utf-8")), []) + + def test_check_flags_each_broken_rule(self): + self.assertEqual(next_plan.check_plan(GOOD_PLAN), []) + cases = { + "missing section": GOOD_PLAN.replace("## Out of Scope\n\n- b\n\n", ""), + "bad id": GOOD_PLAN.replace("PLAN-007", "plan 7"), + "too few tasks": GOOD_PLAN.replace("- [x] two\n- [x] three\n", ""), + "too many tasks": GOOD_PLAN.replace("- [x] three\n", "".join(f"- [ ] t{i}\n" for i in range(6))), + "draft goal": GOOD_PLAN.replace("One clear goal.", "DRAFT: x"), + "no verify": GOOD_PLAN.replace("python3 tools/verify.py", "run it"), + "long goal": GOOD_PLAN.replace("One clear goal.", "a\nb\nc\nd"), + } + for name, text in cases.items(): + with self.subTest(name): + self.assertTrue(next_plan.check_plan(text), name) + + def _tmp_root(self, plan, backlog): + d = Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, d, True) + (d / "CURRENT_PLAN.md").write_text(plan, "utf-8") + (d / "TODO_BACKLOG.md").write_text(backlog, "utf-8") + return d + + def test_open_tasks_are_never_rolled_over(self): + d = self._tmp_root(GOOD_PLAN.replace("- [x] two", "- [ ] two"), "## Now\n\n- [ ] next thing\n") + self.assertEqual(next_plan.main([], root=d), 0) + self.assertIn("PLAN-007", (d / "CURRENT_PLAN.md").read_text("utf-8")) + + def test_finished_plan_rolls_to_a_draft_from_backlog_now_only(self): + backlog = "## Now\n\n" + "".join(f"- [ ] item {i}\n" for i in range(9)) + "\n## Next\n\n- [ ] later thing\n" + d = self._tmp_root(GOOD_PLAN, backlog) + next_plan.main([], root=d) + new = (d / "CURRENT_PLAN.md").read_text("utf-8") + self.assertIn("PLAN-008", new) + self.assertEqual(len(re.findall(r"^- \[ \] ", new, flags=re.M)), 7) # capped at 7 tasks + self.assertNotIn("later thing", new) # only the "Now" list + self.assertEqual(next_plan.main(["--check"], root=d), 1) # a DRAFT must be hand-edited + + def test_empty_backlog_leaves_plan_alone(self): + d = self._tmp_root(GOOD_PLAN, "## Now\n\n## Next\n\n- [ ] later\n") + next_plan.main([], root=d) + self.assertEqual((d / "CURRENT_PLAN.md").read_text("utf-8"), GOOD_PLAN) + + +class VerifyHardeningTests(unittest.TestCase): + def test_verify_uses_explicit_interpreter_and_no_shell(self): + cmds = verify.commands(ROOT) + for c in cmds: + self.assertEqual(c[0], sys.executable) + self.assertIsInstance(c, list) + src = (ROOT / "tools/verify.py").read_text("utf-8") + self.assertNotIn("shell=True", src) + self.assertNotRegex(src, r"""["'](?:python|pytest)\b""") + joined = [" ".join(c[1:]) for c in cmds] + self.assertTrue(any("compileall" in j for j in joined)) + self.assertTrue(any("unittest discover" in j for j in joined)) + self.assertTrue(any("render_readme.py --check" in j for j in joined)) + + def test_no_tool_shells_out_with_a_string_command(self): + for p in (ROOT / "tools").glob("*.py"): + self.assertNotIn("shell=True", p.read_text("utf-8"), p.name) + + +class LocalDailyTests(unittest.TestCase): + def setUp(self): + self.sh = (ROOT / "tools/local_daily.sh").read_text("utf-8") + + def test_script_is_path_independent_and_strict(self): + self.assertIn("set -euo pipefail", self.sh) + self.assertRegex(self.sh, r'export PATH="[^"]*/opt/homebrew/bin[^"]*/usr/bin') + self.assertIn('PY="$(command -v python3)"', self.sh) + # every python call goes through "$PY": no bare `python`, `python3` or `pytest` command lines + for line in self.sh.splitlines(): + code = line.split("#", 1)[0] + if re.search(r"tools/\w+\.py", code): + self.assertIn('"$PY"', code, line) + + def test_dirty_tree_and_off_main_refuse_and_order_is_update_verify_commit(self): + self.assertIn('git status --porcelain', self.sh) + self.assertIn("working tree not clean; refusing to run", self.sh) + self.assertIn('!= "main"', self.sh) + i_up = self.sh.index("tools/update_stars.py") + i_ver = self.sh.index("tools/verify.py") + i_commit = self.sh.index("git commit") + self.assertLess(i_up, i_ver) + self.assertLess(i_ver, i_commit) + + def test_workflow_verifies_before_committing(self): + wf = (ROOT / ".github/workflows/daily-update.yml").read_text("utf-8") + self.assertLess(wf.index("tools/verify.py"), wf.index("git commit")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/local_daily.sh b/tools/local_daily.sh index 6f03ce6..8d52b04 100755 --- a/tools/local_daily.sh +++ b/tools/local_daily.sh @@ -62,6 +62,8 @@ if [ -n "$(git status --porcelain)" ]; then fi git pull --ff-only +# doc-state tests (plan files, governance) must not block the data update; CI still runs them on every PR +export RISERS_DAILY_RUN=1 export I18N_POLISH="${I18N_POLISH:-1}" I18N_TRANSLATE_BACKEND="${I18N_TRANSLATE_BACKEND:-agy}" I18N_POLISH_BACKEND=agy "$PY" tools/update_stars.py --slot "$UPDATE_SLOT" "$PY" tools/verify.py diff --git a/tools/next_plan.py b/tools/next_plan.py index 89b032c..ecf29e2 100755 --- a/tools/next_plan.py +++ b/tools/next_plan.py @@ -1,39 +1,85 @@ #!/usr/bin/env python3 -"""Roll CURRENT_PLAN.md forward: if every task is checked, start the next plan from TODO_BACKLOG.md "Now". +"""Roll CURRENT_PLAN.md forward, or validate it (harness: CoderSkills continuous-project-loop). -Prints the decision; rewrites CURRENT_PLAN.md only when the current plan is complete and the backlog has items. + python3 tools/next_plan.py if every task is checked, start the next plan from TODO_BACKLOG.md "Now" + python3 tools/next_plan.py --check validate CURRENT_PLAN.md against the planning rules; exit 1 if it breaks them + +Rolling rewrites CURRENT_PLAN.md only when the current plan is complete and the backlog has items. The generated plan is a draft: keep one Goal, 3-7 tasks, and edit by hand before continuing. +Planning rules checked: one Plan ID, one Goal, In Scope, Out of Scope, 3-7 verifiable tasks, +a Verification section naming `python3 tools/verify.py`, and Next Candidates. """ from __future__ import annotations +import argparse import re import sys from pathlib import Path ROOT = Path(__file__).resolve().parents[1] +REQUIRED_SECTIONS = ("Plan ID", "Goal", "In Scope", "Out of Scope", "Tasks", "Verification", "Next Candidates") +MIN_TASKS, MAX_TASKS = 3, 7 + + +def section(text: str, name: str) -> str | None: + m = re.search(rf"^## {re.escape(name)}[ \t]*\n(.*?)(?=^## |\Z)", text, flags=re.M | re.S) + return m.group(1).strip() if m else None + +def check_plan(text: str) -> list[str]: + """Return the list of planning-rule violations (empty = the plan is fine).""" + problems = [f"missing section: ## {s}" for s in REQUIRED_SECTIONS if section(text, s) is None] + pid = section(text, "Plan ID") + if pid is not None and not re.fullmatch(r"PLAN-\d{3,}", pid): + problems.append("Plan ID must look like PLAN-001") + goal = section(text, "Goal") + if goal is not None and (not goal or len([ln for ln in goal.splitlines() if ln.strip()]) > 3): + problems.append("Goal must be one short paragraph (at most 3 lines)") + if goal and goal.startswith("DRAFT"): + problems.append("Goal still says DRAFT: edit the generated plan by hand") + tasks_body = section(text, "Tasks") + if tasks_body is not None: + tasks = re.findall(r"^- \[[ x]\] .+$", tasks_body, flags=re.M) + if not MIN_TASKS <= len(tasks) <= MAX_TASKS: + problems.append(f"Tasks must have {MIN_TASKS}-{MAX_TASKS} checkbox items, found {len(tasks)}") + verification = section(text, "Verification") + if verification is not None and "tools/verify.py" not in verification: + problems.append("Verification must name python3 tools/verify.py") + return problems -def main() -> int: - plan = (ROOT / "CURRENT_PLAN.md").read_text("utf-8") - tasks = re.findall(r"^- \[( |x)\] .+$", plan, flags=re.M) + +def roll(root: Path) -> str: + plan = (root / "CURRENT_PLAN.md").read_text("utf-8") + tasks = re.findall(r"^- \[( |x)\] .+$", section(plan, "Tasks") or plan, flags=re.M) if tasks and any(t == " " for t in tasks): - print("CURRENT_PLAN.md still has open tasks; not rolling forward.") - return 0 - backlog = (ROOT / "TODO_BACKLOG.md").read_text("utf-8") + return "CURRENT_PLAN.md still has open tasks; not rolling forward." + backlog = (root / "TODO_BACKLOG.md").read_text("utf-8") m = re.search(r"^## Now\n(.*?)(?=^## |\Z)", backlog, flags=re.M | re.S) items = re.findall(r"^- \[ \] (.+)$", m.group(1), flags=re.M) if m else [] if not items: - print("Backlog 'Now' is empty; leaving CURRENT_PLAN.md as is.") - return 0 + return "Backlog 'Now' is empty; leaving CURRENT_PLAN.md as is." m_id = re.search(r"PLAN-(\d+)", plan) n = int(m_id.group(1)) + 1 if m_id else 1 - body = "\n".join(f"- [ ] {t}" for t in items[:7]) - (ROOT / "CURRENT_PLAN.md").write_text( + body = "\n".join(f"- [ ] {t}" for t in items[:MAX_TASKS]) + (root / "CURRENT_PLAN.md").write_text( f"# CURRENT_PLAN.md\n\n## Plan ID\n\nPLAN-{n:03d}\n\n## Goal\n\nDRAFT: {items[0]}\n\n" f"## In Scope\n\n- Items from TODO_BACKLOG.md 'Now'\n\n## Out of Scope\n\n- Everything else\n\n" f"## Tasks\n\n{body}\n\n## Verification\n\n- `python3 tools/verify.py`\n\n## Next Candidates\n\n- See TODO_BACKLOG.md\n", "utf-8") - print(f"Wrote draft PLAN-{n:03d} with {min(len(items), 7)} task(s). Edit before continuing.") + return f"Wrote draft PLAN-{n:03d} with {min(len(items), MAX_TASKS)} task(s). Edit before continuing." + + +def main(argv: list[str] | None = None, root: Path = ROOT) -> int: + ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + ap.add_argument("--check", action="store_true", help="validate CURRENT_PLAN.md instead of rolling it") + args = ap.parse_args(argv) + if args.check: + problems = check_plan((root / "CURRENT_PLAN.md").read_text("utf-8")) + for p in problems: + print(f"CURRENT_PLAN.md: {p}", file=sys.stderr) + print("CURRENT_PLAN.md follows the planning rules." if not problems else "CURRENT_PLAN.md breaks the planning rules.") + return 1 if problems else 0 + print(roll(root)) return 0