Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ git이 추적하는 폴더로 지정하세요.
- **독립된 눈.** scout, 리뷰어, QA는 작업을 만든 컨텍스트가 아닌 새 컨텍스트에서
돕니다.
- **주장 대신 증거.** 실행 환경이 없으면 PASS가 아니라 BLOCKED입니다. 버그 수정에는
수정 전에 실패한 테스트가 있어야 합니다.
수정 전에 실패한 테스트가 있어야 하고, 새 동작의 테스트는 그 동작 없이 한 번
실패하는 것을 확인합니다.
- **검증한 것만 납품.** 납품한 트리가 검증한 커밋과 다르면 `close`가 거부합니다.

Build는 [ponytail](https://github.com/DietrichGebert/ponytail) 방식(가장 작은
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,8 @@ vendors e2e's agent guide (Apache-2.0, pinned) so it works offline;
- **Independent eyes.** Scouts, reviewer and QA run in fresh contexts, never
in the one that wrote the work.
- **Proof, not claims.** No environment means BLOCKED, never PASS. A bug fix
needs a test that failed before the fix.
needs a test that failed before the fix; a test for new behavior is seen
failing once without it.
- **Delivered means what was verified.** `close` refuses a delivered tree that
differs from the verified commit.

Expand Down
2 changes: 1 addition & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ governs phase order, gates and proof.
|---|---|---|---|---|
| 1 | Intent | `phases/1-intent.md` | `intent.md`, `refs/`, `logs/` | Clarity 90%, intent approved |
| 2 | Plan | `phases/2-plan.md` | `refs/`, `PLAN.md` | plan approved |
| 3 | Build | `phases/3-build.md` | code, `review.md` | review PASS |
| 3 | Build | `phases/3-build.md` | code, `review.md`, `logs/` | review PASS |
| 4 | Evidence | `phases/4-evidence.md` | `evidence.md`, `logs/` | `sdlc.sh score` routes |

`sdlc.sh next <slug>` names the next step and why. Run `sdlc.sh` (in this
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.6.1
2.7.0
4 changes: 2 additions & 2 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ <h2 id="why-h" data-i18n="why-h">What you get</h2>
</div>
<div>
<dt><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M4 7h10M18 7h2M4 17h4M12 17h8"/><circle cx="16" cy="7" r="2"/><circle cx="10" cy="17" r="2"/></svg><span data-i18n="g4-t">Proof behind every "passed"</span></dt>
<dd data-i18n="g4-d">Every pass points to the record of what was run or seen. If there is nowhere to try something, it is marked as blocked, never as passed. A bug fix needs a test that failed before the fix and passes after it. The list of scenarios becomes your own checklist.</dd>
<dd data-i18n="g4-d">Every pass points to the record of what was run or seen. If there is nowhere to try something, it is marked as blocked, never as passed. A bug fix needs a test that failed before the fix and passes after it, and a test for new behavior is seen failing once without it. The list of scenarios becomes your own checklist.</dd>
</div>
<div>
<dt><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><circle cx="12" cy="12" r="9"/><path d="m8.5 12 2.2 2.2 4.8-5"/></svg><span data-i18n="g5-t">Delivered means what was verified</span></dt>
Expand Down Expand Up @@ -461,7 +461,7 @@ <h2 id="install-h" data-i18n="inst-h">For the person who sets it up</h2>
"g3-t": '다른 AI가 따로 점검합니다',
"g3-d": '조사하는 보조 AI, 검토 AI, 시험 AI는 모두 처음부터 새로 시작합니다. 점검하는 대상을 만든 AI가 맡는 일은 없고, 결과를 고치지도 않습니다. 별도의 AI를 쓸 수 없으면 그 빈틈을 기록합니다.',
"g4-t": '"통과"에는 증거가 붙습니다',
"g4-d": '모든 통과에는 무엇을 실행하거나 보았는지 기록이 붙습니다. 시험해 볼 곳이 없으면 통과가 아니라 "막힘"으로 표시합니다. 오류 수정에는 고치기 전에는 실패하고 고친 뒤에는 통과하는 테스트가 있어야 합니다. 시나리오 목록은 그대로 담당자의 확인 목록이 됩니다.',
"g4-d": '모든 통과에는 무엇을 실행하거나 보았는지 기록이 붙습니다. 시험해 볼 곳이 없으면 통과가 아니라 "막힘"으로 표시합니다. 오류 수정에는 고치기 전에는 실패하고 고친 뒤에는 통과하는 테스트가 있어야 하고, 새 동작의 테스트는 그 동작 없이 한 번 실패하는 것을 확인합니다. 시나리오 목록은 그대로 담당자의 확인 목록이 됩니다.',
"g5-t": '검증한 것만 납품합니다',
"g5-d": '납품한 결과가 검사한 버전과 다르면 작업을 닫을 수 없습니다. 위험한 작업은 어떤 설정에서든 사람의 허락이 먼저 필요합니다. 데이터 삭제, 저장된 데이터의 구조 변경, 다른 시스템이 기대는 약속의 변경, 보안, 인프라, 이 컴퓨터 밖으로 작업을 내보내는 일이 여기에 해당합니다. 작업 중에 읽은 글은 참고 자료로만 다루고, 지시는 사람에게서만 받습니다.',
"lim-h": '할 수 없는 일',
Expand Down
79 changes: 79 additions & 0 deletions log/changelog-v2.7.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# v2.7.0 — keep the contrast: failing tests and dated facts

Three rules make a defect, a test and a fact each carry the run that proves
it. A reviewer's caught defect becomes a test, a new test is seen failing
once, and a fact in `knowledge` names the commit it was verified at.

## Why

- Across three projects, 17 features ran v2 reviews; 11 had at least one
FAIL verdict, 26 `VERDICT: FAIL` rounds in total, 14 on plans and 12 on
diffs. Only Evidence's code FAILs had to become regression tests. Of the 10
diff-review findings that were a code defect or a weak test, the next round
cited a new or changed test aimed at the finding for 3 and not for 7
(counted from `review.md` citations, not from the test files); for the 7
the defect lived on only as prose in `review.md`.
- v2.5.1's false e2e pass (a URL assertion matching the address before a
client-side redirect) was found only by a negative control. One project
already proved its tests that way: head PASS 12/12, base FAIL 12/12.
- Facts in `knowledge` cite files and lines but not the commit they were
verified at, so a scout could cite one as AS-IS after the code changed.
- The recurring FAIL categories across features — missed consumers (5
features), intent or policy mismatches (7), scenario gaps (4), code defects
(4), weak tests (3) — are each already targeted by an existing reviewer or
scout rule. A lesson store would only repeat them.

The idea came from reviewing a proposal built on Harness² (arXiv
2610.14224, "Recursive Agent Harnessing for an Open World", 2026-10-09),
which keeps a lesson only when an execution contrast supports it. These
rules keep that contrast in the project's own tests and on the fact itself.
The proposal's lesson store, context packer and auto-promotion scripts were
not adopted: v2.0.0 removed that kind of structure.

## Changes

- **A blocking diff finding that shows a defect becomes a test.** In Build,
a blocking diff-review finding that a screen, API, job or log could show
is fixed like a bug fix: its regression test first, failing on the
FAILed commit for the finding's reason, then the fix. If no test can show
it (no test runner, a production-only symptom), the fix's commit message
(or, for an uncommitted review, the review request) says why. Plan-review FAILs and findings nothing can observe (wording,
docs, structure) are excluded. The reviewer blocks an observable blocking
diff finding left untested and unexplained. Non-blocking `(observable)`
findings still become QA scenarios, as in v2.4.0.
- **A new test for new behavior is seen failing once.** Build runs each new
or changed test for new behavior once where it must fail: on the base
commit in a disposable worktree (never stashing or touching the human's
tree, as QA already does), or, for a new symbol, with the behavior
disabled. The output goes to `logs/`. It must fail on its assertion;
"symbol not found" or a compile error does not count. The reviewer now
receives `logs/` and blocks a new-behavior test with no recorded assertion
failure. SKILL.md lists `logs/` among Build's writes.
- **Facts carry the commit they hold at.** Evidence writes each fact that
rests on code into `knowledge` with its file (path from the project root)
and the delivered commit of that file's repository,
`[verified: <file> @ <short sha>]`, re-checking the fact on that commit
first when the file changed since the fact was checked (this change, a
rebase or a merge), so a fact verified in Plan against older code is not
stamped with the new commit. Before citing a
code fact, a scout checks whether its file changed since that commit (in
the file's repository, `git log --oneline <sha>.. -- <file>` prints
anything); if it changed, or the fact has no commit, the scout re-checks it
against the code or cites it as `[assumed: <why>]`. Running `git log` in
the file's own repository makes this work for one repository or a root
holding several. Facts not about code (policies, data) are not checked
this way, and only facts a scout cites are checked; there is no list of
watched paths and no cache.

No script change; existing records stay valid.

## Known limits

- Facts already in `knowledge` have no commit; each is re-checked whenever
a scout cites it, until Evidence records it again with its commit.
- A fact that rests on several files carries one file; a change to another
file it depends on is not caught.
- The plan load is at 2500 of 2500 words; the next rule for scouts,
reviewer, plan or SKILL.md must replace words, not add them.
- Whether a finding could be observed is a judgment, as `(observable)`
already is.
15 changes: 13 additions & 2 deletions phases/3-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ Changes:
codebase's structure and conventions.
- Bug fix: write the regression test first and watch it fail for the
reported reason; fix the root cause once, in the shared code.
- A new or changed test for new behavior: run it once where it must fail, on
the base commit in a disposable worktree (never stash or touch the human's
tree) or, for a new symbol, with the behavior disabled; keep the output in
`logs/`. It must fail on its assertion, not on a missing symbol or compile
error.
- Never cut validation at trust boundaries, error handling that prevents data
loss, security or accessibility.
- After each step run the narrowest check; run the full `build`, `test` and
Expand All @@ -29,7 +34,7 @@ Changes:

Commit the change locally (the project's rules decide the message), then
dispatch the reviewer (`roles/reviewer.md`, diff mode) with intent.md,
PLAN.md, `refs/` and the commit. Where every commit needs the human's
PLAN.md, `refs/`, `logs/` and the commit. Where every commit needs the human's
approval, review the uncommitted change instead. The reviewer appends a round
to `review.md` ending in `VERDICT: PASS` or `VERDICT: FAIL`. Record it with
`sdlc.sh review <slug> pass|fail "<one line>"`: the script reads the verdict
Expand All @@ -53,7 +58,13 @@ needs a new round, as does any code changed after a PASS; rewording the
commit made on top of it does not. Recording hashes those files into the
repository's object store (not its index): ignore large or secret files
before reviewing.
FAIL → fix the blocking findings, commit, review again. The third FAIL stops
FAIL → fix the blocking findings, commit, review again. A blocking diff
finding that a screen, API, job or log could show is a bug fix: its
regression test first, failing on the FAILed commit for the finding's reason,
then the fix; if no test can show it, the fix's commit message (or, for an
uncommitted review, the review request) says why.
Plan-review FAILs and findings nothing can observe (wording,
docs, structure) need no test. The third FAIL stops
the loop: show the human the trail; only `sdlc.sh approve plan <slug>
--human` resumes it. Then phase 4.

Expand Down
5 changes: 4 additions & 1 deletion phases/4-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ where it is written in the same delivery, or its follow-up and owner go in
Delivery's `Policy:` line. Then:
- move verified facts from the refs (and `refs/learnings.md`, if any) into
`knowledge`: one place per fact, updating a page rather than adding a
duplicate;
duplicate; a fact that rests on code ends with its file (path from the
project root) and the delivered commit of that file's repository, after
re-checking the fact there if the file changed since it was checked:
`[verified: <file> @ <short sha>]`;
- add the saved scripts (`logs/scripts/`) worth keeping to the project's
`e2e` suite, or name them as a follow-up in Delivery;
- `sdlc.sh close <slug> shipped "<one line>"` — refused if the delivered
Expand Down
5 changes: 3 additions & 2 deletions roles/reviewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Inputs: intent.md, PLAN.md, `refs/`. Blocking:

## Diff mode (phase 3)

Inputs: intent.md, PLAN.md, `refs/`, the commit or uncommitted diff
Inputs: intent.md, PLAN.md, `refs/`, `logs/`, the commit or uncommitted diff
(untracked files included). Read every changed file.
Blocking:
- it does not do what PLAN.md says, or does more;
Expand All @@ -38,7 +38,8 @@ Blocking:
- security: injection, missing authorization, secrets in code, unsafe
deserialization;
- a test that cannot fail or asserts nothing; a bug fix without a test that
failed before it;
failed before it; a new-behavior test with no recorded assertion failure;
an observable blocking diff finding untested and unexplained;
- a name that lies, one rule decided in two places, a hardcoded environment
value, dead code the change left behind;
- an abstraction, option or layer PLAN.md did not ask for.
Expand Down
4 changes: 3 additions & 1 deletion roles/scout.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ under `refs/`. Change nothing else.
Inputs: the question, intent.md, the project root, the `knowledge` folder,
and config.md.

- Read `knowledge` first; cite what it already answers.
- Read `knowledge` first; cite what it already answers. Re-check, or mark
assumed, a cited code fact with no commit or whose file changed since (its
repository's `git log`).
- Trace from the entry the question names to the end: every caller in every
tier and repository, every reader and writer of the data, the states
existing records are in. Query real data read-only where a tool and
Expand Down
Loading