Skip to content

Add AI review workflows, aimed at the stale-contradiction class - #667

Merged
Ethan-Arrowood merged 1 commit into
mainfrom
kris/add-ai-review-workflows
Sep 8, 2026
Merged

Add AI review workflows, aimed at the stale-contradiction class#667
Ethan-Arrowood merged 1 commit into
mainfrom
kris/add-ai-review-workflows

Conversation

@kriszyp

@kriszyp kriszyp commented Sep 6, 2026

Copy link
Copy Markdown
Member

The 2026-09-05 DLC post-mortem measured where escaped defects are actually caught. The finding was that the gap is not lens quality but CI review coverage across repos — most Harper repos outside harper/harper-pro run no automated review at all, so their defects are caught by humans and agents after the fact, or not at all.

repo claude gemini
harper yes yes
harper-pro yes yes
rocksdb-js present but pinned to July same
documentation none none
central-manager / host-manager / documentation / skills-internal none none

What this adds

Standard thin callers of the reusables in HarperFast/ai-review-prompts, pinned to af81872e (main, 2026-09-01) — the same ref harper and harper-pro are on.

  • review-layers: universal — the harper/* layers describe core internals and do not apply to prose.
  • repo-specific-checks names what the shared layers cannot know: doc-to-doc stale contradiction (the largest uncovered multi-author defect class in the preceding window — 12 findings across 6 PRs), versioned-docs snapshots, historic-redirects.ts, and the fact that npm run lint is a no-op stub (echo 0) so a green lint here is evidence of nothing.
  • validate-caller-workflows — the fail-closed check that these callers carry no shadow job (a non-uses: job alongside the reusable call would run with the caller's permissions without the auth gate) and that both refs pin to a 40-char SHA.

For the human reviewer

  • Both reviewers stay OPT-IN. CLAUDE_ALWAYS_ON / GEMINI_ALWAYS_ON are repo variables and I deliberately did not set them: turning them on spends money on every future PR, which is your call, not mine. Until then these fire only on the claude-review / gemini-review label. Merging this PR changes nothing on its own — the variable is what makes it live.
  • Make validate a required status check on main once merged; it is written to run unconditionally so it can be required without going permanently pending.
  • I could not execute these workflows. Validation is: the YAML parses, the required inputs and secrets match what the reusable at that ref declares (checked against its workflow_call block), and the caller shape is copied from harper's maintained callers rather than written fresh.
  • The org secrets must resolve for this repo (HARPERFAST_AI_CLIENT_ID, HARPERFAST_AI_APP_PRIVATE_KEY, plus ANTHROPIC_API_KEY / GEMINI_API_KEY). They are org-level, but I have no way to confirm this repo is in their selected-repository list — worth a check before flipping the variable.

🤖 Generated with Claude Code

The 2026-09-05 DLC post-mortem found the review gap across repos is CI coverage
rather than lens quality. This repo had companion-check, deploy, preview and
validate workflows but no claude or gemini caller.

It is also where the measurement pointed hardest: doc-to-doc stale
contradiction was the largest uncovered multi-author defect class in the
preceding window — 12 findings across 6 PRs — and nothing automated was looking
for it here.

Adds the standard thin callers of the reusables in HarperFast/ai-review-prompts
at af81872e (main, 2026-09-01), plus validate-caller-workflows (fail-closed: no
shadow jobs, SHA-pinned refs). Make its `validate` job a required status check
on main.

review-layers is `universal`; the harper/* layers describe core's internals and
do not apply to prose. repo-specific-checks carries what the shared layers
cannot know, led by the contradiction class: when a page changes a documented
default, option, flag, endpoint or version, the sibling pages asserting the old
value are now wrong. Plus versioned-docs snapshots, historic-redirects.ts, and
the fact that `npm run lint` is a no-op stub (`echo 0`) so a green lint here is
evidence of nothing.

Both reviewers stay OPT-IN via label; CLAUDE_ALWAYS_ON / GEMINI_ALWAYS_ON are
repo variables and are deliberately not set here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kriszyp
kriszyp requested a review from a team as a code owner September 6, 2026 13:32
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@Ethan-Arrowood
Ethan-Arrowood merged commit b1001d7 into main Sep 8, 2026
11 checks passed
@Ethan-Arrowood
Ethan-Arrowood deleted the kris/add-ai-review-workflows branch September 8, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants