Skip to content

feat: hybrid onboarding review with Go tool + AI judgment - #114

Open
amanstep wants to merge 6 commits into
mainfrom
feat/update-review-process
Open

amanstep wants to merge 6 commits into
mainfrom
feat/update-review-process

Conversation

@amanstep

@amanstep amanstep commented Jun 14, 2026 •

Copy link
Copy Markdown
Contributor

Replace the single-prompt onboarding review with a two-stage flow: a Go tool runs ~50 deterministic compliance checks and emits JSON; the Claude action reads that report and focuses only on judgment calls (license semantics, control-flow reasoning, security review, advisor swaps). Boolean compliance is now reproducible, CI-gate-able, and runnable locally before opening a PR. Claude's tokens go to real judgment instead of file-presence checks.

cmd/review-onboarding (new): detects action type and upstream from auto_cherry_pick.yml, then runs deterministic rules across four categories.

Common: LICENSE, action.yml author, SECURITY.md contact, 22
forbidden files/dirs (FUNDING, renovate, PULL_REQUEST,
ISSUE_TEMPLATE, CHANGELOG, .vscode, .editorconfig, .gitattributes,
dependabot, .devcontainer, CODE_OF_CONDUCT, NOTICE, CONTRIBUTING,
CODEOWNERS at multiple paths), required workflows present,
pre-commit tooling absent (husky/lefthook/pre-commit/lint-staged,
including package.json deps and install scripts), README banner,
subscription URL using new /maintained-actions-subscription
endpoint, subscription upstream matches auto_cherry_pick.yml, docs
do not reference upstream owner/repo, no on.schedule trigger
outside audit_package.yml.

Node: dist/ present, .npmrc has min-release-age=3, action.yml
runs.using=node24, package.json author/repository contain
step-security (skipped when fields absent).

Docker: .github/workflows/docker.yml exists; jq installed in
Dockerfile when ENTRYPOINT/CMD invokes a .sh script (skipped
otherwise, since the check then lives in the invoked binary).

Composite: first step under runs.steps in action.yml contains the
subscription URL.

CLI: --repo-path, --output (default stdout), --format/-f json|table (default json; table uses go-pretty/v6 with colored status and word-wrap, colors auto-strip when writing to a file), --fail-on-error. Summary always goes to stderr so stdout stays pipeable into jq.

.github/workflows/claude_review.yml: the onboarding path (gated on the existing review-required label) now checks out step-security/reusable-workflows, sets up Go, runs the tool to produce review_report.json, uploads the artifact, and appends a failure table to $GITHUB_STEP_SUMMARY. The Claude direct_prompt was rewritten to (1) read the JSON, (2) spot-check a few tool decisions for regex errors, (3) cover only the rules the tool cannot - license copyright semantics, subscription invoked first, upstream-only workflow classification, latest-major + advisor swaps for uses: references, README major-version tags, security scan, and the action-type-specific judgment items (node workflow inputs, docker subscription-check file location, composite action pinning) - then emit a fixed-shape markdown PR comment. The cherry-pick path is untouched.

go.mod: adds github.com/jedib0t/go-pretty/v6 (same library OSV-Scanner uses) for terminal-table rendering.

amanstep added 6 commits June 14, 2026 17:29
Replace the single-prompt onboarding review with a two-stage flow: a
Go tool runs ~50 deterministic compliance checks and emits JSON; the
Claude action reads that report and focuses only on judgment calls
(license semantics, control-flow reasoning, security review, advisor
swaps). Boolean compliance is now reproducible, CI-gate-able, and
runnable locally before opening a PR. Claude's tokens go to real
judgment instead of file-presence checks.

cmd/review-onboarding (new): detects action type and upstream from
auto_cherry_pick.yml, then runs deterministic rules across four
categories.

  Common: LICENSE, action.yml author, SECURITY.md contact, 22
  forbidden files/dirs (FUNDING, renovate, PULL_REQUEST,
  ISSUE_TEMPLATE, CHANGELOG, .vscode, .editorconfig, .gitattributes,
  dependabot, .devcontainer, CODE_OF_CONDUCT, NOTICE, CONTRIBUTING,
  CODEOWNERS at multiple paths), required workflows present,
  pre-commit tooling absent (husky/lefthook/pre-commit/lint-staged,
  including package.json deps and install scripts), README banner,
  subscription URL using new /maintained-actions-subscription
  endpoint, subscription upstream matches auto_cherry_pick.yml, docs
  do not reference upstream owner/repo, no on.schedule trigger
  outside audit_package.yml.

  Node: dist/ present, .npmrc has min-release-age=3, action.yml
  runs.using=node24, package.json author/repository contain
  step-security (skipped when fields absent).

  Docker: .github/workflows/docker.yml exists; jq installed in
  Dockerfile when ENTRYPOINT/CMD invokes a .sh script (skipped
  otherwise, since the check then lives in the invoked binary).

  Composite: first step under runs.steps in action.yml contains the
  subscription URL.

CLI: --repo-path, --output (default stdout), --format/-f json|table
(default json; table uses go-pretty/v6 with colored status and
word-wrap, colors auto-strip when writing to a file),
--fail-on-error. Summary always goes to stderr so stdout stays
pipeable into jq.

.github/workflows/claude_review.yml: the onboarding path (gated on
the existing review-required label) now checks out
step-security/reusable-workflows, sets up Go, runs the tool to
produce review_report.json, uploads the artifact, and appends a
failure table to $GITHUB_STEP_SUMMARY. The Claude direct_prompt was
rewritten to (1) read the JSON, (2) spot-check a few tool decisions
for regex errors, (3) cover only the rules the tool cannot - license
copyright semantics, subscription invoked first, upstream-only
workflow classification, latest-major + advisor swaps for `uses:`
references, README major-version tags, security scan, and the
action-type-specific judgment items (node workflow inputs, docker
subscription-check file location, composite action pinning) - then
emit a fixed-shape markdown PR comment. The cherry-pick path is
untouched.

go.mod: adds github.com/jedib0t/go-pretty/v6 (same library
OSV-Scanner uses) for terminal-table rendering.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant