[MakerFlyFly]

Summary

This Draft PR previews the frozen version prepared in this chat: 26 scanner tests, 21 changed files, and 697 additions / 86 deletions.
It is based on upstream commit bdbd66d0 and the reviewed diff SHA-256 8636fe29544c2840b0ed4d9f5e5da53880cec2565352f02fa3e1d42ec4209f9b.
The separate later revision in the existing fork PR is preserved.

• Workflow fetch or parse failures previously produced an empty, successful audit. Preserve successful findings, report repository/workflow failures with context, and return a non-zero status for incomplete scans.
• Check Python, PyYAML, and active gh authentication before discovery. Share one download and parse across the requested checks, and publish only complete workflow inventories atomically under a versioned cache filename.
• Document the completion contract and add regression tests and reporting evals, including incomplete zero-finding scans and stale outputs after early failure.

Type of change

• Skill change — ci-runner-audit, with updated eval fixtures
• Documentation — related spec and eval indexes
• Other: Python helper script shipped with the skill

Test plan

• prek run --all-files passes (Linux checkout with native symlinks, jq/gh installed, and a subprocess reaper)
• Scanner regression suite: 26 stdlib unittest tests pass
• Relevant ruff check, ruff format --check, and mypy pass
• tools/dev/run-skill-script-tests.sh passes
• Skill/tool and spec validators pass; skill token and surface-hash stamps are current
• Affected skill eval suite: 11/11 decision-field checks pass for each of gpt-6.1-sol and gpt-6-luna, using prompts assembled from the edited skill by the existing eval harness
• Real CLI / PyYAML end-to-end comparison: listing/download failures return 1, missing PyYAML returns 2, and a failed inventory refresh recovers on a healthy run without --refresh
• Read-only live smoke: all --repo apache/magpie, 13 workflows completed, scan_errors=0

RFC-AI-0004 compliance

• HITL — the audit remains read-only against GitHub; public contribution actions are reviewed before execution
• Vendor neutrality — existing scope placeholders and backend-specific audit boundaries are retained
• Write-access discipline — the helper produces local evidence and does not post messages or change repositories

Linked issues

Closes #2

Notes for reviewers

Existing findings TSV names and columns remain the same.
New <scope>-scan-errors.tsv columns are repo, path, stage, error, and html_url.
Exit codes are 0 for complete scans, 1 for incomplete scans/output failures, and 2 for unavailable prerequisites or invalid arguments.
workflows_scanned counts workflows that completed every requested check.

Old <owner>-workflow-files.tsv inventories are ignored and rebuilt as <owner>-workflow-files-v2.tsv; failed refreshes preserve the last complete cache without using it as the current run's result.
The active authentication check runs even when a workflow inventory is reused.
Runner-label and architecture-matching rules are retained.

Generative AI assistance: this change was developed with Codex (GPT-6); the test and eval results above describe the verification performed.