Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6a40703
fix(ci-runner-audit): report incomplete scans
MakerFlyFly Oct 10, 2026
3849b26
fix(ci-runner-audit): harden discovery and retain partial results
MakerFlyFly Oct 10, 2026
99838b4
chore: merge upstream main into runner audit fix
MakerFlyFly Oct 10, 2026
e378969
fix(ci-runner-audit): validate SHA text and report input encoding errors
MakerFlyFly Oct 10, 2026
de94111
chore: merge upstream main into runner audit fix
MakerFlyFly Oct 10, 2026
de1615f
fix(ci-runner-audit): handle empty repositories and legacy clients
MakerFlyFly Oct 10, 2026
5e62ed1
chore: merge upstream main into runner audit fix
MakerFlyFly Oct 10, 2026
39baf41
fix(ci-runner-audit): batch org discovery and bound gh calls
MakerFlyFly Oct 10, 2026
c8fbea6
fix(ci-runner-audit): preserve errors on unsupported Python
MakerFlyFly Oct 10, 2026
c935376
docs(ci-runner-audit): load scanner details on demand
MakerFlyFly Oct 10, 2026
ab2bf65
test(ci-runner-audit): reject unknown overridden gh routes
MakerFlyFly Oct 10, 2026
6ee7013
chore: merge upstream main into runner audit fix
MakerFlyFly Oct 10, 2026
baa94af
fix(ci-runner-audit): handle renames and quota failures
MakerFlyFly Oct 10, 2026
9b2d061
chore: merge upstream main into runner audit fix
MakerFlyFly Oct 10, 2026
dfbac3a
fix(ci-runner-audit): narrow changes to incomplete scan reporting
MakerFlyFly Oct 10, 2026
8efeb1b
chore: sync upstream before M-20 review fixes
MakerFlyFly Oct 11, 2026
8b59669
fix(ci-runner-audit): keep failed scan evidence current
MakerFlyFly Oct 11, 2026
b256976
fix(ci-runner-audit): qualify audit inventory context
MakerFlyFly Oct 11, 2026
d4c3134
fix(ci-runner-audit): retain archived repository filtering
MakerFlyFly Oct 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions plugins/magpie-repo-health/skills/ci-runner-audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ argument-hint: "[all|retired|macos-arch] [--repo owner/name | --repo-file repos.
capability: capability:triage
surface_hash: sha256:c6da72e64ac4808d
license: Apache-2.0
measured_tokens: 2295
measured_tokens: 2767
---

<!-- SPDX-License-Identifier: Apache-2.0
Expand Down Expand Up @@ -153,6 +153,9 @@ asks for branch-specific analysis.

## Commands

The Python interpreter running the script must be 3.11 or newer and have PyYAML installed.
Install your distribution's `python3-yaml` package or install `pyyaml` in a virtual environment and run the script with that interpreter.

Run from the framework checkout root.

For one repository:
Expand Down Expand Up @@ -186,20 +189,31 @@ skills/ci-runner-audit/scripts/scan_ci_runners.py all \
--owner apache \
--cache-dir /tmp/ci-runner-audit-cache \
--out-dir /tmp/ci-runner-audit \
--workers 20 \
--refresh
--workers 20
```

For only one check, replace `all` with `retired` or `macos-arch`.

Use `--refresh` for org scans when cached repo/workflow inventory may be
stale. Explicit `--repo` and `--repo-file` scans fetch repository
metadata directly.
Cold organization discovery can consume most of the REST API quota; retain validated inventory caches and use `--refresh` deliberately.
Repository discovery excludes archived repositories.
Explicit archived inputs are reported as unchecked, making the invocation incomplete.

---

## Outputs

`<scope>-scan-errors.tsv` records failures with `repo`, `path`, `stage`, `error`, and `html_url`.
Read TSVs with a CSV-aware parser because quoted error fields can contain multiline diagnostics.
The stderr summary includes `scan_complete`, `workflows_scanned` (files completing all selected checks), and `scan_errors`.
Organization scans report `repo_inventory=cache|live` and `workflow_inventory=cache|live` for the inventories actually read.
A workflow cache hit bypasses repository discovery and reports only `workflow_inventory=cache`.
Exit codes are `0` for complete, `1` for incomplete scanning/cache/output, and `2` for unavailable prerequisites or unreadable input.
Selected findings files are replaced with this invocation's results, including header-only files when prerequisites fail.
Prerequisite failures omit finding counts because no workflows were checked; the completion/error summary still reports the failure.

The script writes TSV files under `--out-dir`:

- `<scope>-retired-gh-runners-confirmed.tsv` — confirmed retired-label
Expand All @@ -211,6 +225,7 @@ The script writes TSV files under `--out-dir`:

Use `--scope-name` for stable output names for project or repo-set
scans.
If a repository file cannot be read, the fallback prefix uses only the known command-line repositories; specify `--scope-name` for a stable prefix.

---

Expand All @@ -236,6 +251,16 @@ Before reporting a broad candidate as actionable, inspect `runs-on`,

## Reporting

Check exit status and completion/error counts before interpreting findings.
A failed or incomplete scan must be reported as incomplete, even with empty findings; keep successful findings qualified as partial results.
A clean result requires successful completion, zero scan errors and zero findings.
Prerequisite failures stop before checking workflows.
Completion with cached inventory covers only that inventory, not repositories or workflow paths added since it was built.
State which inventory layers were cached; freshly listed workflows can still use a cached repository list.
Do not claim the current whole organization is clean from a cached inventory; use `--refresh` when current organization coverage is required.
If only cache persistence fails, report that all enumerated workflows were checked but the invocation is incomplete because the cache could not be saved.
When output writing or CLI parsing fails, do not interpret existing findings files as results of that invocation.

Report findings in this order:

1. Scope scanned: owner/repo set, default branches, and number of
Expand Down
Loading