From 4f9f02ed535edc79c75075e981504d1c401260d4 Mon Sep 17 00:00:00 2001 From: d33bs Date: Fri, 16 Oct 2026 14:15:42 -0600 Subject: [PATCH 1/3] Add runner image maintenance skill --- .../github-runner-image-maintenance/SKILL.md | 190 +++++++ .../references/runner-sources.md | 94 ++++ .../scripts/audit.py | 511 ++++++++++++++++++ .../tests/fixtures/catalog.md | 39 ++ .../tests/fixtures/workflow.yml | 31 ++ .../tests/test_audit.py | 344 ++++++++++++ .github/actionlint.yaml | 7 + .github/workflows/runner-skill-tests.yml | 47 ++ 8 files changed, 1263 insertions(+) create mode 100644 .agents/skills/github-runner-image-maintenance/SKILL.md create mode 100644 .agents/skills/github-runner-image-maintenance/references/runner-sources.md create mode 100644 .agents/skills/github-runner-image-maintenance/scripts/audit.py create mode 100644 .agents/skills/github-runner-image-maintenance/tests/fixtures/catalog.md create mode 100644 .agents/skills/github-runner-image-maintenance/tests/fixtures/workflow.yml create mode 100644 .agents/skills/github-runner-image-maintenance/tests/test_audit.py create mode 100644 .github/actionlint.yaml create mode 100644 .github/workflows/runner-skill-tests.yml diff --git a/.agents/skills/github-runner-image-maintenance/SKILL.md b/.agents/skills/github-runner-image-maintenance/SKILL.md new file mode 100644 index 0000000..ad9f58b --- /dev/null +++ b/.agents/skills/github-runner-image-maintenance/SKILL.md @@ -0,0 +1,190 @@ +--- +name: github-runner-image-maintenance +description: Use when auditing or updating GitHub runner images. +version: 0.1.0 +author: Dave Bunten (d33bs), Hermes Agent +license: MIT +platforms: [linux, macos, windows] +metadata: + hermes: + tags: [github-actions, runners, maintenance, packages, ci] + related_skills: [python-minimum-version-maintenance, python-latest-version-maintenance] +--- + +# GitHub Runner Image Maintenance + +Proactively audit GitHub-hosted runner images across maintained, published +packages, then propose small upgrades on separate branches and draft PRs. +Prefer the newest viable, explicit, generally available OS label that is not +being deprecated. A registry entry, a runner catalog row, and a green local +lint check are three different pieces of evidence; none proves hosted CI passed. + +## When to Use + +- Asked to audit runner freshness or retirement exposure across maintained repos. +- Asked to replace deprecated runners or floating `*-latest` OS labels. +- Asked to act on brownout notices without waiting for an email or CI outage. +- Don't use for blanket action-version, language-version, dependency, or hardware + upgrades. Don't migrate self-hosted runners or Azure pipelines with this skill. + +## Prerequisites + +- Authenticated GitHub CLI: check with `terminal(command="gh auth status")`. + Workflow pushes require a credential authorized to change workflow files. + Never print tokens or change permissions to get around a rejected push. +- Python 3.11+ and `uv` for the read-only helper's declared PyYAML dependency. + The helper reads registry/GitHub APIs; it never clones, edits, commits, pushes, + creates PRs, or dispatches workflows. An explicit `--output` writes a local + report only. Use `read_file`, `search_files`, `patch`, and `write_file` for + project inspection and edits; use `terminal` for CLI commands and checks. +- Establish approved repos and publishing scope. Building this skill or asking + for an audit is not permission to update every discovered package. Preserve + local-only/no-commit/no-push instructions. Do not merge PRs or publish releases. + +## How to Run + +From the repository root, use `terminal` to run a read-only PyPI-maintainer audit: + +```sh +uv run --script .agents/skills/github-runner-image-maintenance/scripts/audit.py \ + --pypi-user HANDLE --output runner-audit.json +``` + +Put the report in the task's scratch directory rather than committing it. +Add repeated `--repo OWNER/REPO` arguments to limit the published-package +inventory to selected projects. Run `--help` for the exact supported options. + +The helper covers PyPI discovery. For other formal registries, use their +current official APIs and perform the same publication/maintainership checks +manually; do not claim automated npm, crates.io, CRAN, or Bioconductor discovery. +A source manifest or a GitHub Release alone is not proof of a registry package. + +## Procedure + +1. Establish maintainer and package scope. For PyPI, use the helper's live + owner/maintainer discovery and registry project URLs, then cross-check the + canonical GitHub repo and current maintenance status. When PyPI omits source + URL fields, the helper requires a published-description GitHub link plus a + matching, SHA-pinned package manifest; name similarity alone is not proof. + Unlinked or legacy setup-only sources remain explicit manual-review outcomes. + For other registries, + record the package name, published version, source URL, and maintainer proof. + Exclude archived repos and unrelated mirrors; report unreachable, ambiguous, + or missing sources instead of guessing. The helper conservatively excludes + forks: review a genuinely published canonical fork separately if requested. + Account for every discovered entry, including skipped/error outcomes; persist + batches and compute totals programmatically before reporting a shortlist. + Completion: selected repos are maintained, published, non-archived, and scoped. +2. Refresh official runner evidence on every run. Read the + [runner image catalog](https://github.com/actions/runner-images), its current + [announcements](https://github.com/actions/runner-images/issues?q=label%3AAnnouncement), + the relevant image's software README and completed image releases, and + [GitHub's runner reference](https://docs.github.com/en/actions/reference/runners/github-hosted-runners). + Record UTC access time, catalog commit SHA, source URLs, image status, + architecture, size, and any deprecation/brownout/removal dates. Follow an + announcement before trusting an unbadged catalog row. Check closed notices + too: closed does not necessarily mean a retirement was cancelled. Exclude + previews, betas, retired/deprecating images, and unconfirmed future releases. + Resolve contradictory official sources or flag uncertainty; do not use an + email, search snippet, remembered label mapping, or OS vendor lifecycle alone. + Completion: each proposed label has fresh, source-backed GA/support evidence. +3. Inventory runner selection on each repo's pinned default-branch SHA. Inspect + all workflow files, reusable workflows and their inputs/defaults, job-level + `runs-on`, groups/labels, matrix axes/include/exclude, variables, and invoked + local actions/scripts. Trace dynamic selectors to definitions and callers; + report unresolved expressions and remote reusable workflows for manual review. + Arrays in `runs-on` combine labels; they are not an OS matrix. Preserve + self-hosted/custom-group selectors. A label in an unrelated example/comment + is not an active job, and changing a matrix axis must also update matching + conditional branches, includes/excludes, artifact names, and cache keys where + they actually depend on that value. Inspect test and Dependabot jobs together + so essential setup steps do not drift. + Completion: every affected job has a resolved selector or explicit review flag. +4. Classify each selector: deprecated/retired, floating alias, older supported + GA, current explicit GA, preview, or unknown/custom. Prioritize imminent + removals and brownouts, but also consider supported pins behind the newest GA. + Resolve aliases from current evidence, not string substitution. Select the + newest viable explicit version within the same OS family, architecture, + runner size, and required toolchain flavor. Compare version numbers + numerically; keep Ubuntu Slim/container and Windows client/server variants + distinct. Never invent a label from an OS version or add a patch number that + GitHub does not expose. Catalog-generated suggestions are review candidates, + not proof that the replacement is supported for this particular project. + Completion: old/new label, reason, evidence, and compatibility risks are stated. +5. Check the project's actual compatibility constraints before editing. Inspect + minimum and newest tested language versions, action architecture support, + dependency wheels, native libraries, browser/GPU/headless setup, Xcode/SDKs, + signing, containers, shell behavior, caches, and hardcoded install paths. + Compare old/new included software; explicitly provision required tools rather + than relying on an image's incidental defaults. For Python, read both + `python-minimum-version-maintenance` and `python-latest-version-maintenance` + once to review the whole supported range. A runner change does not authorize + raising the Python floor or dropping an existing OS/architecture test job. + Keep paid runner size/cost and Intel/ARM transitions separate unless explicitly + authorized. If the newest GA cannot run the support matrix, record the blocker + and propose an explicit, non-deprecated fallback with evidence; never silently + use `latest`, preview, fewer tests, or a different architecture to get green. + Completion: the candidate preserves support promises or has an explicit blocker. +6. For approved repos, inspect status/remotes/guidance and fetch the current + upstream default branch. Create a fresh, non-tracking branch from that base, + not from another package upgrade PR. Check existing PRs for duplicates and + avoid overwriting uncommitted work. Make the narrow runner/related selector + edits with `patch`; preserve triggers, permissions, secrets, release guards, + dependencies, Python versions, and test assertions. Do not globally replace + matching strings or retrofit archival docs. Run a dry-run/diff inspection + before staging and account for every changed file. + Completion: a small diff changes only intended runner selection and necessary + compatibility setup, with unrelated work untouched. +7. Run configured workflow/YAML/action lint and relevant local package checks. + Exercise the supported minimum and newest tested runtime locally where + available, plus build/installed-artifact smoke paths when image tools affect + packaging. If current lint tooling lacks an officially confirmed GA label, + update the tool first, then document only the exact source-backed exception + if still necessary; never disable runner validation wholesale. + A local Mac or Linux container is not the GitHub-hosted image. + Record commands, actual interpreters/platforms, skips, and pre-existing errors + separately; keep unexercised remote combinations provisional. + Completion: local checks pass or the draft accurately names remaining evidence. +8. When publishing is authorized, commit only the scoped files, push to the + intended fork, and verify the remote SHA. Open one draft PR per project with + explicit base/head and a short title plus two-sentence body: the runner change + and actual verification. Include the official notice link when deprecation + motivated the change. Keep detailed logs and the full audit in scratch, not + in the PR description. Read back exact URL, draft/open state, base/head repos, + branches, head SHA, title, and body. Do not mark ready or merge automatically. + Completion: the correct draft PR exists for each approved changed project. +9. Inspect hosted CI on the exact published head. Use check-run app ownership + and workflow jobs, not just a review bot's green status. Read the job's actual + runner labels/architecture and image/setup logs; compare expected matrix jobs + with those executed. A required job skipped by a stale condition is missing + coverage, not a pass. Distinguish image brownouts, missing tools, architecture + failures, and code/test regressions. Fix small in-scope issues, rerun checks, + and verify new head state. Report queued, approval-required, draft-skipped, + absent, timed-out, or failing CI honestly. Do not bypass security guards, + dispatch publishing workflows, or mark a draft ready just to force execution. + Completion: PR URLs and observed CI outcomes are reported, with unverified + combinations/blockers explicit and all PRs left as drafts. + +## Pitfalls + +- Read [source notes](references/runner-sources.md) for a dated worked example; + its version labels and retirement dates must be refreshed, not reused as policy. +- `macos-14`, `macos-14-large`, and `macos-14-xlarge` are not interchangeable + hardware classes. The same warning applies to newer labels and `*-latest`. +- An explicit OS label still receives rolling image/tool updates; it is not an + immutable image digest. Record the actual image version from hosted CI. +- An image README's announcement table can mention a different image's retirement. + An issue's unchecked platform list and mitigation labels are not affected labels. +- Preserve UTC date boundaries and the source's exact tokens. Flag impossible or + contradictory brownout intervals rather than silently correcting their dates. +- Draft PRs can be skipped by project policy or need fork approval. Never call + zero check runs, external-only statuses, or a skipped matrix green validation. + +## Verification + +Every scoped repo has a published-package eligibility decision, a complete runner +inventory or explicit gaps, current official evidence, and a no-change/proposed/ +blocked result. Every published change has a read-back-verified draft PR and an +honest hosted-CI outcome. Skill tests and a live read-only audit must run before +calling the helper verified; they do not establish that package runner upgrades +passed hosted CI. diff --git a/.agents/skills/github-runner-image-maintenance/references/runner-sources.md b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md new file mode 100644 index 0000000..d689a12 --- /dev/null +++ b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md @@ -0,0 +1,94 @@ +# Runner Evidence and Migration Notes + +Use live official sources on every run. This worked example was checked on +October 16, 2026; it is evidence for the skill's first trial, not a permanent +replacement map or retirement calendar. + +## Source precedence + +1. [Current runner catalog](https://github.com/actions/runner-images/blob/main/README.md): + supported label spelling, architecture, image status, and image README links. + Record its commit SHA; do not use a cached search result as the catalog. +2. [Announcement issues](https://github.com/actions/runner-images/issues?q=label%3AAnnouncement): + GA promotion, deprecation, image removals, brownouts, alias migrations, and + breaking software changes. Read the body and relevant updates, not just tags. +3. The chosen image's software README and completed release: actual tools, + image version, OS version, and deployment evidence. A software release's + pre-release flag can mean deployment is underway; do not confuse it with an + OS being generally available everywhere. +4. [Runner reference](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) + and [larger-runner labels](https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners/running-jobs-on-larger-runners): + hardware class, access/billing, architecture restrictions, and label syntax. +5. Exact-head hosted job logs: the image and architecture actually exercised. + This final observation, not a local test or an available-image row, establishes + that the proposed project's jobs ran on the intended image. + +If sources disagree, compare dates and the specific image/architecture in the +newer announcement. Stop or flag manual review if the conflict remains. A current +catalog row can lack a deprecation badge even when an active notice exists. + +## First-trial snapshot + +Catalog commit: +`ee1ff4569e381a9649b400338bdc8ceb7e18a80e`. + +The catalog's macOS 26 rows distinguish standard ARM64 `macos-26`, ARM64 larger +`macos-26-xlarge`, and Intel `macos-26-intel`/`macos-26-large`. Preserve size and +architecture, then verify availability and project compatibility. Do not map +all retired macOS labels to one new label or assume a suffix guarantees the same +CPU count, memory, price, or toolchain forever. + +The [macOS 14 notice](https://github.com/actions/runner-images/issues/13518) +sets retirement for November 2, 2026 and affects `macos-14`, `macos-14-large`, +and `macos-14-xlarge`. Its October 16 brownout spans 14:00 UTC on October 16 +through 00:00 UTC on October 17. Keep the cross-midnight boundary explicit. +The catalog identifies the standard/xlarge variants as ARM64 and large as Intel. +The [macOS 26 ARM64 software inventory](https://github.com/actions/runner-images/blob/main/images/macos/macos-26-arm64-Readme.md) +provides a separate check on tools and preinstalled runtimes; provision missing +supported runtimes with the project's setup action rather than removing tests. + +The [Ubuntu 26.04 GA announcement](https://github.com/actions/runner-images/issues/14747) +confirms GA on September 17, 2026 for `ubuntu-26.04` and `ubuntu-26.04-arm`. +The catalog still associates `ubuntu-latest` with 24.04 in this snapshot, so +resolving `latest` alone does not find the newest available explicit GA label. +A public-preview entry in an older documentation/search copy is not enough to +contradict a newer, explicit GA announcement. + +The [Ubuntu 22.04 notice](https://github.com/actions/runner-images/issues/14254) +states retirement on April 17, 2027, although the catalog's 22.04 rows lack a +retirement badge. Several brownout lines in that notice give a same-day end time +earlier than the start time. Preserve that discrepancy, flag it for confirmation, +and do not invent corrected intervals. Its unchecked lists of other images do +not mean that those images are also deprecated. + +## Validator catalog lag + +The current actionlint release, 1.7.12, recognizes macOS 26 but does not yet +recognize the confirmed GA `ubuntu-26.04` label. Update an old validator first; +if its current release still lags GitHub's official catalog, a narrowly documented +additional-label entry can validate the real hosted label without disabling +workflow lint. The skill's draft CI uses that exact exception in +`.github/actionlint.yaml`. Despite actionlint's configuration key being named +`self-hosted-runner`, it does not change how GitHub schedules the workflow. +Remove the exception when the validator's built-in catalog catches up. Never +allow arbitrary unknown labels or suppress all runner-label diagnostics. + +## Required compatibility review + +- Compare image tools against the commands actually used by the workflow: + compilers, SDKs, Xcode, browsers, native dependencies, display/headless setup, + shell features, package manager paths, signing, containers, and caches. +- Check action/runtime and wheel availability for both the package's minimum + and newest tested runtime on the target architecture. Do not infer ARM support + from an x64 wheel, or successful installation from a resolver-only result. +- Preserve Intel/ARM and standard/larger classes by default. A larger-runner + label may require organization configuration and paid access even when the + image itself is GA; ask before changing cost or resource class. +- Keep test selection conditions and matrix include/exclude entries consistent. + A new runner label can cause `if: matrix.os == 'old-label'` to skip the only + coverage job while the rollup still appears green. +- Read duplicated test/Dependabot/release setup paths together. Do not drop a + display service, system package, tool setup, or release guard in one path. +- Retain the draft PR until a maintainer reviews the actual hosted CI result. + Migrating the runner does not authorize merging, publishing, or rewriting + another open Python-version PR. diff --git a/.agents/skills/github-runner-image-maintenance/scripts/audit.py b/.agents/skills/github-runner-image-maintenance/scripts/audit.py new file mode 100644 index 0000000..8ba02d2 --- /dev/null +++ b/.agents/skills/github-runner-image-maintenance/scripts/audit.py @@ -0,0 +1,511 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# dependencies = ["PyYAML>=6,<7"] +# /// +"""Read-only PyPI/GitHub runner inventory. Run with uv run --script audit.py --help. + +Candidates are catalog-derived, NOT verified compatible replacements. Complex +expressions, custom runners, and referenced workflows require manual inspection. +""" +from __future__ import annotations + +import argparse +import base64 +import itertools +import json +import re +import subprocess +import sys +import tomllib +import urllib.parse +import urllib.request +import xmlrpc.client +from datetime import datetime, timezone +from pathlib import Path + +import yaml + +IMAGE_REPO = "actions/runner-images" +LABEL = re.compile(r"^(ubuntu|macos|windows)-(latest|\d+(?:\.\d+)*)(.*)$") + + +def label_parts(label): + match = LABEL.fullmatch(label) + if not match: + return None + family, version, suffix = match.groups() + return family, (() if version == "latest" else tuple(map(int, version.split(".")))), suffix + + +def parse_catalog(text, sha): + """Fail closed if the official Available Images table changes shape.""" + match = re.search(r"^## Available Images\s*\n(.*?)(?=^## |\Z)", text, re.M | re.S) + if not match: + raise ValueError("Available Images section missing") + rows = [line.strip() for line in match[1].splitlines() if line.strip().startswith("|")] + if not rows or [v.strip() for v in rows[0].strip("|").split("|")] != [ + "Image", "Architecture", "YAML Label", "Included Software" + ]: + raise ValueError("unrecognized catalog table schema") + separators = rows[1].strip("|").split("|") if len(rows) > 1 else [] + if len(rows) < 3 or len(separators) != 4 or not all( + re.fullmatch(r":?-{3,}:?", cell.strip()) for cell in separators): + raise ValueError("catalog table separator missing") + refs = dict(re.findall(r"^\[([^\]]+)\]:\s*(https://\S+)", text, re.M)) + catalog = {} + for row in rows[2:]: + cells = [v.strip() for v in row.strip("|").split("|")] + if len(cells) != 4 or cells[1] not in {"x64", "arm64"}: + raise ValueError(f"unrecognized catalog row: {row}") + labels = re.findall(r"`([^`]+)`", cells[2]) + remainder = re.sub(r"`[^`]+`|\bor\b|[,\s]", "", cells[2]) + if not labels or remainder or any(not re.fullmatch(r"[a-z0-9.-]+", label) for label in labels): + raise ValueError(f"unparseable catalog labels: {row}") + software = re.fullmatch(r"\[([^\]]+)\]", cells[3]) + if not software or software[1] not in refs: + raise ValueError(f"missing included software reference: {row}") + url = refs[software[1]].replace("/blob/main/", f"/blob/{sha}/") + status = "deprecated" if "![deprecated]" in cells[0].lower() else ( + "preview" if re.search(r"!\[(preview|beta)\]", cells[0], re.I) else "ga") + for label in labels: + if label in catalog: + raise ValueError(f"duplicate catalog label: {label}") + catalog[label] = {"architecture": cells[1], "status": status, + "software_url": url, "image": cells[0], + "issues": sorted(set(re.findall( + r"https://github.com/actions/runner-images/issues/\d+", cells[0])))} + if not catalog: + raise ValueError("empty image catalog") + return catalog + + +def classify_label(label, catalog): + result = {"literal": label, "status": "unknown_manual_review", "candidate": None, + "requires_review": True} + source = catalog.get(label) + parts = label_parts(label) + if not source or not parts: + return result + family, version, suffix = parts + options = [] + for other, image in catalog.items(): + parsed = label_parts(other) + if (parsed and parsed[0] == family and parsed[1] and parsed[2] == suffix + and (family != "windows" or not version or + (parsed[1][0] >= 2000) == (version[0] >= 2000)) + and image["architecture"] == source["architecture"] and image["status"] == "ga" + and not image.get("announcement_review")): + options.append((parsed[1], other)) + result.update(catalog_status=source["status"], architecture=source["architecture"], + software_url=source["software_url"], announcement_urls=source["issues"]) + if options: + newest, candidate = max(options) + if not version or newest > version or source["status"] != "ga": + result.update(status="candidate_requires_review", candidate=candidate, + candidate_software_url=catalog[candidate]["software_url"]) + else: + result["status"] = "catalog_current" if source["status"] == "ga" else "manual_review" + else: + result["status"] = "manual_review" + if source.get("announcement_review"): + result.update(status="announcement_review_required", + announcement_urls=sorted(set(source["issues"] + source["announcement_review"]))) + return result + + +class WorkflowLoader(yaml.BaseLoader): + def construct_mapping(self, node, deep=False): + result = {} + for key_node, value_node in node.value: + key = self.construct_object(key_node, deep=deep) + if not isinstance(key, str) or key in result: + raise ValueError(f"duplicate or nonscalar YAML key: {key!r}") + result[key] = self.construct_object(value_node, deep=deep) + return result + + +def matrix_rows(job): + """Resolve bounded literal scalar matrices only; include merges original axes.""" + strategy = job.get("strategy", {}) + if not isinstance(strategy, dict): + raise ValueError("dynamic strategy") + matrix = strategy.get("matrix", {}) + if not isinstance(matrix, dict): + raise ValueError("dynamic matrix") + axes = {key: value for key, value in matrix.items() if key not in {"include", "exclude"}} + if any(not isinstance(v, list) or not v or any(not isinstance(x, str) or "${{" in x for x in v) + for v in axes.values()): + raise ValueError("unsupported matrix axes") + count = 1 + for values in axes.values(): + count *= len(values) + if count > 256: + raise ValueError("matrix expansion exceeds 256") + include, exclude = matrix.get("include", []), matrix.get("exclude", []) + for entries in (include, exclude): + if (not isinstance(entries, list) or len(entries) > 256 or + any(not isinstance(e, dict) or any(not isinstance(v, str) or "${{" in v + for v in e.values()) for e in entries)): + raise ValueError("unsupported matrix include/exclude") + originals = [dict(zip(axes, vals)) for vals in itertools.product(*axes.values())] if axes else [] + originals = [r for r in originals if not any(all(r.get(k) == v for k, v in e.items()) for e in exclude)] + rows = [dict(r) for r in originals] + extras = [] + for entry in include: + matched = False + for original, row in zip(originals, rows): + if all(k not in original or original[k] == v for k, v in entry.items()): + row.update(entry) + matched = True + if not matched: + extras.append(dict(entry)) + if not matrix: + return [{}] + return rows + extras + + +def inventory(text, catalog): + if len(text) > 2_000_000: + raise ValueError("workflow exceeds 2 MB limit") + # Reject anchors/aliases: recursive YAML structures are not a workflow resolver. + try: + if any(isinstance(t, (yaml.tokens.AliasToken, yaml.tokens.AnchorToken)) for t in yaml.scan(text)): + raise ValueError("YAML anchors/aliases require manual review") + document = yaml.load(text, Loader=WorkflowLoader) + except yaml.YAMLError as exc: + raise ValueError(f"invalid YAML: {exc}") from exc + if not isinstance(document, dict) or not isinstance(document.get("jobs"), dict): + raise ValueError("workflow/jobs must be a mapping") + findings = [] + for name, job in document["jobs"].items(): + if not isinstance(job, dict): + raise ValueError(f"job {name} must be a mapping") + base = {"job": name, "source_path": f"jobs.{name}.runs-on", "candidate": None, + "requires_review": True, "original_selector": job.get("runs-on")} + if "uses" in job: + findings.append(dict(base, status="reference_manual_review", reference=job["uses"], + source_path=f"jobs.{name}.uses")) + continue + try: + rows = matrix_rows(job) + except ValueError as exc: + findings.append(dict(base, status="unresolved_manual_review", reason=str(exc))) + continue + for row in rows: + selector = job.get("runs-on") + source = base["source_path"] + if isinstance(selector, list) and len(selector) == 1: + selector = selector[0] + expr = re.fullmatch(r"\s*\$\{\{\s*matrix\.([\w-]+)\s*\}\}\s*", selector or "") if isinstance(selector, str) else None + if expr: + selector = row.get(expr[1]) + source = f"jobs.{name}.strategy.matrix.{expr[1]}" + matrix = job["strategy"]["matrix"] + if isinstance(matrix.get(expr[1]), list) and selector in matrix[expr[1]]: + source += f"[{matrix[expr[1]].index(selector)}]" + for i, addition in enumerate(matrix.get("include", [])): + if expr[1] in addition and all(row.get(k) == v for k, v in addition.items()): + source = f"jobs.{name}.strategy.matrix.include[{i}].{expr[1]}" + entry = dict(base, source_path=source, matrix=row, resolved_selector=selector) + if isinstance(selector, (dict, list)): + entry.update(status="custom_runner_manual_review", reason="group or AND-label constraints") + elif not isinstance(selector, str) or "${{" in selector: + entry.update(status="unresolved_manual_review", reason="nonliteral runner expression") + elif selector == "self-hosted": + entry.update(status="custom_runner_manual_review", literal=selector) + else: + entry.update(classify_label(selector, catalog)) + findings.append(entry) + steps = job.get("steps", []) + if not isinstance(steps, list): + findings.append(dict(base, status="unresolved_manual_review", reason="nonliteral steps")) + else: + for i, step in enumerate(steps): + if isinstance(step, dict) and isinstance(step.get("uses"), str) and step["uses"].startswith("./"): + findings.append(dict(base, status="reference_manual_review", reference=step["uses"], + source_path=f"jobs.{name}.steps[{i}].uses")) + return findings + + +class TimeoutTransport(xmlrpc.client.SafeTransport): + def make_connection(self, host): + connection = super().make_connection(host) + connection.timeout = 45 + return connection + + +class Client: + """Only PyPI reads and GitHub REST GETs; no checkout or remote mutations.""" + def gh(self, endpoint): + if endpoint.startswith(("-", "http")): + raise ValueError("expected relative GitHub API endpoint") + result = subprocess.run( + ["gh", "api", "--hostname", "github.com", "--method", "GET", endpoint], + check=True, capture_output=True, text=True, timeout=45) + return json.loads(result.stdout) + + def pages(self, endpoint, per_page=100, max_pages=10): + result = [] + for page in range(1, max_pages + 1): + separator = "&" if "?" in endpoint else "?" + rows = self.gh(f"{endpoint}{separator}per_page={per_page}&page={page}") + if not isinstance(rows, list): + raise ValueError("expected paginated list") + result.extend(rows) + if len(rows) < per_page: + return result + raise ValueError(f"pagination bound reached: {endpoint}; inventory incomplete") + + def packages(self, username): + with xmlrpc.client.ServerProxy("https://pypi.org/pypi", transport=TimeoutTransport()) as proxy: + return proxy.user_packages(username) + + def pypi(self, name): + request = urllib.request.Request( + f"https://pypi.org/pypi/{urllib.parse.quote(name, safe='')}/json", + headers={"User-Agent": "read-only-runner-audit/1"}) + with urllib.request.urlopen(request, timeout=45) as response: + data = response.read(20_000_001) + if len(data) > 20_000_000: + raise ValueError("PyPI response exceeds 20 MB") + return json.loads(data) + + def blob(self, repo, sha): + data = self.gh(f"repos/{repo}/git/blobs/{sha}") + if data.get("encoding") != "base64": + raise ValueError("unsupported Git blob encoding") + return base64.b64decode(data["content"]).decode("utf-8") + + +def repo_from_url(url): + if not isinstance(url, str): + return None + parsed = urllib.parse.urlsplit(url) + if parsed.scheme not in {"https", "http"} or parsed.netloc.lower() != "github.com": + return None + match = re.match(r"^/([\w.-]+)/([\w.-]+)(?:/|$)", parsed.path) + if not match: + return None + return f"{match[1]}/{match[2].removesuffix('.git')}" + + +def unique_repositories(candidates): + """Compare GitHub repo identities case-insensitively, preserving source tokens.""" + return set({repo.casefold(): repo for repo in sorted(candidates, reverse=True)}.values()) + + +def description_sources(client, info, package): + """Use published README links only with matching, SHA-pinned package metadata. + + Name similarity narrows API requests; it is not publication/source proof. + An ambiguous or unreadable manifest remains an explicit discovery failure. + """ + normalize = lambda value: re.sub(r"[-_.]+", "-", value).lower() + name = normalize(info.get("name") or package) + urls = re.findall(r"https?://github\.com/[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+", + info.get("description") or "") + candidates = unique_repositories({repo for url in urls if (repo := repo_from_url(url)) and + normalize(repo.rsplit("/", 1)[-1]) == name}) + if len(candidates) > 20: + raise ValueError("too many published-description source candidates; manual review required") + verified, failures = {}, [] + for repo in sorted(candidates): + try: + metadata = client.gh(f"repos/{repo}") + if metadata.get("full_name", "").lower() != repo.lower(): + raise ValueError("canonical name differs; manual verification required") + branch = urllib.parse.quote(metadata["default_branch"], safe="") + sha = client.gh(f"repos/{repo}/commits/{branch}")["sha"] + blob = client.gh(f"repos/{repo}/contents/pyproject.toml?ref={sha}") + if blob.get("encoding") != "base64": + raise ValueError("manifest is not base64") + manifest = tomllib.loads(base64.b64decode(blob["content"]).decode()) + declared = manifest.get("project", {}).get("name") or manifest.get( + "tool", {}).get("poetry", {}).get("name") + if not isinstance(declared, str) or normalize(declared) != name: + raise ValueError("source manifest package name does not match published package") + verified[repo] = {"kind": "published_description_and_source_manifest", + "description_repository_url": f"https://github.com/{repo}", + "manifest_url": f"https://github.com/{repo}/blob/{sha}/pyproject.toml", + "manifest_name": declared, "sha": sha} + except Exception as exc: + failures.append(f"{repo}: {exc}") + if not verified: + raise ValueError(f"no verified published-description source: {failures or 'no matching links'}") + return verified + + +def discover(client, username, requested): + repositories, excluded, errors = {}, [], [] + try: + packages = sorted(set(tuple(entry) for entry in client.packages(username))) + except Exception as exc: + return {}, [], [f"PyPI user discovery: {exc}"] + for role, name in packages: + if role not in {"Owner", "Maintainer"}: + excluded.append({"package": name, "reason": f"unsupported PyPI role: {role}"}) + continue + try: + data = client.pypi(name) + info = data["info"] + if not info.get("version") or not data.get("urls"): + excluded.append({"package": name, "reason": "no published release files for registry version"}) + continue + urls = dict(info.get("project_urls") or {}) + if info.get("home_page"): + urls["home_page"] = info["home_page"] + candidates = unique_repositories({candidate for url in urls.values() + if (candidate := repo_from_url(url))}) + source_proofs = {} + if not candidates: + source_proofs = description_sources(client, info, name) + candidates = set(source_proofs) + if len(candidates) != 1: + raise ValueError(f"unmapped/ambiguous published GitHub project URLs: {sorted(candidates)}") + repo = candidates.pop() + metadata = client.gh(f"repos/{repo}") + if metadata.get("full_name", "").lower() != repo.lower(): + raise ValueError("canonical GitHub full_name differs from registry source; manual verification required") + if any(not isinstance(metadata.get(flag), bool) for flag in ("archived", "fork")): + raise ValueError("unverified archived/fork metadata") + if metadata.get("archived") or metadata.get("fork"): + excluded.append({"package": name, "repository": repo, + "reason": "archived" if metadata.get("archived") else "fork"}) + continue + if not metadata.get("default_branch"): + raise ValueError("missing default branch") + repo = metadata["full_name"] + entry = repositories.setdefault(repo, {"default_branch": metadata["default_branch"], "packages": []}) + entry["packages"].append({"name": name, "role": role, "version": info["version"], + "registry_url": f"https://pypi.org/pypi/{urllib.parse.quote(name, safe='')}/json", + "project_urls": urls, "verified_repository_url": f"https://github.com/{repo}", + "source_evidence": source_proofs.get(repo, {"kind": "published_project_urls"}), + "release_files": [file.get("filename") for file in data["urls"]]}) + except Exception as exc: + excluded.append({"package": name, "reason": str(exc)}) + errors.append(f"package {name}: {exc}") + selected = {r.lower() for r in requested} + for repo in selected - {r.lower() for r in repositories}: + errors.append(f"unmatched --repo (not verified published/nonarchived/nonfork): {repo}") + if selected: + for repo in list(repositories): + if repo.lower() not in selected: + excluded.append({"repository": repo, "reason": "outside explicit --repo filter"}) + del repositories[repo] + return dict(sorted(repositories.items())), excluded, errors + + +def apply_announcements(catalog, issues): + """Associate positive impact/title label mentions, never platform checkboxes. + + Dates and brownout intervals intentionally remain raw source: contradictory + announcement times must not be silently repaired by this inventory helper. + """ + for issue in issues: + body = issue.get("body") or "" + impact = re.search(r"^#{1,6}\s*Possible impact\s*\n(.*?)(?=^#{1,6}\s|\Z)", body, re.M | re.S | re.I) + scoped = (issue.get("title") or "") + "\n" + (impact[1] if impact else "") + labels = set(re.findall(r"(?- + python -m unittest discover + -s .agents/skills/github-runner-image-maintenance/tests -v + - name: Check CLI entry point + run: python .agents/skills/github-runner-image-maintenance/scripts/audit.py --help From dedd1012470830c890b43b81419c9f04d14958e0 Mon Sep 17 00:00:00 2001 From: d33bs Date: Fri, 16 Oct 2026 15:53:39 -0600 Subject: [PATCH 2/3] Check hook runtimes before runner migrations --- .../github-runner-image-maintenance/SKILL.md | 14 ++++++++++---- .../references/runner-sources.md | 6 ++++++ 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/.agents/skills/github-runner-image-maintenance/SKILL.md b/.agents/skills/github-runner-image-maintenance/SKILL.md index ad9f58b..94f1812 100644 --- a/.agents/skills/github-runner-image-maintenance/SKILL.md +++ b/.agents/skills/github-runner-image-maintenance/SKILL.md @@ -116,7 +116,13 @@ A source manifest or a GitHub Release alone is not proof of a registry package. dependency wheels, native libraries, browser/GPU/headless setup, Xcode/SDKs, signing, containers, shell behavior, caches, and hardcoded install paths. Compare old/new included software; explicitly provision required tools rather - than relying on an image's incidental defaults. For Python, read both + than relying on an image's incidental defaults. Inspect pre-commit's + `default_language_version` and per-hook `language_version` separately from + the package test matrix. Ensure the lint job explicitly installs those + interpreters before hooks run; a local installation or warm cache can hide + a missing runtime on the new image. Align tool setup with the existing hook + requirement rather than downgrading hooks or raising the package minimum. + For Python, read both `python-minimum-version-maintenance` and `python-latest-version-maintenance` once to review the whole supported range. A runner change does not authorize raising the Python floor or dropping an existing OS/architecture test job. @@ -130,9 +136,9 @@ A source manifest or a GitHub Release alone is not proof of a registry package. not from another package upgrade PR. Check existing PRs for duplicates and avoid overwriting uncommitted work. Make the narrow runner/related selector edits with `patch`; preserve triggers, permissions, secrets, release guards, - dependencies, Python versions, and test assertions. Do not globally replace - matching strings or retrofit archival docs. Run a dry-run/diff inspection - before staging and account for every changed file. + dependencies, supported/test Python versions, and test assertions. Do not + globally replace matching strings or retrofit archival docs. Run a dry-run/diff + inspection before staging and account for every changed file. Completion: a small diff changes only intended runner selection and necessary compatibility setup, with unrelated work untouched. 7. Run configured workflow/YAML/action lint and relevant local package checks. diff --git a/.agents/skills/github-runner-image-maintenance/references/runner-sources.md b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md index d689a12..673b70f 100644 --- a/.agents/skills/github-runner-image-maintenance/references/runner-sources.md +++ b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md @@ -81,6 +81,12 @@ allow arbitrary unknown labels or suppress all runner-label diagnostics. - Check action/runtime and wheel availability for both the package's minimum and newest tested runtime on the target architecture. Do not infer ARM support from an x64 wheel, or successful installation from a resolver-only result. +- Check hook-specific runtime declarations as well as the package test matrix. + A lint job that sets up Python 3.11 can still require Python 3.12 through + pre-commit's `default_language_version`. A newer OS image may no longer supply + the latter on PATH. Explicitly install the configured hook runtime; preserve + the package's minimum and matrix. Verify on the hosted image, not just a local + machine that already happens to have both interpreters installed. - Preserve Intel/ARM and standard/larger classes by default. A larger-runner label may require organization configuration and paid access even when the image itself is GA; ask before changing cost or resource class. From a74ea3008cfe2bff54717d76db57062c0608881a Mon Sep 17 00:00:00 2001 From: d33bs Date: Fri, 16 Oct 2026 17:19:54 -0600 Subject: [PATCH 3/3] Check temporary-storage quotas during runner upgrades --- .agents/skills/github-runner-image-maintenance/SKILL.md | 6 ++++++ .../references/runner-sources.md | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/.agents/skills/github-runner-image-maintenance/SKILL.md b/.agents/skills/github-runner-image-maintenance/SKILL.md index 94f1812..d8be01f 100644 --- a/.agents/skills/github-runner-image-maintenance/SKILL.md +++ b/.agents/skills/github-runner-image-maintenance/SKILL.md @@ -115,6 +115,12 @@ A source manifest or a GitHub Release alone is not proof of a registry package. minimum and newest tested language versions, action architecture support, dependency wheels, native libraries, browser/GPU/headless setup, Xcode/SDKs, signing, containers, shell behavior, caches, and hardcoded install paths. + For data-heavy jobs, measure free space, inodes, mount types, and quotas for + `/tmp`, the job temp directory, and workspace on the actual target image. + A memory-backed `/tmp` can exhaust its quota while the regular disk is free. + Use a small hosted probe when documentation is insufficient; only then scope + an appropriate temporary-directory override to the affected step. Do not skip + the large-data test, disable quotas, or assume a paid runner is necessary. Compare old/new included software; explicitly provision required tools rather than relying on an image's incidental defaults. Inspect pre-commit's `default_language_version` and per-hook `language_version` separately from diff --git a/.agents/skills/github-runner-image-maintenance/references/runner-sources.md b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md index 673b70f..c76964f 100644 --- a/.agents/skills/github-runner-image-maintenance/references/runner-sources.md +++ b/.agents/skills/github-runner-image-maintenance/references/runner-sources.md @@ -81,6 +81,14 @@ allow arbitrary unknown labels or suppress all runner-label diagnostics. - Check action/runtime and wheel availability for both the package's minimum and newest tested runtime on the target architecture. Do not infer ARM support from an x64 wheel, or successful installation from a resolver-only result. +- Compare temporary-storage mounts and quotas, not just total VM disk size. + In the October 16 pilot, Ubuntu 26.04 used a roughly 8 GB quota-limited tmpfs + for `/tmp`, while the job temp directory was on an ext4 disk with about 88 GB + free. Ubuntu 24.04 used the disk for both. The unchanged large-data test wrote + beneath `/tmp` and hit `Errno 122`; a larger paid runner was not the first + remedy. Diagnose with `df`/`findmnt`, then scope `TMPDIR` to a measured, + disk-backed directory for the affected test step and rerun the complete test. + Refresh these measurements rather than hardcoding capacities as guarantees. - Check hook-specific runtime declarations as well as the package test matrix. A lint job that sets up Python 3.11 can still require Python 3.12 through pre-commit's `default_language_version`. A newer OS image may no longer supply