Skip to content

Add CI coverage reports and workflow-gated GitHub agents - #38

Open
josephschorr wants to merge 2 commits into
mainfrom
ci-coverage-reports
Open

josephschorr wants to merge 2 commits into
mainfrom
ci-coverage-reports

Conversation

@josephschorr

@josephschorr josephschorr commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

PR coverage agents can now wait for an install-selected set of GitHub Actions workflows, consume existing CI artifacts, and attach an authenticated HTML report through a separate Check Run. They do not rerun tests or require a coverage service.

The platform changes are reusable GitHub input configuration, durable workflow admission with retry/deduplication and revision checks, authenticated report links, resource-scoped memory, and optional conversational outputs.

examples/codecovbot owns all coverage-specific code: a standalone Go module and Docker image installed through the existing toolkit and bundle image mechanisms. It adds nothing coverage-specific to the OAP CLI, default images, or default installation.

The AgentClass's install-defined coveragePolicyPrompt defines functional coverage. Raw coverage preserves the CI measurement; functional coverage subtracts only policy-approved, source-bound whole blocks. The tool defaults to retaining everything. Custom policies use coverage-blind block inspection and audited exclusion manifests, including multi-statement blocks when every statement qualifies. Reports show both metrics, policy/version fingerprints, exclusion reasons, and compatible exact-base comparisons. The threshold, preferred metric, workflow selection, and notification preferences are configurable.

Validation:

  • Standalone program: complete race-enabled test suite, Docker build, and non-root sandbox execution.
  • Generic CLI, image catalog, bundle/install, toolkit validator, notification/output and memory paths: focused unit/integration checks, bundle lint, and formatting.
  • GitHub App/private-repository integration: signed multi-workflow waiting, reruns, worker restart, regression/recovery, memory persistence/recall, missing artifacts, unavailable exact-base evidence, optional output delivery, and authenticated report access. The independent toolkit image published a successful Check Run linking its HTML report.
  • A different AgentClass policy was also tested end to end: source-bound custom exclusions changed functional coverage while preserving raw counts, persisted the policy/results, and published a successful Check Run with its HTML report.
  • CI status is available in the checks below.

The updated CLI, CRDs, and cluster images must be installed together. Bundle format 2 prevents older installers from dropping input-handler configuration. Weekly goal-driven coverage execution remains a separate follow-up.

@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
openagentprimitives Ignored Ignored Preview Oct 22, 2026 10:39pm UTC

Request Review

Persist deferred webhook deliveries until install-selected workflows settle, with cohort deduplication and revision checks. Support handler configuration in agent bundles, resource-scoped memory, and optional conversational outputs for inputs that publish their own reports.
Install an independent coverage program and Docker image through the example toolkit. Read existing CI artifacts, compare raw and policy-defined functional coverage, publish authenticated HTML reports, and honor per-user notification preferences. Keep all coverage commands and libraries outside the OAP CLI and default installation.

This branch was successfully deployed

1 active (outdated) deployment
Preview — 95aeed28 Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant