Report and enforce coverage non-regression for src Python changes

[mnriem]

Problem

Spec Kit requires tests for behavioral changes and regression evidence for bug fixes, but CI currently runs pytest without collecting or comparing code coverage. pytest-cov and coverage configuration already exist. Without a comparable base-versus-PR measurement, coverage regressions can pass unnoticed.

Proposed policy

For pull requests that change src/**/*.py, Python code coverage must not decrease relative to the base revision. A decrease requires human intervention: restore coverage before merging, or have a maintainer explicitly review and document an exceptional decision. Agents must not waive a decrease on their own.

Pull requests that do not change src/**/*.py are exempt from the coverage comparison. Existing behavioral and regression test requirements continue to apply to executable changes elsewhere.

Implementation

• Add one canonical coverage CI job with a fixed OS and Python version, while retaining the existing cross-platform test matrix.
• On relevant PRs, run coverage on the base and PR revisions under comparable conditions. Report both coverage values and their covered/coverable line counts in the Actions summary; retain the reports as artifacts.
• Compare unrounded coverage ratios rather than displayed percentages, and fail the check on a decrease. Make any exception path explicitly human-controlled and documented; do not let an agent silently skip or override the check.
• On pushes to main, publish the coverage report without attempting a PR comparison.
• Document the policy in AGENTS.md and the measurement and intervention procedure in CONTRIBUTING.md.

Acceptance criteria

• In-scope PRs receive a base-versus-PR coverage report and a passing or failing non-regression check.
• PRs without changes to src/**/*.py do not need a coverage comparison and do not fail due to a skipped comparison.
• Coverage decreases cannot be silently accepted or waived by an agent; human intervention is required.
• The existing cross-platform tests and positive/negative and bug-regression test requirements remain in place.

AI disclosure

Drafted with GitHub Copilot (GPT-6 Sol) in an interactive, human-supervised conversation; the requester directed the scope and revised the policy. Reasoning effort was not specified. AI assistance was limited to drafting this issue; no code changes were made.

[CharlesOmeneki]

Hi @mnriem,

I’d be interested in taking this issue forward as an external contributor, if the maintainers are open to it.

The proposed base-vs-PR coverage comparison seems like a focused improvement, and I can work toward the evidence requirements in CONTRIBUTING.md, including a reproducible test case, coverage comparison, and the required CI/documentation changes.

I noticed the issue is currently marked triage-can-wait, so I wanted to check with the maintainers before starting implementation. If this is something you’d be comfortable accepting as a contributor PR, I’d be happy to work on it and keep the change focused.

I’ll also follow the project’s AI-assistance disclosure requirements throughout the contribution.

Thanks!

[mnriem]

@CharlesOmeneki I appreciate you offering, but this is a maintainer only issue. I apologize for not flagging it as such