Skip to content

Harden CI and release workflows - #19

Open
Morry98 wants to merge 3 commits into
mainfrom
feature/ci-hardening
Open

Morry98 wants to merge 3 commits into
mainfrom
feature/ci-hardening

Conversation

@Morry98

@Morry98 Morry98 commented Nov 9, 2026

Copy link
Copy Markdown
Owner

Harden CI and release workflows

What changes

Publishing depends on CI (publish.yml, ci.yml)

  • ci.yml is now also a reusable workflow (workflow_call). publish.yml calls it as its first job (ci), so tests (3.10–3.14), pre-commit and pip-audit must pass on the exact commit before anything is built or uploaded.
  • ci.yml no longer has its own push: main trigger: on main, the same jobs run inside the Publish run (ci / test (3.x), ci / pre-commit, ci / pip-audit). Pull requests still run ci.yml directly, with unchanged check names (pre-commit, test (3.x), coveralls-finished) plus the new pip-audit.

No publishing from pull requests

  • publish.yml triggers: push to main and workflow_dispatch only. pull_request was removed.
  • Official versions (x.y.z, optional .postN) publish only when the ref is main (push or manual run).
  • Pre-releases (anything else, e.g. 1.2.0rc1) publish only from a manual workflow_dispatch run (any branch). Pushes to main with a pre-release version are skipped, as before.
  • concurrency: publish with cancel-in-progress: false so two releases never overlap.

Least privilege

  • Every workflow declares permissions: contents: read at the top level.
  • The release is split into jobs: check-version → build (read-only, uploads dist/ as an artifact) → publish (only id-token: write, environment pypi) → release (only contents: write, creates the tag and GitHub Release with gh release create --target $GITHUB_SHA, no checkout and no git push).
  • All actions/checkout steps use persist-credentials: false.
  • The CI caller no longer passes secrets (secrets: inherit is not used); Coveralls only needs GITHUB_TOKEN.

Trusted publishing with attestations

  • uv publish is replaced by pypa/gh-action-pypi-publish (OIDC trusted publishing). It generates and uploads PEP 740 provenance attestations for every file by default.

Actions pinned to commit SHAs

  • Every third-party action in ci.yml, publish.yml and docs.yml is pinned to a full 40-char SHA with a # vX.Y.Z comment, staying on the major version already in use (checkout v6.1.0, setup-uv v7.6.0, action-pre-commit-uv v1.1.2, coveralls v2.3.8, upload-pages-artifact v4.0.0, deploy-pages v5.0.1). New: pypi-publish v1.14.3, upload-artifact v7.0.2, download-artifact v8.0.2. docs.yml only had its uses: lines changed.
  • dependabot.yml already covers github-actions and bumps SHA + comment. The Python entry moves from pip to uv so Dependabot updates uv.lock too (the pip ecosystem does not).

Dependency audit (FTM-28)

  • New pip-audit job in ci.yml running .github/scripts/pip-audit.sh: uv export --frozen --all-groups (runtime, dev and docs) audited with a pinned pip-audit==2.10.1. It fails on any known vulnerability.
  • The same export is audited once per supported Python version (uvx --python 3.10 … 3.14, list in PYTHON_VERSIONS). pip-audit evaluates environment markers with its own interpreter and silently skips lines that do not match. A single run on 3.11+ would never check the pins limited to 3.10 (python_full_version < '3.11': 17 packages, including starlette==1.7.0, cronsim==2.7, pydantic-settings==2.15.0). The job fails if any single version reports a vulnerability.
  • Allowlist: .github/pip-audit-allowlist.txt, one vulnerability ID per line with a mandatory reason comment (currently empty).
  • uv lock --upgrade: before, pip-audit reported 34 findings in 9 packages (starlette, anyio, idna, certifi, urllib3, pydantic-settings, setuptools, pymdown-extensions, mkdocs-material). After: No known vulnerabilities found on every Python version from 3.10 to 3.14. The upgrade also adds two new transitive packages: opentelemetry-api (required by fastapi 0.162.0) and tzdata (required by python-dateutil 4.0.0, docs group).
  • Three packages are held at their previous locked versions on purpose: ruff 0.15.10, ty 0.0.29, redis 7.4.0. Their latest versions break the gate for reasons unrelated to this PR (ruff 0.17 reformats Markdown code blocks and adds PLR0917 in task_group.py; ty 0.0.97 and the redis 8.x type stubs report new bytes | str errors in src/). None of the three has a known vulnerability. Adopting them needs changes in src/ and belongs in its own PR.

Workflow guard test

  • tests/test_workflows.py (adds pyyaml>=6.0.3 to the dev group) fails if:
    • publish.yml can be triggered by pull_request, pull_request_target or workflow_run;
    • any job with id-token: write or contents: write in publish.yml does not depend, directly or transitively, on the ci job (which must call ./.github/workflows/ci.yml);
    • ci.yml loses its workflow_call trigger;
    • any workflow has no top-level permissions;
    • any non-local uses: is not pinned to a full SHA with a # vX.Y.Z comment;
    • the pip-audit job no longer runs .github/scripts/pip-audit.sh;
    • PYTHON_VERSIONS in the script differs from the union of the Python classifiers in pyproject.toml and the CI test matrix, or the loop stops iterating over the whole list or passing --python to uvx.
      These rules apply to every file in .github/workflows/, including ones added later.

Not changed / decisions left open

  • fastapi floor not raised. Every starlette release before 1.3.1 is affected by at least one advisory (PYSEC-2026-249 affects >=0.4.1,<1.3.1), and every fastapi release from 0.115.13 to 0.162.0 still accepts starlette >=0.40/>=0.46. No fastapi floor can rule out vulnerable starlette. The only option is an explicit starlette>=1.3.1 dependency, which in practice also requires fastapi>=0.133.0 (earlier releases cap starlette below 1.0). That changes the supported range, so it is left to the maintainer. Applications choose their own starlette version anyway; the lock only covers CI and development.
  • No version bump and no CHANGELOG entry: the CHANGELOG has no Unreleased section, and this PR does not change the published package.
  • A --resolution lowest-direct CI job is not part of this PR.

How to verify

uv sync
uv run pytest                                  # 263 passed
uv run coverage run -m pytest && uv run coverage report   # TOTAL 97%
uv run ty check .                              # All checks passed!
uv run ruff check .                            # All checks passed!
uv run ruff format --check .                   # 40 files already formatted
pre-commit run --all-files                     # all hooks Passed
bash .github/scripts/pip-audit.sh              # No known vulnerabilities found (x5: Python 3.10-3.14)
uvx --from actionlint-py actionlint            # no findings

Lowest direct versions (fastapi 0.115.13, starlette 0.46.2, redis 6.2.0, pydantic 2.11.7): uv sync --resolution lowest-direct on a scratch clone, then pytest: 261 passed.

Mutations tried (each applied alone; tests/test_workflows.py went red every time)

# Mutation Result
M1 add pull_request: to publish.yml triggers red
M2 build needs: [ci, check-version] → needs: [check-version] red
M3 first actions/checkout@<sha> # v6.1.0 in ci.yml → actions/checkout@v6 red
M4 remove the top-level permissions: from ci.yml red
M5 remove workflow_call: from ci.yml red
M6 replace workflow_dispatch: with workflow_run: in publish.yml red

| M7 | PYTHON_VERSIONS=("3.10" … → drop "3.10" | red |
| M8 | drop "3.14" from PYTHON_VERSIONS | red |
| M9 | loop over "${PYTHON_VERSIONS[0]}" instead of "${PYTHON_VERSIONS[@]}" | red |
| M10 | uvx --python "$python_version" pip-audit… → uvx pip-audit… | red |

The pip-audit job was checked against the old lock (exit 1, 34 findings) and the new one (exit 0 on all five versions).

Marker check: in a copy of the script, right after uv export, the 3.10-only pin starlette==1.7.0 ; python_full_version < '3.11' was replaced with starlette==1.3.0 (vulnerable, PYSEC-2026-249). The previous single-interpreter script exits 0 on that export ("No known vulnerabilities found"). The new script exits 1 ("Found 2 known vulnerabilities in 1 package" on Python 3.10, clean on 3.11-3.14, then "pip-audit found vulnerabilities on Python: 3.10").

Settings the maintainer must change on GitHub / PyPI

These cannot be set from the repository and the workflow depends on them:

  1. Environment pypi: Settings → Environments → pypi
    • Required reviewers: add yourself and enable it. Optionally enable Prevent self-review if there is a second maintainer.
    • Deployment branches and tags: leave it on No restriction if you want manual pre-releases from feature branches. Choose Selected branches and tags → main if pre-releases should also come only from main.
    • Make sure no PYPI_TOKEN/API-token secret is stored in the environment or the repo (trusted publishing does not need one).
  2. PyPI trusted publisher: pypi.org → project fastapi-task-manager → Manage → Publishing. The GitHub publisher must be: owner Morry98, repository fastapi-task-manager, workflow publish.yml, environment pypi. Add the environment name if it is empty, then delete any leftover project API tokens (Account settings → API tokens).
  3. PyPI account: Account settings → enable 2FA with a security key or TOTP app (and keep the recovery codes).
  4. Branch protection for main: Settings → Rules → Rulesets → New branch ruleset (or Settings → Branches → Add rule), target main:
    • Require a pull request before merging.
    • Require status checks to pass, require branches to be up to date, with these checks: pre-commit, test (3.10), test (3.11), test (3.12), test (3.13), test (3.14), pip-audit. A check only shows in the picker after it has run once, so add them after this PR's first CI run.
    • Block force pushes and deletions.
  5. Tag protection (recommended): Settings → Rules → Rulesets → New tag ruleset, target v*: restrict creation, update and deletion to maintainers, and allow github-actions to create tags (the release job creates them).
  6. Actions permissions: Settings → Actions → General → Workflow permissions → "Read repository contents and packages permissions" (the workflows request more where needed). Optionally: Allow actions and reusable workflows → allow only the owners used here (actions, astral-sh, tox-dev, coverallsapp, pypa).

Limitations

  • The publish path cannot run end-to-end in a PR by design. The first real run will be the next push to main with a new version, or a manual workflow_dispatch.
  • On main, CI results now appear under the Publish run (ci / …) instead of a separate CI run. Coveralls still receives the main coverage from that run (same branch and commit), but this is only confirmed after the first push to main.
  • The new guard test also applies to workflows added later (for example an integration-test workflow): they need top-level permissions and SHA-pinned actions.
  • pip-audit uses --no-deps --disable-pip on the fully pinned export (no hashes), so it prints a warning recommending hashes. This is expected and harmless.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HVysDwSu3eNK7jNjrPxgQy

- Publish now calls the CI workflow (workflow_call) and only builds after
  tests, pre-commit and pip-audit pass on the commit being released.
- Publish no longer runs on pull_request: official versions publish on push
  to main, pre-releases only via workflow_dispatch.
- Split build / publish / release jobs; only publish gets id-token: write and
  only release gets contents: write. Publish uses trusted publishing through
  pypa/gh-action-pypi-publish, which uploads PEP 740 attestations.
- Workflow-level permissions default to contents: read; checkouts use
  persist-credentials: false; all actions pinned to full commit SHAs.
- Add a pip-audit CI job over the full uv export, with an allowlist file.
- Dependabot: use the uv ecosystem so uv.lock is updated.
- Refresh uv.lock (clears all known vulnerabilities in the locked set).
- Add tests/test_workflows.py guarding the rules above.
Newer ruff (0.17) reformats Markdown code blocks and adds PLR0917; newer ty and redis 8 type stubs report new errors in src/. Adopting them is a separate change. Also raise the pyyaml floor to 6.0.3 (first release with Python 3.14 wheels).
pip-audit evaluates environment markers with its own interpreter and silently skips requirement lines that do not match, so pins limited to Python 3.10 (e.g. python_full_version < '3.11') were never audited. Audit the same export on 3.10-3.14 and fail if any version reports a vulnerability. tests/test_workflows.py checks that the version list matches the classifiers and the CI test matrix.
@Morry98
Morry98 marked this pull request as ready for review November 9, 2026 06:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant