Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand Down Expand Up @@ -58,6 +59,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand Down Expand Up @@ -99,6 +101,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand Down Expand Up @@ -127,6 +130,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand All @@ -145,6 +149,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand Down Expand Up @@ -178,6 +183,7 @@ jobs:
version: latest-known
python-version: "3.15"
enable-cache: true
cache-suffix: ${{ github.job }}
cache-dependency-glob: |
pylock.toml
pyproject.toml
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/upgrade-release-tools.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ on:
description: Groups for the secondary lock; empty selects build requirements
type: string
default: ""
additional-lock-selection:
description: Select project dependencies and configured groups for the secondary lock
type: string
default: build-or-groups
additional-matrix-python:
description: Python range for the secondary lock
type: string
Expand Down Expand Up @@ -67,6 +71,7 @@ jobs:
lock-groups: ${{ inputs.lock-groups }}
additional-lock: ${{ inputs.additional-lock }}
additional-lock-groups: ${{ inputs.additional-lock-groups }}
additional-lock-selection: ${{ inputs.additional-lock-selection }}
additional-matrix-python: ${{ inputs.additional-matrix-python }}
additional-matrix-platforms: ${{ inputs.additional-matrix-platforms }}
additional-matrix-implementations: ${{ inputs.additional-matrix-implementations }}
Expand Down
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,17 @@ jobs:
The required `lock-groups` input lists the caller's release dependency groups, separated by spaces or newlines. Include
`build` and `lock-tools` where declared; exclude test, type, and documentation groups. This example lists this
repository's groups; a caller supplies its own. The project configures its Python and platform targets in `[tool.nab]`.
For a separately checked-in build matrix lock, also pass `additional-lock` with an existing `pylock.*.toml` basename,
`additional-matrix-python` with its Python range, and `additional-matrix-platforms` and
Tests install test, type, and documentation groups afresh so they detect upstream dependency changes. The Action tests
fetch immutable, hash-checked release-tool artifacts from the shared lock; their separate package fixture resolves
test-only tools afresh. For a separately checked-in build matrix lock, also pass `additional-lock` with an existing
`pylock.*.toml` basename, `additional-matrix-python` with its Python range, and `additional-matrix-platforms` and
`additional-matrix-implementations` as space-separated Nab values. The secondary lock keeps its specified environment
matrix during refresh. `additional-lock-groups` selects dependency groups for it; when empty, Nab locks the project's
build requirements. These inputs have no effect when `additional-lock` is absent, and matrix inputs without that lock
fail. The writer accepts only the named secondary lock path.
build requirements. Set `additional-lock-selection: project` to include the project's runtime dependencies and any
configured build group. The consumer must select its runtime group when reading that lock. This selection cannot be
combined with `additional-lock-groups`. Secondary-lock inputs without `additional-lock` fail. The writer accepts only
the named secondary lock path. Use `project` only when those runtime dependencies become part of the published artifact,
such as a bundled zipapp; tests still install runtime dependencies afresh.

For a repository that already commits `pylock.build.toml`, the complete caller is:

Expand Down
1 change: 1 addition & 0 deletions changelog.d/+lock-cutoff.security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Enforce the seven-day Nab cutoff for global and package rules, including disabled or shortened cutoffs.
1 change: 1 addition & 0 deletions changelog.d/+release-asset-retry.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove partial GitHub Release assets before retrying an upload in the same run.
1 change: 1 addition & 0 deletions changelog.d/+release-tool-setup.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Allow repeated lock-tool setup on Windows and check caller release-creator versions against the selected groups.
1 change: 1 addition & 0 deletions changelog.d/+runtime-secondary-lock.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Refresh a secondary pylock for project runtime dependencies, including a separate Python and platform matrix.
5 changes: 1 addition & 4 deletions install-nab/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,7 @@ runs:
steps:
- name: Create lock-tool environment
shell: bash
run: |
if [[ ! -x "$RUNNER_TEMP/lock-tools/bin/python" ]]; then
uv venv --python 3.15 "$RUNNER_TEMP/lock-tools"
fi
run: uv venv --allow-existing --python 3.15 "$RUNNER_TEMP/lock-tools"
- name: Install lock reader
shell: bash
run: |
Expand Down
14 changes: 13 additions & 1 deletion publish-github-release/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -172,9 +172,21 @@ runs:
if jq -e '.[0].draft' "$RUNNER_TEMP/release-current.json" >/dev/null; then
for asset in "$ASSETS_DIRECTORY"/*; do
name=${asset##*/}
if ! jq -e --arg name "$name" 'any(.[0].assets[]; .name == $name)' \
if ! jq -e --arg name "$name" --slurpfile assets "$RUNNER_TEMP/release-assets.json" '
any(.[0].assets[]; .name == $name and .state == "uploaded" and
(.digest as $digest | any($assets[0][]; .name == $name and .digest == $digest)))
' \
"$RUNNER_TEMP/release-current.json" >/dev/null; then
for attempt in {1..6}; do
if ((attempt > 1)); then
release=$(gh api "repos/$GH_REPO/releases/$release_id")
asset_ids=$(jq -r --arg name "$name" \
'.assets[] | select(.name == $name and .state != "uploaded") | .id' <<<"$release")
while IFS= read -r asset_id; do
[[ -n "$asset_id" ]] || continue
gh api -X DELETE "repos/$GH_REPO/releases/assets/$asset_id" >/dev/null
done <<<"$asset_ids"
fi
if gh release upload "$TAG" "$asset" --repo "$GH_REPO"; then
break
fi
Expand Down
13 changes: 6 additions & 7 deletions pylock.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2335,18 +2335,17 @@ command-line = [
"--groups",
"build",
"lock-tools",
"release-attestations",
"release-check",
"release-hook-check",
"release-hooks",
"release-news",
"release-notes",
"release-news",
"release-prepare",
"release-shell",
"release-validate",
"release-check",
"release-attestations",
"release-shell",
"release-hooks",
"release-hook-check",
"--output",
"pylock.toml",
"--upgrade",
]
input-path = "pyproject.toml"
mode = "universal"
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ release-shell = [ "prek>=0.2", "yq>=4.4.5" ]
release-hooks = [ "release-creator>=0.6.0", "prek>=0.2" ]
release-hook-check = [ "prek>=0.2", "yq>=4.4.5" ]
test = [
"certifi",
"check-wheel-contents>=0.6.3",
"hatchling>=1.28",
"pytest>=9",
"pytest-httpserver",
Expand All @@ -29,6 +31,7 @@ test = [
"tox>=4.64.7",
"tox-uv>=1.36.1",
"trustme",
"twine>=7",
"ty",
]

Expand Down
26 changes: 23 additions & 3 deletions refresh-release-lock/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ inputs:
additional-lock-groups:
description: Groups for the secondary lock; empty selects build requirements
default: ""
additional-lock-selection:
description: Select project dependencies and configured groups for the secondary lock
default: build-or-groups
additional-matrix-python:
description: Python range for the secondary lock
default: ""
Expand Down Expand Up @@ -48,6 +51,7 @@ runs:
MODE: ${{ inputs.mode }}
ADDITIONAL_LOCK: ${{ inputs.additional-lock }}
ADDITIONAL_GROUPS: ${{ inputs.additional-lock-groups }}
ADDITIONAL_SELECTION: ${{ inputs.additional-lock-selection }}
MATRIX_PYTHON: ${{ inputs.additional-matrix-python }}
MATRIX_PLATFORMS: ${{ inputs.additional-matrix-platforms }}
MATRIX_IMPLEMENTATIONS: ${{ inputs.additional-matrix-implementations }}
Expand Down Expand Up @@ -80,7 +84,8 @@ runs:
"$RUNNER_TEMP/pylock-before-check.toml" pylock.toml
fi
if [[ -z "$ADDITIONAL_LOCK" ]]; then
if [[ -n "$ADDITIONAL_GROUPS$MATRIX_PYTHON$MATRIX_PLATFORMS$MATRIX_IMPLEMENTATIONS" ]]; then
if [[ "$ADDITIONAL_SELECTION" != build-or-groups ||
-n "$ADDITIONAL_GROUPS$MATRIX_PYTHON$MATRIX_PLATFORMS$MATRIX_IMPLEMENTATIONS" ]]; then
echo '::error::secondary matrix inputs require additional-lock' >&2
exit 1
fi
Expand Down Expand Up @@ -111,8 +116,23 @@ runs:
echo '::error::secondary matrix needs platforms and implementations' >&2
exit 1
fi
selection=(--build-requirements)
if ((${#selected_groups[@]})); then selection=(--groups "${selected_groups[@]}"); fi
case "$ADDITIONAL_SELECTION" in
build-or-groups)
selection=(--build-requirements)
if ((${#selected_groups[@]})); then selection=(--groups "${selected_groups[@]}"); fi
;;
project)
if ((${#selected_groups[@]})); then
echo '::error::project secondary lock cannot select dependency groups' >&2
exit 1
fi
selection=()
;;
*)
echo '::error::unknown secondary lock selection' >&2
exit 1
;;
esac
cp "$ADDITIONAL_LOCK" "$RUNNER_TEMP/additional-lock-before.toml"
uv run --no-project --python "$RUNNER_TEMP/lock-tools" nab lock pyproject.toml "${selection[@]}" \
--project-matrix-python "$MATRIX_PYTHON" \
Expand Down
6 changes: 5 additions & 1 deletion tests/build_python_distribution/test_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,11 @@


@pytest.fixture
def build_repository(source_project: Path, nab_executable: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
def build_repository(
source_project: Path, nab_executable: Path, test_package_index: str, monkeypatch: pytest.MonkeyPatch
) -> Path:
monkeypatch.delenv("SSL_CERT_FILE", raising=False)
monkeypatch.setenv("UV_INDEX_URL", test_package_index)
project = source_project / "pyproject.toml"
release_requirement = tomllib.loads((Path(__file__).resolve().parents[2] / "pyproject.toml").read_text())[
"dependency-groups"
Expand All @@ -21,6 +24,7 @@ def build_repository(source_project: Path, nab_executable: Path, monkeypatch: py
project.read_text()
+ f"\n[dependency-groups]\nrelease-notes = [{json.dumps(release_requirement)}]\n"
+ '\n[tool.nab]\nbase-group = "project-runtime"\nbuild-group = "build"\n'
+ f'[[tool.nab.indexes]]\nname = "fixture"\nurl = "{test_package_index}"\nserialization = "html"\n'
)
run(
source_project,
Expand Down
68 changes: 66 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,21 @@
import ssl
import subprocess
import sys
import tomllib
from collections.abc import Iterator, Mapping
from hashlib import sha256
from html import escape
from pathlib import Path
from textwrap import dedent
from typing import Final
from urllib.parse import urlsplit
from urllib.request import urlopen

import certifi
import pytest
import trustme
import yaml
from packaging.utils import canonicalize_name
from pytest_httpserver import HTTPServer
from werkzeug.wrappers import Request, Response

Expand All @@ -27,8 +34,7 @@ def github_api(tmp_path: Path) -> Iterator[tuple[HTTPServer, dict[str, str]]]:
server.start()
bundle = tmp_path / "ca.pem"
authority.cert_pem.write_to_path(bundle)
if system_certs := ssl.get_default_verify_paths().cafile:
bundle.write_bytes(bundle.read_bytes() + Path(system_certs).read_bytes())
bundle.write_bytes(bundle.read_bytes() + Path(certifi.where()).read_bytes())
yield (
server,
{
Expand Down Expand Up @@ -80,6 +86,63 @@ def action_scripts(tmp_path_factory: pytest.TempPathFactory) -> Path:
return scripts


@pytest.fixture(scope="session")
def test_package_lock(tmp_path_factory: pytest.TempPathFactory, nab_executable: Path) -> Path:
fixture = tmp_path_factory.mktemp("fixture-lock")
(fixture / "pyproject.toml").write_text(
'[project]\nname = "release-fixture"\nversion = "0.1.0"\nrequires-python = ">=3.15,<3.16"\n'
'[dependency-groups]\nfixture = ["check-wheel-contents>=0.6.3", "hatchling>=1.28", "twine>=7"]\n'
'[tool.nab]\nmode = "universal"\nbase-group = "project-runtime"\n'
'default-groups = []\n[tool.nab.matrix]\npython = ">=3.15,<3.16"\nplatforms = ["linux_x86_64"]\n'
)
subprocess.run(
[str(nab_executable), "lock", "--groups", "fixture", "--output", "pylock.toml"],
cwd=fixture,
check=True,
capture_output=True,
)
return fixture / "pylock.toml"


@pytest.fixture
def test_package_index(httpserver: HTTPServer, test_package_lock: Path, artifact_cache: dict[str, bytes]) -> str:
packages = [
*tomllib.loads((Path(__file__).resolve().parents[1] / "pylock.toml").read_text())["packages"],
*tomllib.loads(test_package_lock.read_text())["packages"],
]
listings: dict[str, list[str]] = {}
artifacts: dict[str, tuple[str, str]] = {}
for package in packages:
links = listings.setdefault(canonicalize_name(package["name"]), [])
for artifact in [*package.get("wheels", []), *([package["sdist"]] if "sdist" in package else [])]:
filename = artifact.get("name") or urlsplit(artifact["url"]).path.rsplit("/", 1)[-1]
digest = artifact["hashes"]["sha256"]
artifacts[filename] = artifact["url"], digest
links.append(
f'<a href="{escape(httpserver.url_for(f"/files/{filename}"))}#sha256={digest}" '
f'data-upload-time="{artifact["upload-time"].isoformat()}">{escape(filename)}</a>'
)
for name, links in listings.items():
httpserver.expect_request(f"/simple/{name}/").respond_with_data("\n".join(links), content_type="text/html")
for filename, (url, digest) in artifacts.items():

def serve(_request: Request, *, source: str = url, expected: str = digest) -> Response:
if expected not in artifact_cache:
with urlopen(source, timeout=30) as stream:
artifact_cache[expected] = stream.read()
content = artifact_cache[expected]
assert sha256(content).hexdigest() == expected
return Response(content, mimetype="application/octet-stream")

httpserver.expect_request(f"/files/{filename}").respond_with_handler(serve)
return httpserver.url_for("/simple/")


@pytest.fixture(scope="session")
def artifact_cache() -> dict[str, bytes]:
return {}


@pytest.fixture(scope="session")
def nab_executable(tmp_path_factory: pytest.TempPathFactory) -> Path:
root = Path(__file__).resolve().parents[1]
Expand Down Expand Up @@ -389,6 +452,7 @@ def _safe_environment() -> dict[str, str]:
"LANG",
"PATH",
"UV_CACHE_DIR",
"UV_INDEX_URL",
"HTTP_PROXY",
"HTTPS_PROXY",
"NO_PROXY",
Expand Down
Loading
Loading