diff --git a/RELEASING.md b/RELEASING.md index e0b4d6750..3de813882 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -28,129 +28,60 @@ or the initial version for a new distribution. ## Release Permissions and Protection -The GitHub `pypi` environment permits deployments only from **tags** matching: - -- `sdk-v*` -- `otel-v*` -- `testing-v*` -- `telemetry-exporter-v*` - -Branches, including `main` and temporary publishing branches, are not allowed. -GitHub checks the workflow run's ref; checking out an allowed tag or a merged -commit inside a branch-triggered workflow does not make that run eligible. - -The [protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902) -restricts creation, updates, deletion, and force pushes for the same tag -patterns. Have an actor permitted by that ruleset create the release tag at the -intended merged commit after CI passes. Treat published tags as immutable: use -a new version and tag for corrections instead of moving, deleting, or -recreating a published tag, even if your role permits those operations. - -When adding a package with a new tag prefix, configure its pattern in **both** -the [`pypi` environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments) -and the protected tags ruleset before its first release. Updating only the -workflow matrix or this document is not enough. Keep these protections in -place for first-time publications as well as later releases. +The [`pypi-publish.yml`](.github/workflows/pypi-publish.yml) workflow publishes +to PyPI only for a release whose tag matches `sdk-v*`, `otel-v*`, +`testing-v*`, or `telemetry-exporter-v*`. Its `pypi-publish` jobs run in the +`pypi` environment, which accepts only runs on a matching tag. The +[protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902) +restricts who can create, update, delete, or force push a matching tag. + +If a release uses a tag that matches none of the patterns, GitHub creates the +tag and the release, but refuses to run the `pypi-publish` jobs. Before the +first release that uses a new prefix, add the prefix's pattern to both the +ruleset and the +[environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments). + +Do not move, delete, or recreate a published tag. Correct a release with a +new version and a new tag. ## Publishing a Package for the First Time -Each distribution name needs its own PyPI project and trusted publisher. This -also applies when renaming a package: the old project's publisher does not -carry over to the new name. Trusted publishing uses GitHub Actions OIDC, so this -process does not require a PyPI API token on your machine or in GitHub secrets. - -### 1. Prepare the package - -Confirm the distribution name in the package's `pyproject.toml` (`[project].name`) -and its initial version in `__about__.py`. The distribution name is the name used -by `pip install`; it can differ from the Python import name. - -Add the package to **both** matrices in -[`pypi-publish.yml`](.github/workflows/pypi-publish.yml): its name and path in -`release-build`, and its name in `pypi-publish`. Add its path and tag prefix to -the [Packages](#packages) table above, and configure any new tag prefix as -described in [Release Permissions and Protection](#release-permissions-and-protection). -Merge the package and confirm CI passes before publishing. Follow -[CONTRIBUTING.md](CONTRIBUTING.md) for tests and run -`hatch build` from the package directory. From the repository root, run -[`check_dist_legal_files.py`](.github/scripts/check_dist_legal_files.py) with -`packages/` as its argument to verify the wheel and source archive -include `LICENSE` and `NOTICE`. - -### 2. Configure the PyPI publisher - -Sign in to the PyPI account that should own the new project. Open -[Publishing settings](https://pypi.org/manage/account/publishing/), find **Add a -new pending publisher**, select **GitHub**, and fill in: - -| Field | Value | -|-------|-------| -| PyPI Project Name | The exact `[project].name` from the new package's `pyproject.toml` | -| Owner | `aws` | -| Repository name | `aws-durable-execution-sdk-python` | -| Workflow name | `pypi-publish.yml` | -| Environment name | `pypi` | - -**Owner** is the GitHub organization, not the PyPI username. **Workflow name** -is the filename, without `.github/workflows/`; it is not the display name -`Upload PyPI Package`. Enter `pypi` for the environment even though the form -marks it optional, because the publishing job uses that GitHub environment. - -Click **Add**. The publisher remains pending until the first successful upload, -which creates the project and converts it to a regular trusted publisher. The -PyPI account that registered it becomes a project owner. A pending publisher -**does not reserve the name**. See PyPI's -[first-project publishing guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/). - -If you need to claim a name without uploading a release and have access to a -PyPI organization, use the organization's **Manage → Projects → Create and add -a new project** action instead ([instructions](https://docs.pypi.org/organization-accounts/actions/project-actions/#create-a-project)). -Then add the trusted publisher through that project's **Publishing** settings. -A public PyPI 404 alone does not prove a name is free; a project can exist -without any releases ([name availability](https://pypi.org/help/#project-name)). - -### 3. Publish the first version - -Follow [Cutting a Release](#cutting-a-release) below using an allowed release tag. -The initial version only needs to be unused under the new distribution name; -there is no need to bump it just to create the project. - -Check the entire release scope first. The current workflow publishes **all -packages in both matrices**, regardless of the release's tag prefix. After -PyPI publishing succeeds, it calls -[`ecr-release.yml`](.github/workflows/ecr-release.yml), which publishes the -testing emulator image only when the tag includes `testing-v`. Do not -create a GitHub release solely to claim one name if those other publications -are not intended. - -The current PyPI workflow has no package selector or manual-dispatch trigger, -and the environment rules prevent publishing from a temporary branch. If only -the new package should be uploaded, first review and merge workflow changes -that limit publication to the intended package and avoid the unrelated emulator -image release, while retaining the tag protections. Include those changes in -the release's tagged commit. For name reservation without an upload, use the -PyPI organization project-creation option above. - -### 4. Verify the project - -Check that the workflow ran on the intended release tag, that the tag and the -checkout step resolve to the selected merged source commit, and that both the -build and publish jobs succeeded. On PyPI, verify the project name, version, -wheel, and source archive. Compare their SHA-256 hashes with the workflow's -distribution artifact. Confirm the publisher now appears in the project's -**Publishing** settings and arrange the team's project ownership as needed. - -If publishing reports `invalid-pending-publisher` or `invalid-publisher`, check -the repository owner, repository name, workflow filename, and environment -against the form values above. Also check that the built distribution's name -matches **PyPI Project Name**. Correct the configuration before retrying; adding -the pending publisher by itself does not finish publication. - -If GitHub rejects access to the `pypi` environment, check the run's tag ref -against the allowed patterns. Retrying a branch run or changing its checkout -does not satisfy a tag-only deployment rule. If tag creation is rejected, have -an actor permitted by the protected tags ruleset create the intended release -tag. Do not relax the environment or tag protections to retry a publication. +A trusted publisher names the GitHub workflow that may upload to a PyPI +project. A pending publisher is a trusted publisher for a project that does +not exist yet. Each distribution name needs its own project and trusted +publisher. A renamed package needs a new project and publisher too. + +1. Set the package's initial version in its `__about__.py`. Choose its tag + prefix as [Tagging Convention](#tagging-convention) describes. In + [`pypi-publish.yml`](.github/workflows/pypi-publish.yml), add its name and + path to the `release-build` matrix and its name to the `pypi-publish` + matrix. Add it to the legal-files step in + [`ci.yml`](.github/workflows/ci.yml), the [Packages](#packages) table, and + the [Versioning](#versioning) list. +2. The PyPI account that adds the pending publisher becomes the project's + owner. So sign in as `durable-execution-dev`, the account that owns the + existing projects. In + [Publishing settings](https://pypi.org/manage/account/publishing/), add a + pending GitHub publisher + ([PyPI guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/)): + + | Field | Value | + |-------|-------| + | PyPI Project Name | `[project].name` from the package's `pyproject.toml` | + | Owner | `aws` | + | Repository name | `aws-durable-execution-sdk-python` | + | Workflow name | `pypi-publish.yml` | + | Environment name | `pypi` | + + Enter the environment name even though the form marks it optional. Without + it, PyPI accepts uploads from a run of `pypi-publish.yml` on any branch. + With it, PyPI accepts only jobs in the `pypi` environment, so only runs on + a matching tag can upload. Another PyPI user can still register the name + until the first upload. +3. Check the other packages as the note in + [How Publishing Works](#how-publishing-works) describes. Then + [cut a release](#cutting-a-release). The first successful upload creates + the project. ## Cutting a Release @@ -162,11 +93,6 @@ use the initial version prepared above. Commit and merge to `main`. ### 2. Create a GitHub Release -If the release creates a new tag, the actor creating it must be permitted by -the [protected tags ruleset](#release-permissions-and-protection). The tag must -point to the merged commit whose CI passed, including any publishing workflow -changes needed for this release. - 1. Go to the [Releases page](https://github.com/aws/aws-durable-execution-sdk-python/releases) on GitHub. 2. Click **Draft a new release**. 3. Select the intended merged commit as the target and create a new tag @@ -211,17 +137,21 @@ Creating a GitHub Release triggers the [`pypi-publish.yml`](.github/workflows/py 2. **Uploads** the built distributions as artifacts. 3. **Publishes** each package to [PyPI](https://pypi.org/) using trusted publishing (OIDC-based, no API tokens required). -The workflow runs on the `release: [published]` event. Its publishing jobs use -the protected `pypi` environment, so the release tag must satisfy that -environment's deployment rules before they can upload to PyPI. The workflow -and package source come from the tagged revision; later changes on `main` do -not change an existing release's source. - -> **Note:** The tag prefix does not select packages. The workflow builds and -> publishes every package in its matrices, including any unrelated version -> that has not been published yet. Check every package's version before -> creating a release. PyPI rejects an already published filename; with -> `fail-fast: false`, that job can fail while other packages still publish. +The workflow runs on the `release: [published]` event. The workflow file and +the package source both come from the tagged revision. So a re-run of an +existing release uses its original workflow and source, not later changes on +`main`. The `pypi-publish` jobs run in the `pypi` environment, which +[Release Permissions and Protection](#release-permissions-and-protection) +describes. + +> **Note:** The tag prefix does not select packages. The workflow uploads +> every package in its matrices on every release. PyPI publishes each version +> that it does not have yet. So check that each unpublished version belongs in +> this release. PyPI accepts a re-upload of a version it already has only if +> the files match the published ones. If the package changed after PyPI +> published that version, even only its README, PyPI rejects the upload with +> `400 File already exists`. That package's job then fails, and GitHub skips +> the emulator image job. Re-running the failed job repeats the rejection. The [`ecr-release.yml`](.github/workflows/ecr-release.yml) workflow publishes the testing emulator image only when the release tag names `testing-v`, @@ -321,17 +251,12 @@ Before publishing a release: `aws-durable-execution-sdk-python-telemetry-exporter`), follow [Publishing a Package for the First Time](#publishing-a-package-for-the-first-time) and configure its PyPI publisher before uploading -- [ ] Every unpublished version in the PyPI matrices is intended for this - release, along with the testing emulator image publication when the tag - includes `testing-v` +- [ ] Every unpublished version in the PyPI matrices belongs in this release +- [ ] For a release tag that includes `testing-v`, every package that + changed since its latest PyPI version has a new version, or GitHub + skips the emulator image job - [ ] Changes merged to `main` - [ ] CI checks pass on `main` -- [ ] Release tag points to the selected merged, CI-passing commit, including - the publishing workflow used for the release -- [ ] Release tag matches an allowed `pypi` deployment pattern and is covered - by the protected tags ruleset; any new prefix is configured in both -- [ ] The actor creating the release tag is permitted by the protected tags - ruleset - [ ] Release notes written with separate sections per package - [ ] Tag follows the naming convention (`sdk-vX.Y.Z`, `otel-vX.Y.Z`, `testing-vX.Y.Z`, `telemetry-exporter-vX.Y.Z`, or comma-separated)