Skip to content

docs: correct and shorten the release guide #879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yaythomas wants to merge 1 commit into
base: main
Choose a base branch
from
+70 −145
Open

docs: correct and shorten the release guide #879

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
215 changes: 70 additions & 145 deletions RELEASING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,129 +28,60 @@ or the initial version for a new distribution.

## Release Permissions and Protection

The GitHub `pypi` environment permits deployments only from **tags** matching:

- `sdk-v*`
- `otel-v*`
- `testing-v*`
- `telemetry-exporter-v*`

Branches, including `main` and temporary publishing branches, are not allowed.
GitHub checks the workflow run's ref; checking out an allowed tag or a merged
commit inside a branch-triggered workflow does not make that run eligible.

The [protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902)
restricts creation, updates, deletion, and force pushes for the same tag
patterns. Have an actor permitted by that ruleset create the release tag at the
intended merged commit after CI passes. Treat published tags as immutable: use
a new version and tag for corrections instead of moving, deleting, or
recreating a published tag, even if your role permits those operations.

When adding a package with a new tag prefix, configure its pattern in **both**
the [`pypi` environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments)
and the protected tags ruleset before its first release. Updating only the
workflow matrix or this document is not enough. Keep these protections in
place for first-time publications as well as later releases.
The [`pypi-publish.yml`](.github/workflows/pypi-publish.yml) workflow publishes
to PyPI only for a release whose tag matches `sdk-v*`, `otel-v*`,
`testing-v*`, or `telemetry-exporter-v*`. Its `pypi-publish` jobs run in the
`pypi` environment, which accepts only runs on a matching tag. The
[protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902)
restricts who can create, update, delete, or force push a matching tag.

If a release uses a tag that matches none of the patterns, GitHub creates the
tag and the release, but refuses to run the `pypi-publish` jobs. Before the
first release that uses a new prefix, add the prefix's pattern to both the
ruleset and the
[environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments).

Do not move, delete, or recreate a published tag. Correct a release with a
new version and a new tag.

## Publishing a Package for the First Time

Each distribution name needs its own PyPI project and trusted publisher. This
also applies when renaming a package: the old project's publisher does not
carry over to the new name. Trusted publishing uses GitHub Actions OIDC, so this
process does not require a PyPI API token on your machine or in GitHub secrets.

### 1. Prepare the package

Confirm the distribution name in the package's `pyproject.toml` (`[project].name`)
and its initial version in `__about__.py`. The distribution name is the name used
by `pip install`; it can differ from the Python import name.

Add the package to **both** matrices in
[`pypi-publish.yml`](.github/workflows/pypi-publish.yml): its name and path in
`release-build`, and its name in `pypi-publish`. Add its path and tag prefix to
the [Packages](#packages) table above, and configure any new tag prefix as
described in [Release Permissions and Protection](#release-permissions-and-protection).
Merge the package and confirm CI passes before publishing. Follow
[CONTRIBUTING.md](CONTRIBUTING.md) for tests and run
`hatch build` from the package directory. From the repository root, run
[`check_dist_legal_files.py`](.github/scripts/check_dist_legal_files.py) with
`packages/<package-name>` as its argument to verify the wheel and source archive
include `LICENSE` and `NOTICE`.

### 2. Configure the PyPI publisher

Sign in to the PyPI account that should own the new project. Open
[Publishing settings](https://pypi.org/manage/account/publishing/), find **Add a
new pending publisher**, select **GitHub**, and fill in:

| Field | Value |
|-------|-------|
| PyPI Project Name | The exact `[project].name` from the new package's `pyproject.toml` |
| Owner | `aws` |
| Repository name | `aws-durable-execution-sdk-python` |
| Workflow name | `pypi-publish.yml` |
| Environment name | `pypi` |

**Owner** is the GitHub organization, not the PyPI username. **Workflow name**
is the filename, without `.github/workflows/`; it is not the display name
`Upload PyPI Package`. Enter `pypi` for the environment even though the form
marks it optional, because the publishing job uses that GitHub environment.

Click **Add**. The publisher remains pending until the first successful upload,
which creates the project and converts it to a regular trusted publisher. The
PyPI account that registered it becomes a project owner. A pending publisher
**does not reserve the name**. See PyPI's
[first-project publishing guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/).

If you need to claim a name without uploading a release and have access to a
PyPI organization, use the organization's **Manage → Projects → Create and add
a new project** action instead ([instructions](https://docs.pypi.org/organization-accounts/actions/project-actions/#create-a-project)).
Then add the trusted publisher through that project's **Publishing** settings.
A public PyPI 404 alone does not prove a name is free; a project can exist
without any releases ([name availability](https://pypi.org/help/#project-name)).

### 3. Publish the first version

Follow [Cutting a Release](#cutting-a-release) below using an allowed release tag.
The initial version only needs to be unused under the new distribution name;
there is no need to bump it just to create the project.

Check the entire release scope first. The current workflow publishes **all
packages in both matrices**, regardless of the release's tag prefix. After
PyPI publishing succeeds, it calls
[`ecr-release.yml`](.github/workflows/ecr-release.yml), which publishes the
testing emulator image only when the tag includes `testing-v<version>`. Do not
create a GitHub release solely to claim one name if those other publications
are not intended.

The current PyPI workflow has no package selector or manual-dispatch trigger,
and the environment rules prevent publishing from a temporary branch. If only
the new package should be uploaded, first review and merge workflow changes
that limit publication to the intended package and avoid the unrelated emulator
image release, while retaining the tag protections. Include those changes in
the release's tagged commit. For name reservation without an upload, use the
PyPI organization project-creation option above.

### 4. Verify the project

Check that the workflow ran on the intended release tag, that the tag and the
checkout step resolve to the selected merged source commit, and that both the
build and publish jobs succeeded. On PyPI, verify the project name, version,
wheel, and source archive. Compare their SHA-256 hashes with the workflow's
distribution artifact. Confirm the publisher now appears in the project's
**Publishing** settings and arrange the team's project ownership as needed.

If publishing reports `invalid-pending-publisher` or `invalid-publisher`, check
the repository owner, repository name, workflow filename, and environment
against the form values above. Also check that the built distribution's name
matches **PyPI Project Name**. Correct the configuration before retrying; adding
the pending publisher by itself does not finish publication.

If GitHub rejects access to the `pypi` environment, check the run's tag ref
against the allowed patterns. Retrying a branch run or changing its checkout
does not satisfy a tag-only deployment rule. If tag creation is rejected, have
an actor permitted by the protected tags ruleset create the intended release
tag. Do not relax the environment or tag protections to retry a publication.
A trusted publisher names the GitHub workflow that may upload to a PyPI
project. A pending publisher is a trusted publisher for a project that does
not exist yet. Each distribution name needs its own project and trusted
publisher. A renamed package needs a new project and publisher too.

1. Set the package's initial version in its `__about__.py`. Choose its tag
prefix as [Tagging Convention](#tagging-convention) describes. In
[`pypi-publish.yml`](.github/workflows/pypi-publish.yml), add its name and
path to the `release-build` matrix and its name to the `pypi-publish`
matrix. Add it to the legal-files step in
[`ci.yml`](.github/workflows/ci.yml), the [Packages](#packages) table, and
the [Versioning](#versioning) list.
2. The PyPI account that adds the pending publisher becomes the project's
owner. So sign in as `durable-execution-dev`, the account that owns the
existing projects. In
[Publishing settings](https://pypi.org/manage/account/publishing/), add a
pending GitHub publisher
([PyPI guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/)):

| Field | Value |
|-------|-------|
| PyPI Project Name | `[project].name` from the package's `pyproject.toml` |
| Owner | `aws` |
| Repository name | `aws-durable-execution-sdk-python` |
| Workflow name | `pypi-publish.yml` |
| Environment name | `pypi` |

Enter the environment name even though the form marks it optional. Without
it, PyPI accepts uploads from a run of `pypi-publish.yml` on any branch.
With it, PyPI accepts only jobs in the `pypi` environment, so only runs on
a matching tag can upload. Another PyPI user can still register the name
until the first upload.
3. Check the other packages as the note in
[How Publishing Works](#how-publishing-works) describes. Then
[cut a release](#cutting-a-release). The first successful upload creates
the project.

## Cutting a Release

Expand All @@ -162,11 +93,6 @@ use the initial version prepared above. Commit and merge to `main`.

### 2. Create a GitHub Release

If the release creates a new tag, the actor creating it must be permitted by
the [protected tags ruleset](#release-permissions-and-protection). The tag must
point to the merged commit whose CI passed, including any publishing workflow
changes needed for this release.

1. Go to the [Releases page](https://github.com/aws/aws-durable-execution-sdk-python/releases) on GitHub.
2. Click **Draft a new release**.
3. Select the intended merged commit as the target and create a new tag
Expand Down Expand Up @@ -211,17 +137,21 @@ Creating a GitHub Release triggers the [`pypi-publish.yml`](.github/workflows/py
2. **Uploads** the built distributions as artifacts.
3. **Publishes** each package to [PyPI](https://pypi.org/) using trusted publishing (OIDC-based, no API tokens required).

The workflow runs on the `release: [published]` event. Its publishing jobs use
the protected `pypi` environment, so the release tag must satisfy that
environment's deployment rules before they can upload to PyPI. The workflow
and package source come from the tagged revision; later changes on `main` do
not change an existing release's source.

> **Note:** The tag prefix does not select packages. The workflow builds and
> publishes every package in its matrices, including any unrelated version
> that has not been published yet. Check every package's version before
> creating a release. PyPI rejects an already published filename; with
> `fail-fast: false`, that job can fail while other packages still publish.
The workflow runs on the `release: [published]` event. The workflow file and
the package source both come from the tagged revision. So a re-run of an
existing release uses its original workflow and source, not later changes on
`main`. The `pypi-publish` jobs run in the `pypi` environment, which
[Release Permissions and Protection](#release-permissions-and-protection)
describes.

> **Note:** The tag prefix does not select packages. The workflow uploads
> every package in its matrices on every release. PyPI publishes each version
> that it does not have yet. So check that each unpublished version belongs in
> this release. PyPI accepts a re-upload of a version it already has only if
> the files match the published ones. If the package changed after PyPI
> published that version, even only its README, PyPI rejects the upload with
> `400 File already exists`. That package's job then fails, and GitHub skips
> the emulator image job. Re-running the failed job repeats the rejection.

The [`ecr-release.yml`](.github/workflows/ecr-release.yml) workflow publishes the
testing emulator image only when the release tag names `testing-v<version>`,
Expand Down Expand Up @@ -321,17 +251,12 @@ Before publishing a release:
`aws-durable-execution-sdk-python-telemetry-exporter`), follow
[Publishing a Package for the First Time](#publishing-a-package-for-the-first-time)
and configure its PyPI publisher before uploading
- [ ] Every unpublished version in the PyPI matrices is intended for this
release, along with the testing emulator image publication when the tag
includes `testing-v<version>`
- [ ] Every unpublished version in the PyPI matrices belongs in this release
- [ ] For a release tag that includes `testing-v<version>`, every package that
changed since its latest PyPI version has a new version, or GitHub
skips the emulator image job
- [ ] Changes merged to `main`
- [ ] CI checks pass on `main`
- [ ] Release tag points to the selected merged, CI-passing commit, including
the publishing workflow used for the release
- [ ] Release tag matches an allowed `pypi` deployment pattern and is covered
by the protected tags ruleset; any new prefix is configured in both
- [ ] The actor creating the release tag is permitted by the protected tags
ruleset
- [ ] Release notes written with separate sections per package
- [ ] Tag follows the naming convention (`sdk-vX.Y.Z`, `otel-vX.Y.Z`, `testing-vX.Y.Z`,
`telemetry-exporter-vX.Y.Z`, or comma-separated)
Loading