Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/publish_pypi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,12 @@ jobs:
needs: build
runs-on: ubuntu-latest
strategy:
# One package failing must not cancel the other two mid-release. With the
# default fail-fast, the adapter-sdk leg failing on v0.1.1 cancelled the
# engine and api legs and skipped `ghcr` and `docs`, so the release landed
# nowhere. Each leg is an independent upload to an independent project;
# let the survivors finish and re-publish only what actually failed.
fail-fast: false
matrix:
include:
- package: nl2sql-engine
Expand Down Expand Up @@ -116,10 +122,37 @@ jobs:
cp dist/${{ matrix.stem }}-*.tar.gz dist/${{ matrix.stem }}-*.whl upload/
ls -1 upload/

# `attestations` defaults to true in the action, and that is what broke
# v0.1.1. Do not re-enable it while this workflow is reached through
# `workflow_call`.
#
# When release_please.yml calls this workflow, OIDC authentication and
# PEP 740 attestation disagree about which workflow "this" is:
#
# * auth - PyPI matches the token's `job_workflow_ref` claim, which is
# the *called* workflow: publish_pypi.yaml. That is what the
# trusted publishers are registered against, so auth succeeds.
# * attest - Sigstore signs with the entry-point identity, so the
# certificate's Build Config URI is the *calling* workflow:
# release_please.yml.
#
# PyPI verifies the attestation against the single publisher that
# authenticated the request, so the two can never both match one
# registration and the upload fails with:
#
# 400 Invalid attestations supplied during upload: ... Certificate's
# Build Config URI (...release_please.yml@refs/heads/main) does not
# match expected Trusted Publisher (publish_pypi.yaml @ nadeem4/nl2sql)
#
# Trusted publishing itself is unaffected - no API token is introduced.
# The cost is that released artifacts carry no PEP 740 provenance.
# See docs/development/releasing.md and
# https://docs.pypi.org/trusted-publishers/troubleshooting/#reusable-workflows-on-github
- name: Publish ${{ matrix.package }} to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: upload/
attestations: false

# packages/api/Dockerfile installs nl2sql-adapter-sdk, nl2sql-engine and
# nl2sql-api from the local sources, not from PyPI, and its COPY paths are
Expand Down
34 changes: 34 additions & 0 deletions docs/development/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,23 @@ changelog and the version it proposes is the release decision.
Authentication is PyPI **trusted publishing** (OIDC, `id-token: write`). There
is no API token anywhere in the workflow and no secret to rotate.

!!! warning "PEP 740 attestations are off, and must stay off"
The `pypi` step sets `attestations: false`. Because `release_please.yml`
reaches this workflow through `workflow_call`, OIDC auth and attestation
signing disagree about which workflow "this" is: PyPI matches the token's
`job_workflow_ref` claim, which names the *called* workflow
(`publish_pypi.yaml`, the registered publisher), while Sigstore signs with
the *entry-point* identity, putting `release_please.yml` in the
certificate's Build Config URI. PyPI verifies the attestation against the
one publisher that authenticated the request, so the two can never both
match and the upload fails with `400 Invalid attestations supplied during
upload`. This is what broke `v0.1.1`. Trusted publishing is unaffected; the
cost is that released artifacts carry no PEP 740 provenance. See
[PyPI's note on reusable workflows](https://docs.pypi.org/trusted-publishers/troubleshooting/#reusable-workflows-on-github).

The three `pypi` legs run with `fail-fast: false`, so one package failing no
longer cancels the other two and skips `ghcr` and `docs`.

### Why the publish is chained to release-please, not to the tag

**GitHub does not trigger workflows from events created with the default
Expand Down Expand Up @@ -174,6 +191,23 @@ $ gh workflow run publish_pypi.yaml -f tag=v0.1.0

or **Actions → Publish Release → Run workflow** and type the tag.

The same dispatch is how you recover a release that **tagged but never
uploaded** — the `vX.Y.Z` tag and the GitHub Release exist, but PyPI still shows
the previous version and no image reached GHCR. That is what happened to
`v0.1.1`: the `pypi` matrix failed and took `ghcr` and `docs` down with it.
Nothing needs to be re-tagged; fix the cause on `main`, then dispatch the same
tag:

```console
$ gh workflow run publish_pypi.yaml -f tag=v0.1.1
```

The dispatch checks out the tag, so it publishes the tagged commit, not `main`.
Only legs that never uploaded can be re-run — PyPI rejects a re-upload of a
version it already has, so a partially-published release needs `skip-existing`
or a manual publish of just the missing packages, in the
[dependency order below](#package-order).

Every job resolves its tag from `${{ inputs.tag || github.ref_name }}`, never
from the raw ref. On a dispatched run `github.ref_name` is the *branch* you
dispatched from, so the fallback would silently build `main` instead of the
Expand Down
2 changes: 1 addition & 1 deletion docs/observability/error-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

NL2SQL represents failures as structured `PipelineError` objects and propagates them through state. Retries are managed at the subgraph level, and a single circuit breaker provides fast-fail safety for vector retrieval.

One case cannot use state: LangGraph conditional-edge routers may only return routing decisions, so `route_scan_layers()` reports "no compatible subgraph found" by raising `PipelineExecutionError` (`nl2sql.common.exceptions`), an `NL2SQLError` carrying the `PipelineError` payload on `.error`. This propagates out of the router and is caught by `run_with_graph()`'s crash handler, which folds it into `GraphState.errors` as an `UNKNOWN_ERROR` `PipelineError`.
One case cannot use state: LangGraph conditional-edge routers may only return routing decisions, so `route_scan_layers()` reports "no compatible subgraph found" by raising `PipelineExecutionError` (`nl2sql.common.exceptions`), an `NL2SQLError` carrying the `PipelineError` payload on `.error`. This propagates out of the router and is caught by `run_with_graph()`, which unwraps it and returns its `PipelineError` payload unchanged in `GraphState.errors` — preserving `error_code` (`INVALID_STATE`), `severity`, `node` and `is_retryable`; only unrecognised exceptions become `UNKNOWN_ERROR`.

## Error contract

Expand Down
Loading