Skip to content

Commit a5cafaa

Browse files
committed
Compile the Azure CLI's bytecode on the build platform
The 0.2.0 release failed on its arm64 image: compiling the Azure CLI's 12,000 files under emulation hit uv's per-file timeout after 36 minutes. Bytecode is the same on every architecture for one Python, so it is now compiled in a stage on the build machine's platform, from the same base. Pushes and pull requests now build both architectures too, so a failure under emulation is found before a release.
1 parent 2815451 commit a5cafaa

5 files changed

Lines changed: 62 additions & 7 deletions

File tree

‎.github/workflows/container.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -235,14 +235,20 @@ jobs:
235235
echo "changed=true" >> "${GITHUB_OUTPUT}"
236236
fi
237237
238+
# Every platform is built on every change, not only for a release, so a build that
239+
# only breaks under emulation is found before a release depends on it.
238240
- name: Set up emulation for other architectures
239-
if: steps.changed.outputs.changed == 'true'
241+
if: >-
242+
steps.plan.outputs.skip != 'true' &&
243+
(steps.plan.outputs.publish != 'true' || steps.changed.outputs.changed == 'true')
240244
run: |
241245
sudo apt-get update --quiet
242246
sudo apt-get install --yes --quiet qemu-user-static
243247
244248
- name: Build for every platform
245-
if: steps.changed.outputs.changed == 'true'
249+
if: >-
250+
steps.plan.outputs.skip != 'true' &&
251+
(steps.plan.outputs.publish != 'true' || steps.changed.outputs.changed == 'true')
246252
env:
247253
VARIANT: ${{ matrix.variant }}
248254
VERSION: ${{ steps.version.outputs.version }}

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,15 @@ All notable changes to libre-devops-helpers are recorded here. The project follo
55

66
## 0.2.1
77

8+
### Fixed
9+
10+
- The 0.2.0 release stopped before publishing the default image and the GitHub release:
11+
its arm64 build compiled the Azure CLI's 12,000 files under emulation and ran out of
12+
time. The Azure CLI's bytecode is now compiled on the build machine's own platform (it
13+
is the same on every architecture), in seconds. Only `0.2.0-slim` was published.
14+
- Every push and pull request now builds both architectures, so a build that fails only
15+
under emulation is caught before a release.
16+
817
### Security
918

1019
- The default image's Azure CLI now runs cryptography 50.0.1 (CVE-2026-69247,

‎Containerfile‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,21 @@ COPY src ./src
2828
RUN UV_PROJECT_ENVIRONMENT=/opt/ldo uv sync --locked --no-dev --no-editable
2929

3030
# Build the Azure CLI, locked in container/azure-cli, into its own environment at /opt/az.
31+
# Its bytecode is compiled in the next stage instead: the Azure CLI is over 12,000 files,
32+
# and compiling them under emulation, for another architecture's image, takes longer
33+
# than a build may run.
3134
FROM build AS build-az
3235
COPY container/azure-cli/pyproject.toml container/azure-cli/uv.lock /src/azure-cli/
33-
RUN UV_PROJECT_ENVIRONMENT=/opt/az uv sync --locked --no-dev --project /src/azure-cli
36+
RUN UV_PROJECT_ENVIRONMENT=/opt/az UV_COMPILE_BYTECODE=0 \
37+
uv sync --locked --no-dev --project /src/azure-cli
38+
39+
# The Azure CLI's bytecode, compiled on the build machine's own platform. Bytecode is the
40+
# same on every architecture for one Python version, and this is the same base image, so
41+
# the result is what compiling in place would give, in seconds rather than an hour.
42+
# unchecked-hash: the image never changes, so Python need not check the sources.
43+
FROM --platform=$BUILDPLATFORM docker.io/library/python:3.14-slim-trixie@sha256:caaf356f40667c496d405780745b9ac25771c189a51dfcc42430d531ea09f8a2 AS compile-az
44+
COPY --from=build-az /opt/az /opt/az
45+
RUN python -W ignore -m compileall -q -j 0 --invalidation-mode unchecked-hash /opt/az/lib
3446

3547
# Runtime: the same base, patched, with a non-root user and the tool on PATH.
3648
FROM docker.io/library/python:3.14-slim-trixie@sha256:caaf356f40667c496d405780745b9ac25771c189a51dfcc42430d531ea09f8a2 AS runtime
@@ -64,7 +76,7 @@ FROM runtime AS tool
6476
# The default target: the tool with the Azure CLI, for signing in as yourself (the
6577
# default "azure-cli" profiles) and 'ldo az'.
6678
FROM runtime AS az
67-
COPY --from=build-az /opt/az /opt/az
79+
COPY --from=compile-az /opt/az /opt/az
6880
USER root
6981
# A wrapper rather than a link, so az always runs on its own environment's Python.
7082
RUN printf '#!/bin/sh\nexec /opt/az/bin/python -m azure.cli "$@"\n' > /usr/local/bin/az \

‎README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -918,9 +918,10 @@ Every pull request and push to `main` runs `.github/workflows/ci.yml`:
918918
| Coverage | line and branch coverage, failing below the floor in `pyproject.toml` (`fail_under`), with the report in the job summary and the total written for the badge |
919919
| Build | builds the sdist and wheel once, installs the wheel in a clean environment, and keeps both as the run's artifact |
920920

921-
`.github/workflows/container.yml` builds both images on every pull request and push, runs
922-
them (the version, a command's help, a non-root user, and for the default image a
923-
signed-out `az whoami` that must fail cleanly), and scans them with a checksum-pinned Trivy. The `slim` image
921+
`.github/workflows/container.yml` builds both images for `amd64` and `arm64` on every pull
922+
request and push. It runs the native build (the version, a command's help, a non-root user,
923+
and for the default image a signed-out `az whoami` that must fail cleanly) and scans it with
924+
a checksum-pinned Trivy. The `slim` image
924925
fails on any high or critical vulnerability that has a fix; the default image fails on a
925926
fixed critical one and reports the rest, since the Azure CLI inside it pins its own
926927
dependencies. Findings with a fix go to the Security tab. Those without one, mostly Debian
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import re
2+
from pathlib import Path
3+
4+
CONTAINERFILE = Path(__file__).resolve().parents[2] / "Containerfile"
5+
FROM = re.compile(r"^FROM\s+(?:--platform=\S+\s+)?(\S+)(?:\s+AS\s+(\S+))?", re.MULTILINE)
6+
7+
8+
def stages() -> dict[str, str]:
9+
return {name: image for image, name in FROM.findall(CONTAINERFILE.read_text())}
10+
11+
12+
def test_every_python_stage_is_one_pinned_base():
13+
# The Azure CLI's bytecode is compiled in one stage and run in another, so the two must
14+
# be the same Python; a partial base image bump would break that quietly.
15+
pythons = {image for image in stages().values() if "library/python:" in image}
16+
assert len(pythons) == 1, pythons
17+
assert "@sha256:" in pythons.pop()
18+
19+
20+
def test_the_azure_cli_is_compiled_on_the_build_platform_and_shipped_compiled():
21+
text = CONTAINERFILE.read_text()
22+
assert "FROM --platform=$BUILDPLATFORM" in text
23+
assert "COPY --from=compile-az /opt/az /opt/az" in text
24+
assert (
25+
"COPY --from=build-az /opt/az /opt/az"
26+
in text.split("AS compile-az", 1)[1].split("\nFROM", 1)[0]
27+
)

0 commit comments

Comments
 (0)