feat(tool-images): build the runner image with other runner ids

.github/workflows/publish-tool-images.yml

@@ -1,6 +1,6 @@
 name: Publish tool images
 
-# Builds the reference images in tool-images/ (the tool-cache and system-library capability images, see the ARC role README's Capabilities section, and the build runner image, the stock runner image with a C toolchain) for linux/amd64 and linux/arm64 and pushes them to this repository owner's GitHub Container Registry namespace, as ghcr.io/<owner>/github-runner-toolcache-<name>, ghcr.io/<owner>/github-runner-sysroot-<name> and ghcr.io/<owner>/github-runner-runner-<name>, each tagged with the version its Dockerfile pins (its ARG <TOOL>_VERSION default; for the build runner image, the runner version it extends). A version tag that already exists is left alone, so a published version never changes under anyone pinning it and a run publishes only the images whose pinned version is new. The job summary lists each pushed image by digest, the reference to pin in a capability or as a runner image.
+# Builds the reference images in tool-images/ (the tool-cache and system-library capability images, see the ARC role README's Capabilities section, and the build runner image, the stock runner image with a C toolchain) for linux/amd64 and linux/arm64 and pushes them to this repository owner's GitHub Container Registry namespace, as ghcr.io/<owner>/github-runner-toolcache-<name>, ghcr.io/<owner>/github-runner-sysroot-<name> and ghcr.io/<owner>/github-runner-runner-<name>, each tagged with the version its Dockerfile pins (its ARG <TOOL>_VERSION default; for the build runner image, the runner version it extends). A version tag that already exists is left alone, so a published version never changes under anyone pinning it and a run publishes only the images whose pinned version is new. The job summary lists each pushed image by digest, the reference to pin in a capability or as a runner image. A build runner entry with a runner_id builds the same Dockerfile with RUNNER_UID and RUNNER_GID set to it and tags it <runner version>-uid<id>: the runner user at that id, for a fleet whose nodes give the stock id (1001) to an account of their own and reserve this one for the runner (tool-images/build/Dockerfile says why the id reaches the nodes). Each entry's digest artifacts are named with the architecture before the tag, so one tag's download pattern never matches another's artifacts.
 #
 # Runs only when started by hand or by pushing a tag named tool-images-<anything>; the release workflow does not build these images. Each architecture is built natively, on GitHub's amd64 and arm64 hosted runners, and pushed by digest, and a second job joins the two into one multi-architecture tag (https://docs.docker.com/build/ci/github-actions/multi-platform/), because compiling the system libraries under emulation is far slower than natively.
 
@@ -19,7 +19,7 @@ permissions:
 
 jobs:
   build:
-    name: Build ${{ matrix.image.kind }}-${{ matrix.image.name }} (${{ matrix.platform.arch }})
+    name: Build ${{ matrix.image.kind }}-${{ matrix.image.name }}${{ matrix.image.runner_id && format('-uid{0}', matrix.image.runner_id) || '' }} (${{ matrix.platform.arch }})
     runs-on: ${{ matrix.platform.runner }}
     timeout-minutes: 90
     permissions:
@@ -39,6 +39,7 @@ jobs:
           - {name: glpk, kind: sysroot}
           - {name: libmariadb, kind: sysroot}
           - {name: build, kind: runner}
+          - {name: build, kind: runner, runner_id: 549}
         platform:
           - {arch: amd64, runner: ubuntu-latest}
           - {arch: arm64, runner: ubuntu-24.04-arm}
@@ -51,6 +52,7 @@ jobs:
           OWNER: ${{ github.repository_owner }}
           KIND: ${{ matrix.image.kind }}
           NAME: ${{ matrix.image.name }}
+          RUNNER_ID: ${{ matrix.image.runner_id }}
         run: |
           set -euo pipefail
           version="$(sed -n 's/^ARG [A-Z_]*_VERSION=//p' "tool-images/${NAME}/Dockerfile")"
@@ -59,7 +61,7 @@ jobs:
             exit 1
           fi
           echo "repository=ghcr.io/${OWNER,,}/github-runner-${KIND}-${NAME}" >>"$GITHUB_OUTPUT"
-          echo "version=${version}" >>"$GITHUB_OUTPUT"
+          echo "version=${version}${RUNNER_ID:+-uid${RUNNER_ID}}" >>"$GITHUB_OUTPUT"
 
       - uses: docker/login-action@v4
         with:
@@ -88,6 +90,9 @@ jobs:
         with:
           context: tool-images/${{ matrix.image.name }}
           platforms: linux/${{ matrix.platform.arch }}
+          build-args: |
+            ${{ matrix.image.runner_id && format('RUNNER_UID={0}', matrix.image.runner_id) || '' }}
+            ${{ matrix.image.runner_id && format('RUNNER_GID={0}', matrix.image.runner_id) || '' }}
           labels: |
             org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
             org.opencontainers.image.revision=${{ github.sha }}
@@ -106,13 +111,13 @@ jobs:
       - uses: actions/upload-artifact@v7
         if: steps.published.outputs.exists == 'false'
         with:
-          name: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-${{ matrix.platform.arch }}
+          name: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-${{ matrix.platform.arch }}-${{ steps.image.outputs.version }}
           path: ${{ runner.temp }}/digests/*
           if-no-files-found: error
           retention-days: 1
 
   merge:
-    name: Publish ${{ matrix.image.kind }}-${{ matrix.image.name }}
+    name: Publish ${{ matrix.image.kind }}-${{ matrix.image.name }}${{ matrix.image.runner_id && format('-uid{0}', matrix.image.runner_id) || '' }}
     needs: [build]
     # Each image is joined on its own, so one image's failed build does not hold back the others; the join itself refuses an image missing an architecture.
     if: ${{ !cancelled() }}
@@ -135,6 +140,7 @@ jobs:
           - {name: glpk, kind: sysroot}
           - {name: libmariadb, kind: sysroot}
           - {name: build, kind: runner}
+          - {name: build, kind: runner, runner_id: 549}
     steps:
       - uses: actions/checkout@v7
 
@@ -144,16 +150,17 @@ jobs:
           OWNER: ${{ github.repository_owner }}
           KIND: ${{ matrix.image.kind }}
           NAME: ${{ matrix.image.name }}
+          RUNNER_ID: ${{ matrix.image.runner_id }}
         run: |
           set -euo pipefail
           version="$(sed -n 's/^ARG [A-Z_]*_VERSION=//p' "tool-images/${NAME}/Dockerfile")"
           echo "repository=ghcr.io/${OWNER,,}/github-runner-${KIND}-${NAME}" >>"$GITHUB_OUTPUT"
-          echo "version=${version}" >>"$GITHUB_OUTPUT"
+          echo "version=${version}${RUNNER_ID:+-uid${RUNNER_ID}}" >>"$GITHUB_OUTPUT"
 
       - uses: actions/download-artifact@v8
         with:
           path: ${{ runner.temp }}/digests
-          pattern: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-*
+          pattern: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-*-${{ steps.image.outputs.version }}
           merge-multiple: true
 
       - uses: docker/login-action@v4

.github/workflows/runner-build.yml

@@ -0,0 +1,34 @@
+name: Runner build image
+
+# Builds the reference build runner image (tool-images/build) natively on amd64 and arm64, with its default ids and with RUNNER_UID and RUNNER_GID set to another id, and checks the runner user in each (tests/runner_build/run.sh): its ids, no file left with the stock ids once they change, and that it can still write its home, run its own programs, use sudo and reach the docker group. Only on changes that can affect them. Builds nothing it publishes.
+
+on:
+  pull_request:
+    paths:
+      - tool-images/build/**
+      - tests/runner_build/**
+      - .github/workflows/runner-build.yml
+  workflow_dispatch:
+
+concurrency:
+  group: runner-build-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  runner-ids:
+    name: Runner ids (${{ matrix.platform.arch }})
+    runs-on: ${{ matrix.platform.runner }}
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        platform:
+          - {arch: amd64, runner: ubuntu-latest}
+          - {arch: arm64, runner: ubuntu-24.04-arm}
+    steps:
+      - uses: actions/checkout@v7
+      - name: Build and check the image with each set of ids
+        run: tests/runner_build/run.sh

README.md

@@ -70,6 +70,7 @@ Each Docker-in-Docker runner pod starts with an empty Docker daemon, so every jo
   ```
 - **Confirm no disk accumulates across repeated runner jobs:** the Docker host's disk usage (inside the VM, on a host that runs Docker in one) must stay flat across repeated `test-arc-runner.yml` runs.
 - **Test capabilities:** `tests/capabilities/run.sh` builds the reference Node.js tool image and libpq system-library image, renders a profile with capabilities through the ARC role and the scale-set chart, and runs a pod from the chart's runner template in a throwaway kind cluster to check that the tool reaches the tool cache and `PATH` and the library reaches the sysroot and its search paths; `.github/workflows/capabilities.yml` runs it on pull requests. `tests/system_libraries/run.sh [openmpi|highs|libpq|glpk|libmariadb]...` proves each reference system-library image works relocated: it copies the payload to `/opt/sysroot` as the role's init container does, applies the role's job-started hook, and runs the payload's programs in the stock `actions-runner` image and a program compiled against it in the reference build runner image (`tool-images/build`, the stock image with `build-essential` and `pkg-config`), including a two-process `mpirun` for Open MPI; `.github/workflows/system-libraries.yml` runs it on amd64 and arm64 on pull requests. The images under `tool-images/` build with `docker buildx build --platform linux/arm64,linux/amd64 tool-images/<name>`; `.github/workflows/publish-tool-images.yml` publishes them (see Releases), and the release workflow does not.
+- **Test the build runner image's ids:** `tests/runner_build/run.sh` builds `tool-images/build` with its default runner ids and with `RUNNER_UID` and `RUNNER_GID` set to another, and checks the runner user in each; `.github/workflows/runner-build.yml` runs it on amd64 and arm64 for changes to the image.
 - **Test the registry cache:** `tests/registry_cache/run.sh` installs the cache as the ARC role renders it in a throwaway kind cluster, with Docker Hub, ghcr.io and a private registry behind basic auth, runs a pod from the chart's runner template as the role renders a Docker-in-Docker profile with the cache on, and checks in the cache's access logs that `docker pull` and a `docker-container` buildx builder pulled through it from all three without logging in, then scales the cache to nothing and checks that pulls and builds still work directly; `.github/workflows/registry-cache.yml` runs it on pull requests.
 - **Lint:** `.github/workflows/ci.yml` runs Shellcheck, actionlint, yamllint, hadolint, an Ansible syntax check, ansible-lint (at its `production` profile), and a collection build check on every push and pull request, plus commitlint (on every commit in a pull request, and on the pull request's own title) on pull requests specifically - all gated by one `required-checks` job so a repository ruleset only ever has to require that one check regardless of how many lint jobs exist. hadolint's warning-level pinning advice is visible but non-blocking (`--failure-threshold error`): this repo deliberately floats several images/packages on `latest` rather than pinning. yamllint's config lives in `.yamllint`, relaxing the three default rules that conflict with this project's own conventions (long single-line comments, no leading `---`, GitHub Actions' `on:` key).
 - **Commit messages:** `.husky/commit-msg` runs commitlint locally (after `npm ci`), the same check CI's commitlint job applies, so a message semantic-release would mishandle is rejected before it is pushed. The machine-wide hooks dispatcher hands off to it; without one, `git config core.hooksPath .husky` enables it.
@@ -90,7 +91,7 @@ A release:
 
 The `[skip ci]` in the release commit message is deliberate: the image build already happens in this same run using the exact version semantic-release just decided, so there is nothing useful for a second, separately-triggered run of this workflow (or of `ci.yml`) to do against a commit that only changed a version stamp and a changelog - `ci.yml`'s own lint jobs already ran against every commit before it reached `main`.
 
-The reference capability images in `tool-images/` are not part of a release. `.github/workflows/publish-tool-images.yml` builds each one natively on amd64 and arm64 runners and pushes it to `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` or `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version its Dockerfile pins, when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a `tool-images-<anything>` tag. It leaves a version tag that already exists alone, and its run summary gives each pushed image's `<image>:<version>@sha256:<digest>` reference to pin in a capability or as a runner image. A new GitHub Container Registry package starts private; make it public in its package settings so pods can pull it without a credential.
+The reference capability images in `tool-images/` are not part of a release. `.github/workflows/publish-tool-images.yml` builds each one natively on amd64 and arm64 runners and pushes it to `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` or `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version its Dockerfile pins, when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a `tool-images-<anything>` tag. It leaves a version tag that already exists alone, and its run summary gives each pushed image's `<image>:<version>@sha256:<digest>` reference to pin in a capability or as a runner image. A new GitHub Container Registry package starts private; make it public in its package settings so pods can pull it without a credential. A build runner entry with a `runner_id` publishes the same image with its runner user at that uid and gid, tagged `<runner version>-uid<id>` (the ARC role README says when a fleet needs one).
 
 **What the release push needs:** a deploy key with write access on `ExaDev/github-runner`, whose private key is the `RELEASE_DEPLOY_KEY` repository secret `release.yml` checks out with. If `main` gains a ruleset, add deploy keys as its bypass actor so the release commit can land. Publishing to Ansible Galaxy additionally needs a `GALAXY_API_KEY` repository secret (an API key from `https://galaxy.ansible.com/me/preferences`); without it, the publish step logs that it's skipping and the release still completes.