Skip to content

Pull RPM build containers from ghcr.io/simp/simp-<os>-build (fixes fleet-wide RPM build failures) - #48

Open
silug wants to merge 3 commits into
mainfrom
release-rpms-ghcr-build-containers
Open

Pull RPM build containers from ghcr.io/simp/simp-<os>-build (fixes fleet-wide RPM build failures)#48
silug wants to merge 3 commits into
mainfrom
release-rpms-ghcr-build-containers

Conversation

@silug

@silug silug commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Problem

release_rpms.yml resolves its builder image as:

simp_builder_docker_image: 'docker.io/simpproject/simp_build_${{ github.event.inputs.build_container_os }}:latest'

Docker Hub's simpproject/ namespace has no EL8/9/10 build images. The only relevant ones are:

Image Last pushed
simpproject/simp_build_centos7 2024-08-09
simpproject/simp_build_centos8 2023-06-26

(plus older centos6 and *_ruby3_1 variants). So no value of build_container_os can select a current container — simpproject/simp_build_el8 does not exist.

Building against those years-old images fails with:

NameError: uninitialized constant JSON::Fragment

JSON::Fragment was added in the json gem 2.9; the stale images ship older. As of 2026-07-28 this is failing in 70 of 71 non-archived repos that have run the workflow, spanning 2026-06-09 → 2026-07-24 — i.e. through the entire 7.0.0/8.0.0 release wave.

Fix

Point at the current containers, which are public and rebuilt weekly:

Image Tags
ghcr.io/simp/simp-el8-build latest, 2026042820260720
ghcr.io/simp/simp-el9-build latest, 2026042820260720
ghcr.io/simp/simp-el10-build latest, 2026061820260727

Three changes:

  1. simp_builder_docker_imageghcr.io/simp/simp-${os}-build:${tag} — note both the registry host and the image-name shape differ from the old form.
  2. New build_container_tag input (default latest) so a dated tag can be pinned for reproducible rebuilds.
  3. build_container_os default centos8el8 (lowest currently supported EL, so the widest RPM compatibility).

This is already proven in production

pupmod-simp-mockup has carried exactly this configuration for a while and is the only repo with it. The single green release_rpms.yml run anywhere in the org came from it: simp/pupmod-simp-simplib 5.0.3 (build os: el10), dispatched through mockup's workflow via target_repo on 2026-06-22 — signing and release-asset attachment included. This PR ports that working configuration into the baseline so the other 78 repos get it.

Diffed against pupmod-simp-mockup's copy; this PR is the mockup delta minus the runner/action bumps, which #42 already owns (see below).

Relationship to #42

#42 also touches this file, but only bumps runs-on: ubuntu-20.0424.04, actions/github-script@v6@v9, and actions/checkout@v5@v7. It does not touch the image reference, the input defaults, or add build_container_tag. The two changes are complementary and deliberately kept separate.

⚠️ Merge-order note: my image-line hunk ends around template line 259 and one of #42's github-script@v9 hunks starts around line 259, so whichever merges second may need a trivial context resolution. There is no semantic conflict — keep both sides (the ghcr.io image line and the @v9 bump).

Deliberately not included

  • verbose input. mockup un-comments the verbose workflow input, but its build step still hardcodes verbose: 'no' #${{ github.event.inputs.verbose }}, so the input would be exposed while doing nothing. Left as-is here rather than shipping a no-op knob fleet-wide — worth either wiring up properly or leaving commented, as a separate decision.
  • The prerelease-tag defect. release_rpms.yml:105's tag regex captures the prerelease counter as a separate optional group from the prerelease word and has no release (-R) component, which is consistent with simp-core#864 / simp-doc#461 (simp-6.6.0-Alpha.el8.noarch.rpm instead of simp-6.6.0-Alpha10-1…). Same file, separate defect — kept out to keep this reviewable.

Verification suggested before syncing to all 78 repos

A release does not have to wait on the sync: because mockup's workflow accepts target_repo + target_repo_token, RPMs can be built and attached for any repo through mockup today. Recommended order:

  1. Dispatch with dry_run=yes against one module and confirm the container pulls and the build runs.
  2. One real simple module, then one with native/provider bits.
  3. Confirm SIMP_CORE_REF_FOR_BUILDING_RPMS (reportedly still 6.5.0-1) is still viable — a real build exercises it immediately.
  4. Then sync this template out.

silug added 2 commits July 28, 2026 15:20
The docker.io/simpproject/simp_build_<os> images this template referenced have
no EL8/9/10 variants -- the newest are simp_build_centos7 (pushed 2024-08-09)
and simp_build_centos8 (2023-06-26). Builds against those die with
'NameError: uninitialized constant JSON::Fragment' (json >= 2.9 required).

Point at ghcr.io/simp/simp-el{8,9,10}-build, which are public and rebuilt
weekly, and add a build_container_tag input so a dated tag can be pinned.
Matches the proven configuration already deployed in pupmod-simp-mockup.

@hcaballero2 hcaballero2 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skeptical review — I tried to break this and mostly couldn't. What I verified before believing the PR body:

  • All three images exist and pull anonymously: docker manifest inspect ghcr.io/simp/simp-el{8,9,10}-build:latest all succeed (and a dated tag, simp-el8-build:20260720, too). No ghcr.io auth wrinkle for public runners.
  • The head-branch file is byte-identical to pupmod-simp-mockup's proven copy except the verbose input block and the runs-on/github-script/checkout versions — exactly the deltas the body claims to defer.
  • The workflow is workflow_dispatch-only, so github.event.inputs.build_container_tag is always populated; no empty-tag edge case from other triggers.
  • This is the only file in puppetsync referencing simpproject (org code search), so nothing else keeps pointing at the dead registry.
  • The failure diagnosis is real: simplib run 29344542051 (2026-07-14) dies with NameError: uninitialized constant JSON::Fragment under ruby-2.7.8/json_pure-2.6.3 inside the old container.

Two substantive concerns:

1. Syncing this template as-is would regress fleet repos to a retired runner. The baseline still carries runs-on: ubuntu-20.04 (deferred to #42), but the fleet's deployed copies have already moved on — e.g. simplib's .github/workflows/release_rpms.yml line 95 is runs-on: ubuntu-24.04, and mockup's likewise. GitHub retired ubuntu-20.04 hosted runners in April 2025, so a puppetsync run of this file before #42 lands would overwrite working 24.04 fleet copies with a workflow that fails at scheduling — trading a broken build step for a workflow that never starts. The body's "trivial context resolution" framing undersells this: #42 (or at least its runs-on hunk) is a hard prerequisite for syncing, not just a merge-order nicety. Suggest either folding the runs-on bump into this PR, or stating explicitly in the body/CHANGELOG that this file must not be synced until #42 is in.

2. Free-text build_container_os now composes nonexistent image names on stale muscle memory. See inline comment — the very run I pulled logs from was dispatched with build os: centos7 two weeks ago.

Minor: json mechanics aside (the constant is actually raised from json_pure 2.6.3's ext shim, not the 2.9 gem itself), the "container toolchain too old" root cause stands, so no change requested there.

description: "Build container OS"
required: true
default: 'centos8'
default: 'el8'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the valid values are now exactly el8/el9/el10, consider making this a type: choice input:

      build_container_os:
        description: "Build container OS"
        required: true
        type: choice
        options: [el8, el9, el10]
        default: 'el8'

With free text, anyone dispatching with old muscle memory (centos8, centos7) gets ghcr.io/simp/simp-centos7-build:latest — a nonexistent image whose pull failure only surfaces deep inside the build/signing job. This isn't hypothetical: simplib's most recent release_rpms run (29344542051, 2026-07-14) was dispatched with build os: centos7. choice also works fine for the cross-repo target_repo dispatch path (the API rejects values outside options, which is the point). If you'd rather keep it free-text for forward-compat with future ELs, at least name the valid values in the description.

CHANGELOG.md is frozen (see #71); the squash-merge PR title carries
this change's history.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants