deb: install jq from distro package so deb builder images work on arm64 - #1204
Merged
Merged
Conversation
The deb builder installs jq via the amd64-only `jq-linux64` GitHub release binary. jq is used only in the release path of `scripts/fetch_and_build_deb` (GPG tag-signature verification), so this never surfaced on the jq-free nightly path -- but it hard-fails when the builder image runs on arm64, blocking arm64 release .deb builds. Switch to the distribution jq package, which is arch-independent and present in Debian/Ubuntu main across every target distro. The two jq filters used are trivial and work on any jq >= 1.5, so amd64 behavior is unchanged. Regenerated the 10 deb `-all` dockerfiles from the template so `check_docker_files_integrity` (`./update_dockerfiles` + `git diff --exit-code dockerfiles`) stays green. Verified locally: the generator reproduces these dockerfiles byte-for-byte. This is a prerequisite for the gated arm64 release deb legs (Option B); it carries no arch gate of its own and does not alter amd64 builds. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0259dd2c-9208-48f9-a8c2-8031dca8ab75
This was referenced Aug 12, 2026
ibrahim halatci (ihalatci)
marked this pull request as ready for review
August 21, 2026 13:11
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Sep 1, 2026
) Add an arm64 leg to the release build+publish workflow, gated behind the DEB_BUILD_MULTI_ARCH repo variable (default OFF), so amd64 release builds are byte-for-byte unchanged until the gate is flipped. - arch matrix dimension: ["amd64"] by default, ["amd64","arm64"] when the gate is on; arm64 is excluded for the RPM platforms (el/8, el/9, ol/8, ol/9). - arm64 legs run on native ubuntu-24.04-arm runners. - The arm64 builder + debsigner images are not published to Docker Hub yet, so build them in-job by cloning develop's tooling (this branch carries none): TEST=false ./update_image builds citus/packaging:<distro>-all, and debsigner is built from develop's Dockerfile. "docker run" has no --platform and prefers a local image, so citus_package picks up the native arm64 images automatically; amd64 keeps pulling the published images from Docker Hub. Requires the develop jq fix (#1204) so the cloned deb builder images work on arm64. Part of Track 2 of citusdata/citus#8612. Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0259dd2c-9208-48f9-a8c2-8031dca8ab75
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Sep 1, 2026
…1206) Adds a gated arm64 .deb release leg so arm64 hll packages publish to citusdata/community (the apt index feeding the stable Debian Citus images). Mirrors the merged nightly pattern (#1198) and the all-citus release leg (#1205). Everything arm64 is behind the DEB_BUILD_MULTI_ARCH repo variable (default OFF): with the gate unset the matrix stays amd64-only and the existing release pipeline is byte-for-byte unchanged. When the gate is on, arm64 runs on native ubuntu-24.04-arm runners and builds the builder + debsigner images in-job by cloning develop's tooling (this branch carries none). citus_package's docker run has no --platform flag and prefers a local image, so the native arm64 images are used automatically; amd64 keeps pulling the published images from Docker Hub. The matrix is deb-only so no RPM exclude block is needed. One file touched; no changes to debian/control.in (already Architecture: any), the deb entrypoint, or the upload script -> version strings are identical to amd64 by construction. Requires the develop jq fix (#1204) merged first for the arm64 builder images to build. Nothing runs until #1204 lands and the gate is flipped. Part of Track 2 of citusdata/citus#8612 (ARM64 Debian Docker images). Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0259dd2c-9208-48f9-a8c2-8031dca8ab75
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Sep 1, 2026
…1207) Adds a gated arm64 .deb release leg so arm64 topn packages publish to citusdata/community (the apt index feeding the stable Debian Citus images). Mirrors the merged nightly pattern (#1198) and the all-citus release leg (#1205). Everything arm64 is behind the DEB_BUILD_MULTI_ARCH repo variable (default OFF): with the gate unset the matrix stays amd64-only and the existing release pipeline is byte-for-byte unchanged. When the gate is on, arm64 runs on native ubuntu-24.04-arm runners and builds the builder + debsigner images in-job by cloning develop's tooling (this branch carries none). citus_package's docker run has no --platform flag and prefers a local image, so the native arm64 images are used automatically; amd64 keeps pulling the published images from Docker Hub. The matrix is deb-only so no RPM exclude block is needed. One file touched; no changes to debian/control.in (already Architecture: any), the deb entrypoint, or the upload script -> version strings are identical to amd64 by construction. Requires the develop jq fix (#1204) merged first for the arm64 builder images to build. Nothing runs until #1204 lands and the gate is flipped. Part of Track 2 of citusdata/citus#8612 (ARM64 Debian Docker images). Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0259dd2c-9208-48f9-a8c2-8031dca8ab75
ibrahim halatci (ihalatci)
pushed a commit
that referenced
this pull request
Sep 3, 2026
… images Apply the same amd64-only guard to the 4 deb dockerfiles that are not in os-list.csv / the arm64 build matrix (debian-buster, debian-stretch, ubuntu-bionic, ubuntu-focal) so the full deb dockerfile set stays in lockstep with templates/Dockerfile-deb.tmpl and matches the file set touched by #1204. stretch keeps its existing packagecloud.io installer URL; only the amd64 guard + comment are added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
serhat andic (serhatandic)
pushed a commit
that referenced
this pull request
Sep 8, 2026
…lock arm64 debs) (#1216) * deb: skip pg_auto_failover community repo bootstrap on non-amd64 The deb builder image bootstrap runs the PackageCloud community installer (install.citusdata.com/community/deb.sh) solely to add pg_auto_failover repos. On non-x86_64 arches that installer hard-aborts ("the Citus repository does not contain packages for non-x86_64 architectures", exit 123), failing the "Build arm64 builder image" step before any build/sign/publish and blocking all arm64 (.deb) legs. Guard the step on `dpkg --print-architecture` so it runs only on amd64 and becomes a no-op elsewhere. amd64 behavior is unchanged: the installer still runs and a failure still fails the build. Regenerated the 6 live deb dockerfiles from templates/Dockerfile-deb.tmpl. Companion arm64 fix to #1204. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * deb: guard pg_auto_failover bootstrap on the remaining non-matrix deb images Apply the same amd64-only guard to the 4 deb dockerfiles that are not in os-list.csv / the arm64 build matrix (debian-buster, debian-stretch, ubuntu-bionic, ubuntu-focal) so the full deb dockerfile set stays in lockstep with templates/Dockerfile-deb.tmpl and matches the file set touched by #1204. stretch keeps its existing packagecloud.io installer URL; only the amd64 guard + comment are added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Ibrahim Halatci <ihalatci@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What / why
The deb builder installs
jqvia the amd64-onlyjq-linux64GitHub release binary:RUN curl -sL https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 \ -o /usr/bin/jq \ && chmod +x /usr/bin/jqjqis used only in the release path ofscripts/fetch_and_build_deb(GPG tag-signatureverification —
.object.shaand.verification.verified), so it never surfaced on the jq-freenightly path. But that hardcoded x86-64 binary hard-fails when the builder image runs on
arm64, which blocks the upcoming gated arm64 release
.debbuilds.This swaps it for the distribution
jqpackage, which is architecture-independent and present inDebian/Ubuntu
mainacross every target distro:RUN apt-get update \ && apt-get install -y --no-install-recommends jq \ && rm -rf /var/lib/apt/lists/*The two
jqfilters in use are trivial and work on anyjq >= 1.5, so amd64 behavior isunchanged.
Scope (surgical)
templates/Dockerfile-deb.tmpl— the source of truth.dockerfiles/*-all/Dockerfile(debian bookworm/bullseye/buster/stretch/trixie,ubuntu bionic/focal/jammy/noble/resolute) — regenerated from the template.
11 files, identical block swap in each (
git diff --stat: 55 insertions / 44 deletions).RPM (
Dockerfile-rpm.tmpl) andpgxndockerfiles are untouched.Integrity
check_docker_files_integrityruns./update_dockerfilesthengit diff --exit-code dockerfiles.Verified locally: after staging these edits, re-running the generator reproduces the committed
dockerfiles byte-for-byte (
git diff --quiet dockerfiles→ exit 0), so the check stays green.Guardrails
DEB_BUILD_MULTI_ARCH,default OFF) arm64 release deb legs on
all-citus/debian-hll/debian-topn(follow-up PRs).Part of Track 2 of citusdata/citus#8612 (arm64 Debian Docker images). Follows the merged nightly
leg #1198.