Skip to content

refactor(ami): share provisioning steps between qemu and ami builds - #2567

Open
brainrake wants to merge 16 commits into
developfrom
martonboros/mpg-1383-share-provisioning-script
Open

brainrake wants to merge 16 commits into
developfrom
martonboros/mpg-1383-share-provisioning-script

Conversation

@brainrake

@brainrake brainrake commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Moves steps the QEMU and AMI builds duplicated into packer/scripts/provision-common.sh. Packer uploads it to /tmp, the scripts source it, and the AMI chroot scripts get a copy in /mnt/tmp.

Behavior changes

  • AMI: /var/log/wal-g is 0310 and pitr.log is 0340, was 0300. QEMU already had these. Salt sets them on v2 so vector can read pitr.log.
  • QEMU: lastlog and wtmp are emptied after the zero-fill, as on the AMI.
  • QEMU: /usr/share/doc is removed. The old line used an /mnt path and did nothing.
  • QEMU aarch64: libgcc-8-dev is dropped from the purge list. It does not exist on noble.
  • QEMU: apt-mark auto zlib1g* becomes zlib1g zlib1g-dev. Same packages, explicit names. Both return to auto, the base image state, after the Kong purge.
  • AMI stage 1 runs through run_playbook. The sed on hosts: all is gone, and runs use -i localhost, -c local or -i /mnt, -c chroot. Pipelining comes from ANSIBLE_PIPELINING, not ansible.cfg.
  • Stage 2 passes $NIX_STORE_PATHS through run_playbook. The unused postgresql_version, postgresql_major and postgresql_major_version are no longer passed.
  • AMI: the /usr/bin/chown existence guards around the log setup are gone.
  • ansible/qemu-vars.yaml is deleted. It set a var no template reads.

Also moves the qemu branches for UIDs, GIDs, the supascan baseline prefix and five service templates into ansible/vars/aws.yml and qemu.yml, selected by the existing qemu var. Rendered templates and UID and GID pins are identical for both targets.

MPG-1383

Move the Nix install, playbook run, Nix GC and report steps that
qemu-bootstrap-nix.sh and nix-provision.sh duplicated into
provision-common.sh. Both Packer builds upload it to /tmp.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Package version diff: none (aarch64-linux, x86_64-linux)

brainrake and others added 13 commits October 16, 2026 15:26
Move locale, postgres env, sshd password auth and the image cleanup
tail into provision-common.sh. The AMI chroot scripts get the library
copied into /mnt/tmp.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Share the image log setup between both builds. The AMI now ships
/var/log/wal-g as 0310 and pitr.log as 0340, matching QEMU and the
salt vector formula, so vector can read pitr.log without salt.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
clean_system calls setup_image_logs, which lives in provision-common.sh.
The surrogate script copied the library into the chroot but never
sourced it on the host.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add install_ansible to provision-common.sh for all three bootstrap
scripts. Delete ansible/qemu-vars.yaml: it set a var the templates
never read, so the QEMU splay was always the vars.yml default.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
run_playbook takes the inventory host and connection as arguments, so
the chroot run shares it with the local runs. Drop the sed that
rewrote the playbook hosts line and use -i localhost, -c local instead.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Drop the update_and_upgrade_apt copies that matched the library. Rename
the chroot cleanup_apt and report_packages, which differ from the
library versions, to clean_apt_cache and report_chroot_packages.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Move the apt mirror fallback, DEBIAN_FRONTEND export, cloud-init wait
and the unwanted-files removal into provision-common.sh. Mark only
zlib1g-dev for autoremove on QEMU.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The qemu doc removal used an /mnt path and did nothing on the live VM.
Do it in remove_unwanted_files for both targets. noble has no
libgcc-8-dev, so drop it from the aarch64 purge list.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add purge_packages to provision-common.sh with the build leftovers both
targets remove and the shared autoremove, autoclean and upgrade tail.
cleanup-qemu.sh passes its extra names. The purge sets are unchanged.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Load ansible/vars/aws.yml or qemu.yml from the playbook. The UID and
GID pins, the supascan baseline prefix and the differing values in the
adminapi, gotrue, postgres_exporter and admin-agent templates now come
from vars. Rendered templates and pins are unchanged for both targets.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cript

# Conflicts:
#	ebssurrogate/scripts/nix-provision.sh
#	ebssurrogate/scripts/qemu-bootstrap-nix.sh
@blacksmith-sh

This comment has been minimized.

…-share-provisioning-script

# Conflicts:
#	amazon-amd64-nix.pkr.hcl
#	ansible/playbook.yml
#	packer/scripts/nix-provision.sh
#	packer/scripts/provision-common.sh
#	packer/scripts/qemu-bootstrap-nix.sh
#	packer/scripts/surrogate-bootstrap-nix.sh
#	packer/stage1-nix.pkr.hcl
@brainrake
brainrake marked this pull request as ready for review October 25, 2026 10:09
@brainrake
brainrake requested a review from a team as a code owner October 25, 2026 10:09
Restore both packages to the base image state after the Kong purge.
setup-kong.yml marks them manual, and the previous change left zlib1g
manual.

Refs: MPG-1383

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brainrake
brainrake requested a review from mmlb October 25, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant