Repository navigation
template: Update builtin templates from CentOS 5.x to Debian 13 - #14493
weizhouapache wants to merge 33 commits into
Conversation
- templates.sql: use the patched Debian 13 images (20261017-2634) from http://10.0.34.8/templates/debian-13-genericcloud-amd64/ for the KVM, XenServer (vhd.bz2) and VMware (ova) templates, with their SHA-512 checksums - scripts/util/build-acs-image.sh: new script to patch, shrink and convert a cloud image (qcow2) to VHD, OVA and XenServer VHD for CloudStack, and to create SHA512SUMS - schema-21to22.sql, schema-2214to30.sql: remove the inserts of the old CentOS 5 builtin templates (ids 4, 7 and 5)
…n 12 (64-bit) templates.sql cannot reference the 'Debian GNU/Linux 12 (64-bit)' guest OS because it is only added by a later upgrade script with an auto-generated id. Update the guest_os_id of the built-in Debian 13 templates (KVM, XenServer, VMware) in the 4.23.0.0 to 4.24.0.0 upgrade, looking the guest OS up by name. Also rename the VMware built-in template to debian13-x86_64-vmware to match the KVM and XenServer names, and drop the 4.3 upgrade statement that forced the XenServer built-in template back to guest OS 142.
get_suitable_test_template now returns the shared public test template (tiny-kvm on KVM), registered by the admin only when it is missing, and falls back to the built-in template. It was already doing this for XenServer. The hypervisor argument is optional and is detected from the hosts when not given. Tests that only need a small guest now use it instead of the built-in template, and test_ipv4_routing, test_snapshots and test_restore_vm no longer register their own copy of the tiny template. Tests that depend on the built-in template are not changed, for example test_deploy_vm_root_resize (needs a template of at least 1 GiB), test_templates, test_sharedfs_lifecycle and test_vm_autoscaling.
…again TestSnapshotStandaloneBackup takes a snapshot, expunges the VM and then deletes its template, to test the snapshot without its template. It needs its own template, so it cannot use the shared public test template.
The Debian 13 cloud image has no NFS client, so mounting an NFS share in the guest fails with "NFS: mount program didn't pass remote address". The shared file system smoke test mounts the share in the guest. Install nfs-common with the other guest packages.
Moving the smoke tests to the shared tiny-kvm template broke the tests that need more from the guest. Run these on the Debian 13 built-in template, taken with get_template on every hypervisor. - test_attach_multiple_volumes, test_volumes: they detach data disks from a running VM, which the tiny-kvm guest never releases. - test_backup_recovery_nas, test_backup_recovery_kboss: they create a VM from a backup and expect the files written in the guest to be in it. - test_primary_storage: it puts a primary storage with running VMs in maintenance, which needs a guest that shuts down on request. - test_volumes: use the 512 MiB big offering, as the Debian 13 guest does not boot with the 128 MiB tiny or the 256 MiB small offering. - test_vm_autoscaling: the scale up condition needs at least 1 percent CPU, and an idle Debian 13 VM uses about 0.4 percent. Start a CPU loop from the template user data.
… templates The images were rebuilt with the nfs-common package installed. The URLs are unchanged, so the KVM, XenServer and VMware templates get the checksums of the new files.
…n templates The images are built from the Debian 13 genericcloud image of 20261021-2638 with the nfs-common package installed. Update the URLs and the SHA-512 checksums of the KVM, XenServer and VMware templates.
The tests write a file in the guest and take a live backup a few seconds later. The backup is not quiesced, so a file that was not flushed yet can be missing from the backup. The guest root filesystem commits its journal about every 5 seconds, which is also the wait before the backup. Run sync after each touch, so that the file is on the disk before the backup starts.
boring-cyborg
Bot
added
component:integration-test
component:marvin
Python
|
@blueorangutan package |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14493 +/- ##
============================================
+ Coverage 20.75% 20.79% +0.03%
- Complexity 21901 21936 +35
============================================
Files 6496 6496
Lines 591602 591690 +88
Branches 72583 72593 +10
============================================
+ Hits 122814 123017 +203
+ Misses 455456 455322 -134
- Partials 13332 13351 +19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19708 |
|
@blueorangutan test matrix |
|
@weizhouapache a [SL] Trillian-Jenkins matrix job (ol8 mgmt + kvm-ol8, ubuntu24 mgmt + kvm-ubuntu24, ol8 mgmt + vmware-70u3, ol9 mgmt + xcpng82) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-17238) |
|
[SF] Trillian Build Failed (tid-17240) |
|
[SF] Trillian Build Failed (tid-17239) |
|
[SF] Trillian Build Failed (tid-17237) |
|
[SF] Trillian test result (tid-133)
|
|
[SF] Trillian test result (tid-135)
|
|
[SF] Trillian test result (tid-132)
|
|
[SF] Trillian test result (tid-137)
|
The usage text is printed from the header comment, so the line range it prints is moved down by the length of the license.
…d.cloudstack.org The images are the same 20261021-2638 files with the same checksums, now served from download.cloudstack.org instead of the build server.
An error answer, like the timeout when the secondary storage VM disconnects during a download, has no install path. The download callback wrote it to the template_store_ref entry anyway, so the install path became null and the next download of the template went to a directory named "null". Only update the install path when the answer has one.
|
@blueorangutan package |
|
@blueorangutan package |
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
guest_os_idto NULL if the subquery returns no row (which is plausible… · New
Review effort: Lite
|
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✔️ suse15. SL-JID 19912 |
* vmware-base: fix template copy from secondary to primary * vmware-base: refactoring * test: revert curl commands to wget in test_routers_network_ops (PR 13977) Revert test_routers_network_ops.py back to wget commands after fixing underlying macchina VM file system corruption issue. Context: - Previous commit changed wget to curl due to libpcre.so.1 library error in macchina VMs - Error was: 'wget: /usr/lib/libpcre.so.1 is not an ELF file' - Root cause: file system corruption during template copy from secondary to primary storage - Fix: PR 13977 (vmware-base: fix template copy from secondary to primary) resolves the issue - This revert verifies wget is working correctly again after the underlying fix Changes: - Replace 'curl -v -m 1 -o index.html -sL www.google.com' back to 'wget -t 1 -T <timeout> www.google.com' - Affects TestRedundantIsolateNetworks and TestIsolatedNetworks test classes - Apply timeout values of 5 or 1 second depending on test context
Added EXISTS clause to ensure guest OS is available before updating vm_template records for Debian 13.
e913f50 to
31fcf80
Compare
|
@blueorangutan package |
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
4 open issues
download.cloudstack.org, but… · Newdownload.cloudstack.org, but… · New
1 resolved since last review
Review effort: Lite
There was a problem hiding this comment.
Copilot review overview
4 open issues
download.cloudstack.org, but…download.cloudstack.org, but…
Review effort: Balanced
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19915 |
There was a problem hiding this comment.
🔵 Needs a closer look
Human assessment is needed to evaluate unpublished risks involving zone-specific image selection, template userdata overrides, stalled VMware uploads, and stale image checksums.
No open issues 🎉
4 resolved since last review
download.cloudstack.org, but…download.cloudstack.org, but…
Review effort: Balanced
|
@blueorangutan package |
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19924 |
|
@blueorangutan test matrix |
|
@weizhouapache a [SL] Trillian-Jenkins matrix job (ol8 mgmt + kvm-ol8, ubuntu24 mgmt + kvm-ubuntu24, ol8 mgmt + vmware-70u3, ol9 mgmt + xcpng82) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-17499) |
|
[SF] Trillian Build Failed (tid-17498) |
|
[SF] Trillian Build Failed (tid-17497) |
|
[SF] Trillian Build Failed (tid-17496) |
Description
This PR replaces the built-in CentOS 5.x templates with Debian 13 (trixie) templates, and moves the smoke tests to them. More discussion on #13797
Templates
setup/db/templates.sql: the KVM (qcow2), XenServer (vhd.bz2) and VMware (ova) built-in templates are the Debian 13genericcloudamd64 images (20261026-2643) patched for CloudStack, downloaded fromdownload.cloudstack.org, with their SHA-512 checksums.scripts/util/build-acs-image.sh: new script that patches, shrinks and converts a cloud image (qcow2) to VHD, OVA and XenServer VHD for CloudStack, and createsSHA512SUMS. It installsnfs-commonin the Debian and Ubuntu images, because the guest needs an NFS client to mount a shared file system.schema-21to22.sqlandschema-2214to30.sql.Debian GNU/Linux 12 (64-bit), looked up by name because the guest OS is only added by a later upgrade script with an auto-generated id. The VMware template is renameddebian13-x86_64-vmwareto match the KVM and XenServer names. The 4.3 upgrade statement that forced the XenServer built-in template back to guest OS 142 is dropped.Fix for the template download
template_store_refentry. Before, the install path became null and the next download of the template went to a directory namednull, which failed.Smoke tests
get_suitable_test_templatereturns the shared public test template (tiny-kvmon KVM), registered by the admin only when it is missing, and falls back to the built-in template. Tests that only need a small guest use it.get_templateon every hypervisor:test_attach_multiple_volumes,test_volumes(with the 512 MiB offering),test_backup_recovery_nas,test_backup_recovery_kboss,test_primary_storage,test_vm_autoscaling.test_snapshots: the standalone backup class registers its own template again, because it deletes the template.test_backup_recovery_nas:syncthe guest after writing the marker files, so that they are in the backup.Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
DOWNLOAD_ERRORand kept its install path (template/tmpl/<account id>/<template id>). After the next sync the template was downloaded to that path, and nonulldirectory was created. Before the fix the install path was cleared.BaseImageStoreDriverImplTest(the install path is kept for an error answer without one and set when the answer has one).How did you try to break this feature and the system with this change?