Repository navigation
Conversation
SDK never had any package listings, so I didn't bother with generating the basic ones. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
That way sys-devel/gcc will show up in a package listings without weird hacks. Also removes an obstacle for moving to a newer binary package format. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
sys-devel/gcc is installed normally, so it will show up in package database. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Implicit packages were mostly packages stored in the initramfs. Previously we took package names sys-kernel/coreos-kernel DEPEND, filtered out all the entries not ending with a slot followed by `=` and filtering out the entries already installed in the production image. It had following weaknesses: - Taking into account only packages with a slot ending with `=` was too limiting, and I don't know what was the justification here. sys-apps/busybox and app-admin/google-guest-configs were skipped because of that (and both of those packages are in initramfs only). - Doing heuristics around equery listing all available versions of the package to pick one that would be installed in the initramfs isn't really necessary. Instead, we can just do a query for the package in the board sysroot - the sysroot should contain all the packages by now. - sys-boot/grub, sys-boot/shim and sys-boot/shim-signed were missing from the package listings, because they weren't installed into the image using emerge, but some of our scripts. Also, refactor this code a bit, so I can add another package listings with a different format later. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
…and developer containers Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Earlier the flatcar_target was using ROOT and SYSROOT variables to
determine the target. The problem is that ROOT may be undefined in
src_* phase functions and SYSROOT may be undefined in most of pkg_*
phase functions. We got away with using those because portage wasn't
really strict with it, but it recently had some work done about not
exporting some variables into the environment, so it's better to
become a bit future-proof.
The improved flatcar_target function started using extended patterns
which are enabled by using "shopt -s extglob". The problem with it is
that extglob needs to be enabled by the time the code with extended
patterns is parsed, which means enabling it at the toplevel
scope. This could be done in two ways. First is to call `shopt -s
extglob` before function definition and revert to the previous state
after it:
revert=$(shopt -p extglob)
shopt -s extglob
function_with_extended_patterns() { … }
${revert}
Second way is to put the body of the function to a separate file and
have a stub function that will enable extglob, source the file (thus
parsing and executing the code there) and revert the change:
actual_function() {
revert=$(shopt -p extglob)
shopt -s extglob
source the_external_file
${revert}
}
I went with the latter - the former resulted in errors inside
portage's ebuild script, because the function body with an extended
pattern ended up being stored inside the some environment file and the
script is not enabling extglob, thus not being able to parse the
environment file:
/build/amd64-usr/var/tmp/portage/coreos-base/ajwaj-0/temp/environment: line 461: syntax error near unexpected token `('
/build/amd64-usr/var/tmp/portage/coreos-base/ajwaj-0/temp/environment: line 461: ` /build/+([^/])-usr)'
* ERROR: coreos-base/ajwaj-0::coreos-overlay failed (unpack phase):
* error sourcing environment
*
* Call stack:
* ebuild.sh, line 580: Called die
* The specific snippet of code:
* source "${T}"/environment || die "error sourcing environment"
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
I think it's better to check if current target is what we want instead of not being what we don't want, e.g. if current target is some generic image, instead of relying on anything not being an SDK being a generic image. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
This does not seem to always work for mysterious reasons (didn't do squat for sys-devel/gcc, for example), so replace it with post pre_pkginst hook that does the removals. Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
Signed-off-by: Krzesimir Nowak <knowak@microsoft.com>
There was a problem hiding this comment.
🟡 Changes recommended
GCC file ownership, cached GLib installation, and report generation for reused SDK stages remain unresolved.
3 open issues
What changed in this PR
This PR expands package and file inventories across Flatcar images and SDK builds, replaces manual GCC library extraction with package hooks, and refines build-target detection.
Changes:
- Add structured package listings and SDK stage content reports, including publication through CI.
- Include more indirectly installed packages in image inventories.
- Replace the GCC extraction workaround and simplify target-specific hooks and shell helpers.
| File | Description |
|---|---|
sdk_lib/sdk_container_common.sh |
Replace external version-parsing commands with Bash builtins. |
sdk_container/src/third_party/coreos-overlay/profiles/coreos/targets/generic/prod/package.provided |
Mark compiler-only dependencies as provided. |
sdk_container/src/third_party/coreos-overlay/profiles/coreos/base/profile.bashrc.flatcar-target |
Add phase-aware build-target detection. |
sdk_container/src/third_party/coreos-overlay/profiles/coreos/base/profile.bashrc |
Wrap target detection with extglob handling. |
sdk_container/src/third_party/coreos-overlay/coreos/config/env/sys-fs/lvm2 |
Explicitly select generic targets. |
sdk_container/src/third_party/coreos-overlay/coreos/config/env/sys-devel/gcc |
Retain and relocate production runtime libraries. |
sdk_container/src/third_party/coreos-overlay/coreos/config/env/net-misc/openssh |
Move file exclusions into a pre-install hook. |
sdk_container/src/third_party/coreos-overlay/coreos/config/env/dev-libs/glib |
Remove introspection files through a pre-install hook. |
sdk_container/src/third_party/coreos-overlay/coreos/config/env/app-admin/google-guest-configs |
Normalize the board-target comparison. |
sdk_container/src/third_party/coreos-overlay/coreos-base/coreos/coreos-0.0.1.ebuild |
Add GCC as a runtime dependency. |
sdk_container/src/third_party/coreos-overlay/coreos-base/coreos/coreos-0.0.1-r322.ebuild |
Add the revised metapackage with the GCC dependency. |
ci-automation/sdk_bootstrap.sh |
Digest, sign, and upload SDK listings. |
build_sysext |
Generate extended system-extension package listings. |
build_library/sysext_prod_builder |
Pass the base-extension build target explicitly. |
build_library/reports_util.sh |
Simplify the include guard. |
build_library/prod_image_util.sh |
Drop manual GCC extraction and add extended listings. |
build_library/dev_container_util.sh |
Add extended listings and explicit developer targeting. |
build_library/catalyst.sh |
Generate package and content reports for SDK stages. |
build_library/build_image_util.sh |
Expand inventory helpers and implicit package coverage. |
build_library/board_options.sh |
Remove the unused package-provides helper. |
Review effort: Balanced
| --file "$TEMPDIR/${stage}.spec" | ||
| ln -sf "$stage-${ARCH}-${FLAGS_version}.tar.bz2" \ | ||
| "$BUILDS/$stage-${ARCH}-latest.tar.bz2" | ||
| generate_stage_listings "${stage}" |
Comment on lines
+10
to
+12
| for f in "${ED}/usr/lib/gcc/${CTARGET}/${GCCMAJOR}/"*.so*; do | ||
| mv "${f}" "${EROOT}/usr/lib64/" || die | ||
| done |
| fi | ||
| cros_post_pkg_preinst_drop_gobject_introspection() { | ||
| [[ $(flatcar_target) = 'generic-'* ]] || return | ||
| rm -r "${ED}"/usr/bin/gi-* "${ED}"/usr/lib*/libgirepository-2.0* || die |
chewi
requested changes
Oct 22, 2026
| grep -v '^!' | \ | ||
| grep -v -F 'sys-kernel/coreos-sources' | \ | ||
| sort -u) | ||
| shopt -s nullglob |
Contributor
There was a problem hiding this comment.
You need to unset this afterwards. Checking with [[ -d ... ]] in the loop would be safer.
Comment on lines
+38
to
+42
| local revert_shopt=$(shopt -p extglob) | ||
| shopt -s extglob | ||
| source "${BASH_SOURCE[0]}.flatcar-target" target_type | ||
| echo "${target_type}" | ||
| ${revert_shopt} |
Contributor
There was a problem hiding this comment.
I think you can use a subshell to avoid using a separate file and having to revert. I've done this in games-fps/ut2004-data.
Contributor
There was a problem hiding this comment.
Yeah, I can't remember precisely, but I think that's why I did it the new way in app-admin/google-guest-configs.
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI: https://jenkins.flatcar.org/job/container/job/sdk/260/cldsv/
This looks like a bag of assorted fixes and improvements (probably best reviewed commit by commit):
${packlage} ${version} ${slot} ${repository}. Easier to parse by scripts.Example extended package listing: https://bincache.flatcar-linux.net/images/amd64/4858.0.0+more-package-listings-13/flatcar_production_image_packages_extended.txt
Same for SDK (stage 4): https://bincache.flatcar-linux.net/sdk/amd64/4858.0.0+more-package-listings-13/flatcar-sdk-amd64-4858.0.0+more-package-listings-13-stage4-packages_extended.txt
SDK contents (stage 1): https://bincache.flatcar-linux.net/sdk/amd64/4858.0.0+more-package-listings-13/flatcar-sdk-amd64-4858.0.0+more-package-listings-13-stage1-contents_wtd.txt