e2e coverage for git and OCI remote stacks - #14226
Conversation
FromRemote switches a scenario's -f to a reference handled by compose's remote loaders (git URL, oci:// artifact) with optional root flags such as --insecure-registry, while the anchored testdata copy keeps serving as the local content the remote is built from. ContainerEnv joins the check vocabulary: the container-config environment is the natural observable that a remote project's bundled files (env_file) were consumed, whatever the source of the model. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The remote loaders had almost no end-to-end coverage: one test consumed an OCI artifact (config only, plus an up declined at the prompt) and nothing exercised the git loader at all. Git scenarios run against a throwaway repository served over the smart HTTP protocol by an in-process server — git http-backend as a CGI per httptest request: no daemon, no container, no fixed port. Smart HTTP is a hard requirement: the loader ls-remotes the ref then shallow-fetches the raw commit, which the dumb protocol cannot serve (no shallow capability) and which needs uploadpack.allowAnySHA1InWant. Covered: deploying from the default branch with repository-relative files, selecting a #branch, selecting a #ref:subdir project. OCI scenarios publish the fixture to a throwaway local registry (the TestPublish pattern) then deploy from oci://: a full up consuming the bundled env-file layer, and tag selection between two published revisions. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
3 findings in new test infrastructure. No issues with existing code.
| r.t.Helper() | ||
| cmd := exec.Command("git", args...) | ||
| cmd.Dir = dir | ||
| cmd.Env = append(os.Environ(), |
There was a problem hiding this comment.
[medium] git() appends env overrides to os.Environ() — first-match semantics on Linux mean pre-existing CI values shadow the /dev/null overrides
cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", ...) produces a slice where the original env vars come first and the isolation overrides come last. On Linux, glibc's getenv() — which git uses internally — returns the first match when a key appears multiple times. If the CI runner already exports GIT_CONFIG_GLOBAL (e.g. pointing at a signing config), the override at the tail of the slice has no effect: git reads the runner's config and the intended isolation breaks.
The same applies to GIT_AUTHOR_NAME/GIT_COMMITTER_NAME if the CI sets those via standard git env vars.
Fix: filter out the keys you're about to override before appending:
| cmd.Env = append(os.Environ(), | |
| env := slices.DeleteFunc(os.Environ(), func(e string) bool { | |
| return strings.HasPrefix(e, "GIT_CONFIG_GLOBAL=") || | |
| strings.HasPrefix(e, "GIT_CONFIG_SYSTEM=") || | |
| strings.HasPrefix(e, "GIT_AUTHOR_NAME=") || | |
| strings.HasPrefix(e, "GIT_AUTHOR_EMAIL=") || | |
| strings.HasPrefix(e, "GIT_COMMITTER_NAME=") || | |
| strings.HasPrefix(e, "GIT_COMMITTER_EMAIL=") | |
| }) | |
| cmd.Env = append(env, |
| Confidence | Score |
|---|---|
| 🟡 moderate | 75/100 |
There was a problem hiding this comment.
Not applicable to processes spawned through os/exec: exec.Cmd dedupes the environment before starting the child, keeping the LATER entry (dedupEnv, os/exec/exec.go: 'removed, in favor of later values'). Verified empirically: with GIT_CONFIG_GLOBAL=/ci/injected/config exported in the parent and cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null"), the child observes /dev/null. glibc first-match semantics never come into play because the duplicate never reaches the child's environment block. Keeping the idiomatic append.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
git init -b requires git >= 2.28; init followed by symbolic-ref names the initial branch on any version. Branch() now returns the working tree to main so successive calls cut from the same base instead of stacking on the previous branch. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
docker-agent
left a comment
There was a problem hiding this comment.
This review covers only the commits pushed since 5109a75.
Assessment: 🟢 APPROVE
The two changes in this increment are both correct:
-
Version-proof git fixture init — replacing
git init -q -b main .withgit init -q .followed bygit symbolic-ref HEAD refs/heads/mainis the canonical, version-agnostic way to name an unborn branch. Git allows HEAD to be a symbolic ref pointing to a not-yet-existing ref (refs/heads/main), and the first commit creates that ref. The sequence (init → symbolic-ref → add → commit → clone) is correctly ordered. This directly addresses the finding from the previous review cycle. -
Branch cuts from main — adding
r.git(r.work, "checkout", "-q", "main")at the end ofBranch()is safe: by that pointgit add -Aandgit commithave already run, so the working tree is clean and the checkout cannot fail on an unclean-tree basis. This ensures eachBranch()call cuts from the same base commit, directly addressing the second finding from the previous review cycle.
No new issues found in the changed lines.
What this PR does, in one sentence
End-to-end coverage for remote compose sources: five scenarios deploying projects from git repositories and OCI artifacts, against fully hermetic in-process infrastructure.
Context
The remote loaders are almost invisible to the e2e suite today: a single test consumes an OCI artifact —
configoutput plus anupdeliberately declined at the confirmation prompt — and nothing exercises the git loader at all. No test ever deploys a remote project, so the behaviors users actually rely on (relative files resolved against the fetched copy, branch and subdirectory selection, bundled env files, tag routing) are locked by nothing. Issue #14224 recently showed what lives in that blind spot.What the PR brings
Three git scenarios — deploy from the default branch with repository-relative
env_file, select a#branch, select a#ref:subdirproject past a decoy at the repository root — and two OCI scenarios: a fullupfromoci://consuming the bundled env-file layer (the first e2e that actually deploys an artifact), and tag selection between two published revisions.The infrastructure stays hermetic. Git repositories are throwaway fixtures served over the smart HTTP protocol by an in-process server:
git http-backendrun as a CGI perhttptestrequest — no daemon process, no container, no fixed port. Smart HTTP is a hard requirement, not a preference: the loader resolves the ref withls-remotethen shallow-fetches the raw commit, which the dumb protocol cannot serve (no shallow capability) and which needsuploadpack.allowAnySHA1InWant. OCI scenarios reuse theTestPublishpattern: publish to a throwaway localregistry:3, consume with--insecure-registry. Every scenario isolates its remote-resource cache (XDG_CACHE_HOME), and the git helper neutralizes any developer configuration (GIT_CONFIG_GLOBAL=/dev/null) so signing or hook setups cannot leak into fixtures.The scenario DSL gains the one primitive this needs:
FromRemote(source, rootFlags...)switches the scenario's-fto a remote reference while the anchored testdata copy keeps serving as the local content the remote is built from — steps before the switch (thepublish) still run against it.ContainerEnvjoins the check vocabulary as the state-based observable that a remote project's bundled files were consumed.Why this is the right next brick
Remote-source behavior is where reconciliation, path resolution and caching intersect (see #14224): before touching any of it, the intended behaviors need to be pinned. This suite establishes the harness those future fixes will extend — a regression test for #14224 becomes a one-scenario addition.