Skip to content

e2e coverage for git and OCI remote stacks - #14226

Merged
glours merged 3 commits into
docker:mainfrom
ndeloof:e2e-remote-stacks
Sep 16, 2026
Merged

glours merged 3 commits into
docker:mainfrom
ndeloof:e2e-remote-stacks

Conversation

@ndeloof

@ndeloof ndeloof commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

What this PR does, in one sentence

End-to-end coverage for remote compose sources: five scenarios deploying projects from git repositories and OCI artifacts, against fully hermetic in-process infrastructure.

Context

The remote loaders are almost invisible to the e2e suite today: a single test consumes an OCI artifact — config output plus an up deliberately declined at the confirmation prompt — and nothing exercises the git loader at all. No test ever deploys a remote project, so the behaviors users actually rely on (relative files resolved against the fetched copy, branch and subdirectory selection, bundled env files, tag routing) are locked by nothing. Issue #14224 recently showed what lives in that blind spot.

What the PR brings

Three git scenarios — deploy from the default branch with repository-relative env_file, select a #branch, select a #ref:subdir project past a decoy at the repository root — and two OCI scenarios: a full up from oci:// consuming the bundled env-file layer (the first e2e that actually deploys an artifact), and tag selection between two published revisions.

The infrastructure stays hermetic. Git repositories are throwaway fixtures served over the smart HTTP protocol by an in-process server: git http-backend run as a CGI per httptest request — no daemon process, no container, no fixed port. Smart HTTP is a hard requirement, not a preference: the loader resolves the ref with ls-remote then shallow-fetches the raw commit, which the dumb protocol cannot serve (no shallow capability) and which needs uploadpack.allowAnySHA1InWant. OCI scenarios reuse the TestPublish pattern: publish to a throwaway local registry:3, consume with --insecure-registry. Every scenario isolates its remote-resource cache (XDG_CACHE_HOME), and the git helper neutralizes any developer configuration (GIT_CONFIG_GLOBAL=/dev/null) so signing or hook setups cannot leak into fixtures.

The scenario DSL gains the one primitive this needs: FromRemote(source, rootFlags...) switches the scenario's -f to a remote reference while the anchored testdata copy keeps serving as the local content the remote is built from — steps before the switch (the publish) still run against it. ContainerEnv joins the check vocabulary as the state-based observable that a remote project's bundled files were consumed.

Why this is the right next brick

Remote-source behavior is where reconciliation, path resolution and caching intersect (see #14224): before touching any of it, the intended behaviors need to be pinned. This suite establishes the harness those future fixes will extend — a regression test for #14224 becomes a one-scenario addition.

FromRemote switches a scenario's -f to a reference handled by compose's
remote loaders (git URL, oci:// artifact) with optional root flags such
as --insecure-registry, while the anchored testdata copy keeps serving
as the local content the remote is built from. ContainerEnv joins the
check vocabulary: the container-config environment is the natural
observable that a remote project's bundled files (env_file) were
consumed, whatever the source of the model.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The remote loaders had almost no end-to-end coverage: one test consumed
an OCI artifact (config only, plus an up declined at the prompt) and
nothing exercised the git loader at all.

Git scenarios run against a throwaway repository served over the smart
HTTP protocol by an in-process server — git http-backend as a CGI per
httptest request: no daemon, no container, no fixed port. Smart HTTP is
a hard requirement: the loader ls-remotes the ref then shallow-fetches
the raw commit, which the dumb protocol cannot serve (no shallow
capability) and which needs uploadpack.allowAnySHA1InWant. Covered:
deploying from the default branch with repository-relative files,
selecting a #branch, selecting a #ref:subdir project.

OCI scenarios publish the fixture to a throwaway local registry (the
TestPublish pattern) then deploy from oci://: a full up consuming the
bundled env-file layer, and tag selection between two published
revisions.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
@ndeloof
ndeloof requested review from a team as code owners September 16, 2026 06:45

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

3 findings in new test infrastructure. No issues with existing code.

Comment thread pkg/e2e/remote_git_test.go Outdated
r.t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] git() appends env overrides to os.Environ() — first-match semantics on Linux mean pre-existing CI values shadow the /dev/null overrides

cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", ...) produces a slice where the original env vars come first and the isolation overrides come last. On Linux, glibc's getenv() — which git uses internally — returns the first match when a key appears multiple times. If the CI runner already exports GIT_CONFIG_GLOBAL (e.g. pointing at a signing config), the override at the tail of the slice has no effect: git reads the runner's config and the intended isolation breaks.

The same applies to GIT_AUTHOR_NAME/GIT_COMMITTER_NAME if the CI sets those via standard git env vars.

Fix: filter out the keys you're about to override before appending:

Suggested change
cmd.Env = append(os.Environ(),
env := slices.DeleteFunc(os.Environ(), func(e string) bool {
return strings.HasPrefix(e, "GIT_CONFIG_GLOBAL=") ||
strings.HasPrefix(e, "GIT_CONFIG_SYSTEM=") ||
strings.HasPrefix(e, "GIT_AUTHOR_NAME=") ||
strings.HasPrefix(e, "GIT_AUTHOR_EMAIL=") ||
strings.HasPrefix(e, "GIT_COMMITTER_NAME=") ||
strings.HasPrefix(e, "GIT_COMMITTER_EMAIL=")
})
cmd.Env = append(env,
Confidence Score
🟡 moderate 75/100

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not applicable to processes spawned through os/exec: exec.Cmd dedupes the environment before starting the child, keeping the LATER entry (dedupEnv, os/exec/exec.go: 'removed, in favor of later values'). Verified empirically: with GIT_CONFIG_GLOBAL=/ci/injected/config exported in the parent and cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null"), the child observes /dev/null. glibc first-match semantics never come into play because the duplicate never reaches the child's environment block. Keeping the idiomatic append.

Comment thread pkg/e2e/remote_git_test.go
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

git init -b requires git >= 2.28; init followed by symbolic-ref names
the initial branch on any version. Branch() now returns the working
tree to main so successive calls cut from the same base instead of
stacking on the previous branch.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review covers only the commits pushed since 5109a75.

Assessment: 🟢 APPROVE

The two changes in this increment are both correct:

  1. Version-proof git fixture init — replacing git init -q -b main . with git init -q . followed by git symbolic-ref HEAD refs/heads/main is the canonical, version-agnostic way to name an unborn branch. Git allows HEAD to be a symbolic ref pointing to a not-yet-existing ref (refs/heads/main), and the first commit creates that ref. The sequence (init → symbolic-ref → add → commit → clone) is correctly ordered. This directly addresses the finding from the previous review cycle.

  2. Branch cuts from main — adding r.git(r.work, "checkout", "-q", "main") at the end of Branch() is safe: by that point git add -A and git commit have already run, so the working tree is clean and the checkout cannot fail on an unclean-tree basis. This ensures each Branch() call cuts from the same base commit, directly addressing the second finding from the previous review cycle.

No new issues found in the changed lines.

@glours glours left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@glours
glours merged commit 1a1df10 into docker:main Sep 16, 2026
92 of 95 checks passed
@ndeloof
ndeloof deleted the e2e-remote-stacks branch September 16, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants