fix(up): make the launch barrier mean what dependents need — completed, and resolvable

[Mikimoto]

Summary

Two ways up released a service before its dependents could actually use it. Both predate concurrent launch; launching one at a time simply lost the races almost never.

Stacked on #145. The first two commits are #144 and #145 (OrtegaMatias); this PR's own commits are the last two. The fixes touch configService's signature and the launch loop, which are that PR's shape — re-basing them onto today's main would mean re-authoring against a loop that is about to be replaced. Happy to rebase once #145 lands.

A one-shot recorded as running

waitUntilServiceStarted returned .running the moment it observed that status. A one-shot only passes through .running on its way to .stopped, so an init container still working at the first 0.5s poll was recorded as running, and every dependent declaring service_completed_successfully was then refused:

Service 'etcd2' depends on 'etcd-init' with condition
'service_completed_successfully', but 'etcd-init' has not completed successfully.

Whether a service must be waited out is not readable from the service itself — it lives on whoever depends on it. Service.servicesRequiredToComplete collects those names and configService passes awaitCompletion down, so the readiness wait keeps polling for .stopped. maxWait still backstops a compose file that declares the condition on something which never exits.

With a chown -R init and six VMs booting at once this was reliable rather than occasional.

A container running before its name resolves

Docker's embedded DNS updates as the container joins the network, so depends_on there implies resolvability and compose files rely on that. Apple Container publishes the record a moment after the container is up.

haproxy resolves every server line while parsing its config and refuses to start if one fails, so a proxy in the next wave died on

[ALERT] 'server postgres_replicas/patroni2' : could not resolve address 'patroni2'.
[ALERT] Failed to initialize server(s) addr.

with patroni2 running and healthy — the same name resolved from another container seconds later.

After recording a service's IP, up now polls the host resolver for the name the daemon registers (<container>.<dns domain>, answerable via /etc/resolver once a DNS domain is configured) until it appears. The wave barrier then also gates on DNS, which is what the next wave actually needs.

Advisory rather than fatal: on timeout it warns and proceeds, since a compose file may name a container the daemon never registers.

Rejected first

Giving haproxy a resolvers section with init-addr last,libc,none, so it would start with unresolvable backends marked down and re-resolve later. haproxy's own resolver does not expand search domains, so it queried the bare patroni2, cached NXDOMAIN, and parked every backend in maintenance permanently — strictly worse than the race it was meant to fix. The asymmetry is between the two runtimes' DNS timing, so the fix belongs here.

Verification

swift test --filter Container_Compose_StaticTests
✔ Test run with 252 tests in 24 suites passed

Ten new tests cover the two pure pieces: which services must be waited out (list vs map depends_on, each condition, one init shared by several dependents), and the registration name (plain, already-qualified, multi-label domain).

The waits themselves are not covered — both need a live daemon, as does waitUntilServiceStarted, which has no test today either. They are covered only by an end-to-end run: 23 services, all up, health checks green.

[Mcrich23]

These are useful fixes. A one-shot that runs silently for more than 30 seconds still appears to hit the inactivity timeout, even after reaching .running. Can you separate that from waiting for startup and add a dynamic regression? For DNS, please verify resolution from a dependent container; a successful host lookup doesn't establish that the dependent can resolve the name.

[Mikimoto]

These are useful fixes. A one-shot that runs silently for more than 30 seconds still appears to hit the inactivity timeout, even after reaching .running. Can you separate that from waiting for startup and add a dynamic regression? For DNS, please verify resolution from a dependent container; a successful host lookup doesn't establish that the dependent can resolve the name.

Thanks. Both are right, and the DNS one has a cause worth naming before I write the test.

One-shot vs. startup. Confirmed — idleTimeout defaults to 30s and applies to the same wait that is being used for "has it started", so a one-shot that runs silently past that window trips it even after reaching .running. Those are two different questions and I'll split them: reaching .running ends the startup wait, and a restart: "no" service then gets waited to completion under its own policy rather than an inactivity clock. Dynamic regression with a one-shot that sleeps quietly past the window and then exits 0.

DNS from a dependent container. Agreed, and the reason a host lookup is misleading here is worse than sampling the wrong place. On this branch the domain is still derived from the compose project name:

 if let derived = ComposeProject.sanitizeDnsDomain(projectName) {
     dnsDomain = derived
     dnsAvailable = await checkDnsDomainRegistered(derived)

An operator who registered a domain that isn't their project's name gets dnsAvailable == false, so --dns-domain is never passed and the barrier this PR adds never runs at all — it falls back to the /etc/hosts cross-patcher, which is best-effort by design and reports nothing. The cluster comes up looking fine. A host-side container system dns list check passes the whole time, which is exactly the gap you're pointing at.

We hit this downstream and fixed it as a separate commit: read the globally configured [dns] domain from container system property list and try it first, keeping the project-derived name as the second candidate so projects that did register their own name are unaffected. Both decisions are pure and unit-tested. Happy to bring it into this PR — the barrier isn't really testable from a dependent container until it runs — or keep it separate if you'd rather review it on its own. Your call.

Rebase. Noticed #144 merged, so bba4162 on this branch is now a duplicate of what's on main and will drop on the next rebase. 85a13fc (#145) stays until that one lands.

[Mikimoto]

Pushed, rebased onto current main — which dropped bba4162 automatically, since #144 is now upstream. Three commits, down from four.

One-shot completion vs. startup. You were right that they were entangled, and fixing the first one exposed a second cap underneath it.

The idle timeout exists for a container stuck coming up: an image pull streams progress, so silence with nothing running means genuinely stuck rather than slow. Once the container has been seen running that reasoning stops holding — a chown -R over a populated volume prints nothing for minutes, which is the normal case. waitVerdict now applies the idle budget only before the container starts; maxWait still bounds the whole wait. The post-start message no longer claims the container timed out "waiting to be running", which it demonstrably was not.

With that out of the way the 45-second regression still failed — at ten seconds, on XPC timeout for request to com.apple.container.apiserver/containerWait. containerWait answers when the container exits, so its responseTimeout: .seconds(10) was never a request budget; it was a cap on how long a one-shot is allowed to run. Any init doing real work hit it, with an error naming neither the service nor what was being waited for. It now matches the wait loop's ceiling, so one number bounds "how long may a one-shot take".

Seven static tests on waitVerdict, plus a dynamic regression: a one-shot that sleeps 45 seconds in silence with a dependent gated on service_completed_successfully. It asserts up both succeeds and actually waited — ~52s. Mutation-checked in both directions: re-applying the idle budget to a started container reddens three of the static tests, and putting the XPC timeout back to ten seconds reproduces the original XPC failure exactly.

DNS from a dependent container. Added, and your reasoning holds — the barrier's getaddrinfo runs in the compose process against the host's resolver, while a dependent resolves through its own /etc/resolv.conf against the daemon's DNS from inside the container network. The new test starts a real pair and asks the dependent: its resolv.conf carries the domain, and getent hosts resolves the dependency by both short service name and registered dotted name.

The project is named after a domain discovered from container system dns list rather than a hardcoded one, since up derives its DNS domain from the project name — under any other name the DNS path is never taken and there is nothing to assert. (The existing ComposeUpDnsTests hardcodes dnstest, so it silently skips anywhere that domain is not registered, which is how I noticed.)

Worth flagging: writing this produced a convincing false positive first. On the DNS path containers are named <service>.<domain>, not <project>-<service>. My first version used the wrong shape, execed into a container that did not exist, and read the empty output as a resolution failure — I nearly reported a DNS defect that was not there. The test now proves the exec target is reachable before interpreting any miss, and pointing it at the wrong name fails with "the assertions below would be vacuous".

Static suite: 259 tests / 25 suites green.

Still open from my earlier comment, and still your call: the DNS domain on this branch is derived from the project name, so an operator who registered a domain that isn't their project's name gets dnsAvailable == false and the barrier never runs at all. The fix is a separate commit downstream; say the word and I'll bring it in or file it on its own.