adopt compose-go jobs and container-spec layering by ndeloof · Pull Request #14093 · docker/compose

ndeloof · 2026-08-19T13:44:47Z

Adopts compose-spec/compose-go#866 (jobs top-level element, container specification layered as ContainerSpec/WorkloadSpec, pre_start hooks carrying the full container specification).

This first step adapts the types and gives jobs their first runtime entry point:

composite literals setting moved fields are wrapped into the embedded ContainerSpec/WorkloadSpec (no behavior change; promoted field access was already source-compatible)
pre_start handling is typed against PreStartHook; exec hooks (post_start/pre_stop) lose the image field they never honored — now enforced by construction
up warns about declared jobs (pointing at docker compose run to trigger them) and fails on active (profile-enabled) scheduled jobs with scheduled jobs are not supported in this version: silently not scheduling would break expectations, while manual jobs just wait for an explicit trigger
docker compose run <job> executes a job exactly like a service: the job — a ContainerSpec+WorkloadSpec, the same layers a service is made of — is materialized as a service for the one-off machinery; its profile is activated, its dependencies start as they would for a service, the exit code flows back. Per the spec, manual execution is always available — scheduled jobs included — unless the job explicitly opts out with triggers.manual: false (tri-state *bool in compose-go), which run rejects. All locked by e2e scenarios (scheduled rejection by up before any resource is created; manual run starting its depends_on services first; manual: false refusal).
pre_start hooks inherit the whole container specification, resolved at load time: compose-go completes each hook with the service's container-spec attributes while loading (idempotently across config round-trips, thanks to the strictly-identical-entry merge dedup), so the model itself carries the spec a hook runs with and the runtime consumes it as-is through the standard create path — every ContainerSpec attribute (resources, capabilities, dns, sysctls, ...) inherits with zero attribute-specific code, following the exact multi-file merge rules (command replaces, environment merges per key, extra_hosts accumulate). The only deliberate exception is volumes, inherited at runtime through volumes_from (the only way to share anonymous and image volumes), with hook-declared mounts taking precedence per target. This closes the two standing asks against init containers: inheriting/declaring extra_hosts (fixes pre_start containers don't inherit/support extra_hosts #13939) and giving the hook read-write access to a volume the service mounts read-only (fixes pre_start/init-Containers should support own/overwritten Volumes #13934) — both locked by e2e scenarios.

compose-spec/compose-go#866 is merged: go.mod now points at the merged compose-go head (v2.15.1-0.20260910154416-11feead015fc), no replace directive involved — this PR is self-contained and mergeable. Jobs are hereby exposed end to end: declared in the model, warned about and guarded by up (scheduled jobs rejected until a scheduler exists), and runnable through docker compose run <job>. A compose-go tagged release can still substitute the pseudo-version before merge if preferred.

🤖 Generated with Claude Code

Update (independent review follow-ups). An adversarial review pass surfaced real gaps, all addressed in the rebuilt stack:

volumes_from (and service:-scoped network_mode/ipc/pid) inherited into a pre_start hook now resolve to live container IDs like the service create path — the daemon used to receive raw service names and fail up for any service combining volumes_from with a hook (e2e-locked). Hook labels merge into the container's labels (the runtime identification set wins), and the <API 1.44 network fallback joins the hook's own networks.
A job's env_file/label_file resolve through run: materialization now happens during loading, before environment resolution — and the retry is keyed on "the target is a declared job", never on matching an error message, so a plain typo keeps the precise selection error and unrelated services' env_file still don't need to exist.
Job→job depends_on executes: the whole job closure materializes, each dependency running to completion through the machinery a service dependency uses (e2e-locked). Materialized jobs carry the standard compose labels — without them their containers were invisible to every label-driven path — and their x-* extensions.
create and start refuse active scheduled jobs like up does, keeping the "before any resource is created" promise on every entry point.
publish safeguards and image pinning cover jobs: sensitive-env/env_file/bind-mount/build-only checks, digest override, application index, and config --images.

Note on config hashes: the container-spec layering reorders ServiceConfig fields, which would have changed every service's config-hash (mass recreation on upgrade) — #14215 pins the hash byte layout to its historical form — fully backward compatible, zero recreation — and must land first; this PR then deletes its transient continuity test in the compose-go bump commit, as that test documents.

glours

A few things worth fixing before this lands, even given the interim/client-side nature of this jobs support:

cmd/compose/up.go:362 — warnIgnoredJobs iterates project.AllJobs() (includes profile-disabled jobs), while rejectScheduledJobs (up.go:379, also called from create.go:104 and start.go:63) correctly uses project.Jobs (profile-enabled
only). Result: up can warn about jobs that aren't even part of this invocation's profile selection. Should use project.Jobs here too.

cmd/compose/run.go:287-292 — the job-detection fallback now retries the full project load on any narrowed-load error, not just "no such service". Any unrelated load failure (bad include:, interpolation error) now triggers a second full
load — duplicating remote include: fetches and unsupported-attribute warnings — before falling through to the (still correct) original error. Worth narrowing the retry condition back to the selection-failure case, since job detection doesn't
need the broader retry.

pkg/compose/pre_start_test.go — still no test exercising the security-sensitive fields (Privileged, CapAdd/CapDrop, SecurityOpt, ReadOnly, Sysctls) that hooks now inherit from the service by default. This is pre-existing/permanent
hook behavior, not something the interim jobs work will replace — worth covering given what's at stake if inheritance breaks silently.

pkg/compose/publish.go — the new job branches (pushApplicationIndex, generateImageDigestsOverride, collectEnvCheckFindings, checkOnlyBuildSection, checkForBindMount, checkForSensitiveData) are correct on inspection but have zero
dedicated test coverage — publish_test.go's only changes are mechanical ContainerSpec adaptations. This is the secret-scanning gate for publish; a future regression here would go undetected by CI.

Nits, non-blocking:

pre_start.go: resolveHookServiceReferences rebuilds the per-service container map inline instead of reusing getContainersByService.
publish.go: findings for a job are still labeled "service %q: ..." in the merged prompt message.

ndeloof · 2026-09-18T11:37:38Z

Addressed in d04a245, 447c867, dff0e0e:

warnIgnoredJobs now uses project.Jobs (profile-enabled only), matching rejectScheduledJobs.
runProject's job-detection fallback now only retries the full/unselected load on the specific "no such service" selection failure (via a new isNoSuchServiceErr helper), not on any load error.
Added TestPreStart_SecuritySensitiveFieldsHonored covering hook inheritance of Privileged/CapAdd/CapDrop/SecurityOpt/ReadOnly/Sysctls.
Added dedicated job-path coverage for pushApplicationIndex, generateImageDigestsOverride, collectEnvCheckFindings, checkOnlyBuildSection, and checkForBindMount (only checkForSensitiveData had a service-level test to mirror before; pushApplicationIndex had none at all — the job case is proven via an invalid job image surfacing its own reference error, since exercising the real push path needs a live registry).
Nits: resolveHookServiceReferences now reuses getContainersByService; job findings in the publish prompt are no longer mislabeled "service %q".

All new/changed tests fail without their respective fix, confirmed locally. build/vet/lint/tests all green.

compose-spec/compose-go#866 is merged: jobs top-level element, container specification layered as ContainerSpec/WorkloadSpec, pre_start hooks carrying the full container specification resolved at load time. Bump to the merged head and adapt in the same movement — composite literals setting moved fields wrap them into the embedded ContainerSpec / WorkloadSpec (promoted field access was already source-compatible, so this is literal-only, no behavior change), and pre_start handling is typed against PreStartHook. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

… start The model may declare jobs this runtime cannot execute yet. Commands that materialize the application warn about them, and refuse an active (profile-enabled) scheduled job before any resource is created — up, create and start alike: silently not scheduling would break expectations, while manual jobs just wait for an explicit trigger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ndeloof · 2026-09-18T12:55:50Z

Rebased onto main (07f3de8) — root cause of the lint/test failure: the relay work merged into main after this branch was cut added tests using the pre-layering ServiceConfig shape (DependsOn/Networks set directly, PreStart as []ServiceHook), which don't compile against the ContainerSpec/WorkloadSpec split this branch adopts. GitHub tests the hypothetical merge of the PR into main, which is why this only showed up in CI and not locally against the branch's own tip. Adapted both files to the new shape; not a real regression in the reviewed commits. Full build/vet/lint/test green locally afterward (pkg/watch's fsevents flakes aside, unrelated and pre-existing).

`docker compose run <job>` materializes the job — a ContainerSpec + WorkloadSpec, the same layers a service is made of — as a service for the one-off machinery: its profile is activated, the project narrowed to its dependencies by WithSelectedJob, and the exit code flows back. The materialization happens during project loading, BEFORE service selection and environment resolution: the job's own env_file and label_file resolve exactly like a selected service's would, unrelated services' env_file still don't need to exist, and a target that is neither a service nor a job keeps the precise selection error (the retry is keyed on "the target is a declared job", never on matching an error message). A service and a job may share a name: the service wins, matching what the selector resolved. A job may depend on other jobs: the whole job closure materializes, so every depends_on edge resolves to a runnable service satisfying its declared condition — a dependency job runs to completion through the exact machinery a service dependency does — instead of dangling as an unresolvable name. Job extensions (x-*) survive materialization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

pre_start hooks carry the full container specification, but the init-container runner only consumed a handful of attributes (image, command, user, env, workdir) — everything else was silently dropped. Instead of wiring attributes one by one, the hook's specification is merged over the service's ContainerSpec through compose-go's own file merge machinery (override.Merge on the canonical yaml tree): command and entrypoint replace, environment merges per key with the hook winning, extra_hosts and dns accumulate entries, ulimits merge — the exact per-attribute rules of multi-file compose, maintained in one place. Every ContainerSpec attribute inherits this way, current and future, with zero attribute-specific code. The merged spec then runs through the standard create path (getCreateConfigs) as a service-shaped one-off, so resources, capabilities, dns, sysctls, logging... materialize exactly as they would for a service container. Hook containers keep their minimal labels and carry no container-number, so tooling telling replicas apart does not count them. The only deliberate exception is volumes: mounts inherit at runtime through volumes_from — the only mechanism that shares the service's anonymous and image volumes — and the hook's own volume declarations, materialized by the standard path, take precedence per target. This is what lets an init container get read-write access to a volume the service mounts read-only (fixes: see PR). e2e scenarios lock extra_hosts inheritance and accumulation, volume override and completion, and the unit tests pin the merge rules. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

…chain Scenario-DSL coverage for the jobs entry points: - up refuses a project declaring an active scheduled job before creating any resource; - run executes a manual job like a service, starting its depends_on services first, and refuses a schedule-only job; - a job's own env_file feeds its environment through run — the materialization happens before environment resolution; - a job depending on another job runs the dependency to completion first, through the exact machinery a service dependency uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Per the spec, any job can be triggered manually by an explicit run command, its automated triggers notwithstanding — so run now accepts scheduled jobs too. The exception is a job explicitly declared with triggers.manual: false, which run rejects: meant for scheduled jobs whose out-of-schedule execution would be harmful. compose-go is bumped to the jobs-branch commit making Manual tri-state (*bool) and allowing manual and schedule to be combined. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

compose-go resolves each hook against its service at load time: the model itself carries the full container specification a hook runs with, and the runtime consumes it as-is through the standard create path — the runtime-side merge helper goes away. Consuming the full spec means honoring ALL of it: - service references it may carry — volumes_from entries, and service:-scoped network_mode/ipc/pid, inherited or declared — resolve to live container IDs exactly like the service create path does; the daemon knows nothing about service names and rejected them, failing up for any service combining volumes_from with a pre_start hook (locked by an e2e scenario). - hook labels — declared or inherited — merge into the container's labels, the runtime identification set winning on conflicts. - the <API 1.44 network-connect fallback joins the HOOK's networks, not the parent service's: a hook overriding networks was connected to the wrong ones. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Every publish safeguard and image-pinning path used to walk only project.Services, leaving jobs invisible: - the sensitive-data checks (literal environment values, env_file scans, bind-mount warnings, build-only rejection) now cover jobs — a job declaring AWS_SECRET_ACCESS_KEY=... was published without the guard rail a service gets; - the image-digest override pins job images too (jobs dressed as services run through the exact WithImagesResolved semantics), so the published artifact is reproducible for jobs as well; - the application index references job images; - `config --images` lists them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

Rebasing onto main pulled in the relay work (added after this branch was cut), whose tests build ServiceConfig/PreStart literals against the flat, pre-layering shape: DependsOn/Networks set directly, and PreStart typed as []ServiceHook. Both are incompatible with the ContainerSpec/WorkloadSpec split this branch adopts -- the promoted DependsOn/Networks fields need go1.27 to set via a flat literal (this module stays on 1.26.3), and PreStart is now []PreStartHook, a full container specification, not a plain hook. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

The fallback reload runProject uses to detect a job target passed warnUnsupportedAttributes, the same option as the first, selective load right before it -- which had already fired every warning once. For the common case where the target is neither a service nor a job (a plain typo), every finding got logged twice before falling through to the original, correct error anyway. skipUnsupportedAttributesWarning already exists for exactly this secondary-load case (see its other call sites in compose.go and completion.go); use it here too. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

…mage-digests Only the interpolated path (imagesOnly) and pkg/compose/publish.go walked project.Jobs the same way they walk project.Services. Three sibling paths in config.go were missed: - runConfigImages (--images) only printed service images. - lockModel (--no-interpolate --lock-image-digests) dropped the top-level "jobs" key entirely along with every other non-services key, instead of filtering it down to image/type:image-volumes like services. - resolveImageDigests (--no-interpolate --resolve-image-digests) only built its pseudo-project from model["services"], so job images and type: image volume sources never resolved -- confirmed by a new test that failed against the first attempt at this fix (job image stayed unpinned) until dressed as its own pseudo-project the same way generateImageDigestsOverride in publish.go already does, since WithImagesResolved only walks a project's Services, never its Jobs. The extra complexity that last one picked up is split into resolveServiceImageDigests (services, unchanged from before this commit) and resolveJobImageDigests (new), called from a small dispatcher. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

docker compose create <job> / start <job> failed with the raw compose-go error "no such service: <name>", since project loading rejects the job name before rejectScheduledJobs or any job-aware logic ever runs -- unlike run, neither command knows how to detect that a name is a declared job. The result gave no hint that jobs exist or that run is the right command. Jobs stay run-only by design; this only replaces the error message when the target is a declared job, via a small helper (jobTargetErr) shared by both commands, reloading unselected exactly like run's own job-detection fallback. Any other "no such service" (a real typo) keeps the original, precise error. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

materializeJobClosure walked job-typed depends_on edges without re-checking triggers.manual: false on anything but the top-level run target, so `docker compose run A` where A depends_on job B (manual: false) silently ran B anyway. manual: false declares a job harmful to trigger outside its schedule; depends_on doesn't change who caused the execution or when -- pulling B in to satisfy A's dependency is still the run command causing that out-of-schedule execution, one hop removed. Refuse it the same way the top-level check already does, before anything in the closure is created. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ndeloof · 2026-09-21T09:58:12Z

Addressed the follow-up review:

f761e08 — fix 1: the fallback reload in run.go now uses skipUnsupportedAttributesWarning instead of warnUnsupportedAttributes, matching the existing convention for secondary loads.
2bc29e3 — fix 2: --images, --lock-image-digests and --resolve-image-digests now scan jobs the same way they scan services. Added TestResolveImageDigestsWithJobs, which caught a real bug in my first attempt: WithImagesResolved only walks a project's Services, so job images silently stayed unpinned until dressed as their own pseudo-project the same way generateImageDigestsOverride in publish.go already does. Split resolveImageDigests into resolveServiceImageDigests (unchanged) + resolveJobImageDigests to keep cognitive complexity under the lint threshold.
c116397 — fix 3: create/start now give job %q can only be triggered with "docker compose run" instead of the raw compose-go selection error, via a small shared helper (jobTargetErr) mirroring run's own job-detection fallback. A genuine typo still gets the original, precise error.
8fc4918 — open question: decided manual: false should also block a job pulled in transitively via depends_on. depends_on doesn't change who caused the execution or when — pulling the job in to satisfy a dependency is still the run command causing that out-of-schedule execution, one hop removed, and manual: false exists specifically to prevent that. Added the check in materializeJobClosure, a unit test, and a locking e2e scenario (TestRunJobDependsOnManualFalseJob).

All four verified locally: build/vet/lint/test green (pkg/watch's pre-existing fsevents flakes aside), plus manual runs of create/start against a job target.

jobTargetErr was called with the full [SERVICE...] argument list, but compose-go's selection error only ever names the one target it actually failed on. The loop matched any name in the list against the job set, so `docker compose create totally-bogus migrate` (a real typo plus a legitimate job) reported the job error instead of the real "no such service: totally-bogus", masking the typo. Match the name the error actually names, not any job name that happens to share the invocation. unselectedJobs is extracted out of jobTargetErr so start's own project-resolution fallback (next commit) can reuse the same unselected-reload logic. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

projectOrName falls back to a label-driven, file-less project on any load failure when COMPOSE_PROJECT_NAME is set -- swallowing the load error entirely, including "no such service" for a target that's actually a declared job. Since that branch returns err == nil, jobTargetErr (gated on err != nil) and rejectScheduledJobs (gated on project != nil) were both skipped: `docker compose start migrate` fell through to a label-driven start that finds no container for a job that was never run, and exited 0 having silently done nothing. Refuse a named job explicitly on that fallback path too, reusing unselectedJobs from the previous commit. Also adds the test coverage this and the previous fix (job-target error messages on create/start) were entirely missing: create.go and start.go had no unit tests at all, and pkg/e2e/jobs_test.go only exercised up/run. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ndeloof · 2026-09-21T12:31:18Z

Addressed a third-party review of the previous commit (c116397) plus an independent fresh-eyes pass I ran separately, both converging on the same two real gaps in jobTargetErr/create/start:

06f51dd17 — jobTargetErr matched any name in the [SERVICE...] argument list against the job set, not the one name compose-go's selection actually failed on. docker compose create totally-bogus migrate (a real typo plus a legitimate job) reported the job error instead of the real no such service: totally-bogus, masking the typo. Fixed by correlating against the specific failing name; extracted unselectedJobs out of jobTargetErr for reuse below.
2e5a6876c — two things from the same review round:
- create.go/start.go had zero test coverage for their job-target-error wiring (no create_test.go/start_test.go, and pkg/e2e/jobs_test.go only exercised up/run). Added TestJobTargetErr (unit) and TestCreateRefusesJob/TestStartRefusesJob (e2e).
- A real bug this gap let through: when COMPOSE_PROJECT_NAME is set, start's underlying projectOrName helper falls back to a label-driven, file-less project on any load failure — including "no such service" for a job — discarding the error entirely. Both jobTargetErr (gated on err != nil) and rejectScheduledJobs (gated on project != nil) were skipped on that path, so docker compose start migrate silently exited 0 having done nothing (no container exists for a job that was never run). Fixed, with TestStartRefusesJobWithProjectNameEnv locking in the repro.

All verified locally: build/vet/lint/test green, including a manual regression check on the correlation fix (reverting it back-to-back against the new unit test confirmed it catches the bug).

The previous fix only covered create/start. projectOrName backs every other service-targeting command too (stop, kill, pause, unpause, logs, rm, down, ps, events), so they still surfaced the same two problems: a raw "no such service" error for a job name, and -- when COMPOSE_PROJECT_NAME is set -- a silent no-op, since projectOrName's label-driven fallback swallows the load error entirely. Fold the check into projectOrName so every caller is fixed at once, instead of duplicating the wiring per command. start.go's own job-handling code, now redundant, is removed; jobTargetErr and unselectedJobs (cmd/compose/run.go) are unchanged, just called from one more place. Adds unit coverage for projectOrName itself (including the COMPOSE_PROJECT_NAME path and the typo-correlation edge case) and e2e coverage for two more callers (stop, down) to confirm the fix lives in the shared helper rather than being re-implemented per command. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

glours

Sounds like a good 1st iteration

… order The plan documented in both PRs had #14215 land first, then #14093's compose-go bump delete the transient TestServiceHashContinuity. Merged in the opposite order, main's pkg/compose tests no longer even compile: richServiceFixture sets fields the container-spec layering turned into promoted embeds (struct literals require go1.27 for that), and once compilable two tests fail by construction. - richServiceFixture is built by assignment; - TestServiceHashContinuity is deleted, exactly as its own comment prescribes: the layered compose-go is the reorder it existed to outlive, its proof duty (pinned == historical bytes) is done, and TestHashGoldenValues carries the continuity contract alone — it PASSES on the layered compose-go, confirming the pinning preserved every historical hash through the layering; - pull_refresh_after, newly exposed at the service root, is appended to serviceHashKeyOrder: the fallback already emitted it in that exact position, so no hash moves — listing it only freezes the layout and satisfies the coverage walker. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

…wed load error, strict service validation projectOrName and toProjectName resolved the project with opposite precedences, and projectOrName silently swallowed any load error when COMPOSE_PROJECT_NAME was set: a broken compose file sent stop, down, ps... into label-based reconstruction without a word — even when the file was named explicitly with --file. One precedence now, documented on both resolvers and applied identically by compose-go while loading: --project-name, then COMPOSE_PROJECT_NAME, then the model's name. The failure policy becomes explicit: an unreadable explicit --file is a hard error; no file around with COMPOSE_PROJECT_NAME set stays the silent file-less workflow; a present-but-broken implicit file falls back to label-based mode with a warning. Service-name validation follows one rule — strict whenever a model is available: restart and wait no longer silently no-op on a typo (validateServiceNames, profile-disabled services remain legitimate targets), and the hand-rolled checks in ps and volumes are removed as dead code, the load-time selection already rejecting unknown names (pinned by test). Epic docker#14074, F.4. Rebased onto main, which since merged the jobs work (docker#14093, docker#14234): projectOrName's job-target detection (jobTargetErr) is restored ahead of the new explicit-file hard-error branch -- the file loaded fine here, only the target's selection failed -- and validateServiceNames now checks project.AllJobs() too, since restart/wait route their service arguments through it instead of projectOrName's own selection. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

…wed load error, strict service validation projectOrName and toProjectName resolved the project with opposite precedences, and projectOrName silently swallowed any load error when COMPOSE_PROJECT_NAME was set: a broken compose file sent stop, down, ps... into label-based reconstruction without a word — even when the file was named explicitly with --file. One precedence now, documented on both resolvers and applied identically by compose-go while loading: --project-name, then COMPOSE_PROJECT_NAME, then the model's name. The failure policy becomes explicit: an unreadable explicit --file is a hard error; no file around with COMPOSE_PROJECT_NAME set stays the silent file-less workflow; a present-but-broken implicit file falls back to label-based mode with a warning. Service-name validation follows one rule — strict whenever a model is available: restart and wait no longer silently no-op on a typo (validateServiceNames, profile-disabled services remain legitimate targets), and the hand-rolled checks in ps and volumes are removed as dead code, the load-time selection already rejecting unknown names (pinned by test). Epic docker#14074, F.4. Rebased onto main, which since merged the jobs work (docker#14093, docker#14234): projectOrName's job-target detection (jobTargetErr) is restored ahead of the new explicit-file hard-error branch -- the file loaded fine here, only the target's selection failed -- and validateServiceNames now checks project.AllJobs() too, since restart/wait route their service arguments through it instead of projectOrName's own selection. docker-agent review: the "compose file found but could not be loaded" warning's suppression guard only matched errdefs.IsNotFoundError (compose-go's own ErrNotFound sentinel) -- a raw os.ErrNotExist (e.g. a nonexistent --project-directory) wasn't recognized and would have printed a misleading warning. Added errors.Is(err, os.ErrNotExist) as a fallback, and TestProjectOrNameResolution now asserts the warning's presence/absence in both directions instead of just the fallback name. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

pre_start hook containers become first-class resources of the reconciliation plan. The reconciler plans one OpCreateHookContainer per declared hook — gated on the same predicate the start phase uses to run the hooks: a replica to start and no running replica SURVIVING the plan (a recreated replica is not running at start time, so its runners are planned too). Runner creation depends on every replica node (the executor resolves the VolumesFrom target against a final live view), on the infrastructure, and on the purge of previously observed runners: names are deterministic (<project>-<service>-pre_start-<i>) so repeated plans converge on the same container instead of accumulating anonymous ones, and a new HookIndexLabel ties each runner to its hook. runPreStart becomes pure execution: it looks up the created-state runner for each hook by label and never creates one. A declared hook without a prepared runner is an actionable error naming the reconciliation command (docker compose up <service>) — the accepted consequence is that stop-then-start of a hooked service errors, since runners are consumed on success (locked by e2e). Success removal, failure retention and cancellation cleanup are unchanged. Runners stay invisible to the start/ps container listings — they carry no ConfigHashLabel, which getDefaultFilters requires — and down keeps removing them by hook label. Rebased onto main, which diverged substantially since this branch was opened (the unrelated container-spec-layering refactor, docker#14093, restructured compose-go's ServiceConfig/PreStartHook types this PR also touches). Reconciling pre_start.go/pre_start_test.go against that refactor surfaced a real bug: createPreStartContainer builds the hook's config via the shared getCreateConfigs, which unconditionally stamps a ConfigHashLabel — silently contradicting this same commit's own "invisible to ps/start" guarantee above, since that label is exactly what getDefaultFilters requires. Fixed by deleting the label after getCreateConfigs returns, with a test asserting its absence (an independent local review, via the GitHub Copilot CLI, and a fresh-context review agent both converged on catching this during rebase verification). Also restored two security/label tests that only existed on one side of the rebase (SecuritySensitiveFieldsHonored, HookLabelsMerged), ported to call createPreStartContainer directly. Also addresses glours' three review comments on this PR: hook-container create operations for a service no longer chain serially (each depends on the same shared containerNodes+infraDeps+purges set, letting the executor create them in parallel), the corresponding plan-string test assertion, and a clearer missing-runner error message hinting this isn't necessarily a planning bug. Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

ndeloof changed the title ~~adopt compose-go jobs and container-spec layering (types only)~~ adopt compose-go jobs and container-spec layering Aug 19, 2026

ndeloof force-pushed the adopt-compose-go-jobs branch 10 times, most recently from f6ea219 to ccaa8ce Compare August 26, 2026 14:01

ndeloof force-pushed the adopt-compose-go-jobs branch from ccaa8ce to 31714e5 Compare August 31, 2026 07:37

stasadev mentioned this pull request Aug 31, 2026

Docker Compose v5.3.0: new pre_start hooks could simplify existing DDEV logic ddev/ddev#8773

Open

glours mentioned this pull request Aug 31, 2026

jobs: add support for one-shot and scheduled jobs #14155

Draft

ndeloof force-pushed the adopt-compose-go-jobs branch from 8114744 to 3811fa7 Compare September 11, 2026 06:59

ndeloof marked this pull request as ready for review September 11, 2026 08:56

ndeloof requested review from a team as code owners September 11, 2026 08:56

ndeloof requested a review from glours September 11, 2026 08:56

ndeloof mentioned this pull request Sep 11, 2026

fix(hash): pin the service config-hash byte layout, decoupled from compose-go struct refactorings #14215

Merged

ndeloof force-pushed the adopt-compose-go-jobs branch from 3811fa7 to 0a8ee1b Compare September 11, 2026 17:21

ndeloof requested a review from docker-agent September 11, 2026 17:21

glours reviewed Sep 14, 2026

View reviewed changes

ndeloof and others added 2 commits September 18, 2026 14:51

ndeloof force-pushed the adopt-compose-go-jobs branch from dff0e0e to 07f3de8 Compare September 18, 2026 12:55

ndeloof and others added 2 commits September 18, 2026 16:20

ndeloof and others added 5 commits September 18, 2026 16:20

ndeloof force-pushed the adopt-compose-go-jobs branch from 95afff2 to d28ad4c Compare September 18, 2026 14:21

ndeloof added 4 commits September 21, 2026 11:54

ndeloof added 2 commits September 21, 2026 14:28

ndeloof mentioned this pull request Sep 21, 2026

stop/kill/pause/logs/rm/down/ps/events silently no-op or give a raw error when targeting a job name #14249

Closed

glours approved these changes Sep 21, 2026

View reviewed changes

ndeloof merged commit 0d206e6 into main Sep 21, 2026
62 checks passed

ndeloof deleted the adopt-compose-go-jobs branch September 21, 2026 14:35

ndeloof mentioned this pull request Sep 22, 2026

fix(hash): repair main after #14093/#14215 merged in reverse order #14253

Merged

ndeloof mentioned this pull request Sep 22, 2026

cli: one project-resolution story — single name precedence, no swallowed load error, strict service validation #14151

Open

Navigation Menu

adopt compose-go jobs and container-spec layering - #14093

adopt compose-go jobs and container-spec layering#14093
ndeloof merged 16 commits into
mainfrom
adopt-compose-go-jobs

ndeloof commented Aug 19, 2026 •

edited

Loading

Uh oh!

glours left a comment

Uh oh!

ndeloof commented Sep 18, 2026

Uh oh!

ndeloof commented Sep 18, 2026

Uh oh!

ndeloof commented Sep 21, 2026

Uh oh!

ndeloof commented Sep 21, 2026

Uh oh!

glours left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ndeloof commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

ndeloof commented Sep 18, 2026

Uh oh!

ndeloof commented Sep 18, 2026

Uh oh!

ndeloof commented Sep 21, 2026

Uh oh!

ndeloof commented Sep 21, 2026

Uh oh!

glours left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ndeloof commented Aug 19, 2026 •

edited

Loading