Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,15 @@ variable's value is a fact about the machine the script runs on, which is not th
machine that read the compose file.

**Closure**:
The target service and everything reachable from it through `depends_on`
(`graph.startup_order`). It is the unit of scope for almost everything: only the
closure joins the pod, so only the closure is emitted, only its hostnames are
resolvable, and pod-level options (`dns`, `sysctls`, `extra_hosts`) are unioned and
conflict-checked across it and nothing else. A service outside the closure never runs,
which is why `profiles` is inert. Validation is not scoped to it, though: the dependency
graph is checked document-wide (`graph.validate_graph`), because whether Docker rejects a
document cannot depend on which service the caller happened to target
([#87](https://github.com/modern-python/compose2pod/issues/87)).
The target service and everything reachable from it through `depends_on` or `links`
(`graph.startup_order`, over the one graph `graph.depends_on` returns for both keys). It is
the unit of scope for almost everything: only the closure joins the pod, so only the closure
is emitted, only its hostnames are resolvable, and pod-level options (`dns`, `sysctls`,
`extra_hosts`) are unioned and conflict-checked across it and nothing else. A service outside
the closure never runs, which is why `profiles` is inert. Validation is not scoped to it,
though: the dependency graph is checked document-wide (`graph.validate_graph`), because
whether Docker rejects a document cannot depend on which service the caller happened to
target ([#87](https://github.com/modern-python/compose2pod/issues/87)).

**Rule one / rule two**:
The two directions of the Docker-rejection parity rule
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ Within that boundary it covers most of what real compose files use:
- **Services** — `image`/`build`, `command`/`entrypoint`, `environment` and
`env_file` (string and long-form `{path, required, format}`), `volumes`
(short-form and long-form `--mount`, including the `bind` and `tmpfs`
option maps), `tmpfs`, `healthcheck`, `depends_on` (all conditions), network
option maps), `tmpfs`, `healthcheck`, `depends_on` (all conditions), `links`
(read as a dependency plus a hostname alias, as Docker reads it), network
`aliases`, `hostname`/`container_name`.
- **Confinement & metadata** — `user`, `working_dir`, `read_only`, `init`,
`privileged`, `cap_add`/`cap_drop`, `security_opt`, `devices`, `group_add`,
Expand Down
6 changes: 3 additions & 3 deletions compose2pod/emit.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,8 +347,8 @@ def _plan(compose: dict[str, Any], options: EmitOptions) -> PlannedScript:
host_tokens = hosts_file_tokens(services, order, hosts)
completion_gated = {
dep
for svc in services.values()
for dep, condition in depends_on(svc).items()
for svc_name, svc in services.items()
for dep, condition in depends_on(svc_name, svc).items()
if condition == "service_completed_successfully"
}
names: set[str] = set()
Expand Down Expand Up @@ -381,7 +381,7 @@ def _plan(compose: dict[str, Any], options: EmitOptions) -> PlannedScript:
names |= stores.referenced_variables(compose, order, options.project_dir)
waited: set[str] = set()
for name in order:
for dep, condition in depends_on(services[name]).items():
for dep, condition in depends_on(name, services[name]).items():
if condition == "service_healthy" and dep not in waited:
interval = interval_seconds((services[dep].get("healthcheck") or {}).get("interval"))
attempts = max(HEALTHY_WAIT_BUDGET_SECONDS // interval, 1)
Expand Down
77 changes: 73 additions & 4 deletions compose2pod/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ def _depends_on_entry_condition(dep: str, spec: dict[str, Any]) -> str:
return condition


def depends_on(svc: dict[str, Any]) -> dict[str, str]:
"""Normalize dependencies of a service to a name -> condition mapping."""
def _declared_depends_on(svc: dict[str, Any]) -> dict[str, str]:
"""Normalize the `depends_on` key alone to a name -> condition mapping."""
deps = svc.get("depends_on")
if deps is None:
# Explicitly absent, not merely falsy. `or {}` treated `depends_on: ""`
Expand All @@ -101,6 +101,54 @@ def depends_on(svc: dict[str, Any]) -> dict[str, str]:
return {dep: _depends_on_entry_condition(dep, spec) for dep, spec in deps.items()}


def _link_entries(name: str, svc: dict[str, Any]) -> list[tuple[str, str]]:
"""Split each `links` entry into the service it names and the alias it adds.

Measured against `docker compose config` v5.1.2. An entry splits into
`service:alias` on a single colon and not otherwise: `db:a:b` is refused as
`undefined service "db:a:b"`, so the whole entry is the name, and `:db` is
refused as `undefined service ""`, so the empty half is the name too. Both
verdicts fall out of returning the string unsplit or split as measured and
letting `validate_graph` refuse a name no service answers to.

The alias half is empty for the plain form (`links: [db]`, which carries the
dependency and no new name, `db` already resolving) and for the measured
`links: ['db:']`, which docker ACCEPTS with a blank alias -- a name with no
characters cannot be written into a hosts file, so it contributes none.
"""
links = svc.get("links")
if links is None:
return []
if not isinstance(links, list):
msg = f"service {name!r}: 'links' must be a list"
raise UnsupportedComposeError(msg)
entries: list[tuple[str, str]] = []
for entry in links:
# A bool is an int and an int is not a string, so `isinstance(entry, str)`
# is the whole check -- docker refuses `links: [1]` and `links: [true]` alike.
if not isinstance(entry, str):
msg = f"service {name!r}: 'links' entry {entry!r} must be a string"
raise UnsupportedComposeError(msg)
service, _sep, alias = entry.partition(":") if entry.count(":") == 1 else (entry, "", "")
entries.append((service, alias))
return entries


def depends_on(name: str, svc: dict[str, Any]) -> dict[str, str]:
"""Dependencies of a service as a name -> condition mapping, from `depends_on` and `links`.

`docker compose config` v5.1.2 normalises `links: [db]` into a `depends_on`
entry on `db` with `condition: service_started`, so the two keys feed one
graph -- which is why a self-link and a `links` cycle are both refused as
cycles, exactly as docker refuses them. Where both keys name the same
service the declared entry keeps its own condition, which is what docker's
normalised output carries: `links: [db]` beside a `service_healthy`
`depends_on` leaves the condition `service_healthy`.
"""
declared = _declared_depends_on(svc)
return {service: "service_started" for service, _alias in _link_entries(name, svc)} | declared


# Docker validates container_name against this exact pattern (measured:
# `container_name '' does not match pattern '[a-zA-Z0-9][a-zA-Z0-9_.-]+'`).
# It is a *search*, not a fullmatch -- JSON-schema `pattern` semantics, which is
Expand All @@ -125,6 +173,19 @@ def _validated_name(name: str, key: str, svc: dict[str, Any]) -> str | None:
return value


def _link_aliases(name: str, svc: dict[str, Any]) -> list[str]:
"""Names this service's `links` add, each belonging to the service it links to.

Kept out of `_host_names`, which answers a different question: the names one
service is reachable by, declared on that service. A `links` alias is declared
on the service doing the linking and names another one. It needs no address of
its own -- every name compose2pod writes into the pod's hosts file resolves to
127.0.0.1 -- only for the service it names to be running, which is what the
edge `depends_on` takes from the same entry guarantees.
"""
return [alias for _service, alias in _link_entries(name, svc) if alias]


def _host_names(name: str, svc: dict[str, Any]) -> list[str]:
"""Names one service is reachable by: hostname, container_name, and network aliases."""
result: list[str] = [value for key in ("hostname", "container_name") if (value := _validated_name(name, key, svc))]
Expand All @@ -147,10 +208,18 @@ def _host_names(name: str, svc: dict[str, Any]) -> list[str]:


def hostnames(services: dict[str, Any]) -> list[str]:
"""All names other services may use to reach a service: names, hostnames/container names, then aliases."""
"""All names other services may use to reach a service: names, hostnames/container names, then aliases.

`links` aliases are collected here too, and so are scoped to whichever
services are passed in: at emit time that is the closure, so an alias
declared by a service that never runs never reaches the pod's hosts file.
A closure service's own alias always names a closure service, because the
same entry put it there.
"""
names = list(services)
for name, svc in services.items():
names.extend(_host_names(name, svc))
names.extend(_link_aliases(name, svc))
return names


Expand Down Expand Up @@ -181,7 +250,7 @@ def visit(name: str, declared_by: str) -> None:
msg = f"service {declared_by!r}: unknown dependency {name!r}"
raise UnsupportedComposeError(msg)
state[name] = "visiting"
for dep in depends_on(services[name]):
for dep in depends_on(name, services[name]):
visit(dep, name)
state[name] = "done"
order.append(name)
Expand Down
1 change: 1 addition & 0 deletions compose2pod/keys.py
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,7 @@ def _emit_ulimits(value: Any) -> list[Token]: # noqa: ANN401 - Compose values a
# "tmpfs" removed — now a SERVICE_KEYS registry key (_scalar_or_list("--tmpfs")).
"healthcheck",
"depends_on",
"links",
"networks",
"hostname",
"container_name",
Expand Down
18 changes: 3 additions & 15 deletions compose2pod/parsing.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,18 +65,6 @@ def _validate_string_list(name: str, key: str, value: Any) -> None: # noqa: ANN
"'extra_hosts' instead"
),
}
# Refused because compose2pod does not read the key yet, which ADR-0006 calls a tracked
# limitation rather than a design position. Kept apart from the table above so the two
# never share a sentence: this one is expected to shrink.
_UNIMPLEMENTED_REFUSALS = {
"links": (
"docker reads it as a dependency on the linked service plus a hostname alias "
"(measured, v5.1.2), and compose2pod takes neither from this key -- declare the "
"dependency in 'depends_on' and the alias in 'networks.<network>.aliases'"
),
}
# The categories above are the documented distinction; the gate only needs the reason.
_REFUSAL_REASONS = _POD_MODEL_REFUSALS | _UNIMPLEMENTED_REFUSALS
# The only service keys Docker tolerates an explicit null on, where it means
# "not specified" (measured against `docker compose config`). Every other key
# with a bare `key:` is refused -- see `_reject_null_values`.
Expand Down Expand Up @@ -815,8 +803,8 @@ def _validate_service(name: str, svc: Any) -> list[str]: # noqa: ANN401 - Compo
if key in IGNORED_SERVICE_KEYS:
IGNORED_SERVICE_KEYS[key](name, key, svc[key])
warnings.append(f"service {name!r}: ignoring '{key}'")
elif key in _REFUSAL_REASONS:
msg = f"service {name!r}: {key!r} is not supported: {_REFUSAL_REASONS[key]}"
elif key in _POD_MODEL_REFUSALS:
msg = f"service {name!r}: {key!r} is not supported: {_POD_MODEL_REFUSALS[key]}"
raise UnsupportedComposeError(msg)
elif key not in SUPPORTED_SERVICE_KEYS:
msg = f"service {name!r}: unsupported key '{key}'"
Expand Down Expand Up @@ -1030,7 +1018,7 @@ def _validate_depends_on(services: dict[str, Any]) -> None:
"""
validate_graph(services)
for name, svc in services.items():
for dep, condition in depends_on(svc).items():
for dep, condition in depends_on(name, svc).items():
if condition not in DEPENDS_ON_CONDITIONS:
msg = f"service {name!r}: depends_on {dep!r} has unsupported condition {condition!r}"
raise UnsupportedComposeError(msg)
Expand Down
14 changes: 9 additions & 5 deletions docs/adr/0003-the-shared-namespace-decides-key-classification.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,15 @@ supported way to name one. This clause once also swept up `links` and `expose`,
the same sentence and belong in neither category
([#120](https://github.com/modern-python/compose2pod/issues/120), measured against
`docker compose config` v5.1.2). `links` normalises to a `depends_on` edge plus a hostname alias
-- docker refuses `links: [ghost]` exactly as it refuses a ghost `depends_on` -- so it neither
escapes the namespace nor is satisfied by it, and ignoring it would drop a dependency the
`--target` closure is built from. It is refused as a tracked limitation under
[ADR-0006](0006-docker-rejection-parity.md), and both halves are mechanisms compose2pod already
has, so this one is expected to shrink. `expose` carries no edge, is never published, and is
-- docker refuses `links: [ghost]` exactly as it refuses a ghost `depends_on`, and a self-link as
a cycle -- so it neither escapes the namespace nor is satisfied by it, and ignoring it would drop
a dependency the `--target` closure is built from. Both halves were mechanisms compose2pod already
had, which is why the tracked limitation was expected to shrink and did
([#132](https://github.com/modern-python/compose2pod/issues/132)): the edge joins the mapping
`graph.depends_on` returns, so one graph is closed over and validated whichever key declared it,
and the alias joins `graph.hostnames`, landing in the pod's hosts file at `127.0.0.1` like every
other name in it. An alias needs no address of its own for exactly the reason `external_links`
cannot have one. `expose` carries no edge, is never published, and is
validated by docker no further than its list shape (it keeps `expose: [banana]`), which makes it
inert exactly as `ports` is: it sits in `IGNORED_SERVICE_KEYS` with a warning, not refused.
Per-container namespace overrides (`ipc`, `uts`, `domainname`, `cgroup`, `userns_mode`) are
Expand Down
10 changes: 10 additions & 0 deletions tests/conformance/corpus/service_links_empty_alias.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Measured ACCEPT by `docker compose config` v5.1.2: `links: ['db:']` keeps the entry
# verbatim, blank alias and all. The edge stands; a name with no characters cannot be
# written into the pod's hosts file, so it contributes none (issue 132).
services:
db:
image: nginx
app:
image: nginx
links:
- "db:"
9 changes: 9 additions & 0 deletions tests/conformance/corpus/service_links_multi_colon.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Docker rejects this as `undefined service "db:a:b"`: an entry splits into service and
# alias on a single colon and not otherwise, so the whole string is the service name.
services:
db:
image: nginx
app:
image: nginx
links:
- db:a:b
7 changes: 7 additions & 0 deletions tests/conformance/corpus/service_links_self.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Docker rejects this as "dependency cycle detected: app -> app". `links` feeds the same
# dependency graph `depends_on` does, so a self-link is a self-edge.
services:
app:
image: nginx
links:
- app
28 changes: 20 additions & 8 deletions tests/conformance/test_corpus.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,19 +130,31 @@ def test_volumes_long_form_image_type_is_no_longer_an_over_rejection(
assert assert_rule(yaml.safe_load(path.read_text())) == "both-accept"


def test_service_links_is_a_catalogued_over_rejection(
def test_service_links_is_no_longer_an_over_rejection(
assert_rule: Callable[[dict[str, Any]], str],
) -> None:
"""Docker accepts `links: [db:database]`; compose2pod does not read the key yet.
"""Docker accepts `links: [db:database]`, and since issue 132 so does compose2pod.

Asserted rather than left to the generic corpus run because `over-reject` is an allowed
verdict either way. What it pins is the measurement that reclassified this key in issue
120: docker normalises it to a `depends_on` edge plus an alias, so ignoring it with a
warning would drop a dependency the closure is built from. The day compose2pod reads
both halves, this flips to `both-accept` and the assertion says so.
This assertion is the one the issue said would report when the limitation closed: it
asserted `over-reject` while the key was refused, and the verdict it asserts now is the
measurement that replaced it. Left to the generic corpus run it would stay green either
way, `over-reject` being an allowed verdict -- which is exactly why the flip needs saying.
"""
path = Path(__file__).parent / "corpus" / "service_links_alias.yaml"
assert assert_rule(yaml.safe_load(path.read_text())) == "over-reject"
assert assert_rule(yaml.safe_load(path.read_text())) == "both-accept"


def test_service_links_empty_alias_is_accepted_by_both(
assert_rule: Callable[[dict[str, Any]], str],
) -> None:
"""`links: ['db:']` is accepted by docker with a blank alias, so it cannot be refused here.

Asserted on the verdict for the same reason as the row above: an over-rejection of this
shape would pass the generic run silently, and refusing a document docker runs over an
alias that names nothing would be a limitation invented rather than measured.
"""
path = Path(__file__).parent / "corpus" / "service_links_empty_alias.yaml"
assert assert_rule(yaml.safe_load(path.read_text())) == "both-accept"


def test_service_external_links_is_a_catalogued_over_rejection(
Expand Down
6 changes: 2 additions & 4 deletions tests/integration/refusals.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,8 @@
Four claims, four experiments. The keys refused under ADR-0003 have no rows, because
their reason is the pod model rather than rule two: podman honours `network_mode` for a
container that joined a pod (#115), and `external_links` names a container the generated
script never creates, which is a fact about the script. `links` has none either, for a
third reason -- it is a form compose2pod does not read yet (#120). The gate that every
rule-two site has a row is issue #109 phase 3, and those are the exemptions it has to
know about.
script never creates, which is a fact about the script. The gate that every rule-two site
has a row is issue #109 phase 3, and those are the exemptions it has to know about.

A `subpath` row measures the floor, not podman as such: podman gained the option above
the supported minimum (ADR-0006), so the row goes red on a runner newer than the floor,
Expand Down
Loading
Loading