diff --git a/docs/specs/004-domains-apply.md b/docs/specs/004-domains-apply.md new file mode 100644 index 0000000..8881566 --- /dev/null +++ b/docs/specs/004-domains-apply.md @@ -0,0 +1,125 @@ +# S-004: Domains apply + +status: approved + +## Problem + +A declared Domain reaches its server only through `domains.yml`. The CLI runs that playbook +only inside `cloudfall migrate` (`sdk/src/cloudfall/migrate.py`, steps `domains-http`, +`dns-verify` and `domains-tls`), and the same plan converges `baseline` and `services` on +every server first. The `converge_domains` MCP tool runs `domains.yml` without the baseline, +but it takes no domain argument and connects to every server in `cloudfall_servers`. So an +operator who may deploy to a server but must not converge its baseline (a server lent from +another project, cloudfall-dev/cloudfall#87) has no safe way to apply one Domain. The vhost +and the certificate get written by hand, outside the receipts, and the next `domains.yml` +run overwrites that vhost with no record of what it replaced. + +## Behaviour + +Two commands in a new `domains` group: + +``` +cloudfall domains apply [--issue-certificate] [--yes] +cloudfall domains observe +``` + +`` is a Domain's `metadata.id`. Both commands validate the project first, as +`deploy` does. Invalid config exits `CONFIG_INVALID`. A valid config with no Domain of that +id exits `PRECONDITION` with context code `lifecycle_domain_missing`, the way `deploy` +refuses an unknown component (`lifecycle_component_missing`). Both refusals happen before +the inventory render and before any SSH connection. + +**domains apply** + +- Runs `domains.yml` limited to the Domain's `proxy.server` and to that one Domain. No other host is contacted, and the proxy server renders no other Domain +- The play runs only the `cloudfall_nginx_site` role, so no baseline, firewall, time, access or service task runs. It is safe on a server whose baseline another project owns +- `origin.server` gets nothing, even when it differs from the proxy server: `domains.yml` has no origin-side task today, and no playbook reads the rendered `cloudfall_origin_domains` variable +- Without `--yes` it changes nothing and exits 0 with a plan, like `deploy`: `status: "plan"`, `action: "domains-apply"`, `domain`, `servers` (the proxy server), `wouldRun`, `instruction` and `effect: "would_update"`. The plan runs every controller-side check the run does, the DNS check included +- With `--yes` it answers `status: "ok"`, `action`, `domain`, `servers`, `receipt` (the receipt path) and `effect: "updated"` +- `--issue-certificate` checks DNS on the controller first, then runs the role with `cloudfall_domains_issue_certificates` true. The role's own sequence is the one `migrate` relies on: the vhost goes live over HTTP to serve the ACME challenge, certbot issues the certificate by webroot, the vhost is rendered again with TLS, and `certbot.timer` is enabled +- The DNS check covers `primaryName` and every alias. Each name must resolve to at least one address of the proxy server, computed as `migrate`'s `dns-verify` does (the server address, the names it resolves to, and `network.ipv4`). Otherwise the command exits `PRECONDITION` with context code `domains_dns_unverified`, naming each failing name and what it resolved to +- A Domain with `edge.mode: proxied` skips the DNS check, as `dns-verify` does, and the answer says `dnsCheck: "skipped-proxied"`. A checked Domain answers `dnsCheck: "verified"`. Without `--issue-certificate` there is no DNS check and no `dnsCheck` value +- `--issue-certificate` on a Domain with `tls.mode: disabled` exits `PRECONDITION` before any engine step +- A failed playbook run, such as a vhost that `nginx -t` rejects (the role rolls it back) or a failed issuance, exits `ENGINE_STEP_FAILED` +- Receipts go to `--deployments` (default `tmp/deployments`), the folder `migrate` and `converge_domains` use + +**Receipt** + +Every `domains.yml` run keeps writing one `DeploymentReceipt` per Domain at +`/.json` (`config/schemas/v1/deployment-receipt.schema.json`). Its +`spec` gains an optional `previousConfigurationSha256`: the sha256 of the file at +`proxy.configurationPath` before this run rendered it, or `null` when no file was there. The +role already reads that file before rendering +(`engine/ansible/roles/cloudfall_nginx_site/tasks/apply_virtual_host.yml`), so it hashes it +there. The role writes the receipt, so runs from `migrate` and `converge_domains` record the +field too. A receipt without the field still loads. + +**domains observe** + +Read-only. It runs a new read-only playbook against the proxy server only, and writes +nothing to the server or to the project. It reports: + +- `server`: the proxy server +- `vhost`: `path` (`proxy.configurationPath`), `present`, and `sha256` when present +- `receipt`: `configurationSha256` and `deployedAt` from `/.json`, and `matches` saying whether the vhost on disk has that hash; `null` when there is no receipt +- `certificate`: `path` (`/etc/letsencrypt/live//fullchain.pem`), `present`, and `notAfter` when present +- `certbotTimer`: `enabled` and `active` + +It exits 0 whatever it finds: a missing vhost or an expired certificate is a reading, not an +error. An unreachable server exits `ENGINE_STEP_FAILED`. The playbook hands its reading back +through a controller-side file, as `backup.yml` does with its receipt. `services observe` +stays the public-network view of the route; `domains observe` is the server-side view. + +**MCP** + +`domains.apply` and `domains.observe` join `PROJECT_COMMANDS` in +`sdk/src/cloudfall/agent_server.py`, so `cloudfall mcp serve` serves them as tools generated +from the commands. The provided `converge_domains` tool (`_engine_tools` in +`sdk/src/cloudfall/app.py`) gains an optional `domain` string. Given, the tool runs on that +Domain's proxy server only and renders only that Domain, with `issue_certificates` working +as today (no DNS check). An unknown id exits `PRECONDITION`. Omitted, the tool behaves as +today. + +Touches `sdk/src/cloudfall/app.py`, `sdk/src/cloudfall/commands.py`, +`sdk/src/cloudfall/agent_server.py`, `engine/ansible/playbooks/domains.yml`, a new read-only +playbook under `engine/ansible/playbooks/`, `engine/ansible/roles/cloudfall_nginx_site`, +`config/schemas/v1/deployment-receipt.schema.json`, `domains apply`, `domains observe` and +the `converge_domains` MCP tool. + +## Acceptance criteria + +- S-004-1: In a project with servers `a` and `b` and a Domain whose `proxy.server` is `a`, `cloudfall domains apply --yes` runs `domains.yml` with only `a` in its play recap, and no task from `baseline.yml`, `services.yml` or the `cloudfall_firewall` role runs +- S-004-2: With Domains `d1` and `d2` both proxied on `a`, `domains apply d1 --yes` renders only `d1`'s `proxy.configurationPath`, and `d2`'s vhost file and receipt are unchanged +- S-004-3: Without `--yes`, `domains apply ` exits 0 with `status: "plan"` and `effect: "would_update"`, runs no `ansible-playbook`, and writes no receipt +- S-004-4: `domains apply nope` and `domains observe nope` in a valid project with no Domain `nope` exit `PRECONDITION` with context code `lifecycle_domain_missing`, and neither the inventory render nor `ansible-playbook` runs +- S-004-5: `domains apply ` in a project whose config fails validation exits `CONFIG_INVALID` before any engine step +- S-004-6: `--issue-certificate` on a `dns-only` Domain with an alias that resolves to no address of the proxy server exits `PRECONDITION` with context code `domains_dns_unverified` naming that alias, with and without `--yes`, and runs no `ansible-playbook` +- S-004-7: `--issue-certificate` on a `proxied` Domain runs no DNS check and answers `dnsCheck: "skipped-proxied"` +- S-004-8: `--issue-certificate` on a Domain with `tls.mode: disabled` exits `PRECONDITION` before any engine step +- S-004-9: `domains apply --issue-certificate --yes` on a fresh `dns-only` Domain whose names resolve to the proxy server passes `cloudfall_domains_issue_certificates: true` to `domains.yml`, ends with `/etc/letsencrypt/live//fullchain.pem` present and `certbot.timer` enabled and active, answers `dnsCheck: "verified"`, and writes `/.json` +- S-004-10: A second `domains apply --issue-certificate --yes` skips the `Issue the declared certificate` task, and its receipt has `configurationSha256` equal to `previousConfigurationSha256` +- S-004-11: A `domains.yml` run on a proxy server with a hand-written file at `proxy.configurationPath` writes a receipt whose `previousConfigurationSha256` is that file's sha256, and a run with no file there writes `null` +- S-004-12: A `DeploymentReceipt` without `previousConfigurationSha256` still loads in `services status` +- S-004-13: A vhost that `nginx -t` rejects makes `domains apply --yes` exit `ENGINE_STEP_FAILED` +- S-004-14: `domains observe ` reports `vhost.present`, `vhost.sha256`, `certificate.notAfter`, `certbotTimer.enabled` and `certbotTimer.active` as read on the proxy server, exits 0 when the vhost file is missing, and changes no file on the server or in the project +- S-004-15: `domains observe ` reports `receipt.matches: false` once the vhost file was edited after the last receipt, `true` while it is unchanged, and `receipt: null` when no receipt exists +- S-004-16: `cloudfall mcp serve` on a project lists a tool for `domains apply` and one for `domains observe` +- S-004-17: The `converge_domains` tool called with `domain: ` runs only on `d`'s proxy server and renders only `d`, called without `domain` runs on every server in `cloudfall_servers` as before, and called with an unknown `domain` exits `PRECONDITION` + +## Out of scope + +- Origin-side tasks: nothing in `domains.yml` acts on `origin.server` today. A spec that adds an origin allowlist or a private upstream can extend `domains apply` to the origin then +- Opening ports 80 and 443: the firewall belongs to the baseline, which this command must not touch on a lent server. If the firewall blocks them, issuance fails with `ENGINE_STEP_FAILED` +- Removing a Domain's vhost or certificate: deletion needs its own plan and refusal rules +- A `--server` flag or several domains per call: one Domain has one proxy server, and a loop over ids covers several +- Storing `domains observe` readings as evidence files: `services observe` already stores the public view, and the server view can join it later if the dashboard needs it +- Compatibility of new receipts with an older `cloudfall`: the receipt schema sets `additionalProperties: false`, so an older `services status` refuses a receipt that carries `previousConfigurationSha256`. Upgrading `cloudfall` first is the path, as for any new receipt field +- `migrate --only domains-http,domains-tls`: a step filter on a resumable plan is easy to get wrong and still reads as a migration + +## Decisions relied on + +- none + +## Issues + +## Verification