diff --git a/README.md b/README.md index e7faa3b2..c9a8b904 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ There are two ways to deploy. A fleet's hosts are managed from a control machine with Ansible, from the fleet's own inventory (see [Ansible](#ansible) below). `bootstrap.sh`, described here, brings up a single machine from a checkout of this repository without a control node or inventory. It runs the same Ansible role against the machine itself, so both paths deploy identically. -Prerequisites on the host: Docker (or Colima) with the Compose plugin, any release but 2.37.1 to 2.38.x (the cluster role refuses those; see Docker Compose versions in `roles/github_runner_cluster/README.md`), `python3`, `jq`, `openssl`, and `gh`. On macOS, the playbook installs `helm`, `kubectl` and Docker through Homebrew itself, and the cluster role starts a Colima host's profile at every login and fails unless the runtime is up (Docker Desktop needs its own start-at-login setting; see Container runtime at login in `roles/github_runner_cluster/README.md`). Elsewhere, install `helm` and `kubectl` from the host's package manager first. +Prerequisites on the host: Docker (or Colima) with the Compose plugin, any release but 2.37.1 to 2.38.x (the cluster role refuses those; see Docker Compose versions in `roles/github_runner_cluster/README.md`), `python3`, `jq`, `openssl`, and `gh`. On macOS, the playbook installs `helm`, `kubectl` and Docker through Homebrew itself, and the cluster role starts a Colima host's profile again after a reboot (the node's profile at boot, with nobody logged in, any other at login) and fails unless the runtime is up (Docker Desktop needs its own start-at-login setting; see Container runtime after a reboot in `roles/github_runner_cluster/README.md`). Elsewhere, install `helm` and `kubectl` from the host's package manager first. 1. **Create a GitHub App** on the organisation with **Self-hosted runners: read & write** permission, and install it on the organisation. `playbooks/github_app_setup.yml` does this through GitHub's manifest flow (see `roles/github_runner_arc/README.md`). For one repository's runners instead, which is how a repository of a personal account uses the fleet, the App needs repository **Administration: read & write** and **Metadata: read**, installed where it reaches that repository, and `RUNNER_REPOSITORY` names it (see Repository scope in `roles/github_runner_arc/README.md`). 2. **Configure the environment.** Copy `.env.bootstrap.example` to `.env.bootstrap` and fill in the following (the file's own comments cover the rest, such as joining an existing cluster or running as an agent): diff --git a/roles/github_runner_cluster/README.md b/roles/github_runner_cluster/README.md index 35102e4e..7102319d 100644 --- a/roles/github_runner_cluster/README.md +++ b/roles/github_runner_cluster/README.md @@ -136,21 +136,26 @@ By default the role drives whichever Docker daemon the host's current Docker con Set `github_runner_cluster_docker_context` in the host's variables to pin the context by name (`colima` in that example) on any host with more than one engine. Every `community.docker` module call in the role then passes it as `cli_context`, and every `docker` command the role runs (Compose, `exec`, `inspect`, `volume`, image pulls, and the commands in `playbooks/recover_in_cluster_mesh.yml` and `playbooks/promote_headscale_standby.yml`) runs with `DOCKER_CONTEXT` set to it, so the modules and the CLI always reach the same daemon. The role checks the pin before it touches the host and fails if the context does not exist there, or if `DOCKER_HOST` is also set, since the two would name the daemon in conflicting ways. It never creates a context and never changes the host's current context; `docker context show` on the host reads the same before and after a run. The Headscale tasks that act on another host (the `headscale_hosted` server and standby, and the promotion playbook) use that host's own value. Run `docker context show` on each host to see what it currently points at before choosing a value. -## Container runtime at login (macOS) +## Container runtime after a reboot (macOS) On a Mac the container runtime is a Linux VM (Colima, or Docker Desktop) that stays stopped after a reboot until something starts it. The node's k3s container has `restart: unless-stopped`, so it comes back by itself once the runtime does, but a host whose runtime never starts never rejoins the cluster. With three servers, two such hosts leave the cluster without quorum and no runner pods anywhere. Only macOS hosts are affected by any of this: the role does nothing about the runtime on a Linux host, where Docker is a system service. Which runtime a Mac runs is taken from the Docker context the role drives (see [Docker contexts](#docker-contexts)): `colima`, or `colima-` for another profile, is Colima, and `desktop-linux` is Docker Desktop. `github_runner_cluster_runtime` says so explicitly for a host whose context does not tell, for example one that uses `DOCKER_HOST`, and `github_runner_cluster_colima_profile` names the profile to start when the context does not. -**Colima.** With `github_runner_cluster_runtime_autostart` on (the default, which acts on Colima hosts only), the role: +**Colima.** With `github_runner_cluster_runtime_autostart` on (the default, which acts on Colima hosts only), the role starts the profile if it is stopped, with `colima start --profile --activate=false`, so the profile does not make its context the host's current one, and installs the launchd job that starts it again after a reboot. There are two kinds, chosen by `github_runner_cluster_runtime_autostart_at` (`login`, `boot` or `auto`, the default). `auto` is `boot` for the profile named by `github_runner_cluster_runtime_boot_profile` (default `node`, the profile the bridged k3s node runs in) and `login` for any other, such as a `default` profile a host keeps for other workloads. A profile has one kind at a time: installing either retires the other's property list, unloading it first, so the profile is not started twice. + +**At login (an agent).** The role: -- starts the profile if it is stopped, with `colima start --profile --activate=false`, so the profile does not make its context the host's current one; - installs a launchd agent, `com.exadev.github-runner.colima.`, as `~/Library/LaunchAgents/