Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions roles/github_runner_linux_host/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,22 @@ Its tasks that change the host use `become` on their own, so the connecting user

## Wi-Fi power save

`github_runner_linux_host_wifi_powersave_off: true` tells NetworkManager not to power-save Wi-Fi. A radio in power-save mode sleeps between beacons and delays or drops packets, which on a node of the cluster shows as the node flapping `NotReady`. It matters only for a host on Wi-Fi, and does nothing to a host without a Wi-Fi device; on a host without NetworkManager it fails, since the setting is explicitly asked for.
`github_runner_linux_host_wifi_powersave_off: true` keeps the host's Wi-Fi from power saving. A radio in power-save mode sleeps between beacons and delays or drops packets, which on a node of the cluster shows as the node flapping `NotReady`. It matters only for a host on Wi-Fi, and does nothing to a host without a Wi-Fi device. The mechanism depends on what manages the Wi-Fi, which the role reads from the host: NetworkManager when `nmcli` lists a Wi-Fi device that is not `unmanaged`, else systemd-networkd when it is running and the kernel has a Wi-Fi network device (one with a `phy80211` link under `/sys/class/net`). A host where neither manages a Wi-Fi device fails, since the setting is explicitly asked for; a host with no Wi-Fi device at all is left alone as long as one of the two answers.

It writes `/etc/NetworkManager/conf.d/zz-github-runner-wifi-powersave.conf` with `wifi.powersave = 2` (disable), which sets the default for every connection that does not set its own, and has NetworkManager read it (`nmcli general reload conf`). NetworkManager applies a connection's power save setting each time it activates the connection, so that is what keeps it off across reconnects and reboots. The drop-in's name sorts after Ubuntu's `default-wifi-powersave-on.conf` (`wifi.powersave = 3`), because NetworkManager reads its configuration files in alphabetical order and the last one to set a value wins. To switch it off on a radio that is running now, the role runs `iw dev <device> set power_save off` for each Wi-Fi device that reports it on, which changes the radio without taking the link down; `nmcli device reapply` does not change a running radio. A host without `iw` is told so in the run, and gets the setting at the next activation of its connection.
### NetworkManager

The role writes `/etc/NetworkManager/conf.d/zz-github-runner-wifi-powersave.conf` with `wifi.powersave = 2` (disable), which sets the default for every connection that does not set its own, and has NetworkManager read it (`nmcli general reload conf`). NetworkManager applies a connection's power save setting each time it activates the connection, so that is what keeps it off across reconnects and reboots. The drop-in's name sorts after Ubuntu's `default-wifi-powersave-on.conf` (`wifi.powersave = 3`), because NetworkManager reads its configuration files in alphabetical order and the last one to set a value wins. To switch it off on a radio that is running now, the role runs `iw dev <device> set power_save off` for each Wi-Fi device that reports it on, which changes the radio without taking the link down; `nmcli device reapply` does not change a running radio. A host without `iw` is told so in the run, and gets the setting at the next activation of its connection.

### systemd-networkd

Nothing configures Wi-Fi power save here: neither `systemd.network` nor `systemd.link` (checked in the v259 manual pages, where the only wireless settings are matching ones such as `WLANInterfaceType=` and `Type=wlan`) nor netplan has such a setting, and the kernel's per-interface preference is set with `iw`. So the role installs `iw` with apt (a host that is not Debian or Ubuntu fails) and writes two files:

- `/etc/udev/rules.d/90-github-runner-wifi-powersave.rules`, which gives each Wi-Fi network device (`DEVTYPE=wlan`, set by cfg80211) a `SYSTEMD_WANTS` for the unit below when it is added, named by its final name (`$name`, after a `.link` file's rename), which is what systemd names the interface's device unit by. The role runs `udevadm control --reload` when the rule changes.
- `/etc/systemd/system/github-runner-wifi-powersave@.service`, a oneshot unit with `RemainAfterExit` that runs `iw dev <interface> set power_save off`. It is bound to and ordered after `sys-subsystem-net-devices-<interface>.device` (the same pattern as `wpa_supplicant@.service`), so it runs once udev has finished with the interface, has a status and a journal, and runs again if the interface goes away and comes back. The role runs `systemctl daemon-reload` when the unit changes.

The rule matches every Wi-Fi interface, so one that appears later (a USB adapter) is covered without running the role again. The setting survives a reconnect, because the kernel keeps it per interface; brcmfmac, for one, stores the preference and applies it each time it brings the firmware up. The unit has not run for an interface that is already up, so the role also runs `iw dev <interface> set power_save off` on each one that reports it on. Unlike the NetworkManager path, an `iw` that cannot read an interface fails the run, since there is no later activation to rely on. Not every driver supports the setting; one that does not makes the unit fail, and shows in `systemctl status github-runner-wifi-powersave@<interface>`.

Under `--check` both paths report each change and make none. A second run changes nothing.

A Wi-Fi-only node still drops when the radio does. These settings only stop the radio adding drops of its own; see [A Wi-Fi-only host](../github_runner_cluster/README.md#a-wi-fi-only-host).

Expand Down Expand Up @@ -60,7 +73,7 @@ A host that is not Debian or Ubuntu, a host without the package, and a package n
## Variables

- `github_runner_linux_host_keep_awake` (default `false`): mask the sleep targets and have logind ignore the lid switch.
- `github_runner_linux_host_wifi_powersave_off` (default `false`): turn Wi-Fi power save off in NetworkManager and on the radio.
- `github_runner_linux_host_wifi_powersave_off` (default `false`): turn Wi-Fi power save off, in NetworkManager or, on a host without it that runs systemd-networkd, with a udev rule and a unit that run `iw`, and on the radio.
- `github_runner_linux_host_k3s_prerequisites` (default `false`): load the modules and set the sysctls k3s needs, now and at boot.
- `github_runner_linux_host_battery_charge_limit` (default `false`), `github_runner_linux_host_battery_charge_limit_percent` (default 80) and `github_runner_linux_host_battery_device` (default `BAT0`): limit the battery's charge, to what, and which battery.
- `github_runner_linux_host_headless` (default `false`): boot to `multi-user.target` and stop and disable the display manager.
Expand All @@ -70,9 +83,9 @@ A host that is not Debian or Ubuntu, a host without the package, and a package n
- `github_runner_linux_host_upgrade` (default `false`): upgrade the packages, reboot if due, and autoremove, for one run.
- `github_runner_linux_host_upgrade_reboot` (default `true`), `github_runner_linux_host_upgrade_reboot_running_node` (default `false`), `github_runner_linux_host_node_services` and `github_runner_linux_host_reboot_timeout` (default 600): whether the upgrade reboots, whether it may reboot a running node, which units make a host a running node, and how long it waits for the host to come back.
- `github_runner_linux_host_sleep_targets`, `github_runner_linux_host_lid_switch_settings`, `github_runner_linux_host_kernel_modules`, `github_runner_linux_host_sysctl_settings` and `github_runner_linux_host_default_target`: the targets masked, the logind settings set to `ignore`, the modules loaded, the sysctls set and the target booted to. Constants, not settings; variables only so the tests can narrow them.
- `github_runner_linux_host_logind_dropin`, `github_runner_linux_host_networkmanager_dropin`, `github_runner_linux_host_modules_load_file`, `github_runner_linux_host_sysctl_file`, `github_runner_linux_host_sys_module_dir`, `github_runner_linux_host_power_supply_dir`, `github_runner_linux_host_battery_unit`, `github_runner_linux_host_unattended_upgrades_file`, `github_runner_linux_host_reboot_required_file`: where the role's files go and what it reads and runs. Not settings; variables only so the tests can point them at a stand-in.
- `github_runner_linux_host_logind_dropin`, `github_runner_linux_host_networkmanager_dropin`, `github_runner_linux_host_sysfs_net_dir`, `github_runner_linux_host_wifi_udev_rule`, `github_runner_linux_host_wifi_powersave_unit`, `github_runner_linux_host_modules_load_file`, `github_runner_linux_host_sysctl_file`, `github_runner_linux_host_sys_module_dir`, `github_runner_linux_host_power_supply_dir`, `github_runner_linux_host_battery_unit`, `github_runner_linux_host_unattended_upgrades_file`, `github_runner_linux_host_reboot_required_file`: where the role's files go and what it reads and runs. Not settings; variables only so the tests can point them at a stand-in.
- `github_runner_linux_host_tailscale_update` (default `true`): keep the installed Tailscale package current and its service running, as under Keeping Tailscale current above.

## Tests

`tests/unit/test_linux_host.py` runs the role (`tests/linux_host/run.yml`) against stand-ins for `systemctl`, `nmcli`, `iw`, `modprobe`, `sysctl`, `apt-get`, `dpkg-query`, `uname` and `shutdown` on `PATH`, with every file the role writes or reads redirected and the host's facts stood in for. It checks for each step but the Tailscale update that it is off by default (and that the update is on, and leaves the host alone without the package), does what it says, changes nothing on a second run, and under `--check` reports each change and makes none; and in particular the order of the modules and the sysctls, a battery without the attribute, a limit outside 1 to 100, a display manager worked out from the alias, from its own unit when the alias is not loaded and from a unit the host names, a host with none, services the host does not have, the apt steps refused on a host without apt, the order of the upgrade, a running node's reboot refused with the packages already upgraded, and the defaults, argument specs and this README agreeing.
`tests/unit/test_linux_host.py` runs the role (`tests/linux_host/run.yml`) against stand-ins for `systemctl`, `nmcli`, `iw`, `udevadm`, `modprobe`, `sysctl`, `apt-get`, `dpkg-query`, `uname` and `shutdown` on `PATH`, with every file the role writes or reads redirected and the host's facts stood in for. It checks for each step but the Tailscale update that it is off by default (and that the update is on, and leaves the host alone without the package), does what it says, changes nothing on a second run, and under `--check` reports each change and makes none; and in particular the order of the modules and the sysctls, a battery without the attribute, a limit outside 1 to 100, a display manager worked out from the alias, from its own unit when the alias is not loaded and from a unit the host names, a host with none, services the host does not have, the apt steps refused on a host without apt, the order of the upgrade, a running node's reboot refused with the packages already upgraded, and the defaults, argument specs and this README agreeing.
6 changes: 5 additions & 1 deletion roles/github_runner_linux_host/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,14 @@ github_runner_linux_host_lid_switch_settings:
# Not settings: where the logind drop-in goes, which systemd-logind reads on top of /etc/systemd/logind.conf. Variables only so the tests can point it at a stand-in.
github_runner_linux_host_logind_dropin: /etc/systemd/logind.conf.d/90-github-runner-keep-awake.conf

# Whether NetworkManager is told not to power-save Wi-Fi (wifi.powersave = 2, disable). A Wi-Fi radio in power-save mode sleeps between beacons and drops or delays packets, which on a node of the cluster shows as the node flapping NotReady; it matters only for a host that is on Wi-Fi, and does nothing to a host without a Wi-Fi device. It sets the default for every connection that does not set its own, and applies to the radio now.
# Whether the host's Wi-Fi is kept from power saving. A Wi-Fi radio in power-save mode sleeps between beacons and drops or delays packets, which on a node of the cluster shows as the node flapping NotReady; it matters only for a host that is on Wi-Fi, and does nothing to a host without a Wi-Fi device. On a host whose Wi-Fi NetworkManager manages, NetworkManager is told not to (wifi.powersave = 2, disable) for every connection that does not set its own; on a host without NetworkManager that runs systemd-networkd, a udev rule and a systemd unit run `iw dev <interface> set power_save off` for each Wi-Fi interface when it appears, and iw is installed. Both apply to the radio now. A host where neither stack manages a Wi-Fi device fails.
github_runner_linux_host_wifi_powersave_off: false
# Not settings: where the NetworkManager drop-in goes. The name sorts after Ubuntu's default-wifi-powersave-on.conf (wifi.powersave = 3), because NetworkManager reads its configuration files in alphabetical order and the last one to set a value wins. Variables only so the tests can point them at a stand-in.
github_runner_linux_host_networkmanager_dropin: /etc/NetworkManager/conf.d/zz-github-runner-wifi-powersave.conf
# Not settings: where the sysfs network class is read for the host's Wi-Fi devices, and where the udev rule and the systemd unit template of the systemd-networkd path go. The unit's name must end in @.service, since the rule instantiates it for each interface. Variables only so the tests can point them at a stand-in.
github_runner_linux_host_sysfs_net_dir: /sys/class/net
github_runner_linux_host_wifi_udev_rule: /etc/udev/rules.d/90-github-runner-wifi-powersave.rules
github_runner_linux_host_wifi_powersave_unit: /etc/systemd/system/github-runner-wifi-powersave@.service

# What k3s needs of the kernel (https://docs.k3s.io/installation/requirements and the sysctls kube-proxy and the CNI plugins rely on). true loads the kernel modules now and at boot (a modules-load.d file), and sets the sysctls now and at boot (a sysctl.d file). k3s loads the modules and sets some of the sysctls itself when it starts, but the bridge ones only exist once br_netfilter is loaded, and a host that sets them before k3s starts does not depend on that.
github_runner_linux_host_k3s_prerequisites: false
Expand Down
14 changes: 13 additions & 1 deletion roles/github_runner_linux_host/meta/argument_specs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,23 @@ argument_specs:
github_runner_linux_host_wifi_powersave_off:
type: bool
default: false
description: Whether NetworkManager is told not to power-save Wi-Fi (wifi.powersave = 2), as the default for every connection and on the radio now.
description: Whether the host's Wi-Fi is kept from power saving, through NetworkManager (wifi.powersave = 2, as the default for every connection) when it manages a Wi-Fi device, else through a udev rule and a systemd unit that run iw for each Wi-Fi interface when systemd-networkd is running, and on the radio now. Fails when neither manages a Wi-Fi device.
github_runner_linux_host_networkmanager_dropin:
type: path
default: /etc/NetworkManager/conf.d/zz-github-runner-wifi-powersave.conf
description: Where the NetworkManager drop-in goes. Not a setting, and a variable only so the tests can point it at a stand-in.
github_runner_linux_host_sysfs_net_dir:
type: path
default: /sys/class/net
description: Where the sysfs network class is read for the host's Wi-Fi devices. Not a setting, and a variable only so the tests can point it at a stand-in.
github_runner_linux_host_wifi_udev_rule:
type: path
default: /etc/udev/rules.d/90-github-runner-wifi-powersave.rules
description: Where the udev rule goes that starts the power save unit for each Wi-Fi interface under systemd-networkd. Not a setting, and a variable only so the tests can point it at a stand-in.
github_runner_linux_host_wifi_powersave_unit:
type: path
default: /etc/systemd/system/github-runner-wifi-powersave@.service
description: Where the power save unit template goes, whose name must end in @.service. Not a setting, and a variable only so the tests can point it at a stand-in.
github_runner_linux_host_k3s_prerequisites:
type: bool
default: false
Expand Down
Loading
Loading