Repository navigation
fix(cluster): load br_netfilter in the Colima VM now and at every boot - #170
Merged
Merged
Conversation
A pod that talks to a Service whose only backend pod is on the same node gets no reply when the Colima VM has not loaded br_netfilter, because bridged hairpin traffic skips the reverse NAT without bridge-nf-call-iptables. Pods on other nodes keep working, so the gap stays latent until a single-backend Service such as CoreDNS is rescheduled onto the node, and then every runner pod on it loses DNS. On a Colima host the cluster role now writes a modules-load.d file and a sysctl.d file in the VM, loads the module when the VM has no bridge sysctl directory, and sets both bridge sysctls when they differ, through colima ssh on the host's profile. It runs whatever the autostart setting says, changes nothing on a second run, and reports each change under --check.
Mearman
marked this pull request as ready for review
November 2, 2026 17:56
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
🎉 This PR is included in version 2.18.1 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On a Colima host the node runs in a Linux VM that does not load
br_netfilter. With host-network k3s (flannelcni0bridge), a pod that talks to a Service whose single backend pod is on the same node gets no reply, since bridged hairpin traffic skips the reverse NAT withoutbridge-nf-call-iptables. Pods on other nodes are fine, so it stays latent until something like CoreDNS is rescheduled onto the node, and then every runner pod on it loses DNS and cannot reach GitHub.The cluster role now has
tasks/colima_kernel.yml, included fromruntime.ymlonce the Colima VM answers, whatevergithub_runner_cluster_runtime_autostartsays. Throughcolima ssh --profile <profile>it writes/etc/modules-load.d/br_netfilter.confand/etc/sysctl.d/90-github-runner-br-netfilter.confin the VM, runsmodprobe br_netfilterwhen/proc/sys/net/bridgeis absent, and setsnet.bridge.bridge-nf-call-iptablesand-ip6tablesto 1 when they differ. A second run changes nothing, and--checkreports each change and makes none.The Linux paths need nothing new. A native node already runs
ExecStartPre=-/sbin/modprobe br_netfilter(k3s-agent.service.j2), andgithub_runner_linux_host_k3s_prerequisitesloads the module and sets the same sysctls with boot files. A Compose node on a real Linux host relies on the Docker daemon, which loadsbr_netfilterand setsbridge-nf-call-iptablesitself when it sets up a bridge network (setup_bridgenetfiltering.goin moby). k3s triesmodprobe br_netfilterinside the node's container too (pkg/agent/syssetup/setup.go), but the Compose file mounts no/lib/modulesthere, so that attempt only warns; that is why a Colima VM, where nothing else loads the module, was the one missing it. A Linux Compose host that setsgithub_runner_linux_host_k3s_prerequisites: truegets it from the linux host role regardless.Tests are in
tests/unit/test_runtime_autostart.py, against a stand-in VM behindcolima ssh. Removing the include, the file write, themodprobeor the sysctl task each fails at least one of them.