setup_docker: make LXC resource limits visible to nested containers

[andrebrait]

✍️ Description

Inside an LXC container, Docker gives every container it starts a fresh procfs in its own mount namespace. The lxcfs bind mounts that LXC placed on /proc/meminfo and friends live in the CT's namespace and never reach it, so every Docker container reads the physical host's totals instead of the CT's limits:

CT sees:        MemTotal:  8388608 kB     # the CT's 8 GiB
container sees: MemTotal: 65648168 kB     # the whole 62 GiB hypervisor
cgroup limit:   max

Enforcement was never affected — the CT cgroup still caps every descendant. Visibility was, and anything that self-sizes from those numbers (JVM heap ergonomics, Node, Go, OpenMP thread pools) sizes against the wrong value and gets OOM-killed instead of throttling.

When enabled, this registers a small runc wrapper as Docker's default-runtime. On each container create it re-binds the lxcfs-backed files into the container and, when the container sets no limit of its own, applies the CT's memory limit to the container cgroup.

Both halves are needed, because runtimes read the limit two different ways:

• glibc's sysconf(_SC_PHYS_PAGES), procps free and friends read /proc/meminfo → fixed by the binds.
• musl and Node's uv_get_constrained_memory() consult the cgroup → fixed by the limit.

CPU needs no handling: cpuset is inherited down the cgroup tree, so nested containers already observe the CT's core count. The one exception is /sys/devices/system/cpu/online, which glibc's sysconf(_SC_NPROCESSORS_ONLN) reads, so it is bound alongside the /proc files. Without it, a container on a 2-core CT reports getconf _NPROCESSORS_ONLN = 6 on a 6-core host while nproc correctly says 2 (coreutils uses sched_getaffinity).

Toggle (opt-in)

Off by default. DOCKER_LXCFS_VISIBILITY=yes|true enables it, and so does the app variable var_docker_lxcfs=yes (for the app_vars entry proposed in review). DOCKER_LXCFS_VISIBILITY takes precedence. lxc-attach inherits the environment, so:

var_docker_lxcfs=yes bash -c "$(curl -fsSL .../ct/docker.sh)"

On the update path nothing changes unless the variable is set explicitly. When it is set, a running Docker is reloaded (SIGHUP; default-runtime and runtimes are reloadable) so no restart is needed.

Safety

• No-op unless enabled, and outside an LXC container or wherever lxcfs is not mounted (guard: grep -qs 'fuse\.lxcfs' /proc/mounts), so the five non-Docker installers that call setup_docker are unaffected.
• Never replaces an existing default-runtime (e.g. nvidia): it warns and skips.
• Fails safe. Setup leaves Docker untouched when runc or jq is missing or daemon.json cannot be parsed. If the wrapper cannot patch a container's config, it hands that config to the real runc unmodified.
• runc resolved at runtime, since distro docker.io ships /usr/sbin/runc while containerd.io ships /usr/bin/runc.
• Never overrides an explicit --memory / mem_limit.
• docker run --runtime=runc bypasses the wrapper for a single container.
• The jq merge preserves any existing daemon.json keys (registry-mirrors, insecure-registries, log-driver).
• Covered by tests/docker-lxcfs.sh (wired into verify-split-files.yml).

Known limitations, documented in docs/lib.md ("Docker inside LXC")

The limit is written at container creation. After changing a CT's memory with pct set, existing containers keep the old value until recreated, or re-synced with docker update --memory=<new> --memory-swap=<new> <container>. New containers pick up the new value immediately, and CPU changes propagate live with no action.

The alternative — binds only, no cgroup limit — tracks resizes live (lxcfs walks up to the nearest ancestor with a limit) but leaves Node and musl reading the host. I judged that the worse default; happy to flip it if maintainers disagree.

🔗 Related Issue

Fixes community-scripts/ProxmoxVE#16756

✅ Prerequisites (X in brackets)

• Self-review completed – Code follows project standards.
• Tested thoroughly – Changes work as expected (Proxmox VE and/or Incus, as applicable).
• No security risks – No hardcoded secrets, unnecessary privilege escalations, or permission issues.
• API.txt regenerated – If a function signature or behavior changed, API.txt was updated to match.

---

🤖 AI Assistance (X in brackets)

• No AI used – Code was written without AI assistance.
• AI was used – The output has been reviewed and corrected, and I take responsibility for it matching this repo's conventions.

Model: Claude Opus 5 (1M context), reasoning effort high, via Claude Code.

---

🛠️ Type of Change (X in brackets)

• 🐞 Bug fix – Resolves an issue without breaking functionality.
• ✨ New feature – Adds new, non-breaking functionality.
• 💥 Breaking change – Alters existing functionality in a way that may require updates.
• 🆕 New script – A fully functional and tested script or script set.
• 🌍 Website update – Changes to script metadata (PocketBase/website data).
• 🔧 Refactoring / Code Cleanup – Improves readability or maintainability without changing functionality.
• 📝 Documentation update – Changes to README, AppName.md, CONTRIBUTING.md, or other docs.

---

Test evidence

Run against a real Proxmox VE 9.2.10 host (PVE kernel 7.0.14-11-pve, lxcfs 7.0.0-pve1) in an unprivileged Debian 13 CT with features: nesting=1, 8192 MB / 6 cores, Docker 29.7.2, on a 62 GiB host. The CT was reset to a clean pre-patch state, then the patched _docker_setup_lxcfs_visibility was extracted from lib/runtime.func and executed:

RED: default=runc
MemTotal:       65648168 kB
max

[info] Configuring LXC resource visibility for Docker
[ok] Configured LXC resource visibility for Docker

GREEN: default=lxcfs
MemTotal:        8388608 kB
8589934592

Additional cases, same CT:

[B --memory=1g respected]  MemTotal:  1048576 kB   memory.max: 1073741824
[C --runtime=runc bypass]  MemTotal: 65648168 kB   memory.max: max
[D cpu/online bound]       1 mount   (8 lxcfs binds total)
[E second run idempotent]  {"dr":"lxcfs","rt":["lxcfs"],"mirrors":1}
[F toggle off]             returned 0, no action taken
[G survives daemon restart] MemTotal: 8388608 kB   memory.max: 8589934592

Note [B]: lxcfs resolves per reading process's cgroup, so a container with its own --memory reports its limit through the same bind — no per-container wiring needed.

CI gates run locally:

shellcheck --shell=bash --severity=error --exclude=SC1091 --format=gcc lib/runtime.func   -> rc=0
shellcheck --severity=warning, added range                                               -> clean
verify-split-files API check: lib/tools.func vs lib/API.txt                               -> matches (151 functions)

Verification one-liner for reviewers

echo "CT sees:        $(head -1 /proc/meminfo)"
echo "container sees: $(docker run --rm alpine head -1 /proc/meminfo)"
echo "cgroup limit:   $(docker run --rm alpine cat /sys/fs/cgroup/memory.max)"

Two commands that look fine while this is broken, worth avoiding: docker info reports the correct MemTotal throughout, because the daemon is a plain CT process reading lxcfs; and docker run --rm alpine free -m prints the host figure even once fixed, because busybox free uses sysinfo(2), which lxcfs cannot intercept.

[github-actions]

Try this branch

The engine and the scripts resolve independently, so a production script can
be run against the engine from this PR by setting one variable:

COMMUNITY_SCRIPTS_CORE_URL=https://raw.githubusercontent.com/andrebrait/core/fix/docker-lxc-resource-visibility \
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/debian.sh)"

Swap ct/debian.sh for whatever exercises the change.

Run a script from a fork as well

curl -fsSL https://raw.githubusercontent.com/andrebrait/core/fix/docker-lxc-resource-visibility/tools/run.sh |
  bash -s -- https://raw.githubusercontent.com/YOU/DevScripts/your-branch ct/debian.sh \
             https://raw.githubusercontent.com/andrebrait/core/fix/docker-lxc-resource-visibility

Note that run.sh is reached through a pipe, so the script it starts inherits
an exhausted stdin. Whiptail is fine — it opens /dev/tty — but a plain read
would see EOF. The single-variable form above does not have that problem.

Useful flags while testing

dev_mode=net logs every engine fetch with status and duration, which is the
quickest way to confirm the branch is really being used. dev_mode=keep stops a
failed build from deleting the container along with the evidence.

[andrebrait]

Adding a caveat found in live testing after opening this, now documented in the function header.

Monitoring containers are the one case that wants the opposite of this patch.

lxcfs resolves usage by the reading process's own cgroup. Once /proc/meminfo is bound in, a container that graphs "system memory used" reports only its own footprint. MemTotal is correct; MemFree/MemAvailable/Cached are not. Measured on a 32 GiB CT with ~290 MB in use:

CT itself:           MemFree: 33254596 kB   Cached: 164344 kB
with these binds:    MemFree: 33553052 kB   Cached:      0 kB

Two things worth noting about the scope of this:

• Dropping the cgroup limit does not avoid it. I checked, expecting usage to walk up to the CT the way MemTotal does. It does not — only the limit walks up; usage stays scoped to the reading container's leaf cgroup either way. So this is inherent to lxcfs, not a consequence of the memory.limit half of the patch.
• CPU is unaffected. /proc/stat is not cgroup-scoped; the cpu line inside a bound container tracks the CT's within a few ticks.

Tools that read the Docker API rather than /proc are fine, because the daemon is a plain CT process reading the CT's own lxcfs values — that covers Portainer's dashboard and Arcane's Docker info panel. Tools using gopsutil's mem.VirtualMemory() (Arcane's live system-stats websocket, netdata, glances) are affected.

Escape hatch is per container, no config change needed elsewhere:

services:
  arcane:
    runtime: runc

I don't think this should change the default. The patch fixes every workload that self-sizes from the wrong number, which is the common case; monitoring containers are a small, identifiable set with a one-line opt-out. But it is a real behaviour change for them and maintainers should weigh it — happy to gate it differently if you'd prefer, e.g. skipping containers that mount /var/run/docker.sock.

[MickLesk]

So, now i had time to check all of this;

At first, good research. But we Needed before merge: opt-in (DOCKER_LXCFS_VISIBILITY default off, accept yes|true), never override an existing default-runtime, don't apply it on the update path unless explicitly requested, and cut the header comment down to a line or two (the monitoring caveat can go into the docs).

app_vars for Docker:

[
  { "name": "var_docker_tcp_socket", "label": "Expose Docker TCP socket (insecure)", "type": "select", "options": ["no", "local", "all"], "default": "no" },
  { "name": "var_docker_lxcfs", "label": "Show CT resource limits inside containers (lxcfs)", "type": "boolean", "default": "no" },
  { "name": "var_docker_compose", "label": "Install Docker Compose (Alpine only)", "type": "boolean", "default": "no" }
]

[MickLesk]

@andrebrait ?

[andrebrait]

@MickLesk oh, sorry, I had not realized that was a request for a change. Let me take a look.

[andrebrait]

Done in cb5b2c1 and f4ace17, rebased on current main:

• Opt-in: DOCKER_LXCFS_VISIBILITY defaults to off and accepts yes/true. var_docker_lxcfs=yes is also honoured, to match your app_vars entry. ct/docker.sh will need export var_docker_lxcfs="${var_docker_lxcfs:-}" per CONTRIBUTING.
• An existing default-runtime is never replaced; it warns and skips.
• Updates leave daemon.json alone unless the variable is set. If it is set, Docker is reloaded (SIGHUP), not restarted.
• The header is cut to two lines. The monitoring caveat and the other notes are in docs/lib.md → "Docker inside LXC".
• Added tests/docker-lxcfs.sh to verify-split-files.

The workflows need your approval to run on this fork PR.