cgroup: explain a rejected cpuset fence write instead of surfacing bare ENOSPC - #192
Merged
Conversation
…re ENOSPC Changing cgroup_cpus while VM scopes are live makes the kernel reject the cpuset write with ENOSPC. That reaches the operator as "vm clone: prepare cgroup scope: no space left on device", which points at disk rather than at the fence, and checkScopePlacements does not catch it: it only inspects scopes carrying an explicit placement, while inheriting scopes are the common case. The clear-fence path had no check at all. Wrap the write error with the current fence, the requested one and the number of running VMs so the message says what to do.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changing
cgroup_cpuswhile VM scopes are live makes the kernel reject thecpuset.cpuswrite withENOSPC. What the operator sees is:That points at the disk, not at the fence. Hit while benchmarking a 120-VM Windows fleet; it cost about half an hour to trace, and the first hypothesis was a cocoon bug rather than an operator error.
checkScopePlacementsdoes not catch this: it only inspects scopes that carry an explicitcpuset.cpus, and skips inheriting ones — which are the common case, since most fleets never set a per-VM placement. The clear-fence branch (fence == "") had no check at all, and that is the path that failed.This wraps the write error with the current fence, the requested one, and the number of running VMs:
Deliberately an annotation rather than a pre-flight refusal: narrowing the fence did succeed on a live fleet in the same session, so refusing every change while scopes exist would regress a path that works today. The wrap only fires when the kernel has already said no.
make lintclean on bothGOOS,asl ./...clean on both,go test ./cgroup/...green.