Skip to content

helm: add nodeSelector to upgrade-crd hook Job to prevent Windows scheduling#2640

Open
iacker wants to merge 1 commit into
NVIDIA:mainfrom
iacker:fix/upgrade-crd-nodeselector
Open

helm: add nodeSelector to upgrade-crd hook Job to prevent Windows scheduling#2640
iacker wants to merge 1 commit into
NVIDIA:mainfrom
iacker:fix/upgrade-crd-nodeselector

Conversation

@iacker

@iacker iacker commented Jul 14, 2026

Copy link
Copy Markdown

What this PR does / why we need it:

The pre-upgrade CRD hook Job (deployments/gpu-operator/templates/upgrade_crd.yaml) renders tolerations from .Values.operator.tolerations but no nodeSelector, so on mixed-OS clusters with untainted Windows nodes the hook pod can be scheduled onto a Windows node and hang in ContainerCreating forever, blocking Helm upgrades.

This commit adds a nodeSelector block to the Job's pod template that:

  • renders .Values.operator.nodeSelector if the user has set it (for consistency with the operator Deployment and user-controlled node targeting)
  • defaults to kubernetes.io/os: linux when no user nodeSelector is provided, ensuring the hook always lands on Linux

Which issue(s) this PR fixes:

Fixes #2608

Special notes for reviewers:

  • Tested locally with helm template (no live cluster available).
  • Blast radius: one template file, one pod spec, no user-facing API changes.
  • The default kubernetes.io/os: linux is safe and standard for Linux-only container images.

…eduling

The pre-upgrade CRD hook Job lacked a nodeSelector, so on mixed-OS
clusters with untainted Windows nodes the hook pod could be scheduled
onto Windows and hang in ContainerCreating forever, blocking Helm upgrades.

This commit adds a nodeSelector block that renders .Values.operator.nodeSelector
if set, or defaults to kubernetes.io/os: linux as a safe fallback.

Fixes: NVIDIA#2608
Signed-off-by: Billard <82095453+iacker@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

upgrade-crd hook Job lacks nodeSelector, strands on Windows nodes in mixed-OS clusters

1 participant