Skip to content

fix: when MIG is enabled, start the DRA after MIG service - #9853

Open
runzhen wants to merge 6 commits into
mainfrom
runzhen/dra-mig
Open

runzhen wants to merge 6 commits into
mainfrom
runzhen/dra-mig

Conversation

@runzhen

@runzhen runzhen commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

  • when MIG is enabled, start the DRA after MIG service
  • add the unit test case

Which issue(s) this PR fixes:

Fixes #

Copilot AI balanced review requested due to automatic review settings October 21, 2026 21:21
@runzhen
runzhen requested review from a team as code owners October 21, 2026 21:21
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   18 suites   1m 10s ⏱️
563 tests 563 ✅ 0 💤 0 ❌
585 runs  585 ✅ 0 💤 0 ❌

Results for commit 1c8918c.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The condition checks the wrong layout variable, leaving layout-only MIG nodes without the intended DRA startup ordering.

1 open issue
What changed in this PR

This PR updates Linux GPU provisioning to start the NVIDIA DRA driver after MIG partitioning when MIG is configured.

Changes:

  • Adds conditional systemd startup ordering for the DRA driver.
  • Adds an Ubuntu 24.04 E2E scenario covering MIG Mixed mode, DRA service startup, and workload scheduling.
File Description
parts/​linux/​cloud-init/​artifacts/​cse_config_gpu.sh Adds MIG-dependent ordering to the DRA service override.
e2e/​scenario/​scenario_gpu_managed_experience.go Adds a managed DRA scenario using an A100 GPU with MIG Mixed mode.

Review effort: Balanced

Comment thread parts/linux/cloud-init/artifacts/cse_config_gpu.sh Outdated
Updated comment to reflect correct variable name for MIG profile.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 21:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped ordering change has no identified blocking defects; deterministic ShellSpec coverage remains a non-blocking improvement.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Low severity Add tests for DRA drop-in ordering across GPU profile inputs

parts/​linux/​cloud-init/​artifacts/​cse_config_gpu.sh:451

🟡 Medium Risk — 🧪 Test Coverage: Please extend the existing startNvidiaManagedExpServices ShellSpec cases in spec/parts/linux/cloud-init/artifacts/cse_config_gpu_spec.sh to assert the generated DRA drop-in's ordering. None currently checks this directive, and the new E2E scenario sets only GPUInstanceProfile and checks eventual health, so it cannot reliably catch removal of the ordering or a broken layout-only condition. Capture the DRA drop-in and assert After=mig-partition.service kubelet.service for profile-only and layout-only inputs, and After=kubelet.service when both inputs are empty.

1 resolved since last review

Review effort: Balanced

Comment thread e2e/scenario/scenario_gpu_managed_experience.go
Comment thread e2e/scenario/scenario_gpu_managed_experience.go Outdated
Copilot AI balanced review requested due to automatic review settings October 22, 2026 06:53
@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
test 0% ❌
windows 27% ❌
staging_cse_windows Package 1 65% ➖
debug 0% ❌
provisioningscripts 2% ❌
parts_windows Package 1 73% ➖
Summary 38% (1652 / 6834) ❌

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
github.com/Azure/agentbaker/aks-node-controller 72% ➖
github.com/Azure/agentbaker/aks-node-controller/common 100% ✔
github.com/Azure/agentbaker/aks-node-controller/helpers 68% ➖
github.com/Azure/agentbaker/aks-node-controller/parser 88% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu 100% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils 89% ✔
github.com/Azure/agentbaker/aks-node-controller/utils 0% ❌
github.com/Azure/agentbaker/apiserver 24% ❌
github.com/Azure/agentbaker/cmd 0% ❌
github.com/Azure/agentbaker/cmd/starter 0% ❌
github.com/Azure/agentbaker/fuzz/api 0% ❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata 0% ❌
github.com/Azure/agentbaker/pkg/agent 78% ➖
github.com/Azure/agentbaker/pkg/agent/datamodel 77% ➖
github.com/Azure/agentbaker/pkg/agent/toggles 0% ❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel 87% ✔
Summary 75% (5873 / 7861) ➖

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
shellspec spec 28% ❌
Summary 28% (4557 / 15973) ❌

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The provisioning change still needs the required ShellSpec regression coverage for profile-only, layout-only, and non-MIG service ordering.

1 open issue

Review effort: Balanced


tee "${DRA_DRIVER_NVIDIA_GPU_OVERRIDE_DIR}/10-dra-driver-nvidia-gpu.conf" > /dev/null <<EOF
# If either GPU_INSTANCE_PROFILE or NVIDIA_MIG_PROFILE_LAYOUT is set, start the DRA driver after MIG is ready.
if [ -n "${GPU_INSTANCE_PROFILE}" ] || [ -n "${NVIDIA_MIG_PROFILE_LAYOUT}" ]; then

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants