Skip to content

feat: bake NPD into Azure Linux 3 Kata VHDs - #9844

Open
chmill-zz wants to merge 1 commit into
mainfrom
kataNPD
Open

chmill-zz wants to merge 1 commit into
mainfrom
kataNPD

Conversation

@chmill-zz

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Bakes Node Problem Detector (NPD) into Azure Linux 3 Kata (Pod Sandboxing) VHDs. Kata now uses the same VHD-owned NPD as regular Azure Linux 3, including live config updates through generic live patching (LPC). This completes the Kata host-component onboarding started with node-exporter (#9810) and Inspektor Gadget (#9836).

Kata was left extension-managed when NPD moved into Ubuntu and Azure Linux 3 VHDs (#9725) only to keep that PR's scope small.

Changes

  • components.json: adds azurelinuxkata entries for node-problem-detector-kubernetes (cloud-native) and node-problem-detector-aks-config (ms-oss), mirroring azurelinux v3.0. Kata builds look up packages under azurelinuxkata, so before this no version resolved.
  • install-dependencies.sh: removes the IS_KATA exclusion.
  • Content test: Kata now has to have baked NPD (packages, tdnf lock, cached recovery RPM, marker, service disabled for capture).
  • docs/npd-onboarding.md: describes Kata behavior.
  • No CSE, handler or E2E logic change. configureNodeProblemDetector, npd-update.sh and ValidateBakedNPD already apply to any Azure Linux 3 node with the marker, and Kata reports ID=azurelinux, VERSION_ID=3.0 with no VARIANT_ID.

⚠️ Kata owners: Kata hosts will receive NPD updates through LPC

With this PR, Kata nodes on new images process NPD live-patching components exactly like Azure Linux 3 nodes:

  • npdConfig (today). The RP publishes one azureLinuxPackageVersions["3.0"] selection per cluster. It is not per pool, so Kata pools receive the same NPD config version as regular Azure Linux 3 pools in that cluster. The Kata host's snapshot-update timer (already enabled on Kata) runs npd-update.sh, which:
    • installs that node-problem-detector-aks-config package with rpm -U;
    • restarts NPD and checks its health;
    • rolls back to the cached previous package if activation fails.
      Selections are forward-only.
  • npdBinary (follow-up, feat: update the NPD binary through LPC as a separate npdBinary component #9833). Once that merges, the NPD binary package is updated the same way.

What does not change on Kata:

  • Only NPD's own held packages move. There's no tdnf update, kernel change, Kata or guest-VM package change, or reboot.
  • Kata OS, kernel and Kata packages still update only through node images:
    • dnf-automatic remains disabled on Kata (cse_main.sh);
    • the RP still excludes Kata from securityPatch, so that component reports no action for Kata pools.
  • NPD runs on the Kata host only. It does not run inside Kata guest VMs.

For comparison, Kata hosts already get NPD changes outside the image cycle today: the VM extension reinstalls NPD when the extension version changes, with no version pin or rollback.

If Kata should take NPD updates only through images, please say so. That would be a small node-side guard in npd-update.sh that reports no action for npdConfig (and npdBinary) on Kata. The RP cannot exclude Kata, because the selection is per OS release, not per pool.

Expected behavior on Kata

Validation

  • PR-gate pipeline on this branch (all VHDs and E2E):
    • The Kata VHD baked binary 0.8.25-24.azl3 and config 1.0.0-2.azl3, and its content test passed.
    • AzureLinuxV3Gen2Kata passed, including baked NPD: enabled and healthy alongside baked node-exporter and IG.
    • The only failure was the unrelated Ubuntu2604Minimal_CustomCa, a kernel drm_atomic_helper_wait_for_vblanks trace seen on other PRs too.
  • Disposable Kata node pool from this branch's VHD (host kernel 6.18.34.mshv3-1.azl3):
    • Startup: configureNodeProblemDetector took 0.6s. NPD was active with all monitors loaded and 10 NPD node conditions published; there were no ML-KEM/TLS errors.
    • Extension coexistence: AKSLinuxExtension 1.519 honored skip_vhd_npd through install, uninstall and reinstall. NPD's PID, file hashes and rpm database were unchanged.
    • LPC, through the real Mariner dispatcher with a shadow ConfigMap and annotations:
      • Selecting the installed version: no action; securityPatch reported no action for the Kata pool.
      • Real PMC download of a newer config despite the tdnf lock: Succeeded.
      • Upgrade to a test package that adds a monitor: Succeeded, monitor loaded; a rerun found the goal already converged.
      • Broken package (startup exits 1): rolled back to the previous package, NPD healthy, reported Failed PackageActivation.
      • Selecting an older version: no action.
      • Interrupted transaction: recovered.
    • Reboot: NPD active after kubelet, 10 conditions with fresh heartbeats, and a Kata pod running on the node.
    • The pool's CSE came from the released AgentBaker service, which predates configureNodeProblemDetector, so that function was run from main for the live test. The PR-gate E2E covers it end to end.
  • Local: CUE validation, shellcheck, and ShellSpec (511 examples across the VHD-builder, npd-update and Mariner reconcile specs).

Which issue(s) this PR fixes:

Related: #9725, #9810, #9836, #9833

Kata was left extension-managed when NPD moved into Azure Linux 3 VHDs.
Resolve both NPD packages through azurelinuxkata and require baked NPD in
content tests. Kata uses the same packages, tdnf lock, recovery cache,
CSE activation and npdConfig handler as Azure Linux 3; its OS, kernel and
Kata packages still update only through images.
Copilot AI balanced review requested due to automatic review settings October 21, 2026 07:48
@chmill-zz
chmill-zz requested review from a team as code owners October 21, 2026 07:49
@github-actions github-actions Bot added the components This pull request updates cached components on Linux or Windows VHDs label Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   18 suites   1m 9s ⏱️
563 tests 563 ✅ 0 💤 0 ❌
585 runs  585 ✅ 0 💤 0 ❌

Results for commit d52083f.

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
parts_windows Package 1 73% ➖
staging_cse_windows Package 1 65% ➖
debug 0% ❌
provisioningscripts 2% ❌
test 0% ❌
windows 27% ❌
Summary 38% (1652 / 6834) ❌

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
github.com/Azure/agentbaker/aks-node-controller 73% ➖
github.com/Azure/agentbaker/aks-node-controller/common 100% ✔
github.com/Azure/agentbaker/aks-node-controller/helpers 68% ➖
github.com/Azure/agentbaker/aks-node-controller/parser 88% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu 100% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils 89% ✔
github.com/Azure/agentbaker/aks-node-controller/utils 0% ❌
github.com/Azure/agentbaker/apiserver 24% ❌
github.com/Azure/agentbaker/cmd 0% ❌
github.com/Azure/agentbaker/cmd/starter 0% ❌
github.com/Azure/agentbaker/fuzz/api 0% ❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata 0% ❌
github.com/Azure/agentbaker/pkg/agent 78% ➖
github.com/Azure/agentbaker/pkg/agent/datamodel 77% ➖
github.com/Azure/agentbaker/pkg/agent/toggles 0% ❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel 87% ✔
Summary 75% (5735 / 7646) ➖

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Enabling shared NPD live-patching on Kata warrants final Kata-owner confirmation despite the reported validation and absence of identified defects.

No open issues 🎉

What changed in this PR

Extends VHD-owned Node Problem Detector (NPD) to Azure Linux 3 Kata, completing the host-component onboarding alongside node-exporter and Inspektor Gadget.

Changes:

  • Adds Kata package pins and enables NPD installation during VHD builds.
  • Includes Kata in existing baked-NPD content checks.
  • Documents host-only operation and shared live-patching behavior.
File Description
vhdbuilder/​packer/​test/​linux-vhd-content-test.sh Requires baked NPD on Kata images.
vhdbuilder/​packer/​install-dependencies.sh Removes the Kata installation exclusion.
parts/​common/​components.json Adds Kata NPD binary and config package pins.
docs/​npd-onboarding.md Documents Kata ownership, updates, and host limitations.

Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
shellspec spec 28% ❌
Summary 28% (4518 / 15934) ❌

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

components This pull request updates cached components on Linux or Windows VHDs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants