Skip to content

fix(security): restrict node secret file perms and avoid logging AD realm password - #9776

Open
Calvin S. (calvin197) wants to merge 1 commit into
mainfrom
security/node-runtime-secret-hardening
Open

Calvin S. (calvin197) wants to merge 1 commit into
mainfrom
security/node-runtime-secret-hardening

Conversation

@calvin197

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:
Hardens three node-provisioning scripts so secrets aren't exposed via file permissions or provisioning logs. Addresses Mythos security findings (tracked under ADO 37518878):

• bootstrap-kubeconfig (ADO 37518906/911/915) —  /var/lib/kubelet/bootstrap-kubeconfig  embeds the TLS bootstrap token but was written world-readable ( chmod 0644 ). Tightened to  chmod 0600  so unprivileged local users can't read the token.
• kubelet client key (ADO 37518920/916) —  /etc/kubernetes/certs/client.key  (kubelet client private key) was written with no explicit mode (umask-dependent, typically 0644). Now  touch  +  chmod 0600  before writing, so there's never a world-readable window.
• AD realm password in logs (ADO 37518908) —  setup-custom-search-domains.sh  runs under  set -x , which echoed the AD realm-join password into provisioning logs. Wrapped the  realm join  in  set +x  /  set -x .

No functional behavior change — only file modes and log-tracing scope. Validated with  go test ./pkg/agent/...  and  aks-node-controller ./parser/...  (both pass),  bash -n  on all three scripts, and confirmed no ShellSpec assertions reference the changed lines.

Which issue(s) this PR fixes:

Fixes #

…ealm password

Addresses Mythos findings tracked under ADO 37518878:

- 37518906/911/915: bootstrap-kubeconfig embeds the TLS bootstrap token but
  was written world-readable (chmod 0644). Restrict to 0600 so unprivileged
  local users can't read the token.
- 37518920/916: the kubelet client private key (/etc/kubernetes/certs/client.key)
  was written with no explicit mode (umask-dependent, typically 0644). touch +
  chmod 0600 before writing so it's never world-readable.
- 37518908: setup-custom-search-domains.sh runs under `set -x`, echoing the AD
  realm join password into provisioning logs. Wrap the realm join in
  `set +x` / `set -x`.

Verified: go test ./pkg/agent/... and aks-node-controller ./parser/... pass;
bash -n on all three scripts; no ShellSpec assertions reference the changed lines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 350c2577-7031-4d0d-adc1-e99d053fc981
Copilot AI balanced review requested due to automatic review settings October 14, 2026 04:05
@calvin197
Calvin S. (calvin197) requested review from a team as code owners October 14, 2026 04:05
@calvin197 Calvin S. (calvin197) changed the title fix(security): restrict node secret file perms and avoid logging AD r… fix(security): restrict node secret file perms and avoid logging AD realm password Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   18 suites   1m 24s ⏱️
554 tests 554 ✅ 0 💤 0 ❌
557 runs  557 ✅ 0 💤 0 ❌

Results for commit 5d3b920.

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
test 0% ❌
windows 27% ❌
staging_cse_windows Package 1 65% ➖
debug 0% ❌
provisioningscripts 2% ❌
parts_windows Package 1 74% ➖
Summary 38% (1649 / 6827) ❌

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Restoring tracing masks realm-join failures, and the new security invariants lack required ShellSpec coverage.

3 open findings
What changed in this PR

Hardens Linux node provisioning against secret exposure.

Changes:

  • Restricts kubelet private key and bootstrap kubeconfig permissions.
  • Suppresses tracing while passing the AD realm password.
File Description
setup-custom-search-domains.sh Disables tracing around realm join.
cse_config.sh Sets client key permissions to 0600.
cse_config_kubelet.sh Sets bootstrap kubeconfig permissions to 0600.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

# disable command tracing so the realm password isn't written to provisioning logs
set +x
echo "${CUSTOM_SEARCH_REALM_PASSWORD}" | realm join -U ${CUSTOM_SEARCH_REALM_USER}@$(echo "${CUSTOM_SEARCH_DOMAIN_NAME}" | tr /a-z/ /A-Z/) $(echo "${CUSTOM_SEARCH_DOMAIN_NAME}" | tr /a-z/ /A-Z/)
set -x
Comment on lines +509 to +511
# client.key is the kubelet client private key; restrict to root before writing
touch /etc/kubernetes/certs/client.key
chmod 0600 /etc/kubernetes/certs/client.key
Comment on lines +309 to +310
# bootstrap-kubeconfig embeds the TLS bootstrap token; restrict to root only
chmod 0600 "${BOOTSTRAP_KUBECONFIG_FILE}"
@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
github.com/Azure/agentbaker/aks-node-controller 72% ➖
github.com/Azure/agentbaker/aks-node-controller/common 100% ✔
github.com/Azure/agentbaker/aks-node-controller/helpers 68% ➖
github.com/Azure/agentbaker/aks-node-controller/parser 88% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu 100% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils 88% ✔
github.com/Azure/agentbaker/aks-node-controller/utils 0% ❌
github.com/Azure/agentbaker/apiserver 24% ❌
github.com/Azure/agentbaker/cmd 0% ❌
github.com/Azure/agentbaker/cmd/starter 0% ❌
github.com/Azure/agentbaker/fuzz/api 0% ❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata 0% ❌
github.com/Azure/agentbaker/pkg/agent 76% ➖
github.com/Azure/agentbaker/pkg/agent/datamodel 76% ➖
github.com/Azure/agentbaker/pkg/agent/toggles 0% ❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel 87% ✔
Summary 74% (5610 / 7570) ➖

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
shellspec spec 25% ❌
Summary 25% (3783 / 15360) ❌

touch "${BOOTSTRAP_KUBECONFIG_FILE}"
chmod 0644 "${BOOTSTRAP_KUBECONFIG_FILE}"
# bootstrap-kubeconfig embeds the TLS bootstrap token; restrict to root only
chmod 0600 "${BOOTSTRAP_KUBECONFIG_FILE}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calvin S. (@calvin197) you can drop this change, addressing it here: #9797

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants