Skip to content

fix(installer): trust the corporate MITM CA in the k3d nodes (#424) - #453

Draft
shujaatTracebloc wants to merge 3 commits into
developfrom
fix/424-in-cluster-ca-trust
Draft

fix(installer): trust the corporate MITM CA in the k3d nodes (#424)#453
shujaatTracebloc wants to merge 3 commits into
developfrom
fix/424-in-cluster-ca-trust

Conversation

@shujaatTracebloc

Copy link
Copy Markdown
Contributor

Closes #424.

Problem

Proxy reachability reaches the k3d nodes, but on a TLS-inspecting (break-and-inspect) network the nodes still don't trust the corporate CA — so every in-node containerd pull (rancher/k3s, ghcr.io, tracebloc images) fails x509. Because helm upgrade runs without --wait the failure is masked (companion #425) and surfaces as a root-cause-free "an image couldn't be pulled." This is the enterprise/hospital archetype, on all three OSes.

Fix

1. Inject the CA at create time (the core fix). When TRACEBLOC_CA_BUNDLE (or CURL_CA_BUNDLE) is set, mount the bundle into every k3d node (-v …:/etc/ssl/certs/tracebloc-mitm-ca.crt@all) and write a registries.yaml (configs.<registry>.tls.ca_file) pointing containerd at it for docker.io / registry-1.docker.io / ghcr.io, hooked into the same --config/create path that already carries proxy env. Parity across cluster.sh (Linux/macOS) and install-k8s.ps1 (Windows). A CA var set but unreadable fails loudly, not silently.

2. Name the env var where the user hits the wall. The TLS-interception preflight hint (both OSes), docs/INSTALL.md, and the PS -Help env-var list.

3. CA-aware diagnosis. _diagnose_not_ready / Get-NotReadyState detect x509 / "certificate signed by unknown authority" pull events → a dedicated image_pull_ca state → "the cluster does not trust your network's TLS-inspection CA" + the exact remedy (never the generic pull error). Mirrored in summary.sh and Print-Summary.

Drift gate. New check-drift.sh parity check (_drift_ca_trust) so neither installer can drop the CA wiring for the other's OS.

Design choices (as flagged)

  • Env var: support both — prefer explicit TRACEBLOC_CA_BUNDLE, fall back to CURL_CA_BUNDLE.
  • Injection: k3d registries.yaml ca_file as primary (targets the actual registry-pull failure) plus mounting the bundle into the nodes so the ca_file path resolves.

Acceptance criteria

  • Behind a MITM proxy with the CA supplied: the nodes trust it for in-node pulls (k3d-native registries.yaml + node mount).
  • Without the CA: preflight and the diagnosis name the CA problem + the env var — never a generic pull error.

Tests (+21)

cluster.bats +8 (resolver 4 cases, registries.yaml gen, create wires mount + --registry-config, no-CA path, unreadable→hard error), summary.bats +3 (x509→image_pull_ca, non-x509 stays image_pull, report names CA+env var), check-drift.bats +2, Pester +8.

Verified locally (CI-identical):

  • Pester full: 183 passed, 0 failed, 8 skipped (Windows-only)
  • all bats: 3 failures are pre-existing macOS-local flakes (all 21 new tests pass); ubuntu CI is the arbiter
  • PSScriptAnalyzer: 0 Error-severity · check-style / drift: clean · manifest regenerated

Note: a real break-and-inspect network can't be exercised in CI; the wiring is the documented k3d registries.yaml + node-mount mechanism and is unit-tested end-to-end.

Proxy REACHABILITY reaches the nodes, but on a TLS-inspecting (break-and-
inspect) network the nodes still don't TRUST the corporate CA, so every in-node
containerd pull (rancher/k3s, ghcr.io, tracebloc images) fails x509 — then
masked (helm runs without --wait) into a root-cause-free "an image couldn't be
pulled." Enterprise/hospital archetype, all three OSes.

- Inject the CA at create time: when TRACEBLOC_CA_BUNDLE (or CURL_CA_BUNDLE) is
  set, mount the bundle into every k3d node and write a registries.yaml pointing
  containerd at it per-registry (docker.io, registry-1.docker.io, ghcr.io), via
  the same --config/create path that already carries proxy env. Parity across
  scripts/lib/cluster.sh (Linux/macOS) and install-k8s.ps1 (Windows). A CA var
  set but unreadable fails loudly instead of silently skipping.
- Name the env var where the user hits the wall: the TLS-interception preflight
  hint (both OSes), docs/INSTALL.md, and the PS -Help env-var list.
- CA-aware diagnosis: detect x509 / "certificate signed by unknown authority"
  pull events and report a dedicated image_pull_ca state — "the cluster does not
  trust your network's TLS-inspection CA" + the exact remedy — instead of the
  generic pull error. Mirrored in summary.sh and Print-Summary.
- New check-drift.sh parity check (_drift_ca_trust) so neither installer can drop
  the CA wiring for the other's OS.

Tests: +8 cluster.bats, +3 summary.bats, +2 check-drift.bats, +8 Pester.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTracebloc shujaatTracebloc self-assigned this Jul 28, 2026
…-ca-trust

# Conflicts:
#	scripts/manifest.sha256
#	scripts/tests/install-k8s.Tests.ps1
…-ca-trust

# Conflicts:
#	scripts/install-k8s.ps1
#	scripts/manifest.sha256
#	scripts/tests/check-drift.bats
#	scripts/tests/check-drift.sh
#	scripts/tests/install-k8s.Tests.ps1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants