Skip to content

Add containerd patch to handle dmverity referrers - #99

Merged
Dallas Delaney (dallasd1) merged 4 commits into
aclmainfrom
dadelan/containerd2-erofs-patch
Oct 21, 2026
Merged

Dallas Delaney (dallasd1) merged 4 commits into
aclmainfrom
dadelan/containerd2-erofs-patch

Conversation

@dallasd1

@dallasd1 Dallas Delaney (dallasd1) commented Oct 10, 2026 •

Copy link
Copy Markdown

Summary

Patch containerd to be able to discover dmverity OCI referrers if new config option enable_dmverity_referrers is present. This patch represents changes to support this in the CRI, transfer service, unpacker, differ, snapshotter, and mount paths.

Tests have intentionally been stripped from this patch to reduce its size.

Additions

  • Discover and fetch signed dm-verity referrers if they are present. The transfer service discovers referrers with artifactType application/vnd.containerd.erofs.dmverity.v1 for an image manifest and stores its per-layer EROFS metadata, Merkle tree, and detached PKCS#7 root-hash signature in the content store.
  • Update the container resolver to identify parent snapshots that have a signed ChainID key
  • Keep signed and unsigned snapshots independent of each other. Unsigned snapshots are unchanged. They retain pre-existing OCI ChainID keys. Signed snapshots use a separate key appended with "-dmverity". Images sharing layers can therefore use either key without replacing snapshots underneath existing containers.
  • Reuse cached signed snapshots. Signed cache hits from snapshot key IDs and concurrent snapshot commits are checked against the root-hash label identity before reuse.
  • Fetch without unpacking to allow AgentBaker to precache images without going through the transfer service. When the snapshotter advertises dm-verity-referrer support, transfer fetch-only requests retain the dmverity bundle and its payloads without creating snapshots.
  • Create containers offline from a complete cache. If referrer data was fetched without unpacking, CRI can reconstruct signed materializations from retained image content and referrer artifacts without contacting the registry.
  • Preserve ordinary paths. OverlayFS unpack remains unchanged.
  • With EROFS in auto mode, images without a dmverity referrer continue through ordinary unsigned materialization.
  • During FS mount, the kernel verifies the root-hash signature against its trusted keyrings. During container runtime, dm-verity checks reads against the Merkle tree.
  • Setting config option enable_dmverity_referrers indicates the snapshotter is capable of referrer fetching. The snapshotter will materialize the published artifacts alongside the original layer tar data.

Limitations and Prerequisites

  • This is an opt-in feature. Signed referrer execution requires enable_dmverity_referrers  enabled in the config, a compatible dm-verity mode, and the required kernel support.
  • There must only be one dmverity referrer per image manifest. Multiple distinct referrers are rejected. There is no newest-referrer selection. This would require replacing ignoring cached snapshotter metadata and always replacing signatures, even for layers that haven't changed.
  • A pre-existing race condition exists where two container process try to mount the same block device at the same time. The first one wins and the second fails with EBUSY. This is intentionally not resolved in this patch as it is outside the scope of this feature.
  • Signed Kubernetes image volumes are unsupported. CRI rejects them on dm-verity-capable snapshotters instead of silently unpacking them unsigned. This restriction does not apply to ordinary OverlayFS volumes.
  • Legacy client paths are not automatically integrated. The supported integration is in the transfer service plus CRI. Direct client Fetch() and ordinary client unpack/snapshot calls do not automatically exercise behavior.

PR that will invoke this behavior when IPE is enabled: #98

Change Log

  • Add containerd patch based on three containerd functionality PRs

Type of Change

  • Image build change (base image, sysexts, OEM images)
  • Package/SPEC update
  • CI/automation change
  • SDK/toolchain update
  • Configuration change
  • Documentation update
  • Bug fix

Does this affect the image build?

  • Yes
  • No

Associated Issues

Test Methodology

Test methods:

  • acl-pipelines
  • MAP pre-release tests
  • MAP conformance tests
  • AKS e2e tests
  • local validation of non-IPE OverlayFS (default) and IPE EROFS expected behavior

Merge Checklist

All applicable boxes should be checked before merging

  • Image builds successfully with this change (or image build is not affected)
  • Any updated packages/SPECs build successfully
  • Relevant kola tests pass
  • All package sources are available
  • Source files have up-to-date hashes/manifests
  • Documentation has been updated to match any changes
  • Ready to merge

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The security-critical referrer discovery and validation pipeline lacks focused automated tests.

1 open finding
What changed in this PR

Adds opt-in signed dm-verity referrer support to containerd’s EROFS snapshotter and CRI integration.

Changes:

  • Discovers, caches, validates, and materializes signed dm-verity artifacts.
  • Separates signed and unsigned snapshot identities and supports offline reuse.
  • Enables the patch in the containerd2 RPM.
File Description
acl/​SPECS/​containerd2/​erofs-signed-dmverity.patch Implements signed referrer handling across transfer, CRI, unpack, EROFS, and dm-verity paths.
acl/​SPECS/​containerd2/​containerd2.spec Applies the patch and increments the package release.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
@dallasd1
Dallas Delaney (dallasd1) marked this pull request as ready for review October 10, 2026 01:00
@dallasd1
Dallas Delaney (dallasd1) requested a review from a team as a code owner October 10, 2026 01:00
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
@dallasd1

Copy link
Copy Markdown
Author

Adding a comment from the config toggle PR as a limitation of this change:

"AgentBaker skips existing images by name and otherwise imports the pause image using the default overlayfs snapshotter. If it is only prepared for overlayfs, switching to EROFS triggers another pull.

On network-isolated nodes whose bootstrap registry requires authentication, that runtime pull cannot authenticate. Pods then cannot start, even though the pause image is already cached.

Azure/AgentBaker#9761 updates VHD precaching but does not change this bootstrap logic."

@dallasd1
Dallas Delaney (dallasd1) force-pushed the dadelan/containerd2-erofs-patch branch from 249201b to 5805876 Compare October 19, 2026 17:04
Copilot AI balanced review requested due to automatic review settings October 19, 2026 17:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Test-compilation failures, image-selection and cache regressions, and unbounded signature reads remain unresolved.

6 open issues

Review effort: Balanced

Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch Outdated
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch Outdated
Comment thread acl/SPECS/containerd2/multi-snapshotters-support.patch
Copilot AI balanced review requested due to automatic review settings October 19, 2026 21:15

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch Outdated
Comment thread acl/SPECS/containerd2/erofs-signed-dmverity.patch
…/signed snapshot chain

- preserve signed observations when metadata-only traversal filters non-target platform layers
Copilot AI balanced review requested due to automatic review settings October 20, 2026 21:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-critical integration across referrer discovery, snapshot caching, and kernel verification requires final human validation.

No open issues 🎉

5 resolved since last review

Review effort: Balanced

@aadhar-agarwal

Copy link
Copy Markdown

suggestion (non-blocking): Please retain the focused signed-referrer and snapshot-cache regression tests, possibly in a separate test-only patch. Removing them means the RPM's make test does not exercise the new signed-lifecycle cases when this patch is updated later.

@dallasd1
Dallas Delaney (dallasd1) merged commit f630e3e into aclmain Oct 21, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants