feat(snapshot): kind-based DataImport targetRef, snapshot CRUD commands, and full-tree aggregator import#391
Merged
kkozoriz merged 7 commits intoJun 29, 2026
Conversation
… archive + Mode B
Mode A (d8 snapshot import):
- EnsureDataImport now emits targetRef{group,kind,name} and echoes the leaf's
storageClassName/size/volumeMode instead of the removed dataArtifactType; the
controller infers the artifact and matches the leaf by GroupKind reverse-lookup.
- leafTargetRef returns the leaf kind directly (no RESTMapper resolution).
- Every import marker is the unified spec.source.import: {} (drop dataImportName,
dataSource.name and sourceRef); isImportModeMarker keys solely off it.
- snapshot.yaml VolumeInfo carries storageClassName/size/volumeMode (from
SnapshotContent.status.dataRef); PlannedNode reads it back for the DataImport spec.
Mode B (d8 data import): align the typed API with the unified CRD (KindPersistentVolumeClaim
constant, doc comments) and validate the PVC template carries metadata.name before create.
Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
Align the d8 DataExport consumer with the storage-volume-data-manager CRD
that now keys targetRef on Kind instead of the resource plural:
- typed TargetRefSpec.Resource -> Kind (Group stays); the controller resolves
the served version via its RESTMapper, so only group needs deriving.
- KindToGroupResource -> KindToGroup (drop the plural); CreateDataExport,
CreateDataExporterIfNeeded and the recreate-on-expiry path now pass kind.
- getExportStatus allowlist checks targetRef.Kind against the Kind constants.
- cmd/create resolves group via KindToGroup and forwards the kind verbatim.
- snapshot path: aggapi.LeafDataExportTarget returns {group, kind} (no
RESTMapping), exporter.EnsureDataExport/OpenExport take kind, pipeline wires
it through.
- tests assert Kind instead of the resource plural.
Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
- dataimport README: document that create drives standalone PVC import
(targetRef.kind: PersistentVolumeClaim) and that the PVC template must carry
metadata.name (rejected client-side otherwise).
- dataexport README: note targets are referenced by kind and list supported
target kinds with their CLI aliases; fix stale snapshot/ alias example to vs/.
- CHEATSHEET_E2E: describe the unified spec.source.import: {} marker and the
per-leaf DataImport with kind-based targetRef plus volume params in spec.
Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
Add `d8 snapshot list` (alias `ls`) to list storage.deckhouse.io/v1alpha1 Snapshot resources in kubectl style: - `-n/--namespace` for a single namespace, `-A/--all-namespaces` for all, otherwise the kubeconfig context namespace (`-A` and `-n` are mutually exclusive). - `-o table|json|yaml` (default table); table columns NAME, READY, SNAPSHOTCONTENT, CHILDREN, AGE (plus NAMESPACE under `-A`); json/yaml emit the raw Snapshot objects. Add `utilk8s.KubeconfigNamespace` to resolve the context namespace, factoring the shared kubeconfig loading-rules logic out of `SetupK8sClientSet`. Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
Extend 'd8 snapshot list' with an optional positional [DIR] argument that lists snapshot nodes from a downloaded archive on disk instead of querying the cluster. - Recursively walk DIR for snapshot.yaml nodes and print a table with columns NAME/KIND/NAMESPACE/CHILDREN/VOLUMES/AGE/PATH; -o json|yaml emit the same rows. - Surface filesystem I/O errors (stat/ReadDir) instead of silently skipping nodes or under-reporting children; only ErrNotExist is benign. - Keep Namespace raw in rows so cluster-scoped nodes omit it in json/yaml; the "-" placeholder is applied only when rendering the table. - Reject -A/--all-namespaces and -n with a local DIR argument. Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
Add 'd8 snapshot create' to create a Snapshot object and 'd8 snapshot delete' to remove Snapshot objects, and wire both into the snapshot command group. Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
Domain aggregators (e.g. an intermediate DemoVirtualMachineSnapshot that references child snapshots) were rejected by 'd8 snapshot import' as "not client-drivable", which blocked importing any archive that contained them. They are in fact reconstructed server-side by the genericbinder from the same inputs the CLI already uploads for structural nodes: the unified spec.source.import marker plus the node's manifests and child refs (no DataImport, since the aggregator carries no own volume data). This is the exact flow the e2e suite client-drives. Stop rejecting aggregators in preflight and in importMarkerCR so a full archive imports as-is, with the aggregator materialised as a non-root node. The only remaining restriction is that an aggregator (or a manifest-only domain node) cannot be selected as a standalone --node root, since it has no parent SnapshotContent to attach to; import the full archive or select an ancestor Snapshot instead. Update the command help and unit tests accordingly, and add coverage for a full aggregator-tree import. Signed-off-by: Aleksandr Zimin <alexandr.zimin@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch builds on
feat/d8-snapshot-new-endpointsand bundles several related improvements to thedataandsnapshotCLI surfaces:targetReffrom a resource to a kind, and add a dual-mode (Mode A / Mode B) DataImport flow with an import marker and volume metadata persisted in the archive.d8 snapshot list(cluster + local-archive modes).d8 snapshot createandd8 snapshot delete.d8 snapshot importto import full snapshot trees, including domain aggregators (e.g.DemoVirtualMachineSnapshot).What's included
data: kind-based targetRef + dual-mode DataImport
targetRefnow identifies the target by kind instead of resource (group/kind/name), so leaves are matched by GroupKind without RESTMapper resolution.spec.source.import: {}) and captured volume metadata are persisted in the archive.snapshot:
listd8 snapshot listlists top-levelSnapshotobjects in a namespace (with child counts) and supports a local-archive mode that walks a downloaded archive tree and prints each node's KIND/PATH.snapshot:
create/deleted8 snapshot createcreates aSnapshotobject (built viaunstructuredto preserve all CRD fields).d8 snapshot deleteremovesSnapshotobjects, with explicit flag/argument validation.snapshot: full-tree import including domain aggregators
d8 snapshot importpreviously rejected any archive containing a domain aggregator (a domain snapshot that references children but carries no own volume data) as "not client-drivable".spec.source.importmarker plus the node's manifests and child refs (no DataImport, since the aggregator has no own data) — the exact flow the e2e suite client-drives.importMarkerCRno longer reject aggregators, so a full archive imports as-is with the aggregator materialised as a non-root node.--noderoot (no parent SnapshotContent to attach to); import the full archive or select an ancestorSnapshotinstead.Behavior changes / compatibility
targetRefshape changes from resource to kind — any consumer reading/writing DataExport/DataImporttargetRefmust use kind.d8 snapshot importnow succeeds on archives that previously errored out due to a domain aggregator; the--node <aggregator>case still errors by design.