Merge upstream worker, snapshot, and preview gate updates

.agents/skills/detect-flaky-tests/SKILL.md

@@ -51,7 +51,7 @@ For each run, you need logs from **two jobs**:
 | Job name | Coverage |
 |---|---|
 | `run-tests` | Unit + integration tests (`go test -race -v ./...`) |
-| `e2e-test` | E2E suite, both sandbox classes — two sequential steps in the one job |
+| `e2e-test`, `e2e-test (envoy, preview)` | E2E suite, both sandbox classes — two sequential steps in each job |
 
 ```bash
 # List all jobs for a run
@@ -62,10 +62,17 @@ gh api "repos/agent-substrate/substrate/actions/runs/<RUN_ID>/jobs" \
 gh api "repos/agent-substrate/substrate/actions/jobs/<JOB_ID>/logs" > /tmp/job_<JOB_ID>.log
 ```
 
-The two e2e lanes are NOT a matrix — `e2e-test` is a single job that runs the step
+`e2e-test` is a matrix over preview mode: `e2e-test` runs with no preview features,
+skipping their tests, and `e2e-test (envoy, preview)` runs with every one on. Other
+dataplanes, where they run, are named `e2e-test (<dataplane>)` and `e2e-test
+(<dataplane>, preview)`. Match all of them by the `e2e-test` prefix, and keep the
+preview mode with each result: a test that fails only without preview features is a
+gating bug, not a flake.
+
+The two sandbox-class lanes are NOT a matrix — each `e2e-test` job runs the step
 "Run E2E tests (gVisor)" followed by "Run E2E tests (micro-VM)" (same
 `hack/run-e2e-kind.sh` command, the second with `E2E_SANDBOX_CLASS: microvm`). Split the
-one `e2e-test` job log at the "Run E2E tests (micro-VM)" step boundary: PASS/FAIL lines
+each `e2e-test` job log at the "Run E2E tests (micro-VM)" step boundary: PASS/FAIL lines
 before it belong to the gVisor lane, lines after it to the microVM lane. Because the
 steps are sequential, a gVisor-lane failure means the micro-VM step never ran — record
 no microVM results for that run rather than counting them as failures.
@@ -77,7 +84,8 @@ grep -E '^--- (PASS|FAIL): ' /tmp/job_<JOB_ID>.log \
 ```
 
 Track results per (test_name, job_type) where job_type is one of:
-`unit`, `e2e-gvisor`, `e2e-microvm`.
+`unit`, `e2e-gvisor`, `e2e-microvm`, `e2e-gvisor-preview`, `e2e-microvm-preview`. The
+`-preview` job types come from the preview jobs; the others from the jobs without.
 
 ---
 
@@ -167,7 +175,7 @@ gh issue create \
 ## Flaky test detected
 
 **Test:** `<TEST_NAME>`
-**Job:** `<e2e-gvisor | e2e-microvm | unit>`
+**Job:** `<e2e-gvisor | e2e-microvm | e2e-gvisor-preview | e2e-microvm-preview | unit>`
 
 ### Evidence (last 7 days)
 
@@ -219,7 +227,7 @@ gh issue create \
 ## Recurring infrastructure failure in CI
 
 **Pattern:** `<infra error pattern>`
-**Job type:** `<unit | e2e-gvisor | e2e-microvm>`
+**Job type:** `<unit | e2e-gvisor | e2e-microvm | e2e-gvisor-preview | e2e-microvm-preview>`
 
 ### Evidence
 

.agents/skills/post-draft-review/SKILL.md

@@ -142,7 +142,7 @@ mutation($review: ID!, $path: String!, $line: Int!, $body: String!) {
     path: $path, line: $line, side: RIGHT, body: $body
   }) { thread { id } }
 }' -f review="PRR_kwDO..." \
-   -f path="cmd/ateom-microvm/restore.go" -F line=214 \
+   -f path="cmd/ateworker-microvm/restore.go" -F line=214 \
    -F body=@c1.md
 ```
 

.agents/skills/triage-issues/SKILL.md

@@ -52,8 +52,8 @@ Apply all areas that are touched. Most issues need one or two; a few need more.
 | `area/api-machinery` | API server internals: RPC handlers, storage layer, syncer, controllers (`cmd/ateapi/`, `cmd/atecontroller/`) |
 | `area/network` | Networking: atenet-router, Envoy, ext_proc, DNS, xDS, ingress (`cmd/atenet/`) |
 | `area/node` | Node agent and worker lifecycle: atelet, sandbox launch, OCI, cgroups (`cmd/atelet/`) |
-| `area/gvisor` | gVisor sandbox specifics: runsc integration, GPU in gVisor, gVisor OCI (`cmd/ateom-gvisor/`) |
-| `area/microVM` | Micro-VM sandbox specifics: cloud-hypervisor, kata, snapshot/restore (`cmd/ateom-microvm/`) |
+| `area/gvisor` | gVisor sandbox specifics: runsc integration, GPU in gVisor, gVisor OCI (`cmd/ateworker-gvisor/`) |
+| `area/microVM` | Micro-VM sandbox specifics: cloud-hypervisor, kata, snapshot/restore (`cmd/ateworker-microvm/`) |
 | `area/storage` | Snapshot storage, image cache, GCS/S3 backends, retention (`internal/imagecache/`, ActorSnapshot) |
 | `area/scheduling` | WorkerPool sizing, actor placement, HPA, resource allocation |
 | `area/observability` | Metrics, tracing, logging, OTLP, OTel SDK integration |

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -98,7 +98,7 @@ body:
     id: logs
     attributes:
       label: Relevant Logs and Diagnostic Output
-      description: Please provide relevant logs from atelet, ateom, ate apiserver, or kubectl describe.
+      description: Please provide relevant logs from atelet, ateworker, ate apiserver, or kubectl describe.
       render: shell
       placeholder: |
         # Paste relevant logs or command output here

.github/SECURITY.md

@@ -52,7 +52,7 @@ See the [threat model](../docs/threat-model.md) for the trust boundaries and
 assumptions this section is based on.
 
 In scope: the Agent Substrate control plane (`ateapi`), node supervisor
-(`atelet`, `ateom`), networking stack (`atenet`), and CLI (`kubectl-ate`).
+(`atelet`, `ateworker`), networking stack (`atenet`), and CLI (`kubectl-ate`).
 
 Agent Substrate multiplexes many actors onto a smaller pool of shared
 workers, so its most important security boundaries are between actors and
@@ -70,7 +70,7 @@ valuable, listed roughly from most to least severe:
   `ateapi` RPCs without valid credentials (mTLS or JWT auth modes),
   misusing workerpool client certificates, or acting beyond what valid
   credentials are authorized to do.
-- **Actor-to-platform escalation**: an actor compromising `atelet`, `ateom`,
+- **Actor-to-platform escalation**: an actor compromising `atelet`, `ateworker`,
   or the control plane, whether while the actor is still actively running
   or beyond its own lifecycle.
 - **Node lateral movement**: using a compromised worker node to reach other

.github/workflows/helm-e2e.yaml

@@ -26,6 +26,8 @@ jobs:
       VERSION: helm-e2e
       E2E_ATENET_DATAPLANE: agentgateway
       E2E_CREDENTIAL_PROVIDER: "1"
+      # Retain external-volume coverage after the preview gate is introduced.
+      ATE_PREVIEW: ALL
       ARTIFACTS: ${{ github.workspace }}/_artifacts
     steps:
     - name: Checkout
@@ -97,6 +99,7 @@ jobs:
           --set postgres.clientCertificates.enabled=true \
           --set image.registry=localhost:5001 \
           --set image.tag=helm-e2e \
+          --set 'preview[0]=ALL' \
           --set 'atelet.extraArgs[0]=--localhost-registry-replacement=kind-registry:5000' \
           --set otel.endpoint=http://opentelemetry-collector.otel-system.svc:4317 \
           --wait --timeout=10m
@@ -124,20 +127,20 @@ jobs:
       # tests keep a stable trust bundle and gateway signing certificate.
       env:
         E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-gvisor.xml
-      run: hack/run-e2e-kind.sh -v -skip '^(TestActorIdentity_|TestActorEgressCredentialInjection$)' -args --no-color
+      run: hack/run-e2e-kind.sh -v -skip '^(TestActorIdentity_|TestActorEgressCredentialInjection$)' -args --no-color --preview="${ATE_PREVIEW}"
     - name: Run E2E tests (micro-VM)
       env:
         E2E_SANDBOX_CLASS: microvm
         E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-microvm.xml
-      run: hack/run-e2e-kind.sh ./internal/e2e/suites/demo -v -args --no-color
+      run: hack/run-e2e-kind.sh ./internal/e2e/suites/demo -v -args --no-color --preview="${ATE_PREVIEW}"
     - name: Run E2E tests (egress credential injection)
       env:
         E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-egress-credentials.xml
-      run: hack/run-e2e-kind.sh ./internal/e2e/suites/egresscredinject -v -args --no-color
+      run: hack/run-e2e-kind.sh ./internal/e2e/suites/egresscredinject -v -args --no-color --preview="${ATE_PREVIEW}"
     - name: Run E2E tests (identity and trust rotation)
       env:
         E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-identity.xml
-      run: hack/run-e2e-kind.sh ./internal/e2e/suites/identity -v -args --no-color
+      run: hack/run-e2e-kind.sh ./internal/e2e/suites/identity -v -args --no-color --preview="${ATE_PREVIEW}"
     - name: Require every lane to have run tests
       if: always()
       run: |

.github/workflows/pr-workflow.yaml

@@ -90,8 +90,8 @@ jobs:
           ~/.cargo/registry/index
           ~/.cargo/registry/cache
           ~/.cargo/git/db
-          cmd/dataplane/envoy/dynamic-modules/*/target
-        key: cargo-${{ runner.os }}-${{ hashFiles('cmd/dataplane/envoy/dynamic-modules/*/Cargo.lock') }}
+          cmd/dataplane/envoy/dynamic-modules/target
+        key: cargo-${{ runner.os }}-${{ hashFiles('cmd/dataplane/envoy/dynamic-modules/Cargo.lock') }}
         restore-keys: cargo-${{ runner.os }}-
     - name: dynamic module tests
       run: |
@@ -122,32 +122,42 @@ jobs:
   # One kind cluster exercises BOTH runtimes. Free x86-64 ubuntu-latest runners
   # expose /dev/kvm (with a udev rule), so create-kind-cluster.sh mounts it and the
   # micro-VM (kata + cloud-hypervisor) sandbox class works alongside gVisor.
+  # Each preview mode runs on a fresh cluster with agentgateway.
+  # The default install retains the required e2e-test check name.
   e2e-test:
-    name: e2e-test
+    name: ${{ matrix.preview == 'off' && 'e2e-test' || 'e2e-test (preview)' }}
     runs-on: ubuntu-latest
     # Cluster bring-up, image builds and two sequential lanes; generous, but far
     # short of the six-hour platform default a hung lane would otherwise consume.
     timeout-minutes: 120
     strategy:
       fail-fast: false
       matrix:
-        include:
-        - dataplane: agentgateway
+        dataplane: ['agentgateway']
+        preview: ['off', 'on']
     env:
       E2E_ATENET_DATAPLANE: ${{ matrix.dataplane }}
       # The installer requires a credential provider selection, and deploys the
       # bundled Kubernetes Secrets provider with the control plane. Job-wide
       # because every ate-setup invocation reads it, including the control
       # plane redeploy in run-microvm-demo-kind.sh.
       ATE_CREDENTIAL_PROVIDER: '{"name":"k8s.io"}'
+      # Job-wide for the same reason. Empty is an explicit "no preview
+      # features", not an unset value. The e2e steps pass it to the suites as
+      # --preview, so tests of a disabled gate skip themselves (see
+      # internal/e2e/preview.go).
+      ATE_PREVIEW: ${{ matrix.preview == 'on' && 'ALL' || '' }}
+      # Appended to the lane and artifact names: e2e-gvisor runs without
+      # preview features, e2e-gvisor-preview with them.
+      LANE_SUFFIX: ${{ matrix.preview == 'on' && '-preview' || '' }}
       # E2E_JUNIT_FILE is set per step, not here: the lanes below share this
       # job, so one job-level path would have each overwrite the last.
       ARTIFACTS: ${{ github.workspace }}/_artifacts
       # Cache the envoy-dataplane Rust build. Only main writes it, so PRs
       # share one copy instead of each storing their own.
       DOCKER_BUILD_FLAGS: >-
         --cache-from type=gha,scope=envoy-dataplane
-        ${{ github.ref == 'refs/heads/main' && matrix.dataplane == 'envoy'
+        ${{ github.ref == 'refs/heads/main' && matrix.preview == 'off'
           && '--cache-to type=gha,mode=max,scope=envoy-dataplane,ignore-error=true' || '' }}
     steps:
     - name: Checkout
@@ -209,8 +219,10 @@ jobs:
       run: hack/create-kind-cluster.sh
     - name: Install Agent Substrate (${{ matrix.dataplane }})
       # The dataplane selection applies to both the ingress router and egress
-      # gateway.
-      run: hack/install-ate-kind.sh --deploy-ate-system --atenet-dataplane=${{ matrix.dataplane }} --rollout-timeout=300s
+      # gateway. ATE_PREVIEW above selects the preview features.
+      env:
+        ATE_ATENET_DATAPLANE: ${{ matrix.dataplane }}
+      run: hack/install-ate-kind.sh --deploy-ate-system --rollout-timeout=300s
     - name: Enable NFS
       # Load NFS kernel modules so in-cluster NFS server and CSI driver can run.
       run: |
@@ -240,8 +252,8 @@ jobs:
     - name: Run E2E tests (gVisor)
       env:
         E2E_PARALLELISM: 8
-        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/e2e-gvisor.xml
-      run: hack/run-e2e-kind.sh -v -args --no-color
+        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/e2e-gvisor${{ env.LANE_SUFFIX }}.xml
+      run: hack/run-e2e-kind.sh -v -args --no-color --preview="${ATE_PREVIEW}"
     - name: Focus the node on micro-VM
       run: hack/e2e-focus-sandbox-class.sh microvm
     - name: Run E2E tests (micro-VM)
@@ -250,8 +262,8 @@ jobs:
       env:
         E2E_SANDBOX_CLASS: microvm
         E2E_PARALLELISM: 8
-        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/e2e-microvm.xml
-      run: hack/run-e2e-kind.sh -v -args --no-color
+        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/e2e-microvm${{ env.LANE_SUFFIX }}.xml
+      run: hack/run-e2e-kind.sh -v -args --no-color --preview="${ATE_PREVIEW}"
     # Every lane must have reported tests; a `-run` filter that stops matching
     # otherwise leaves its lane green and empty. The set comes from the manifest
     # each lane registers itself in, so adding a lane needs no change here, and
@@ -262,7 +274,7 @@ jobs:
     - name: Save Go build cache
       # Main only, so PRs restore the latest main build rather than each
       # storing its own copy.
-      if: ${{ !cancelled() && github.ref == 'refs/heads/main' }}
+      if: ${{ !cancelled() && github.ref == 'refs/heads/main' && matrix.preview == 'off' }}
       uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
       with:
         path: |
@@ -273,7 +285,7 @@ jobs:
       if: always()
       uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
       with:
-        name: test-results-e2e-${{ matrix.dataplane }}
+        name: test-results-e2e-${{ matrix.dataplane }}${{ env.LANE_SUFFIX }}
         path: ${{ env.ARTIFACTS }}
         if-no-files-found: error
     - name: Dump diagnostics on failure
@@ -289,7 +301,7 @@ jobs:
         done
         # Every worker pod in any namespace: the demo pools plus the e2e suites'
         # randomly-named per-test namespaces, which the suites keep on failure.
-        # The failing actor runs in one of these, so this is where its ateom logs
+        # The failing actor runs in one of these, so this is where its worker logs
         # (and, for a micro-VM worker, the guest console tail) live.
         kubectl --context kind-kind get pods -A -l ate.dev/worker-pool \
           -o 'custom-columns=:.metadata.namespace,:.metadata.name' --no-headers 2>/dev/null \

.github/workflows/release.yaml

@@ -88,7 +88,7 @@ jobs:
         run: |
           set -o errexit -o nounset -o pipefail
 
-          for component in ateapi atecontroller atelet ateom-gvisor ateom-microvm podcertcontroller atenet objectstore-plugin credential-provider/kubernetes-secrets; do
+          for component in ateapi atecontroller atelet ateworker-gvisor ateworker-microvm podcertcontroller atenet objectstore-plugin credential-provider/kubernetes-secrets; do
             KO_DOCKER_REPO="${IMAGE_REPOSITORY}/${component##*/}" \
             ./hack/run-tool.sh ko build \
               --tags "${IMAGE_TAGS}" \

.golangci.yaml

@@ -27,10 +27,25 @@ linters:
   default: standard
 
   enable:
+    # Import restrictions, see depguard settings below.
+    - depguard
     # Spelling errors in comments, strings, and identifiers.
     - misspell
 
   settings:
+    depguard:
+      rules:
+        # internal/ateworker is the ateworker binaries' shared code. Anything
+        # else should import a flat package under internal/ instead.
+        ateworker:
+          files:
+            - "$all"
+            - "!**/cmd/ateworker-*/**"
+            - "!**/internal/ateworker/**"
+          deny:
+            - pkg: github.com/agent-substrate/substrate/internal/ateworker
+              desc: only the ateworker binaries may import internal/ateworker
+
     errcheck:
       # Functions where ignoring the return is idiomatic in this codebase.
       # Note: errcheck's exclude-functions matches on the static receiver

AGENTS.md

@@ -31,7 +31,7 @@ tools/        # Standalone Go tools (go run ./tools/<name>) for Dev/CI
 | Shared across binaries, not for external import | `internal/<pkg>` |
 | Public API for external consumers | `pkg/<pkg>` |
 | Public proto (control-plane gRPC API) | `pkg/proto/<name>` |
-| Internal proto (atelet / ateom) | `internal/proto/<name>` |
+| Internal proto (atelet / ateworker) | `internal/proto/<name>` |
 | Dev/CI scripts | `hack/` |
 | Standalone Go dev/CI tools | `tools/<name>` with its own `go.mod` |
 | Self-contained plugin that nothing in the repo depends on | `internal/plugins/<name>` with its own `go.mod`, importing only `pkg/`; see `internal/plugins/README.md` |

README.md

@@ -243,7 +243,7 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
 * [Architecture](docs/architecture.md): How the control plane, node supervisor, and networking stack fit together.
 * [API Configuration Guide](docs/api-guide.md): Detailed reference for configuring WorkerPools, ActorTemplates, Secrets, and Volumes.
 * [Full CLI Documentation](cmd/kubectl-ate/README.md): Installation and usage for `kubectl-ate`.
-* [Glossary](docs/glossary.md): Core terms (Actor, Atespace, ActorTemplate, WorkerPool, Worker, ate-api-server, atenet, atelet, ateom) and how they relate.
+* [Glossary](docs/glossary.md): Core terms (Actor, Atespace, ActorTemplate, WorkerPool, Worker, ate-api-server, atenet, atelet, ateworker) and how they relate.
 * [Integration Repositories](docs/integration-repos.md): Where integrations live, how their repositories are named, and how fixes flow back to core.
 * [Observability Guide](docs/observability.md): Guide to actor logging, metrics, and distributed tracing.
 * [Authentication Guide](docs/authentication.md): Configure trusted JWT providers and human credentials.
@@ -263,8 +263,8 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
 * `cmd/atelet`: A node-level DaemonSet that supervises physical worker pods, coordinates snapshotting, and manages state transfers.
 * `cmd/atecontroller`: A Kubernetes controller that reconciles WorkerPool custom resources.
 * `cmd/atenet`: A combined networking controller providing Envoy routing and proxy sidecars.
-* `cmd/ateom-gvisor`: An interior-pod helper running inside sandboxed worker pods to execute `runsc` checkpoint and restore commands.
-* `cmd/ateom-microvm`: The micro-VM peer of `ateom-gvisor`, running actors as cloud-hypervisor VMs.
+* `cmd/ateworker-gvisor`: An interior-pod helper running inside sandboxed worker pods to execute `runsc` checkpoint and restore commands.
+* `cmd/ateworker-microvm`: The micro-VM peer of `ateworker-gvisor`, running actors as cloud-hypervisor VMs.
 * `cmd/podcertcontroller`: A "polyfill" that provides Pod Certificate signers that
   will eventually ship in upstream Kubernetes (with different names).
 * `cmd/kubectl-ate`: A CLI tool for managing Agent Substrate resources. See its [README](cmd/kubectl-ate/README.md).