Automated Sync from main to stable

.agents/skills/test-release-canary/SKILL.md

@@ -16,7 +16,7 @@ The Release Canary (`.github/workflows/release-canary.yml`) smoke-tests the arti
 | `macos` | `macos-latest-xlarge` | Installs the dev Homebrew artifacts, reaches the VM gateway, and creates, executes in, and deletes a sandbox. |
 | `ubuntu-deb` | `ubuntu-latest` | Installs the dev Debian package, reaches the Docker gateway, and creates, executes in, and deletes a sandbox. |
 | `fedora` | `fedora:latest` container | Installs the dev RPM packages, reaches the Podman gateway, and creates, executes in, and deletes a sandbox. |
-| `ubuntu-snap-system-docker` | `ubuntu-latest` | Uses `install.sh` to install the snap from `latest/edge`, reuses system Docker, reaches the Docker gateway, and creates, executes in, and deletes a sandbox, and verifies that the Docker snap is not installed. |
+| `ubuntu-snap-system-docker` | `ubuntu-latest` | Uses `install.sh` to install the snap from `latest/edge`, reuses system Docker, verifies the packaged prover version and a local policy boundary check, reaches the Docker gateway, creates, executes in, and deletes a sandbox, and verifies that the Docker snap is not installed. |
 | `ubuntu-snap-docker-preflight` | `ubuntu-latest` | Verifies that `install.sh` rejects the OpenShell Snap path when Docker is absent or supplied by the Docker snap, without installing OpenShell. |
 | `kubernetes` | `ubuntu-latest` + kind | Installs the dev Helm chart, reaches the in-cluster gateway, and creates, executes in, and deletes a sandbox using the published runtime images. |
 
@@ -144,6 +144,7 @@ Loopback registration auto-derives the gateway name to `openshell` if `--name` i
 | Sandbox create or exec fails | Published sandbox and supervisor artifacts are missing, incompatible, or cannot establish the protected runtime channel. | Gateway logs plus Docker, Podman, VM, Snap, or Kubernetes runtime diagnostics for the job. |
 | `macos`/`ubuntu-deb`/`fedora` job fails on `openshell status` | Local gateway service did not start (systemd/brew/podman). Often a driver issue. | Service logs in the job log; `OPENSHELL_COMPUTE_DRIVER` env in the "Ensure …" step. |
 | `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable, the edge revision or automatic interfaces were unavailable, or the gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, gateway and snapd journals, snap logs, and port 17670 listeners. |
+| `ubuntu-snap-system-docker` fails during the prover checks | The prover artifact is missing or packaged for the wrong architecture, `openshell.prover` is not exposed or confined to read the test policies, or its solver linkage is not runnable. | The `Verify Snap installation` and `Check a policy boundary with the Snap prover` steps, plus `snap info openshell` and `snap connections openshell`. |
 | `ubuntu-snap-docker-preflight` unexpectedly succeeds | The installer no longer fails before installing the OpenShell snap when Docker is absent or supplied by the Docker snap. | Inspect `install.log`, `docker-snap.log`, `snap list`, and snapd changes. |
 | `kubernetes` job fails on `helm install --wait` | Chart did not deploy in 5 min — usually image pull failure or readiness probe failing. | "Diagnostics on failure" step dumps `helm status`, manifest, pod describe, pod logs. |
 | `kubernetes` job fails on `kubectl wait` | Gateway pod stuck `CrashLoopBackOff` or `ImagePullBackOff`. | Diagnostics dump; check `:dev` image existence at `ghcr.io/nvidia/openshell/gateway`. |

.agents/skills/watch-github-actions/SKILL.md

@@ -125,6 +125,13 @@ gh run list --json databaseId,status,headBranch,url --jq '.[] | {id: .databaseId
 
 ## View Job Logs
 
+`setup-nix` retries development-shell preparation once when `prepare-shell`
+is enabled. Inspect both attempts in the job log; `setup-rust` assumes the
+shell has already been prepared. Cargo, lint, and test commands are not retried.
+Direct Nix builds and app dependency preparation also retry once; apps run
+once after preparation succeeds. Skipped dependent E2E suites indicate blocked
+coverage.
+
 For `Trivy Changes`, inspect the `Resolve PR baseline` step for the base and head
 SHAs. PR runs compare the tested merge commit with its
 first parent; change detection and scans must use the same pair. On reruns, do

.github/actions/build-rust-binary/action.yml

@@ -29,6 +29,10 @@ inputs:
     description: Additional flags passed to cargo build
     required: false
     default: ""
+  entitlements-file:
+    description: Path to an entitlements plist for macOS ad-hoc codesigning (skipped when empty or not on macOS)
+    required: false
+    default: ""
 runs:
   using: composite
   steps:
@@ -66,6 +70,15 @@ runs:
         # Confirm Syft can decode the embedded cargo-auditable metadata.
         SYFT_CHECK_FOR_APP_UPDATE=false syft "file:${binary}" -o cyclonedx-json | grep 'pkg:cargo/' > /dev/null
 
+    - name: Codesign with entitlements
+      if: inputs.entitlements-file != '' && runner.os == 'macOS'
+      shell: bash
+      env:
+        INPUTS_ENTITLEMENTS: ${{ inputs.entitlements-file }}
+        INPUTS_TRIPLE: ${{ inputs.triple }}
+        INPUTS_BINARY: ${{ inputs.binary }}
+      run: /usr/bin/codesign --entitlements "${INPUTS_ENTITLEMENTS}" --force -s - "target/${INPUTS_TRIPLE}/release/${INPUTS_BINARY}"
+
     - name: Upload ${{ inputs.binary }}
       uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
       with:

.github/actions/check-protobuf-compatibility/action.yml

@@ -19,4 +19,7 @@ runs:
       env:
         CHECK_REF: ${{ inputs.ref }}
       run: |
+        # Retry dependency preparation, then run the compatibility check once.
+        nix build --no-link --no-write-lock-file .#check-protobuf-compatibility ||
+          nix build --no-link --no-write-lock-file .#check-protobuf-compatibility
         nix run .#check-protobuf-compatibility --no-write-lock-file -- "$CHECK_REF"

.github/actions/setup-nix/action.yml

@@ -2,14 +2,23 @@
 # SPDX-License-Identifier: Apache-2.0
 
 name: Setup Nix
-description: Install Nix and configure the OpenShell Cachix cache
+description: Install Nix, configure Cachix, and optionally prepare the development shell
 
 inputs:
   cachix-auth-token:
     description: Token used to write build outputs to Cachix
     required: false
     default: ""
 
+  prepare-shell:
+    description: Prepare the development shell, retrying once on failure
+    required: false
+    default: "false"
+  shell-installable:
+    description: Development shell to prepare
+    required: false
+    default: "."
+
 runs:
   using: composite
   steps:
@@ -22,3 +31,16 @@ runs:
         name: openshell
         authToken: ${{ inputs.cachix-auth-token }}
         skipPush: ${{ inputs.cachix-auth-token == '' }}
+
+    - name: Prepare Nix development shell
+      if: inputs.prepare-shell == 'true'
+      shell: bash
+      env:
+        NIX_SHELL_INSTALLABLE: ${{ inputs.shell-installable }}
+      run: |
+        # HTTP 416 is not retried by Nix; a fresh invocation restarts downloads.
+        if nix develop "$NIX_SHELL_INSTALLABLE" -c true; then
+          exit 0
+        fi
+        echo "::warning::Nix shell preparation failed; retrying once."
+        nix develop "$NIX_SHELL_INSTALLABLE" -c true

.github/actions/setup-rust/action.yml

@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: Apache-2.0
 
 name: Setup Rust
-description: Configure the Nix development shell and Rust caches
+description: Configure Rust caches after setup-nix has prepared the development shell
 
 inputs:
   cache-key:
@@ -23,10 +23,6 @@ runs:
         shell_drv=$(nix eval --raw --impure .#devShells --apply 'shells: shells.${builtins.currentSystem}.default.drvPath')
         echo "hash=$(nix hash file --type sha256 --base16 "$shell_drv")" >> "$GITHUB_OUTPUT"
 
-    - name: Realize Nix development shell
-      shell: bash
-      run: nix develop -c true
-
     - name: Cache Rust target and registry
       uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
       with:

.github/workflows/branch-checks.yml

@@ -23,7 +23,7 @@
 
 jobs:
   pr_metadata:
     if: false # CARRY: disable upstream-only job (requires runners unavailable in downstream fork)
     name: Resolve PR metadata
     runs-on: ubuntu-latest
     permissions:
@@ -65,7 +65,7 @@
     runs-on: linux-amd64-cpu8
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
@@ -90,7 +90,7 @@
     runs-on: linux-amd64-cpu8
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
@@ -115,14 +115,11 @@
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
-        with:
-          github_access_token: ${{ secrets.GITHUB_TOKEN }}
-
-      - uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
+      - uses: ./.github/actions/setup-nix
         with:
-          name: openshell
-          authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}
+          cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
+          prepare-shell: "true"
+          shell-installable: .#devShells.x86_64-linux.default
 
       - name: Check dependencies
         run: cargo deny check licenses bans sources
@@ -138,6 +135,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - name: Format
@@ -159,6 +157,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - name: Verify Cargo lockfiles
@@ -186,6 +185,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -222,6 +222,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -258,6 +259,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -307,6 +309,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -331,6 +334,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -357,6 +361,7 @@
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - uses: ./.github/actions/setup-rust
@@ -382,7 +387,7 @@
     runs-on: ${{ matrix.runner }}
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
@@ -418,7 +423,7 @@
     runs-on: linux-amd64-cpu8
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
@@ -438,7 +443,7 @@
     runs-on: linux-amd64-cpu8
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
@@ -458,7 +463,7 @@
     runs-on: linux-amd64-cpu8
     timeout-minutes: 30
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/branch-e2e.yml

@@ -144,6 +144,7 @@ jobs:
           ref: ${{ github.sha }}
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
       - uses: ./.github/actions/build-rust-binary
         with:
@@ -184,6 +185,7 @@ jobs:
           ref: ${{ github.sha }}
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
       - uses: ./.github/actions/build-rust-binary
         with:

.github/workflows/build-binaries.yml

@@ -51,6 +51,7 @@ jobs:
           ref: ${{ inputs.checkout-ref || github.sha }}
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
       - uses: ./.github/actions/build-rust-binary
         with:

.github/workflows/build-vm-driver.yml

@@ -39,6 +39,7 @@ jobs:
           - arch: aarch64
             triple: aarch64-apple-darwin
             runner: macos-15-xlarge
+            entitlements-file: crates/openshell-driver-vm/entitlements.plist
     runs-on: ${{ matrix.runner }}
     timeout-minutes: 60
     defaults:
@@ -52,6 +53,7 @@ jobs:
 
       - uses: ./.github/actions/setup-nix
         with:
+          prepare-shell: "true"
           cachix-auth-token: ${{ secrets.CACHIX_AUTH_TOKEN }}
 
       - name: Download openshell-sandbox
@@ -81,7 +83,9 @@ jobs:
           path: vm-init
 
       - name: Build VM runtime
-        run: nix build .#vm-runtime
+        run: |
+          # HTTP 416 is not retried by Nix; restart the build once on failure.
+          nix build .#vm-runtime || nix build .#vm-runtime
 
       - name: Assemble compressed VM runtime
         run: |
@@ -106,3 +110,4 @@ jobs:
           triple: ${{ matrix.triple }}
           cargo-version: ${{ inputs['cargo-version'] }}
           supervisor-image-tag: ${{ inputs['supervisor-image-tag'] }}
+          entitlements-file: ${{ matrix.entitlements-file }}

.github/workflows/cargo-deny.yml

@@ -35,7 +35,7 @@ jobs:
     name: Cargo Deny
     runs-on: linux-amd64-cpu8
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/deb-package.yml

@@ -37,7 +37,7 @@ jobs:
     runs-on: ${{ matrix.runner }}
     timeout-minutes: 20
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/e2e-docker-test.yml

@@ -50,7 +50,7 @@ jobs:
       matrix:
         include: ${{ fromJSON(inputs.suite-matrix) }}
     container:
-      image: ghcr.io/nvidia/openshell/ci:37072ee81cd7b294c714bfa5ecc829b6927b3d70
+      image: ghcr.io/nvidia/openshell/ci:37072ee81cd7b294c714bfa5ecc829b6927b3d70@sha256:ffa96b8009de6e28bbf157060440c3abb312d3bcda444e9571c539c8c6115615
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/e2e-gpu-test.yaml

@@ -38,7 +38,7 @@ jobs:
             runner: wsl-amd64-gpu-rtxpro6000-latest-1
             experimental: true
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/helm-lint.yml

@@ -24,7 +24,7 @@
 
 jobs:
   pr_metadata:
     if: false # CARRY: disable upstream-only job (requires runners unavailable in downstream fork)
     name: Resolve PR metadata
     runs-on: ubuntu-latest
     permissions:
@@ -82,7 +82,7 @@
     if: needs.pr_metadata.outputs.should_run == 'true' && needs.helm_changes.outputs.should_run == 'true'
     runs-on: linux-amd64-cpu8
     container:
-      image: ghcr.io/nvidia/openshell/ci:latest
+      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
       credentials:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/integration-runner.yml

@@ -85,4 +85,7 @@ jobs:
           ENVIRONMENT: ${{ matrix.environment }}
           INSTALLER: ${{ matrix.installer }}
           TESTSUITE: ${{ matrix.testsuite }}
-        run: nix run .#tmachine -- test "${ENVIRONMENT}" "${INSTALLER}" "${TESTSUITE}"
+        run: |
+          # Retry dependency preparation, then execute the test suite once.
+          nix build --no-link .#tmachine || nix build --no-link .#tmachine
+          nix run .#tmachine -- test "${ENVIRONMENT}" "${INSTALLER}" "${TESTSUITE}"