Skip to content

feat(gpu): support runtime-owned Docker GPU compatibility for WSL and Jetson #4837

Open

Description

@prekshivyas

User Story

We use OpenShell as the sandbox runtime for NemoClaw. While qualifying the OpenShell 0.1.3 upgrade in NVIDIA/NemoClaw#12603, we encountered two failing Docker GPU compatibility scenarios.

NemoClaw retains compatibility routes for Docker Desktop WSL and Jetson/Tegra. We need a supported GPU provisioning workflow that preserves sandbox security and lifecycle ownership. The real-host evidence below covers ARM64 Docker Desktop WSL only; Jetson/Tegra requirements remain unqualified.

Terminology and Requested Behavior

  • OpenShell Runtime owns sandbox creation, supervision, isolation, credentials, and lifecycle.
  • NVIDIA Container Runtime is Docker's nvidia runtime, selected with --runtime=nvidia. Earlier references to “NVIDIA runtime” meant this component, not OpenShell Runtime.
  • CDI describes GPU devices and their required resources; OpenShell currently uses NVIDIA CDI discovery for its native GPU path.

In response to elezar's question, the Docker compatibility behaviors under discussion are equivalents of:

docker run --gpus={{ .Request }} ...
docker run --gpus={{ .Request }} --runtime=nvidia ...

These describe the host behaviors we want to accommodate, not a requirement to add two new OpenShell modes. On the tested host, --gpus all alone works for GPU visibility with Docker's default runtime runc; explicitly selecting --runtime=nvidia also works but is not necessary for that check. Other host configurations must be tested separately.

We prefer a supported CDI solution if #4204 and #2846 can provide NVIDIA device discovery and the required sandbox access on this host. Whether any additional non-CDI mechanism belongs in OpenShell remains an open maintainer design decision.

Problem Statement

The released OpenShell 0.1.7 Docker GPU path rejects the tested WSL host before sandbox allocation because no NVIDIA CDI devices are discovered, while Docker GPU visibility checks succeed. The evidence below motivates resolving that host gap; it does not establish that an additional compatibility mode is preferable to the ongoing CDI work.

NemoClaw's older compatibility code recreates a workload container externally. That approach does not carry forward to the split supervisor/workload runtime: OpenShell owns bootstrap staging, launch credentials, workload networking, and restart identity.

The original single-container selector was a NemoClaw defect. Beyond selection, NemoClaw's legacy restart/recreation validator also assumes the older container contract and rejects the split workload. Those integration defects belong in NemoClaw; they are separate from the native OpenShell CDI discovery failure reproduced below. Correcting role selection alone does not make external workload recreation safe or resolve missing NVIDIA CDI discovery.

This issue requests a supported runtime workflow. It does not assert that OpenShell has already promised support for every affected host.

Impact / Why This Matters

The upgrade cannot qualify the retained GPU compatibility behavior. The previous full NemoClaw run failed both scenarios:

Those CI results establish the compatibility-path failure on Linux. They are not physical WSL or Jetson validation.

Real-Host Reproduction — 2026-11-16

Component Tested configuration
Hardware ARM64 NVIDIA RTX Spark N1X
Windows / NVIDIA driver Windows 28000.3231 / driver 621.14
WSL / Linux WSL 2.7.14.0; kernel 6.18.33.2-microsoft-standard-WSL2; Ubuntu 24.04.5
Docker Desktop 4.97.0.244584; Engine 29.9.2; aarch64; default runtime runc
OpenShell Official released 0.1.7 ARM64 package openshell_0.1.7-1_arm64.deb
Networking WSL mirrored networking, unchanged

OpenShell package SHA256 (matched to release metadata):
354f74db54b21ed34f7bacdc5c97ee35749e4d3690f2c3b83b3a6e32ac5bed05.

Docker discovery fields from docker info --format '{{json .}}':

{
  "CDISpecDirs": ["/etc/cdi", "/var/run/cdi"],
  "DiscoveredDevices": [
    {"Source": "cdi", "ID": "docker.com/gpu=webgpu"}
  ],
  "DefaultRuntime": "runc"
}

CDI directories and a WebGPU CDI entry exist, but no NVIDIA CDI device is discovered. This is not a claim that all CDI discovery is empty.

Using the same public ARM64 Ubuntu image for the controls and native creation request:

IMAGE=ubuntu@sha256:73b999393d07f2bf65a40373778a44dc9a6d54ad7718d29247d8ff9413c4d6a8

docker run --rm --network none --gpus all "$IMAGE" \
  nvidia-smi --query-gpu=name,driver_version --format=csv,noheader

docker run --rm --network none --gpus all --runtime=nvidia "$IMAGE" \
  nvidia-smi --query-gpu=name,driver_version --format=csv,noheader

Both exit 0 and print:

NVIDIA RTX Spark N1X (6144-core Blackwell RTX GPU), 621.14

Against an isolated, authenticated OpenShell 0.1.7 gateway:

openshell --gateway TASK_GATEWAY sandbox create \
  --name TASK_SANDBOX --from "$IMAGE" --gpu --detach -- sleep 600

This exits 1 in approximately 0.098 seconds:

no NVIDIA CDI GPU devices were discovered

The gateway log records RPC status 9 rejecting creation before allocation. The gateway used mTLS user authentication and gateway-issued sandbox JWTs; no insecure flags, external container cloning, bootstrap replay, generated CDI specs, or daemon/network configuration changes were used. Task-owned runtime resources were cleaned up.

Evidence limits: the Docker controls prove GPU visibility, not CUDA kernel execution or inference. Native OpenShell creation did not reach workload execution, so this does not qualify sandbox CUDA, lifecycle, endpoint connectivity, or NemoClaw E2E behavior. Jetson and x86 WSL remain untested. Although this host uses mirrored networking, this failure happens before the seccomp listener path in #4902; it neither reproduces nor resolves that separate issue.

Proposed Design

A caller should request a GPU sandbox through OpenShell's managed creation workflow. First determine whether the existing CDI work can support the observed host configuration. If an additional GPU injection mechanism is needed, its necessity and supported host contract require maintainer agreement before implementation. OpenShell must retain lifecycle ownership.

GPU attachment alone is insufficient. Landlock policies must permit the required device nodes and libraries, with the CDI-derived policy work in #4204 considered as part of the solution. The workflow must preserve required devices, libraries, and supplementary groups together with supervisor/workload identity, bootstrap credentials, networking, restart, rollback, and cleanup. Non-root execution, seccomp, dropped capabilities, and network isolation must remain enforced.

Please establish the supported host requirements and configuration contract before implementation. This proposal does not require a particular new CLI flag or internal replacement mechanism.

Scope and Ownership

The broader acceptance criteria below remain outstanding. This description separates the workstreams in response to triage; it does not claim that separate deliverables have been accepted or completed.

Acceptance Criteria

  • Establish a supported OpenShell workflow for the existing Docker GPU compatibility requirements, including hosts without usable NVIDIA CDI devices, or demonstrate an equivalent CDI path on those exact host configurations.
  • Demonstrate real CUDA initialization through that workflow on Docker Desktop WSL and Jetson/Tegra. Record runtime versions, host profiles, tested commits, commands, and results.
  • Preserve the selected GPU devices, required libraries, supplementary groups, and inference behavior through sandbox stop/start and gateway restart.
  • Keep workload networking isolated and keep launch credentials under runtime ownership. Callers must not copy bootstrap private keys or externally clone the workload to obtain GPU access.
  • On creation or transition failure, preserve existing workload data and rollback authority. Reject foreign or ambiguous workload identities.
  • Qualify both NemoClaw compatibility-only and native-rejection/fallback scenarios using the published runtime artifact. Preserve the existing assertions.

Alternatives Considered

  • CDI-first solution: feat(gpu): derive Docker sandbox requirements from CDI specs #4204 and fix(supervisor): preserve standard sbin paths in sandbox shells #2846 are the maintainer-identified work to assess before adding compatibility behavior. The historical assessment of feat(gpu): derive Docker sandbox requirements from CDI specs #4204 at 33350a881f55011286e7c3a68d8007e43e50e0bf covered CDI-derived devices, mounts, sockets, and groups; it was not a qualification of the current revisions on the host above. We have not established that those proposals cannot solve this case.
  • Existing design: #1606 covers CDI-derived sandbox access. This issue records the remaining compatibility requirement rather than duplicating that resolver work.
  • External container recreation: fixing role selection leaves bootstrap staging, network isolation, and restart admission unresolved.
  • Middleware or inference providers: these extension points do not own Docker GPU device injection or workload creation.
  • Disable GPU support: that does not preserve NemoClaw's retained support requirements.

Agent Investigation

Historical source inspection used OpenShell 0.1.3 revision e1f3c82caa3ed3b65de22889ae7ef32a774878ef and release tag v0.1.5. These are historical references, not the release used for the new reproduction above.

In the v0.1.5 Docker driver, DockerSandboxDriverConfig exposes cdi_devices and mounts, and GPU DeviceRequest uses driver cdi. Source inspection does not establish a live failure on v0.1.5.

The 0.1.3 driver stages runtime/bootstrap files after workload creation and requires workload network mode none. Restart selection cannot target an externally created replacement by immutable Docker ID while an original with the same sandbox labels remains retained.

The new physical ARM64 WSL reproduction is recorded above. Jetson remains untested. Separately, rsliter's earlier prototype report describes CUDA execution, isolation, and lifecycle testing on ARM64 N1X WSL using an unpublished prototype based on #4204. That attributed prototype evidence is not released OpenShell 0.1.7 behavior, Jetson/x86 WSL qualification, or completion of NemoClaw's compatibility E2E scenarios.

Checklist

  • Reviewed relevant existing issues, PRs, runtime source, and repository issue templates.
  • This is a workflow proposal with observable acceptance criteria.

Activity

  1. rsliter commented on Oct 22, 2026

    @rsliter

    We have a local prototype extending #4204 with OpenShell-owned NVIDIA GPU injection for Docker Desktop WSL, tested on real ARM64 N1X hardware. This provides a path to unblock the most critical flow in this issue for the upgrade to OpenShell 0.1.x.

    What it does

    • Adds an explicit per-sandbox configuration: --gpu --driver-config-json '{"docker":{"nvidia_wsl_gpus":"all"}}'. The gateway operator must enable driver configuration.
    • Requires a Docker Desktop WSL daemon and explicitly selects all GPUs. It rejects conflicting CDI selection or an explicit GPU count; the existing CDI path remains unchanged.
    • Adds the NVIDIA GPU request during normal OpenShell workload creation. It does not externally recreate containers or copy bootstrap credentials.
    • Records the selection in protected runtime metadata and checks it during admission and restart.
    • Retains non-root execution, Landlock, seccomp, dropped capabilities, and workload network isolation.

    What we tested

    On an NVIDIA RTX Spark N1X running Ubuntu 24.04 under WSL2, Docker Desktop 4.96.0 / Engine 29.9.1, and NVIDIA driver 620.66:

    • CUDA initialization, device enumeration, memory allocation, PTX kernel execution, and result retrieval passed inside OpenShell as UID 998.
    • Filesystem writes outside the allowed paths and direct network connections were denied. A control test confirmed the filesystem write succeeds for the same non-root user outside OpenShell.
    • CUDA and isolation checks passed again after sandbox stop/start and a dedicated gateway restart.
    • The workload retained its exact container identity, GPU request, and sandbox state across those operations.
    • Sandbox and supervisor cleanup completed, and the dedicated test gateway was stopped.

    The Docker driver suite passed 165 tests on both macOS and ARM64 Linux, with two existing tests ignored. Strict Clippy and formatting checks also passed.

    Scope and remaining work

    This is an unpublished prototype based on #4204 at 33350a881f55011286e7c3a68d8007e43e50e0bf. It demonstrates the explicit all-GPU path on this ARM64 WSL host. It does not yet qualify Jetson, x86 WSL, individual GPU selection on multi-GPU hosts, model inference, or NemoClaw’s two compatibility scenarios against a published runtime artifact.

    The hardware test used an isolated development gateway configuration, so production gateway transport/authentication is also outside this evidence.

    This supports a focused Docker Desktop WSL implementation slice; it does not establish completion of all #4837 acceptance criteria.

    Feedback or Approval Requested

    We'd love feedback on this approach, or approval to proceed.

  2. purp commented on Nov 5, 2026

    @purp
    Collaborator

    📋 triage-agent

    Thanks for the detailed write-up. The problem is clear: OpenShell's Docker driver does not support the Docker Desktop WSL and Jetson/Tegra configurations that NemoClaw needs. A few specifics are needed before a maintainer can decide.

    What the current code does. On main, the Docker driver treats CDI as supported when docker info reports non-empty CDISpecDirs. It builds its GPU inventory only from DiscoveredDevices entries with source cdi. The only injection path is a DeviceRequest with driver cdi. If no NVIDIA CDI devices are discovered, --gpu fails with "no NVIDIA CDI GPU devices were discovered". If the daemon reports no CDI spec dirs, it fails with "docker GPU sandboxes require Docker CDI support".

    The existing WSL2 nvidia.com/gpu=all fallback applies only when the daemon already lists nvidia.com/gpu=all as a discovered device. It does not help when discovery returns nothing. This is consistent with the issue's premise, but the code alone cannot show what Docker Desktop WSL and Jetson hosts actually report.

    Information needed

    1. For each target host (Docker Desktop WSL, Jetson/Tegra): the docker info CDI fields (CDISpecDirs, DiscoveredDevices), plus the exact error from openshell sandbox create --gpu on current main or the latest release.
    2. Which NemoClaw failures remain once the NemoClaw single-container selector defect is fixed. That part is outside OpenShell and should be tracked in NemoClaw.

    Scope. This issue combines two host families, hardware qualification, rollback guarantees, and cross-repo NemoClaw qualification. Please split it into separately deliverable issues, each with its own acceptance criteria:

    Then narrow or retitle this issue to what remains. Related: #4204 (CDI base, currently open and stale), #1606 (closed), #4327.

    Next human decision: a maintainer decides whether to accept the split work once the above is provided.

  3. added
    state:needs-infoAssessment needs specific evidence or reproduction details
    and removed
    state:triage-neededOpened without agent diagnostics and needs triage
    on Nov 5, 2026
  4. elezar commented on Nov 5, 2026

    @elezar
    Member

    @prekshivyas could you please update the issue description to make the distinction betwen the OpenShell Runtime and the NVIDIA Container Runtime (which I assume is referred to as the "NVIDIA runtime") more clear.

    If I understand that ask, what you are asking for is a mechanism to switch between our current CDI-only-based injection to at least the following alternatives:

    • Something equivalent to: docker run --gpus={{ .Request }}
    • Something equivalent to: docker run --gpus={{ .Request }} --runtime=nvidia

    Is this correct?

    In general, I don't think pushing the compatibility behaviour into OpenShell is the right approach here (or at least not without further motivation). The existing work in #4204 and #2846 attempts to address platform-specific inconsistencies and should remove the need for most of the workarounds that are being put in place on these systems.

    One thing to note: Injecting a GPU into the sandbox using an alternative mechansim does not guarantee that the injected paths (device nodes, libraries, etc.) are accessible in a sandbox. These may still be restricted by the landlocked policies that we're applying. The intent of #4204 is to ensure that we also use the CDI specification to populate the required sandbox policies.

  5. prekshivyas commented on Nov 16, 2026

    @prekshivyas
    ContributorAuthor

    @elezar Updated the issue description in place to address your feedback:

    • Distinguished OpenShell Runtime from NVIDIA Container Runtime, and clarified the two Docker command equivalents you asked about.
    • Added real ARM64 N1X WSL evidence: both --gpus all and --gpus all --runtime=nvidia pass GPU visibility checks, while released OpenShell 0.1.7 rejects creation with no NVIDIA CDI GPU devices were discovered. Docker reports a WebGPU CDI entry, but no NVIDIA CDI devices; its default runtime is runc.
    • Clarified that we prefer the CDI route through feat(gpu): derive Docker sandbox requirements from CDI specs #4204 / fix(supervisor): preserve standard sbin paths in sandbox shells #2846 if it covers this host, and that GPU attachment must also preserve the required Landlock access. An additional compatibility mechanism remains an open design question.
    • Separated the NemoClaw recreation defects and the unqualified Jetson/x86 WSL and mirrored-networking paths from this reproduction.

    Does this provide the motivation and host evidence you needed to assess whether the existing CDI work covers this case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:needs-infoAssessment needs specific evidence or reproduction details

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions