Environment:
- Windows 10 (26200)
- Docker Desktop 4.92.0
- GPU: AMD Radeon HD 8570 (1 GB VRAM)
- Docker Model Runner: llama.cpp b9879-cpu
Problem:
Docker Model Runner tries to use Vulkan even when installed with --gpu none.
Models >= 1B parameters crash with ggml_vulkan: ErrorOutOfDeviceMemory.
Small model (smollm2:360M) runs but produces garbage output.
Steps to reproduce:
- docker model install-runner --backend llama.cpp --gpu none
- docker model run ai/gemma3:1B-Q4_K_M "Bonjour"
- Observe crash
Expected behavior:
Model should run on CPU when --gpu none is specified.
Actual behavior:
llama.cpp still attempts Vulkan allocation, fails, and aborts.
Error logs:
_layers already set by user to 999, abort
ggml_vulkan: Device memory allocation of size 57671680 failed.
ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache
Note:
The environment variable DOCKER_MODEL_RUNNER_LLAMA_CPP_ARGS="--n-gpu-layers 0 --no-vulkan" did not resolve the issue.
Docker Desktop requirements list only NVIDIA GPUs for Windows (amd64), but the error suggests Vulkan is still being attempted on AMD hardware.
Environment:
Problem:
Docker Model Runner tries to use Vulkan even when installed with
--gpu none.Models >= 1B parameters crash with
ggml_vulkan: ErrorOutOfDeviceMemory.Small model (smollm2:360M) runs but produces garbage output.
Steps to reproduce:
Expected behavior:
Model should run on CPU when
--gpu noneis specified.Actual behavior:
llama.cpp still attempts Vulkan allocation, fails, and aborts.
Error logs:
_layers already set by user to 999, abort
ggml_vulkan: Device memory allocation of size 57671680 failed.
ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache
Note:
The environment variable DOCKER_MODEL_RUNNER_LLAMA_CPP_ARGS="--n-gpu-layers 0 --no-vulkan" did not resolve the issue.
Docker Desktop requirements list only NVIDIA GPUs for Windows (amd64), but the error suggests Vulkan is still being attempted on AMD hardware.