Skip to content

[Bug] Vulkan ErrorOutOfDeviceMemory on Windows with AMD Radeon HD 8570 despite --gpu none #1073

Description

@PORTRAITART1

Environment:

  • Windows 10 (26200)
  • Docker Desktop 4.92.0
  • GPU: AMD Radeon HD 8570 (1 GB VRAM)
  • Docker Model Runner: llama.cpp b9879-cpu

Problem:
Docker Model Runner tries to use Vulkan even when installed with --gpu none.
Models >= 1B parameters crash with ggml_vulkan: ErrorOutOfDeviceMemory.
Small model (smollm2:360M) runs but produces garbage output.

Steps to reproduce:

  1. docker model install-runner --backend llama.cpp --gpu none
  2. docker model run ai/gemma3:1B-Q4_K_M "Bonjour"
  3. Observe crash

Expected behavior:
Model should run on CPU when --gpu none is specified.

Actual behavior:
llama.cpp still attempts Vulkan allocation, fails, and aborts.

Error logs:
_layers already set by user to 999, abort
ggml_vulkan: Device memory allocation of size 57671680 failed.
ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache

Note:
The environment variable DOCKER_MODEL_RUNNER_LLAMA_CPP_ARGS="--n-gpu-layers 0 --no-vulkan" did not resolve the issue.
Docker Desktop requirements list only NVIDIA GPUs for Windows (amd64), but the error suggests Vulkan is still being attempted on AMD hardware.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions