[romkey]

Summary

Each base docker-compose.yml for ollama, llama-cpp and vllm is now hardware-neutral (runs on CPU). Hardware overrides are chosen with COMPOSE_FILE in .env, following the same layout as experiments/comfyui.

Hardware	ollama	llama.cpp	vLLM
NVIDIA cards (nvidia.yml)	stock image	server-cuda	vllm-openai (CUDA 13)
DGX Spark	nvidia.yml	nvidia.yml + spark.yml (server-cuda13)	nvidia.yml, optionally + spark.yml (spark-arena build)
Jetson Thor (jetson.yml)	runtime: nvidia	server-cuda13 + runtime: nvidia	vllm-openai + runtime: nvidia
AMD Strix Halo / Point (strix.yml)	rocm image	server-vulkan	vllm-openai-rocm
Intel Arc (arc.yml)	Vulkan	server-intel (SYCL)	vllm-openai-xpu
CPU	base file	base file	base file (vllm-openai-cpu)

.env.example and the README for each app document every COMPOSE_FILE line and the per-backend variables.

Behavior changes

• The NVIDIA GPU reservation moved out of the ollama and vllm base files into docker-compose.nvidia.yml. A host whose .env doesn't set COMPOSE_FILE will now run on the CPU. .env.example defaults to the NVIDIA line for both.
• The llama.cpp GPU files default LLAMA_ARG_N_GPU_LAYERS to 999. An existing LLAMA_ARG_N_GPU_LAYERS=0 in .env still overrides that.

Testing

• docker compose config resolves for every hardware combination, with and without the NFS override.
• yamllint passes. .github/scripts/validate_compose.py reports only the three !override errors in the NFS files, which are already on main.
• Not run on real hardware.

🤖 Generated with Claude Code

---

Note

Medium Risk
Existing deployments that relied on NVIDIA settings baked into the base compose (without COMPOSE_FILE) will silently fall back to CPU until .env is updated; vLLM’s default image change has the same effect.

Overview
ollama, llama-cpp, and vllm now follow the same pattern as experiments/comfyui: each base docker-compose.yml is CPU-only, and you pick the stack via COMPOSE_FILE in .env. New override files cover NVIDIA (nvidia.yml), Jetson Thor (jetson.yml, runtime: nvidia instead of GPU reservations), AMD Strix (strix.yml), Intel Arc (arc.yml), plus DGX Spark layers (spark.yml for llama-cpp/vllm). NFS snippets are documented as the last file in the chain (with a special ordering note for vLLM Arc).

NVIDIA GPU deploy.resources blocks move out of the ollama and vllm base compose files into docker-compose.nvidia.yml, so a plain docker compose up without the right COMPOSE_FILE runs on CPU. vllm’s default image switches from CUDA vllm-openai to vllm-openai-cpu until nvidia.yml (or another override) is selected.

llama-cpp stops using a single IMAGE_VERSION table for backend choice: each override sets its own default tag (server-cuda, server-vulkan, server-intel, etc.), GPU overrides default LLAMA_ARG_N_GPU_LAYERS to 999 (replacing -1 / base 0), and .env.example / README are expanded for backend-specific env vars.

^{Reviewed by Cursor Bugbot for commit e318533. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e318533. Configure here.}

[cursor]

Install path drops GPU overrides

High Severity

start_app invokes docker compose -f docker-compose.yml, which ignores COMPOSE_FILE from .env. GPU reservations now live only in the hardware overrides, so install-ai-satellite.sh and install-chaos.sh start ollama on CPU even when .env selects docker-compose.nvidia.yml. start_experiment already omits -f so those overrides are honored.

Additional Locations (1)

• apps/vllm/docker-compose.yml#L7-L29

 

^{Reviewed by Cursor Bugbot for commit e318533. Configure here.}