[romkey]

Summary

• vLLM entrypoint fix. config/entrypoint.sh set VLLM_USE_FLASHINFER_MOE_FP4, VLLM_MOE_BACKEND, VLLM_FP4_BACKEND and VLLM_MLA_DISABLE without export, so vLLM never saw them. Three of the four also no longer exist in vLLM: they are now the --moe-backend / --linear-backend flags. The assignments are removed. The Spark section of .env.example now has VLLM_MLA_DISABLE=1, a VLLM_ARGS line with --moe-backend marlin --linear-backend cutlass, and the old names for older builds, all commented out.
• Tuned settings per platform. Each runner's .env.example now has a commented-out block for NVIDIA discrete GPUs, DGX Spark, Jetson Thor, AMD Strix (Halo and Point), Intel Arc and CPU. Uncomment the block for your hardware. vLLM has one VLLM_ARGS line per platform, because it is a single variable. Every variable and flag was checked against current ollama envconfig, llama.cpp common/arg.cpp and ggml backends, and vLLM envs.py / arg_utils.py.
• llama.cpp fit. llama.cpp now fits GPU layers and context to free memory by default (--fit on, -ngl -1). The LLAMA_ARG_N_GPU_LAYERS=999 default that ollama, llama-cpp, vllm: add per-hardware compose overrides #15 added to the GPU overrides turned that off, so this removes it. arc.yml sets ZES_ENABLE_SYSMAN=1 so SYCL reports free VRAM to --fit.
• Inline comments. Compose reads VAR= # note as VAR="# note". Empty-value lines carry their comment on the line above instead.

Testing

• Uncommented every variable line of each .env.example into a temporary .env and ran docker compose config for every hardware override: no errors, and no values containing comment text.
• yamllint passes. validate_compose.py reports only the three !override errors already on main. sh -n passes on the entrypoint.
• Not run on real hardware; the tuned values are starting points.

🤖 Generated with Claude Code

---

Note

Low Risk
Mostly documentation and example env; runtime impact is dropping llama-cpp's pinned GPU layers default and removing ineffective vLLM entrypoint env vars, which should improve defaults rather than break production configs that already set overrides in .env.

Overview
Adds commented, hardware-specific tuning blocks to .env.example for llama-cpp, ollama, and vllm (NVIDIA discrete, DGX Spark, Jetson Thor, AMD Strix Halo/Point, Intel Arc, CPU). READMEs now point users to uncomment the block for their platform.

llama-cpp: Drops the compose default LLAMA_ARG_N_GPU_LAYERS=999 from GPU override files so server memory fitting (--fit) can auto-size GPU layers and context; Intel Arc compose sets ZES_ENABLE_SYSMAN=1 so SYCL can report free VRAM for fitting. Example env documents fit-related vars and many server flags.

vLLM: Removes unused/non-exported MoE/MLA assignments from config/entrypoint.sh (Spark tuning moves to commented .env.example via VLLM_ARGS and optional legacy env names). Restructures VLLM_ARGS guidance with one suggested line per platform.

Ollama: Reorganizes .env.example with the same platform sections (context, parallel, KV cache, GPU overhead, etc.) without compose behavior changes beyond documentation.

^{Reviewed by Cursor Bugbot for commit 2e44106. Bugbot is set up for automated code reviews on this repo. Configure here.}