Make lcode bench count the part of a loaded model in RAM

CHANGELOG.md

@@ -6,6 +6,23 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+### Fixed
+
+- **The line after a load counts the part of the model in system RAM.** With Ollama 0.40, `qwen3.6-35b`
+  at 32K on a 12 GB GPU showed as `10.5 GB, 98% on the GPU and the rest in system RAM`, because Ollama
+  leaves out the ~12 GB of experts that stay mapped from the model's file. For the models in the catalog,
+  lcode now counts the whole model at its estimate, the one `lcode setup` and `lcode doctor` show:
+  `~23 GB, 42% on the GPU and the rest in system RAM`. With Ollama's numbers, the warning that another
+  program is using GPU memory couldn't fire. Now it does, and it says how much of the model fits on the GPU.
+- **`lcode doctor` shows the loaded model.** It looked for `lcode-qwen3.6-35b` while Ollama lists it as
+  `lcode-qwen3.6-35b:latest`, so the `Loaded` line never appeared for the models `lcode setup` creates.
+  It now says how the model fits, in the same words as the line after a load.
+- **`lcode bench` counts the part of the model in system RAM, too.** For `qwen3.6-35b` at 32K on a
+  12 GB GPU, it said `10.4 GB, 98% on the GPU` after loading, and the same in its summary. It now says
+  `~23 GB, 42% on the GPU and the rest in system RAM`, as the line after a load does. `--json` results
+  keep Ollama's count in `memory_gb` and `gpu_percent`, so they still compare with older results, and
+  add lcode's estimate in `estimated_memory_gib` and `estimated_gpu_percent`.
+
 ## [0.62.0] - 2026-11-06
 
 ### Changed

docs/bench.md

@@ -25,7 +25,7 @@ its memory use, so those columns stay empty.
 lcode bench · NVIDIA GeForce RTX 4080 Laptop GPU (12 GB VRAM), 31 GB RAM · Ollama 0.40.13
 
 lcode-qwen3.6-35b · 32K context · reasoning on · 8 tasks · 3 attempts each
-  Loaded in 18s · 10.4 GB, 98% on the GPU · reads prompts at 905 tok/s
+  Loaded in 18s · ~23 GB, 42% on the GPU and the rest in system RAM · reads prompts at 905 tok/s
   Attempt 1 of 3
   ✓ fix-bug       Fix a bug so the tests pass         22s  the tests pass
   ✓ find-code     Answer with file:line                8s  next_wait at backoff.py:13
@@ -182,7 +182,7 @@ quick tasks once, as `lcode bench` did before 0.57.
 | **Generation** | Output tokens per second, across all the model's responses |
 | **Prompt reading** | Prompt tokens per second: how fast the model reads files and tool output. Measured per model with fresh 4,000-token prompts (the faster of two readings, since the first one after loading includes warm-up), because during the tasks Ollama serves most of each request from its prompt cache |
 | **Tool-call errors** | Tool calls that failed (wrong arguments, editing without reading first, unmatched edits) or that the model wrote as invalid JSON |
-| **Memory** | The model's memory use and how much of it is on the GPU, from Ollama |
+| **Memory** | The model's memory and how much of it is on the GPU, counted as in the [line after a load](usage.md#whats-happening-under-the-hood). Ollama leaves out what stays mapped from the model's file in system RAM, so for a model in `lcode models` that's partly off the GPU, it's lcode's estimate of the whole model (`~23 GB`). Otherwise it's what Ollama says |
 
 Results depend on the context window, so `lcode bench` uses **32K** by default on every machine,
 which makes results comparable. When comparing models, each one is unloaded before the next starts
@@ -217,7 +217,7 @@ RTX 4080 Laptop GPU (12 GB) with 31 GB of RAM, Ollama 0.40.13, 32K context, reas
 | Generation | 64.0 tok/s |
 | Prompt reading | 851 tok/s |
 | Tool-call errors | 4 |
-| Memory | 10.5 GB, 98% on GPU |
+| Memory | ~23 GB, 42% on GPU |
 
 What it missed:
 - one `spec-fix` attempt fixed the reported case but not all the docstring's rules;
@@ -226,6 +226,9 @@ What it missed:
 - every `runtime-bug` attempt settled for a band-aid: the window still grew, or still couldn't be
   made smaller.
 
+The table shows memory as lcode counts it now. lcode 0.58.0 printed Ollama's count,
+`10.5 GB, 98% on GPU`, which leaves out the ~12 GB of experts in system RAM.
+
 Two `wide-feature` attempts were first counted as failed. Their `budget list` printed `No budgets
 set.` without budgets, and one listed the budgeted categories first. The task asks for neither
 behavior, so the check now accepts both (0.59.0), and the table counts them as passed.
@@ -302,6 +305,8 @@ existing fields keep their meaning until `schema` changes.
       "prompt_tps": 904.6,
       "memory_gb": 10.4,
       "gpu_percent": 98,
+      "estimated_memory_gib": 22.7,
+      "estimated_gpu_percent": 42,
       "attempts": 3,
       "passed": 24,
       "total": 24,
@@ -328,7 +333,8 @@ existing fields keep their meaning until `schema` changes.
 | `ollama`, `server` | The Ollama version, or `null` with [another model server](servers.md); `server` names the server either way |
 | `runs[].model` | The name you gave (catalog key or Ollama tag); `ollama_model` is the model that ran |
 | `runs[].context`, `num_batch`, `think` | The settings used. lcode lowers the batch size or context if the GPU runs out of memory, and reports what it ended up using |
-| `runs[].memory_gb`, `gpu_percent` | From Ollama's `/api/ps`; `null` if unavailable |
+| `runs[].memory_gb`, `gpu_percent` | The model's memory in GB and the share of it on the GPU, as Ollama's `/api/ps` counts them; `null` if unavailable. Ollama 0.40 leaves out what stays mapped from the model's file in system RAM: `qwen3.6-35b` at 32K on a 12 GB GPU shows as 10.4 GB, 98% of it on the GPU |
+| `runs[].estimated_memory_gib`, `estimated_gpu_percent` | When Ollama's count leaves out part of a model in `lcode models`: lcode's estimate of the whole model in GiB, and the share of it on the GPU, as the summary shows them (`~23 GB, 42% on GPU`). `null` when Ollama counts the whole model (all of it on the GPU, or a model outside the catalog) and in older results |
 | `runs[].generation_tps` | Output tokens per second across the tasks; `null` if Ollama reported no timings |
 | `runs[].prompt_tps` | Prompt tokens per second for a fresh 4,000-token prompt (the faster of two readings); `null` if it couldn't be measured |
 | `runs[].usage` | Totals over the tasks as Ollama reports them. `prompt_tokens` includes tokens served from the prompt cache |

docs/models.md

@@ -258,7 +258,8 @@ RAM needed ≈ memory − GPU memory + ~10 GB for Ollama's server and prompt cac
 
 - **The part of the model that doesn't fit on the GPU.** A Mixture-of-Experts model's experts stay
   mapped from the model's file, so they have to stay in Linux's page cache. `ollama ps` leaves them out:
-  it shows `qwen3.6-35b` at 32K as 10.5 GB, with ~12 GB more in RAM.
+  it shows `qwen3.6-35b` at 32K as 10.5 GB, with ~12 GB more in RAM. lcode counts them: after a load,
+  in `lcode doctor` and in `lcode bench`, it says `~23 GB, 42% on the GPU and the rest in system RAM`.
 - **Ollama's prompt cache**, which grows to 8 GB in a long session, next to 1–2 GB of the server's own.
 
 On a 12 GB GPU, `qwen3.6-35b` needs ~21 GB of RAM at 32K, ~23 GB at 128K and ~26 GB at 256K. On Linux,

docs/troubleshooting.md

@@ -84,8 +84,10 @@ need `lcode mcp login <name>`. More in [MCP servers: Troubleshooting](mcp.md#tro
   starts, and `keep_alive` keeps it loaded between requests.
 - Reading many large files takes a while the first time (~280 tokens/s on a 12 GB GPU, ~500 with
   `lcode config set num_batch 1024` if it fits); follow-up turns reuse the cache.
-- `ollama ps` shows how much of the model is on the GPU. Dense models are much slower when split;
-  prefer the MoE models in `lcode models` on smaller GPUs.
+- `lcode doctor` shows how much of the loaded model is on the GPU, for example
+  `Loaded ~23 GB, 42% on the GPU and the rest in system RAM · 32K context`. `ollama ps` leaves out the
+  part of a Mixture-of-Experts model that's in system RAM, so it can say 98%. Dense models are much
+  slower when split; prefer the MoE models in `lcode models` on smaller GPUs.
 - `--no-think` skips reasoning for simple requests.
 
 ### It starts fast, then gets about three times slower

docs/usage.md

@@ -409,7 +409,7 @@ stays loaded:
 
 | When | What it says |
 |---|---|
-| The model was loaded | `◆ Loaded qwen3.6-35b in 15s: 22.3 GB, 45% on the GPU and the rest in system RAM.`, and a warning when less of it is on the GPU than usual, because another program or model is using GPU memory |
+| The model was loaded | `◆ Loaded qwen3.6-35b in 15s: ~23 GB, 42% on the GPU and the rest in system RAM.`, and a warning when less of it is on the GPU than usual, because another program or model is using GPU memory |
 | A dense model doesn't fit on the GPU (at startup and on `/model`) | `⚠ qwen3.8-27b is a dense model: all 27B parameters work on every token. It's capable, but slow here. Only about 61% of it fits in the 12 GB of GPU memory…`, with what to do instead |
 | The model that looks at images needs the memory | `Loading the vision model … to look at bug.png; qwen3.6-35b makes room and reloads afterwards`, then `◆ qwen3.6-35b was unloaded to make room…`, and the next step says `Reloading qwen3.6-35b into memory (it made room for the vision model …)` |
 | A tool needs the GPU to itself (an MCP server with `free_gpu`) | `◆ Unloaded qwen3.6-35b so the server can use the GPU; it reloads afterwards.` |
@@ -421,6 +421,12 @@ While the model loads in the background at startup, the status bar says
 part of them in system RAM. Dense ones slow down a lot, to a few tokens a second, which is why they
 get the warning.
 
+The `~` marks lcode's estimate of the model's memory, the same one `lcode setup` and `lcode doctor`
+show. Ollama leaves out what stays mapped from the model's file in system RAM: `ollama ps` shows
+qwen3.6-35b at 32K on a 12 GB GPU as 10.5 GB, 98% of it on the GPU, while ~12 GB of its experts are
+in RAM. So when part of a model in `lcode models` is off the GPU, lcode counts all of it. For other
+models it repeats what Ollama says. `lcode doctor` shows the same line for the model in memory.
+
 ## Steer while it works
 
 A request can take minutes with a local model. You don't have to stop it to change course: type

src/lcode/bench.py

@@ -27,12 +27,12 @@
 from rich.table import Table
 from rich.text import Text
 
-from lcode import __version__, escalate
+from lcode import __version__, escalate, residency
 from lcode.agent import Agent, Settings
 from lcode.bench_checks import Check, Task
 from lcode.bench_quick import TASKS
 from lcode.config import format_tokens
-from lcode.hardware import Hardware
+from lcode.hardware import GIB, Hardware
 from lcode.ollama import Ollama, OllamaError
 
 SCHEMA_VERSION = 1
@@ -188,8 +188,10 @@ class ModelRun:
     think: bool = True
     load_seconds: float = 0.0
     prompt_tps: float | None = None  # measured with a fresh prompt; see prompt_speed()
-    memory_gb: float | None = None
-    gpu_percent: int | None = None
+    memory_gb: float | None = None  # as Ollama's /api/ps counts it, which can leave part of the model out
+    gpu_percent: int | None = None  # of memory_gb, on the GPU
+    estimated_memory_gib: float | None = None  # all of the model by the catalog's estimate, when Ollama leaves part out
+    estimated_gpu_percent: int | None = None  # of estimated_memory_gib, on the GPU
     usage: dict = field(default_factory=dict)
     tasks: list[TaskResult] = field(default_factory=list)
     error: str | None = None  # the model couldn't run at all
@@ -219,6 +221,14 @@ def generation_tps(self) -> float | None:
     def tool_errors(self) -> int:
         return sum(t.tool_errors for t in self.tasks)
 
+    def note_memory(self, where: residency.Placement) -> None:
+        """Ollama's count keeps the names it always had, so results compare with older ones. The estimate that the line
+        after the load shows goes next to it."""
+        self.memory_gb, self.gpu_percent = round(where.size / 1e9, 1), round(100 * where.vram / where.size)
+        if where.estimate > where.size:
+            self.estimated_memory_gib = round(where.estimate / GIB, 1)
+            self.estimated_gpu_percent = round(100 * where.gpu_share)
+
     def to_json(self) -> dict:
         data = asdict(self)
         rates = self.rates()
@@ -410,13 +420,11 @@ def _read_prompt(ollama: Ollama, settings: Settings, options: dict) -> float | N
     return round(count / (ns / 1e9), 1) if ns and count >= 1000 else None
 
 
-def memory_use(ollama: Ollama, model: str) -> tuple[float | None, int | None]:
-    for entry in ollama.running():
-        if entry.get("name") in (model, f"{model}:latest") or entry.get("model") in (model, f"{model}:latest"):
-            size, vram = entry.get("size") or 0, entry.get("size_vram") or 0
-            if size:
-                return round(size / 1e9, 1), round(100 * vram / size)
-    return None, None
+def memory_use(ollama: Ollama, model: str, context: int) -> residency.Placement | None:
+    """How the loaded model sits in memory, counted as the line after a load counts it: Ollama 0.40 leaves out the
+    part of a Mixture-of-Experts model that stays mapped from its file in RAM (see residency.with_estimate)."""
+    where = residency.placement(ollama, model)  # Ollama lists lcode-qwen3.6-35b as lcode-qwen3.6-35b:latest
+    return residency.with_estimate(where, model, context) if where and where.size else None
 
 
 def run_model(
@@ -455,14 +463,15 @@ def run_model(
         run.interrupted = True
         return run
     run.load_seconds = round(time.monotonic() - started, 1)
-    run.memory_gb, run.gpu_percent = memory_use(ollama, settings.model)
+    if placed := memory_use(ollama, settings.model, settings.context):
+        run.note_memory(placed)
     try:
         with console.status(Elapsed("  Measuring prompt speed")):
             run.prompt_tps = prompt_speed(ollama, settings, options)
     except KeyboardInterrupt:
         run.interrupted = True
         return run
-    where = f" · {run.memory_gb} GB, {run.gpu_percent}% on the GPU" if run.memory_gb else ""
+    where = f" · {placed.describe()}" if placed else ""
     reads = f" · reads prompts at {run.prompt_tps:,.0f} tok/s" if run.prompt_tps else ""
     if getattr(ollama, "kind", "ollama") == "ollama":
         console.print(f"  [dim]Loaded in {run.load_seconds:.0f}s{where}{reads}[/]")
@@ -503,8 +512,8 @@ def run_model(
                 run.pruned, run.compacted = pruned + shared.pruned, compacted + shared.compacted
         if run.interrupted:
             break
-    if run.memory_gb is None:
-        run.memory_gb, run.gpu_percent = memory_use(ollama, settings.model)
+    if run.memory_gb is None and (placed := memory_use(ollama, settings.model, settings.context)):
+        run.note_memory(placed)
     # lcode lowers the batch size or context if the GPU runs out of memory; report what was used.
     run.context, run.num_batch, run.think = settings.context, settings.num_batch, settings.think
     return run
@@ -521,6 +530,13 @@ def percent(share: float | None) -> str:
     return "–" if share is None else f"{share * 100:.0f}%"
 
 
+def memory(run: ModelRun) -> str:
+    """'~23 GB, 42% on GPU' by lcode's estimate, as the line after the load counts it, or else what Ollama says."""
+    if run.estimated_memory_gib:
+        return f"~{run.estimated_memory_gib:.0f} GB, {run.estimated_gpu_percent}% on GPU"
+    return f"{run.memory_gb} GB, {run.gpu_percent}% on GPU" if run.memory_gb else "?"
+
+
 def summary_rows(runs: list[ModelRun], tasks: list[Task]) -> list[tuple[str, list[str]]]:
     rows = []
     for task in {t.id: t for t in tasks}.values():  # a task that ran several times: one row, a mark each
@@ -554,10 +570,7 @@ def cell(run: ModelRun, value: str) -> str:
         ("Generation", [cell(r, f"{r.generation_tps:.1f} tok/s" if r.generation_tps else "?") for r in runs]),
         ("Prompt reading", [cell(r, f"{r.prompt_tps:,.0f} tok/s" if r.prompt_tps else "?") for r in runs]),
         ("Tool-call errors", [cell(r, str(r.tool_errors)) for r in runs]),
-        (
-            "Memory",
-            [cell(r, f"{r.memory_gb} GB, {r.gpu_percent}% on GPU" if r.memory_gb else "?") for r in runs],
-        ),
+        ("Memory", [cell(r, memory(r)) for r in runs]),
         ("Context", [cell(r, format_tokens(r.context)) for r in runs]),
     ]
     if any(r.session for r in runs):

src/lcode/doctor.py

@@ -110,15 +110,9 @@ def line(label: str, value: str, good: bool | None = True) -> None:
                 f"here (delete {limits.path()} to try again)",
                 None,
             )
-        loaded = [m for m in ollama.running() if m.get("name") == model or m.get("model") == model]
-        if loaded:
-            m = loaded[0]
-            line(
-                "Loaded",
-                f"{m.get('size_vram', 0) / 1e9:.1f} GB on GPU of {m.get('size', 0) / 1e9:.1f} GB, "
-                f"context {m.get('context_length', '?')}",
-                True,
-            )
+        if where := residency.placement(ollama, model):  # Ollama lists lcode-qwen3.6-35b as lcode-qwen3.6-35b:latest
+            where = residency.with_estimate(where, model, ctx)
+            line("Loaded", where.describe() + (f" · {format_tokens(where.context)} context" if where.context else ""))
     except cli.NotInstalled as e:
         line("Model", str(e), False)
         ok = False