Count the part of a loaded model in RAM, and show it in lcode doctor

CHANGELOG.md

@@ -6,6 +6,18 @@ All notable changes to lcode are documented here. The format follows
 
 ## [Unreleased]
 
+### Fixed
+
+- **The line after a load counts the part of the model in system RAM.** With Ollama 0.40, `qwen3.6-35b`
+  at 32K on a 12 GB GPU showed as `10.5 GB, 98% on the GPU and the rest in system RAM`, because Ollama
+  leaves out the ~12 GB of experts that stay mapped from the model's file. For the models in the catalog,
+  lcode now counts the whole model at its estimate, the one `lcode setup` and `lcode doctor` show:
+  `~23 GB, 42% on the GPU and the rest in system RAM`. With Ollama's numbers, the warning that another
+  program is using GPU memory couldn't fire. Now it does, and it says how much of the model fits on the GPU.
+- **`lcode doctor` shows the loaded model.** It looked for `lcode-qwen3.6-35b` while Ollama lists it as
+  `lcode-qwen3.6-35b:latest`, so the `Loaded` line never appeared for the models `lcode setup` creates.
+  It now says how the model fits, in the same words as the line after a load.
+
 ## [0.62.0] - 2026-11-06
 
 ### Changed

docs/models.md

@@ -258,7 +258,8 @@ RAM needed ≈ memory − GPU memory + ~10 GB for Ollama's server and prompt cac
 
 - **The part of the model that doesn't fit on the GPU.** A Mixture-of-Experts model's experts stay
   mapped from the model's file, so they have to stay in Linux's page cache. `ollama ps` leaves them out:
-  it shows `qwen3.6-35b` at 32K as 10.5 GB, with ~12 GB more in RAM.
+  it shows `qwen3.6-35b` at 32K as 10.5 GB, with ~12 GB more in RAM. lcode counts them: after a load,
+  and in `lcode doctor`, it says `~23 GB, 42% on the GPU and the rest in system RAM`.
 - **Ollama's prompt cache**, which grows to 8 GB in a long session, next to 1–2 GB of the server's own.
 
 On a 12 GB GPU, `qwen3.6-35b` needs ~21 GB of RAM at 32K, ~23 GB at 128K and ~26 GB at 256K. On Linux,

docs/troubleshooting.md

@@ -84,8 +84,10 @@ need `lcode mcp login <name>`. More in [MCP servers: Troubleshooting](mcp.md#tro
   starts, and `keep_alive` keeps it loaded between requests.
 - Reading many large files takes a while the first time (~280 tokens/s on a 12 GB GPU, ~500 with
   `lcode config set num_batch 1024` if it fits); follow-up turns reuse the cache.
-- `ollama ps` shows how much of the model is on the GPU. Dense models are much slower when split;
-  prefer the MoE models in `lcode models` on smaller GPUs.
+- `lcode doctor` shows how much of the loaded model is on the GPU, for example
+  `Loaded ~23 GB, 42% on the GPU and the rest in system RAM · 32K context`. `ollama ps` leaves out the
+  part of a Mixture-of-Experts model that's in system RAM, so it can say 98%. Dense models are much
+  slower when split; prefer the MoE models in `lcode models` on smaller GPUs.
 - `--no-think` skips reasoning for simple requests.
 
 ### It starts fast, then gets about three times slower

docs/usage.md

@@ -409,7 +409,7 @@ stays loaded:
 
 | When | What it says |
 |---|---|
-| The model was loaded | `◆ Loaded qwen3.6-35b in 15s: 22.3 GB, 45% on the GPU and the rest in system RAM.`, and a warning when less of it is on the GPU than usual, because another program or model is using GPU memory |
+| The model was loaded | `◆ Loaded qwen3.6-35b in 15s: ~23 GB, 42% on the GPU and the rest in system RAM.`, and a warning when less of it is on the GPU than usual, because another program or model is using GPU memory |
 | A dense model doesn't fit on the GPU (at startup and on `/model`) | `⚠ qwen3.8-27b is a dense model: all 27B parameters work on every token. It's capable, but slow here. Only about 61% of it fits in the 12 GB of GPU memory…`, with what to do instead |
 | The model that looks at images needs the memory | `Loading the vision model … to look at bug.png; qwen3.6-35b makes room and reloads afterwards`, then `◆ qwen3.6-35b was unloaded to make room…`, and the next step says `Reloading qwen3.6-35b into memory (it made room for the vision model …)` |
 | A tool needs the GPU to itself (an MCP server with `free_gpu`) | `◆ Unloaded qwen3.6-35b so the server can use the GPU; it reloads afterwards.` |
@@ -421,6 +421,12 @@ While the model loads in the background at startup, the status bar says
 part of them in system RAM. Dense ones slow down a lot, to a few tokens a second, which is why they
 get the warning.
 
+The `~` marks lcode's estimate of the model's memory, the same one `lcode setup` and `lcode doctor`
+show. Ollama leaves out what stays mapped from the model's file in system RAM: `ollama ps` shows
+qwen3.6-35b at 32K on a 12 GB GPU as 10.5 GB, 98% of it on the GPU, while ~12 GB of its experts are
+in RAM. So when part of a model in `lcode models` is off the GPU, lcode counts all of it. For other
+models it repeats what Ollama says. `lcode doctor` shows the same line for the model in memory.
+
 ## Steer while it works
 
 A request can take minutes with a local model. You don't have to stop it to change course: type

src/lcode/doctor.py

@@ -110,15 +110,9 @@ def line(label: str, value: str, good: bool | None = True) -> None:
                 f"here (delete {limits.path()} to try again)",
                 None,
             )
-        loaded = [m for m in ollama.running() if m.get("name") == model or m.get("model") == model]
-        if loaded:
-            m = loaded[0]
-            line(
-                "Loaded",
-                f"{m.get('size_vram', 0) / 1e9:.1f} GB on GPU of {m.get('size', 0) / 1e9:.1f} GB, "
-                f"context {m.get('context_length', '?')}",
-                True,
-            )
+        if where := residency.placement(ollama, model):  # Ollama lists lcode-qwen3.6-35b as lcode-qwen3.6-35b:latest
+            where = residency.with_estimate(where, model, ctx)
+            line("Loaded", where.describe() + (f" · {format_tokens(where.context)} context" if where.context else ""))
     except cli.NotInstalled as e:
         line("Model", str(e), False)
         ok = False

src/lcode/residency.py

@@ -12,17 +12,18 @@
 
 import threading
 import time
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
 from statistics import median
 from urllib.parse import urlsplit
 
 from lcode import catalog
 from lcode.cloud import is_cloud
 from lcode.config import format_tokens
-from lcode.hardware import Hardware
+from lcode.hardware import GIB, Hardware
 from lcode.ollama import OllamaError
 
 GB = 1e9
+ALL = 0.995  # this share of a model on the GPU counts as all of it, and 1 - ALL as none of it
 RELEASE_WAIT = 15.0  # seconds to wait for Ollama to give an unloaded model's GPU memory back
 SPLIT_WARNING = 0.95  # a dense model with less than this on the GPU runs much slower
 SHORTFALL = 0.25  # this much less on the GPU than expected: something else is using GPU memory
@@ -36,24 +37,30 @@
 
 @dataclass
 class Placement:
-    """A loaded model, from Ollama's /api/ps."""
+    """A loaded model, from Ollama's /api/ps (see `with_estimate` for what that leaves out)."""
 
     name: str
-    size: float  # bytes in memory
+    size: float  # bytes in memory, as Ollama counts them
     vram: float  # bytes of it on the GPU
     context: int = 0  # the context window it was loaded with
+    estimate: float = 0.0  # bytes the model takes by the catalog's estimate, when Ollama's count leaves part out
+
+    @property
+    def total(self) -> float:
+        return max(self.size, self.estimate)
 
     @property
     def gpu_share(self) -> float:
-        return self.vram / self.size if self.size else 0.0
+        return self.vram / self.total if self.total else 0.0
 
     def describe(self) -> str:
-        """'22.3 GB, 44% on the GPU and the rest in system RAM'"""
-        size = f"{self.size / GB:.1f} GB"
+        """'6.6 GB, all on the GPU', or with the catalog's estimate: '~23 GB, 42% on the GPU and the rest in system
+        RAM'. The estimate is shown as `lcode setup` and `lcode doctor` show it."""
+        size = f"~{self.estimate / GIB:.0f} GB" if self.estimate > self.size else f"{self.size / GB:.1f} GB"
         share = self.gpu_share
-        if share >= 0.995:
+        if share >= ALL:
             return f"{size}, all on the GPU"
-        if share <= 0.005:
+        if share <= 1 - ALL:
             return f"{size}, all in system RAM (on the CPU)"
         return f"{size}, {share:.0%} on the GPU and the rest in system RAM"
 
@@ -117,6 +124,20 @@ def placement(ollama, model: str) -> Placement | None:
     return found[0] if found else None
 
 
+def with_estimate(where: Placement, model: str, context: int) -> Placement:
+    """`where`, counting the whole model when part of it is off the GPU: by the catalog's estimate (models outside
+    the catalog stay as Ollama counts them).
+
+    /api/ps counts the memory Ollama's runner allocates. The weights it keeps in system RAM aren't allocated: they
+    stay mapped from the model's file. So with Ollama 0.40, qwen3.6-35b at 32K on a 12 GB GPU shows as 10.5 GB, 98%
+    of it on the GPU, while ~12 GB of its experts are in RAM. A model all on the GPU is counted in full.
+    """
+    spec = catalog.find(model)
+    if spec is None or where.vram >= ALL * where.size:
+        return where
+    return replace(where, estimate=spec.memory_gib(where.context or context) * GIB)
+
+
 def is_moe(ollama, model: str) -> bool | None:
     """Whether the model is a Mixture-of-Experts (None: unknown)."""
     spec = catalog.find(model)
@@ -169,14 +190,17 @@ def load_report(
 ) -> str:
     """The line after a load: how long it took, how it fits, and a warning if it fits worse than it should."""
     took = f" in {seconds:.0f}s" if seconds >= 1 else ""
-    line = f"{'Reloaded' if reload else 'Loaded'} {model}{took}" + (f": {where.describe()}." if where else ".")
+    line = f"{'Reloaded' if reload else 'Loaded'} {model}{took}"
     if where is None:
-        return line
+        return f"{line}."
+    where = with_estimate(where, model, context)
+    line += f": {where.describe()}."
     expected = expected_share(model, context, hardware)
     if expected is not None and where.gpu_share < expected - SHORTFALL:
+        fits = "all of it fits" if expected >= ALL else f"up to {expected:.0%} fits"
         line += (
-            f" Less of it is on the GPU than usual ({expected:.0%}): another program or model is using GPU "
-            "memory, so it runs slower until that's closed."
+            f" Less of it is on the GPU than usual ({fits}): another program or model is using GPU memory, so it "
+            "runs slower until that's closed."
         )
     elif dense and where.gpu_share < SPLIT_WARNING:
         line += " It's a dense model split between GPU and CPU, so expect it to be slow."

tests/test_residency.py

@@ -21,6 +21,11 @@ def entry(name, size_gb, gpu_gb, context=0):
     return {"name": name, "size": size_gb * 1e9, "size_vram": gpu_gb * 1e9, "context_length": context}
 
 
+# What Ollama 0.40.13 lists for lcode-qwen3.6-35b at 32K on the laptop: the ~12 GB of experts that stay mapped from
+# the model's file in RAM aren't in its size, so 98% of it seems to be on the GPU.
+OLLAMA_0_40 = {**entry("lcode-qwen3.6-35b:latest", 10.49, 10.29, 32768), "model": "lcode-qwen3.6-35b:latest"}
+
+
 def text(agent):
     return " ".join(output(agent).split())
 
@@ -31,6 +36,24 @@ def test_where_a_model_sits():
     assert Placement("m", 6e9, 0).describe() == "6.0 GB, all in system RAM (on the CPU)"
 
 
+def estimated(model, size_gb, gpu_gb, context=32768, fallback=32768):
+    return residency.with_estimate(Placement(model, size_gb * 1e9, gpu_gb * 1e9, context), model, fallback).describe()
+
+
+def test_the_part_of_a_catalog_model_ollama_leaves_out_is_counted():
+    assert estimated("lcode-qwen3.6-35b", 10.49, 10.29) == "~23 GB, 42% on the GPU and the rest in system RAM"
+    assert f"~{catalog.find('qwen3.6-35b').memory_gib(32768):.0f} GB" == "~23 GB"  # as lcode setup and doctor say
+    assert estimated("qwen3.6-35b", 10.49, 10.29, context=262144) == "~28 GB, 35% on the GPU and the rest in system RAM"
+    assert estimated("qwen3.6-35b", 10.49, 10.29, context=0) == "~23 GB, 42% on the GPU and the rest in system RAM"
+    assert estimated("qwen3.5-4b", 1.2, 0) == "~5 GB, all in system RAM (on the CPU)"  # no GPU: all of it is in RAM
+
+
+def test_what_ollama_counts_stands_when_nothing_is_left_out_or_nothing_is_known():
+    assert estimated("qwen3.5-9b", 8.5, 8.5, context=65536) == "8.5 GB, all on the GPU"  # below the estimate
+    assert estimated("qwen3.6-35b", 26, 10) == "26.0 GB, 38% on the GPU and the rest in system RAM"  # above it
+    assert estimated("custom:7b", 10.49, 10.29) == "10.5 GB, 98% on the GPU and the rest in system RAM"  # no catalog
+
+
 def test_dense_models_that_dont_fit_get_a_warning(make_agent):
     ollama = make_agent().ollama
     warning = residency.dense_warning(ollama, "qwen3.8-27b", 32768, LAPTOP)
@@ -54,29 +77,42 @@ def test_models_outside_the_catalog_are_asked_about(make_agent):
 
 
 def test_a_load_is_reported_once_with_how_it_fits(make_agent):
-    agent = make_agent([reply("hi"), reply("again")], model="qwen3.6-35b", context=32768)
+    agent = make_agent([reply("hi"), reply("again")], context=32768)
     agent.hardware = LAPTOP
     agent.ollama.loaded = []
 
     def chat_stream(payload, on_open=None):
         agent.ollama.payloads.append(payload)
-        agent.ollama.loaded = [entry("qwen3.6-35b:latest", 22.3, 9.8, 32768)]
+        agent.ollama.loaded = [OLLAMA_0_40]
         yield from agent.ollama.scripts.pop(0)
 
     agent.ollama.chat_stream = chat_stream
     agent.run_turn("hello")
-    assert "◆ Loaded qwen3.6-35b: 22.3 GB, 44% on the GPU and the rest in system RAM." in text(agent)
+    assert "◆ Loaded lcode-qwen3.6-35b: ~23 GB, 42% on the GPU and the rest in system RAM." in text(agent)
+    assert "⚠" not in output(agent)  # as much is on the GPU as usual
     agent.console.file.truncate(0)
     agent.run_turn("and again")
     assert "◆" not in output(agent) and "⚠" not in output(agent)  # in memory: nothing to say
 
 
-def test_less_on_the_gpu_than_usual_is_explained(make_agent):
-    report = residency.load_report("qwen3.6-35b", 14, Placement("m", 22.3e9, 2e9), 32768, LAPTOP, dense=False)
-    assert report.startswith("Loaded qwen3.6-35b in 14s: 22.3 GB, 9% on the GPU")
-    assert "another program or model is using GPU memory" in report
-    dense = residency.load_report("qwen3.8-27b", 40, Placement("m", 18e9, 11e9), 32768, LAPTOP, dense=True)
-    assert dense.endswith("It's a dense model split between GPU and CPU, so expect it to be slow.")
+def report(model, size_gb, gpu_gb, context=32768, dense=False):
+    placement = Placement("m", size_gb * 1e9, gpu_gb * 1e9, context)
+    return residency.load_report(model, 14, placement, context, LAPTOP, dense=dense)
+
+
+def test_less_on_the_gpu_than_usual_is_explained():
+    # Another program holds GPU memory: Ollama counts 4.2 GB, 95% of it on the GPU, and leaves out the rest in RAM.
+    assert report("qwen3.6-35b", 4.2, 4.0) == (
+        "Loaded qwen3.6-35b in 14s: ~23 GB, 16% on the GPU and the rest in system RAM. Less of it is on the GPU than "
+        "usual (up to 53% fits): another program or model is using GPU memory, so it runs slower until that's closed."
+    )
+    assert report("qwen3.6-35b", 10.49, 10.29) == (
+        "Loaded qwen3.6-35b in 14s: ~23 GB, 42% on the GPU and the rest in system RAM."  # the usual 9-10 GB
+    )
+    assert "(all of it fits)" in report("qwen3.5-9b", 7.7, 7.5, context=131072, dense=True)
+    split = "It's a dense model split between GPU and CPU, so expect it to be slow."
+    assert report("qwen3.8-27b", 18, 11, dense=True).endswith(split)
+    assert report("qwen3.8-27b", 11.2, 11, dense=True).endswith(split)  # with the part in RAM left out, too
 
 
 def test_the_status_line_says_what_comes_before_the_first_token(make_agent):
@@ -390,6 +426,18 @@ def test_doctor_says_nothing_about_ram_it_cant_tell(cli_here, machine):
     assert "RAM     " not in cli_here()
 
 
+def test_doctor_shows_how_the_loaded_model_fits(cli_here, machine, monkeypatch):
+    machine(available=25.0, ollama_private=0.0)
+    ollama = LocalOllama()
+    ollama.loaded = [OLLAMA_0_40]  # as lcode-qwen3.6-35b:latest, while the model resolves to lcode-qwen3.6-35b
+    monkeypatch.setattr(cli, "Ollama", lambda host: ollama)
+    assert run_doctor() == 0
+    assert "✓ Loaded     ~23 GB, 42% on the GPU and the rest in system RAM · 32K context\n" in cli_here()
+    ollama.loaded = [entry("qwen3.5-9b", 6.6, 6.6, 131072)]  # another model
+    assert run_doctor() == 0
+    assert cli_here().count("Loaded") == 1
+
+
 def test_setup_warns_about_ram_on_its_recommendation(cli_here, machine, monkeypatch):
     machine(available=22.0, ollama_private=0.0)
     monkeypatch.setattr(cli.Confirm, "ask", lambda *args, **kwargs: False)