feat(verdict): keep the verdict per product, and judge over HTTP too

CHANGELOG.md

@@ -26,6 +26,12 @@ what has already shipped, and for nothing else. See `RELEASING.md`.
   split, one what an update closes, and the NVD caveat is always printed. Entries without a
   version are scanned by name, and the scan says which. `list` shows version and CPE. The
   verdict is not stored yet (§5.5): `status`, the API and the web UI are the next step.
+- **The verdict is kept, and the API judges too** (docs/DESIGN.md §5.5). In `cve_software`, per
+  product, with the installed version it was given for: after a new `--version` the old verdict
+  is not shown until a scan judges the new one. `patchradar status` and `/api/cves` show it;
+  `POST /api/watchlist/{name}` takes `version` and `cpe` with the CLI's checks, `GET
+  /api/watchlist` adds `entries`, and `POST /api/scan` judges with the CLI's function
+  (`affected.mark`) and returns the split per product in `verdicts`.
 - **docs/DESIGN.md.** The code referred to "the versionradar design, §5" for these columns; the
   text was never committed and could not be found. It is reconstructed from what `affected.py`,
   `inventory.py` and `tests/test_priority_wiring.py` already committed to.

README.md

@@ -98,8 +98,10 @@ affected ones — the rest stay open because NVD does not state where they were
 fixed, not because the update falls short — and a count from NVD is always
 followed by the reminder that it counts identifiers, not flaws. Entries without
 a version are scanned by name as before, and the scan names them. The verdict
-is not stored yet, so `status`, the API and the web UI still show CVEs by name
-([docs/DESIGN.md](docs/DESIGN.md) §5.5).
+is stored with the version it was given for: `patchradar status` and
+`/api/cves` show it while that version is still the installed one, and the API
+takes the version and CPE too (`POST /api/watchlist/<name>?version=…&cpe=…`).
+The web UI does not show it yet ([docs/DESIGN.md](docs/DESIGN.md) §5.5, §5.6).
 
 ### Asking Debian what it is actually doing
 

docs/DESIGN.md

@@ -104,16 +104,25 @@ The total line keeps its meaning: CVEs found by name.
 
 ### 5.5 What is stored
 
-PR-01a judges in memory, during the scan. Storing the verdict — so that
-`status`, the API and the web UI can show it — needs columns in `cves` and is the
-migration `tests/test_priority_wiring.py` waits for, together with the KEV fields.
+The verdict is kept on the link between a CVE and a product, `cve_software`
+(`verdict`, `verdict_reason`, `verdict_fixed_version`, `judged_version`), because
+one CVE stored once can be affected for one watched product and not for another.
+The latest scan decides it. It is read back only while the installed version is
+still `judged_version`: after `add --version` with a new version, the old verdict
+is not shown until a scan judges the new one. `status` and `/api/cves` show it;
+the API records version and CPE (`POST /api/watchlist/{name}?version=&cpe=`) and
+judges with the same function as the CLI (`affected.mark`).
+
+Still in memory only: the KEV fields that decide a rank — the migration
+`tests/test_priority_wiring.py` waits for.
 
 ### 5.6 Order of work
 
 1. **PR-01a** — the columns, `add --version --cpe`, the verdict for NVD records
    in the command-line scan, `--all`, the README.
-2. **PR-01b** — KEV and MSRC entries judged through NVD's record of the same
-   CVE; the verdict and the KEV fields stored; the API.
+2. **PR-01b** — in two parts: the verdict stored and the API (01b-1); the KEV
+   fields stored, with the database-backed listings ranked by priority, and KEV
+   and MSRC entries judged through NVD's record of the same CVE (01b-2).
 3. **PR-01c** — the web UI; `patchradar import` writing the mappings a person
    confirmed, `channel` included.
 

src/patchradar/affected.py

@@ -26,11 +26,11 @@
 ever assigned an identifier. `Summary.caveat` carries that sentence so a caller
 cannot print "no known vulnerabilities" without it.
 
-**Wired into the command-line scan** for a watchlist entry that has both an
-installed version and a CPE (docs/DESIGN.md §5): `judge` gives each CVE found by
-name one of three verdicts — affected, not affected, not evaluable — and
-`closed_by` says what one update closes. Storing the verdict, and the API, are
-the next step (§5.5); `tests/test_priority_wiring.py` holds that boundary.
+**Wired into both scans**, the CLI's and the API's, for a watchlist entry that
+has both an installed version and a CPE (docs/DESIGN.md §5): `judge` gives each
+CVE found by name one of three verdicts — affected, not affected, not evaluable
+— `mark` applies it to a scan's records, and `closed_by` says what one update
+closes. The verdict is stored with the version it was given for (§5.5).
 """
 from __future__ import annotations
 
@@ -451,3 +451,21 @@ def judge(record: dict, *, version: str, vendor: str, cpe_product: str) -> Verdi
                                       f"compared with {version}")
     return Verdict(NOT_AFFECTED, f"NVD states the affected versions ({stated}); "
                                  f"{version} is not among them")
+
+
+def mark(records: list[dict], installed: tuple[str, str, str] | None) -> None:
+    """Give each scan record its verdict for `installed` — (version, vendor,
+    product), or None for an entry matched by name — and take NVD's
+    configurations off it, which nothing after the verdict reads.
+
+    The one place the CLI and the API judge, so the two cannot disagree. The
+    record keeps the version it was judged for, which is what lets a stored
+    verdict be told apart from one about a version no longer installed.
+    """
+    for record in records:
+        if installed is not None:
+            version, vendor, product = installed
+            verdict = judge(record, version=version, vendor=vendor, cpe_product=product)
+            record.update(verdict=verdict.status, verdict_reason=verdict.reason,
+                          verdict_fixed_version=verdict.fixed_version, judged_version=version)
+        record.pop("configurations", None)

src/patchradar/api/main.py

@@ -15,14 +15,25 @@
 from fastapi.security import APIKeyHeader
 from fastapi.staticfiles import StaticFiles
 
+from patchradar.affected import AFFECTED, NOT_AFFECTED, NOT_EVALUABLE, mark, orderable
 from patchradar.collectors.debian import fetch_cves as debian_fetch
 from patchradar.collectors.errors import CollectorError
 from patchradar.collectors.kev import fetch_cves as kev_fetch
 from patchradar.collectors.msrc import fetch_cves as msrc_fetch
 from patchradar.collectors.nvd import fetch_cves as nvd_fetch
 from patchradar.collectors.vuldb import api_key as vuldb_api_key
 from patchradar.collectors.vuldb import fetch_cves as vuldb_fetch
-from patchradar.db.database import add_to_watchlist, get_cves, get_watchlist, init_db, remove_from_watchlist
+from patchradar.cpe import CpeError, parse_cpe
+from patchradar.db.database import (
+    WatchEntry,
+    add_to_watchlist,
+    get_cves,
+    get_watch_entries,
+    get_watchlist,
+    init_db,
+    remove_from_watchlist,
+    set_installed,
+)
 from patchradar.epss import enrich as epss_enrich
 from patchradar.names import SOFTWARE_NAME_MAX_LENGTH, SOFTWARE_NAME_PATTERN, normalise_software_name
 from patchradar.priority import merge_by_cve
@@ -194,15 +205,23 @@ def sanitize_cve(cve: dict) -> dict:
     # Numeric fields. The EPSS pair is passed through as stored, including a
     # missing one: `.get` yields None, and None here means FIRST does not score
     # the CVE — which a 0.0 would misreport as "almost certainly not exploited".
+    for field in ("verdict", "verdict_reason", "verdict_fixed_version", "judged_version"):
+        value = cve.get(field)
+        safe[field] = _strip_control_chars(value).strip()[:300] if isinstance(value, str) else None
     safe["cvss_score"] = cve.get("cvss_score")
     safe["epss_score"] = cve.get("epss_score")
     safe["epss_percentile"] = cve.get("epss_percentile")
     return safe
 
 @app.get("/api/watchlist")
 async def api_watchlist():
-    items = await get_watchlist()
-    return {"watchlist": items}
+    # `watchlist` keeps its shape for what already reads it; `entries` adds
+    # what is installed (docs/DESIGN.md §5.1). One query for both: two made the
+    # endpoint 24% slower (CodSpeed, #59).
+    entries = await get_watch_entries()
+    return {"watchlist": [e.name for e in entries],
+            "entries": [{"name": e.name, "installed_version": e.installed_version, "cpe": e.cpe}
+                        for e in entries]}
 
 @app.post(
     "/api/watchlist/import",
@@ -243,14 +262,29 @@ async def api_watchlist_import(payload: dict):
 async def api_add(
     software: str = Path(
         ..., min_length=1, max_length=SOFTWARE_NAME_MAX_LENGTH, pattern=SOFTWARE_NAME_PATTERN
-    )
+    ),
+    version: str | None = Query(None, min_length=1, max_length=64),
+    cpe: str | None = Query(None, min_length=1, max_length=256),
 ):
     # Normalise here too, so both routes store the same canonical value.
     name = normalise_software_name(software)
     if name is None:
         raise HTTPException(status_code=422, detail="Invalid software name")
+    # The same checks as `patchradar add`, before anything is written.
+    if version is not None and not orderable(version):
+        raise HTTPException(status_code=422, detail=f"{version} is not a version the scan can compare")
+    parsed = None
+    if cpe is not None:
+        try:
+            parsed = parse_cpe(cpe)
+        except CpeError as exc:
+            raise HTTPException(status_code=422, detail=str(exc)) from None
     added = await add_to_watchlist(name)
-    return {"added": added, "software": name}
+    if version is not None or parsed is not None:
+        await set_installed(name, version=version, cpe=parsed)
+    entry = next(e for e in await get_watch_entries() if e.name == name)
+    return {"added": added, "software": name,
+            "installed_version": entry.installed_version, "cpe": entry.cpe}
 
 
 @app.delete(
@@ -272,7 +306,8 @@ async def api_cves(software: str = Query(None, min_length=1, max_length=100), li
     sanitized = [sanitize_cve(c) for c in cves]
     return {"cves": sanitized, "total": len(sanitized)}
 
-async def _scan_one(sw: str, days: int, errors: list[dict]) -> int:
+async def _scan_one(sw: str, days: int, errors: list[dict], entry: WatchEntry | None = None,
+                    verdicts: dict | None = None) -> int:
     """Collect and persist every source for a single package.
 
     A source that fails is recorded in `errors` rather than silently counted
@@ -321,6 +356,13 @@ async def _scan_one(sw: str, days: int, errors: list[dict]) -> int:
         logger.warning("scan of %r unranked by EPSS: %s", sw, exc)
         errors.append({"software": sw, "source": exc.source,
                        "reason": exc.reason, "status": exc.status})
+    # Judged by the function the CLI judges with, before anything is stored, so
+    # the verdict lands with the link and the two scans give one answer.
+    installed = entry.installed() if entry is not None else None
+    mark(collected, installed)
+    if installed is not None and verdicts is not None:
+        found = [cve["verdict"] for cve in collected]
+        verdicts[sw] = {status: found.count(status) for status in (AFFECTED, NOT_EVALUABLE, NOT_AFFECTED)}
     for cve in collected:
         await save_cve(cve)
     return len(collected)
@@ -334,6 +376,8 @@ async def _scan_one(sw: str, days: int, errors: list[dict]) -> int:
 )
 async def api_scan(days: int = Query(7, ge=1, le=90)):
     watchlist = await get_watchlist()
+    entries = {entry.name: entry for entry in await get_watch_entries()}
+    verdicts: dict[str, dict[str, int]] = {}
     total = 0
     results = {}
     errors: list[dict] = []
@@ -343,7 +387,7 @@ async def api_scan(days: int = Query(7, ge=1, le=90)):
         # scan that never returns pins a worker indefinitely.
         async with asyncio.timeout(SCAN_TIMEOUT_SECONDS):
             for sw in watchlist:
-                count = await _scan_one(sw, days, errors)
+                count = await _scan_one(sw, days, errors, entries.get(sw), verdicts)
                 results[sw] = count
                 total += count
     except TimeoutError:
@@ -357,6 +401,9 @@ async def api_scan(days: int = Query(7, ge=1, le=90)):
     return {
         "total": total,
         "by_software": results,
+        # Per product judged against its installed version: how many of the
+        # CVEs found by name affect it, could not be evaluated, do not affect it.
+        "verdicts": verdicts,
         "timed_out": timed_out,
         "scanned": len(results),
         "watched": len(watchlist),

src/patchradar/cli.py

@@ -14,7 +14,7 @@
 from rich.text import Text
 
 import patchradar
-from patchradar.affected import AFFECTED, CAVEAT, NOT_AFFECTED, NOT_EVALUABLE, closed_by, judge, orderable
+from patchradar.affected import AFFECTED, CAVEAT, NOT_AFFECTED, NOT_EVALUABLE, closed_by, mark, orderable
 from patchradar.collectors.debian import DEBIAN_RELEASE as debian_release
 from patchradar.collectors.debian import get_tracker_snapshot
 from patchradar.collectors.errors import CollectorError
@@ -419,15 +419,7 @@ async def _scan_target(
     # listed, unless --all asks for them.
     installed = entry.installed() if entry is not None else None
     judged = installed is not None
-    if installed is not None:
-        version, vendor, product = installed
-        for cve in all_cves:
-            verdict = judge(cve, version=version, vendor=vendor, cpe_product=product)
-            cve["verdict"] = verdict.status
-            cve["verdict_reason"] = verdict.reason
-            cve["verdict_fixed_version"] = verdict.fixed_version
-    for cve in all_cves:
-        cve.pop("configurations", None)
+    mark(all_cves, installed)
     for cve in all_cves:
         await save_cve(cve)
     if collected is not None:
@@ -637,18 +629,26 @@ def _print_cves_table(cves: list):
     table.add_column("Score", justify="center", width=6)
     table.add_column("Severity", justify="center", width=10)
     table.add_column("Source", width=6)
+    # Only when a stored verdict is current for some row: a watchlist matched by
+    # name alone gets the table it always had.
+    judged = any(cve.get("verdict") for cve in cves)
+    if judged:
+        table.add_column("Verdict")
 
     for cve in cves:
         score_str = _format_score(cve.get("cvss_score"))
         severity = _normalise_severity(cve.get("severity"))
         severity_color = SEVERITY_STYLES.get(severity, "white")
+        verdict = cve.get("verdict")
+        extra = [Text(verdict or "—", style=VERDICT_STYLES.get(verdict, "dim"))] if judged else []
 
         table.add_row(
             Text(str(cve.get("id", ""))),
             Text(str(cve.get("software") or "")),
             score_str,
             Text(severity, style=severity_color),
             Text(str(cve.get("source") or "")),
+            *extra,
         )
     console.print(table)