Skip to content

fix(mcp): harden semantic search inputs and store-error handling - #2067

Merged
DeusData merged 1 commit into
mainfrom
fix/semantic-search-residuals-938
Sep 6, 2026
Merged

fix(mcp): harden semantic search inputs and store-error handling#2067
DeusData merged 1 commit into
mainfrom
fix/semantic-search-residuals-938

Conversation

@DeusData

@DeusData DeusData commented Sep 5, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

Distilled from #938 with co-author credit to @dpersek — the four residual hardening points that were still open after the semantic-search work landed on main, adapted to the current design (semantic_limit/semantic_offset, sg_semantic_page_t, the certified top-K fetch loop). Refs #915.

  1. extract_semantic_keywords (src/mcp/mcp.c) — a semantic_query array containing a non-string element (["publish", 42], [null], [["publish"]]) was silently narrowed to its string members, so a malformed call ran as a different query and the caller never learned. It now returns -1 and search_graph answers with the same isError response as a bare-string semantic_query (message extended to say every element must be a string).

  2. run_semantic_query_core fails closed on a store error — a vector scan that FAILED was rendered as "0 semantic matches" plus the "use a moderate/full index or broader keywords" hint, sending the caller off to broaden keywords against a broken index. cbm_store_vector_search now distinguishes the two states main conflated:

    • no node_vectors table at all (lean index) → CBM_STORE_NOT_FOUND → unchanged behaviour: empty page + the moderate/full-index hint. On main this case only worked because the no such table prepare failure was swallowed as zero matches; a sqlite_master probe makes it explicit.
    • the table exists but the scan fails → CBM_STORE_ERRisError "semantic search failed …", for semantic-only and combined (structural + semantic) calls alike — no structural page pretends the semantic half found nothing.
  3. vs_min_cosine_score (src/store/store.c) — the keyword L2 norms and the node norm were recomputed inside the per-(node, keyword) loop. Keyword norms are now computed once per search (vs_keyword_norms), the node norm once per node. Math-identical: the integer sums are exact, so sqrt((double)ma) * sqrt((double)mb) yields the same doubles whether or not it is factored out of the pair loop.

  4. vs_append_resultstrdup results were unchecked. It now returns CBM_STORE_OK/CBM_STORE_ERR, frees the partial row on failure, and only counts a row once every copy succeeded, so the caller's existing free path stays valid.

Left out of #938 (already covered or contradicted by main's current design): its three run_semantic_query_core call sites (main has one), the fixed 250-candidate window (main certifies top-K via the fetch-expansion loop so limit cannot change ranking), and the row memset (a failed row is never counted, so it is never read).

Verification

  • Reproduce-first: tool_search_graph_semantic_query_rejects_non_string_elements and tool_search_graph_semantic_store_error_fails_closed (mcp suite) were built against main's src/ with only the tests applied — exactly those two fail (382 passed, 2 failed, 4 skipped, both at the isError assertion). With the change: mcp mcp_mutation_guard store_search cli714 passed, 0 failed, 4 skipped (the four are pre-existing Windows-platform skips).
  • make -f Makefile.cbm lint-ci green (cppcheck, clang-format, NOLINT check); scripts/check-no-test-skips.sh OK; no raw fopen( in the diff.

Checklist

  • Every commit is signed off (git commit -s) — required, CI rejects
    unsigned commits (DCO, see CONTRIBUTING.md)
  • Tests pass locally (make -f Makefile.cbm test) — the four suites exercising semantic search; full 3-OS ladder is left to CI
  • Lint passes (make -f Makefile.cbm lint-ci)
  • New behavior is covered by a test (reproduce-first for bug fixes)

Land the four residual hardening points from #938 on the current
semantic-search design (semantic_limit/offset, the certified top-K
fetch loop):

- extract_semantic_keywords rejects a semantic_query array that holds a
  non-string element instead of silently running the string subset; the
  caller gets the same isError as for a bare-string semantic_query.
- run_semantic_query_core fails closed when the vector scan itself
  fails ("semantic search failed", isError) instead of rendering a store
  error as zero matches plus the broaden-your-keywords hint.
  cbm_store_vector_search now tells the two states apart: a store with
  no node_vectors table (lean index) is CBM_STORE_NOT_FOUND and keeps
  the empty-page + moderate/full-index hint that main relied on by
  swallowing the "no such table" prepare failure; a failing scan is
  CBM_STORE_ERR.
- vs_min_cosine_score hoists the keyword norms (once per search) and
  the node norm (once per node) out of the per-(node, keyword) loop.
  The integer sums are exact, so every score stays bit-identical.
- vs_append_result checks its strdup results, frees the partial row on
  failure and counts a row only once every copy succeeded, so the
  caller's free path stays valid.

Two reproduce-first tests in the mcp suite fail on main and pass with
the change: non-string elements produce the type error, and a forced
cbm_cosine_i8 failure produces the fail-closed error for semantic-only
and combined calls while a missing node_vectors table still yields the
lean-index hint.

Refs #915

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
@DeusData
DeusData merged commit 7b0f553 into main Sep 6, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant