fix(mcp): harden semantic search inputs and store-error handling - #2067
Merged
Conversation
Land the four residual hardening points from #938 on the current semantic-search design (semantic_limit/offset, the certified top-K fetch loop): - extract_semantic_keywords rejects a semantic_query array that holds a non-string element instead of silently running the string subset; the caller gets the same isError as for a bare-string semantic_query. - run_semantic_query_core fails closed when the vector scan itself fails ("semantic search failed", isError) instead of rendering a store error as zero matches plus the broaden-your-keywords hint. cbm_store_vector_search now tells the two states apart: a store with no node_vectors table (lean index) is CBM_STORE_NOT_FOUND and keeps the empty-page + moderate/full-index hint that main relied on by swallowing the "no such table" prepare failure; a failing scan is CBM_STORE_ERR. - vs_min_cosine_score hoists the keyword norms (once per search) and the node norm (once per node) out of the per-(node, keyword) loop. The integer sums are exact, so every score stays bit-identical. - vs_append_result checks its strdup results, frees the partial row on failure and counts a row only once every copy succeeded, so the caller's free path stays valid. Two reproduce-first tests in the mcp suite fail on main and pass with the change: non-string elements produce the type error, and a forced cbm_cosine_i8 failure produces the fail-closed error for semantic-only and combined calls while a missing node_vectors table still yields the lean-index hint. Refs #915 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> Co-authored-by: Dustin Persek <dustin.persek@protonmail.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Distilled from #938 with co-author credit to @dpersek — the four residual hardening points that were still open after the semantic-search work landed on main, adapted to the current design (
semantic_limit/semantic_offset,sg_semantic_page_t, the certified top-K fetch loop). Refs #915.extract_semantic_keywords(src/mcp/mcp.c) — asemantic_queryarray containing a non-string element (["publish", 42],[null],[["publish"]]) was silently narrowed to its string members, so a malformed call ran as a different query and the caller never learned. It now returns-1andsearch_graphanswers with the sameisErrorresponse as a bare-stringsemantic_query(message extended to say every element must be a string).run_semantic_query_corefails closed on a store error — a vector scan that FAILED was rendered as "0 semantic matches" plus the "use a moderate/full index or broader keywords" hint, sending the caller off to broaden keywords against a broken index.cbm_store_vector_searchnow distinguishes the two states main conflated:node_vectorstable at all (lean index) →CBM_STORE_NOT_FOUND→ unchanged behaviour: empty page + the moderate/full-index hint. On main this case only worked because theno such tableprepare failure was swallowed as zero matches; asqlite_masterprobe makes it explicit.CBM_STORE_ERR→isError"semantic search failed …", for semantic-only and combined (structural + semantic) calls alike — no structural page pretends the semantic half found nothing.vs_min_cosine_score(src/store/store.c) — the keyword L2 norms and the node norm were recomputed inside the per-(node, keyword) loop. Keyword norms are now computed once per search (vs_keyword_norms), the node norm once per node. Math-identical: the integer sums are exact, sosqrt((double)ma) * sqrt((double)mb)yields the same doubles whether or not it is factored out of the pair loop.vs_append_result—strdupresults were unchecked. It now returnsCBM_STORE_OK/CBM_STORE_ERR, frees the partial row on failure, and only counts a row once every copy succeeded, so the caller's existing free path stays valid.Left out of #938 (already covered or contradicted by main's current design): its three
run_semantic_query_corecall sites (main has one), the fixed 250-candidate window (main certifies top-K via the fetch-expansion loop solimitcannot change ranking), and the rowmemset(a failed row is never counted, so it is never read).Verification
tool_search_graph_semantic_query_rejects_non_string_elementsandtool_search_graph_semantic_store_error_fails_closed(mcp suite) were built against main'ssrc/with only the tests applied — exactly those two fail (382 passed, 2 failed, 4 skipped, both at theisErrorassertion). With the change:mcp mcp_mutation_guard store_search cli→714 passed, 0 failed, 4 skipped(the four are pre-existing Windows-platform skips).make -f Makefile.cbm lint-cigreen (cppcheck, clang-format, NOLINT check);scripts/check-no-test-skips.shOK; no rawfopen(in the diff.Checklist
git commit -s) — required, CI rejectsunsigned commits (DCO, see CONTRIBUTING.md)
make -f Makefile.cbm test) — the four suites exercising semantic search; full 3-OS ladder is left to CImake -f Makefile.cbm lint-ci)