improvement(knowledge): match keyword chunks before authorizing them

[waleedlatif1]

Problem

The org-scoped keyword retrieval leg built its candidate set in the order visibility → match → rank:

1. visible_keyword_documents — every document in the knowledge base that passes the visibility predicate. No limit, and not restricted to documents the query matched.
2. scored_keyword_candidates — the full-text match plus ts_rank_cd, filtered to those document ids.
3. ranked_keyword_candidates — order by rank, take the page.

That predicate carries a correlated subquery per connector plus a search-integration check, so step 1 is a full pass over the base evaluated before the query terms are consulted. The cost of a search is therefore set by how many documents the base holds, not by how many the query matched — the same search costs about the same whether it matches a few thousand chunks or a few hundred thousand.

Change

Reorder to match → authorize → rank:

1. matched_keyword_chunks — the full-text match, chunk identifiers only.
2. visible_keyword_documents — the identical visibility predicate, restricted to the documents that matched.
3. ranked_keyword_candidates — ts_rank_cd for the matches that survive authorization, then the existing order/limit/offset.

Same predicate, same ordering, same page contract, no schema or write-path change.

Two details keep the reorder from paying the saving back, both measured:

• Restricting with document.id = ANY (ARRAY(...)) rather than IN (SELECT ...) keeps the narrowed lookup on a bitmap scan, which prefetches. The subquery form plans as a nested-loop index walk that does not, and on a cold cache it gives back most of what the smaller candidate set saves.
• The match stage carries identifiers only. Ranking there instead would detoast one text-search vector per match rather than per visible match, which on a mid-frequency term costs more than the pass it replaces.

The EXISTS (SELECT 1 FROM visible_keyword_documents) conjunct is dropped: it sat beside document_id IN (SELECT id FROM visible_keyword_documents), and with a non-null document_id the IN already yields no rows when that set is empty. Verified by running both shapes under an access scope that grants nothing — 0 rows from each.

Measurements

Synthetic corpus shaped like a large search index (a few million chunks over a base of documents behind a mix of workspace, admin-verified and member-observed connectors), Postgres 17, shared_buffers sized to about 8% of the working set. Buffer counts are the hardware-independent figure; wall-clock is warm, single machine, and is direction only.

Paired warm runs, base vs this change:

matching chunks	base hit	base read	new hit	new read	base ms	new ms
39,650	3,090,022	134,504	136,400	0	1,940	157
168,188	3,231,123	265,565	844,101	210,926	6,654	5,672
335,893	3,418,769	336,168	1,590,297	350,457	17,766	18,060
1,191,775	4,618,037	387,349	4,153,278	278,784	15,263	13,882

Cold (shared buffers and page cache dropped, single shot):

matching chunks	base read → new read	base ms → new ms
39,650	139,529 → 11,513	3,904 → 599
335,893	348,471 → 387,071	20,595 → 19,831

Reading of that: at the selectivity a real query has, the leg stops reading altogether — the candidate set is small enough to sit in cache, and total buffer touches fall by more than an order of magnitude. As the term matches more of the corpus the two shapes converge, because the full-text scan itself becomes the cost and neither shape can avoid it. The worst cell measured is 1.02x slower, inside this machine's run-to-run spread (the same base query measured between 3.3 s and 17.8 s across runs on a term matching a third of the corpus).

An earlier variant that ranked inside the match stage and capped it was measured as slower than today between roughly 150k and 350k matches — that is the shape this PR deliberately does not use, and why matched_keyword_chunks carries identifiers only.

SELECT DISTINCT inside the ARRAY(...) was measured and made no difference at any selectivity (identical buffer counts); Postgres already aggregates the array for the bitmap, so it is not included.

Correctness

EXCEPT in both directions against the previous statement, 0 rows each way, across six configurations: 39,650 / 168,188 / 335,893 / 1,191,775 matching chunks, page sizes 50 and 200, and a page at offset 1,000. No cap and no truncation anywhere in the new shape, so the result set is identical by construction rather than by sampling.

Tests

New unit test asserts the ordering contract — the match stage precedes the visibility stage, carries no rank, and the visibility stage is restricted to matched identifiers. Mutation-verified three ways: reverting the source fails it, moving the rank back into the match stage fails it, and swapping = ANY (ARRAY(...)) for a subquery fails it. Three existing statement markers moved to the new CTE name, including the staging-only latency harness.

apps/sim/lib/knowledge (2,933 tests) and apps/sim/app/api/knowledge (187 tests) pass.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 18, 2026 6:06pm UTC

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge with the authorization and result-ordering contracts preserved.

Summary

This PR restructures live-scope keyword retrieval so full-text matching narrows the document set before the existing visibility predicate runs.

• Materializes matching chunk identifiers before evaluating document authorization.
• Ranks and paginates only chunks whose documents survive visibility checks.
• Updates query-shape unit tests and latency-plan instrumentation for the new CTE ordering.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Full-text keyword match] --> B[Matched chunk identifiers]
  B --> C[Matched document lookup]
  C --> D[Document visibility and source-access checks]
  D --> E[Rank authorized chunks]
  E --> F[Limit and offset]
  F --> G[Hydrate with live authorization recheck]

Loading

_{Reviews (1) · Last reviewed commit: "improvement(knowledge): match keyword ch..."}

[cubic-dev-ai]

No issues found across 3 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}