perf(query): memoize the accepted catalog and compiled queries, skip idle full-text validation

[azimafroozeh]

What & why

Every query rebuilds three things from text that has not changed: the catalog from the schema contract, the plan from the .gq text, and a full-text coverage check for scans that read no full-text index. Each is a pure function of inputs the query already reads, so computing it once per input changes no answer. The cost scales with the schema contract and the .gq text, not the corpus. Kept out of the #665 follow-up so that PR stays about the retry ladder.

• Accepted catalog memo (ReadCaches::accepted_catalog). The catalog is memoized on the exact bytes of _schema.pg, _schema.ir.json and __schema_state.json. The bytes are read on every query, so a SchemaApply by any handle misses on the next query, and a drifted byte fails the full validation. Open and init seed it. One entry per handle. Probe catalog_builds.
• Compiled-query cache (ReadCaches::compiled_queries). The IR of a named query is cached per handle under SHA-256(source) + name; a hit also requires the same catalog Arc (the entry holds a Weak, upgraded and ptr_eq-checked), so a rebuilt catalog recompiles everything and a dropped one pins nothing. 256-entry LRU of IRs, no source copies, errors never cached. Probe query_compiles.
• Full-text validation on demand (FtsFilterDemand). Each typed filter is inspected for contains_tokens as it is set on the ScanTuning. A scan with no full-text query, no SQL-string filter and no contains_tokens demand runs neither get_expr_filter (which rebuilds Lance's filterable schema for a set filter) nor the certificate check. SQL-string filters still go through get_expr_filter; a contains_tokens whose first argument is not a plain column fails closed to every column; staged scan and staged count check only when a filter is set, like count_rows. Probe fts_validations.

Measured effect

From a scratch bench outside this PR (22,000 docs, IVF + full-text index, release build, file://, one machine), so not rule-13 evidence. Each number is one WARM run of one query: the query is repeated, the first runs are discarded, and the median of the next 30 to 45 is shown, same rows on both trees. On this branch a warm run is a cache hit. The cold cost is paid once per query, not once per run: the catalog is memoized at open, and the first run of each query compiles it and caches the plan. Measured first runs on this branch in the 101-type setting: key lookup 1.6 ms, plain nearest 1.7, filtered nearest 2.1, bm25 2.9, rrf 4.9, each about 0.6 to 0.9 ms above its warm run. On main every run is a cold run. The tables differ in how much text the engine handles before the query runs: the schema, and the .gq text sent with the request (our CLI sends its whole stored-queries file each time).

One node type, request text with these 5 queries:

Query (limit 10)	main	this branch	faster
key lookup	0.69 ms	0.67 ms	1.0×
plain nearest	0.67 ms	0.65 ms	1.0×

31 node types, request text with 55 stored queries (about our deployment):

Query (limit 10)	main	this branch	faster
key lookup	2.70 ms	0.93 ms	2.9×
plain nearest	2.44 ms	0.86 ms	2.8×
nearest with a pushed filter	3.36 ms	1.53 ms	2.2×
search + bm25	3.83 ms	2.16 ms	1.8×
rrf(nearest, bm25)	6.91 ms	5.01 ms	1.4×

101 node types, request text with 205 stored queries:

Query (limit 10)	main	this branch	faster
key lookup	4.91 ms	0.95 ms	5.2×
plain nearest	4.95 ms	0.93 ms	5.3×
nearest with a pushed filter	5.66 ms	1.52 ms	3.7×
search + bm25	6.01 ms	2.09 ms	2.9×
rrf(nearest, bm25)	8.61 ms	4.44 ms	1.9×

Small schema, short request: nothing to save. As both grow, main pays about 2 ms then 4 ms per run before the query starts; this branch stays near 0.9 ms on the light queries. The heavier shapes add scan time, equal on both trees.

Backing issue / RFC

• Fixes an accepted issue
• Implements an accepted RFC
• Other: performance change on the schema-contract read path and the full-text validation seam, kept separate from the bug: nearest retry reruns with the same limit #665 retry-ladder PR.

Checklist

• Change is focused: warm-read work that is a pure function of already-read inputs is computed once per input, one probe per cut.
• Tests: warm_read_cost::warm_query_memoizes_catalog_and_compiled_query_until_schema_apply (warm queries after open build nothing and compile once; a second query with a typed filter compiles once more and records zero full-text validations; the same name in a second source is a second compile answering with its own rows; a query naming a type only the new schema declares is refused with the typecheck's unknown-type error, then answered through the applying handle and a second handle, which rebuilds once and recompiles); runtime_cache::compiled_query_cache_hit_requires_the_live_compiled_under_catalog (equal catalog under another Arc misses; a dropped catalog pins nothing and never hits); table_store::fts_filter_demand_names_contains_tokens_columns_and_fails_closed (plain column, CAST-wrapped, literal in column position, no arguments, nested under NOT/AND, no call); search::plain_nearest_skips_the_full_text_validation (unfiltered control); the existing uncertified-index tests keep the full-text-query and SQL-string routes red.
• Docs: docs/releases/v0.11.0.md Highlights; docs/dev/execution.md §Filters and pushdown says when a scan inspects its filter and when it checks coverage.
• Invariants: rule 3 "Every operation uses one coherent accepted view" (the contract bytes are read per call and the catalog is validated against the one snapshot); rule 7 "Physical acceleration is derived state" (both caches are derived, keyed on inputs, a miss recomputes); rule 12 "One source of truth, cheaply derived" (the contract files stay the only authority; a hit is served only for bytes read on that call, so no "maintained parallel truth").

Local verification

• cargo fmt --all --check clean; cargo clippy --workspace --all-targets -- -D warnings silent
• cargo test --workspace --no-fail-fast: 102 targets, 3056 passed, 24 ignored, 1 failed = blob::tests::external_blob_file_policy_rejects_special_files (binds a Unix socket, denied by the local sandbox; blob.rs untouched)
• cargo test -p omnigraph-engine --test search --test warm_read_cost --test lance_surface_guards --test rrf_prefilter_gate --test schema_apply: 42 / 17 / 34 / 12 / 28; --lib runtime_cache 7; --lib table_store 62; omnigraph-gqt --test gq_logic_tests 16
• python3 scripts/check-docs.py 129 files OK; typos docs clean
• Base: upstream/main at d520d2bb

Notes for reviewers

• No staleness risk: build_accepted_catalog_with_schema_gate_held performs the three read_text and two exists calls on every query, and the memo key is those bytes. A hit skips compile_schema_source, the two JSON parses, the cross-checks and build_catalog_from_ir. warm_query_validates_schema_contract_once pins the per-query reads.
• One read-and-validate sequence, read_schema_contract_text + validate_schema_contract_text, backs open, refresh, accepted_schema, the write path and the query path, so a validation step cannot drift between copies; every message comes from one invalid_contract_file and two constants. Precedence: a missing or incomplete contract reports an unparseable source first; a storage error from the existence probes or a failing IR/state read is reported before a parse error.
• The compiled-query cache is bounded by IRs: an entry is a 32-byte digest, the name, an Arc<QueryIR> and a Weak<Catalog>. An entry from a replaced catalog stays until evicted but pins nothing and cannot hit. A digest collision between live sources is not defended against.
• "A hit is exactly what a rebuild would produce" rests on build_catalog_from_ir and fixup_physical_schemas reading only the IR (reviewed at source); Catalog has no equality, so no differential test can state it.
• invalidate_read_caches leaves both caches alone: content-keyed, nothing to clear on a branch refresh.
• The full-text demand is the union over every filter_expr call while Lance keeps only the last filter: a scan may validate more columns than it reads, never fewer.
• No engine path emits a typed contains_tokens today; the typed arm serves SDK-shaped callers and is pinned by the table_store unit test. The warm test's zero validations on a typed-filter read pin the shape that pays.
• Not here: the ANN probe ladder, overfetch and nearest gate (bug: nearest retry reruns with the same limit #665 follow-up), and the ann-shapes bench scenario.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[azimafroozeh]

The Test Workspace red is actor_provenance_concurrent_first_use_has_one_identity, not this PR. It's a race in the actor first-use path from #663: the strict OmniActor insert is staged against live HEAD before the read-set revalidation, so a concurrent winner's row surfaces as a terminal KeyConflict instead of a retry. This PR only touches the read path (catalog memo, compiled-query cache, idle full-text validation skip), and the test passes locally. #678 withdraws the feature and the test with it, so merging.