BE-618: Rank semantic searches on a quantized embedding column with per-policy-branch reads by TimDiekmann · Pull Request #9124 · hashintel/hash

TimDiekmann · 2026-07-30T15:41:11Z

🌟 What is the purpose of this PR?

Top-nav semantic search took 16s+ locally (BE-618). The generic filter path sorted all entities by exact cosine distance before cutting to the limit, and the cross-table permission disjunction forced the planner to materialize the whole visible set.

This PR makes the search index-driven: entities are ranked on a new binary-quantized embedding column via a partial HNSW index, permission policies are decomposed into one conjunctive branch per permit (each individually plannable), and the candidates are re-scored against the full vector before hydration. Measurements and query plans are documented in BE-618.

🔗 Related links

BE-618 — the investigation
BE-735 — follow-up merging the per-branch reads into one statement (stacked PR, to be merged together with this one)
BE-734 — follow-up collapsing structurally identical web permits
BE-732 — production embedding backfill (only 31k of 726k prod entities have embeddings)

🚫 Blocked by

Nothing. The BE-735 follow-up PR is stacked on this branch and follows it into main.

🔍 What does this change?

Commit 1 — quantized ranking (2a1ab997):

Migration V57: generated embedding_bits bit(3072) column (binary_quantize, 392B inline vs 12kB TOASTed vector) on entity_embeddings and entity_type_embeddings, plus a partial HNSW index (bit_hamming_ops, WHERE property IS NULL) on entities; shadow migrations updated
SQL-AST: BinaryOperator::HammingDistance (<~>), Function::BinaryQuantize, PostgresType::Bit
SelectCompiler::rank_by_quantized_distance + restrict_embedding_property: ranks on the quantized column with an INNER embeddings join, forbids cursors, overfetches limit × 4 candidates
search_entities / search_entity_types: ranked key-read + exact <=> rerank with the distance threshold, then hydration with rank restore, in one REPEATABLE READ transaction; SET LOCAL hnsw.ef_search sized to the candidate pool with iterative_scan = relaxed_order
Filter::CosineDistance removed entirely (compile arm, error variants, Distance pseudo-columns, keys-first gate); snapshot restore switched to explicit column lists (generated columns reject INSERT ... SELECT *)

Commit 2 — policy branches (HEAD):

Filter::for_policy_branches: splits the permit disjunction at construction (no tree parsing), one branch per permit with all forbids conjoined; duplicate and unmatchable permits produce no branch; for_policies re-expressed over the same parts extraction with byte-identical output
search_entities_impl moved to knowledge/entity/search.rs and restructured: one candidate key-read per branch, union-dedup, exact rerank over unnest arrays with deterministic tie-breaking, divergence warning if hydration disagrees with the ranking, branch count recorded on the tracing span
New compile goldens (branch shape, end-to-end branch split) and a property test asserting the branch union selects the same entities as the combined filter across 144 policy configurations

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

do not affect the execution graph

⚠️ Known issues

Behaviour change: results were previously exact over all entities; they are now approximate-then-exact — an entity whose quantized rank is more than limit × 4 positions away from its exact rank can differ. The binary-quantization recall has not been measured yet.
The per-branch reads run sequentially (N+3 round trips per search) until the stacked BE-735 PR lands.
The candidate pool counts rows duplicated by to-many filter joins, so the effective pool can fall below limit × 4 and miss a nearer neighbour. Deduplicating before the limit is part of BE-735, which rewrites the reads into one statement.
Actors with many webs get one branch per web role (BE-734); duplicate policies in prod (BE-696) would each cost a read — identical permits are deduplicated at branch assembly as a stopgap.
search_entity_types keeps the single-statement shape: entity_type_embeddings has no HNSW index yet and stays small. It also ranks and hydrates without a transaction, unlike the entity search — BE-738, which needs a trait change the compiler does not allow today.

🐾 Next steps

BE-735: statement-layer UNION + shared parameter registry, rerank composed from the AST, single statement per search
BE-734: collapse structurally identical web permits into web_id = ANY(...)
BE-732: production embedding backfill
Measure binary-quantization recall against exact ranking; the overfetch factor 4 is unmeasured

🛡 What tests cover this?

tests/graph/integration/postgres/semantic_search.rs (new): 7 tests against constructed ground truth (exact cosine distances 0/0.5/1/2) — ranking with rank restore, distance threshold, limit, actor isolation, cross-branch deduplication via entity-scoped policies, draft handling, request filters
filter::tests::policy_conversion: 15 unit tests incl. a property test comparing for_policy_branches against for_policies over a synthetic universe
Compile goldens pinning the ranked statement shapes, parameters, and the branch split end to end
libs/@local/graph/postgres-store/tests/semantic_search/main.rs: ignored perf harness against a seeded database (measures, does not assert)

❓ How to test this?

Run the integration tests: cargo nextest run --package hash-graph-integration --test postgres semantic_search (requires a freshly migrated database)
With a seeded database: start the graph (cargo run --bin hash-graph -- server) and POST /entities/search with an embedding, maximumSemanticDistance and limit — results are ordered by ascending distance
EXPLAIN ANALYZE on a branch statement shows the HNSW index driving broad branches and conventional index scans on selective ones

📹 Demo

Plans and measurement details are documented in the Linear issues.

Adds a generated bit(3072) column plus a partial HNSW index over the combined per-entity embeddings, and reworks both search endpoints to rank candidates on it under the permission and request filters, re-score them against the full vector, and hydrate the survivors. Replaces the `Filter::CosineDistance` special case, which scanned every embedding row unindexed; keys-first entity reads are unconditional now.

vercel · 2026-07-30T15:41:19Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
petrinaut	Ready	Preview	Aug 4, 2026 10:34am

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
hash	Ignored	Preview	Aug 4, 2026 10:34am
hashdotdesign-tokens	Ignored	Preview	Aug 4, 2026 10:34am

…slow-top-nav-search-query-performance # Conflicts: # libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs

codecov · 2026-07-30T16:14:38Z

Codecov Report

❌ Patch coverage is 62.39669% with 273 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.61%. Comparing base (639beae) to head (16d8f47).
⚠️ Report is 27 commits behind head on main.

Files with missing lines	Patch %	Lines
...tore/src/store/postgres/knowledge/entity/search.rs	0.00%	196 Missing ⚠️
.../graph/postgres-store/src/snapshot/entity/batch.rs	0.00%	24 Missing ⚠️
libs/@local/graph/store/src/filter/mod.rs	95.82%	15 Missing and 2 partials ⚠️
...s-store/src/snapshot/ontology/entity_type/batch.rs	0.00%	15 Missing ⚠️
...s-store/src/store/postgres/ontology/entity_type.rs	0.00%	11 Missing ⚠️
...rc/store/postgres/query/ast/expression/function.rs	50.00%	4 Missing ⚠️
...h/postgres-store/src/store/postgres/query/table.rs	33.33%	4 Missing ⚠️
...gres-store/src/store/postgres/query/compile/mod.rs	97.95%	0 Missing and 1 partial ⚠️
...es-store/src/store/postgres/query/postgres_type.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9124      +/-   ##
==========================================
+ Coverage   59.55%   59.61%   +0.06%     
==========================================
  Files        1408     1409       +1     
  Lines      137734   138283     +549     
  Branches     6418     6424       +6     
==========================================
+ Hits        82028    82438     +410     
- Misses      54711    54843     +132     
- Partials      995     1002       +7

Flag	Coverage Δ
apps.hash-ai-worker-ts	`1.99% <ø> (ø)`
apps.hash-api	`12.09% <ø> (ø)`
local.hash-backend-utils	`2.55% <ø> (ø)`
local.hash-graph-sdk	`10.02% <ø> (ø)`
local.hash-isomorphic-utils	`6.37% <ø> (+0.62%)`	⬆️
rust.hash-graph-api	`7.37% <ø> (ø)`
rust.hash-graph-postgres-store	`29.33% <19.74%> (-0.34%)`	⬇️
rust.hashql-compiletest	`28.39% <ø> (ø)`
rust.hashql-eval	`79.82% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…llow

cursor · 2026-07-31T17:39:35Z

PR Summary

Medium Risk
Touches core entity search, authorization filter shape, and a DB migration with new indexes; results are approximate-then-exact and policy-branch fan-out can affect latency, though behavior is heavily tested.

Overview
Replaces the old full-table cosine-distance filter path for semantic search with an index-backed, two-stage pipeline: rank on stored embedding_bits (HNSW on combined entity embeddings), then re-score with exact vector distance, apply the threshold, and hydrate while preserving rank.

Schema: adds generated embedding_bits (binary_quantize) on entity_embeddings and entity_type_embeddings, plus a partial HNSW index on entity combined embeddings (property IS NULL). Snapshot restore omits generated columns on insert.

Query compiler: drops Filter::CosineDistance and the embeddings distance subquery rewrite; adds Hamming distance / binary_quantize, rank_by_quantized_distance, restrict_embedding_property, and QUANTIZED_RANK_OVERFETCH (4×). Entity list queries can use keys-first again without an embeddings-filter gate.

Entity search: new search.rs runs inside a read-only transaction; splits view policies via Filter::for_policy_branches (one conjunctive branch per permit + shared forbids), fetches candidates per branch with SET LOCAL hnsw.*, dedupes, reranks, then hydrates. Entity type search uses the same quantized-then-exact pattern with a single combined policy filter.

Tests: integration tests with known cosine distances; policy-branch unit/property tests; compile goldens updated.

^{Reviewed by Cursor Bugbot for commit 16d8f47. Bugbot is set up for automated code reviews on this repo. Configure here.}

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…ing tables

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 16d8f47. Configure here.}

github-actions · 2026-07-31T19:28:51Z

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2002	$$26.4 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}0.609 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.84 \mathrm{ms} \pm 16.4 \mathrm{μs}\left({\color{gray}0.979 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 1002	$$11.9 \mathrm{ms} \pm 73.6 \mathrm{μs}\left({\color{gray}0.544 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 3314	$$38.6 \mathrm{ms} \pm 314 \mathrm{μs}\left({\color{gray}-0.478 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$11.7 \mathrm{ms} \pm 76.3 \mathrm{μs}\left({\color{gray}-1.251 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 1527	$$21.4 \mathrm{ms} \pm 214 \mathrm{μs}\left({\color{gray}0.050 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 2078	$$27.1 \mathrm{ms} \pm 196 \mathrm{μs}\left({\color{gray}-1.757 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.14 \mathrm{ms} \pm 16.0 \mathrm{μs}\left({\color{gray}0.355 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 1033	$$12.7 \mathrm{ms} \pm 92.2 \mathrm{μs}\left({\color{gray}-1.751 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_medium

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 102	$$3.15 \mathrm{ms} \pm 21.9 \mathrm{μs}\left({\color{gray}-0.841 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.45 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.581 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 52	$$2.78 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}1.61 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 269	$$4.46 \mathrm{ms} \pm 42.6 \mathrm{μs}\left({\color{gray}1.71 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$2.91 \mathrm{ms} \pm 17.5 \mathrm{μs}\left({\color{gray}-0.080 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 108	$$3.47 \mathrm{ms} \pm 19.8 \mathrm{μs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 133	$$3.75 \mathrm{ms} \pm 22.4 \mathrm{μs}\left({\color{gray}0.287 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.87 \mathrm{ms} \pm 18.5 \mathrm{μs}\left({\color{gray}2.07 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 63	$$3.38 \mathrm{ms} \pm 23.6 \mathrm{μs}\left({\color{gray}-0.206 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_none

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2	$$2.15 \mathrm{ms} \pm 11.8 \mathrm{μs}\left({\color{gray}0.627 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.03 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}0.044 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 2	$$2.11 \mathrm{ms} \pm 11.6 \mathrm{μs}\left({\color{gray}-0.709 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 8	$$2.36 \mathrm{ms} \pm 18.0 \mathrm{μs}\left({\color{gray}0.028 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.18 \mathrm{ms} \pm 12.9 \mathrm{μs}\left({\color{gray}-0.506 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 3	$$2.33 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.780 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_small

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 52	$$2.48 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.643 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.22 \mathrm{ms} \pm 11.0 \mathrm{μs}\left({\color{gray}-0.388 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 26	$$2.45 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}4.59 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 94	$$2.79 \mathrm{ms} \pm 18.5 \mathrm{μs}\left({\color{gray}0.204 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$2.42 \mathrm{ms} \pm 16.5 \mathrm{μs}\left({\color{gray}0.640 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 27	$$2.64 \mathrm{ms} \pm 14.1 \mathrm{μs}\left({\color{gray}1.95 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 66	$$2.71 \mathrm{ms} \pm 16.3 \mathrm{μs}\left({\color{gray}-0.556 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.39 \mathrm{ms} \pm 13.3 \mathrm{μs}\left({\color{gray}1.39 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 29	$$2.70 \mathrm{ms} \pm 22.1 \mathrm{μs}\left({\color{gray}3.64 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_complete

Function	Value	Mean	Flame graphs
entity_by_id;one_depth	1 entities	$$37.1 \mathrm{ms} \pm 208 \mathrm{μs}\left({\color{gray}-0.655 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	10 entities	$$29.0 \mathrm{ms} \pm 246 \mathrm{μs}\left({\color{gray}-0.448 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	25 entities	$$30.4 \mathrm{ms} \pm 156 \mathrm{μs}\left({\color{gray}-1.183 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	5 entities	$$28.1 \mathrm{ms} \pm 183 \mathrm{μs}\left({\color{gray}3.03 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	50 entities	$$37.1 \mathrm{ms} \pm 281 \mathrm{μs}\left({\color{gray}2.21 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	1 entities	$$44.3 \mathrm{ms} \pm 179 \mathrm{μs}\left({\color{gray}-0.476 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	10 entities	$$35.1 \mathrm{ms} \pm 188 \mathrm{μs}\left({\color{gray}0.309 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	25 entities	$$81.4 \mathrm{ms} \pm 429 \mathrm{μs}\left({\color{gray}-1.023 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	5 entities	$$29.3 \mathrm{ms} \pm 204 \mathrm{μs}\left({\color{lightgreen}-34.219 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	50 entities	$$267 \mathrm{ms} \pm 1.35 \mathrm{ms}\left({\color{gray}-2.318 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	1 entities	$$9.43 \mathrm{ms} \pm 63.0 \mathrm{μs}\left({\color{gray}0.726 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	10 entities	$$9.35 \mathrm{ms} \pm 48.6 \mathrm{μs}\left({\color{gray}-1.035 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	25 entities	$$9.41 \mathrm{ms} \pm 53.9 \mathrm{μs}\left({\color{gray}1.04 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	5 entities	$$9.38 \mathrm{ms} \pm 62.9 \mathrm{μs}\left({\color{gray}0.859 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	50 entities	$$9.17 \mathrm{ms} \pm 42.3 \mathrm{μs}\left({\color{gray}-1.953 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_linkless

Function	Value	Mean	Flame graphs
entity_by_id	1 entities	$$9.25 \mathrm{ms} \pm 55.2 \mathrm{μs}\left({\color{gray}0.408 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10 entities	$$9.37 \mathrm{ms} \pm 47.8 \mathrm{μs}\left({\color{gray}-1.714 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	100 entities	$$9.31 \mathrm{ms} \pm 54.5 \mathrm{μs}\left({\color{gray}0.259 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	1000 entities	$$9.38 \mathrm{ms} \pm 66.7 \mathrm{μs}\left({\color{gray}-3.611 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10000 entities	$$9.80 \mathrm{ms} \pm 58.9 \mathrm{μs}\left({\color{gray}1.40 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity

Function	Value	Mean	Flame graphs
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/block/v/1`	$$9.54 \mathrm{ms} \pm 63.7 \mathrm{μs}\left({\color{gray}-2.621 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/book/v/1`	$$9.56 \mathrm{ms} \pm 50.6 \mathrm{μs}\left({\color{gray}-1.842 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/building/v/1`	$$9.63 \mathrm{ms} \pm 58.5 \mathrm{μs}\left({\color{gray}-0.284 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/organization/v/1`	$$9.69 \mathrm{ms} \pm 55.4 \mathrm{μs}\left({\color{gray}-0.772 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/page/v/2`	$$9.66 \mathrm{ms} \pm 50.0 \mathrm{μs}\left({\color{gray}-1.887 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/person/v/1`	$$9.66 \mathrm{ms} \pm 47.1 \mathrm{μs}\left({\color{gray}-1.729 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/playlist/v/1`	$$9.66 \mathrm{ms} \pm 53.1 \mathrm{μs}\left({\color{gray}0.577 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/song/v/1`	$$9.66 \mathrm{ms} \pm 48.1 \mathrm{μs}\left({\color{gray}-4.144 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1`	$$9.84 \mathrm{ms} \pm 67.1 \mathrm{μs}\left({\color{gray}0.677 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity_type

Function	Value	Mean	Flame graphs
get_entity_type_by_id	Account ID: `bf5a9ef5-dc3b-43cf-a291-6210c0321eba`	$$7.21 \mathrm{ms} \pm 37.6 \mathrm{μs}\left({\color{gray}3.16 \mathrm{\%}}\right) $$	Flame Graph

representative_read_multiple_entities

Function	Value	Mean	Flame graphs
entity_by_property	traversal_paths=0	0	$$52.0 \mathrm{ms} \pm 356 \mathrm{μs}\left({\color{lightgreen}-7.768 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$96.5 \mathrm{ms} \pm 503 \mathrm{μs}\left({\color{gray}-2.291 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$58.8 \mathrm{ms} \pm 349 \mathrm{μs}\left({\color{gray}-4.343 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$66.9 \mathrm{ms} \pm 461 \mathrm{μs}\left({\color{gray}-4.912 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$75.1 \mathrm{ms} \pm 394 \mathrm{μs}\left({\color{gray}-2.956 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$80.4 \mathrm{ms} \pm 469 \mathrm{μs}\left({\color{gray}-2.011 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=0	0	$$41.1 \mathrm{ms} \pm 245 \mathrm{μs}\left({\color{gray}-0.923 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$64.6 \mathrm{ms} \pm 319 \mathrm{μs}\left({\color{gray}-1.136 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$46.5 \mathrm{ms} \pm 226 \mathrm{μs}\left({\color{gray}-0.428 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$54.9 \mathrm{ms} \pm 515 \mathrm{μs}\left({\color{gray}0.288 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$56.2 \mathrm{ms} \pm 378 \mathrm{μs}\left({\color{gray}-0.609 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$56.0 \mathrm{ms} \pm 391 \mathrm{μs}\left({\color{gray}-0.620 \mathrm{\%}}\right) $$

scenarios

Function	Value	Mean	Flame graphs
full_test	query-limited	$$108 \mathrm{ms} \pm 719 \mathrm{μs}\left({\color{red}5.76 \mathrm{\%}}\right) $$	Flame Graph
full_test	query-unlimited	$$118 \mathrm{ms} \pm 506 \mathrm{μs}\left({\color{red}5.16 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-limited	$$16.3 \mathrm{ms} \pm 95.3 \mathrm{μs}\left({\color{gray}-2.372 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-unlimited	$$477 \mathrm{ms} \pm 1.04 \mathrm{ms}\left({\color{gray}0.089 \mathrm{\%}}\right) $$	Flame Graph

Copilot

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 1 comment.

-          {
-            "type": "command",
-            "command": "jq -r '.tool_input.file_path | select(endswith(\".toml\"))' | xargs -r mise exec --env dev taplo -- taplo fmt"
+            "command": "node .vscode/setup.mjs"


Copilot

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

TimDiekmann added 2 commits July 29, 2026 18:07

BE-618: Rank semantic-search candidates per policy branch

ed25fc4

TimDiekmann self-assigned this Jul 30, 2026

github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Jul 30, 2026

vercel Bot deployed to Preview – hash July 30, 2026 16:00 View deployment

Merge remote-tracking branch 'origin/main' into t/be-618-investigate-…

bfc75b3

…slow-top-nav-search-query-performance # Conflicts: # libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs

vercel Bot temporarily deployed to Preview – hashdotdesign-tokens July 30, 2026 16:11 Inactive

BE-618: Box the hydration read to keep the search future's layout sha…

96c3aae

…llow

vercel Bot temporarily deployed to Preview – petrinaut July 30, 2026 16:19 Inactive

vercel Bot deployed to Preview – hash July 30, 2026 16:33 View deployment

TimDiekmann marked this pull request as ready for review July 31, 2026 17:39

TimDiekmann requested a review from a team as a code owner July 31, 2026 17:39

Copilot AI balanced review requested due to automatic review settings July 31, 2026 17:39

Copilot started reviewing on behalf of TimDiekmann July 31, 2026 17:39 View session

Copilot AI reviewed Jul 31, 2026

cursor Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread libs/@local/graph/postgres-store/src/store/postgres/ontology/entity_type.rs

Comment thread libs/@local/graph/postgres-store/src/snapshot/entity/batch.rs

BE-618: Keep the quantized column and its index off the snapshot stag…

16d8f47

…ing tables

Copilot AI review requested due to automatic review settings July 31, 2026 18:04

vercel Bot temporarily deployed to Preview – petrinaut July 31, 2026 18:04 Inactive

Copilot started reviewing on behalf of TimDiekmann July 31, 2026 18:05 View session

Copilot AI reviewed Jul 31, 2026

cursor Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread libs/@local/graph/postgres-store/src/store/postgres/ontology/entity_type.rs

vercel Bot deployed to Preview – hash July 31, 2026 18:18 View deployment

Copilot AI review requested due to automatic review settings August 4, 2026 10:25

Copilot started reviewing on behalf of TimDiekmann August 4, 2026 10:25 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Comment thread .claude/settings.json Outdated

{

"type": "command",

"command": "jq -r '.tool_input.file_path | select(endswith(\".toml\"))' | xargs -r mise exec --env dev taplo -- taplo fmt"

"command": "node .vscode/setup.mjs"

vercel Bot temporarily deployed to Preview – petrinaut August 4, 2026 10:34 Inactive

Copilot AI review requested due to automatic review settings August 4, 2026 14:13

TimDiekmann force-pushed the t/be-618-investigate-slow-top-nav-search-query-performance branch from 653360c to 16d8f47 Compare August 4, 2026 14:13

Copilot started reviewing on behalf of TimDiekmann August 4, 2026 14:13 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Uh oh!

Conversation

TimDiekmann commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌟 What is the purpose of this PR?

🔗 Related links

🚫 Blocked by

🔍 What does this change?

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

📜 Does this require a change to the docs?

🕸️ Does this require a change to the Turbo Graph?

⚠️ Known issues

🐾 Next steps

🛡 What tests cover this?

❓ How to test this?

📹 Demo

Uh oh!

vercel Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

policy_resolution_medium

policy_resolution_none

policy_resolution_small

read_scaling_complete

read_scaling_linkless

representative_read_entity

representative_read_entity_type

representative_read_multiple_entities

scenarios

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

TimDiekmann commented Jul 30, 2026 •

edited

Loading

vercel Bot commented Jul 30, 2026 •

edited

Loading

codecov Bot commented Jul 30, 2026 •

edited

Loading

cursor Bot commented Jul 31, 2026 •

edited

Loading