[schmitch]

Fixes #536

What was wrong

A group asked each local member (hosted, or a proxy searching its cache, which in browse is every member) for its first MAX_TAKE = 1,000 hits. Each hit came with every version and the id's casing, on every page and every debounced search. The merge used only the ids. The members were searched one after another.

Because a local member's search had no window, it was never widened. So a group stopped at each member's first 1,000 ids. With the new test's setup on main (a proxy cache of 1,500 ids plus a hosted member), browsing the group reports a total of 1,001, and nothing past the proxy's first 1,000 can be paged to.

The fix

The fix follows the issue's suggestion. The member search is split in two: first the ids, then the hits of the final page.

• Two RepoReadOps methods (crates/core/src/repo/mod.rs):
  • search_ids(repo, params, filter): the ids of every hit of a search of what the repository holds, without the upstream, in its order.
  • search_hits(repo, params, ids): the unenriched hits of those ids, in their order. An id with no matching version left is skipped.
  • Both defaults page through search_unenriched, so any implementation (the test stubs, OfflineMember) stays correct. PackageFilteredReadOps applies its filter to both.
• Hosted and proxy (hosted.rs, proxy.rs): search_filtered is split into search_names (the sorted id list both already computed: the SELECT DISTINCT name …) and hits_of (versions and casing for some ids). The direct search uses the two halves as before. The proxy's two halves share one set of listable bounds (ListableBounds).
• Group (group.rs):
  • A local member answers with all its ids, and the merge works on ids (MemberSearch::ids). Upstream searches are unchanged.
  • The final page then loads its hits through the member that won each id (page_hits → search_hits), one batch per member, and enrich_page enriches them as before.
  • If a member fails to load its page's hits, the search fails. It answered for those ids a moment before, and a page left short would read as the last one.
  • Members are searched four at a time (MEMBER_SEARCHES), as are the page loads.
• Per-id guard work: the group now evaluates every id of its local members, not at most 1,000 per member, so two shortcuts keep that cheap:
  • with no package filters on the graph, visible_leaves memoizes one entry for all ids;
  • serving_leaves returns the visible leaves without a guarded walk when no group's guard is active for the id. With no active guard, the guarded walk visits exactly what the unguarded one does.

The total is exact for local members, over all their ids, under the guards. That includes ids an owner holds that the cache lists (the second core test). The prefer-hosted ownership lookup therefore now covers every candidate id instead of at most 1,000 per member.

Measured

Same shape as the issue: Postgres 18, a hosted member with 300 ids × 15 versions, and a proxy cache of 4,000 ids with 219,000 versions. Times are psql timings of the queries a page runs (3 runs each), on this session's container:

query	before (group page)	after (group page)
all matching ids of the proxy	~80 ms	~80 ms (unchanged)
versions of the proxy member's hits	1,000 ids: 120–190 ms	20 ids: ~3.5 ms
prefer-hosted ownership lookup	1,000 ids: ~6 ms	4,300 ids: ~21 ms

On top of that, before the change the server decoded and sorted about 55,000 version rows per page, and read the casing of 1,000 ids. Now it handles the page's 20 ids.

Tests

• crates/server/tests/browse.rs, a_group_browses_every_cached_id_of_its_proxy:
  • setup: a group over a proxy whose cache lists 1,500 ids (seeded in SQL) and a hosted member that owns one of them;
  • browsing the group reports a total of 1,500, reaches the last page, and serves the owned id from the hosted member;
  • a search for one id finds it.
  • On main it fails with total 1,001.
• crates/core/tests/group_read.rs:
  • a_group_pages_local_ids_and_loads_only_its_page: two hosted members with 1,200 and 300 ids, 100 of them shared. A recording wrapper shows that each member lists its ids once and loads hits only for the ids it won on the page: search_hits gets exactly those 10 + 10 ids, and nothing calls search/search_unenriched. Versions come from the winning member, casing is kept, the total is 1,400, and the last page is reached.
  • a_guard_takes_ids_past_the_first_thousand_off_the_total: a hosted owner holds 10 ids only as unlisted versions, and a cache of 1,100 ids lists them too. The prefer-hosted guard takes them off the total (1,090) and off the page, past the first 1,000.
  • an_offline_proxy_is_searched_like_a_local_member: the stub now sees two unwindowed searches (the ids, then the page's hits) instead of one.

Checks run

• cargo test -p forge-core: all suites; group_read 40 passed, proxy_read 58, hosted_read 12.
• cargo test -p forge-server --lib --test browse --test admin --test admin_rbac --test dispatch --test group_proxy --test plugin_contract --test proxy_search: all passed.
• cargo clippy -p forge-core -p forge-server --all-targets -- -D warnings, cargo fmt --all --check, cargo xtask changelog --check, cargo xtask docs --check.
• No query text changed, so .sqlx/ is unchanged.

Changelog: docs/changes/2.2.0/536.fixed.md. No admin API change, so no web UI screen.

---

Generated by Claude Code