improvement(knowledge): stop consuming access batches once both overview probes saturate - #7970
Merged
Merged
Conversation
…iew probes saturate The source overview loops over the access batches the read-access generator yields. The first batch is free, but every later one costs a connector discovery query plus a per-connector live source proof over the network. Both probes in the loop already stop issuing queries once they saturate — the searchable probe on its first hit, the indexing probe once every configured provider type is accounted for — but the loop kept pulling batches afterwards, paying the producer's cost for no probe at all. Hoist the two probe guards into closures so the loop body and the exit share one definition of each, and break once neither can change the result. Output is unchanged; only the work is dropped.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
Collaborator
Author
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
readSearchSourceOverviewloops over the access batchesknowledgeReadAccessBatchesyields. The first batch is free, but every later one costs a connector-discovery query plus a per-connector live-source proof over the network.Diagnostics
accessBatchCountandliveProofConnectorCountare annotated inside the generator before each yield, so they now report fewer batches when the loop exits early. That matches what those fields already document — batches yielded to the caller and connector identities sent for live proof, i.e. work actually spent, never a count of the batches the owner could produce. The generator records them before the yield specifically so an abandoning caller leaves correct counts, and it holds for every batch, not just the first. There is notry/finallyin the generator, sobreak(which invokes itsreturn()) skips nothing.Type of Change
Testing
search-source-overview.test.tswith a consumed-batch counter: one test that the loop exits after the first batch once both probes are satisfied, and two that it keeps consuming while either probe is still unsatisfied (one per probe).breakturns the early-exit test red; making thebreakunconditional turns the keep-consuming tests red.providers[].isSyncingandhasSearchableDocumentsare pinned unchanged.apps/sim/lib/knowledge/application+read-access.test.ts: 886 passed, 18 skipped.bun run lint,check-block-registry.ts origin/staging,bun run check:audits(46 audits),bun run docs-manifest:check,bun run type-checkall pass. No migrations or schema changes.Checklist