Skip to content

feat(query-engine): prototype streaming query engine#360

Draft
forshev wants to merge 16 commits into
mainfrom
0-one-phase-search
Draft

feat(query-engine): prototype streaming query engine#360
forshev wants to merge 16 commits into
mainfrom
0-one-phase-search

Conversation

@forshev

@forshev forshev commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Introduces the foundation for a new streaming search engine that will enable searching by non-indexed fields.

Key Changes

  • New Query Engine with streaming RecordProducer interface
  • SeqQL parser extensions: filter, sort, limit, stats, fields pipes
  • gRPC streaming API for one-phase search (store side)

Architecture

Instead of the traditional two-phase approach (find IDs → fetch documents), this engine streams documents directly through a chain of executors (filter → sort → limit → project), enabling efficient search on non-indexed fields.


  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

If you have used LLM/AI assistance please provide model name and full prompt:

Model: {{model-name}}
Prompt: {{prompt}}

@forshev forshev self-assigned this Feb 20, 2026
@forshev forshev added the feature New feature or request label Feb 20, 2026
@codecov-commenter

codecov-commenter commented Feb 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.38028% with 591 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.90%. Comparing base (23d27e1) to head (5b47f6e).

Files with missing lines Patch % Lines
proxyapi/grpc_one_phase_search.go 0.00% 175 Missing ⚠️
proxyapi/grpc_v1.go 14.28% 80 Missing and 4 partials ⚠️
storeapi/grpc_one_phase_search.go 60.76% 64 Missing and 18 partials ⚠️
proxy/search/one_phase_search.go 53.98% 63 Missing and 12 partials ⚠️
parser/seqql_pipes.go 71.71% 32 Missing and 24 partials ⚠️
query/exec/data_source.go 76.19% 21 Missing and 14 partials ⚠️
storeapi/client.go 0.00% 25 Missing ⚠️
proxyapi/grpc_async_search.go 0.00% 16 Missing ⚠️
query/exec/merger.go 89.79% 9 Missing and 1 partial ⚠️
query/record.go 79.48% 6 Missing and 2 partials ⚠️
... and 7 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #360      +/-   ##
==========================================
- Coverage   70.83%   69.90%   -0.93%     
==========================================
  Files         225      236      +11     
  Lines       17512    18922    +1410     
==========================================
+ Hits        12404    13227     +823     
- Misses       4183     4691     +508     
- Partials      925     1004      +79     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggWide/size=10000-4 8aefcc afad98
47761.00 ns/op 58671.00 ns/op 1.23 🔴
AggWide/size=1000000-4 8aefcc afad98
516.00 B/op 623.00 B/op 1.21 🔴
5419714.00 ns/op 6613404.00 ns/op 1.22 🔴
FindSequence_Random/small-4 8aefcc afad98
6638.44 MB/s 5400.47 MB/s 0.81 🔴

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggDeep/size=1000-4 8aefcc 5e2344
4817.00 ns/op 5772.00 ns/op 1.20 🔴
AggDeep/size=1000000-4 8aefcc 5e2344
4837044.00 ns/op 5950444.00 ns/op 1.23 🔴
AggWide/size=10000-4 8aefcc 5e2344
47761.00 ns/op 57932.00 ns/op 1.21 🔴
FindSequence_Random/small-4 8aefcc 5e2344
6638.44 MB/s 5084.65 MB/s 0.77 🔴

@forshev forshev force-pushed the 0-one-phase-search branch 2 times, most recently from cf22e58 to f9534c5 Compare April 14, 2026 06:28
@forshev forshev force-pushed the 0-one-phase-search branch from a1659f5 to 64a9b26 Compare April 24, 2026 12:21
@forshev forshev force-pushed the 0-one-phase-search branch from 7a58d66 to 8d3d79d Compare May 4, 2026 12:23
@forshev forshev force-pushed the 0-one-phase-search branch from d167fff to 695b657 Compare May 14, 2026 11:15
@forshev forshev force-pushed the 0-one-phase-search branch 3 times, most recently from 4877d35 to 21cd3f4 Compare May 25, 2026 13:50
@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
AggWide/size=1000000-4 2c48a6 6a5e64
467.00 B/op 545.00 B/op 1.17 🔴

@forshev forshev force-pushed the 0-one-phase-search branch 6 times, most recently from b417d52 to 67e6e8a Compare June 4, 2026 11:48
@forshev forshev force-pushed the 0-one-phase-search branch from 67e6e8a to 5b47f6e Compare June 17, 2026 14:07
@eguguchkin eguguchkin changed the title feat: one phase search feat(query-engine): prototype streaming query engine Jul 3, 2026
@eguguchkin eguguchkin requested review from cheb0, dkharms, eguguchkin and moflotas and removed request for cheb0 July 3, 2026 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

epic/query-engine feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants