Skip to content

feat(global-index): align row filtering and BTree Bloom support - #432

Open
lxy-9602 wants to merge 3 commits into
apache:mainfrom
lxy-9602:bitmap-btree-not
Open

lxy-9602 wants to merge 3 commits into
apache:mainfrom
lxy-9602:bitmap-btree-not

Conversation

@lxy-9602

Copy link
Copy Markdown
Member

Purpose

Linked issue: close #316

  • Track exact row counts in global-index metadata.
  • Use the complete row-ID domain for NOT EQUAL, NOT IN, and IS NOT NULL.
  • Pass shard-local row-range hints to index readers and apply them to BTree queries.
  • Align BTree IN/NOT IN null semantics with Java.
  • Add optional BTree Bloom filters through btree-index.bloom-filter.enabled.
  • Add safe full-match optimization for BTree readers.

Tests

Added tests for:

  • BTree row-range filtering and null semantics.
  • Negative predicates and complete row counts.
  • BTree full-match optimization.
  • Bloom filter builders and all-null indexes.
  • Global-to-local row-range conversion.
  • Primary-key sorted-index negative predicates.

API and Format

  • GlobalIndexIOMeta now includes row_count.
  • GlobalIndexer::CreateReader now accepts total_row_count and optional local row ranges.
  • Add optional BTree Bloom filters through btree-index.bloom-filter.enabled.

Documentation

Documented btree-index.bloom-filter.enabled.

Generative AI tooling

Generated-by: OpenAI Codex (GPT-5)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Reimplement bitmap Global Index for compatibility with Paimon Java

1 participant