perf: buffer BlobFile sequential reads in Rust

[geruh]

fixes #8831.

Small sequential reads on BlobFile previously submitted a storage range for every call. File consumers reading in small chunks could spend most of their time waiting for object storage.

This adds lazy read-ahead to Rust BlobFile, shared by Python, Java, and Rust callers. It replaces the Python BufferedReader from the earlier revision.

The default is 4 MiB. buffer_size=0 disables read-ahead. Explicit read_range and read_ranges calls bypass the buffer and leave the cursor unchanged. Seeks within the buffered span reuse it; seeks outside discard it.

blob = ds.take_blobs("assets", indices=[0])[0]
blob = ds.take_blobs("assets", indices=[0], buffer_size=0)[0]

Buffering reduces requests for sequential reads but can fetch unused bytes for header probes and scattered reads. The size is configurable; the default has not yet been validated against S3.

Testing

added coverage for sequential read coalescing, disabled buffering, seeks, buffer resizing, and independent range reads. Python tests also cover reads across buffer boundaries, TextIOWrapper, ZIP, read-only destinations, and closed handles. Java tests cover disabled buffering and invalid buffer sizes.

cc: @wjones127

[LuQQiu]

Should this live in Rust instead? so this only fixes Python, while the Java binding and Rust callers hit the same request amplification. Not blocking — the Python buffer still avoids per-read FFI overhead. Suggest a follow-up to move buffering/prefetch into the Rust BlobFile.

[geruh]

@LuQQiu, thanks for the review! yeah, at first I was gonna just keep it in Python to unblock the issue. But yeah I agree this definitely can be in the rust logic.

Now the PR moved it there and exposed the buffer as a param consistent with other file apis!

Also, gatekeeper doesn't want to re-review lol

[LuQQiu]

Try to trigger lance gate keeper again, and will also rereview

[LuQQiu]

@lancedb-robot review this PR again

[LuQQiu]

Thanks for moving the buffering into Rust — this is exactly the cross-language design I was hoping for, and the test coverage (deterministic submission counts in Rust/Python, TextIOWrapper/ZIP/closed-handle cases, Java validation) is strong.

Requesting changes narrowly on two items; the rest are comments/follow-ups:

1. read_up_to can now return short reads mid-file (inline comment) — a silent semantics change for Java/Rust callers. Please fix or document loudly.
2. .pyi stub drift: LanceBlobFile in python/python/lance/lance/__init__.pyi is missing set_buffer_size and range_submission_count. BlobFile.__init__ now calls set_buffer_size unconditionally, so the stub is actively wrong.

Everything else inline is non-blocking.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

This revision closes all opened Java blob handles when buffer setup raises an I/O or runtime failure and hides the request counter as test instrumentation; the prior short-read and stub fixes remain intact. The remaining accepted risk is the 4 MiB default: probe-then-close reads can overfetch, and a small header read followed by read() can refetch buffered bytes. The author chose to keep this window for now. Callers with those workloads can use buffer_size=0, read_range, or continue sequential reads.

[LuQQiu]

looks good!

[geruh]

Thanks for the review @LuQQiu, i don't got merge perms so feel free once you feel like it's ready!

[LuQQiu]

Following up on my earlier review with a longer-term concern about customers who aren't reading sequentially, plus a concrete menu of acceptable ways to land this. Any one of the three end-states below works for me.

The concern: non-sequential readers pay for the buffer too

Two independent amplification paths:

1. Byte amplification. Every cursor-read miss fetches max(buffer_size, still_need) — 4 MiB by default. Random access (seek + small read) turns every jump into a 4 MiB fetch, and this gets triggered "innocently": zipfile, tarfile, or any format parser over a blob does seek-to-directory / jump-to-member patterns. read_range bypasses the buffer, but file-interface consumers won't know to switch APIs.

2. Memory retention. The retained prefetch is ~buffer_size per open handle, and take_blobs hands out handles in batches, so the real bound is open handles × buffer_size: probing 64 bytes from each of 1,000 large blobs transfers 4 GB and keeps 4 GB resident until the handles close. On top of that, the current code retains the fetch even when it is fully consumed (*prefetch = Some(...) unconditionally when buffer_size > 0), so an oversized read (still_need > buffer_size) pins the whole fetch — the 4 MiB "cap" doesn't hold on that branch — and a handle read to EOF keeps its last block until close.

Small blobs (< buffer size) are actually fine — the first read caches the whole blob and later seeks hit it. The bad combination is large blobs + scattered reads + many concurrent handles.

Three acceptable end-states (either is fine)

	What lands in this PR	Result
A	Release-on-consume fix + ramp-up readahead	Default-on, ideal — but ramp-up is a real algorithm change and may drag this PR out
B	Release-on-consume fix + default lowered to 256 KiB (one constant); ramp-up as an issue-tracked fast-follow that raises the effective max back to 4 MiB	Default-on, worst case capped
C	Neither	Then the buffer should default to buffer_size = 0 (explicit opt-in)

Why I'd rank B over C: opt-in defeats the point of the PR — #8831 is precisely about the default behavior of file-like consumers who will never discover buffer_size. And 256 KiB keeps most of the win: 8 KiB sequential reads still see 32× fewer requests, while the worst case drops from 4 GB resident / 4 MiB per jump to 256 MB / 256 KiB. (4 MiB is within ecosystem norms — s3fs uses 5 MB blocks — but those are single-file opens; take_blobs batching makes it multiplicative, and the PR itself notes the 4 MiB default is unvalidated against S3.)

The release-on-consume fix is tiny

Only read_up_to changes — keep the prefetch only when this read didn't drain it:

let result = data.slice(0..still_need.min(data.len()));
*cursor = fetch_cursor + result.len() as u64;
*prefetch = (*buffer_size > 0 && result.len() < data.len())
    .then(|| BlobPrefetch { start: fetch_cursor, bytes: data });

result.len() == data.len() covers exactly the three sharp corners: oversized reads, the EOF tail block, and a read that exactly drains a block. Header probes (64 < 4 MiB) still retain — that's legitimate caching, governed by the default size / ramp-up, not by this fix.

I checked every submission-count assertion in this PR (7 Rust + 5 Python): none changes, so no existing test needs touching. Two notes:

• One deliberate tradeoff: today, seeking back into a fully-drained span still hits the old buffer (seek's contains check keeps it); after the fix it refetches. Pinning buffer_size per handle for that rare pattern isn't worth it, but it should be a conscious choice.
• Please add a pinning test, since the field is private and only observable via the counter:

#[tokio::test]
async fn fully_consumed_prefetch_is_released() {
    // 40-byte payload, buffer_size = 16
    let _ = blob.read_up_to(16).await.unwrap(); // drains the block
    let after = blob.range_submission_count();
    blob.seek(0).await.unwrap();
    let _ = blob.read_up_to(4).await.unwrap();  // must refetch
    assert_eq!(blob.range_submission_count(), after + 1);
}

Regardless of A/B/C

Please document the memory model in blob.md and the docstrings: each open handle retains up to buffer_size; with batched take_blobs that is multiplicative; random-access consumers should pass buffer_size=0 or use read_range / read_ranges.