fix: harden storage consistency and blob admission - #251
Merged
beinan merged 2 commits intoSep 11, 2026
Merged
Conversation
beinan
marked this pull request as ready for review
September 10, 2026 22:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix data loss and inconsistent reads during rollout schema upgrades, peer WAL merges, and checkout; enforce blob admission before allocating payloads; and make dataset deletion remove data through its storage backend.
refresh_latest(). Periodic and already-prepared merges preserve the pin. Closes Exclude live MemWAL generations from explicitly checked-out snapshots #249.Unknown-size admission is deliberately conservative when ROLLOUT_MAX_INFLIGHT_BLOB_BYTES is enabled and can reduce concurrency. Uniqueness serialization is per handle; applications still coordinate keys across processes. Dataset deletion requires remote writers to be quiesced.
Testing
Project CI helper completed successfully: workspace formatting, Clippy (
--workspace --all-targets -- -D warnings), all-target Rust tests, Python tests, Ruff formatting/lint, and Pyright.Rust workspace: 392 passed, 0 failed, 25 ignored. The 22 etcd-backed master tests also passed separately on
948b970against local etcd 3.7.0; this follow-up does not change master code. The remaining 3 ignored tests are existing benchmarks/manual checks.Rebuilt the Python extension from this tree with maturin. Python 3.13: 215 passed, 6 skipped, 1 existing xfail for per-append version advancement under MemWAL. The new merged-version checkout regressions pass.
Added 21 regression tests covering both key types, concurrent schema upgrades, stale and interrupted list scans, checkout during a prepared merge, read-only snapshot mutations, post-commit WAL cleanup and bounded retries, blob admission before loading, cancellation, counter overflow, URI deletion/recreation, evicted live handles across all LRU store kinds, weak-registry cleanup, and deletion failure/retry. Updated the legacy-schema fixture to actually construct the old schema before testing migration.
All 10 GitHub CI checks passed for
b7d1669, including Rust tests and coverage, Python 3.11 tests, all three wheel builds, and style checks.