Add an optional hybrid RW mode for LSMT writable layers. - #443
Open
ZarrianShi wants to merge 2 commits into
Open
Add an optional hybrid RW mode for LSMT writable layers.#443ZarrianShi wants to merge 2 commits into
ZarrianShi wants to merge 2 commits into
Conversation
Add an optional hybrid writable-layer type and persist it in the layer header. Keep append RW as the default. Signed-off-by: Zehuan Shi <xocoder@gmail.com>
Reuse existing non-zero mappings in the current writable layer while continuing to append uncovered, zeroed, and lower-layer ranges. Signed-off-by: Zehuan Shi <xocoder@gmail.com>
BigVan
reviewed
Aug 21, 2026
| } | ||
|
|
||
| IFileRW *create_file_rw(bool sparse = false) { | ||
| IFileRW *create_file_rw(bool sparse = false, bool hybrid = false) { |
| HeaderTrailer *pht = (HeaderTrailer *)buf_top; | ||
| layer.virtual_size = pht->virtual_size; | ||
| layer.sparse_rw = pht->is_sparse_rw(); | ||
| layer.rw_type = pht->is_sparse_rw() ? RWType::Sparse |
Member
There was a problem hiding this comment.
use a method like pht->filetype() ?
| } | ||
|
|
||
| // Seed upper [0,64); rewrite [8,56) in place; then write [0,80), appending only [64,80). | ||
| TEST_F(FileTest, hybrid_rw_reuses_upper_data) { |
Member
There was a problem hiding this comment.
This test looks too simple...
do_randwrite() & verify() is better
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an optional “hybrid RW” mode for LSMT writable layers, allowing overwrites to reuse existing non-zero mappings in the current writable layer (rewrite-in-place) while continuing to append for holes/zeroed ranges/lower-layer-only mappings. This aims to reduce RW layer growth for overwrite-heavy workloads while keeping append-only as the default.
Changes:
- Add a
--hybridoption tooverlaybd-createand persist hybrid mode via a writable-layer header flag. - Introduce writable-layer-only range iteration (cursor) and implement hybrid overwrite handling in
LSMTFile::pwrite. - Add/extend unit tests to cover hybrid rewrite/append behavior across overlaps, holes, zeroed mappings, and stacked layers.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/tools/overlaybd-create.cpp | Adds CLI flag and maps it to the new RW type selection. |
| src/test/image_service_test.cpp | Updates tests to use LayerInfo::rw_type instead of the removed sparse_rw. |
| src/overlaybd/lsmt/test/test.cpp | Adds extensive hybrid RW unit tests and updates sparse RW test setup for rw_type. |
| src/overlaybd/lsmt/test/lsmt-filetest.h | Updates test helpers to support rw_type and creating hybrid layers. |
| src/overlaybd/lsmt/index.h | Adds writable-layer cursor interface to support upper-only iteration. |
| src/overlaybd/lsmt/index.cpp | Implements writable-layer cursor and exposes it from Index0. |
| src/overlaybd/lsmt/format_spec.md | Documents the new hybrid_rw header flag bit. |
| src/overlaybd/lsmt/file.h | Replaces sparse_rw boolean with RWType enum in LayerInfo. |
| src/overlaybd/lsmt/file.cpp | Persists hybrid flag in headers and implements hybrid rewrite-in-place logic in pwrite. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
360
to
+363
| HeaderTrailer *pht = (HeaderTrailer *)buf_top; | ||
| layer.virtual_size = pht->virtual_size; | ||
| layer.sparse_rw = pht->is_sparse_rw(); | ||
| layer.rw_type = pht->is_sparse_rw() ? RWType::Sparse | ||
| : (pht->is_hybrid_rw() ? RWType::Hybrid : RWType::Append); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR adds an optional hybrid RW mode for LSMT writable layers.
When enabled, writes reuse existing non-zero mappings in the current writable layer by overwriting their data payload in place. Ranges not covered by the writable layer—including holes, zeroed mappings, and mappings only present in lower layers—continue to append data and new index records.
This reduces RW layer growth for overwrite-heavy workloads while preserving the existing append-only behavior as the default. It does not introduce online compaction.
The change includes:
--hybridoption for creating a hybrid RW upper layer.Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #442
Please check the following list: