Skip to content

test(storage-opendal): split backend tests into per-backend modules - #3569

Open
dragonknightk wants to merge 1 commit into
apache:mainfrom
dragonknightk:test/split-opendal-backend-tests
Open

dragonknightk wants to merge 1 commit into
apache:mainfrom
dragonknightk:test/split-opendal-backend-tests

Conversation

@dragonknightk

Copy link
Copy Markdown

Which issue does this PR close?

What changes are included in this PR?

Moves each backend's tests out of the crate-level tests module into the
module that owns the backend, following the hdfs_native split in #3111:
memory.rs, fs.rs, s3.rs, gcs.rs, oss.rs and azdls.rs each get
their own mod tests.

lib.rs keeps the cross-backend tests: client config parsing, factory
construction, the storage-agnostic writer contract, and the IO timeout
test, which exercises the TimeoutLayer that resolve applies to every
backend.

Feature gates are not repeated on the moved tests. lib.rs already guards
each backend module behind its opendal-* feature in cfg_if!, so a test
in gcs.rs can only compile when that module exists.

No test was added, removed or renamed; the moved bodies are unchanged.

Are these changes tested?

cargo fmt --all -- --check, cargo clippy -p iceberg-storage-opendal --all-targets -- -D warnings and the same with --all-features, plus
cargo test -p iceberg-storage-opendal --lib (17 tests) and with
--all-features (74 tests).

AI Disclosure

Codex and Grok Bot, working for dragonknightk, contributed to this code.

@dragonknightk

Copy link
Copy Markdown
Author

Hi @mixermt — this implements the split from #3385, following the
hdfs_native pattern from #3111. lib.rs keeps only the cross-backend
tests and loses 180 lines with no additions; CI is green across all 26 checks.

Could you take a look when you have a moment?

}

#[derive(Debug)]
struct EmptyCredentialLoader;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this helper sits between s3_config_parse_path_style_access and test_relativize_path_s3, but only test_s3_factory_custom_credential_loader_serialization_fails uses it. Could it move up next to parse_with so the helpers are together at the top of the module?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — moved EmptyCredentialLoader up next to parse_with so the helpers sit together at the top of the module. cargo fmt --all -- --check and cargo test -p iceberg-storage-opendal --lib s3 both pass, and CI is green ʕ•ᴥ•ʔ

@mixermt mixermt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Verified the moved test bodies are unchanged and each backend module's cfg gate covers its tests. The s3.rs nit is optional.

Move each backend's tests out of the crate-level `tests` module into the
module that owns the backend, following the `hdfs_native` split in apache#3111:
`memory.rs`, `fs.rs`, `s3.rs`, `gcs.rs`, `oss.rs` and `azdls.rs` each get
their own `mod tests`.

`lib.rs` keeps the cross-backend tests: client config parsing, factory
construction, the storage-agnostic writer contract, and the IO timeout
test, which exercises the `TimeoutLayer` that `resolve` applies to every
backend.

Feature gates are not repeated on the moved tests: `lib.rs` already guards
each backend module behind its `opendal-*` feature in `cfg_if!`, so a test
in `gcs.rs` can only be compiled when that module exists.

Generated-by: OpenAI Codex 0.165.0-alpha.1.2
@dragonknightk
dragonknightk force-pushed the test/split-opendal-backend-tests branch from cf9357b to 0aaee05 Compare October 29, 2026 11:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split the iceberg-storage-opendal lib.rs tests into per-backend modules

2 participants