Skip to content

benchmark tests: extract into an empty directory, fixes #10057 regression - #10395

Merged
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-benchmark-extract-10057
Sep 19, 2026
Merged

ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:fix-benchmark-extract-10057

Conversation

@ThomasWaldmann

Copy link
Copy Markdown
Member

Since #10057 (commit 1b7ff2b), borg extract refuses to extract into a non-empty directory unless --continue is given. src/borg/testsuite/benchmark_test.py was missed by that change, so all 8 parametrisations of test_extract fail with rc 33 (ExtractionDirNotEmpty) on current master.

The test ran extract with the cwd set to the tmpdir that the repo_url fixture also uses for the repository, the keys directory and the cache directory. That directory is never empty, so the very first (and, with the benchmark.pedantic defaults rounds=1, iterations=1, warmup_rounds=0, only) extract already aborts.

Fix: extract into a dedicated empty subdirectory, created per round by pedantics setup hook, which is not timed. Passing --continue would also make the test green, but it makes extract stat and compare the files that are already there, which is not the extract this benchmark is meant to measure. Creating the directory per round also keeps the benchmark correct if rounds or warmup_rounds are ever raised above the defaults.

The other benchmark tests are not affected - test_create_*, test_delete, test_list, test_info and test_check do not write into the cwd. borg benchmark crud extracts with --dry-run, which is exempt from the check.

CI does not catch this: every pytest invocation in .github/workflows/ and in the tox config passes --benchmark-skip, so the benchmark tests are never run there.

Test-only change, so no docs/changes.rst entry.

Before: 8 failed, 44 passed. After: 52 passed.

🤖 Generated with Claude Code

 regression

test_extract ran "borg extract" with the cwd set to the tmpdir that the
repo_url fixture also uses for the repository, the keys dir and the cache dir.
That directory is therefore never empty, so since extract refuses to extract
into a non-empty directory (see borgbackup#10057), all 8 parametrisations failed with
rc 33 (ExtractionDirNotEmpty) already on the first (and, with the pedantic
defaults rounds=1/iterations=1/warmup_rounds=0, only) call.

Extract into a dedicated, per-round empty subdirectory created by pedantic's
setup hook, which is not timed. Using --continue instead would keep the test
green, but it makes extract stat and compare the files that are already there,
which is not the extract that this benchmark is supposed to measure. The
per-round directory also keeps the benchmark correct if rounds or warmup_rounds
are ever raised above the defaults.

The other benchmark tests are not affected: they do not write into the cwd.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.23%. Comparing base (1b7ff2b) to head (6489248).
⚠️ Report is 10 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10395      +/-   ##
==========================================
+ Coverage   88.16%   88.23%   +0.07%     
==========================================
  Files         103      103              
  Lines       18840    18876      +36     
  Branches     2924     2933       +9     
==========================================
+ Hits        16611    16656      +45     
+ Misses       1553     1544       -9     
  Partials      676      676              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ThomasWaldmann
ThomasWaldmann merged commit ec4fe8a into borgbackup:master Sep 19, 2026
24 of 25 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the fix-benchmark-extract-10057 branch September 19, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant