Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/push-batch-investigation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Push batch lifecycle investigation

Original failure: main commit 6e24bcbead3d352f305fb92bee9d894de441f554, run 34237080747, Debian full-suite context.
The partial-transfer case entered lifecycle failure before the following batch failed accounting.

Current probes require exact ordinary failure status and preserve lifecycle barriers. Eight fixed rounds stop on the first failure; no retry-to-green.
Selective push-only runs have not reproduced the original failure. This document is an unknown dependency under the existing tests/run.sh mapping, intentionally exercising the existing full-suite fallback for this diagnostic PR. No workflow or selection rules are changed.

Capture only Bash version, return codes and lifecycle maps. No credentials, full environment dump or global xtrace.
A green full run is not proof of a fix. Retain original logs and distinguish fixture instrumentation from production behavior.

Next evidence needed: first failing boundary and pre-teardown state under the full suite. Production code remains unchanged; PR 47 stays blocked pending an explained resolution.
29 changes: 27 additions & 2 deletions tests/test-push.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1386,13 +1386,38 @@ EOF
runtime_has_published_worker_state && fail "sliding batch left worker-session state"
runtime_has_worker_registration_state && fail "sliding batch left registration state"
: > "$SUCCESS_FILE"; : > "$FAILED_FILE"; : > "$root/state/current"; : > "$root/state/max"
diagnose_batch_contract() {
printf "DIAG: bash=%s status=%s lifecycle_failed=%s worker_error=%s\n" "$BASH_VERSION" "$1" "$BATCH_WORKER_FAILED" "$BATCH_WORKER_ERROR" >&2
declare -p ACTIVE_WORKERS ACTIVE_WORKER_STATE_FILES ACTIVE_WORKER_STATE_STARTS REGISTERING_WORKERS WORKER_REGISTRATION_READY_FILES WORKER_REGISTRATION_RELEASE_FILES WORKER_REGISTRATION_STARTS WORKER_REGISTRATION_STAGE_FILES >&2
}
# Keep production call stacks intact; diagnose only after return.
assert_transfer_failure_contract() {
local label=$1 status=$2
if [[ $status != 1 || $BATCH_WORKER_FAILED != false || $BATCH_WORKER_ERROR != false ]]; then
diagnose_batch_contract "$status"
fail "$label must be ordinary transfer failure, not lifecycle/accounting failure"
fi
pass "$label preserves ordinary failure status"
}
# Fixed probes: stop on first failure, never reset lifecycle barriers.
batch_probe_deadline=$((SECONDS + 90))
for batch_probe in 1 2 3 4 5 6 7 8; do
(( SECONDS < batch_probe_deadline )) || fail "batch probe budget exhausted"
printf "DIAG: consecutive failure batch round=%s/8\n" "$batch_probe"
: > "$SUCCESS_FILE"; : > "$FAILED_FILE"; : > "$root/state/current"; : > "$root/state/max"
SERVERS=(good1 bad1 good2)
assert_fail "partial server failure returns nonzero" run_transfer source destination
partial_status=0
run_transfer source destination || partial_status=$?
assert_transfer_failure_contract "partial failure" "$partial_status"
assert_eq 2 "$(wc -l < "$SUCCESS_FILE")" "partial failure completes both unrelated servers"
assert_eq 1 "$(wc -l < "$FAILED_FILE")" "partial failure records one failed server"
: > "$SUCCESS_FILE"; : > "$FAILED_FILE"; : > "$root/state/current"; : > "$root/state/max"
SERVERS=(bad1 bad2)
assert_fail "all server failures return nonzero" run_transfer source destination
all_status=0
run_transfer source destination || all_status=$?
assert_transfer_failure_contract "all failures" "$all_status"
assert_eq 2 "$(wc -l < "$FAILED_FILE")" "all failures record every server"
done
cleanup_runtime
)

Expand Down