fix(git): retry transient upstream clone and fetch failures

[hzqst]

What kind of change does this PR introduce?

• fix
• feat
• refactor
• perf
• docs
• test
• build / ci
• chore

Summary

Related issue: #35

A transient upstream TLS interruption currently fails a clone/fetch request immediately with HTTP 502, even when a cached mirror exists. This caused intermittent release failures on self-hosted Windows runners using git-cache-proxy 0.1.11 in Docker on WSL2.

During the incident, the client received upstream fetch failed and HTTP 502. The proxy logged the underlying error while fetching an existing public repository:

TLS connect error: error:0A000126:SSL routines::unexpected eof while reading

Related incident: HLND2T/CS2_VibeSignatures#956

This change adds a shared retry helper for upstream git clone --mirror and git fetch:

• Retry recognized transient failures up to three times after the initial attempt, with 1, 2, and 4 seconds of backoff. Recognized failures include TLS EOF, connection resets/timeouts, DNS resolution failures, and HTTP 408, 429, 500, 502, 503, and 504.
• Fail immediately for recognized permanent errors, including authentication, missing repositories, certificate validation, and local filesystem failures. Unrecognized errors are not retried.
• Keep the existing per-repository lock throughout retries, clean failed clone staging directories, preserve existing fetch mirrors, and update freshness only after success.
• Classify bounded stderr diagnostics with the Git subprocess locale fixed to C. Log retry metadata and error categories without exposing raw upstream stderr.

Exhausted retries still return 502; there is no fallback to stale refs. The retry policy does not apply to LFS or local upload-pack, and does not impose a new transfer timeout. It adds at most seven seconds of backoff, excluding time spent in Git operations.

Deployed this implementation in my WSL2 docker and have not observed the HTTP 502 problem again since then.

Tests

• Added / updated tests (unit, plus integration where it fits)
• Not relevant, because: ...

Added diagnostic classification tests and integration tests using real local Git repositories, injected failures, and Tokio's paused clock. Coverage includes transient recovery, retry exhaustion, permanent failures, clone staging cleanup, freshness after failure, concurrent request coalescing, and large stderr output.

Validation on the source corresponding to commit 6cdc8f9:

• Formatting and Clippy checks passed.
• cargo test --all-features --locked: all 65 tests passed.
• Line coverage: 96.73% overall and 97.53% for src/git.rs, exceeding the 80% local coverage gate.
• Release build passed.
• The packaged Docker image completed real clone/fetch requests after injected first-attempt TLS EOF and HTTP 503 failures; each recovered on its second attempt.
• After deployment, health checks and a full clone of the previously failing BinSync repository through the production proxy succeeded. The original Actions job was not rerun as part of validation.

Checklist

• CI is green locally: cargo fmt --all --check, cargo clippy --all-targets --all-features --locked -- -D warnings, cargo test --all-features
• Commits follow Conventional Commits; AI-assisted commits carry an Assisted-by: trailer (The commit uses a Conventional Commit subject, but currently carries Co-Authored-By: Codex <codex@openai.com> rather than the required Assisted-by)
• Preserves the read-only, pull-only invariant (no push or proactive replication to upstream)
• Docs / README updated if behavior or flags changed

Breaking change?

No. Flags, environment variables, and the on-disk cache layout are unchanged; no migration is required. Recognized transient upstream failures may take longer to return an error because they now receive a bounded number of retries.

[codecov]

Codecov Report

❌ Patch coverage is 97.28261% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.69%. Comparing base (05ec23c) to head (1139e74).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/git.rs	97.28%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #37      +/-   ##
==========================================
+ Coverage   96.37%   96.69%   +0.32%     
==========================================
  Files           6        6              
  Lines        1682     1846     +164     
==========================================
+ Hits         1621     1785     +164     
  Misses         61       61              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[rolandjitsu]

Thanks for your contribution @hzqst . I've left a couple of minor comments. Also, please check https://github.com/rolandjitsu/git-cache-proxy/blob/main/CONTRIBUTING.md#commit-messages and fixup the commit.

[hzqst]

Thanks for your contribution @hzqst . I've left a couple of minor comments.

will fix soon

[rolandjitsu]

@hzqst one of the CI jobs is failing - the dependency check.