http: restart pack downloads when servers ignore Range - #78
Draft
ttaylorr-oai wants to merge 12 commits into
Draft
http: restart pack downloads when servers ignore Range#78ttaylorr-oai wants to merge 12 commits into
ttaylorr-oai wants to merge 12 commits into
Conversation
Integrate the current tb/codex/automation topic into the internally distributed codex branch. Codex-Integration: tb/codex/automation@9b7652c87e369dafb6a50e97d24670ccd512f1bc
Integrate the current tb/codex/geometric-maintenance-promisor topic into the internally distributed codex branch. Codex-Integration: tb/codex/geometric-maintenance-promisor@bccc1fd882f27143f7331192a75c6281ba76e2f7
Integrate the current tb/codex/release topic into the internally distributed codex branch. Codex-Integration: tb/codex/release@2bb6c1c18aa3170efff362eb3fb8f096134955f7
Integrate the current dr/codex/dugite topic into the internally distributed codex branch. Codex-Integration: dr/codex/dugite@5b75aebc9071c08a87ddf7bf00b173e05a5bfc96
Integrate the current tb/codex/lto-pgo topic into the internally distributed codex branch. Codex-Integration: tb/codex/lto-pgo@67ba20645b1792f43c8c8ea69c716687bda87ec3
Integrate the current tb/codex/packfile-uri-concurrency topic into the internally distributed codex branch. Codex-Integration: tb/codex/packfile-uri-concurrency@2c3adbb2c475981e340c79fdc5e7f4f9b5d9054e
Integrate the current af/codex/pack-bytes topic into the internally distributed codex branch. Codex-Integration: af/codex/pack-bytes@a6bb1bb89133e5eccf6e3ca3aaa58f560eb3744c
Integrate the current tb/codex/parallel-packfile-uris topic into the internally distributed codex branch. Codex-Integration: tb/codex/parallel-packfile-uris@6f0edce407d5e8c1d212514281f49c9bc433ee3c
HTTP pack downloads resume a saved .pack.temp file by sending a Range header and writing at the end of the saved prefix. A server may ignore the header and return the complete pack with status 200. Appending that body leaves another pack header inside the saved file, which index-pack rejects as corrupt. Use a write callback to inspect the first nonempty response chunk. On 200, rewind before writing and truncate any old tail when the transfer finishes. Delaying truncation keeps concurrent downloads from losing bytes still present in the shared temporary file. Continue appending 206 responses at the requested offset. Preserve empty-response handling so index-pack can verify an already complete temporary file. Add coverage for ignored Range with short and oversized partial files, an empty response, and an interrupted replacement that must preserve the existing prefix. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
CI exits during workflow setup because the repository requires action references to name full commit IDs. The moving version refs in the upstream workflows prevent even the config, style, and whitespace jobs from running. Pin the actions used by these workflows to the commits named by their existing version refs, retaining those refs as comments. This satisfies the repository policy without changing the selected action versions. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
The ignored-Range tests build a separate random pack even though the script already has a pack for HTTP downloads. Reuse that pack and prepare short and oversized inputs once, removing the per-test case statement and size calculations. Compare each completed download with the original pack. Retain coverage for empty and interrupted responses; the latter needs only a 64-byte saved prefix and a response that stops after 32 bytes. Use test_env to set GIT_TRACE_CURL for test_must_fail, as required by the shell portability lint. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
Align the continuation lines in fwrite_pack_file() and use the expected spacing for the new bitfields, as reported by git clang-format in CI. Signed-off-by: Taylor Blau <ttaylorr@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An HTTP pack download resumes from its saved
.pack.tempprefix. If the server ignoresRangeand returns a complete pack with status200, Git appends that body and reports pack corruption. HTTP permits servers to ignore Range; the client must handle the resulting full response. See RFC 9110, section 14.2.Rewind on the first nonempty
200response chunk and remove any stale tail after the transfer succeeds. Delaying truncation preserves bytes used by concurrent fetches sharing the temporary file.206resumes and empty responses retain their existing behavior.Pin the existing CI action versions to full commit IDs so the repository's action policy permits the workflows to run.
Validation on macOS:
codex(47c847ce2641677af3b6525c2867a1f3726178c8).t5550-http-fetch-dumb.shandt5702-protocol-v2.shpass with the patch, including empty-response and existing concurrent-download coverage. The developer build passes with warnings treated as errors.t5550-http-fetch-dumb.sh.