Repository navigation
Add more consistency checks in zipfile #139700
Description
zipfile supports ZIP archives with prepended data (such as SFX executable or zipapp program). Normally, ZIP archive contains offsets from the beginning of the ZIP file, but if it detects that the central directory is located at other position, it corrects locations of all files by the corresponding offset.
There is also "Zip64 end of central directory record" offset for which is written explicitly and can be used to calculate the size of prepended data. Currently its consistency with the central directory offset is not tested.
Linked PRs
- gh-139700: Check consistency of the zip64 end of central directory record #139702
- [3.14] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) #139706
- [3.13] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) #139708
- [3.12] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) (GH-139708) #139712
- [3.11] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) (GH-139708) #139713
- [3.10] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) (GH-139708) #139714
- [3.9] gh-139700: Check consistency of the zip64 end of central directory record (GH-139702) (GH-139708) #139715
Activity
pythongh-139700: Check consistency of the zip64 end of central direct…
gh-139700: Check consistency of the zip64 end of central directory re…
pythongh-139700: Check consistency of the zip64 end of central direct…
[3.13] pythongh-139700: Check consistency of the zip64 end of central…
13 remaining items
zanieb commentedon Oct 14, 2025
@sparrowt it's a category of vulnerability called a "parser differential", e.g., see https://astral.sh/blog/uv-security-advisory-cve-2025-54368 and https://about.gitlab.com/blog/how-to-exploit-parser-differentials/
The exploitable security issue is mainly obfuscation of the payload, i.e., you scan for malicious content in a ZIP file with one ZIP parser (let's say a Rust one) then unpack it in production with another (e.g., the Python one) and get malicious content that the other parser did not see.
The ZIP specification is ambiguous enough that these can be contrived and impossible to eliminate, but they are security issues.
sparrowt commentedon Oct 15, 2025
I see, thanks for explaining 👍
shantanujhalt commentedon Oct 17, 2025
Hello. Am I missing something or this fix is not part of 3.13.8 nor 3.13.9 release? I reviewed release notes and even downloaded 3.13.8 and 3.13.9 but I can't find the fix in them.
serhiy-storchaka commentedon Oct 17, 2025
No, it will only be in 3.13.10.
shantanujhalt commentedon Oct 17, 2025
Thank you for the confirmation. Is there any place I could have found this info myself (as in what release is a certain fix going to be put in)?
serhiy-storchaka commentedon Oct 17, 2025
git logfor tags v3.13.8, v3.13.9 and branch 3.13.Update python-handy-archives to version 0.2.0 / rev 5 via SR 1311959
sparrowt commentedon Oct 22, 2025
https://docs.python.org/3.13/whatsnew/changelog.html is the easiest way to check IMO (search for 139700 and see it's under "Python next" and not one of the released versions below)
cosmicexplorer commentedon Oct 28, 2025
I have implemented the zip specification multiple times in both rust and python as well as sharing logic across both languages via pyo3. I am currently working on a self-bootstrapping installer format which can run on python 2 or 3. I specifically implemented the rust zip crate's current approach which uses SIMD instructions to parse for the directory record zip-rs/zip2#93. I began to add support for a builtin native module in cpython which adds some very basic SIMD byte search operations beyond the memchr loops in fastsearch.h with a conditional check for sse/avx support in the configure script.
But I have spent a ridiculous amount of time trying to understand the proposed ambiguity Astral is referring to, and I also don't understand why the comparison is made between "python" vs "rust" zip parsing, when my implementations for pip pypa/pip#12208 as well as the rust zip crate that everyone actually uses have never exhibited this problem. I truly do not understand which numeric offset value is being described as ambiguously relative vs absolute and it makes Astral's post completely useless for other implementors to understand how to avoid the same mistake. There are no outstanding patent claims on the format and I find the claim of legal uncertainty making it impossible for Astral to quote the ambiguity to be specious at best.
I also do not understand why Astral's post describes metadata streaming as specific to uv:
I personally developed this technique for pip several years ago, spent an incredible amount of time iterating in response to community feedback, and eventually gave an entire conference talk about the experience, including how it led to PEP 658 as a long-term way to avoid the need to rely on zip file hacks: https://web.archive.org/web/20250425181258/https://cfp.packaging-con.org/2023/talk/hpuhu7/ The poetry maintainer had some great review feedback for pip on performance benchmarking the zip streaming technique after we met in person at that conference. PEP 777 describes the use of HTTP range requests like this as a standard technique. I would be delighted to put together a seminar or tech talk on this subject if your employer would ever be interested.
Seth Larson's white paper https://alpha-omega.dev/wp-content/uploads/sites/22/2025/10/ao_wp_102725a.pdf mentions fuzzing—I have also recently refactored the rust zip crate's fuzzing harness: zip-rs/zip2#437 and would love to know if Astral is doing anything differently, or why they're not using the normal zip crate. I spent multiple months in 2023 investigating using rust's async model for zip extraction and decompression and could never produce anything approaching the performance of explicit threading (I now understand the reason for this). The maintainer is super great to work with too! So I don't understand why Astral is portraying this as a general problem with the zip format rather than a mistake made by a single implementation of the format which is not the standard rust crate. Are there benchmarks I'm missing?
I am currently working on a C++ version of my zip file work, under the hope that it could be made portable enough to consume in CPython (via pkg-config, maybe vendored like sre). @serhiy-storchaka I would love to know what kinds of expectations/requirements the cpython project has for external dependencies and/or if this is unlikely to be a useful contribution to the project.
zanieb commentedon Oct 28, 2025
I don't understand the motivation for an extensive attack on my comment / Astral — I was merely giving context on why parser differentials are considered a security issue. I'm happy to discuss the uv blog post and ZIP parser hardening, but this issue does not seem like an appropriate place to do so. Feel free to open an issue in the uv tracker or email me (zanie at astral.sh) instead.
serhiy-storchaka commentedon May 18, 2026
There was also an older related issue #126834.
[CVE-2025-8291] check consistency of the zip64 end of central directo…
[CVE-2025-8291] check consistency of the zip64 end of central directo…
[CVE-2025-8291] check consistency of the zip64 end of central directo…
[CVE-2025-8291] check consistency of the zip64 end of central directo…