Skip to content

Don't treat directories such as archive.zip_ as zip files - #2086

Merged
Rich Chiodo (rchiodo) merged 2 commits into
microsoft:mainfrom
Thanatos9404:fix/1964-zip-boundary
Oct 19, 2026
Merged

Rich Chiodo (rchiodo) merged 2 commits into
microsoft:mainfrom
Thanatos9404:fix/1964-zip-boundary

Conversation

@Thanatos9404

Copy link
Copy Markdown
Contributor

Fixes #1964

To support code imported from zip files, pydevd_file_utils looks for .zip (or .egg when there is no .zip) anywhere in a path and treats what comes before it as the archive. Nothing checks that the extension ends a path segment, so a normal directory such as archive.zip_, foo.zipx, x.zip.bak or bar.egg-info matches too. _apply_func_and_normalize_case() then joins the two halves with a separator, which turns C:\repro\archive.zip_\files\app\script.py into C:\repro\archive.zip\_\files\app\script.py. That is the path shown in the issue: the client can't open the source, and breakpoints come back as "Breakpoint in file that does not exist." exists() makes the same split, so for a missing file under archive.zip_ it can open an unrelated archive.zip next to it and look the rest of the path up inside it.

Both functions now get the split point from a small helper, _get_zip_path_end(). It accepts .zip/.egg only when the next character is /, \ or the end of the string, optionally after the ! that both functions already handle for archives named like x.egg!. When a match doesn't qualify it keeps looking further along the path, so C:\work\foo.zipx\lib.zip\pkg\mod.py finds lib.zip. .zip is still tried before .egg, and paths inside real archives are split exactly as before. One deliberate change: a ! now counts only when it ends the segment, so a directory such as x.zip!old is no longer read as the archive x.zip!.

This is the same bug as fabioz/PyDev.Debugger#265 (.zipx). I can send the change to PyDev.Debugger as well if that's useful.

Tests, in pydevd's tests_python:

  • test_convert_utilities.py: test_zip_like_names_are_not_split checks that archive.zip_, foo.zipx, x.zip.bak, x.zip!old, bar.egg_info and bar.egg-info are left alone, with / and \ separators. test_zip_paths_split_after_archive_name checks that a.zip/b.py, a.zip\b.py, a.egg, a.zip! and a.egg! paths are still split after the archive name, that a path ending in the archive isn't split, and that a real archive after a zip-like directory is still found. test_zip_like_directory_names creates such directories on disk and checks get_abs_path_real_path_and_base_from_file(), exists() and map_file_to_client(). test_exists_does_not_look_into_zip_with_similar_name covers the exists() case above. All 19 new cases fail on main; the file passes with this change (29 passed, 4 skipped).
  • test_debugger_json.py: test_breakpoint_in_zip_like_directory launches a script from archive.zip_/files/, sets a breakpoint and checks that it is verified and hit and that the stack frame has the real path. On main the breakpoint isn't verified ("Breakpoint in file that does not exist.").

I also ran the full tests_python suite on Windows 11 with CPython 3.13.2. The only failures fail the same way on main in my environment (trio not installed, attach to PID without the native binaries, symlinks without the needed privilege, and a few timing-sensitive tests). From tests/debugpy, test_path_mapping.py, test_source_mapping.py, test_breakpoints.py and test_justmycode.py pass (95 tests). ruff check . passes.

pydevd_file_utils looked for ".zip" (or ".egg") anywhere in a path to
find the archive a file is in, so a directory named archive.zip_,
foo.zipx or bar.egg-info matched too. _apply_func_and_normalize_case()
turned C:\repro\archive.zip_\files\script.py into
c:\repro\archive.zip\_\files\script.py, breakpoints in that file were
reported as "Breakpoint in file that does not exist." and the client
could not open the source. exists() did the same lookup, so a missing
file under archive.zip_ could be found in an unrelated archive.zip.

Both now use _get_zip_path_end(), which only accepts the extension when
it ends a path segment: it must be followed by "/", "\" or the end of
the string, optionally after the "!" that the existing code already
handles for archives such as "x.egg!". Later matches are still checked,
so a real archive after such a directory is still found.

Fixes microsoft#1964
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Comment on lines +497 to +498
for ext in (".zip", ".egg"):
ind = filename.find(ext)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we scan path components left-to-right, considering .zip and .egg together, rather than searching the whole path for .zip first? For C:\base\lib.egg\pkg\data.zip\mod.py, with lib.egg as the real archive and pkg/data.zip/mod.py as its member, this selects lib.egg\pkg\data.zip as the archive. Python can import that module, but exists() returns false and the breakpoint is rejected. Once we find the archive component, the remaining member path should be treated as opaque, even if it contains another .zip or .egg component. This limitation predates the PR, but the shared helper seems like the right place to address it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 1d42f0a. _get_zip_path_end() now checks .zip and .egg names together from left to right, and the first archive it accepts ends the search, so lib.egg\pkg\data.zip\mod.py splits after lib.egg and pkg\data.zip\mod.py stays as the member path. test_zip_path_inside_archive_is_not_split_again covers it with a real egg.

Comment on lines +504 to +505
if end == len(filename) or filename[end] in ("/", "\\"):
return ind

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we skip a candidate when it is known to be a directory, rather than accepting it solely because the extension ends a component? For C:\base\outer.zip\lib.zip\pkg\mod.py, where outer.zip is an ordinary directory and lib.zip is the real archive, this returns the end of outer.zip. exists() then tries to open that directory as a ZIP and returns false, even though Python can import the module from lib.zip. This is also an existing limitation, not a new regression. I'd like the helper to continue to the next candidate in this case, while retaining a lexical fallback for unavailable/synthetic paths. Archive opening and validation should stay in exists(), not the normalization path; the existing ! handling and member-path case should remain unchanged.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in the same commit. A candidate that os.path.isdir() reports as a directory is skipped, so outer.zip\lib.zip\pkg\mod.py resolves to lib.zip, and paths that aren't on disk still split at the first archive name. Archive opening stays in exists() and the ! handling is unchanged; test_zip_path_skips_directory_with_archive_name covers your example. One side effect: a real directory named like an archive with no archive under it (an unpacked foo.egg, say) is now normalized like any other path.

_get_zip_path_end() now checks .zip and .egg names together, from left to
right, and the first archive found ends the search. A path such as
lib.egg/pkg/data.zip/mod.py is split after lib.egg and pkg/data.zip/mod.py
is the path inside it.

A segment which is a directory on disk, such as outer.zip in
outer.zip/lib.zip/pkg/mod.py, is skipped so that lib.zip is used. Paths
which aren't on disk are still split at the first archive name.
@rchiodo

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@rchiodo
Rich Chiodo (rchiodo) merged commit 671f6c7 into microsoft:main Oct 19, 2026
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Source lookup fails for paths with “.zip_\” segment

2 participants