Repository navigation
Don't treat directories such as archive.zip_ as zip files - #2086
Conversation
pydevd_file_utils looked for ".zip" (or ".egg") anywhere in a path to find the archive a file is in, so a directory named archive.zip_, foo.zipx or bar.egg-info matched too. _apply_func_and_normalize_case() turned C:\repro\archive.zip_\files\script.py into c:\repro\archive.zip\_\files\script.py, breakpoints in that file were reported as "Breakpoint in file that does not exist." and the client could not open the source. exists() did the same lookup, so a missing file under archive.zip_ could be found in an unrelated archive.zip. Both now use _get_zip_path_end(), which only accepts the extension when it ends a path segment: it must be followed by "/", "\" or the end of the string, optionally after the "!" that the existing code already handles for archives such as "x.egg!". Later matches are still checked, so a real archive after such a directory is still found. Fixes microsoft#1964
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
| for ext in (".zip", ".egg"): | ||
| ind = filename.find(ext) |
There was a problem hiding this comment.
Could we scan path components left-to-right, considering .zip and .egg together, rather than searching the whole path for .zip first? For C:\base\lib.egg\pkg\data.zip\mod.py, with lib.egg as the real archive and pkg/data.zip/mod.py as its member, this selects lib.egg\pkg\data.zip as the archive. Python can import that module, but exists() returns false and the breakpoint is rejected. Once we find the archive component, the remaining member path should be treated as opaque, even if it contains another .zip or .egg component. This limitation predates the PR, but the shared helper seems like the right place to address it.
There was a problem hiding this comment.
Done in 1d42f0a. _get_zip_path_end() now checks .zip and .egg names together from left to right, and the first archive it accepts ends the search, so lib.egg\pkg\data.zip\mod.py splits after lib.egg and pkg\data.zip\mod.py stays as the member path. test_zip_path_inside_archive_is_not_split_again covers it with a real egg.
| if end == len(filename) or filename[end] in ("/", "\\"): | ||
| return ind |
There was a problem hiding this comment.
Could we skip a candidate when it is known to be a directory, rather than accepting it solely because the extension ends a component? For C:\base\outer.zip\lib.zip\pkg\mod.py, where outer.zip is an ordinary directory and lib.zip is the real archive, this returns the end of outer.zip. exists() then tries to open that directory as a ZIP and returns false, even though Python can import the module from lib.zip. This is also an existing limitation, not a new regression. I'd like the helper to continue to the next candidate in this case, while retaining a lexical fallback for unavailable/synthetic paths. Archive opening and validation should stay in exists(), not the normalization path; the existing ! handling and member-path case should remain unchanged.
There was a problem hiding this comment.
Done in the same commit. A candidate that os.path.isdir() reports as a directory is skipped, so outer.zip\lib.zip\pkg\mod.py resolves to lib.zip, and paths that aren't on disk still split at the first archive name. Archive opening stays in exists() and the ! handling is unchanged; test_zip_path_skips_directory_with_archive_name covers your example. One side effect: a real directory named like an archive with no archive under it (an unpacked foo.egg, say) is now normalized like any other path.
_get_zip_path_end() now checks .zip and .egg names together, from left to right, and the first archive found ends the search. A path such as lib.egg/pkg/data.zip/mod.py is split after lib.egg and pkg/data.zip/mod.py is the path inside it. A segment which is a directory on disk, such as outer.zip in outer.zip/lib.zip/pkg/mod.py, is skipped so that lib.zip is used. Paths which aren't on disk are still split at the first archive name.
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Fixes #1964
To support code imported from zip files,
pydevd_file_utilslooks for.zip(or.eggwhen there is no.zip) anywhere in a path and treats what comes before it as the archive. Nothing checks that the extension ends a path segment, so a normal directory such asarchive.zip_,foo.zipx,x.zip.bakorbar.egg-infomatches too._apply_func_and_normalize_case()then joins the two halves with a separator, which turnsC:\repro\archive.zip_\files\app\script.pyintoC:\repro\archive.zip\_\files\app\script.py. That is the path shown in the issue: the client can't open the source, and breakpoints come back as "Breakpoint in file that does not exist."exists()makes the same split, so for a missing file underarchive.zip_it can open an unrelatedarchive.zipnext to it and look the rest of the path up inside it.Both functions now get the split point from a small helper,
_get_zip_path_end(). It accepts.zip/.eggonly when the next character is/,\or the end of the string, optionally after the!that both functions already handle for archives named likex.egg!. When a match doesn't qualify it keeps looking further along the path, soC:\work\foo.zipx\lib.zip\pkg\mod.pyfindslib.zip..zipis still tried before.egg, and paths inside real archives are split exactly as before. One deliberate change: a!now counts only when it ends the segment, so a directory such asx.zip!oldis no longer read as the archivex.zip!.This is the same bug as fabioz/PyDev.Debugger#265 (
.zipx). I can send the change to PyDev.Debugger as well if that's useful.Tests, in pydevd's
tests_python:test_convert_utilities.py:test_zip_like_names_are_not_splitchecks thatarchive.zip_,foo.zipx,x.zip.bak,x.zip!old,bar.egg_infoandbar.egg-infoare left alone, with/and\separators.test_zip_paths_split_after_archive_namechecks thata.zip/b.py,a.zip\b.py,a.egg,a.zip!anda.egg!paths are still split after the archive name, that a path ending in the archive isn't split, and that a real archive after a zip-like directory is still found.test_zip_like_directory_namescreates such directories on disk and checksget_abs_path_real_path_and_base_from_file(),exists()andmap_file_to_client().test_exists_does_not_look_into_zip_with_similar_namecovers theexists()case above. All 19 new cases fail onmain; the file passes with this change (29 passed, 4 skipped).test_debugger_json.py:test_breakpoint_in_zip_like_directorylaunches a script fromarchive.zip_/files/, sets a breakpoint and checks that it is verified and hit and that the stack frame has the real path. Onmainthe breakpoint isn't verified ("Breakpoint in file that does not exist.").I also ran the full
tests_pythonsuite on Windows 11 with CPython 3.13.2. The only failures fail the same way onmainin my environment (trio not installed, attach to PID without the native binaries, symlinks without the needed privilege, and a few timing-sensitive tests). Fromtests/debugpy,test_path_mapping.py,test_source_mapping.py,test_breakpoints.pyandtest_justmycode.pypass (95 tests).ruff check .passes.