IO: Windows paths crash with 'Unrecognized filesystem type in URI: c' #3720
Copy link
Copy link
Closed
Description
Apache Iceberg version
Latest main (also affects all released versions)
Please describe the bug
On Windows, passing a local absolute path (e.g. C:\Users\data\warehouse) to any PyIceberg IO operation crashes with:
from pyiceberg.io.pyarrow import PyArrowFileIO
io = PyArrowFileIO()
f = io.new_input(r"C:\Users\me\warehouse\table\metadata.json")
# ValueError: Unrecognized filesystem type in URI: can example from #1005 is:
from pyiceberg.catalog import load_catalog
catalog = load_catalog(
"my_catalog",
**{
"type": "sql",
"uri": "sqlite:///catalog.db",
"warehouse": r"C:\Users\me\iceberg_warehouse",
},
)
# UserWarning: No preferred file implementation for scheme: cThe root cause: Python's urlparse treats Windows drive letters as URI schemes (C:\... scheme='c'). Since 'c' is not in PyIceberg's scheme-to-FileIO mapping, all local file operations fail.
This affects three independent parse sites:
_infer_file_io_from_scheme()inpyiceberg/io/__init__.pyPyArrowFileIO.parse_location()inpyiceberg/io/pyarrow.pyFsspecFileIO._get_fs_from_uri()inpyiceberg/io/fsspec.py
Related issues
- infra: run tests for windows #2477 (CI: run tests on Windows lists this as the primary failure)
- Cannot specify "file://" uri or direct local file location for warehouse in Windows #1005 (user report closed as stale without fix)
- PR Enhance URI scheme validation for Windows paths #3161 (partial fix only pyarrow, no tests, went stale)
- PR Support execution in Windows using Local File System and NFS #996 (partial fix only pyarrow, went stale)
Proposed fix
A platform-guarded predicate that detects when urlparse has misidentified a Windows drive letter as a URI scheme, and remaps it to 'file':
import sys
def _is_windows_drive_letter(scheme: str) -> bool:
return sys.platform == 'win32' and len(scheme) == 1 and scheme.isalpha()Applied at all three parse sites. Zero behavior change on non-Windows (the platform check short-circuits). Includes platform-conditional tests.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
sidharth-05 commentedon Aug 7, 2026
Hi! I’d like to work on this issue. I plan to add a shared, platform-aware Windows drive-letter path check, apply it consistently at the three parsing sites described here, and add focused tests covering PyArrowFileIO, FsspecFileIO, and FileIO inference without changing non-Windows URI behavior.
If this is still available, could you please assign it to me? Thanks!
qzyu999 commentedon Aug 7, 2026
Hi @sidharth-05, this has been completed as of #3723.