Skip to content

Dataset.open fails with PermissionError when the recorded absolute .svar path is unreadable, instead of falling back to the sibling .svar #335

Description

@d-laub

Summary

_resolve_svar walks a fallback chain (overridelink.relative_pathlink.absolute_path → sibling *.svar), but the link.absolute_path probe can raise PermissionError instead of returning False, aborting the walk before the correct sibling fallback is ever tried.

This makes a .gvl dataset unopenable on any machine other than the one that built it, whenever the recorded absolute path happens to live under a directory the current user cannot stat.

Reproducer

A .gvl built inside a container recorded its genotypes store as /root/gvf-brca/data/gvl/gdc_wgs_DR45.brca.svar. On a normal multi-user host, /root is mode 0700:

from pathlib import Path
p = Path("/root/gvf-brca/data/gvl/gdc_wgs_DR45.brca.svar")
p.is_dir()
# PermissionError: [Errno 13] Permission denied: '/root/gvf-brca/data/gvl/gdc_wgs_DR45.brca.svar'

So gvl.Dataset.open("F2Mb_W16Mb.gvl") raises PermissionError even though the correct .svar is sitting right next to the .gvl and the sibling fallback would have found it immediately.

Root cause

python/genvarloader/_dataset/_svar_link.py:

    if link is not None:
        rel = (gvl_path / link.relative_path).resolve()
        if rel.is_dir():          # <- can also raise
            return rel
        absp = Path(link.absolute_path)
        if absp.is_dir():         # <- raises PermissionError, aborting the chain
            return absp

    siblings = sorted(gvl_path.parent.glob("*.svar"))   # never reached

The code reads as if is_dir() is total, but it is not: pathlib only swallows ENOENT/ENOTDIR/EBADF/ELOOP (pathlib._ignore_error). EACCES propagates.

Why this is easy to miss

The behavior is Python-version dependent. pathlib widened exists()/is_dir() to swallow OSError broadly in 3.13, so:

Python Path('/root/...').is_dir()
3.11.15 raises PermissionError
3.14.6 returns False

Both measured on the same host, same path. genvarloader declares requires-python = ">=3.10", so the whole 3.10–3.12 range is affected while a 3.13+ developer machine looks fine.

Suggested fix

Every probe in the chain should be total — a path that cannot be stat'd is simply not a hit, and the walk should continue to the next candidate:

def _is_dir(p: Path) -> bool:
    """Total is_dir: an unstat-able candidate is a miss, not an error.

    Path.is_dir() only swallows ENOENT/ENOTDIR/EBADF/ELOOP on Python < 3.13;
    EACCES propagates. A recorded absolute path under someone else's $HOME or
    under /root must fall through to the next candidate, not abort resolution.
    """
    try:
        return p.is_dir()
    except OSError:
        return False

and use it for rel, absp, and (with the existing error message) the override probe. Note the override case should still report the failure rather than fall through silently — but it should report "not a usable directory", not leak an errno 13 traceback.

The .resolve() on the relative candidate is a second, smaller instance of the same issue: on Python < 3.13 Path.resolve(strict=False) can also raise on an unreadable intermediate component.

Workaround

Symlink the recorded absolute path, or pass svar= to Dataset.open(...). Neither is discoverable from the traceback, which points at is_dir and looks like a permissions problem with the dataset rather than a resolution-order bug.

Suggested test

def test_resolve_svar_falls_through_an_unreadable_absolute_path(tmp_path):
    gvl_path = tmp_path / "ds.gvl"
    gvl_path.mkdir()
    sibling = tmp_path / "real.svar"
    sibling.mkdir()

    unreadable = tmp_path / "locked"
    unreadable.mkdir(mode=0o000)
    try:
        link = SvarLink(
            relative_path="../nope.svar",
            absolute_path=str(unreadable / "gone.svar"),
            fingerprint=...,
        )
        assert _resolve_svar(gvl_path, link, None) == sibling
    finally:
        unreadable.chmod(0o700)

(skip as root, where the mode is not enforced)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions