[Parquet][Python][Dataset] Mixed FileInfo and path sources can crash Python in dataset() #53033

Describe the bug, including details regarding any error messages, version, and platform.

Describe the bug

pyarrow.dataset.dataset() behaves inconsistently when its source list contains both filesystem paths and pyarrow.fs.FileInfo objects.

Both homogeneous input types work correctly. However, mixed inputs behave differently depending on their order:

• [path, FileInfo] raises a normal TypeError.
• [FileInfo, path] terminates the Python process.

The crash occurs instead of a safe Python exception.

Reproduction

import pathlib
import subprocess
import sys
import tempfile
import textwrap

import pyarrow as pa
import pyarrow.parquet as pq

child_code = textwrap.dedent("""
    import sys
    import pyarrow.dataset as ds
    import pyarrow.fs as fs

    path, case = sys.argv[1:3]
    info = fs.LocalFileSystem().get_file_info(path)

    sources = {
        "paths_only": [path, path],
        "infos_only": [info, info],
        "path_first": [path, info],
        "info_first": [info, path],
    }[case]

    print("Starting:", case, flush=True)

    try:
        result = ds.dataset(
            sources, format="parquet"
        ).to_table().to_pydict()
        print("Result:", result, flush=True)
    except Exception as exc:
        print("Exception:", type(exc).__name__, str(exc), flush=True)
""")

print("Python:", sys.version.split()[0])
print("PyArrow:", pa.__version__)
print("Platform:", sys.platform)

with tempfile.TemporaryDirectory() as directory:
    path = pathlib.Path(directory) / "tiny.parquet"
    pq.write_table(pa.table({"x": [1]}), path)

    for case in ("paths_only", "infos_only", "path_first", "info_first"):
        run = subprocess.run(
            [sys.executable, "-X", "faulthandler", "-c",
             child_code, str(path), case],
            capture_output=True,
            text=True,
            timeout=30,
        )
        print(case, "returncode:", run.returncode)
        print("stdout:", run.stdout.strip())
        print("stderr:", run.stderr.strip())

Actual behavior

Observed with PyArrow 23.0.1 on Kaggle/Linux:

paths_only: returncode 0
infos_only: returncode 0
path_first: TypeError: not a path-like object
info_first: returncode -11
Fatal Python error: Segmentation fault

The same four-case assessment was repeated with PyArrow 26.0.0, where both homogeneous cases succeeded, path-first raised TypeError, and FileInfo-first terminated the subprocess.

The behavior was also reproduced locally on Windows with PyArrow 16.1.0 and 24.0.0.

Expected behavior

Mixing file paths and FileInfo objects should either:

1. Be supported consistently in both orders; or
2. Raise a clear Python exception in both orders.

Invalid or unsupported source combinations should not terminate the interpreter.

Possible root cause

In python/pyarrow/dataset.py, the public dataset() function accepts lists containing either paths or FileInfo objects, but the downstream _filesystem_dataset() logic dispatches based on the first element.

When the first element is FileInfo, the native FileSystemDatasetFactory receives a mixed list. Its Cython path assumes the remaining entries are also FileInfo objects, which appears to cause the native crash.

A possible minimal fix is to reject mixed lists safely before entering native code. Alternatively, mixed inputs could be normalized if supporting them is the intended API contract.

I'd appreciate maintainer guidance on the preferred behavior.

Environment

• Kaggle/Linux, Python 3.13.15, PyArrow 23.0.1
• Kaggle/Linux, PyArrow 26.0.0
• Windows, PyArrow 16.1.0 and 24.0.0

The investigation and initial report drafting were AI-assisted. I independently ran and reviewed the Kaggle reproductions.

Component(s)

Python, Parquet