You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Python] DataFrame interchange: from_dataframe rejects every pandas float16 column #51696
Describe the bug, including details regarding any error messages, version, and platform.
pa.interchange.from_dataframe raises NotImplementedError for a pandas float16 column, with or without a NaN in it. pandas reports NaN as the null kind for every float column, and validity_buffer_nan_sentinel in python/pyarrow/interchange/from_dataframe.py:549 refuses float16 with that null kind. The comment there says pyarrow.compute.is_nan has no float16 kernel. That kernel arrived in #46866 (GH-45083), so the guard is stale.
$ python -c 'import numpy as np, pandas as pd, pyarrow as pa
from pyarrow.interchange import from_dataframe
print(from_dataframe(pd.DataFrame({"h": np.array([1.5], dtype="<f2")})))'
NotImplementedError: (<DtypeKind.FLOAT: 2>, 16, 'e', '=') with 1 is not yet supported.
# pyarrow 26.0.0.dev350 nightly wheel, pandas 3.2.0.dev0, CPython 3.12, Linux x86_64
With the float16 branch removed, so the NaN mask goes through pc.is_nan like float32 and float64:
$ python -m pytest -q pyarrow/tests/interchange/
# main e85e181
3045 passed, 2 skipped
# branch removed, test_pandas_to_pyarrow_float16_with_missing changed to expect [0, None, 2] as float16
3045 passed, 2 skipped
# that test change alone, from_dataframe.py at main
1 failed, 3044 passed, 2 skipped
The test test_pandas_to_pyarrow_float16_with_missing pins the old NotImplementedError, so it changes with the fix.
Component(s)
Python
I saw #49629. Until the protocol is removed, pa.table(df) imports the same column:
$ python -c 'import numpy as np, pandas as pd, pyarrow as pa
print(pa.table(pd.DataFrame({"h": np.array([1.5, np.nan], dtype="<f2")})))'
pyarrow.Table
h: halffloat
----
h: [[1.5,null]]
# pyarrow 26.0.0.dev350 nightly wheel, pandas 3.2.0.dev0
I have the patch, 14 lines added and 20 removed across those two files. Is it worth opening against this issue, or would you rather close this against #49629?
No, there is no use case behind it. I found the stale guard by reading from_dataframe.py, and as the issue shows, pa.table(df) already imports a float16 column with NaN.
I will close this against #49629 unless you want it kept open for #51716 from @amanmaurya92, which targets this issue.
Activity
amanmaurya92 commentedon Oct 4, 2026
I am up for this :) please assign it to me. I am opening a pr
jorisvandenbossche commentedon Oct 6, 2026
Note that we are planning to deprecate the interchange protocol (#49629, as you linked), so I am not sure this is worth fixing.
Is there a specific use case you are using it for, where using
pa.table(df)does not work?chrikrah commentedon Oct 10, 2026
No, there is no use case behind it. I found the stale guard by reading
from_dataframe.py, and as the issue shows,pa.table(df)already imports afloat16column with NaN.I will close this against #49629 unless you want it kept open for #51716 from @amanmaurya92, which targets this issue.