Repository navigation
[C++][Parquet] Be better to check null_count, distinct_count and nan_count are negative when reading files #50692
Description
Describe the enhancement requested
In parquet.thrift, null_count, distinct_count, and nan_count are defined as optional i64, so it is possible to encounter a negative value (whether due to an incorrect file writer implementation, file corruption, or other reasons). We should check for negative values. However, the Parquet format does not specify whether to throw an error or treat such values as "missing" when encountered; I lean towards the latter, as it avoids blocking the reading of other data.
nan_count is not supported now. We can check null_count and distinct_count first, and check nan_count when we implement #50689.
Component(s)
C++, Parquet
Activity
KHARSHAVARDHAN-eng commentedon Jul 28, 2026
Hi @HuaHuaY,
I'd like to work on this enhancement if it's available.
From my understanding, the change involves validating that
null_countanddistinct_countare not negative while reading Parquet metadata, and handling invalid values appropriately instead of blindly accepting them.I'll first inspect the metadata reading path and existing validation logic to ensure the change is consistent with the current error-handling behavior.
Please let me know if there are any implementation preferences or edge cases you'd like me to consider. Thanks!
HuaHuaY commentedon Jul 28, 2026
Hi @KHARSHAVARDHAN-eng,
This is a minor issue I found when I try to implement #50689. And I have a PR to fix it. If you'd like, you can help to review my PR or take a look at other issues in the repository. Sorry for taking up your time, and thank you for wanting to help fix it.
Reranko05 commentedon Jul 29, 2026
Hi @KHARSHAVARDHAN-eng, You've already opened some PRs in the last couple of days and we have already commented on some of those. They have several issues, could you fix them instead of working on new ones? Thanks.
HuaHuaY commentedon Jul 29, 2026
Close this issue. Referring to #50694 (comment), we will not check whether
xxx_countis negative.