You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Parquet reader: fall back to positional field IDs when a file has no field IDs and no name mapping #1046
#1044 adds name mapping support to the Parquet reader. When a Parquet file has no field IDs and no name mapping is provided, the reader returns InvalidSchema, which matches the current C++ Avro reader (avro_reader.cc).
The Java Parquet reader handles this case differently. As pointed out in #1044, ReadConf falls back to addFallbackIds and assigns positional IDs to top-level columns, so such files can still be read.
Proposal
When the Parquet file schema has no field IDs and no name mapping is provided, assign fallback IDs by top-level column position, as Java does, instead of returning an error.
Open questions:
Should the Avro reader get the same fallback for consistency?
Activity