Repository navigation
[Python][Parquet] Check write_table options are accepted by dataset writer #51220
Description
Describe the enhancement requested
Some ParquetWriter options accepted by pq.write_table are not registered in ParquetFileWriteOptions, so pq.write_to_dataset and ds.write_dataset reject them:
write_time_adjusted_to_utcGH-47441: [Python][Parquet] Allow passing write_time_adjusted_to_utc to Python's ParquetWriter #47745bloom_filter_optionsGH-49376: [Python][Parquet] Add ability to write Bloom filters from pyarrow #49377 (also missing inParquetWriter.__init__)
pq.write_to_dataset(table, path, bloom_filter_options={'a': True})Output
File "_/python/pyarrow/parquet/core.py", line 2279, in write_to_dataset
write_options = parquet_format.make_write_options(**kwargs)
pyarrow._dataset_parquet.ParquetFileFormat.make_write_options
(<ParquetFileWriteOptions> opts).update(**kwargs)
File "pyarrow/_dataset_parquet.pyx", line 628, in pyarrow._dataset_parquet.ParquetFileWriteOptions.update
raise TypeError("unexpected parquet write option: " + name)
TypeError: unexpected parquet write option: bloom_filter_optionsstore_schemacommit 56b90a4
Similar to use_content_defined_chunking getting added in #47499 / #47498
See also implementor's note
A test that every option in inspect.signature(pq.write_table).parameters is accepted by ParquetFileFormat.make_write_options may be considered to prevent this in the future.
Component(s)
Python
Activity
KHARSHAVARDHAN-eng commentedon Sep 9, 2026
Hi @tadeja, I’d be interested in working on this issue. Is it still available for someone to pick up?
[-][Python] Check `write_table` options are accepted by dataset writer[/-][+][Python][Parquet] Check `write_table` options are accepted by dataset writer[/+]rok commentedon Sep 29, 2026
Issue resolved by pull request 51461
#51461
apacheGH-51220: [Python][Parquet] Accept write_table options in datas…