[Data] Dataset.write_parquet(**arrow_parquet_args) not work

[bbtfr]

What happened + What you expected to happen

ray.data.Dataset.write_parquet(**arrow_parquet_args)

arrow_parquet_args does not work anymore, since it has not passed to pq.ParquetWriter here

ray/python/ray/data/datasource/parquet_datasink.py

Line 75 in d9e795c

with pq.ParquetWriter(file, schema) as writer:

And so does arrow_parquet_args_fn

Versions / Dependencies

Ubuntu 22.04 LTS
Python 3.10.14
Ray 2.22.0

Reproduction script

ray.data.read_parquet(
    input_path, 
    columns=columns,
).write_parquet(
    output_path, 
    compression="ZSTD", 
    compression_level=100,
)

Issue Severity

High: It blocks me from completing my task.

[raulchen]

thanks for reporting. would you like to submit a fix? I can review the PR.

[MaxVanDijck]

@raulchen I have opened a PR to resolve #45772