Python: Support date32 and decimal statistics in PyArrow writer

[wjones127]

Description

In PyArrow, date32 and decimal statistics from parquet metadata aren't translated into appropriate type, but are instead raw bytes. We can either decode them ourselves, or wait for PyArrow to implement that conversion. Upstream issue: https://issues.apache.org/jira/browse/ARROW-7350

Use Case

Related Issue(s)

• Python PyArrow Dataset Writer #542

[wjones127]

The upstream issue is solved and will be released soon in PyArrow 8.0.0.

We can modify the conditional to also accept date32 and decimal columns if int(pyarrow.__version__.split('.', 1)[0]) >= 8 in here:

delta-rs/python/deltalake/writer.py

Lines 277 to 280 in f48489e

	#
	if logical_type not in ["STRING", "INT", "TIMESTAMP", "NONE"]:
	continue
	# import pdb; pdb.set_trace()

Also should remove the import pdb; pdb.set_trace() 🤦

[Bernolt]

I will try to pick this up.

[wjones127]

Oh sorry @Bernolt I think we've already addressed this in #659