[C++][Parquet] Parquet writer supports writing int32/int64 for decimal type

[wgtmac]

Describe the enhancement requested

As the parquet specs states below, decimal types with small precision can use int32/int64 physical types.

DECIMAL can be used to annotate the following types:

- int32: for 1 <= precision <= 9
- int64: for 1 <= precision <= 18; precision < 10 will produce a warning
- fixed_len_byte_array: precision is limited by the array size. Length n can store <= floor(log_10(2^(8*n - 1) - 1)) base-10 digits
- binary: precision is not limited, but is required. The minimum number of bytes to store the unscaled value should be used.

The aim of this issue is to provide a writer option to write decimal types using int32 when 1 <= precision <= 9 and int64 when 10 <= precision <= 18.

Component(s)

C++, Parquet

[wgtmac]

I will work on it shortly. cc @emkornfield @pitrou

[alippai]

When talking about datasets (multiple parquet files) are the mixed physical types supported? Some files written using the old way, some files with the improved physical type.

[wjones127]

When talking about datasets (multiple parquet files) are the mixed physical types supported? Some files written using the old way, some files with the improved physical type.

The physical type does not change the logical type in the Parquet file, just how the data is serialized. Datasets shouldn't care about the Parquet physical type; it should only care about the logical one.

[alippai]

🥳 thanks!