perf: Start on better Parquet delta decoding

[coastalwhite]

Since Delta Lengths Byte Array will be the default encoding for strings, we want to speed up decoding this and get it up to power with other encodings. This starts the work of implementing collect, sum and skip methods and removing the allocations that it currently does.

[coastalwhite]

I got quite far on this today. And although it does not show any failures in the CI anymore, doing some fuzzing still breaks it. I am hoping to get this fixed tomorrow and get it ready for merging.

[codspeed-hq]

CodSpeed Performance Report

Merging #18049 will not alter performance

_{Comparing coastalwhite:parquet-delta-bytearray (91ce34f) with main (8c19d18)}

Summary

✅ 37 untouched benchmarks

[codecov]

Codecov Report

Attention: Patch coverage is 66.04538% with 419 lines in your changes missing coverage. Please review.

Project coverage is 80.31%. Comparing base (4233d61) to head (91ce34f).
Report is 56 commits behind head on main.

Files	Patch %	Lines
...et/src/parquet/encoding/delta_bitpacked/decoder.rs	74.61%	148 Missing ⚠️
...parquet/src/arrow/read/deserialize/binary/basic.rs	0.00%	118 Missing ⚠️
...rquet/src/parquet/encoding/delta_bitpacked/fuzz.rs	0.00%	44 Missing ⚠️
...olars-parquet/src/arrow/read/deserialize/nested.rs	25.00%	24 Missing ⚠️
...quet/src/arrow/read/deserialize/binary/decoders.rs	14.81%	23 Missing ⚠️
...arquet/src/arrow/read/deserialize/primitive/mod.rs	69.56%	14 Missing ⚠️
...lars-parquet/src/arrow/read/deserialize/binview.rs	90.16%	12 Missing ⚠️
...t/src/parquet/encoding/delta_byte_array/decoder.rs	82.25%	11 Missing ⚠️
...s-parquet/src/parquet/encoding/bitpacked/decode.rs	71.42%	10 Missing ⚠️
...arquet/encoding/delta_length_byte_array/decoder.rs	60.86%	9 Missing ⚠️
... and 4 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #18049      +/-   ##
==========================================
- Coverage   80.32%   80.31%   -0.01%     
==========================================
  Files        1496     1498       +2     
  Lines      197751   198556     +805     
  Branches     2822     2822              
==========================================
+ Hits       158848   159478     +630     
- Misses      38381    38556     +175     
  Partials      522      522              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coastalwhite]

I did quite extensive testing for this PR with many fuzz tests and integration tests against pyarrow's parquet writer. While I cannot guarantee that it is bug free, I am pretty confident it is close to bug free.

[ritchie46]

Alright!