perf: Ensure better chunk sizes

[ritchie46]

If a chunk in a DataFrame was 100k rows and we split at 100k - 1 we wrote row-groups of 1, which is terrible. This ensures we keep reasonable sized chunks.

[codecov]

Codecov Report

Attention: Patch coverage is 83.92857% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 80.92%. Comparing base (cb9db1c) to head (504701d).
Report is 2 commits behind head on main.

❗ Current head 504701d differs from pull request most recent head c690aa3. Consider uploading reports for the commit c690aa3 to get more accurate results

Files	Patch %	Lines
crates/polars-core/src/utils/mod.rs	82.35%	6 Missing ⚠️
crates/polars-core/src/frame/mod.rs	50.00%	1 Missing ⚠️
crates/polars-parquet/src/parquet/write/stream.rs	0.00%	1 Missing ⚠️
...es/polars-pipe/src/executors/sinks/group_by/ooc.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #16071      +/-   ##
==========================================
- Coverage   80.97%   80.92%   -0.06%     
==========================================
  Files        1386     1386              
  Lines      178380   178445      +65     
  Branches     3059     3075      +16     
==========================================
- Hits       144448   144410      -38     
- Misses      33442    33545     +103     
  Partials      490      490              

Flag	Coverage Δ
python	74.47% <82.14%> (-0.06%)	⬇️
rust	78.12% <83.92%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ritchie46]

@alexander-beedie the benchmarks fails because it cannot find torch. :/

[alexander-beedie]

@alexander-beedie the benchmarks fails because it cannot find torch. :/

@ritchie46: This should fix it; added the new reqs to bench/coverage jobs (#16072).