feat(optimizer): Add join reordering as an optimizer rule

[desmondcheongzx]

Applies the naive left deep join order from #3616 as an optimizer rule.
This optimizer rule is gated behind an environment variable that allows us to validate the rule on our current workloads.

Currently join reordering results in errors for 50% of TPC-H queries during join graph building. We'll tackle these in a follow-up PR.

[codspeed-hq]

CodSpeed Performance Report

Merging #3642 will improve performances by 16.8%

_{Comparing desmondcheongzx:apply-join-reorder-optimization (215b815) with main (e6c084f)}

Summary

⚡ 1 improvements
✅ 26 untouched benchmarks

Benchmarks breakdown

	Benchmark	main	desmondcheongzx:apply-join-reorder-optimization	Change
⚡	test_count[1 Small File]	4 ms	3.4 ms	+16.8%

[samster25]

aren't the stats already enriched from the previous batch?

[desmondcheongzx]

Yeah, but since we're creating new operators while join reordering, I re-enrich the new operators. Another possibility is to simply make operators enrich themselves with stats while we perform join reordering

[samster25]

I think it may be worthwhile to refactor this to a OptimizerBuilder where we can call methods to add batches.

let builder = OptimizerBuilder::default()
  .join_reordering()
  .simplify_expressions();

let optimizer = builder.build()

[desmondcheongzx]

Sounds good, done!

[codecov]

Codecov Report

Attention: Patch coverage is 54.73251% with 110 lines in your changes missing coverage. Please review.

Project coverage is 78.02%. Comparing base (2aac957) to head (215b815).
Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...src/optimization/rules/reorder_joins/join_graph.rs	35.36%	53 Missing ⚠️
...l-plan/src/optimization/rules/reorder_joins/mod.rs	0.00%	37 Missing ⚠️
...rc/daft-logical-plan/src/optimization/optimizer.rs	82.95%	15 Missing ⚠️
src/common/daft-config/src/lib.rs	62.50%	3 Missing ⚠️
...l-plan/src/optimization/rules/enrich_with_stats.rs	75.00%	1 Missing ⚠️
...daft-physical-plan/src/physical_planner/planner.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3642      +/-   ##
==========================================
- Coverage   78.09%   78.02%   -0.08%     
==========================================
  Files         721      727       +6     
  Lines       89145    89633     +488     
==========================================
+ Hits        69621    69935     +314     
- Misses      19524    19698     +174     

Files with missing lines	Coverage Δ
src/daft-logical-plan/src/builder.rs	92.30% <100.00%> (+0.07%)	⬆️
src/daft-logical-plan/src/logical_plan.rs	73.63% <100.00%> (+1.38%)	⬆️
.../rules/reorder_joins/naive_left_deep_join_order.rs	95.49% <100.00%> (+0.04%)	⬆️
...l-plan/src/optimization/rules/enrich_with_stats.rs	90.90% <75.00%> (-9.10%)	⬇️
...daft-physical-plan/src/physical_planner/planner.rs	3.37% <0.00%> (ø)
src/common/daft-config/src/lib.rs	81.48% <62.50%> (-2.58%)	⬇️
...rc/daft-logical-plan/src/optimization/optimizer.rs	94.90% <82.95%> (-3.94%)	⬇️
...l-plan/src/optimization/rules/reorder_joins/mod.rs	0.00% <0.00%> (ø)
...src/optimization/rules/reorder_joins/join_graph.rs	88.79% <35.36%> (-5.91%)	⬇️

... and 39 files with indirect coverage changes

[samster25]

typical builder pattern should do the following instead

pub fn reorder_joins(self) -> Self {

so you can chain builder calls

[desmondcheongzx]

Gotcha, fixed

[samster25]

Spoke offline, put join conditions into JoinOrderTree instead since this should just be a 1:1 translation

[desmondcheongzx]

Thanks for the discussion! Moved join condition resolution to join order construction time. Much easier to follow now