Add cluster key statistics in block meta

[zhyass]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Add cluster key statistics in block meta.

Changelog

• New Feature

Related Issues

Fixes #issue

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
databend	✅ Ready (Inspect)	Visit Preview	May 6, 2022 at 10:37AM (UTC)

[mergify]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[dantengsky]

Not sure if we can define a meaningful total order relation between DataValues.
@sundy-li any comments about this? thanks

[dantengsky]

@sundy-li, shall/can we reuse the impl<S> ChangeIf<S> for CmpMin/Max for the comparession of DataValue

https://github.com/datafuselabs/databend/blob/a5748357bdbe5e5ee694ac6c1d750d1b0fe78e3e/common/functions/src/aggregates/aggregate_scalar_state.rs#L45-L73

[sundy-li]

aggregate_scalar_state is generic comparision impls without matching each DataValue, so it's optimized by performance.

In cluster key, we already have min/max indexes, so we don't need to copy this indexes into a Column and apply eval_aggr.

[dantengsky]

aggregate_scalar_state is generic comparision impls without matching each DataValue, so it's optimized by performance...

got it.

Two other concerns:

1. Although we can define our own total order for DataValue (even for weird things like NaN...)
   but a PartialOrder may be easier to define (or just use rust's derive mechanism)
2. The min/max logic is also implemented by the aggregate_min/max
   It would be better if these two keep in coherent.

A suggestion:

• derive PartialOrd for DataValue

• implements the min/max like the aggregate_scalar_state does
  maybe something like this (hope it is doing the same thing as aggregate min max does:-)

          let min = min_stats
                .iter()
                .filter(|x| !x.is_null())
                .min_by(|x, y| min.partial_cmp(&r).unwrap_or(Ordering::Equal))
                .unwrap_or(DataValue::Null);
  
           let max = max_stats
                .iter()
                .filter(|x| !x.is_null())
                .max_by(|x, y| max.partial_cmp(&r).unwrap_or(Ordering::Equal))
                .unwrap_or(DataValue::Null);
  

[dantengsky]

The logics want to follow:

https://github.com/datafuselabs/databend/blob/6623a328b70d793c6f00bb97807ff9df2d4fc1f2/common/functions/src/aggregates/aggregate_scalar_state.rs#L126-L135

https://github.com/datafuselabs/databend/blob/6623a328b70d793c6f00bb97807ff9df2d4fc1f2/common/functions/src/aggregates/aggregate_scalar_state.rs#L67-L71

[zhyass]

Thanks @dantengsky . This is a great suggestion. I will try it later

[dantengsky]

Yeah, hope it helps.
As long as the format kept backward compatible, I think we could merge this first and polish it later.

[BohuTANG]

There is a CI error:

---- servers::http::clickhouse_handler::test_insert_format_ndjson stdout ----
thread 'servers::http::clickhouse_handler::test_insert_format_ndjson' panicked at 'Cannot compare different types with non-numeric type', common/datavalues/src/data_value.rs:314:13

https://github.com/datafuselabs/databend/runs/6318429474?check_suite_focus=true#step:3:2198

[dantengsky]

/LGTM

[dantengsky]

LGTM