feat: histogram aggregate function

[suimenno3002]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Briefly describe what this PR aims to solve. Include background context that will help reviewers understand the purpose of the PR.

• Feat Feature: histogram aggregate function #14588

This Draft PR is only used to display task disassembly, current progress, and exchange ideas. The quality of the code is currently low. Today I will improve it to a state that can be reviewed.

Tests

• Unit Test
• Logic Test

Type of change

• New Feature (non-breaking change which adds functionality)

Progress

• Migrate doris' equal height histogram implementation and unit test
• make code clean
• license compliance using doris code
• histogram aggre for types covered by RangeIndex::supported_type
• [-] move the equal-weight histogram code into a common path
• [-] discussion on the output format of aggregation results
• add test case for aggregate function, and complete testing
• make histogram of date date_time decimal readable
• docs for histogram aggregate function

For Reviewers

The implementation refers to doris' histogram function, so these may be helpful:

• https://github.com/apache/doris/blob/master/be/src/vec/utils/histogram_helpers.hpp
• https://github.com/apache/doris/blob/master/be/test/vec/utils/histogram_helpers_test.cpp

WIP

---

This change is 

[suimenno3002]

Is there anyway to cast ScalarRef to &Scalar?

[sundy-li]

Is there anyway to cast ScalarRef to &Scalar?

to_owned

[suimenno3002]

Is there anyway to format Scalar to be readable? I think neither Scalar nor ScalarRef implement Display. Or maybe I should change the output format of histogram, from one line string to multi line flat bucket

[suimenno3002]

= =, I found some small improvements. Should I submit a new PR directly or open an issue?

diff --git a/src/query/functions/src/aggregates/aggregate_histogram.rs b/src/query/functions/src/aggregates/aggregate_histogram.rs
index b1935c0a1d..7c0e2c4169 100644
--- a/src/query/functions/src/aggregates/aggregate_histogram.rs
+++ b/src/query/functions/src/aggregates/aggregate_histogram.rs
@@ -145,8 +145,8 @@ where
             &buckets
                 .drain(..)
                 .map(|raw| Bucket {
-                    lower: format_scalar(raw.lower.clone()),
-                    upper: format_scalar(raw.upper.clone()),
+                    lower: format_scalar(raw.lower),
+                    upper: format_scalar(raw.upper),
                     ndv: raw.ndv,
                     count: raw.count,
                     pre_sum: raw.pre_sum,
@@ -352,7 +352,7 @@ fn calculate_bucket_max_values<T: Ord>(value_map: &BTreeMap<T, u64>, num_buckets
     // Assume that the value map is not empty
     debug_assert!(!value_map.is_empty());
 
-    // Calculate the total number of values in the map using std::accumulate()
+    // Calculate the total number of values in the map
     let total_values = value_map.values().sum();
 
     // If there is only one bucket, then all values will be assigned to that bucket
diff --git a/src/query/sql/src/planner/semantic/type_check.rs b/src/query/sql/src/planner/semantic/type_check.rs
index c691f9a450..d11fe9b734 100644
--- a/src/query/sql/src/planner/semantic/type_check.rs
+++ b/src/query/sql/src/planner/semantic/type_check.rs
@@ -1726,7 +1726,7 @@ impl<'a> TypeChecker<'a> {
             } && arg_types[1].is_integer();
             if !is_positive_integer {
                 return Err(ErrorCode::SemanticError(
-                    "The delimiter of `histogram` must be a constant positive int",
+                    "The max_num_buckets of `histogram` must be a constant positive int",
                 ));
             }
 

[BohuTANG]

= =, I found some small improvements. Should I submit a new PR directly or open an issue?

diff --git a/src/query/functions/src/aggregates/aggregate_histogram.rs b/src/query/functions/src/aggregates/aggregate_histogram.rs
index b1935c0a1d..7c0e2c4169 100644
--- a/src/query/functions/src/aggregates/aggregate_histogram.rs
+++ b/src/query/functions/src/aggregates/aggregate_histogram.rs
@@ -145,8 +145,8 @@ where
             &buckets
                 .drain(..)
                 .map(|raw| Bucket {
-                    lower: format_scalar(raw.lower.clone()),
-                    upper: format_scalar(raw.upper.clone()),
+                    lower: format_scalar(raw.lower),
+                    upper: format_scalar(raw.upper),
                     ndv: raw.ndv,
                     count: raw.count,
                     pre_sum: raw.pre_sum,
@@ -352,7 +352,7 @@ fn calculate_bucket_max_values<T: Ord>(value_map: &BTreeMap<T, u64>, num_buckets
     // Assume that the value map is not empty
     debug_assert!(!value_map.is_empty());
 
-    // Calculate the total number of values in the map using std::accumulate()
+    // Calculate the total number of values in the map
     let total_values = value_map.values().sum();
 
     // If there is only one bucket, then all values will be assigned to that bucket
diff --git a/src/query/sql/src/planner/semantic/type_check.rs b/src/query/sql/src/planner/semantic/type_check.rs
index c691f9a450..d11fe9b734 100644
--- a/src/query/sql/src/planner/semantic/type_check.rs
+++ b/src/query/sql/src/planner/semantic/type_check.rs
@@ -1726,7 +1726,7 @@ impl<'a> TypeChecker<'a> {
             } && arg_types[1].is_integer();
             if !is_positive_integer {
                 return Err(ErrorCode::SemanticError(
-                    "The delimiter of `histogram` must be a constant positive int",
+                    "The max_num_buckets of `histogram` must be a constant positive int",
                 ));
             }
 

Cool, only a new PR is ok.