Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): approximate quantiles #9881

Merged
merged 2 commits into from
Aug 23, 2024
Merged

Conversation

jcrist
Copy link
Member

@jcrist jcrist commented Aug 20, 2024

This adds a new approx_quantile method for computing approximate quantiles. Like the corresponding exact version, it can take:

  • A single quantile (col.approx_quantile(0.5)), returning a single scalar result
  • An array of quantiles (col.approx_quantile([0.25, 0.75])), returning an array of results

Unlike col.quantile, this is only implemented for numeric types (for now). Most backends support only numeric values for the approximate version, as the algorithms don't easily adapt to non-numeric types. If needed, we could move this to be on a generic type, but most implementations would error for other dtypes.

Fixes #9541.

@jcrist jcrist added feature Features or general enhancements ci-run-cloud Add this label to trigger a run of BigQuery, Snowflake, and Databricks backends in CI labels Aug 20, 2024
@ibis-docs-bot ibis-docs-bot bot removed the ci-run-cloud Add this label to trigger a run of BigQuery, Snowflake, and Databricks backends in CI label Aug 20, 2024
Copy link
Member

@cpcloud cpcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, a couple of questions!

ibis/backends/sql/compilers/snowflake.py Show resolved Hide resolved
ibis/expr/operations/reductions.py Show resolved Hide resolved
@cpcloud cpcloud added this to the 9.4 milestone Aug 22, 2024
@jcrist jcrist merged commit dcdb7a7 into ibis-project:main Aug 23, 2024
82 checks passed
@jcrist jcrist deleted the approx-quantiles branch August 23, 2024 19:03
cpcloud pushed a commit that referenced this pull request Aug 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(bigquery): general approx_quantile similar to approx_median but for arbitrary quantile
2 participants