Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cardinality estimate stat #913

Open
a10y opened this issue Sep 23, 2024 · 4 comments
Open

Add cardinality estimate stat #913

a10y opened this issue Sep 23, 2024 · 4 comments

Comments

@a10y
Copy link
Contributor

a10y commented Sep 23, 2024

Useful for compressor to decide if Dict compression is worthwhile.

There's a Rust crate already implementing it: https://docs.rs/hyperloglogplus/latest/hyperloglogplus/struct.HyperLogLogPlus.html

Can be used:

@lwwmanning
Copy link
Member

I think we want this instead of HLL++: https://www.cidrdb.org/cidr2019/papers/p23-freitag-cidr19.pdf

@lwwmanning
Copy link
Member

lwwmanning commented Sep 23, 2024

(In particular, it gives good estimates of cardinality of arbitrary combinations of attributes rather than just attributes, which is cool / handy for compound join keys)

@lwwmanning
Copy link
Member

if we're taking off the shelf, this crate looks potentially better: https://github.com/cloudflare/cardinality-estimator/tree/main

@robert3005
Copy link
Member

previously mentioned in #85

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants