Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BENCHMARKING] Implement asv benchmarks similar to scikit-learn #27

Open
adam2392 opened this issue Feb 6, 2023 · 0 comments · Fixed by #124
Open

[BENCHMARKING] Implement asv benchmarks similar to scikit-learn #27

adam2392 opened this issue Feb 6, 2023 · 0 comments · Fixed by #124
Labels
unit test Creating unit test for existing code

Comments

@adam2392
Copy link
Collaborator

adam2392 commented Feb 6, 2023

Setup and Work Needed

Currently on main, we have a few files that should serve as a boiler-plate for setting up the ASV benchmarking suite: https://github.com/neurodata/scikit-tree/blob/main/asv.conf.json, https://github.com/neurodata/scikit-tree/tree/main/benchmarks.

What really is needed is:

i) an end-to-end demonstration of asv benchmarking on a PR branch vs say main showing that our setup is correct
ii) a documentation for how to run the ASV benchmarks
iii) [nice-to-have] if it is possible to have a simple Docker container that runs the ASV benchmarks that we can launch w/ a very simple command on any Linux-computer, that would be nice. The Docker container should probably have some fixed-amount of RAM, and CPU resources assigned to it too.

Experiments to include

Prolly use similar simulation setup they have and Increase feature dimensionality and sample dimensionality and run a separate benchmark for each tree/forest method.

We can leverage these results to eventually compare against what is in sklearn.

@adam2392 adam2392 added the unit test Creating unit test for existing code label Feb 7, 2023
@adam2392 adam2392 reopened this Oct 31, 2023
@adam2392 adam2392 changed the title Implement asv benchmarks similar to scikit-learn [BENCHMARKING] Implement asv benchmarks similar to scikit-learn Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
unit test Creating unit test for existing code
Projects
None yet
1 participant