Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set up criterion/asv benchmarks? #215

Open
bluegenes opened this issue Feb 15, 2024 · 2 comments
Open

set up criterion/asv benchmarks? #215

bluegenes opened this issue Feb 15, 2024 · 2 comments

Comments

@bluegenes
Copy link
Contributor

bluegenes commented Feb 15, 2024

..to facilitate comparisons between any utility function changes + prevent regression when developing.

would be good to set up some macro benchmarks too, e.g. using SRR606249.sig.gz in test-data as query...

@bluegenes bluegenes changed the title set up criterion benchmarks for commands? set up criterion/asv benchmarks for commands? Feb 15, 2024
@bluegenes
Copy link
Contributor Author

bluegenes commented Feb 15, 2024

Note, can't use criterion with cdylib-only build. Recommendations online to add rlib, but that doesn't build on my system (arm mac)

ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

could probably get around this - not sure if worthwhile

@bluegenes bluegenes changed the title set up criterion/asv benchmarks for commands? set up criterion/asv benchmarks? Feb 15, 2024
@ctb
Copy link
Collaborator

ctb commented Feb 18, 2024

see also: #214 (comment) which says -

scripts in /home/ctbrown/scratch3/SRR606249-feb13-2024

github repo for scripts: https://github.com/sourmash-bio/2024-benchmark-branchwater-plugin

srun command I used:

srun -p bmh --time=24:00:00 --nodes=1 --cpus-per-task=64 --mem=50GB --pty /bin/bash

I feel like we could probably do them manually before each release easily enough. The 45 minute one (sourmash gather) should not be automated :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants