Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up benchmarking using touchstone #3

Open
kaitejohnson opened this issue Jan 28, 2025 · 1 comment
Open

Set up benchmarking using touchstone #3

kaitejohnson opened this issue Jan 28, 2025 · 1 comment

Comments

@kaitejohnson
Copy link
Collaborator

kaitejohnson commented Jan 28, 2025

Feature

We would like to set up performance benchmarking in CI using touchstone, consistent with the other epinowcast org packages

Preferred solution

This will require writing actual package functions and comparing to some baseline performance, as in the example here from primarycensoreddist. This will take some more thought but could be something as simple as scoring a nowcast from the package and comparing it to a naive multiplicative factor nowcast that assumes some set delay distribution.

Alternatives

We could also do this with a vignette, similar to the synthetic_validation workflow in EpiNow2 but its probably better to use CI for at least some simple functions and consider a more complicated workflow for a vignette.

@seabbs
Copy link
Collaborator

seabbs commented Jan 30, 2025

I would split this into sub issues for the infra and content

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants