-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add vignette on benchmarking model options #695
base: main
Are you sure you want to change the base?
Conversation
As an update, the Error when using library(EpiNow2)
library(cmdstanr)
# Set the number of cores to use
options(mc.cores = 4)
# Generation time
generation_time <- Gamma(
shape = Normal(1.3, 0.3),
rate = Normal(0.37, 0.09),
max = 14
)
# Incubation period
incubation_period <- LogNormal(
meanlog = Normal(1.6, 0.05),
sdlog = Normal(0.5, 0.05),
max = 14
)
# Reporting delay
reporting_delay <- LogNormal(
meanlog = 0.5,
sdlog = 0.5,
max = 10
)
# Combine the incubation period and reporting delay into one delay
delay <- incubation_period + reporting_delay
# Observation model options
obs <- obs_opts(
scale = list(mean = 0.1, sd = 0.025),
return_likelihood = TRUE
)
# Run model
epinow(
data = example_confirmed,
generation_time = generation_time_opts(generation_time),
delays = delay_opts(delay),
obs = obs,
horizon = 0,
rt = NULL,
stan = stan_opts(
method = "laplace",
backend = "cmdstanr"
)
)
Error in fit_model_approximate(args, id = id) :
Approximate inference failed due to: Error: 'jacobian' argument to optimize and laplace must match!
laplace was called with jacobian=TRUE
optimize was run with jacobian=TRUE Not an informative error message from After playing around with the example above with different combinations of Moreover, according to the documentation of |
I can't reproduce this - do you need to update My versions are: devtools::package_info("cmdstanr") |>
dplyr::filter(package == "cmdstanr")
#> package * version date (UTC) lib source
#> cmdstanr 0.8.1.9000 2024-06-23 [1] Github (stan-dev/cmdstanr@9878dda)
#>
#> [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
cmdstanr::cmdstan_version()
#> [1] "2.35.0" Created on 2024-07-12 with reprex v2.1.0 |
Thanks. It's fixed now after updating. |
I have now pushed the vignette with the run of all the models using MCMC (a472439). To do:
Settings and errors so farPathfinder errors
|
I noticed something interesting about @sbfnk The only thing in the way of this vignette getting merged is the struggle to get |
From meeting with Seb today:
|
a6f38ba
to
fef8991
Compare
62a974e
to
15ec8e0
Compare
Pull request was converted to draft
I'll be picking up this work this week after a long break. |
The plan as stated above sounds really good - just a couple of comments:
I'd suggest to add the nonmechanistic model.
With this can we really get at the stated intention of highlighting differences between real-time vs. retrospective use? It would make sense to me to do this on Rt and across the whole "estimate" part of the time series (for retrospective) vs. partial data (for real-time) but ignore forecasts as that introduces additional complexity of which model to use (latest vs. predicted Rt). I'd also consider using CRPS in addition to bias/coverage so as to be able to do an overall comparison.
This would be a fantastic outcome! |
Haven't pushed to this PR in a while because I was away. I have some local changes that I'll be pushing soon. Current main challenge is with some stochastic failures when using the approximate methods. If they continue, we may have to revise the scope again. |
Description
This PR closes #629.
Initial submission checklist
devtools::test()
anddevtools::check()
).devtools::document()
).lintr::lint_package()
).After the initial Pull Request