Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CmdStanMCMC/MLE/VB from read_cmdstan_csv ouput #412

Merged
merged 19 commits into from
Dec 20, 2020

Conversation

jgabry
Copy link
Member

@jgabry jgabry commented Dec 18, 2020

Submission Checklist

  • Run unit tests
  • Declare copyright holder and agree to license (see below)

Summary

Closes #411

Allows creating CmdStanMCMC, CmdStanVB, and CmdStanMLE objects from the list returned by read_cmdstan_csv().
In reality these objects are not exactly the same because they don't have an associated CmdStanRun object. This means that methods like cmdstan_diagnose() and save_output_files() are unavailable, but the most important methods (e.g., draws(), summary()) work fine.

Copyright and Licensing

Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Columbia University

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

Copy link
Member Author

@jgabry jgabry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rok-cesnovar Just added a bunch of review comments that hopefully help explain the changes.

R/fit.R Show resolved Hide resolved
R/fit.R Show resolved Hide resolved
R/fit.R Show resolved Hide resolved
R/fit.R Show resolved Hide resolved
R/fit.R Show resolved Hide resolved
R/read_csv.R Outdated Show resolved Hide resolved
R/read_csv.R Outdated Show resolved Hide resolved
R/read_csv.R Outdated Show resolved Hide resolved
@jgabry jgabry requested a review from rok-cesnovar December 18, 2020 21:21
DESCRIPTION Outdated
@@ -1,6 +1,6 @@
Package: cmdstanr
Title: R Interface to 'CmdStan'
Version: 0.3.0
Version: 0.3.0.9000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bumping to dev version

R/read_csv.R Outdated Show resolved Hide resolved
@jgabry jgabry marked this pull request as ready for review December 18, 2020 22:49
@jgabry
Copy link
Member Author

jgabry commented Dec 18, 2020

@yizhang-yiz I think this is now ready to play around with if you want to try it out.

@yizhang-yiz
Copy link

> fit <- cmdstanr::as_cmdstan_mcmc(dir(pattern=".csv", full.name=TRUE))

This works

bayesplot::mcmc_dens_overlay(fit$draws(), pars = c("CL", "Q", "V1", "V2", "ka", "sigma"), facet_args = list(nrow = 2))

but this doesn't

> print(fit$time(), digits = 3)
Error: This method is not available for objects created using as_cmdstan_mcmc().

I wonder what the rationale is.

@jgabry
Copy link
Member Author

jgabry commented Dec 19, 2020

I wonder what the rationale is.

No good one other than it would take more effort to get that to work. read_cmdstan_csv() doesn't currently give you the timing information. When model fitting happens we create a CmdStanRun object

cmdstanr/R/model.R

Lines 804 to 806 in 46ae80f

runset <- CmdStanRun$new(args = cmdstan_args, procs = cmdstan_procs)
runset$run_cmdstan()
CmdStanMCMC$new(runset)

which handles preparing the call to cmdstan, setting up the external processes, and then running cmdstan. It's also the object that report the run times (because we don't wait for reading in the csv to report the time). The way I created as_cmdstan_mcmc() it avoids creating this object entirely and just uses the output from read_cmdstan_csv(). So to get time information with as_cmdstan_mcmc() we'd just need to add it to read_cmdstan_csv() (probably a good idea anyway).

@rok-cesnovar
Copy link
Member

we'd just need to add it to read_cmdstan_csv() (probably a good idea anyway).

Agree. Will do that separately.

@codecov-io
Copy link

codecov-io commented Dec 19, 2020

Codecov Report

Merging #412 (06145b6) into master (d7190ba) will increase coverage by 0.15%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #412      +/-   ##
==========================================
+ Coverage   88.29%   88.45%   +0.15%     
==========================================
  Files          12       12              
  Lines        2793     2823      +30     
==========================================
+ Hits         2466     2497      +31     
+ Misses        327      326       -1     
Impacted Files Coverage Δ
R/model.R 80.33% <ø> (ø)
R/fit.R 97.10% <100.00%> (+0.41%) ⬆️
R/read_csv.R 98.97% <100.00%> (+0.08%) ⬆️
R/utils.R 91.94% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7190ba...06145b6. Read the comment docs.

@jgabry
Copy link
Member Author

jgabry commented Dec 20, 2020

I changed to as_cmdstan_fit(), good idea. I haven't changed the class names yet but I'm leaning towards CmdStanMCMC_CSV. That's definitely better than CmdStanMCMC2.

Let's wait until #414 is merged and then we can update this PR to include time before merging it.

@jgabry
Copy link
Member Author

jgabry commented Dec 20, 2020

Ok I changed the class names to CmdStanMCMC_CSV, CmdStanMLE_CSV, CmdStanVB_CSV and I added the time() method for CmdStanMCMC_CSV after pulling in the changes from #414.

Copy link
Member

@rok-cesnovar rok-cesnovar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks.

@rok-cesnovar rok-cesnovar merged commit 6bb7f7d into master Dec 20, 2020
@rok-cesnovar rok-cesnovar deleted the CmdStanFit-from-CSV branch December 20, 2020 19:20
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.45%. Comparing base (d7190ba) to head (06145b6).
Report is 1652 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #412      +/-   ##
==========================================
+ Coverage   88.29%   88.45%   +0.15%     
==========================================
  Files          12       12              
  Lines        2793     2823      +30     
==========================================
+ Hits         2466     2497      +31     
+ Misses        327      326       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create CmdStanMCMC R6 object from CSV
5 participants