Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle messaging around model treatment of missingness #774

Merged
merged 39 commits into from
Oct 16, 2024
Merged

Conversation

jamesmbaazam
Copy link
Contributor

@jamesmbaazam jamesmbaazam commented Sep 16, 2024

Description

This PR closes #771 by adding a message to inform users about how NA's are treated in the default model. It also points to alternatives given the user's use case. This PR also adds missing tests for obs_opts().

Initial submission checklist

  • My PR is based on a package issue and I have explicitly linked it.
  • I have tested my changes locally (using devtools::test() and devtools::check()).
  • I have added or updated unit tests where necessary.
  • I have updated the documentation if required and rebuilt docs if yes (using devtools::document()).
  • I have followed the established coding standards (and checked using lintr::lint_package()).
  • I have added a news item linked to this PR.

After the initial Pull Request

  • I have reviewed Checks for this PR and addressed any issues as far as I am able.

R/opts.R Outdated Show resolved Hide resolved
Copy link
Contributor

@sbfnk sbfnk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice - some suggestions for (I think) slightly clearer wording. I think we should be more restrictive about when this message is shown (e.g. it should be suppressed when there is no missing data).

R/opts.R Outdated Show resolved Hide resolved
NEWS.md Outdated Show resolved Hide resolved
R/opts.R Outdated Show resolved Hide resolved
R/opts.R Outdated Show resolved Hide resolved
R/opts.R Outdated Show resolved Hide resolved
tests/testthat/test-obs_opts.R Outdated Show resolved Hide resolved
@jamesmbaazam jamesmbaazam changed the title Add message about NA treatment in obs_opts() Conditionally throw message for obs_opts(na = "missing") Sep 17, 2024
NEWS.md Show resolved Hide resolved
Copy link
Contributor

@sbfnk sbfnk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good.

It's adding a fair amount of infrastructure (an additional function + return value in obs_opts() that needs to be removed) just to be able to show that message. Shall we give it an expiry (and use keyword "deprecated" somewhere)?

R/checks.R Outdated Show resolved Hide resolved
R/checks.R Outdated Show resolved Hide resolved
R/checks.R Outdated Show resolved Hide resolved
R/estimate_infections.R Outdated Show resolved Hide resolved
R/opts.R Outdated Show resolved Hide resolved
R/estimate_infections.R Outdated Show resolved Hide resolved
@jamesmbaazam
Copy link
Contributor Author

Thanks, this looks good.

It's adding a fair amount of infrastructure (an additional function + return value in obs_opts() that needs to be removed) just to be able to show that message. Shall we give it an expiry (and use keyword "deprecated" somewhere)?

Was this what you meant? https://github.com/epiforecasts/EpiNow2/pull/774/files#diff-6cd987920443d42098a4963d43b2e0efec9fd16f9198d66705e6d95094e3b87aR185-R194.

@sbfnk
Copy link
Contributor

sbfnk commented Sep 24, 2024

Thanks, this looks good.
It's adding a fair amount of infrastructure (an additional function + return value in obs_opts() that needs to be removed) just to be able to show that message. Shall we give it an expiry (and use keyword "deprecated" somewhere)?

Was this what you meant? https://github.com/epiforecasts/EpiNow2/pull/774/files#diff-6cd987920443d42098a4963d43b2e0efec9fd16f9198d66705e6d95094e3b87aR185-R194.

Yes, I think this will do.

@jamesmbaazam
Copy link
Contributor Author

jamesmbaazam commented Oct 9, 2024

Converting this PR to a draft to start a discussion before opening for review.

I'm a little torn and need your opinion (@sbfnk & @seabbs). I have two options here:

  1. Allow check_na_setting_against_data() to call test_data_complete() so that the check is self-contained; small issue is they will share the cols_to_check argument. I think that argument doesn't make sense in the signature of check_na_setting_against_data(). This couples the two functions and there isn't complete separation of concerns.
  2. Only call check_na_setting_against_data() if test_data_complete() returns false

Here, the change required in estimate_infections() and estiamte_secondary() will be:

if (!test_data_complete(data, cols_to_check)) {
		check_na_setting_against_data(data, obs)
}

This second option would mean that check_na_setting_against_data(data, obs) would not check if the data is complete and would unconditionally throw the message about treatment of NA's.

Which one makes sense to you? Is there an option I'm missing?

UPDATE:

  • I decided to go with option 1.

@jamesmbaazam jamesmbaazam marked this pull request as ready for review October 10, 2024 12:06
sbfnk
sbfnk previously approved these changes Oct 11, 2024
Copy link
Contributor

@sbfnk sbfnk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - just needs the reviewer / PR number added.

@jamesmbaazam

This comment was marked as outdated.

@sbfnk
Copy link
Contributor

sbfnk commented Oct 11, 2024

I think the check failures might be related to epinowcast/actions#24 - I'll clear the cache and run again.

@jamesmbaazam
Copy link
Contributor Author

I think the check failures might be related to epinowcast/actions#24 - I'll clear the cache and run again.

It was related to future being required to to run test-setup-future. I now install it in the CI in #827. Will rebase this on main when #827 is merged.

@sbfnk
Copy link
Contributor

sbfnk commented Oct 14, 2024

I think the check failures might be related to epinowcast/actions#24 - I'll clear the cache and run again.

It was related to future being required to to run test-setup-future. I now install it in the CI in #827. Will rebase this on main when #827 is merged.

That was an additional issue - this one was related to the cache (and was successful after clearing it): https://github.com/epiforecasts/EpiNow2/actions/runs/11290308841

Agreed.

@jamesmbaazam jamesmbaazam changed the title Conditionally throw message for obs_opts(na = "missing") Handle messaging around model treatment of missingness Oct 15, 2024
@sbfnk sbfnk added this pull request to the merge queue Oct 16, 2024
Merged via the queue into main with commit f5927c4 Oct 16, 2024
9 checks passed
@sbfnk sbfnk deleted the na-warning branch October 16, 2024 16:15
@jamesmbaazam jamesmbaazam mentioned this pull request Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add message explaining new behaviour when obs_opts(na = "missing")
2 participants