Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDN 2016-20 #436

Open
eclee25 opened this issue May 15, 2023 · 23 comments
Open

SDN 2016-20 #436

eclee25 opened this issue May 15, 2023 · 23 comments
Labels
multiyr_obs These models have multi-year observations

Comments

@eclee25
Copy link
Collaborator

eclee25 commented May 15, 2023

No description provided.

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 6, 2023

Data pull:
HASH: 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8
config

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 7, 2023

Model run:
HASH: 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 8, 2023

After double checking, SDN2016-2020 config was missing before.need to regenerate config and rerun from data pull

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 8, 2023

new config

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 8, 2023

rerun from data pull:
data pull on dev: HASH c9a89d4ac5e4f63cb75fa5c006a900ece2839199

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 9, 2023

Model run:
HASH 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8

@QLLZ
Copy link
Collaborator

QLLZ commented Jun 9, 2023

Country data report

@eclee25
Copy link
Collaborator Author

eclee25 commented Jun 9, 2023

Convergence, fits, Rhats look good, but there is a strong effect of the adjacency matrix on the rate maps that seems unusual.

Opinion: Investigate adjacency matrix

@javierps
Copy link
Collaborator

Maybe same issue on population density as in NER. Large and sparsely populated places are hard for the model to fit?

@javierps
Copy link
Collaborator

javierps commented Jul 6, 2023

Dagar fixed should solve this, waiting for ID 45 report.

@QLLZ
Copy link
Collaborator

QLLZ commented Jul 24, 2023

Config

Data pull:

HASH: 211913e

@QLLZ
Copy link
Collaborator

QLLZ commented Jul 24, 2023

@eclee25 eclee25 added the multiyr_obs These models have multi-year observations label Jul 31, 2023
@QLLZ
Copy link
Collaborator

QLLZ commented Aug 4, 2023

Rerun on dev_u_combs_fix

HASH: e56580fa5ddab00293e31ff90139351f80ae3f6c

@QLLZ
Copy link
Collaborator

QLLZ commented Aug 8, 2023

country data report

@QLLZ
Copy link
Collaborator

QLLZ commented Aug 8, 2023

Overall cases are overestimated. convergency looks good/

@eclee25
Copy link
Collaborator Author

eclee25 commented Aug 9, 2023

Instability in sd_w traceplots but otherwise the model convergence looks good. There is overestimation in 2016 and 2019 and it's not clear why. Perhaps the 2016 overestimation is related to the multi-year observation from 2016-2018?

Investigate the 3-year multi-year obs in OCs 20189? 21107? It's hard to tell which OCs they are in based on the cdr -- may be better to verify which OCs have these observations in the stan input file.

I think we could perhaps try running it after removing or handling the 3-year multi-year obs in a different way?

@QLLZ
Copy link
Collaborator

QLLZ commented Aug 9, 2023

double checked the 3-year observation in 21089. it looks correct based on the source doc

@javierps
Copy link
Collaborator

The 2016 data is an imputed observation, so may make sense it is hard for the model to fit.

For std_dev_w, given the very high values of rho I don't think this has a large impact (the w traceplots look very good).

I don't think the multi-year observations are breaking things here, but could try a run without it.

@eclee25
Copy link
Collaborator Author

eclee25 commented Aug 17, 2023

Rerun with updated national od parameter

@eclee25
Copy link
Collaborator Author

eclee25 commented Aug 23, 2023

@javierps determined that the initialization failures we're having with the new od admin0 parameter are caused by numerical stability issues in the evaluation of the censored observation probability -- added a possible fix to the Stan code that improves numerical stability of the probability is close to 0 or 1 --> implemented as dummy censoring variables

@javierps
Copy link
Collaborator

Sep 2023 Production run: chain differences in std_w, rest convergence Ok, Rhats Ok.

Suggestion: accept.

@eclee25
Copy link
Collaborator Author

eclee25 commented Sep 19, 2023

Noting that Tab 6 2017 estimates are between the two observations and the PI does not include either obs, likely due to the slightly restricted overdispersion param here. Other diags look okay.

Discuss this but leaning to Approve. We are okay with the 2017 estimate but we want to rerun the cdr to show the 2016 imputed observations.

Action Required - rerun cdr

@eclee25
Copy link
Collaborator Author

eclee25 commented Oct 24, 2023

2016 imputed obs is ok.
Approve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multiyr_obs These models have multi-year observations
Projects
None yet
Development

No branches or pull requests

3 participants