-
Notifications
You must be signed in to change notification settings - Fork 0
/
search_index.json
1 lines (1 loc) · 72.6 KB
/
search_index.json
1
[["index.html", "Understanding Propensity Score Matching Preamble Description Prerequisites", " Understanding Propensity Score Matching Ehsan Karim 2023-03-19 Preamble Description Propensity score matching is widely used in analyzing observational datasets to reduce the impact of confounding due to observed covariates. This workshop will provide a basic overview of related causal inference concepts, explain propensity score matching analysis steps, illustrate propensity score matching diagnostics, and provide examples of when this method may be preferable to a regression. Main references We present may references inside, but the following are the key references Propensity score matching steps Austin (2011a) Reporting guideline Karim et al. (2020) Version history Materials were updated over time through various deliveries of the content: ‘Understanding Propensity Score Matching’: June 3, 2021, prepared as a Post Conference Workshop for 2021 Conference - CSEB. ‘A Practical Introduction to Propensity Score Analysis using R’: Sept 30, 2020, prepared for an invited webinar for Canadian Statistics Student Society, in collaboration with TI Methods. ‘Introduction to Causal Inference: Propensity Score Analysis in Healthcare Data’: May 14, 2020, prepared for Population Data BC (in partnership with IC/ES). ‘Introduction to Causal inference’ March 20, 2023, March 21, 2022, March 22, 2021, March 23, 2020, March 25, 2019, prepared for Guest Lecture in SPPH 500/007 (Analytical Methods in Epidemiological Research) at UBC Also see further resources at the very end of the document. Prerequisites The prerequisites are knowledge of multiple regression analysis and basic probability. Software demonstrations and codes will be provided in R, although proficiency in R is not required for understanding the concepts. If you are not familiar with R, and want to gain further understanding, I would suggest the following tutorial. R tutorial Karim ME, Hoang A and Qu Y “Introduction to R for health data analysis” URL: ehsanx.github.io/intro2R/ Packages that we will use in this demonstration: # devtools::install_github('osofr/simcausal', build_vignettes = FALSE) require(simcausal) require(summarytools) require(skimr) require(jtools) require(cobalt) require(tableone) require(MatchIt) require(twang) require(Matching) require(SuperLearner) require(ltmle) require(DoubleML) require(AIPW) require(ggplot2) library(mlr3) library(mlr3learners) License The online version of this book is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. You may share, adapt the content and may distribute your contributions under the same license (CC BY-NC-SA 4.0), but you have to give appropriate credit, and cannot use material for the commercial purposes. How to cite Karim, ME (2021) “Understanding Propensity Score Matching”, URL: ehsanx.github.io/psw/ Comments For any comments regarding this document, reach out to me. References "],["terms.html", "Chapter 1 Defining Parameter 1.1 Epidemiological research goals 1.2 Potential outcome 1.3 Parameters of interest", " Chapter 1 Defining Parameter 1.1 Epidemiological research goals Two common goals for epidemiological research are prediction and causal inference: Prediction goal: The primary objective of a prediction goal is to forecast the occurrence or risk of an outcome (\\(Y\\)) based on one or more risk factors (\\(A\\)). The focus of this goal is often on making accurate predictions. Causal goal: The causal goal focuses on understanding the causal relationship between a risk factor (often a treatment, \\(A\\)) and a health outcome (\\(Y\\)). Control for confounding factors (\\(L\\)) is often a necessary step in understanding such a relationship. The focus of this goal is often on estimating the parameter ‘treatment effect’. We only focus on estimating treatment effect today. For that, let us define the notations first. 1.2 Potential outcome \\(A\\): Exposure status \\(1\\) = takes Rosuvastatin \\(0\\) = does not take rosuvastatin \\(Y\\): Outcome: Total cholesterol levels \\(Y(A=1)\\) = potential outcome when exposed \\(Y(A=0)\\) = potential outcome when not exposed Relationship between \\(Y\\) and \\([Y(A=1), Y(A=0)]\\) can be expressed as follows: \\(Y = A \\times Y(A=1) + (1-A) \\times Y(A=0)\\) 1.3 Parameters of interest When assessing the effect of an exposure on an outcome, we are interested about the following estimands treatment effect for an individual (TE) average treatment effect (ATE) average treatment effect on the treated (ATT) 1.3.1 TE John takes Rosuvastatin \\((A=1)\\) and his total cholesterol level is = \\(Y(A=1)\\) = \\(195\\) mg/dL (milligrams per deciliter) after 3 months John does not take Rosuvastatin \\((A=0)\\) and his total cholesterol level is = \\(Y(A=0)\\) = \\(245\\) mg/dL after 3 months Effect of Rosuvastatin on John is = \\(TE = Y(A=1) - Y(A=0) = 195 - 245 = - 50\\) TE is not estimable as we generally can’t observe outcomes under both treatment conditions. 1.3.2 ATE Person <- c("John","Jim","Jake","Cody","Luke") Y1 <- c( 195, 100, 210, 155, 165) Y0 <- c(245, 160, 270, 210, 230) PotentialOutcomes <- data.frame(Person, Y1, Y0, TE = Y1-Y0) mean.values <- c(NA, mean(PotentialOutcomes$Y1), mean(PotentialOutcomes$Y0), mean(PotentialOutcomes$TE)) PotentialOutcomes <- rbind(PotentialOutcomes, mean.values) kable(PotentialOutcomes, booktabs = TRUE, col.names = c("Person", "Y(1)", "Y(0)", "TE")) %>% row_spec(6, bold = T, color = "white", background = "#D7261E") Person Y(1) Y(0) TE John 195 245 -50 Jim 100 160 -60 Jake 210 270 -60 Cody 155 210 -55 Luke 165 230 -65 165 223 -58 \\(ATE = E[Y(A=1)-Y(A=0)]\\) mean(PotentialOutcomes$Y1 - PotentialOutcomes$Y0) ## [1] -58 1.3.3 Interpretation of ATE This is a treatment effect (on an average) of the following hypothetical situation having the entire population as treated, vs. having the entire population as untreated. Entire population is the reference goup here. 1.3.4 Identifiability Assumptions Real-world scenario (both outcomes under different treatments can not be observed): Person <- c("John","Jim","Jake","Cody","Luke") Y1 <- c( NA, 100, NA, 155, NA) Y0 <- c(245, NA, 270, NA, 230) PotentialOutcomes <- data.frame(Person, Y1, Y0, TE = Y1-Y0) mean.values <- c(NA, mean(PotentialOutcomes$Y1, na.rm = TRUE), mean(PotentialOutcomes$Y0, na.rm = TRUE), mean(PotentialOutcomes$TE)) PotentialOutcomes <- rbind(PotentialOutcomes, round(mean.values,1)) PotentialOutcomes[6,4] <- round(mean(PotentialOutcomes$Y1, na.rm = TRUE)- mean(PotentialOutcomes$Y0, na.rm = TRUE),1) kable(PotentialOutcomes, booktabs = TRUE, col.names = c("Person", "Y(1)", "Y(0)", "TE")) %>% row_spec(6, bold = T, color = "white", background = "#D7261E") Person Y(1) Y(0) TE John 245.0 Jim 100.0 Jake 270.0 Cody 155.0 Luke 230.0 127.5 248.3 -120.8 We can rearrange it as follows: Person <- c("John","Jim","Jake","Cody","Luke") A <- c( 0, 1, 0, 1, 0) Y <- c(245, 100, 270, 155, 230) RealOutcomes <- data.frame(Person, A, Y) kable(RealOutcomes, booktabs = TRUE, col.names = c("Person", "A", "Y")) Person A Y John 0 245 Jim 1 100 Jake 0 270 Cody 1 155 Luke 0 230 If we can compute a causal quantity, such as \\(ATE = E[Y(A=1)-Y(A=0)]\\) or mean(PotentialOutcomes$Y1 - PotentialOutcomes$Y0) using a statistical quantity, such as \\(E[Y|A=1]-E[Y|A=0]\\) or mean(Y[A=1]) - mean(Y[A=0]), we say that the causal quantity is identifiable. For such identifiability, we need to meet the following assumptions: Exchangeability \\(Y(1), Y(0) \\perp A\\) Treatment assignment is independent of the potential outcome Positivity \\(0 < P(A=1) < 1\\) Subjects are eligible to receive both treatment Consistency \\(Y = Y(a) \\forall A=a\\) No multiple version of the treatment No interference Treated one patient will not impact outcome for others Note here, from data we get the estimate of average TE is (100+155)/2 - (245+270+230)/3 = -120.8. Alternatively, we can calculate the beta coefficient associated with \\(A\\) as follows: round(coef(lm(Y~A)),1) ## (Intercept) A ## 248.3 -120.8 Here, beta coefficient associated with \\(A\\) is -120.8, which is different than average TE -58 that we obtained from the potential outcome data table above. Part of it is because of finite sample bias (having only 5 data points) instead of infinite population. If we had a large enough sample, we would expect the estimate to be close to the true average TE. You can find more detailed exploration of estimation in a different tutorial using a real data. Extending these assumptions when confounders exist: Conditional Exchangeability \\(Y(1), Y(0) \\perp A | L\\) Treatment assignment is independent of the potential outcome, given L Positivity \\(0 < P(A=1 | L) < 1\\) Subjects are eligible to receive both treatment, given L Here, - \\(L\\): Confounder: Age, could be an example 1.3.5 ATT Assume that the following are the confounders that impact the relationship between rosuvastatin and cholesterol levels race sex age We have 5 Rosuvastatin-treated subjects who are all white, male, 50 years of age We recruited additional 5 subjects (same characteristics) to non-rosuvastatin group. Treated group: Person <- c("John","Jim","Jake","Cody","Luke") Y1 <- c( 195, 100, 210, 155, 165) Y0 <- rep(NA, length(Y1)) Treated <- data.frame(Person, Y1, Y0, TE = Y1-Y0) Treated[6,2] <- mean(Treated$Y1) kable(Treated, booktabs = TRUE, col.names = c("Person", "Y(1)", "Y(0)", "TE"))%>% row_spec(6, bold = T, color = "white", background = "#D7261E") Person Y(1) Y(0) TE John 195 Jim 100 Jake 210 Cody 155 Luke 165 165 Untreated group: New folks with characteristics similar to the treated group. Person <- c( "Jack", "Dustin", "Cole", "Lucas", "Dylan") Y0 <- c( 245, 160, 270, 210, 165) Y1 <- rep(NA, length(Y0)) Untreated <- data.frame(Person, Y1, Y0, TE = Y1-Y0) Untreated[6,3] <- mean(Untreated$Y0) kable(Untreated, booktabs = TRUE, col.names = c("Person", "Y(1)", "Y(0)", "TE"))%>% row_spec(6, bold = T, color = "white", background = "#D7261E") Person Y(1) Y(0) TE Jack 245 Dustin 160 Cole 270 Lucas 210 Dylan 165 210 \\(ATT = E[Y(A=1)-Y(A=0) | A = 1]\\) mean(Treated$Y1) - mean(Untreated$Y0) ## [1] -45 1.3.6 Interpretation of ATT This is a treatment effect (on an average) of the treated population (reference group), vs. untreated population, but have similar characteristics to the reference group/treated population. It is also possible to change the reference population to untreated population. Then it is called Average Treatment Effect for the Untreated (ATU). 1.3.7 ATT vs. ATE In a RCT (enough n), the ATT & ATE are equivalent In an observational study the ATT and ATE are not necessarily the same. "],["balance.html", "Chapter 2 Balance and Overlap 2.1 Balance 2.2 Adjustment 2.3 Lack of overlap", " Chapter 2 Balance and Overlap 2.1 Balance Table 1 in any RCT paper is very important to assess the balance of the baseline characteristics between the two treatment arms. Balance in RCT: Compare the proportions in each age categories by eye-balling in making an evaluation. In absence of randomization: 2.1.1 Measures of Balance To compare baseline characteristics between the two treatment groups, we use t-test (for continuous variables) and chi-square test (for categorical variables) Today we will introduce a new concept, known as standardized mean differences (SMD) that can be also used to compare baseline characteristics between the two treatment groups. 2.1.1.1 SMD Austin (2011b) For continuous confounders: \\(SDM_{continuous} = \\frac{\\bar{L}_{A=1} - \\bar{L}_{A=0}}{\\sqrt{\\frac{(n_1 - 1) \\cdot s^2_{A=1} + (n_0 - 1) \\cdot s^2_{A=0}}{n_1 + n_0 - 2}}}\\) \\(\\bar{L}_{A=1}\\): mean of the continuous variable L for the treated group (A=1) \\(\\bar{L}_{A=0}\\): mean of the continuous variable L for the untreated group (A=0) \\(n_1\\): number of treated individuals (A=1) \\(n_0\\): number of untreated individuals (A=0) \\(s^2_{A=1}\\): variance of the continuous variable L for the treated group (A=1) \\(s^2_{A=0}\\): variance of the continuous variable L for the untreated group (A=0) For binary confounders: \\(SDM_{binary} = \\frac{\\hat{p}_{A=1} - \\hat{p}_{A=0}}{\\sqrt{\\frac{\\hat{p}_{A=1} \\times (1 - \\hat{p}_{A=1}) + \\hat{p}_{A=0} \\times (1 - \\hat{p}_{A=0})}{2}}}\\) where \\(\\hat{p}{A=1}\\) and \\(\\hat{p}{A=0}\\) are the proportions of the two binary groups. For categorical confounders (more than 2 categories): Calculating SMD for categorical variables is slightly more complicated, requiring some additional calculation and averaging out the results from category-specific SMDs. See formulas in Yang and Dalton (2012). It is also possible to report SMD for each category separately (similar to binary). Generally, \\(0.1\\) is used as a cut-point. But some suggest more liberal cut-points. More on that later. COVID example from Gautret et al. (2020) p-value vs. SMD Statistical tests are affected by sample size t-test McNemar tests Wilcoxon rank test Balance of what? statistical tests make inference about balance at the population level but we are really interested in balance at the sample level 2.1.1.2 Variance ratio Variances of baseline characteristics between comparator groups under consideration. Suggested cut-point rages are (0.5 to 2). More liberal cut-points are also used in the literature. More on this later. 2.2 Adjustment 2.2.1 Why adjust? In absence of randomization, treatment effect estimate ATE = \\(E[Y|A=1] - E[Y|A=0]\\) includes Treatment effect Systematic differences in 2 groups (‘confounding’) Doctors may prescribe tx more to frail and older age patients. In here, \\(L\\) = age is a confounder. In absence of randomization, if age is a known confounder, conditioning can solve the problem: Causal effect for young (\\(<50\\)) \\(E[Y|A=1, L =\\) younger age\\(]\\) - \\(E[Y|A=0, L =\\) younger age\\(]\\) Causal effect for old (\\(\\ge 50\\)) \\(E[Y|A=1, L =\\) older age\\(]\\) - \\(E[Y|A=0, L =\\) older age\\(]\\) Conditional exchangeability; only works if \\(L\\) is measured. 2.2.2 Adjustment Methods Adjustment of imbalance could mean exact matching (Rubin 1973) stratification, restriction When L includes a large number of covariates, matching method would result in a small sample size. Regression and machine learning methods are popular adjustment methods. Below is a list of adjustment methods that uses different combinations of exposure and outcome modelling. Method Exposure model \\((\\color{green}{\\text{A}} \\sim ...)\\) Outcome Model \\((Y \\sim ...)\\) Regression (Gauss 1821 \\(\\dagger\\)) No \\(\\color{green}{\\text{A}}\\) modelling Yes \\((Y \\sim \\color{green}{\\text{A}}+\\color{red}{\\text{L}})\\) Instrumental variable methods (2SLS) \\(\\dagger\\dagger\\) \\(A \\sim IV + L\\), but explicitly to obtain \\(\\hat{A}\\) \\(Y \\sim \\hat{A} + L\\) Propensity score matching (Rosenbaum and Rubin 1983) Yes \\((\\color{green}{\\text{A}} \\sim \\color{red}{\\text{L}})\\) Crude comparison on matched data \\((Y \\sim \\color{green}{\\text{A}})\\) Propensity score Weighting (Rosenbaum and Rubin 1983) Yes \\((\\color{green}{\\text{A}} \\sim \\color{red}{\\text{L}})\\) Crude comparison on weighted data \\((Y \\sim \\color{green}{\\text{A}})\\) Propensity score double adjustment Yes \\((\\color{green}{\\text{A}} \\sim \\color{red}{\\text{L}})\\) Yes \\((Y \\sim \\color{green}{\\text{A}}+ \\color{red}{\\text{L}})\\) Decision tree-based method (Breiman et al. 1984) No \\(\\color{green}{\\text{A}}\\) modelling Yes \\((Y \\sim \\color{green}{\\text{A}}+\\color{red}{\\text{L}})\\) G-Computation (J. Robins 1986) No \\(\\color{green}{\\text{A}}\\) modelling Yes \\((Y \\sim \\color{green}{\\text{A=1 vs. 0}}+\\color{red}{\\text{L}})\\) Random Forest (Ho 1995) No \\(\\color{green}{\\text{A}}\\) modelling Yes \\((Y \\sim \\color{green}{\\text{A}}+\\color{red}{\\text{L}})\\) Double robust (J. M. Robins and Rotnitzky 2001), (Van Der Laan and Rubin 2006) (augmented weighted, or TMLE), causal forest (Athey and Wager 2019), double machine learning (DML) (Chernozhukov et al. 2017), potentially using machine learning Yes \\((\\color{green}{\\text{A}} \\sim \\color{red}{\\text{L}})\\) Yes \\((Y \\sim \\color{green}{\\text{A}}+ \\color{red}{\\text{L}})\\) \\(\\dagger\\) (Stanton 2001). \\(\\dagger\\dagger\\) other economic models also exists: Structural equation modeling, Difference-in-differences, Regression discontinuity, Interrupted time series 2.3 Lack of overlap “Lack of complete overlap” happens if there is a baseline covariate space where there are exposed patients, but no control or vice versa. Region of ‘no overlap’ is an inherent limitation of the data. Regression adjustment usually do not offer any solution to this. Consequently, inference is not generalizable beyond the region of overlap. References "],["ps.html", "Chapter 3 Propensity score 3.1 Motivating problem 3.2 Defining Propensity score 3.3 PS Matching Steps", " Chapter 3 Propensity score 3.1 Motivating problem \\(Y\\) : Outcome Cholesterol levels (high vs. low) \\(A\\) : Exposure Diabetes \\(L\\) : Known Confounders gender, age, race, education, married, BMI Search literature for the confounder variables, and look for those variables in the data source (NHANES 2017-2018). Data recourses: All of the data files used in this workshop are available in the GitHub repo. If you are interested in how to obtain NHANES datasets and prepare analytic data, here is an example (outside of the scope of the current tutorial). load(file="data/NHANES17.RData") require(dplyr) analytic <- dplyr::select(analytic, cholesterol, # outcome gender, age, race, education, married, bmi, # confounders diabetes) # exposure analytic$cholesterol <- ifelse(analytic$cholesterol > 240, 1, 0) analytic$diabetes <- ifelse(analytic$diabetes == "Yes", 1, 0) require(summarytools) dfSummary(analytic) ## Data Frame Summary ## analytic ## Dimensions: 1562 x 8 ## Duplicates: 2 ## ## ------------------------------------------------------------------------------------------------------------------------------------------------- ## No Variable Label Stats / Values Freqs (% of Valid) Graph Valid Missing ## ---- --------------------- --------------------------- ------------------------- --------------------- --------------------- ---------- --------- ## 1 cholesterol Min : 0 0 : 1391 (89.1%) IIIIIIIIIIIIIIIII 1562 0 ## [numeric] Mean : 0.1 1 : 171 (10.9%) II (100.0%) (0.0%) ## Max : 1 ## ## 2 gender 1. Female 603 (38.6%) IIIIIII 1562 0 ## [character] 2. Male 959 (61.4%) IIIIIIIIIIII (100.0%) (0.0%) ## ## 3 age Age in years at screening Mean (sd) : 53.2 (17.2) 61 distinct values : : : 1562 0 ## [labelled, integer] min < med < max: . : . . : : : : : (100.0%) (0.0%) ## 20 < 55 < 80 : : : : : : : : : : ## IQR (CV) : 29 (0.3) : : : : : : : : : : ## : : : : : : : : : : ## ## 4 race 1. Black 324 (20.7%) IIII 1562 0 ## [character] 2. Hispanic 284 (18.2%) III (100.0%) (0.0%) ## 3. Other 228 (14.6%) II ## 4. White 726 (46.5%) IIIIIIIII ## ## 5 education 1. College 806 (51.6%) IIIIIIIIII 1562 0 ## [character] 2. High.School 658 (42.1%) IIIIIIII (100.0%) (0.0%) ## 3. School 98 ( 6.3%) I ## ## 6 married 1. Married 921 (59.0%) IIIIIIIIIII 1562 0 ## [character] 2. Never.married 228 (14.6%) II (100.0%) (0.0%) ## 3. Previously.married 413 (26.4%) IIIII ## ## 7 bmi Body Mass Index (kg/m2) Mean (sd) : 30 (7.3) 314 distinct values : 1562 0 ## [labelled, numeric] min < med < max: : . (100.0%) (0.0%) ## 14.8 < 28.9 < 64.2 : : : ## IQR (CV) : 8.8 (0.2) : : : : ## . : : : : : . ## ## 8 diabetes Min : 0 0 : 1232 (78.9%) IIIIIIIIIIIIIII 1562 0 ## [numeric] Mean : 0.2 1 : 330 (21.1%) IIII (100.0%) (0.0%) ## Max : 1 ## ------------------------------------------------------------------------------------------------------------------------------------------------- 3.2 Defining Propensity score Conditional Probability of getting treatment, given the observed covariates Prob(treatment: \\(A = 1\\) | baseline or pre-treatment covariates: \\(L\\)) Prob(\\(A = 1\\): Has diabetes | \\(L\\): gender, age, race, education, married, bmi) PS = \\(Prob(A=1|L)\\) Condensing multiple variables (in L) into one summary variable (PS). Essentially a dimension reduction exercise! 3.2.1 Theoretical result Rosenbaum and Rubin (1983) showed: For potential outcomes \\(Y(1), Y(0)\\), if you have sufficient observed covariate list \\(L\\) to reduce confounding (`strong ignoribility’): i.e., if \\((Y(1), Y(0)) \\perp A | L\\) Note that is this NOT \\(Y \\perp A | L\\) then \\((Y(1), Y(0)) \\perp A | PS\\) and \\(A \\perp L | PS\\) 3.2.2 Assumptions Conditional Exchangeability \\(Y(1), Y(0) \\perp A | L\\) Treatment assignment is independent of the potential outcome, given L Positivity \\(0 < P(A=1 | L) < 1\\) Subjects are eligible to receive both treatment, given L Consistency \\(Y = Y(a) \\forall A=a\\) No multiple version of the treatment 3.2.3 Ways to use PS Many ways to use propensity scores (PS) in the analysis PS matching [our focus today] PS weighting PS stratification PS used as a covariate 3.3 PS Matching Steps Propensity score matching has 4 steps (Austin 2011a) Step 1 exposure modelling: \\(PS = Prob(A=1|L)\\) Step 2 Match by \\(PS\\) Step 3 Assess balance and overlap (\\(PS\\) and \\(L\\)) Step 4 outcome modelling: \\(Prob(Y=1|A=1)\\) References "],["s1.html", "Chapter 4 Step 1: Exposure modelling 4.1 Model specification 4.2 Variables to adjust 4.3 Model selection 4.4 Alternative modelling strategies 4.5 PS estimation", " Chapter 4 Step 1: Exposure modelling 4.1 Model specification Specify the propensity score model to estimate propensity scores, and fit the model: \\(A \\sim L\\) baselinevars <- c("gender", "age", "race", "education", "married", "bmi") ps.formula <- as.formula(paste("diabetes", "~", paste(baselinevars, collapse = "+"))) ps.formula ## diabetes ~ gender + age + race + education + married + bmi # fit logistic regression to estimate propensity scores PS.fit <- glm(ps.formula,family="binomial", data=analytic) require(jtools) summ(PS.fit) Observations 1562 Dependent variable diabetes Type Generalized linear model Family binomial Link logit χ²(10) 282.89 Pseudo-R² (Cragg-Uhler) 0.26 Pseudo-R² (McFadden) 0.18 AIC 1349.94 BIC 1408.83 Est. S.E. z val. p (Intercept) -8.38 0.58 -14.49 0.00 genderMale 0.34 0.15 2.26 0.02 age 0.06 0.01 11.26 0.00 raceHispanic 0.15 0.23 0.64 0.52 raceOther 0.76 0.23 3.25 0.00 raceWhite -0.23 0.18 -1.23 0.22 educationHigh.School 0.14 0.15 0.95 0.34 educationSchool 0.52 0.27 1.92 0.05 marriedNever.married -0.04 0.25 -0.16 0.88 marriedPreviously.married -0.02 0.16 -0.15 0.88 bmi 0.10 0.01 10.14 0.00 Standard errors: MLE Coef of PS model fit is not of concern Model can be rich: to the extent that prediction is better But look for multi-collinearity issues SE too high? 4.1.1 Updating model specification What other model specifications are possible? 4.1.1.1 Interactions Common terms to add (indeed based on biological plausibility; requiring subject area knowledge) # Interactions ps.formula2 <- as.formula(paste("diabetes", "~", paste(baselinevars, collapse = "+"), "+ education:bmi + gender:age")) ps.formula2 ## diabetes ~ gender + age + race + education + married + bmi + ## education:bmi + gender:age 4.1.1.2 Polynomial terms # polynomials or splines ps.formula3 <- as.formula(paste("diabetes", "~", paste(baselinevars, collapse = "+"), "+ age^2 + age^3")) ps.formula3 ## diabetes ~ gender + age + race + education + married + bmi + ## age^2 + age^3 4.1.1.3 More complex functions # transformations ps.formula4 <- as.formula(paste("diabetes", "~", paste(baselinevars, collapse = "+"), "+ log(age)")) ps.formula4 ## diabetes ~ gender + age + race + education + married + bmi + ## log(age) 4.1.2 Stability of PS How many variables in PS model are too many? Depends on the sample size Too many variables (and too many interaction + polynomials) means too many parameters \\(p\\) to be estimated If large data is available, might not be a problem Again look at the stability: the exposure model coef SEs 4.2 Variables to adjust 4.2.1 Best approach Subject area expertise known from literature Try drawing causal diagram to determine which variables to include 4.2.2 General guideline of type of variables See Brookhart et al. (2006) for a guideline (not based on empirical association in the same data) Observed covariates are used to fix design Which covariates should be selected (based on subject area expertise; not based on empirical correlation analysis): known to be a confounder (causes of \\(Y\\) and \\(A\\)) known to be a cause of the outcome (risk factors of \\(Y\\)) 4.2.3 What NOT to include Two types avoid known instruments or noise variables: SE suffers mediating factors should be avoided (total effect = goal) 4.2.4 Mediators Why not incorporate mediator variable in the PS analysis? One example of a mediator variable in out analysis could be ‘physical exercise’. In the current framework, we do not include mediator variables as we are primarily interested about ‘total effect’, not any decomposition. 4.2.5 Unmeasured confounding What if an important variable is unmeasured / not available in the data? Find a proxy variable that may be associated with that unmeasured variable’s concept. Exists some sensitivity analysis to assess the impact of the unmeasured variable in the analysis. 4.3 Model selection Not encouraged, but popularly done! not encouraged, as this is utilizing empirical associations creates post-selection problem There are debate about this (ideal vs. pragmatism) see Karim, Pang, and Platt (2018) for an example. 4.3.1 Based on association with outcome Selecting just based on association with the outcome (\\(Y\\)) in your data to select covariates is not encouraged separation between outcome and exposure modelling is broken! Usually done in a situation when we are not sure whether a variable should be included in the PS model no clear indication in the literature, or based on subject area knowledge. we are unsure if this is a confounder or risk factor or noise We show here an example that can be considered as a middle-ground keep known confounders + risk factors of outcome use variable selection only on the variables about which we are unsure # Assuming that you are not sure if education and married # variables should be included in the PS analysis # Try outcome modelling as follows: formula.full <- as.formula(paste("cholesterol", "~", "gender + age + race + education+ married + bmi")) fit.0 <- glm(formula.full, family=binomial, data = analytic) scope <- list(upper = ~ gender + age + race + education+ married + bmi, # upper included all variables (known + unsure) lower = ~ gender + age + race + bmi) # lower included only known confounders + risk factors of outcome fitstep <- step(fit.0, scope = scope, trace = FALSE, k = 2, direction = "backward") # k = 2 is equivalant to AIC formula(fitstep) ## cholesterol ~ gender + age + race + bmi # if education, married (one or both) survives this # stepwise, then consider adding that/those in the PS model # If not, discard from the PS model. formula.chosen <- as.formula(paste("diabetes", "~", "gender + age + race + bmi")) formula.chosen ## diabetes ~ gender + age + race + bmi # We, however, did not use this approach below. Stepwise (p-value or criterion based) not recommended depending on sample size, different values can get selected 4.3.2 Based on association with exposure Selecting based on association with the exposure (\\(A\\)) in your data to select covariates can be the worst! May attract instruments strongly discouraged! Below is an example of what NOT to do. # Assume again that you are not sure if education and married # variables should be included in the PS analysis # Try exposure modelling as follows: formula.full.e <- as.formula(paste("diabetes", "~", "gender + age + race + education+ married + bmi")) fit.1 <- glm(formula.full.e, family=binomial, data = analytic) scope <- list(upper = ~ gender + age + race + education+ married + bmi, lower = ~ gender + age + race + bmi) fitstep.e <- step(fit.1, scope = scope, trace = FALSE, k = 2, direction = "backward") formula(fitstep.e) ## diabetes ~ gender + age + race + bmi # This is the chosen PS model by this approach. # We, however, did not use this approach below. 4.4 Alternative modelling strategies Other machine learning alternatives are possible to use instead of logistic regression. tree based methods have better ability to detect non-linearity / non-additivity (model-specification aspect) shrinkage methods - lasso / elastic net may better deal with multi-collinearity ensemble learners / super learners were successfully used shallow/deep learning! 4.5 PS estimation PS is unknown, and needs to be estimated from the fitted exposure model: # extract estimated propensity scores from the fit analytic$PS <- predict(PS.fit, newdata = analytic, type="response") require(cobalt) bal.plot(analytic, var.name = "PS", treat = "diabetes", which = "both", data = analytic) Don’t loose sight that better balance is the ultimate goal for propensity score Prediction of \\(A\\) is just a means to that end (as true PS is unknown) May attract variables highly associated with \\(A\\) References "],["s2.html", "Chapter 5 Step 2: Propensity score Matching 5.1 Matching method NN 5.2 Initial fit 5.3 Fine tuning: add caliper 5.4 Things to keep track of 5.5 Matches 5.6 Other matching algorithms", " Chapter 5 Step 2: Propensity score Matching PS is a continuous variable. Exact matching is not feasible. Below is an example of control patient (treatment = 0) with PS = 0.25 We want to find a treated patient (treatment = 1) with PS closest to 0.25. require(cobalt) library(ggplot2) bal.plot(analytic, var.name = "PS", treat = "diabetes", which = "both", data = analytic) + geom_vline(xintercept=0.25, linetype="dashed", color = "red") 5.1 Matching method NN Match using estimates propensity scores with the following choices (simplest choices) Matching method: nearest-neighbor (NN) matching Can the same subject be chosen only once?: matching without replacement Closeness of the treated-untreated subjects: with caliper = .2*SD of logit of propensity score Ratio of treated-untreated subjects: with 1:1 ratio (pair-matching) 5.2 Initial fit 1:1 NN Match using estimates propensity scores set.seed(123) require(MatchIt) match.obj <- matchit(ps.formula, data = analytic, distance = 'logit', method = "nearest", replace=FALSE, ratio = 1) analytic$PS <- match.obj$distance summary(match.obj$distance) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.003916 0.068128 0.169946 0.211268 0.312987 0.925132 match.obj ## A matchit object ## - method: 1:1 nearest neighbor matching without replacement ## - distance: Propensity score ## - estimated with logistic regression ## - number of obs.: 1562 (original), 660 (matched) ## - target estimand: ATT ## - covariates: gender, age, race, education, married, bmi 5.3 Fine tuning: add caliper 2 SD of logit of the propensity score is suggested as a caliper to allow better comparability of the groups. logitPS <- -log(1/analytic$PS - 1) # logit of the propensity score .2*sd(logitPS) # suggested in the literature ## [1] 0.2606266 # choosing too strict PS has unintended consequences set.seed(123) require(MatchIt) match.obj <- matchit(ps.formula, data = analytic, distance = 'logit', method = "nearest", replace=FALSE, caliper = .2*sd(logitPS), ratio = 1) analytic$PS <- match.obj$distance summary(match.obj$distance) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.003916 0.068128 0.169946 0.211268 0.312987 0.925132 match.obj ## A matchit object ## - method: 1:1 nearest neighbor matching without replacement ## - distance: Propensity score [caliper] ## - estimated with logistic regression ## - caliper: <distance> (0.045) ## - number of obs.: 1562 (original), 632 (matched) ## - target estimand: ATT ## - covariates: gender, age, race, education, married, bmi 5.4 Things to keep track of original sample size matched sample size percent reduction in sample how many matched sets some can be discarded because of no match; whether some sets are unequal 5.5 Matches Taking a closer look at the matches # Ref: https://lists.gking.harvard.edu/pipermail/matchit/2013-October/000559.html matches <- as.data.frame(match.obj$match.matrix) colnames(matches)<-c("matched_unit") matches$matched_unit<-as.numeric( as.character(matches$matched_unit)) matches$treated_unit<-as.numeric(rownames(matches)) matches.only<-matches[!is.na(matches$matched_unit),] head(matches.only) ## matched_unit treated_unit ## 40 8496 40 ## 56 3139 56 ## 65 4192 65 ## 66 94 66 ## 86 2212 86 ## 110 7154 110 matched.data <- match.data(match.obj) head(table(matched.data$subclass)) ## ## 1 2 3 4 5 6 ## 2 2 2 2 2 2 length(table(matched.data$subclass)) ## [1] 316 5.6 Other matching algorithms Variable NN ratio (e.g., 1:10) is usually better. matching with replacement also possible But creates issue when calculating variances (but can be easily handled by controlling for ‘matching weights’). Other possibilities Optimal genetic matching CEM "],["s3.html", "Chapter 6 Step 3: Balance and overlap 6.1 Assessment of Balance by SMD 6.2 SMD vs. P-values 6.3 Vizualization for Overlap 6.4 Variance ratio 6.5 Close inspection of boundaries 6.6 Unsatirfactory balance", " Chapter 6 Step 3: Balance and overlap Balance is more important than prediction! Criteria to assess success of step 2: PS estimation better balance better overlap [no extrapolation!] PS = 0 or PS = 1 needs close inspection 6.1 Assessment of Balance by SMD balance = similarity of the covariate distributions Full data: tab1e <- CreateTableOne(vars = baselinevars, data = analytic, strata = "diabetes", includeNA = TRUE, test = TRUE, smd = TRUE) print(tab1e, showAllLevels = FALSE, smd = TRUE, test = TRUE) ## Stratified by diabetes ## 0 1 p test SMD ## n 1232 330 ## gender = Male (%) 738 (59.9) 221 (67.0) 0.023 0.147 ## age (mean (SD)) 50.54 (17.23) 63.04 (12.87) <0.001 0.822 ## race (%) 0.110 0.151 ## Black 253 (20.5) 71 (21.5) ## Hispanic 220 (17.9) 64 (19.4) ## Other 169 (13.7) 59 (17.9) ## White 590 (47.9) 136 (41.2) ## education (%) 0.005 0.186 ## College 649 (52.7) 157 (47.6) ## High.School 518 (42.0) 140 (42.4) ## School 65 ( 5.3) 33 (10.0) ## married (%) <0.001 0.282 ## Married 727 (59.0) 194 (58.8) ## Never.married 201 (16.3) 27 ( 8.2) ## Previously.married 304 (24.7) 109 (33.0) ## bmi (mean (SD)) 29.14 (7.03) 33.01 (7.65) <0.001 0.526 Matched data: matched.data <- match.data(match.obj) tab1m <- CreateTableOne(vars = baselinevars, strata = "diabetes", data = matched.data, includeNA = TRUE, test = TRUE, smd = TRUE) Compare the similarity of baseline characteristics between treated and untreated subjects in a the propensity score-matched sample. In this case, we will compare SMD < 0.1 or not. In some literature, other generous values (0.25) are proposed. (Austin 2011b) print(tab1m, showAllLevels = FALSE, smd = TRUE, test = FALSE) ## Stratified by diabetes ## 0 1 SMD ## n 316 316 ## gender = Male (%) 218 (69.0) 212 (67.1) 0.041 ## age (mean (SD)) 63.03 (13.48) 62.67 (12.87) 0.027 ## race (%) 0.105 ## Black 79 (25.0) 68 (21.5) ## Hispanic 58 (18.4) 61 (19.3) ## Other 44 (13.9) 53 (16.8) ## White 135 (42.7) 134 (42.4) ## education (%) 0.007 ## College 153 (48.4) 152 (48.1) ## High.School 133 (42.1) 134 (42.4) ## School 30 ( 9.5) 30 ( 9.5) ## married (%) 0.099 ## Married 183 (57.9) 186 (58.9) ## Never.married 20 ( 6.3) 27 ( 8.5) ## Previously.married 113 (35.8) 103 (32.6) ## bmi (mean (SD)) 32.38 (7.62) 32.63 (7.20) 0.035 require(cobalt) bal.plot(match.obj, var.name = "distance", which = "both", type = "histogram", mirror = TRUE) bal.tab(match.obj, un = TRUE, thresholds = c(m = .1)) ## Call ## matchit(formula = ps.formula, data = analytic, method = "nearest", ## distance = "logit", replace = FALSE, caliper = 0.2 * sd(logitPS), ## ratio = 1) ## ## Balance Measures ## Type Diff.Un Diff.Adj M.Threshold ## distance Distance 0.9493 0.0276 Balanced, <0.1 ## gender_Male Binary 0.0707 -0.0190 Balanced, <0.1 ## age Contin. 0.9715 -0.0278 Balanced, <0.1 ## race_Black Binary 0.0098 -0.0348 Balanced, <0.1 ## race_Hispanic Binary 0.0154 0.0095 Balanced, <0.1 ## race_Other Binary 0.0416 0.0285 Balanced, <0.1 ## race_White Binary -0.0668 -0.0032 Balanced, <0.1 ## education_College Binary -0.0510 -0.0032 Balanced, <0.1 ## education_High.School Binary 0.0038 0.0032 Balanced, <0.1 ## education_School Binary 0.0472 0.0000 Balanced, <0.1 ## married_Married Binary -0.0022 0.0095 Balanced, <0.1 ## married_Never.married Binary -0.0813 0.0222 Balanced, <0.1 ## married_Previously.married Binary 0.0835 -0.0316 Balanced, <0.1 ## bmi Contin. 0.5051 0.0338 Balanced, <0.1 ## ## Balance tally for mean differences ## count ## Balanced, <0.1 14 ## Not Balanced, >0.1 0 ## ## Variable with the greatest mean difference ## Variable Diff.Adj M.Threshold ## race_Black -0.0348 Balanced, <0.1 ## ## Sample sizes ## Control Treated ## All 1232 330 ## Matched 316 316 ## Unmatched 916 14 love.plot(match.obj, binary = "std", thresholds = c(m = .1)) 6.2 SMD vs. P-values Possible to get p-values to check balance: but strongly discouraged P-value based balance assessment can be influenced by sample size print(tab1m, showAllLevels = FALSE, smd = FALSE, test = TRUE) ## Stratified by diabetes ## 0 1 p test ## n 316 316 ## gender = Male (%) 218 (69.0) 212 (67.1) 0.670 ## age (mean (SD)) 63.03 (13.48) 62.67 (12.87) 0.733 ## race (%) 0.629 ## Black 79 (25.0) 68 (21.5) ## Hispanic 58 (18.4) 61 (19.3) ## Other 44 (13.9) 53 (16.8) ## White 135 (42.7) 134 (42.4) ## education (%) 0.996 ## College 153 (48.4) 152 (48.1) ## High.School 133 (42.1) 134 (42.4) ## School 30 ( 9.5) 30 ( 9.5) ## married (%) 0.465 ## Married 183 (57.9) 186 (58.9) ## Never.married 20 ( 6.3) 27 ( 8.5) ## Previously.married 113 (35.8) 103 (32.6) ## bmi (mean (SD)) 32.38 (7.62) 32.63 (7.20) 0.662 Assessment of balance in the matched data smd.res <- ExtractSmd(tab1m) t(round(smd.res,2)) ## gender age race education married bmi ## 1 vs 2 0.04 0.03 0.11 0.01 0.1 0.03 6.3 Vizualization for Overlap boxplot(PS ~ diabetes, data = analytic, lwd = 2, ylab = 'PS') stripchart(PS ~ diabetes, vertical = TRUE, data = analytic, method = "jitter", add = TRUE, pch = 20, col = 'blue') plot(match.obj, type = "jitter") ## [1] "To identify the units, use first mouse button; to stop, use second." ## integer(0) Vizualization for assessing overlap issues plot(match.obj, type = "hist") 6.4 Variance ratio Variance ratios \\(\\sim\\) 1 means: equal variances in groups group balance could vary from 1/2 to 2 other cut-points are suggested as well (0.8 to 1.2) See Stuart (2010) and Austin (2009) require(cobalt) baltab.res <- bal.tab(x = match.obj, data = analytic, treat = analytic$diabetes, disp.v.ratio = TRUE) baltab.res ## Call ## matchit(formula = ps.formula, data = analytic, method = "nearest", ## distance = "logit", replace = FALSE, caliper = 0.2 * sd(logitPS), ## ratio = 1) ## ## Balance Measures ## Type Diff.Adj V.Ratio.Adj ## distance Distance 0.0276 1.0992 ## gender_Male Binary -0.0190 . ## age Contin. -0.0278 0.9114 ## race_Black Binary -0.0348 . ## race_Hispanic Binary 0.0095 . ## race_Other Binary 0.0285 . ## race_White Binary -0.0032 . ## education_College Binary -0.0032 . ## education_High.School Binary 0.0032 . ## education_School Binary 0.0000 . ## married_Married Binary 0.0095 . ## married_Never.married Binary 0.0222 . ## married_Previously.married Binary -0.0316 . ## bmi Contin. 0.0338 0.8928 ## ## Sample sizes ## Control Treated ## All 1232 330 ## Matched 316 316 ## Unmatched 916 14 6.5 Close inspection of boundaries boxplot(PS ~ diabetes, data = matched.data, lwd = 2, ylab = 'PS', ylim=c(0,1)) stripchart(PS ~ diabetes, vertical = TRUE, data = matched.data, method = "jitter", add = TRUE, pch = 20, col = 'blue') abline(h=c(0+0.05,1-0.05), col = "red", lty = 2) Sensitivity analysis should be done with trimming. Have consequences in interpretation target population may be unclear 6.6 Unsatirfactory balance Iterative process Best strategy is to go back to step 2, and make changes in the PS model specification References "],["s4.html", "Chapter 7 Step 4: Outcome modelling 7.1 Crude outcome model 7.2 Double-adjustment 7.3 Adjusted outcome model 7.4 Variance considerations 7.5 Estimate obtained", " Chapter 7 Step 4: Outcome modelling Some flexibility in choosing outcome model considered independent of exposure modelling some propose double robust approach adjusting imbalanced covariates only? double-adjustment may address residual confounding (Nguyen et al. 2017) 7.1 Crude outcome model Estimate the effect of treatment on outcomes using propensity score-matched sample fit3 <- glm(cholesterol~diabetes, family=binomial, data = matched.data) publish(fit3) ## Variable Units OddsRatio CI.95 p-value ## diabetes 0.90 [0.54;1.50] 0.6984 7.2 Double-adjustment Estimate the effect of treatment on outcomes using propensity score-matched sample, and adjust for imbalanced covariate fit3r <- glm(cholesterol~diabetes + race, family=binomial, data = matched.data) publish(fit3r) ## Variable Units OddsRatio CI.95 p-value ## diabetes 0.89 [0.54;1.49] 0.6657 ## race Black Ref ## Hispanic 0.96 [0.46;2.02] 0.9165 ## Other 1.32 [0.63;2.78] 0.4581 ## White 0.58 [0.30;1.13] 0.1095 7.3 Adjusted outcome model Adjust for all covariates, again! (suggested) out.formula <- as.formula(paste("cholesterol", "~ diabetes +", paste(baselinevars, collapse = "+"))) out.formula ## cholesterol ~ diabetes + gender + age + race + education + married + ## bmi fit3b <- glm(out.formula, family=binomial, data = matched.data) publish(fit3b) ## Variable Units OddsRatio CI.95 p-value ## diabetes 0.86 [0.51;1.46] 0.5794126 ## gender Female Ref ## Male 0.38 [0.21;0.69] 0.0012767 ## age 0.95 [0.93;0.97] < 1e-04 ## race Black Ref ## Hispanic 0.72 [0.31;1.65] 0.4346787 ## Other 0.77 [0.34;1.73] 0.5224157 ## White 0.51 [0.25;1.04] 0.0649791 ## education College Ref ## High.School 0.70 [0.39;1.24] 0.2215142 ## School 0.93 [0.35;2.43] 0.8791455 ## married Married Ref ## Never.married 0.48 [0.15;1.54] 0.2173180 ## Previously.married 0.84 [0.45;1.57] 0.5900732 ## bmi 0.93 [0.89;0.97] 0.0005547 The above analysis do not take matched pair into consideration while regressing. 7.4 Variance considerations Literature proposes different strategies: do not control for pairs / clusters use glm as is control for pairs / clusters use cluster option (preferred) use GEE or use conditional logistic 7.4.1 Cluster option Here is an example using cluster option: require(jtools) summ(fit3b, rubust = "HC0", confint = TRUE, digists = 3, cluster = "subclass", model.info = FALSE, model.fit = FALSE, exp = TRUE) exp(Est.) 2.5% 97.5% z val. p (Intercept) 100.02 8.74 1144.55 3.70 0.00 diabetes 0.86 0.51 1.46 -0.55 0.58 genderMale 0.38 0.21 0.69 -3.22 0.00 age 0.95 0.93 0.97 -4.47 0.00 raceHispanic 0.72 0.31 1.65 -0.78 0.43 raceOther 0.77 0.34 1.73 -0.64 0.52 raceWhite 0.51 0.25 1.04 -1.85 0.06 educationHigh.School 0.70 0.39 1.24 -1.22 0.22 educationSchool 0.93 0.35 2.43 -0.15 0.88 marriedNever.married 0.48 0.15 1.54 -1.23 0.22 marriedPreviously.married 0.84 0.45 1.57 -0.54 0.59 bmi 0.93 0.89 0.97 -3.45 0.00 Standard errors: MLE 7.4.2 Bootstrap Bootstrap for matched pair for WOR (Austin and Small 2014) straightforward method to estimate SE may not be appropriate for WR Following is an example of block bootstrap; see MatchIt package vignettes for additional details. # For binary outcomes # with covariate adjustment and bootstrapping pair_ids <- levels(matched.data$subclass) est_fun <- function(pairs, i) { # See MatchIt vignettes #Compute number of times each pair is present numreps <- table(pairs[i]) #For each pair p, copy corresponding matched.data row indices numreps[p] times ids <- unlist(lapply(pair_ids[pair_ids %in% names(numreps)], function(p) rep(which(matched.data$subclass == p), numreps[p]))) #Subset matched.data with block bootstrapped ids matched.data_boot <- matched.data[ids,] #Fitting outcome the model out.formula <- as.formula(paste("cholesterol", "~ diabetes +", paste(baselinevars, collapse = "+"))) fit_boot <- glm(out.formula, family = binomial(link = "logit"), weights = weights, data = matched.data_boot) #Estimate potential outcomes for each unit #Under control matched.data_boot$diabetes <- 0 P0 <- weighted.mean(predict(fit_boot, matched.data_boot, type = "response"), w = matched.data_boot$weights) Odds0 <- P0 / (1 - P0) #Under treatment matched.data_boot$diabetes <- 1 P1 <- weighted.mean(predict(fit_boot, matched.data_boot, type = "response"), w = matched.data_boot$weights) Odds1 <- P1 / (1 - P1) #Return marginal odds ratio return(Odds1 / Odds0) } boot_est <- boot(pair_ids, est_fun, R = 1000) # R must be larger than # of data rows boot_est ## ## ORDINARY NONPARAMETRIC BOOTSTRAP ## ## ## Call: ## boot(data = pair_ids, statistic = est_fun, R = 1000) ## ## ## Bootstrap Statistics : ## original bias std. error ## t1* 0.8706933 0.04061083 0.2376491 boot.ci(boot_est, type = "bca") ## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS ## Based on 1000 bootstrap replicates ## ## CALL : ## boot.ci(boot.out = boot_est, type = "bca") ## ## Intervals : ## Level BCa ## 95% ( 0.5239, 1.3936 ) ## Calculations and Intervals on Original Scale 7.5 Estimate obtained The example compared diabetic (a treated group; target) vs Not diabetic (untreated). Thc corresponding treatment effect estimate is known as Average Treatment Effects on the Treated (ATT) Other estimates from PS analysis (e.g., PS weighting) are possible that compared the whole population what if everyone treated vs. what if nobody was treated (ATE) You can see further comparison of results elsewhere. References "],["compare.html", "Chapter 8 PS vs. Regression 8.1 Data Simulation 8.2 Treatment effect from counterfactuals 8.3 Treatment effect from Regression 8.4 Treatment effect from PS 8.5 Non-linear Model", " Chapter 8 PS vs. Regression 8.1 Data Simulation Simplified simulation example, so that we know the true parameter \\(\\theta\\). \\(Y\\) : Outcome Cholesterol levels (continuous) \\(A\\) : Exposure Diabetes \\(L\\) : Known Confounders age (continuous) Confounder \\(L\\) (continuous) \\(L\\) ~ N(mean = 10, sd = 1) Treatment \\(A\\) (binary 0/1) Logit \\(P(A = 1)\\) ~ 0.4 L Outcome \\(Y\\) (continuous) Y ~ N(mean = 3 L + \\(\\theta\\) A, sd = 1) True parameter: \\(\\theta = 0.7\\) We want to see, how close the estimates (compared to this \\(\\theta = 0.7\\)) are when we try to estimate this parameter using different methods: regression PS Note that, given the data generating mechanism, \\(L\\) is a confounders, and should be adjusted. require(simcausal) D <- DAG.empty() D <- D + node("L", distr = "rnorm", mean = 10, sd = 1) + node("A", distr = "rbern", prob = plogis(0.4*L)) + node("Y", distr = "rnorm", mean = 3 * L + 0.7 * A, sd = 1) Dset <- set.DAG(D) plotDAG(Dset, xjitter = 0.1, yjitter = .9, edge_attrs = list(width = 0.5, arrow.width = 0.4, arrow.size = 1.7), vertex_attrs = list(size = 18, label.cex = 1.8)) ## using the following vertex attributes: ## 181.8NAdarkbluenone0 ## using the following edge attributes: ## 0.50.41.7black1 # Data generating function fnc <- function(n = 10, seedx = 123){ require(simcausal) set.seed(seedx) D <- DAG.empty() D <- D + node("L", distr = "rnorm", mean = 10, sd = 1) + node("A", distr = "rbern", prob = plogis(0.4*L)) + node("Y", distr = "rnorm", mean = 3 * L + 0.7 * A, sd = 1) Dset <- set.DAG(D) A1 <- node("A", distr = "rbern", prob = 1) Dset <- Dset + action("A1", nodes = A1) A0 <- node("A", distr = "rbern", prob = 0) Dset <- Dset + action("A0", nodes = A0) Cdat <- sim(DAG = Dset, actions = c("A1", "A0"), n = n, rndseed = 123) generated.data <- round(cbind(Cdat$A1[c("ID", "L", "Y")],Cdat$A0[c("Y")]),2) names(generated.data) <- c("ID", "L", "Y1", "Y0") generated.data <- generated.data[order(generated.data$L, generated.data$ID),] generated.data$A <- sample(c(0,1),n, replace = TRUE) generated.data$Y <- ifelse(generated.data$A==0, generated.data$Y0, generated.data$Y1) counterfactual.dataset<- generated.data[order(generated.data$ID) , ][c("ID","L","A","Y1","Y0")] observed.dataset<- generated.data[order(generated.data$ID) , ][c("ID","L","A","Y")] return(list(counterfactual=counterfactual.dataset, observed=observed.dataset)) } 10 observations from the data generation: result.data <- fnc(n=10) result.data ## $counterfactual ## ID L A Y1 Y0 ## 1 1 9.44 0 30.24 29.54 ## 2 2 9.77 1 30.37 29.67 ## 3 3 11.56 0 35.78 35.08 ## 4 4 10.07 0 31.02 30.32 ## 5 5 10.13 1 30.53 29.83 ## 6 6 11.72 0 37.63 36.93 ## 7 7 10.46 1 32.58 31.88 ## 8 8 8.73 0 24.94 24.24 ## 9 9 9.31 0 29.34 28.64 ## 10 10 9.55 1 28.89 28.19 ## ## $observed ## ID L A Y ## 1 1 9.44 0 29.54 ## 2 2 9.77 1 30.37 ## 3 3 11.56 0 35.08 ## 4 4 10.07 0 30.32 ## 5 5 10.13 1 30.53 ## 6 6 11.72 0 36.93 ## 7 7 10.46 1 32.58 ## 8 8 8.73 0 24.24 ## 9 9 9.31 0 28.64 ## 10 10 9.55 1 28.89 8.2 Treatment effect from counterfactuals True \\(\\theta\\) can be obtained from counterfactual data: result.data$counterfactual$TE <- result.data$counterfactual$Y1- result.data$counterfactual$Y0 result.data$counterfactual ## ID L A Y1 Y0 TE ## 1 1 9.44 0 30.24 29.54 0.7 ## 2 2 9.77 1 30.37 29.67 0.7 ## 3 3 11.56 0 35.78 35.08 0.7 ## 4 4 10.07 0 31.02 30.32 0.7 ## 5 5 10.13 1 30.53 29.83 0.7 ## 6 6 11.72 0 37.63 36.93 0.7 ## 7 7 10.46 1 32.58 31.88 0.7 ## 8 8 8.73 0 24.94 24.24 0.7 ## 9 9 9.31 0 29.34 28.64 0.7 ## 10 10 9.55 1 28.89 28.19 0.7 8.3 Treatment effect from Regression What happens in observed data for a sample of size 10? round(coef(glm(Y ~ A, family="gaussian", data=result.data$observed)),2) ## (Intercept) A ## 30.79 -0.20 round(coef(glm(Y ~ A + L, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L ## -5.76 0.38 3.61 What happens in observed data for a sample of size 10000? result.data <- fnc(n=10000) round(coef(glm(Y ~ A, family="gaussian", data=result.data$observed)),2) ## (Intercept) A ## 30.06 0.56 round(coef(glm(Y ~ A + L, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L ## -0.07 0.70 3.01 8.4 Treatment effect from PS Propensity score model fitting: require(MatchIt) match.obj <- matchit(A ~ L, method = "nearest", data = result.data$observed, distance = 'logit', caliper = 0.001, replace = FALSE, ratio = 1) ## Warning: Fewer control units than treated units; not all treated units will get ## a match. match.obj ## A matchit object ## - method: 1:1 nearest neighbor matching without replacement ## - distance: Propensity score [caliper] ## - estimated with logistic regression ## - caliper: <distance> (0) ## - number of obs.: 10000 (original), 8290 (matched) ## - target estimand: ATT ## - covariates: L Results from step 4: crude matched.data <- match.data(match.obj) Results from step 4: adjusted round(coef(glm(Y ~ A, family="gaussian", data=matched.data)),2) ## (Intercept) A ## 30.01 0.70 round(coef(glm(Y ~ A+L, family="gaussian", data=matched.data)),2) ## (Intercept) A L ## -0.06 0.70 3.01 8.5 Non-linear Model 8.5.1 Data generation \\(Y\\) : Outcome Cholesterol levels (continuous) \\(A\\) : Exposure Diabetes \\(L\\) : Known Confounders age (continuous) Confounder \\(L\\) (continuous) \\(L\\) ~ N(mean = 10, sd = 1) Treatment \\(A\\) (binary 0/1) Logit \\(P(A = 1)\\) ~ 0.4 L Outcome \\(Y\\) (continuous) Y ~ N(mean = 3 \\(L^3\\) + \\(\\theta\\) A, sd = 1) The only difference is \\(L^3\\) instead of \\(L\\) in the outcome mode. Again, \\(\\theta = 0.7\\) # Data generating function fnc2 <- function(n = 10, seedx = 123){ require(simcausal) set.seed(seedx) D <- DAG.empty() D <- D + node("L", distr = "rnorm", mean = 10, sd = 1) + node("A", distr = "rbern", prob = plogis(0.4*L)) + node("Y", distr = "rnorm", mean = 3 * L^3 + 0.7 * A, sd = 1) Dset <- set.DAG(D) A1 <- node("A", distr = "rbern", prob = 1) Dset <- Dset + action("A1", nodes = A1) A0 <- node("A", distr = "rbern", prob = 0) Dset <- Dset + action("A0", nodes = A0) Cdat <- sim(DAG = Dset, actions = c("A1", "A0"), n = n, rndseed = 123) generated.data <- round(cbind(Cdat$A1[c("ID", "L", "Y")],Cdat$A0[c("Y")]),2) names(generated.data) <- c("ID", "L", "Y1", "Y0") generated.data <- generated.data[order(generated.data$L, generated.data$ID),] generated.data$A <- sample(c(0,1),n, replace = TRUE) generated.data$Y <- ifelse(generated.data$A==0, generated.data$Y0, generated.data$Y1) counterfactual.dataset<- generated.data[order(generated.data$ID) , ][c("ID","L","A","Y1","Y0")] observed.dataset<- generated.data[order(generated.data$ID) , ][c("ID","L","A","Y")] return(list(counterfactual=counterfactual.dataset, observed=observed.dataset)) } 8.5.2 Regression result.data <- fnc2(n=10000) Crude estimates round(coef(glm(Y ~ A, family="gaussian", data=result.data$observed)),2) ## (Intercept) A ## 3082.75 10.44 Adjusted estimates fit <- glm(Y ~ A + L, family="gaussian", data=result.data$observed) round(coef(fit),2) ## (Intercept) A L ## -6001.49 -3.56 909.34 In regression adjustments, the results could be subject to “model extrapolation” based on linearity assumption. It is sometimes difficult to know whether the adjusted effect is based on extrapolation. Especially true in observational settings. PS may not need such linearity assumption (when non-parametric approaches used for prediction). Don’t necessarily mean non-parametric approaches are the best option though! 8.5.3 PS Matching with PS match.obj <- matchit(A ~ L, method = "nearest", data = result.data$observed, distance = 'logit', replace = FALSE, caliper = 0.001, ratio = 1) match.obj ## A matchit object ## - method: 1:1 nearest neighbor matching without replacement ## - distance: Propensity score [caliper] ## - estimated with logistic regression ## - caliper: <distance> (0) ## - number of obs.: 10000 (original), 8202 (matched) ## - target estimand: ATT ## - covariates: L matched.data <- match.data(match.obj) Results from step 4: crude round(coef(glm(Y ~ A, family="gaussian", data=matched.data)),2) ## (Intercept) A ## 3082.38 0.69 Results from step 4: adjusted round(coef(glm(Y ~ A+L, family="gaussian", data=matched.data)),2) ## (Intercept) A L ## -5994.17 0.69 907.17 8.5.4 Machine learning Using gradient boosted method for PS estimation require(twang) result.data$observed$S <- 0 ps.gbm <- ps(A ~ L + S,data = result.data$observed,estimand = "ATT",n.trees=1000) names(ps.gbm) summary(ps.gbm$ps$es.mean.ATT) result.data$observed$ps <- ps.gbm$ps$es.mean.ATT Matching with PS generated from gradient boosted method require(Matching) match.obj2 <- Match(Y=result.data$observed$Y, Tr=result.data$observed$A, X=result.data$observed$ps, M=1, caliper = 0.001, replace=FALSE) summary(match.obj2) ## ## Estimate... -1.5796 ## SE......... 2.1149 ## T-stat..... -0.7469 ## p.val...... 0.45512 ## ## Original number of observations.............. 10000 ## Original number of treated obs............... 4987 ## Matched number of observations............... 4579 ## Matched number of observations (unweighted). 4579 ## ## Caliper (SDs)........................................ 0.001 ## Number of obs dropped by 'exact' or 'caliper' 408 matched.data2 <- result.data$observed[c(match.obj2$index.treated, match.obj2$index.control),] mb <- MatchBalance(A~L, data=result.data$observed, match.out=match.obj2, nboots=10) Results from step 4: crude round(coef(glm(Y ~ A, family="gaussian", data=matched.data2)),2) ## (Intercept) A ## 3078.72 -1.58 Results from step 4: adjusted round(coef(glm(Y ~ A+L, family="gaussian", data=matched.data2)),2) ## (Intercept) A L ## -6007.72 0.89 909.14 Powerful machine learning method is good at prediction. Propensity score methods rely on obtaining good balance. Always a good idea to check analysis with multiple sensitivity analysis. 8.5.5 Regression is doomed? Not really. Always a god idea to check the diagnostic plots to find any indication of assumption violation: par(mfrow=c(2,2)) plot(fit) residuals <- residuals(fit) par(mfrow=c(1,1)) plot(y=residuals,x=result.data$observed$Y) Residual plot has a pattern! Indication that we may meed to reset the model-specification. fit2 <- glm(Y ~ A + poly(L,2), family="gaussian", data=result.data$observed) round(coef(fit2),2) ## (Intercept) A poly(L, 2)1 poly(L, 2)2 ## 3087.50 0.92 90803.84 12753.21 fit3 <- glm(Y ~ A + poly(L,3), family="gaussian", data=result.data$observed) round(coef(fit3),2) ## (Intercept) A poly(L, 3)1 poly(L, 3)2 poly(L, 3)3 ## 3087.61 0.70 90803.92 12753.02 723.60 "],["misspecify.html", "Chapter 9 PS vs. Double robust methods 9.1 Complex Data Simulation 9.2 Understanding finite sample bias 9.3 Estimation using different methods", " Chapter 9 PS vs. Double robust methods Understanding sources of bias is important Bias = Average of our estimates in repeated sampling - true treatment effect = bias due to finite sample + bias due to not designing the study properly + bias due to model misspecification The past part is often less talked about. Nonparametric or machine learning methods could be an advantage if used properly to reduce bias due to this source. A topic for a different tutorial, but here we will show an example of the implication of avoiding the possibility of model-misspecification. 9.1 Complex Data Simulation Complex simulation example, so that we know the true parameter \\(\\theta\\). \\(Y\\) : Outcome (continuous) \\(A\\) : Exposure (binary) \\(C\\) (1-4): true confounders (continuous) \\(L\\) (1-4): measured and transformed version of true confounders (continuous) True Exposure Model \\[\\begin{equation} \\begin{aligned} \\pi = P(A=1|C) &= f(C) \\\\ &= \\frac{1}{1+exp[\\alpha_0 + \\alpha_1 C_1 + \\alpha_2 C_2 + \\alpha_3 C_3 + \\alpha_4 C_4 ]}\\nonumber \\end{aligned} \\end{equation}\\] True Outcome Model \\[\\begin{equation} \\begin{aligned} E(Y|A,C) &= g(A,C)\\\\ &= \\beta_0 + \\theta A + \\beta_1 C_1 + \\beta_2 C_2 + \\beta_3 C_3 + \\beta_4 C_4 \\nonumber \\end{aligned} \\end{equation}\\] In our example, \\(\\theta\\) = 6, which is our treatment effect. Outcomes and exposures are complex functions of measured covariates Using example from Kang and Schafer (2007), Naimi, Mishler, and Kennedy (2017) and Balzer and Westling (2021), let us assume that we don’t observed the \\(C\\) variables directly, and we only have access to a set of transformed covariates \\(L\\)s. \\[\\begin{equation} \\begin{aligned} L_1 &= \\exp(C1/2) \\nonumber\\\\ L_2 &= [(C2/(1+\\exp(C1)) + 10)]\\nonumber\\\\ L_3 &= [(C1*C3/25 + 0.6)^3] \\nonumber\\\\ L_4 &= [(C2 + C4 + 20)^2\\nonumber \\end{aligned} \\end{equation}\\] create.data <- function(n, te = 6){ # original covariates that generates A and Y C1 <- rnorm(n,0,1) C2 <- rnorm(n,0,1) C3 <- rnorm(n,0,1) C4 <- rnorm(n,0,1) pscore <- plogis(-1 +log(1.75)*(C1+C2+C3+C4) ) # treatment generation model A <- rbinom(n, 1, pscore) eps <- rnorm(n,0,6) # error term get.Y <- function(A, TE=te) {120+TE*A+3*(C1+C2+C3+C4)+eps} Y1 <- get.Y(A=1, TE=te) Y0 <- get.Y(A=0, TE=te) # potential outcomes PO <- data.frame(cbind(pscore, Y1,Y0, C1, C2, C3, C4)) # outcome generation model Y <- get.Y(A=A, TE=te) # misspecified versions of covariates L1 <- exp(C1/2) L2 <- C2/(1+exp(C1)) + 10 L3 <- (C1*C3/25 + 0.6)^3 L4 <- (C2 + C4 + 20)^2 obs <- data.frame(cbind(L1,L2,L3,L4,A,Y)) return(list(observed=obs, Potential.Outcome= PO, ate = mean(PO$Y1)-mean(PO$Y0))) } create.data(6) ## $observed ## L1 L2 L3 L4 A Y ## 1 2.2374436 10.06754 0.1625330 419.8183 1 120.3914 ## 2 1.7679300 10.02612 0.1537718 441.0633 1 113.7917 ## 3 1.8338882 10.06432 0.2636953 489.8636 0 133.9537 ## 4 1.3741692 10.04336 0.1926407 389.0893 0 124.1439 ## 5 0.9565624 11.25774 0.2182527 556.0792 1 138.0037 ## 6 0.8111285 10.77576 0.2057078 512.2704 0 127.6085 ## ## $Potential.Outcome ## pscore Y1 Y0 C1 C2 C3 C4 ## 1 0.4265123 120.3914 114.3914 1.6106679 0.4056800 -0.8423000 0.08378755 ## 2 0.3564975 113.7917 107.7917 1.1396187 0.1077560 -1.4095507 0.89375158 ## 3 0.7938446 139.9537 133.9537 1.2128768 0.2806335 0.8504563 1.85222880 ## 4 0.2154750 130.1439 124.1439 0.6356986 0.1252420 -0.8832483 -0.39989653 ## 5 0.6518198 138.0037 132.0037 -0.0888186 2.4085868 -0.5850746 1.17274430 ## 6 0.6371594 133.6085 127.6085 -0.4186576 1.2861567 0.5783549 1.34723419 ## ## $ate ## [1] 6 9.2 Understanding finite sample bias First try: set.seed(1) result.data <- create.data(100) round(coef(glm(Y ~ A + L1 + L2 + L3 + L4, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L1 L2 L3 L4 ## 91.23 8.46 3.50 0.81 -19.83 0.05 Second try: set.seed(22) result.data <- create.data(100) round(coef(glm(Y ~ A + L1 + L2 + L3 + L4, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L1 L2 L3 L4 ## 70.05 10.20 4.48 2.29 9.53 0.05 Third try: set.seed(33) result.data <- create.data(100) round(coef(glm(Y ~ A + L1 + L2 + L3 + L4, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L1 L2 L3 L4 ## 97.51 7.13 2.32 -0.40 12.16 0.05 9.3 Estimation using different methods We now work with a larger data: \\(n = 10,000\\) set.seed(123) result.data <- create.data(10000) 9.3.1 Regression round(coef(glm(Y ~ A + L1 + L2 + L3 + L4, family="gaussian", data=result.data$observed)),2) ## (Intercept) A L1 L2 L3 L4 ## 88.23 7.70 4.40 -0.12 -3.13 0.07 9.3.2 Propensity score Propensity score model fitting: require(MatchIt) match.obj <- matchit(A ~ L1 + L2 + L3 + L4, method = "nearest", data = result.data$observed, distance = 'logit', caliper = 0.2, replace = FALSE, ratio = 1) match.obj ## A matchit object ## - method: 1:1 nearest neighbor matching without replacement ## - distance: Propensity score [caliper] ## - estimated with logistic regression ## - caliper: <distance> (0.034) ## - number of obs.: 10000 (original), 5790 (matched) ## - target estimand: ATT ## - covariates: L1, L2, L3, L4 matched.data <- match.data(match.obj) Results from step 4: crude round(coef(glm(Y ~ A, family="gaussian", data=matched.data)),2) ## (Intercept) A ## 121.51 7.93 Results from step 4: double adjustment round(coef(glm(Y ~ A + L1 + L2 + L3 + L4, family="gaussian", data=matched.data)),2) ## (Intercept) A L1 L2 L3 L4 ## 89.56 7.66 4.11 -0.36 8.38 0.07 9.3.3 Double machine learning method We are using DML with only one learner: res <- result.data$observed require(DoubleML) dd <- DoubleMLData$new(res, y_col = "Y", d_cols = "A", x_cols = c("L1","L2","L3","L4")) library(mlr3) library(mlr3learners) lgr::get_logger("mlr3")$set_threshold("warn") learner = lrn("regr.ranger", num.trees=500, mtry=floor(sqrt(5)), max.depth=5, min.node.size=2) ml_g = learner$clone() # outcome modelling ml_m = learner$clone() # exposure modelling set.seed(1234) dml.res <- DoubleMLPLR$new(dd, ml_g=ml_g, ml_m=ml_m) dml.res$fit() print(dml.res) ## ================= DoubleMLPLR Object ================== ## ## ## ------------------ Data summary ------------------ ## Outcome variable: Y ## Treatment variable(s): A ## Covariates: L1, L2, L3, L4 ## Instrument(s): ## No. Observations: 10000 ## ## ------------------ Score & algorithm ------------------ ## Score function: partialling out ## DML algorithm: dml2 ## ## ------------------ Machine learner ------------------ ## ml_g: regr.ranger ## ml_m: regr.ranger ## ## ------------------ Resampling ------------------ ## No. folds: 5 ## No. repeated sample splits: 1 ## Apply cross-fitting: TRUE ## ## ------------------ Fit summary ------------------ ## Estimates and significance testing of the effect of target variables ## Estimate. Std. Error t value Pr(>|t|) ## A 7.0114 0.1532 45.76 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 dml.res$summary() ## Estimates and significance testing of the effect of target variables ## Estimate. Std. Error t value Pr(>|t|) ## A 7.0114 0.1532 45.76 <2e-16 *** ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 9.3.4 Augmented Inverse probability weighting SL.lib=c('SL.glm', 'SL.step.interaction', 'SL.earth', 'SL.mean') require(AIPW) require(ggplot2) res <- result.data$observed aipw_sl <- AIPW$new(Y=res$Y, A=res$A, W=res[c("L1","L2","L3","L4")], Q.SL.library=SL.lib,g.SL.library=SL.lib, k_split=1,verbose=TRUE) aipw_sl$fit() aipw_sl$summary(g.bound = 0.025) print(aipw_sl$result, digits = 2) ## Estimate SE 95% LCL 95% UCL N ## Risk of exposure 126.3 0.18 126.0 126.7 3055 ## Risk of control 119.9 0.10 119.7 120.1 6945 ## Risk Difference 6.4 0.19 6.1 6.8 10000 9.3.5 Double robust method (TMLE) TMLE with superlearner tmle.res <- ltmle(data=result.data$observed, Anodes='A', Ynodes='Y', abar=list(1,0), SL.library=c('SL.glm', 'SL.step.interaction', 'SL.earth', 'SL.mean'), estimate.time=F, SL.cvControl=list(V=10), gbounds=c(0.025, 0.975)) summary(tmle.res) ## Estimator: tmle ## Call: ## ltmle(data = result.data$observed, Anodes = "A", Ynodes = "Y", ## abar = list(1, 0), gbounds = c(0.025, 0.975), SL.library = c("SL.glm", ## "SL.step.interaction", "SL.earth", "SL.mean"), SL.cvControl = list(V = 10), ## estimate.time = F) ## ## Treatment Estimate: ## Parameter Estimate: 126.3 ## Estimated Std Err: 0.15795 ## p-value: <2e-16 ## 95% Conf Interval: (125.99, 126.61) ## ## Control Estimate: ## Parameter Estimate: 119.92 ## Estimated Std Err: 0.099995 ## p-value: <2e-16 ## 95% Conf Interval: (119.73, 120.12) ## ## Additive Treatment Effect: ## Parameter Estimate: 6.382 ## Estimated Std Err: 0.16961 ## p-value: <2e-16 ## 95% Conf Interval: (6.0496, 6.7144) You can also try with more candidate learners within super learner # Time consuming # tmle.res1 <- ltmle(data=result.data$observed, Anodes='A', Ynodes='Y', abar=list(1,0), # SL.library=c('SL.glm', 'SL.step.interaction', # 'SL.gam', 'SL.glmnet', 'SL.cforest', # "SL.ranger", "SL.xgboost", # "SL.rpart", "SL.rpartPrune", # 'SL.earth', 'SL.mean'), # estimate.time=F, # SL.cvControl=list(V=10), # gbounds=c(0.025, 0.975)) # summary(tmle.res1) References "],["guide.html", "Chapter 10 Reporting Guidelines 10.1 Discipline-specific Reviews 10.2 Suggested Guidelines 10.3 Additional topics", " Chapter 10 Reporting Guidelines While writing journal articles or reports, what are the components we should report? 10.1 Discipline-specific Reviews Propensity score matching most popular Guidelines available for some discipline-specific areas: Cardiovascular (Austin 2007), Infective endocarditis, Intensive care Critical care, anesthesiology, Sepsis, Psychology Cancer (Yao et al. 2017), Multiple sclerosis (Karim et al. 2020) 10.2 Suggested Guidelines Population Be specific about population of interest - ATT vs. ATE - exclusion criteria Intervention Be specific about exposure - no multiple version of treatment - no interference - comparator Covariates How variables are selected - Any important variables not measured? Proxy? - Large list of covariates? See King and Nielsen (2019) PS Model Model selection - interaction or polynomials - logistic vs. machine learning - Residual imbalance and refit PS model PS approach Why PS matching (or other approach) was selected? Sample size Reduction % of the matched data: major issue! Diagnostics Overlap vs. balance assessments - numeric and visual Software Report software, packages 10.3 Additional topics Some of the advanced topics not covered here. Sensitivity analysis - unmeasured confounding: proxy, or how much of an effect of unmeasured confounder necessary to nullify the results (e-value) - any positivity issue? Deleting extremes has consequences! - ad-hoc methods: truncation / trimming: bias-variance trade-off - different matching methods and allowing different thresholds: caliper, ratio, WR/WOR Subgroup analysis Refit within each group for matching - See Ali et al. (2019), Rassen et al. (2012), Radice et al. (2012), Kreif et al. (2012), Green and Stuart (2014), Girman et al. (2014), Eeren et al. (2015), Wang et al. (2018), Liu et al. (2020), Dong et al. (2020) for a more complete list Missing data Report clearly about missing data - how missing data handled References "],["final.html", "Chapter 11 Final Words 11.1 Common misconception 11.2 Benifits of PS 11.3 Limitations of PS 11.4 When PS may not be useful? 11.5 Software 11.6 Further Resources", " Chapter 11 Final Words 11.1 Common misconception PS results = ‘causal’; regression = ‘non-causal’. No. ‘Results from both methods should lead to the same conclusions.’ (D’Agostino Jr 1998) When the results deviate, important to investigate why! Establishing causality requires establishing temporarily and integration of subject area expertise. 11.2 Benifits of PS Intuitive: compare two similar groups 2-step process Encourages researchers to think about the treatment generation process Fit outcome model with only important variables. Allowing to think more about design stage (nice separation from outcome model building process). Fit rich PS model (with higher order terms); focusing on prediction; worry less about overparameterization. Reduce dimension, helpful when exposure frequent but outcome rare (event per variable). Smaller outcome model may be helpful in diagnostic checks. Diagnostics Diagnostics (balance checking) much easier compared to residual plot/influence Graphical comparison helps identify areas of non-overlap. 11.3 Limitations of PS Matching population vs. target population: often not the same. PS matching may give effect estimate of a subset, which may be difficult to identify in the actual population! May delete a lot of subjects from the study! SMD is very commonly used, but may not be enough to judge balance. Check other useful summaries. 11.4 When PS may not be useful? When outcome is common (5 times the available number of variables), then PS may not have any advantage over rregression modelling [ref, 17-5; March 20, 2022]. PS can do nothing about unmeasured confounding, neither can outcome regression. Consider instrumental variable (IV) approaches. Non-parametric (ML) approaches can be used to relax linearity assumption in estimating PS, but variance estimation becomes difficult. Double robust methods should be used when non-parametric (ML) approaches are used. See more on Lee, Lessler, and Stuart (2010), Pirracchio, Petersen, and Van Der Laan (2015), Alam, Moodie, and Stephens (2019), Naimi, Mishler, and Kennedy (2017) and Balzer and Westling (2021) 11.5 Software Useful R packages MatchIt cobalt Matching twang Also see Elizabeth Stuart’s Propensity Score Software Page for SAS, STATA, SPSS, Excel packages 11.6 Further Resources My workshop page My YouTube channel for related PS materials Teaching by WebApps: particularly this one. Understanding propensity score weighting More advanced methods, such as TMLE References "],["references.html", "References", " References "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]]