1. Comparing high-dimensional propensity score versus lasso variable selection for confounding adjustment in a novel simulation framework
Jessica Franklin
Instructor in Medicine
Division of Pharmacoepidemiology & Pharmacoeconomics
Brigham and Women’s Hospital and Harvard Medical School
QMC, Department of Quantitative Health Sciences
University of Massachusetts Medical School
April 15, 2014

2. Background
Administrative healthcare claims data are a popular data source for nonrandomized studies of interventions.
Because treatments are not randomized, addressing confounding is the primary methodological challenge.

3. Claims Data
Comprehensive claims databases contain information on patient insurance enrollment and demographics, as well as every healthcare encounter, including:
Diagnoses
Procedures
Hospitalizations
Medications dispensed
Dates of encounters provide a complete longitudinal record of patients’ healthcare interactions.

4. Follow-up for outcome events
New user design
Potential confounders are measured prior to initiation of exposure.
Active treatment comparator group reduces biases associated with non-user comparators.
End of:
Data
Enrollment
Exposure initiation
Covariates assessed
Follow-up for outcome events

5. Principles of variable selection
Brookhart et al. (2006) showed that the best PS model is the model that includes all predictors of outcome (regardless of whether they are associated with exposure).
Pearl (2010) and Myers et al. (2011) further noted that including instrumental varaibles (IVs) can increase bias from unmeasured confounding.
IVs are associated with exposure, but not associated with outcome except through exposure.

6. hd-PS variable selection
The high-dimensional propensity score (hd-PS) algorithm screens thousands of diagnoses, medications, and procedure codes and ranks variables according to likelihood of confounding.
Relies on the idea that a large number of “proxy” variables can reduce bias from unmeasured confounding.
Empirical evidence has shown a reduction in bias.

7. Shrinkage methods
Greenland (2008) suggested regularization methods as preferable to variable selection.
Shrinking coefficients allows for efficient estimation, even in models with many degrees of freedom.
Lasso regression provides both shrinkage and principled variable selection.
Shrinkage allows for direct modeling of the outcome even with many potential confounders
Some coefficients are shrunk all the way to 0.

8. Objective To compare the performance of
hd-PS variable selection
Ridge regression of the outcome on all potential confounders
Lasso regression of the outcome on all potential confounders
The goal is maximum reduction in confounding bias.

9. Comparing high-dimensional methods
How can we answer this question?
Empirical studies are useful when we “know” the true treatment effect, but even then we can’t determine the contributions of bias and variance to overall error.
Ordinary simulation techniques with completely synthetic data cannot capture the complex correlation structure among covariates in claims data.

10. Plasmode simulation We start with a real empirical cohort study:
49,653 patients
Exposed to either ns-NSAIDs or Cox-2 inhibitors (X)
Followed for gastrointestinal events (Y)
Pre-defined covariates include age, sex, race, and 16 diagnosis/medication/procedure variables (C1)
To get reasonable values for associations between covariates and outcome, we estimated a model with:
Y ~ X + all pre-defined covariates + interactions between age and binary covariates

11. Simulation setup True outcome generation model:
Estimated coefficient values from the observed outcome model
Except for the coefficient on exposure:
To create simulated datasets:
Sample with replacement rows from (X, C)
Calculate for each patient in the sample.
Simulate outcome
We created 500 datasets, each of size 30,000, outcome prevalence set to 5%, exposure prevalence set to 40%.

12. True causal diagram
Any variables associated with exposure remain associated with exposure. C
Any correlations among covariates and true confounders remain intact. C1 X Y
Associations with outcome are determined by chosen simulation model.
C1 = True confounders, a subset of C = all measured covariates.

13. Outcome generation Variable True OR Age 1.030928413 Black race
Male gender
Congestive heart failure
Coronary disease
Prior bleeding
Prior ulcer
Recent hospitalization
Recent nursing home admission
Warfarin
Gastrointestinal drugs

14. The mechanics of hd-PS
For each diagnosis, procedure, medication code, hd- PS creates 3 potential variables:
Code observed ≥ 1 time during baseline period
Code observed ≥ median number of times
Code observed ≥ 75th percentile number of times
There are 2 potential ranking methods:
Exposure-based: A simple RR association measure between exposure and each variable.
Bias-based: Bross’s bias formula that considers the association of each varaible with exposure and outcome

15. hd-PS Analyses PSs were constructed using:
The top 500 exposure-ranked variables + demographics
The top 500 bias-ranked variables + demographics
The top 30 exposure-ranked variables + demographics
The top 30 bias-ranked variables + demographics
Logistic regression on exposure + deciles of each PS

16. Shrinkage analyses
Regression of the outcome on all hdPS-screened variables (4800 – those that never occur) + exposure + demographics
Ridge regression
Lasso regression
We apply no shrinkage to the coefficient on exposure.
Calculate the crude estimate for comparison

17. Combination approaches
Using the variables selected by the lasso regression:
Include them in a PS analysis
Include them in an ordinary logistic regression outcome model
Using the 500 variables chosen by bias-based hd-PS:
Include them in a lasso outcome model
Include them in a ridge outcome model

18. Results – Variable selection
Lasso selected 103 variables on average.
66% were also selected by at least one hdPS algorithm
IQR: 62-70%
Age was selected in 100% of simulations.
Race was selected in 28%.

19. Results - Bias

20. Results - Bias
Crude confounding bias of 0.19.

21. Results - Bias
Ridge and lasso regression with all variables reduces bias by 41% and 63%, respectively.

22. Results - Bias
Ridge and lasso do better when they start with pre-screened variables. Bias is reduced by 70% and 83%, respectively.

23. Results - Bias
Ordinary regression and PS approaches performed better. Exposure-based hdPS with 500 variables completely eliminated bias.

24. Results - Bias
Bias-based hdPS varaible selection also performed well, with 93% and 91% bias reduction in the PS and ordinary regression models.

25. Results - Bias
PS and regular regression models performed well using lasso variable selection as well (95% and 96% bias reduction).

26. Results - Bias
When restricting variables to a very small set, bias-based hdPS was much preferred.

27. Conclusion
The variable selection method had relatively little importance.
The estimation method mattered much more.
Shrinkage of coefficient estimates led to insufficient bias control.
Focus on including a large number of potential confounders or confounder proxies.

28. Limitations There are many “instruments” in current simulation setup.
Variables associated with exposure that are not included in the outcome simulation model are essentially IVs, which is unrealistic.
There is no unmeasured confounding in these data.
Variable selection is an easier task when all important confounders are measured.

29. Future work Enrich the outcome model Vary the true treatment effect
Non-linear associations, more interactions, more true confounders
Vary the true treatment effect
Modify the coefficient on treatment in the outcome generation model.
Vary exposure prevalence
Can be accomplished by sampling within exposure group.
Vary outcome prevalence
Modify the intercept in the outcome generation model.
Unmeasured confounding
Set aside one or more true confounders and don’t allow methods to utilize these variables.
Other base datasets

30. Thanks! Co-authors: Contact: Wesley Eddings Jeremy A Rassen
Robert J Glynn
Sebastian Schneeweiss
Contact:
franklin/