Send to

Choose Destination
Stat Med. 2017 May 30;36(12):1946-1963. doi: 10.1002/sim.7250. Epub 2017 Feb 16.

Comparing the performance of propensity score methods in healthcare database studies with rare outcomes.

Author information

Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, U.S.A.
Institute for Clinical Evaluative Sciences, Toronto, Canada.
Departments of Mental Health, Biostatistics, and Healthcare Policy and Management, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, U.S.A.


Nonrandomized studies of treatments from electronic healthcare databases are critical for producing the evidence necessary to making informed treatment decisions, but often rely on comparing rates of events observed in a small number of patients. In addition, studies constructed from electronic healthcare databases, for example, administrative claims data, often adjust for many, possibly hundreds, of potential confounders. Despite the importance of maximizing efficiency when there are many confounders and few observed outcome events, there has been relatively little research on the relative performance of different propensity score methods in this context. In this paper, we compare a wide variety of propensity-based estimators of the marginal relative risk. In contrast to prior research that has focused on specific statistical methods in isolation of other analytic choices, we instead consider a method to be defined by the complete multistep process from propensity score modeling to final treatment effect estimation. Propensity score model estimation methods considered include ordinary logistic regression, Bayesian logistic regression, lasso, and boosted regression trees. Methods for utilizing the propensity score include pair matching, full matching, decile strata, fine strata, regression adjustment using one or two nonlinear splines, inverse propensity weighting, and matching weights. We evaluate methods via a 'plasmode' simulation study, which creates simulated datasets on the basis of a real cohort study of two treatments constructed from administrative claims data. Our results suggest that regression adjustment and matching weights, regardless of the propensity score model estimation method, provide lower bias and mean squared error in the context of rare binary outcomes.


epidemiology; healthcare databases; propensity score; risk ratio; simulation

[Indexed for MEDLINE]

Supplemental Content

Loading ...
Support Center