The specification of the propensity score in multilevel observational studies

An Evaluation of Weighting Methods Based on Propensity Scores to Reduce Selection Bias in Multilevel Observational Studies

Observational studies of multilevel data to estimate treatment effects must consider both the nonrandom treatment assignment mechanism and the clustered structure of the data. We present an approach for implementation of four propensity score (PS) methods with multilevel data involving creation of weights and three types of weight scaling (normalized, cluster-normalized and effective), followed by estimation of multilevel models with the multilevel pseudo-maximum likelihood estimation method. Using a Monte Carlo simulation study, we found that the multilevel model provided unbiased estimates of the Average Treatment Effect on the Treated (ATT) and its standard error across manipulated conditions and combinations of PS model, PS method, and type of weight scaling. Estimates of between-cluster variances of the ATT were biased, but improved as cluster sizes increased. We provide a step-by-step demonstration of how to combine PS methods and multilevel modeling to estimate treatment effects using multilevel data from the Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K).

Evaluating possible confounding by prescriber in comparative effectiveness research

In nonrandomized studies of comparative effectiveness of medications, the prescriber may be the most important determinant of treatment assignment, yet the majority of analyses ignore the prescriber. Via Monte Carlo simulation, we evaluated the bias of 3 approaches that utilize the prescriber in analysis compared against the default approach that ignores the prescriber. Prescriber preference instrumental variable (IV) analyses were unbiased when IV criteria were met, which required no clustering of unmeasured patient characteristics within prescriber. In all other scenarios, IV analyses were highly biased, and stratification on the prescriber reduced confounding bias at the patient or prescriber levels. Including a prescriber random intercept in the propensity score model reversed the direction of confounding from measured patient factors and resulted in unpredictable changes in bias. Therefore, we recommend caution when using the IV approach, particularly when the instrument is weak. Stratification on the prescriber may be more robust; this approach warrants additional research.

Impact of infectious disease consultation on the clinical and economic outcomes of solid organ transplant recipients admitted for infectious complications

BACKGROUND

There has been a paucity of data on the healthcare resource utilization of infectious disease-related complications in solid organ transplant recipients. The aims of this study were to report the clinical and economic burden of infectious disease-related complications, along with the impact of infectious disease consultation.

METHODS

This cohort study evaluated patients requiring admission to a tertiary-care center during 2007, 2008, and 2011. Propensity score matching was used to estimate the effects of patient demographics, comorbidities, and transplant- and infection-related factors on 28-day hospital survival, length of stay (LOS), and medical costs.

RESULTS

Infectious disease-related complications occurred in 603 of 1414 (43%) admissions in 306 of 531 (58%) patients. Unadjusted 28-day mortality did not differ between those who received infectious disease consultations vs those who did not (2.9% vs 3.6%, P = .820), however, after propensity score matching, infectious disease consultation resulted in significantly greater 28-day survival estimates (hazard ratio = 0.33; log-rank P = .026), and reduced 30-day rehospitalization rates (16.9% vs 23.9%, P = .036). The median LOS and hospitalization costs were significantly increased for patients receiving an infectious disease consultation than in those managed by the attending team alone (7.0 vs 5.0 days, P = .002, and $9652 vs $6192, P = .003). However, the median LOS (5.5 vs 5.1 days, P = .31) and hospitalization costs ($8106 vs $6912, P = .63) did not differ significantly among those receiving an early infectious disease consultation (<48 hours) vs no consultation, respectively.

CONCLUSIONS

Infectious disease consultation in recipients of solid organ transplant is associated with increased LOS and hospitalization costs but decreased mortality and reduced rehospitalization rates. Early consultation with infectious disease specialists decreases healthcare resource utilization compared with delayed referrals.

Propensity score weighting with multilevel data

Propensity score methods are being increasingly used as a less parametric alternative to traditional regression to balance observed differences across groups in both descriptive and causal comparisons. Data collected in many disciplines often have analytically relevant multilevel or clustered structure. The propensity score, however, was developed and has been used primarily with unstructured data. We present and compare several propensity-score-weighted estimators for clustered data, including marginal, cluster-weighted, and doubly robust estimators. Using both analytical derivations and Monte Carlo simulations, we illustrate bias arising when the usual assumptions of propensity score analysis do not hold for multilevel data. We show that exploiting the multilevel structure, either parametrically or nonparametrically, in at least one stage of the propensity score analysis can greatly reduce these biases. We applied these methods to a study of racial disparities in breast cancer screening among beneficiaries of Medicare health plans.

Propensity scores used for analysis of cluster randomized trials with selection bias: a simulation study

Cluster randomized trials (CRTs) are often prone to selection bias despite randomization. Using a simulation study, we investigated the use of propensity score (PS) based methods in estimating treatment effects in CRTs with selection bias when the outcome is quantitative. Of four PS-based methods (adjustment on PS, inverse weighting, stratification, and optimal full matching method), three successfully corrected the bias, as did an approach using classical multivariable regression. However, they showed poorer statistical efficiency than classical methods, with higher standard error for the treatment effect, and type I error much smaller than the 5% nominal level.

Patient-level hospital costs and length of stay after conventional versus minimally invasive total hip replacement: a propensity-matched analysis

OBJECTIVES

A current trend in total hip replacement (THR) is the use of minimally invasive surgery. Little is known, however, about the impact of minimally invasive THR on resource use and length of stay. This study analyzed the effect of minimally invasive surgery on hospital costs and length of stay in German hospitals compared with conventional treatment in THR.

METHODS

We used patient-level administrative hospital data from three German hospitals participating in the national cost data study. We conducted a propensity score matching to account for baseline differences between minimally invasively and conventionally treated patients. Subsequently, we estimated the treatment effect on costs and length of stay by conducting group comparisons, via paired t tests and Wilcoxon signed-rank tests, and regression analyses.

RESULTS

The three hospitals provided data from 2886 THR patients. The propensity score matching led to 812 matched pairs. Length of stay was significantly higher for conventionally treated patients (11.49 days vs. 10.90 days; P < 0.05), but total costs did not differ significantly (€6018 vs. €5986; P = 0.67). We found a difference in the allocation of costs, with significantly higher implant costs for minimally invasively treated patients (€1514 vs. €1375; P < 0.001) in contrast to significantly higher staff and overhead costs for conventionally treated patients.

CONCLUSIONS

Minimally invasive surgery was compared with conventional THR and was found to be associated with a reduced length of stay. Total hospital costs, however, did not differ between the two treatment groups, because of higher implant costs for minimally invasively treated patients.

Effects of a universal parenting program for highly adherent parents: a propensity score matching approach

This paper examines the effectiveness of a group-based universal parent training program as a strategy to improve parenting practices and prevent child problem behavior. In a dissemination trial, 56 schools were first selected through a stratified sampling procedure, and then randomly allocated to treatment conditions. 819 parents of year 1 primary school children in 28 schools were offered Triple P. 856 families in 28 schools were allocated to the control condition. Teacher, primary caregiver and child self-report data were collected at baseline, post, and two follow-up assessments. Analyses were constrained to highly adherent parents who completed all four units of the parenting program. A propensity score matching approach was used to compare parents fully exposed to the intervention with parents in the control condition, who were matched on 54 baseline characteristics. Results suggest that the intervention had no consistent effects on either five dimensions of parenting practices or five dimensions of child problem behavior, assessed by three different informants. These findings diverge from findings reported by program developers and distributors. Potential explanations for the discrepancy and implications for future research are discussed.

The Use of Propensity Scores for Nonrandomized Designs With Clustered Data

In this article we propose several modeling choices to extend propensity score analysis to clustered data. We describe different possible model specifications for estimation of the propensity score: single-level model, fixed effects model, and two random effects models. We also consider both conditioning within clusters and conditioning across clusters. We examine the underlying assumptions of these modeling choices and the type of randomized experiment approximated by each approach. Using a simulation study, we compare the relative performance of these modeling and conditioning choices in reducing bias due to confounding variables at both the person and cluster levels. An applied example based on a study by Hughes, Chen, Thoemmes, and Kwok (2010) is provided in which the effect of retention in Grade 1 on passing an achievement test in Grade 3 is evaluated. We find that models that consider the clustered nature of the data both in estimation of the propensity score and conditioning on the propensity score performed best in our simulation study; however, other modeling choices also performed well. The applied example illustrates practical limitations of these models when cluster sizes are small.