Comparison of methods for handling covariate missingness in propensity score estimation with a binary exposure

Propensity Score Weighting with Missing Data on Covariates and Clustered Data Structure

Propensity score (PS) analyses are increasingly popular in behavioral sciences. Two issues often add complexities to PS analyses, including missing data in observed covariates and clustered data structure. In previous research, methods for conducting PS analyses with considering either issue alone were examined. In practice, the two issues often co-occur; but the performance of methods for PS analyses in the presence of both issues has not been evaluated previously. In this study, we consider PS weighting analysis when data are clustered and observed covariates have missing values. A simulation study is conducted to evaluate the performance of different missing data handling methods (complete-case, single-level imputation, or multilevel imputation) combined with different multilevel PS weighting methods (fixed- or random-effects PS models, inverse-propensity-weighting or the clustered weighting, weighted single-level or multilevel outcome models). The results suggest that the bias in average treatment effect estimation can be reduced, by better accounting for clustering in both the missing data handling stage (such as with the multilevel imputation) and the PS analysis stage (such as with the fixed-effects PS model, clustered weighting, and weighted multilevel outcome model). A real-data example is provided for illustration.

Balance diagnostics in propensity score analysis following multiple imputation: A new method

The combination of propensity score analysis and multiple imputation has been prominent in epidemiological research in recent years. However, studies on the evaluation of balance in this combination are limited. In this paper, we propose a new method for assessing balance in propensity score analysis following multiple imputation. A simulation study was conducted to evaluate the performance of balance assessment methods (Leyrat's, Leite's, and new method). Simulated scenarios varied regarding the presence of missing data in the control or treatment and control group, and the imputation model with/without outcome. Leyrat's method was more biased in all the studied scenarios. Leite's method and the combine method yielded balanced results with lower mean absolute difference, regardless of whether the outcome was included in the imputation model or not. Leyrat's method had a higher false positive ratio and Leite's and combine method had higher specificity and accuracy, especially when the outcome was not included in the imputation model. According to simulation results, most of time, Leyrat's method and Leite's method contradict with each other on appraising the balance. This discrepancy can be solved using new combine method.

Predicting the Climate Impact of Healthcare Facilities Using Gradient Boosting Machines

Health care accounts for 9-10% of greenhouse gas (GHG) emissions in the United States. Strategies for monitoring these emissions at the hospital level are needed to decarbonize the sector. However, data collection to estimate emissions is challenging, especially for smaller hospitals. We explored the potential of gradient boosting machines (GBM) to impute missing data on resource consumption in the 2020 survey of a consortium of 283 hospitals participating in Practice Greenhealth. GBM imputed missing values for selected variables in order to predict electricity use and beef consumption (R2=0.82) and anesthetic gas desflurane use (R2=0.51), using administrative data readily available for most hospitals. After imputing missing consumption data, estimated GHG emissions associated with these three examples totaled over 3 million metric tons of CO2 equivalent emissions (MTCO2e). Specifically, electricity consumption had the largest total carbon footprint (2.4 MTCO2e), followed by beef (0.6 million MTCO2e) and desflurane consumption (0.03 million MTCO2e) across the 283 hospitals. The approach should be applicable to other sources of hospital GHGs in order to estimate total emissions of individual hospitals and to refine survey questions to help develop better intervention strategies.

Assessment of the effectiveness of weight-adjusted antibiotic administration, for reduced duration, in surgical prophylaxis of primary hip and knee arthroplasty

BACKGROUND

Prophylactic antibiotics have significantly led to a reduction in the risk of post-operative surgical site infections (SSI) in orthopaedic surgery. The aim of using antibiotics for this purpose is to achieve serum and tissue drug levels that exceed, for the duration of the operation, the minimum inhibitory concentration of the likely organisms that are encountered. Prophylactic antibiotics reduce the rate of SSIs in lower limb arthroplasty from between 4% and 8% to between 1% and 3%. Controversy, however, still surrounds the optimal frequency and dosing of antibiotic administration.

AIM

To evaluate the impact of introduction of a weight-adjusted antibiotic prophylaxis regime, combined with a reduction in the duration of administration of post-operative antibiotics on SSI incidence during the 2 years following primary elective total hip and knee arthroplasty.

METHODS

Following ethical approval, patients undergoing primary total hip arthroplasty (THA)/total knee arthroplasty (TKA) with the old regime (OR) of a preoperative dose [cefazolin 2 g intravenously (IV)], and two subsequent doses (2 h and 8 h), were compared to those after a change to a new regime (NR) of a weight-adjusted preoperative dose (cefazolin 2 g IV for patients < 120 kg; cefazolin 3g IV for patients > 120 kg) and a post-operative dose at 2 h. The primary outcome in both groups was SSI rates during the 2 years post-operatively.

RESULTS

A total of n = 1273 operations (THA n = 534, TKA n = 739) were performed in n = 1264 patients. There was no statistically significant difference in the rate of deep (OR 0.74% (5/675) vs NR 0.50% (3/598); fishers exact test P = 0.72), nor superficial SSIs (OR 2.07% (14/675) vs NR 1.50% (9/598); chi-squared test P = 0.44) at 2 years post-operatively. With propensity score weighting and an interrupted time series analysis, there was also no difference in SSI rates between both groups [RR 0.88 (95%CI 0.61 to 1.30) P = 0.46].

CONCLUSION

A weight-adjusted regime, with a reduction in number of post-operative doses had no adverse impact on SSI incidence in this population.

Handling Missing Data in Health Economics and Outcomes Research (HEOR): A Systematic Review and Practical Recommendations

BACKGROUND

Missing data in costs and/or health outcomes and in confounding variables can create bias in the inference of health economics and outcomes research studies, which in turn can lead to inappropriate policies. Most of the literature focuses on handling missing data in randomized controlled trials, which are not necessarily always the data used in health economics and outcomes research.

OBJECTIVES

We aimed to provide an overview on missing data issues and how to address incomplete data and report the findings of a systematic literature review of methods used to deal with missing data in health economics and outcomes research studies that focused on cost, utility, and patient-reported outcomes.

METHODS

A systematic search of papers published in English language until the end of the year 2020 was carried out in PubMed. Studies using statistical methods to handle missing data for analyses of cost, utility, or patient-reported outcome data were included, as were reviews and guidance papers on handling missing data for those outcomes. The data extraction was conducted with a focus on the context of the study, the type of missing data, and the methods used to tackle missing data.

RESULTS

From 1433 identified records, 40 papers were included. Thirteen studies were economic evaluations. Thirty studies used multiple imputation with 17 studies using multiple imputation by chained equation, while 15 studies used a complete-case analysis. Seventeen studies addressed missing cost data and 23 studies dealt with missing outcome data. Eleven studies reported a single method while 20 studies used multiple methods to address missing data.

CONCLUSIONS

Several health economics and outcomes research studies did not offer a justification of their approach of handling missing data and some used only a single method without a sensitivity analysis. This systematic literature review highlights the importance of considering the missingness mechanism and including sensitivity analyses when planning, analyzing, and reporting health economics and outcomes research studies.

Tutorial on causal mediation analysis with binary variables: An application to health psychology research

Mediation analysis has been widely applied to explain why and assess the extent to which an exposure or treatment has an impact on the outcome in health psychology studies. Identifying a mediator or assessing the impact of a mediator has been the focus of many scientific investigations. This tutorial aims to introduce causal mediation analysis with binary exposure, mediator, and outcome variables, with a focus on the resampling and weighting methods, under the potential outcomes framework for estimating natural direct and indirect effects. We emphasize the importance of the temporal order of the study variables and the elimination of confounding. We define the causal effects in a hypothesized causal mediation chain in the context of one exposure, one mediator, and one outcome variable, all of which are binary variables. Two commonly used and actively maintained R packages, mediation and medflex, were used to analyze a motivating example. R code examples for implementing these methods are provided. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Initiation of continuous renal replacement therapy versus intermittent hemodialysis in critically ill patients with severe acute kidney injury: a secondary analysis of STARRT-AKI trial

BACKGROUND

There is controversy regarding the optimal renal-replacement therapy (RRT) modality for critically ill patients with acute kidney injury (AKI).

METHODS

We conducted a secondary analysis of the STandard versus Accelerated Renal Replacement Therapy in Acute Kidney Injury (STARRT-AKI) trial to compare outcomes among patients who initiated RRT with either continuous renal replacement therapy (CRRT) or intermittent hemodialysis (IHD). We generated a propensity score for the likelihood of receiving CRRT and used inverse probability of treatment with overlap-weighting to address baseline inter-group differences. The primary outcome was a composite of death or RRT dependence at 90-days after randomization.

RESULTS

We identified 1590 trial participants who initially received CRRT and 606 who initially received IHD. The composite outcome of death or RRT dependence at 90-days occurred in 823 (51.8%) patients who commenced CRRT and 329 (54.3%) patients who commenced IHD (unadjusted odds ratio (OR) 0.90; 95% confidence interval (CI) 0.75-1.09). After balancing baseline characteristics with overlap weighting, initial receipt of CRRT was associated with a lower risk of death or RRT dependence at 90-days compared with initial receipt of IHD (OR 0.81; 95% CI 0.66-0.99). This association was predominantly driven by a lower risk of RRT dependence at 90-days (OR 0.61; 95% CI 0.39-0.94).

CONCLUSIONS

In critically ill patients with severe AKI, initiation of CRRT, as compared to IHD, was associated with a significant reduction in the composite outcome of death or RRT dependence at 90-days.

How to use the Surveillance, Epidemiology, and End Results (SEER) data: research design and methodology

In the United States (US), the Surveillance, Epidemiology, and End Results (SEER) program is the only comprehensive source of population-based information that includes stage of cancer at the time of diagnosis and patient survival data. This program aims to provide a database about cancer incidence and survival for studies of surveillance and the development of analytical and methodological tools in the cancer field. Currently, the SEER program covers approximately half of the total cancer patients in the US. A growing number of clinical studies have applied the SEER database in various aspects. However, the intrinsic features of the SEER database, such as the huge data volume and complexity of data types, have hindered its application. In this review, we provided a systematic overview of the commonly used methodologies and study designs for retrospective epidemiological research in order to illustrate the application of the SEER database. Therefore, the goal of this review is to assist researchers in the selection of appropriate methods and study designs for enhancing the robustness and reliability of clinical studies by mining the SEER database.

The Role of Pre-Treatment Traumatic Stress Symptoms in Adolescent Substance Use Treatment Outcomes

Background: Prominent theories suggest that individuals with co-occurring traumatic stress symptoms (TSS) and substance use (SU) may be less responsive to SU treatment compared to those with SU only. However, empirical findings in adult samples are mixed, and there has been limited work among adolescents. This study assesses the association between TSS and SU treatment outcomes among trauma-exposed adolescents, using statistical methods to reduce potential confounding from important factors such as baseline SU severity. Method: 2,963 adolescents with lifetime history of victimization received evidence-based SU treatment in outpatient community settings. At baseline, 3- and 6-months, youth were assessed using the Global Appraisal of Individual Needs Traumatic Stress Scale and the Substance Frequency Scale. Propensity score weighting was used to mitigate potential confounding due to baseline differences in sociodemographic characteristics and SU across youth with varying levels of TSS. Results: Propensity score weighting successfully balanced baseline differences in sociodemographic factors and baseline SU across youth. Among all youth, mean SU was lower at both 3- and 6- month follow-up relative to baseline, indicating declining use. After adjusting for potential confounders, we observed no statistically significant relationship between TSS and SU at either 3- or 6-month follow-up. Conclusions: Based on this investigation, conducted among a large sample of trauma-exposed youth receiving evidence-based outpatient SU treatment, baseline TSS do not appear to be negatively associated with SU treatment outcomes. However, future research should examine whether youth with TSS achieve better outcomes through integrative treatment for both SU and TSS.

Intersectionality-Informed Sex/Gender-Sensitivity in Public Health Monitoring and Reporting (PHMR): A Case Study Assessing Stratification on an "Intersectional Gender-Score"

To date, PHMR has often relied on male/female stratification, but rarely considers the complex, intersecting social positions of men and women in describing the prevalence of health and disease. Stratification on an Intersectional Gender-Score (IG-Score), which is based on a variety of social covariables, would allow comparison of the prevalence of individuals who share the same complex intersectional profile (IG-Score). The cross-sectional case study was based on the German Socio-Economic Panel 2017 (n = 23,269 age 18+). After stratification, covariable-balance within the total sample and IG-Score-subgroups was assessed by standardized mean differences. Prevalence of self-rated health, mental distress, depression and hypertension was compared in men and women. In the IG-Score-subgroup with highest proportion of males and lowest probability of falling into the 'woman'-category, most individuals were in full-time employment. The IG-Score-subgroup with highest proportion of women and highest probability of falling into the 'woman'-category was characterized by part-time/occasional employment, housewife/-husband, and maternity/parental leave. Gender differences in prevalence of health indicators remained within the male-dominated IG-Score-subgroup, whereas the same prevalence of depression and self-rated health was observed for men and women constituting the female-dominated IG-Score-subgroup. These results might indicate that sex/gender differences of depression and self-rated health could be interpreted against the background of gender associated processes. In summary, the proposed procedure allows comparison of prevalence of health indicators conditional on men and women sharing the same complex intersectional profile.

Impact of Integrating Machine Learning in Comparative Effectiveness Research of Oral Anticoagulants in Patients with Atrial Fibrillation

We aimed to compare the ability to balance baseline covariates and explore the impact of residual confounding between conventional and machine learning approaches to derive propensity scores (PS). The Health Insurance Review and Assessment Service database (January 2012-September 2019) was used. Patients with atrial fibrillation (AF) who initiated oral anticoagulants during July 2015-September 2018 were included. The outcome of interest was stroke/systemic embolism. To estimate PS, we used a logistic regression model (i.e., a conventional approach) and a generalized boosted model (GBM) which is a machine learning approach. Both PS matching and inverse probability of treatment weighting were performed. To evaluate balance achievement, standardized differences, p-values, and boxplots were used. To explore residual confounding, E-values and negative control outcomes were used. In total, 129,434 patients were identified. Although all baseline covariates were well balanced, the distribution of continuous variables seemed more similar when GBM was applied. E-values ranged between 1.75 and 2.70 and were generally higher in GBM. In the negative control outcome analysis, slightly more nonsignificant hazard ratios were observed in GBM. We showed GBM provided a better ability to balance covariates and had a lower impact of residual confounding, compared with the conventional approach in the empirical example of comparative effectiveness analysis.

Association of early sedation level with patient outcomes in moderate-to-severe acute respiratory distress syndrome: Propensity-score matched analysis

Purpose

Studies of early depth of sedation in mixed critically ill populations have suggested benefit to light sedation; however, the relationship of early depth of sedation with outcomes in patients with acute respiratory distress syndrome (ARDS) is unknown.

Materials and methods

We performed a propensity-score matched analysis of early light sedation (Richmond Agitation Sedation Scale Score, RASS 0 to −1 or equivalent) versus deep sedation (RASS −2 or lower) in patients enrolled in the non-intervention group of The Reevaluation of Systemic Early Neuromuscular Blockade trial. Primary outcome was 90 day mortality. Secondary outcomes included days free of mechanical ventilation, days not in ICU, days not in hospital at day 28.

Results

137 of 486 participants (28.2%) received early light sedation. Vasopressor usage and Apache III scores significantly differed between groups. Prior to matching, 90-day mortality was higher in the early deep sedation (45.3%) compared to light sedation (34.2%) group. In the propensity score matched cohort, there was no difference in 90-day mortality (Odds Ratio (OR) 0.72, 95% CI 0.41, 1.27, p = 0.26) or secondary outcomes between the groups.

Conclusions

We did not find an association between early depth of sedation and clinical outcomes in this cohort of patients with moderate-to-severe ARDS.

Degree of Safety Against Falls Provided by 4 Different Prosthetic Knee Types in People With Transfemoral Amputation: A Retrospective Observational Study

OBJECTIVE

People with transfemoral amputation have balance and mobility problems and are at high risk of falling. An adequate prosthetic prescription is essential to maximize their functional levels and enhance their quality of life. This study aimed to evaluate the degree of safety against falls offered by different prosthetic knees.

METHODS

A retrospective study was conducted using data from a center for prosthetic fitting and rehabilitation. Eligible individuals were adults with unilateral transfemoral amputation or knee disarticulation. The prosthetic knee models were grouped into 4 categories: locked knees, articulating mechanical knees (AMKs), fluid-controlled knees (FK), and microprocessor-controlled knees (MPK). The outcome was the number of falls experienced during inpatient rehabilitation while wearing the prosthesis. Association analyses were performed with mixed-effect Poisson models. Propensity score weighting was used to adjust causal estimates for participant confounding factors.

RESULTS

Data on 1486 hospitalizations of 815 individuals were analyzed. Most hospitalizations (77.4%) were related to individuals with amputation due to trauma. After propensity score weighting, the knee category was significantly associated with falls. People with FK had the highest rate of falling (incidence rate = 2.81 falls per 1000 patient days, 95% CI = 1.96 to 4.02). FK significantly increased the risk of falling compared with MPK (incidence rate ratio [IRRFK-MPK] = 2.44, 95% CI = 1.20 to 4.96). No other comparison among knee categories was significant.

CONCLUSIONS

Fluid-controlled prosthetic knees expose inpatients with transfemoral amputation to higher incidence of falling than MPK during rehabilitation training.

IMPACT

These findings can guide clinicians in the selection of safe prostheses and reduction of falls in people with transfemoral amputation during inpatient rehabilitation.

Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings

Background

Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.

Methods

We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.

Results

Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.

Conclusions

The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01454-z.

Addressing missing values in routine health information system data: an evaluation of imputation methods using data from the Democratic Republic of the Congo during the COVID-19 pandemic

BACKGROUND

Poor data quality is limiting the use of data sourced from routine health information systems (RHIS), especially in low- and middle-income countries. An important component of this data quality issue comes from missing values, where health facilities, for a variety of reasons, fail to report to the central system.

METHODS

Using data from the health management information system in the Democratic Republic of the Congo and the advent of COVID-19 pandemic as an illustrative case study, we implemented seven commonly used imputation methods and evaluated their performance in terms of minimizing bias in imputed values and parameter estimates generated through subsequent analytical techniques, namely segmented regression, which is widely used in interrupted time series studies, and pre-post-comparisons through paired Wilcoxon rank-sum tests. We also examined the performance of these imputation methods under different missing mechanisms and tested their stability to changes in the data.

RESULTS

For regression analyses, there were no substantial differences found in the coefficient estimates generated from all methods except mean imputation and exclusion and interpolation when the data contained less than 20% missing values. However, as the missing proportion grew, k-NN started to produce biased estimates. Machine learning algorithms, i.e. missForest and k-NN, were also found to lack robustness to small changes in the data or consecutive missingness. On the other hand, multiple imputation methods generated the overall most unbiased estimates and were the most robust to all changes in data. They also produced smaller standard errors than single imputations. For pre-post-comparisons, all methods produced p values less than 0.01, regardless of the amount of missingness introduced, suggesting low sensitivity of Wilcoxon rank-sum tests to the imputation method used.

CONCLUSIONS

We recommend the use of multiple imputation in addressing missing values in RHIS datasets and appropriate handling of data structure to minimize imputation standard errors. In cases where necessary computing resources are unavailable for multiple imputation, one may consider seasonal decomposition as the next best method. Mean imputation and exclusion and interpolation, however, always produced biased and misleading results in the subsequent analyses, and thus, their use in the handling of missing values should be discouraged.

Relationships Between E-cigarette Use and Subsequent Cigarette Initiation Among Adolescents in the PATH Study: an Entropy Balancing Propensity Score Analysis

This study aimed to examine the relationship between electronic cigarette use and subsequent combustible cigarette use, controlling for confounding by using a propensity score method approach. Data from the first three annual waves of the Population Assessment of Tobacco and Health study were analyzed (n = 6309). Participants were tobacco-naïve at Wave 1; used e-cigarettes exclusively (n = 414), used combustible cigarettes exclusively (n = 46), or not used any tobacco products (n = 5849) at Wave 2. We conducted entropy balancing propensity score analysis to examine the association between exclusive e-cigarette or cigarette initiation and subsequent cigarette use at Wave 3, adjusting for non-response bias, sampling bias, and confounding. Among tobacco-naïve youth, exclusive e-cigarette use was associated with greater risk for subsequent combustible cigarette smoking initiation (OR = 3.42, 95% CI = (1.99, 5.93)) and past 30-day combustible cigarette use (OR = 2.88, 95% CI = (1.22, 6.86)) in the following year. However, the latter risk was comparatively lower than the risk if youth started with a combustible cigarette (OR = 25.79, 95% CI = (9.68, 68.72)). Results of sensitivity analyses indicated that estimated effects were robust to unmeasured confounding. Use of e-cigarettes in tobacco-naïve youth is associated with increased risk of subsequent past 30-day combustible cigarette use but the risk is an order of magnitude higher if they start with a combustible cigarette.

Propensity Score Analysis with Partially Observed Baseline Covariates: A Practical Comparison of Methods for Handling Missing Data

(1) Background: Propensity score methods gained popularity in non-interventional clinical studies. As it may often occur in observational datasets, some values in baseline covariates are missing for some patients. The present study aims to compare the performances of popular statistical methods to deal with missing data in propensity score analysis. (2) Methods: Methods that account for missing data during the estimation process and methods based on the imputation of missing values, such as multiple imputations, were considered. The methods were applied on the dataset of an ongoing prospective registry for the treatment of unprotected left main coronary artery disease. The performances were assessed in terms of the overall balance of baseline covariates. (3) Results: Methods that explicitly deal with missing data were superior to classical complete case analysis. The best balance was observed when propensity scores were estimated with a method that accounts for missing data using a stochastic approximation of the expectation-maximization algorithm. (4) Conclusions: If missing at random mechanism is plausible, methods that use missing data to estimate propensity score or impute them should be preferred. Sensitivity analyses are encouraged to evaluate the implications methods used to handle missing data and estimate propensity score.

Impact of the 1115 behavioral health Medicaid waiver on adult Medicaid beneficiaries in New York State

OBJECTIVE

To evaluate the impact of the Health and Recovery Plan (HARP), a capitated special needs Medicaid managed care product that fully integrates physical and behavioral health delivery systems in New York State.

DATA SOURCES

2013-2019 claims and encounters data on continuously enrolled individuals from the New York State Medicaid data system.

STUDY DESIGN

We used a difference-in-difference approach with inverse probability of exposure weights to compare service use outcomes in individuals enrolled in the HARP versus HARP eligible comparison group in two regions, New York City (NYC) pre- (2013-2015) versus post- (2016-2018) intervention periods, and rest of the state (ROS) pre- (2014-2016) versus post- (2017-2019) intervention periods.

DATA COLLECTION/EXTRACTION METHODS

Not applicable.

PRINCIPAL FINDINGS

HARPs were associated with a relative decrease in all-cause (RR = 0.78, 95% CI 0.68-0.90), behavioral health-related (RR = 0.76, 95% CI 0.60-0.96), and nonbehavioral-related (RR = 0.87, 95% CI 0.78-0.97) stays in the NYC region. In the ROS region, HARPs were associated with a relative decrease in all-cause (RR = 0.87, 95% CI 0.80-0.94) and behavioral health-related (RR = 0.80, 95% CI 0.70-0.91) stays. Regarding outpatient visits, the HARPs benefit package were associated with a relative increase in behavioral health (RR = 1.21, 95% CI 1.13-1.28) and nonbehavioral health (RR = 1.08, 95% CI 1.01-1.15) clinic visits in the NYC region. In the ROS region, the HARPs were associated with relative increases in behavioral health (RR = 1.47, 95% CI 1.32-1.64) and nonbehavioral health (RR = 1.17, 95% CI 1.11-1.25) clinic visits.

CONCLUSIONS

Compared to patients with similar clinical needs, HARPs were associated with a relative increase in services used and led to a better engagement in the HARPs group regardless of the overall decline in services used pre- to postperiod.

The effect of a constraint on the maximum number of controls matched to each treated subject on the performance of full matching on the propensity score when estimating risk differences

Many observational studies estimate causal effects using methods based on matching on the propensity score. Full matching on the propensity score is an effective and flexible method for utilizing all available data and for creating well-balanced treatment and control groups. An important component of the full matching algorithm is the decision about whether to impose a restriction on the maximum ratio of controls matched to each treated subject. Despite the possible effect of this restriction on subsequent inferences, this issue has not been examined. We used a series of Monte Carlo simulations to evaluate the effect of imposing a restriction on the maximum ratio of controls matched to each treated subject when estimating risk differences. We considered full matching both with and without a caliper restriction. When using full matching with a caliper restriction, the imposition of a subsequent constraint on the maximum ratio of the number of controls matched to each treated subject had no effect on the quality of inferences. However, when using full matching without a caliper restriction, the imposition of a constraint on the maximum ratio of the number of controls matched to each treated subject tended to result in an increase in bias in the estimated risk difference. However, this increase in bias tended to be accompanied by a corresponding decrease in the sampling variability of the estimated risk difference. We illustrate the consequences of these restrictions using observational data to estimate the effect of medication prescribing on survival following hospitalization for a heart attack.

A Narrative Review of Methods for Causal Inference and Associated Educational Resources

BACKGROUND AND OBJECTIVES

Root cause analysis involves evaluation of causal relationships between exposures (or interventions) and adverse outcomes, such as identification of direct (eg, medication orders missed) and root causes (eg, clinician's fatigue and workload) of adverse rare events. To assess causality requires either randomization or sophisticated methods applied to carefully designed observational studies. In most cases, randomized trials are not feasible in the context of root cause analysis. Using observational data for causal inference, however, presents many challenges in both the design and analysis stages. Methods for observational causal inference often fall outside the toolbox of even well-trained statisticians, thus necessitating workforce training.

METHODS

This article synthesizes the key concepts and statistical perspectives for causal inference, and describes available educational resources, with a focus on observational clinical data. The target audience for this review is clinical researchers with training in fundamental statistics or epidemiology, and statisticians collaborating with those researchers.

RESULTS

The available literature includes a number of textbooks and thousands of review articles. However, using this literature for independent study or clinical training programs is extremely challenging for numerous reasons. First, the published articles often assume an advanced technical background with different notations and terminology. Second, they may be written from any number of perspectives across statistics, epidemiology, computer science, or philosophy. Third, the methods are rapidly expanding and thus difficult to capture within traditional publications. Fourth, even the most fundamental aspects of causal inference (eg, framing the causal question as a target trial) often receive little or no coverage. This review presents an overview of (1) key concepts and frameworks for causal inference and (2) online documents that are publicly available for better assisting researchers to gain the necessary perspectives for functioning effectively within a multidisciplinary team.

CONCLUSION

A familiarity with causal inference methods can help risk managers empirically verify, from observed events, the true causes of adverse sentinel events.