Using a hierarchical model to estimate risk-adjusted mortality for hospitals not included in the reference sample

Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) collects detailed clinical data from participating hospitals using standardized data definitions, analyzes these data, and provides participating hospitals with reports that permit risk-adjusted comparisons with a surgical quality standard. Since its inception, the ACS NSQIP has worked to refine surgical outcomes measurements and enhance statistical methods to improve the reliability and validity of this hospital profiling. From an original focus on controlling for between-hospital differences in patient risk factors with logistic regression, ACS NSQIP has added a variable to better adjust for the complexity and risk profile of surgical procedures (procedure mix adjustment) and stabilized estimates derived from small samples by using a hierarchical model with shrinkage adjustment. New models have been developed focusing on specific surgical procedures (eg, "Procedure Targeted" models), which provide opportunities to incorporate indication and other procedure-specific variables and outcomes to improve risk adjustment. In addition, comparative benchmark reports given to participating hospitals have been expanded considerably to allow more detailed evaluations of performance. Finally, procedures have been developed to estimate surgical risk for individual patients. This article describes the development of, and justification for, these new statistical methods and reporting strategies in ACS NSQIP.

Predicting risk-adjusted mortality for trauma patients: logistic versus multilevel logistic models

BACKGROUND

Theoretical advantages of random-intercept multilevel (ML) logistic regression (LR) modeling over standard LR include separating variability due to patient-level and hospital-level predictors, shrinkage of estimates for lower-volume hospitals toward the overall mean, and fewer hospitals falsely identified as outliers.

STUDY DESIGN

We used Nationwide Inpatient Sample data from 2002 to 2004 to construct LR models of hospital mortality after admission with a principal ICD-9 Clinical Modification injury diagnosis (ICD-9 Clinical Modification codes 800 to 904, 910 to 929, 940 to 957, and 959). After considering various predictors, we used patient-level indicator variables for age groups, gender, maximum Abbreviated Injury Scale (AIS) for the head region (AIS score 3, 4, or 5), maximum AIS for other body regions (AIS score 3, 4, or 5), and mechanisms (eg, fall, gunshot, motor vehicle). Using standard LR and MLLR, we compared predictions based on 2002, 2003, and 2004 data with actual mortality observed in the same hospitals in the 2004, 2005, and 2006 Nationwide Inpatient Samples, respectively.

RESULTS

Patient-level fixed effects were similar for the 2 methods in all years, with mortality associated most strongly with AIS = 5 head injury, other AIS = 5 injury, or higher age groups. ML models identified fewer hospitals as outliers. Differences between actual and predicted mortality were smaller with MLLR models compared with standard LR models.

CONCLUSIONS

Multilevel models might have advantages for the measurement and explanation of interhospital differences in trauma patient outcomes.