Reliability adjustment: A necessity for trauma center ranking and benchmarking

Association Between Emergency Medical Service Agency Volume and Mortality in Trauma Patients

OBJECTIVE

The aim of this study was to evaluate the association of annual trauma patient volume on outcomes for emergency medical services (EMS) agencies.

BACKGROUND

Regionalization of trauma care saves lives. The underlying concept driving this is a volume-outcome relationship. EMS are the entry point to the trauma system, yet it is unknown if a volume-outcome relationship exists for EMS.

METHODS

A retrospective analysis of prospective cohort including 8 trauma centers and 20 EMS air medical and metropolitan ground transport agencies. Patients 18 to 90 years old with injury severity scores ≥9 transported from the scene were included. Patient and agency-level risk-adjusted regression determined the association between EMS agency trauma patient volume and early mortality.

RESULTS

A total of 33,511 were included with a median EMS agency volume of 374 patients annually (interquartile range: 90-580). Each 50-patient increase in EMS agency volume was associated with 5% decreased odds of 6-hour mortality (adjusted odds ratio=0.95; 95% CI: 0.92-0.99, P =0.03) and 3% decreased odds of 24-hour mortality (adjusted odds ratio=0.97; 95% CI: 0.95-0.99, P =0.04). Prespecified subgroup analysis showed EMS agency volume was associated with reduced odds of mortality for patients with prehospital shock, requiring prehospital airway placement, undergoing air medical transport, and those with traumatic brain injury. Agency-level analysis demonstrated that high-volume (>374 patients/year) EMS agencies had a significantly lower risk-standardized 6-hour mortality rate than low-volume (<374 patients/year) EMS agencies (1.9% vs 4.8%, P <0.01).

CONCLUSIONS

A higher volume of trauma patients transported at the EMS agency level is associated with improved early mortality. Further investigation of this volume-outcome relationship is necessary to leverage quality improvement, benchmarking, and educational initiatives.

Predictors of care discontinuity in geriatric trauma patients

BACKGROUND

Readmission to a non-index hospital, or care discontinuity, has been shown to have worse outcomes among surgical patients. Little is known about its effect on geriatric trauma patients. Our goal was to determine predictors of care discontinuity and to evaluate its effect on mortality in this geriatric population.

METHODS

This was a retrospective analysis of Medicare inpatient claims (2014-2015) of geriatric trauma patients. Care discontinuity was defined as readmission within 30 days to a non-index hospital. Demographic and clinical characteristics (including readmission diagnosis category) were collected. Multivariate logistic regression analysis was performed to identify predictors of care discontinuity and to assess its association with mortality.

RESULTS

We included 754,313 geriatric trauma patients. Mean age was 82.13 years (SD, 0.50 years), 68% were male and 91% were White. There were 21,615 (2.87%) readmitted within 30 days of discharge. Of these, 34% were readmitted to a non-index hospital. Overall 30-day mortality after readmission was 25%. In unadjusted analysis, readmission to index hospitals was more likely to be due to surgical infection, GI complaints, or cardiac/vascular complaints. After adjusted analysis, predictors of care discontinuity included readmission diagnoses, patient- and hospital-level factors. Care discontinuity was not associated with mortality (OR, 0.93; 95% confidence interval, 0.86-1.01).

CONCLUSION

More than a third of geriatric trauma patients are readmitted to a non-index hospital, which is driven by readmission diagnosis, travel time and hospital characteristics. However, unlike other surgical settings, this care discontinuity is not associated with increased mortality. Further work is needed to understand the reasons for this and to determine which standardized processes of care can benefit this population.

LEVEL OF EVIDENCE

Prognostic and Epidemiological; Level IV.

The Relationship Between Hospital Stroke Center Designation and TVT Reported Stroke: The Michigan TAVR Experience

BACKGROUND

The 30-day rate of stroke after transcatheter aortic valve replacement (TAVR) has been suggested as a hospital quality metric. Thirty-day stroke rates for nonsurgical, high, and moderate-risk TAVR trials were 3.4% to 6.1%, whereas those in the national Transcatheter Valve Therapy (TVT) Registry for the same patient population were much lower. Hospital comprehensive stroke center (CSC) is the highest designation for integrated acute stroke recognition, management, and care.

OBJECTIVES

Using Michigan TVT data, we assessed whether in-hospital post-TAVR stroke rates varied between CSC and non-CSC institutions.

METHODS

TVT data submitted from the 22 Michigan Transcatheter Aortic Valve Replacement Collaborative participating institutions between January 1, 2016, and June 30, 2019, were included (N = 6,231). Bayesian hierarchical regression models accounting for patient clinical characteristics and hospital clustering were fitted to assess the association between hospital CSC accreditation and in-hospital post-TAVR stroke. Adjusted ORs and 95% credible intervals were estimated. The University of Michigan Institutional Review Board has waived the need for the approval of studies based on the data collected by the Blue Cross Blue Shield of Michigan Cardiovascular Consortium registry.

RESULTS

There were 3,882 (62.3%) patients at 9 CSC sites and 2,349 (37.7%) patients at 13 non-CSC sites. CSC sites had significantly higher rates of in-hospital post-TAVR stroke (CSC: 2.65% vs non-CSC: 1.15%; P < 0.001). After adjustment, patients who underwent TAVR at a CSC hospital had a significantly higher risk of in-hospital stroke (adjusted OR: 2.21; 95% CI: 1.03-4.62). However, CSC designation was not significantly associated with other important post-TAVR clinical outcomes including 30-day mortality.

CONCLUSIONS

Reported Michigan Transcatheter Aortic Valve Replacement Collaborative TVT stroke rates were significantly higher at sites with Joint Hospital Commission stroke designation status; however, other reported important clinical outcomes did not differ significantly based on this designation. CSC designation is a possible factor in stroke rate detection differences between TAVR institutions and might be a factor in the observed differences in stroke rates between TAVR trials and those reported in TVT. In addition, these data suggest that comparison between hospitals based on post-TAVR stroke rates is potentially problematic.

Application of power analysis to determine the optimal reporting time frame for use in statewide trauma system quality reporting

BACKGROUND

Meaningful reporting of quality metrics relies on detecting a statistical difference when a true difference in performance exists. Larger cohorts and longer time frames can produce higher rates of statistical differences. However, older data are less relevant when attempting to enact change in the clinical setting. The selection of time frames must reflect a balance between being too small (type II errors) and too long (stale data). We explored the use of power analysis to optimize time frame selection for trauma quality reporting.

METHODS

Using data from 22 Level III trauma centers, we tested for differences in 4 outcomes within 4 cohorts of patients. With bootstrapping, we calculated the power for rejecting the null hypothesis that no difference exists amongst the centers for different time frames. From the entire sample for each site, we simulated randomly generated datasets. Each simulated dataset was tested for whether a difference was observed from the average. Power was calculated as the percentage of simulated datasets where a difference was observed. This process was repeated for each outcome.

RESULTS

The power calculations for the 4 cohorts revealed that the optimal time frame for Level III trauma centers to assess whether a single site's outcomes are different from the overall average was 2 years based on an 80% cutoff.

CONCLUSION

Power analysis with simulated datasets allows testing of different time frames to assess outcome differences. This type of analysis allows selection of an optimal time frame for benchmarking of Level III trauma center data.

Social determinants of health and patient-level mortality prediction after trauma

BACKGROUND

Social determinants of health (SDOH) impact patient outcomes in trauma. Census data are often used to account for SDOH; however, there is no consensus on which variables are most important. Social vulnerability indices offer the advantage of combining multiple constructs into a single variable. Our objective was to determine if incorporation of SDOH in patient-level risk-adjusted outcome modeling improved predictive performance.

METHODS

We evaluated two social vulnerability indices at the zip code level: Distressed Community Index (DCI) and National Risk Index (NRI). Individual variable combinations from Agency for Healthcare Research and Quality's SDOH data set were used for comparison. Patients were obtained from the Pennsylvania Trauma Outcomes Study 2000 to 2020. These measures were added to a validated base mortality prediction model with comparison of area under the curve and Bayesian information criterion. We performed center benchmarking using risk-standardized mortality ratios to evaluate change in rank and outlier status based on SDOH. Geospatial analysis identified geographic variation and autocorrelation.

RESULTS

There were 449,541 patients included. The DCI and NRI were associated with an increase in mortality (adjusted odds ratio, 1.02; 95% confidence interval, 1.01-1.03 per 10% percentile rank increase; p < 0.01, respectively). The DCI, NRI, and seven Agency for Healthcare Research and Quality variables also improved base model fit but discrimination was similar. Two thirds of centers changed mortality ranking when accounting for SDOH compared with the base model alone. Outlier status changed in 7% of centers, most representing an improvement from worse-than-expected to nonoutlier or nonoutlier to better-than-expected. There was significant geographic variation and autocorrelation of the DCI and NRI (DCI; Moran's I 0.62, p = 0.01; NRI; Moran's I 0.34, p = 0.01).

CONCLUSION

Social determinants of health are associated with an individual patient's risk of mortality after injury. Accounting for SDOH may be important in risk adjustment for trauma center benchmarking.

LEVEL OF EVIDENCE

Prognostic/Epidemiologic, level IV.

Extending Trauma Quality Improvement Beyond Trauma Centers: Hospital Variation in Outcomes Among Nontrauma Hospitals

OBJECTIVE

The American College of Surgeons (ACS) conducts a robust quality improvement program for ACS-verified trauma centers, yet many injured patients receive care at non-accredited facilities. This study tested for variation in outcomes across non-trauma hospitals and characterized hospitals associated with increased mortality.

SUMMARY BACKGROUND DATA

The study included state trauma registry data of 37,670 patients treated between January 1, 2013, and December 31, 2015. Clinical data were supplemented with data from the American Hospital Association and US Department of Agriculture, allowing comparisons among 100 nontrauma hospitals.

METHODS

Using Bayesian techniques, risk-adjusted and reliability-adjusted rates of mortality and interfacility transfer, as well as Emergency Departments length-of-stay (ED-LOS) among patients transferred from EDs were calculated for each hospital. Subgroup analyses were performed for patients ages >55 years and those with decreased Glasgow coma scores (GCS). Multiple imputation was used to address missing data.

RESULTS

Mortality varied 3-fold (0.9%-3.1%); interfacility transfer rates varied 46-fold (2.1%-95.6%); and mean ED-LOS varied 3-fold (81-231 minutes). Hospitals that were high and low statistical outliers were identified for each outcome, and subgroup analyses demonstrated comparable hospital variation. Metropolitan hospitals were associated increased mortality [odds ratio (OR) 1.7, P = 0.004], decreased likelihood of interfacility transfer (OR 0.7, P ≤ 0.001), and increased ED-LOS (coef. 0.1, P ≤ 0.001) when compared with nonmetropolitan hospitals and risk-adjusted.

CONCLUSIONS

Wide variation in trauma outcomes exists across nontrauma hospitals. Efforts to improve trauma quality should include engagement of nontrauma hospitals to reduce variation in outcomes of injured patients treated at those facilities.

Quantifying lives lost due to variability in emergency general surgery outcomes: Why we need a national emergency general surgery quality improvement program

BACKGROUND

Nearly 4 million Americans present to hospitals with conditions requiring emergency general surgery (EGS) annually, facing significant morbidity and mortality. Unlike elective surgery and trauma, there is no dedicated national quality improvement program to improve EGS outcomes. Our objective was to estimate the number of excess deaths that could potentially be averted through EGS quality improvement in the United States.

METHODS

Adults with the American Association for the Surgery of Trauma-defined EGS diagnoses were identified in the Nationwide Emergency Department Sample 2006 to 2014. Hierarchical logistic regression was performed to benchmark treating hospitals into reliability adjusted mortality quintiles. Weighted generalized linear modeling was used to calculate the relative risk of mortality at each hospital quintile, relative to best-performing quintile. We then calculated the number of excess deaths at each hospital quintile versus the best-performing quintile using techniques previously used to quantify potentially preventable trauma deaths.

RESULTS

Twenty-six million EGS patients were admitted, and 6.5 million (25%) underwent an operation. In-hospital mortality varied from 0.3% to 4.1% across the treating hospitals. Relative to the best-performing hospital quintile, an estimated 158,177 (153,509-162,736) excess EGS deaths occurred at lower-performing hospital quintiles. Overall, 47% of excess deaths occurred at the worst-performing hospitals, while 27% of all excess deaths occurred among the operative cohort.

CONCLUSION

Nearly 200,000 excess EGS deaths occur across the United States each decade. A national initiative to enable structures and processes of care associated with optimal EGS outcomes is urgently needed to achieve "Zero Preventable Deaths after Emergency General Surgery."

LEVEL OF EVIDENCE

Care management, level IV.

Changes in error patterns in unanticipated trauma deaths during 20 years: In pursuit of zero preventable deaths

BACKGROUND

A fundamental goal of continuous process improvement programs is to evaluate and improve the ratio of actual to expected mortality. To study this, we examined contributors to error-associated deaths during two consecutive periods from 1996 to 2004 (period 1) and 2005 to 2014 (period 2).

METHODS

All deaths at a level I trauma center with an anticipated probability of death less than 50% and/or identified through process improvement committees were examined. Demographics were assessed for trend only because period 1 data were only available in median and interquartile range. Each death was critically appraised to identify potential error, with subsequent classification of error type, phase, cause, and contributing cognitive processes, with comparison of outcomes made using χ test of independence.

RESULTS

During period 1, there were a total of 44,401 admissions with 2,594 deaths and 64 deaths (2.5%) associated with an error, compared with 60,881 admissions during period 2 with 2,659 deaths and 77 (2.9%) associated with an error. Deaths associated with an error occurred in younger and less severely injured patients in period 1 and were likely to occur during the early phase of care, primarily from failed resuscitation and hemorrhage control. In period 2, deaths occurred in older more severely injured patients and were likely to occur in the later phase of care primarily because of respiratory failure from aspiration.

CONCLUSION

Despite injured patients being older and more severely injured, error-associated deaths during the early phase of care that was associated with hemorrhage improved over time. Successful implementation of system improvements resolved issues in the early phase of care but shifted deaths to later events during the recovery phase including respiratory failure from aspiration. This study demonstrates that ongoing evaluation is essential for continuous process improvement and realignment of efforts, even in a mature trauma system.

LEVEL OF EVIDENCE

Therapeutic/Care Management, level IV.

Pull back the curtain: External data validation is an essential element of quality improvement benchmark reporting

BACKGROUND

Accurate and reliable data are pivotal to credible risk-adjusted modeling and hospital benchmarking. Evidence assessing the reliability and accuracy of data elements considered as variables in risk-adjustment modeling and measurement of outcomes is lacking. This deficiency holds the potential to compromise benchmarking integrity. We detail the findings of a longitudinal program to evaluate the impact of external data validation on data validity and reliability for variables utilized in benchmarking of trauma centers.

METHODS

A collaborative quality initiative-based study was conducted of 29 trauma centers from March 2010 through December 2018. Case selection criteria were applied to identify high-yield cases that were likely to challenge data abstractors. There were 127,238 total variables validated (i.e., reabstracted, compared, and reported to trauma centers). Study endpoints included data accuracy (agreement between registry data and contemporaneous documentation) and reliability (consistency of accuracy within and between hospitals). Data accuracy was assessed by mean error rate and type (under capture, inaccurate capture, or over capture). Cohen's kappa estimates were calculated to evaluate reliability.

RESULTS

There were 185,120 patients that met the collaborative inclusion criteria. There were 1,243 submissions reabstracted. The initial validation visit demonstrated the highest mean error rate at 6.2% ± 4.7%, and subsequent validation visits demonstrated a statistically significant decrease in error rate compared with the first visit (p < 0.05). The mean hospital error rate within the collaborative steadily improved over time (2010, 8.0%; 2018, 3.2%) compared with the first year (p < 0.05). Reliability of substantial or higher (kappa ≥0.61) was demonstrated in 90% of the 20 comorbid conditions considered in the benchmark risk-adjustment modeling, 39% of these variables exhibited a statistically significant (p < 0.05) interval decrease in error rate from the initial visit.

CONCLUSION

Implementation of an external data validation program is correlated with increased data accuracy and reliability. Improved data reliability both within and between trauma centers improved risk-adjustment model validity and quality improvement program feedback.

Assessment of Primary Care Clinician Concordance With Guidelines for Use of Magnetic Resonance Imaging in Patients With Nonspecific Low Back Pain in the Veterans Affairs Health System

IMPORTANCE

Magnetic responance imaging (MRI) of the lumbar spine that is not concordant with treatment guidelines for low back pain represents an unnecessary cost for US health plans and may be associated with adverse effects. Use of MRI in the US Department of Veterans Affairs (VA) primary care clinics remains unknown.

OBJECTIVE

To assess the use of MRI scans during the first 6 weeks (early MRI scans) of episodes of nonspecific low back pain in VA primary care sites and to determine if historical concordance can identify clinicians and sites that are the least concordant with guidelines.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective cohort study of electronic health records from 944 VA primary care sites from the 3 years ending in 2016. Data were analyzed between January 2017 and August 2019. Participants were patients with new episodes of nonspecific low back pain and the primary care clinicians responsible for their care.

EXPOSURES

MRI scans.

MAIN OUTCOMES AND MEASURES

The proportion of early MRI scans at VA primary care clinics was assessed. Clinician concordance with published guidelines over 2 years was used to select clinicians expected to have low concordance in a third year.

RESULTS

A total of 1 285 405 new episodes of nonspecific low back pain from 920 547 patients (mean [SD] age, 56.7 [15.8] years; 93.6% men) were attributed to 9098 clinicians (mean [SD] age, 52.1 [10.1] years; 55.7% women). An early MRI scan of the lumbar spine was performed in 31 132 of the episodes (2.42%; 95% CI, 2.40%-2.45%). Historical concordance was better than a random draw in selecting the 10% of clinicians who were subsequently the least concordant with published guidelines. For primary care clinicians, the area under the receiver operating characteristic curve was 0.683 (95% CI, 0.658-0.701). For primary care sites, the area was under this curve was 0.8035 (95% CI, 0.754-0.855). The 10% of clinicians with the least historical concordance were responsible for just 19.2% of the early MRI scans performed in the follow-up year.

CONCLUSIONS AND RELEVANCE

VA primary care clinics had low rates of use of early MRI scans. A history of low concordance with imaging guidelines was associated with subsequent low concordance but with limited potential to select clinicians most in need of interventions to implement guidelines.

Hospital differences in mortality rates after hip fracture surgery in Denmark

Background

Thirty-day mortality after hip fracture is widely used when ranking hospital performance, but the reliability of such hospital ranking is seldom calculated. We aimed to quantify the variation in 30-day mortality across hospitals and to determine the hospital general contextual effect for understanding patient differences in 30-day mortality risk.

Methods

Patients aged ≥65 years with an incident hip fracture registered in the Danish Multidisciplinary Fracture Registry between 2007 and 2016 were identified (n=60,004). We estimated unadjusted and patient-mix adjusted risk of 30-day mortality in 32 hospitals. We performed a multilevel analysis of individual heterogeneity and discriminatory accuracy with patients nested within hospitals. We expressed the hospital general contextual effect by the median odds ratio (MOR), the area under the receiver operating characteristics curve and the variance partition coefficient (VPC).

Results

The overall 30-day mortality rate was 10%. Patient characteristics including high sociodemographic risk score, underweight, comorbidity, a subtrochanteric fracture, and living at a nursing home were strong predictors of 30-day mortality (area under the curve=0.728). The adjusted differences between hospital averages in 30-day mortality varied from 5% to 9% across the 32 hospitals, which correspond to a MOR of 1.18 (95% CI: 1.12-1.25). However, the hospital general context effect was low, as the VPC was below 1% and adding the hospital level to a single-level model with adjustment for patient-mix increased the area under the receiver operating characteristics curve by only 0.004 units.

Conclusions

Only minor hospital differences were found in 30-day mortality after hip fracture. Mortality after hip fracture needs to be lowered in Denmark but possible interventions should be patient oriented and universal rather than focused on specific hospitals.

Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?

BACKGROUND

Report cards on the health care system increasingly report provider-specific performance on indicators that measure the quality of health care delivered. A natural reaction to the publishing of hospital-specific performance on a given indicator is to create 'league tables' that rank hospitals according to their performance. However, many indicators have been shown to have low to moderate rankability, meaning that they cannot be used to accurately rank hospitals. Our objective was to define conditions for improving the ability to rank hospitals by combining several binary indicators with low to moderate rankability.

METHODS

Monte Carlo simulations to examine the rankability of composite ordinal indicators created by pooling three binary indicators with low to moderate rankability. We considered scenarios in which the prevalences of the three binary indicators were 0.05, 0.10, and 0.25 and the within-hospital correlation between these indicators varied between - 0.25 and 0.90.

RESULTS

Creation of an ordinal indicator with high rankability was possible when the three component binary indicators were strongly correlated with one another (the within-hospital correlation in indicators was at least 0.5). When the binary indicators were independent or weakly correlated with one another (the within-hospital correlation in indicators was less than 0.5), the rankability of the composite ordinal indicator was often less than at least one of its binary components. The rankability of the composite indicator was most affected by the rankability of the most prevalent indicator and the magnitude of the within-hospital correlation between the indicators.

CONCLUSIONS

Pooling highly-correlated binary indicators can result in a composite ordinal indicator with high rankability. Otherwise, the composite ordinal indicator may have lower rankability than some of its constituent components. It is recommended that binary indicators be combined to increase rankability only if they represent the same concept of quality of care.

[Evaluation of surgical practice in the treatment of lung cancer in France from the PMSI national database]

BACKGROUND

In recent years, improving the quality of care has been a concern for health professionals in France, through the certification of institutions, accreditation and continuous professional development. Evaluation of these different measures has rarely been carried out. The objective of the study was to evaluate the quality of surgical management of lung cancer in different regions using hospital mortality as an indicator.

METHOD

From the national database of the Program of Medical Information Systems (PMSI), data on all patients who had undergone surgery for lung cancer were extracted as well as the characteristics of the centers. The main outcome criterion was hospital mortality. Logistic models allowed an estimation of the risk standardized mortality rate for each center.

RESULTS

From January 1, 2015 to December 31, 2015, 10,675 patients underwent surgery for lung cancer in 158 French centers. The hospital mortality rate was 3.43% (n=366). Thirty-nine facilities (25%) performed fewer than 15 pulmonary resections. The minimum activity volume was a single pulmonary resection during the year and the maximum was 300 interventions with a coefficient of variation estimated at 147%. Hospital mortality ranged from 0 % to 50% depending on the entries with a coefficient of variation of 112%. For some regions, it is possible to count up to 5 centers per million inhabitants (Languedoc-Roussillon) or 4 centers per million inhabitants (Limousin, Pays-de-Loire). The majority of regions had 3 centers per million inhabitants. Eleven regions have no centers with a standardized mortality rate below 3%. Five regions (Languedoc-Roussillon, Pays-de-Loire, Aquitaine, Brittany and Provence Alpes Côte d'Azur) have at least two centers with a risk standardized rate of mortality above 4%. Among the academic centers, 20% have a risk standardized mortality rate of less than 3%. Among the centers with a risk standardized rate of mortality<3%, 20% performed more than 39 pulmonary resections, 7% between 39 and 15 procedures and 0% for centers with<15 interventions.

CONCLUSION

This work confirms that hospital volume is one of the components of quality of care. The number of centers should be adapted to the actual needs of the population in order to enable patients to access effective services.

The association between outcome-based quality indicators for intensive care units

Purpose

To assess and improve the effectiveness of ICU care, in-hospital mortality rates are often used as principal quality indicator for benchmarking purposes. Two other often used, easily quantifiable, quality indicators to assess the efficiency of ICU care are based on readmission to the ICU and ICU length of stay. Our aim was to examine whether there is an association between case-mix adjusted outcome-based quality indicators in the general ICU population as well as within specific subgroups.

Materials and methods

We included patients admitted in 2015 of all Dutch ICUs. We derived the standardized in-hospital mortality ratio (SMR); the standardized readmission ratio (SRR); and the standardized length of stay ratio (SLOSR). We expressed association through Pearson’s correlation coefficients.

Results

The SMR ranged from 0.6 to 1.5; the SRR ranged from 0.7 to 2.1; and the SLOSR ranged from 0.7 to 1.3. For the total ICU population we found no significant associations. We found a positive, non-significant, association between SMR and SLOSR for admissions with low-mortality risk, (r = 0.25; p = 0.024), and a negative association between these indicators for admissions with high-mortality risk (r = -0.49; p<0.001).

Conclusion

Overall, we found no association at ICU population level. Differential associations were found between performance on mortality and length of stay within different risk strata. We recommend users of quality information to take these three outcome indicators into account when benchmarking ICUs as they capture different aspects of ICU performance. Furthermore, we suggest to report quality indicators for patient subgroups.

Mortality following surgery for trauma in an Indian trauma cohort

Background

India accounts for 20 per cent of worldwide trauma mortality. Little is known about the quality of trauma surgery in an Indian setting. The aim of this study was to estimate the overall perioperative mortality rate, and to assess the association between type of acute surgical intervention and perioperative mortality among adult patients treated for trauma in an urban Indian setting.

Methods

Data were obtained from injured adult patients enrolled in four urban Indian hospitals during 2013–2015. Those who had surgery within 24 h of arrival at hospital were included in the analysis. Patients with missing data were excluded. The perioperative mortality rate was measured at 48 h and 30 days after arrival at hospital. Generalized linear mixed models were used for risk adjustment of procedure-specific mortality.

Results

Among 2986 patients who underwent trauma surgery, the overall 48-h mortality rate was 6·0 per cent, and the 30-day mortality rate was 23·1 per cent. The highest adjusted odds ratios (ORs) for 48-h mortality were found for patients who underwent surgery on the peripheral vasculature (OR 4·71, 95 per cent c.i. 1·18 to 16·59; P = 0·030) and the digestive system and spleen (OR 3·77, 1·33 to 9·01; P = 0·010) compared with those who had nervous system surgery.

Conclusion

In this study of surgery in an Indian trauma cohort, there was an excess of late perioperative deaths. Mortality differed significantly according to the type of surgery being undertaken.

Impact of Volume Change Over Time on Trauma Mortality in the United States

OBJECTIVE

To evaluate the association of trauma center volume change over time with mortality.

BACKGROUND

Regionalization of trauma systems assumes a volume-outcome relationship for severe injury. Whereas this has been shown for cross-sectional volume, it is unclear whether volume changes over time translate into predictable outcome changes.

METHODS

Retrospective cohort study of severely injured (injury severity score >15) patients from the National Trauma Databank 2000 to 2012. A center-level standardized mortality ratio (SMR) was constructed (ratio of observed to expected deaths). Expected mortality was obtained from multilevel logistic regression model, adjusting for demographics, mechanism, vital signs, and injury severity. Center-level percent volume change was assessed across early (2000-2006) and late (2007-2012) periods. Longitudinal panel modeling evaluated association between annual SMR change and volume change over preceding years.

RESULTS

There were 839,809 patients included from 287 centers. Each 1% increase in volume was associated with 73% increased odds of improving SMR over time [odds ratio (OR) 1.73; 95% confidence interval (CI) 1.03-2.91; P = 0.03]. Each 1% decrease in volume was associated with 2-fold increase in odds of worsening SMR over time (OR 2.14; 95% CI 1.07-4.26, P = 0.03). Significant improvement in the SMR emerged after 3 or more preceding years of increasing volume (SMR change -0.008; 95% CI -0.015, -0.002; P = 0.01). This benefit occurred only in centers that were level I or II verified.

CONCLUSIONS

Increasing volume was associated with improving outcomes, whereas decreasing volume was associated with worsening outcomes. High-level trauma center infrastructure seems to facilitate the volume-outcome relationship. The trauma center designation process should consider volume changes in the overall system.

Performance Measurement to Demonstrate Value

Anesthesiologists are obligated to demonstrate the value of the care they provide. The Centers for Medicare and Medicaid Services has multiple performance-based payment programs to drive high-value care and motivate integrated care for surgical patients and hospitalized patients. These programs rely on diverse arrays of performance measures and complex reporting rules. Among all specialties, anesthesiology has tremendous potential to effect wide-ranging change on diverse measures. Performance measures deserve scrutiny by anesthesiologists as tools to improve care, the means by which payment is determined, and as a means to demonstrate the value of care to surgeons, hospitals, and patients.

Development and Validation of the Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support

OBJECTIVE

To develop and validate the Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support, which estimates the risk of in-hospital death for neonates prior to receiving respiratory extracorporeal membrane oxygenation (ECMO) support.

STUDY DESIGN

We used an international ECMO registry (2008-2013); neonates receiving ECMO for respiratory support were included. We divided the registry into a derivation sample and internal validation sample, by calendar date. We chose candidate variables a priori based on published evidence of association with mortality; variables independently associated with mortality in logistic regression were included in this parsimonious model of risk adjustment. We evaluated model discrimination with the area under the receiver operating characteristic curve (AUC), and we evaluated calibration with the Hosmer-Lemeshow goodness-of-fit test.

RESULTS

During 2008-2013, 4592 neonates received ECMO respiratory support with mortality of 31%. The development dataset contained 3139 patients treated in 2008-2011. The Neo-RESCUERS measure had an AUC of 0.78 (95% CI 0.76-0.79). The validation cohort had an AUC = 0.77 (0.75-0.80). Patients in the lowest risk decile had an observed mortality of 7.0% and a predicted mortality of 4.4%, and those in the highest risk decile had an observed mortality of 65.6% and a predicted mortality of 67.5%.

CONCLUSIONS

Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support offers severity-of-illness adjustment for neonatal patients with respiratory failure receiving ECMO. This score may be used to adjust patient survival to assess hospital-level performance in ECMO-based care.

Does Hospital Experience Rather than Volume Improve Outcomes in Geriatric Trauma Patients?

BACKGROUND

Although high absolute hospital geriatric trauma volume (GTV) is associated with improved outcomes among geriatric trauma patients, the actual geriatric trauma proportion (GTP) might be a better predictor of outcomes.

STUDY DESIGN

Adult trauma admissions were identified in the California State Inpatient Database, 2007 to 2011. Hospital characteristics were extracted from the American Hospital Association database. The annual trauma volume of patients 65 years and older was calculated for each hospital. The GTP was derived by dividing the GTV by the overall adult trauma volume and hospitals were categorized into tertiles of GTP. Outcomes were hospital mortality, failure to rescue (FTR), and 30-day readmission in geriatric trauma patients. Independent risk factors were assessed with clustered multivariate logistic regression models adjusted for patient and hospital characteristics.

RESULTS

There were 61,915 geriatric trauma patients included from 63 trauma centers. Hospital mortality, FTR, and 30-day readmission rates were 4.99%, 16.07%, and 12.03%, respectively. The adjusted odds ratios and 95% CIs for in-hospital mortality and FTR per 100 patient increase in GTV were 0.91 (95% CI, 0.83-1.00) and 1.01 (95% CI, 0.90-1.14), respectively. As compared with hospitals in the lowest tertile, adjusted odds of mortality and FTR in the highest tertile were 0.71 (95% CI, 0.54-0.94) and 0.67 (95% CI, 0.48-0.92), respectively. None of the hospital factors measured was significantly associated with readmission. The Wald test revealed that GTP played a larger role than GTV in predicting hospital mortality (p = 0.018 vs p = 0.048) and FTR (p = 0.015 vs p = 0.985).

CONCLUSIONS

Treatment at hospitals with higher GTP is associated with lower hospital mortality and FTR among geriatric patients. These findings suggest that creation of specialized services for geriatric trauma care can improve outcomes among geriatric trauma patients.

Variation in Patient-reported Outcomes Across Hospitals Following Surgery

BACKGROUND

Although there is growing interest in applying patient-reported outcomes (PROs) toward surgical quality, the extent to which PROs vary across hospitals following surgical procedures is unknown.

OBJECTIVES

We examined variation in PROs, specifically health-related quality of life (HRQOL), across hospitals performing bariatric surgery.

RESEARCH DESIGN

A retrospective cohort study.

SUBJECTS

The Michigan Bariatric Surgery Collaborative is a statewide consortium of 39 hospitals performing laparoscopic gastric bypass, gastric banding, or sleeve gastrectomy (n=11,420 patients between 2008 and 2012).

MEASURES

We examined generic and disease-specific HRQOL measured by the Health and Activities Limitations Index (HALex) and Bariatric Quality of Life index (BQL) preoperatively and at 1 year. We measured the variation in postoperative HRQOL across hospitals, and the effect of risk and reliability adjustment on hospital ranking.

RESULTS

In this cohort, HRQOL varied by 56% (HALex) and 37% (BQL) across hospitals. Patient factors accounted for 58% (HALex) to 71% (BQL) of the variation in HRQOL across hospitals. After risk and reliability adjustment, HRQOL varied by 18% (by HALex) and 14.5% (by BQL) across hospitals, and the proportion of patients who experienced a large improvement in HRQOL by HALex ranged from 33% to 69% and 67% to 92% by BQL. After adjusting for patient factors and reliability, these differences diminished to 55%-64% (HALex) and 79%-84% (BQL).

CONCLUSIONS

Patient factors explain a large proportion of hospital-level variation in PROs following bariatric surgery, underscoring the importance of risk adjustment. However, some variation in PROs across hospitals remains unexplained, suggesting PROs may represent a viable indicator of hospital performance.

Heterogeneity in Trauma Registry Data Quality: Implications for Regional and National Performance Improvement in Trauma

BACKGROUND

Led by the American College of Surgeons Trauma Quality Improvement Program, performance improvement efforts have expanded to regional and national levels. The American College of Surgeons Trauma Quality Improvement Program recommends 5 audit filters to identify records with erroneous data, and the Georgia Committee on Trauma instituted standardized audit filter analysis in all Level I and II trauma centers in the state.

STUDY DESIGN

Audit filter reports were performed from July 2013 to September 2014. Records were reviewed to determine whether there was erroneous data abstraction. Percent yield was defined as number of errors divided by number of charts captured.

RESULTS

Twelve centers submitted complete datasets. During 15 months, 21,115 patient records were subjected to analysis. Audit filter captured 2,901 (14%) records and review yielded 549 (2.5%) records with erroneous data. Audit filter 1 had the highest number of records identified and audit filter 3 had the highest percent yield. Individual center error rates ranged from 0.4% to 5.2%. When comparing quarters 1 and 2 with quarters 4 and 5, there were 7 of 12 centers with substantial decreases in error rates. The most common missed complications were pneumonia, urinary tract infection, and acute renal failure. The most common missed comorbidities were hypertension, diabetes, and substance abuse.

CONCLUSIONS

In Georgia, the prevalence of erroneous data in trauma registries varies among centers, leading to heterogeneity in data quality, and suggests that targeted educational opportunities exist at the institutional level. Standardized audit filter assessment improved data quality in the majority of participating centers.

Reliability of hospital cost profiles in inpatient surgery

BACKGROUND

With increased policy emphasis on shifting risk from payers to providers through mechanisms such as bundled payments and accountable care organizations, hospitals are increasingly in need of metrics to understand their costs relative to peers. However, it is unclear whether Medicare payments for surgery can reliably compare hospital costs.

METHODS

We used national Medicare data to assess patients undergoing colectomy, pancreatectomy, and open incisional hernia repair from 2009 to 2010 (n = 339,882 patients). We first calculated risk-adjusted hospital total episode payments for each procedure. We then used hierarchical modeling techniques to estimate the reliability of total episode payments for each procedure and explored the impact of hospital caseload on payment reliability. Finally, we quantified the number of hospitals meeting published reliability benchmarks.

RESULTS

Mean risk-adjusted total episode payments ranged from $13,262 (standard deviation [SD] $14,523) for incisional hernia repair to $25,055 (SD $22,549) for pancreatectomy. The reliability of hospital episode payments varied widely across procedures and depended on sample size. For example, mean episode payment reliability for colectomy (mean caseload, 157) was 0.80 (SD 0.18), whereas for pancreatectomy (mean caseload, 13) the mean reliability was 0.45 (SD 0.27). Many hospitals met published reliability benchmarks for each procedure. For example, 90% of hospitals met reliability benchmarks for colectomy, 40% for pancreatectomy, and 66% for incisional hernia repair.

CONCLUSION

Episode payments for inpatient surgery are a reliable measure of hospital costs for commonly performed procedures, but are less reliable for lower volume operations. These findings suggest that hospital cost profiles based on Medicare claims data may be used to benchmark efficiency, especially for more common procedures.

National estimates of predictors of outcomes for emergency general surgery

BACKGROUND

Identifying predictors of mortality and surgical complications has led to outcome improvements for a variety of surgical conditions. However, similar work has yet to be done for factors affecting outcomes of emergency general surgery (EGS). The objective of this study was to determine the predictors of in-hospital complications and mortality among EGS patients.

METHODS

The Nationwide Inpatient Sample (2003-2011) was queried for patients with conditions encompassing EGS as determined by the American Association for Surgery of Trauma, categorizing them into predefined EGS groups using DRG International Classification of Diseases-9th Rev.-Clinical Modification codes. Primary outcomes considered included incidence of a major complication (pneumonia, pulmonary emboli, urinary tract infections, myocardial infarctions, sepsis, or septic shock) and in-hospital mortality. Separate multivariate logistic regression analyses for complications and mortality were performed to identify risk factors of either outcome from the following domains: patient demographics (age, sex, insurance type, race, and income quartile), comorbidities, and hospital characteristics (location, teaching status, and bed size).

RESULTS

This study included 6,712,151 discharge records, weighted to represent 32,910,446 visits for EGS conditions. Mean age was 58.50 (19.74) years; slightly more than half (54.66%) were female. Uninsured patients were more likely to die (odds ratio,1.25; 95% confidence interval, 1.20-1.30), whereas patients in the highest income quartile had the least likelihood of mortality (odds ratio, 0.86; 95% confidence interval, 0.84-0.87). Old age was an independent predictor of mortality for all EGS subdiagnoses. The overall mortality rate was 1.76%; the overall complication rate was 10.03%. Of the patients who died, 62% experienced at least one major complication. Patients requiring resuscitation had the highest likelihood of mortality followed by patients with vascular disease and hepatic disease.

CONCLUSION

Death patterns of EGS patients were discerned using an administrative data set. Understanding patterns of mortality and complications derived from studies such as this could improve hospital benchmarking for EGS, akin to trauma surgery's previous success.

LEVEL OF EVIDENCE

Prognostic and epidemiologic study, level III.

Using same-hospital readmission rates to estimate all-hospital readmission rates

BACKGROUND

Since October of 2012, Medicare's Hospital Readmissions Reduction Program has fined 2,200 hospitals a total of $500 million. Although the program penalizes readmission to any hospital, many institutions can only track readmissions to their own hospitals. We sought to determine the extent to which same-hospital readmission rates can be used to estimate all-hospital readmission rates after major surgery.

STUDY DESIGN

We evaluated 3,940 hospitals treating 741,656 Medicare fee-for-service beneficiaries undergoing CABG, hip fracture repair, or colectomy between 2006 and 2008. We used hierarchical logistic regression to calculate risk- and reliability-adjusted rates of 30-day readmission to the same hospital and to any hospital. We next evaluated the correlation between same-hospital and all-hospital rates. To analyze the impact on hospital profiling, we compared rankings based on same-hospital rates with those based on all-hospital rates.

RESULTS

The mean risk- and reliability-adjusted all-hospital readmission rate was 13.2% (SD 1.5%) and mean same-hospital readmission rate was 8.4% (SD 1.1%). Depending on the operation, between 57% (colectomy) and 63% (CABG) of hospitals were reclassified when profiling was based on same-hospital readmission rates instead of on all-hospital readmission rates. This was particularly pronounced in the middle 3 quintiles, where 66% to 73% of hospitals were reclassified.

CONCLUSIONS

In evaluating hospital profiling under Medicare's Hospital Readmissions Reduction Program, same-hospital rates provide unstable estimates of all-hospital readmission rates. To better anticipate penalties, hospitals require novel approaches for accurately tracking the totality of their postoperative readmissions.

Benchmarking trauma centers on mortality alone does not reflect quality of care: implications for pay-for-performance

BACKGROUND

Trauma centers are currently benchmarked on mortality outcomes alone. However, pay-for-performance measures may financially penalize centers based on complications. Our objective was to determine whether the results would be similar to the current standard method of mortality-based benchmarking if trauma centers were profiled on complications.

METHODS

We analyzed data from the National Trauma Data Bank from 2007 to 2010. Patients 16 years or older with blunt or penetrating injuries and an Injury Severity Score (ISS) of 9 or higher were included. Risk-adjusted observed-to-expected (O/E) mortality ratios for each center were generated and used to rank each facility as high, average, or low performing. We similarly ranked facilities on O/E morbidity ratios defined as occurrence of any major complication. Concordance between hospital performance rankings was evaluated using a weighted κ statistic. Correlation between morbidity- and mortality-based O/E ratios was assessed using Pearson coefficients. Sensitivity analyses were performed to mitigate the competing risk of death for the morbidity analyses.

RESULTS

A total of 449,743 patients from 248 facilities were analyzed. The unadjusted morbidity and mortality rates were 10.0% and 6.9%, respectively. No correlation was found between morbidity- and mortality-based O/E ratios (r = -0.01). Only 40% of the centers had similar performance rankings for both mortality and morbidity. Of the 31 high performers for mortality, only 11 centers were also high performers for morbidity. A total of 78 centers were ranked as average, and 11 ranked as low performers on both outcomes. Comparison of hospital performance status using mortality and morbidity outcomes demonstrated poor concordance (weighted κ = 0.03, p = 0.22).

CONCLUSION

Mortality-based external benchmarking does not identify centers with high complication rates. This creates a dichotomy between current trauma center profiling standards and measures used for pay-for-performance. A benchmarking mechanism that reflects all measures of quality is needed.

LEVEL OF EVIDENCE

Prognostic/epidemiologic study, level III.

Reliability of hospital readmission rates in vascular surgery

OBJECTIVE

The Center for Medicare and Medicaid Services recently began assessing financial penalties to hospitals with high readmission rates for a narrow set of medical conditions. Because these penalties will be extended to surgical conditions in the near future, we sought to determine whether readmissions are a reliable predictor of hospital performance with vascular surgery.

METHODS

We examined 4 years of national Medicare claims data from 1576 hospitals on beneficiaries undergoing three common vascular procedures: open or endovascular abdominal aortic aneurysm repair (n = 81,520) or lower extremity arterial bypass (n = 57,190). First, we divided our population into two groups on the basis of operative date (2005-2006 and 2007-2008) and generated hospital risk- and reliability-adjusted readmission rates for each time period. We evaluated reliability through the use of the "test-retest" method; highly reliable measures will show little variation in rates over time. Specifically, we evaluated the year-to-year reliability of readmissions by calculating Spearman rank correlation and weighted κ tests for readmission rates between the two time periods.

RESULTS

The Spearman coefficient between 2005-2006 readmissions rankings and 2007-2008 readmissions rankings was 0.57 (P < .001) and weighted κ was 0.42 (P < .001), indicating a moderate correlation. However, only 32% of the variation in hospital readmission rates in 2007-2008 was explained by readmissions during the 2 prior years. There were major reclassifications of hospital rankings between years, with 63% of hospitals migrating among performance quintiles between 2005-2006 and 2007-2008.

CONCLUSIONS

Risk-adjusted readmission rates for vascular surgery vary substantially year to year; this implies that much of the observed variation in readmission rates is either random or caused by unmeasured factors and not caused by changes in hospital quality that may be captured by administrative data.