Ranking hospitals on surgical mortality: the importance of reliability adjustment

Survival analysis for cardiothoracic surgeons: part 6-interpreting time-to-event data

Survival analysis is critical in clinical research, especially in cardiothoracic surgery, to assess outcomes and compare interventions. Certain key survival analysis tools, including Kaplan-Meier (KM) curves, log-rank tests, and Cox proportional hazards models, are helpful in providing insights into survival probabilities and risk factors. KM curves help analyze time-to-event data, estimating survival probabilities, while log-rank tests compare survival distributions across groups. Cox proportional hazards models identify covariates influencing survival and calculate hazard ratios, which quantify the relative risk associated with specific variables. Together, these methods enable clinicians to make evidence-based decisions, optimize treatments, and improve patient outcomes.

Earned outcomes correlate with reliability-adjusted surgical mortality after abdominal aortic aneurysm repair and predict future performance

OBJECTIVE

Cumulative, probability-based metrics are regularly used to measure quality in professional sports, but these methods have not been applied to health care delivery. These techniques have the potential to be particularly useful in describing surgical quality, where case volume is variable and outcomes tend to be dominated by statistical "noise." The established statistical technique used to adjust for differences in case volume is reliability-adjustment, which emphasizes statistical "signal" but has several limitations. We sought to validate a novel measure of surgical quality based on earned outcomes methods (deaths above average [DAA]) against reliability-adjusted mortality rates, using abdominal aortic aneurysm (AAA) repair outcomes to illustrate the measure's performance.

METHODS

Earned outcomes methods were used to calculate the outcome of interest for each patient: DAA. Hospital-level DAA was calculated for non-ruptured open AAA repair and endovascular aortic repair (EVAR) in the Vascular Quality Initiative database from 2016 to 2019. DAA for each center is the sum of observed - predicted risk of death for each patient; predicted risk of death was calculated using established multivariable logistic regression modeling. Correlations of DAA with reliability-adjusted mortality rates and procedure volume were determined. Because an accurate quality metric should correlate with future results, outcomes from 2016 to 2017 were used to categorize hospital quality based on: (1) risk-adjusted mortality; (2) risk- and reliability-adjusted mortality; and (3) DAA. The best performing quality metric was determined by comparing the ability of these categories to predict 2018 to 2019 risk-adjusted outcomes.

RESULTS

During the study period, 3734 patients underwent open repair (106 hospitals), and 20,680 patients underwent EVAR (183 hospitals). DAA was closely correlated with reliability-adjusted mortality rates for open repair (r = 0.94; P < .001) and EVAR (r = 0.99; P < .001). DAA also correlated with hospital case volume for open repair (r = -.54; P < .001), but not EVAR (r = 0.07; P = .3). In 2016 to 2017, most hospitals had 0% mortality (55% open repair, 57% EVAR), making it impossible to evaluate these hospitals using traditional risk-adjusted mortality rates alone. Further, zero mortality hospitals in 2016 to 2017 did not demonstrate improved outcomes in 2018 to 2019 for open repair (3.8% vs 4.6%; P = .5) or EVAR (0.8% vs 1.0%; P = .2) compared with all other hospitals. In contrast to traditional risk-adjustment, 2016 to 2017 DAA evenly divided centers into quality quartiles that predicted 2018 to 2019 performance with increased mortality rate associated with each decrement in quality quartile (Q1, 3.2%; Q2, 4.0%; Q3, 5.1%; Q4, 6.0%). There was a significantly higher risk of mortality at worst quartile open repair hospitals compared with best quartile hospitals (odds ratio, 2.01; 95% confidence interval, 1.07-3.76; P = .03). Using 2016 to 2019 DAA to define quality, highest quality quartile open repair hospitals had lower median DAA compared with lowest quality quartile hospitals (-1.18 DAA vs +1.32 DAA; P < .001), correlating with lower median reliability-adjusted mortality rates (3.6% vs 5.1%; P < .001).

CONCLUSIONS

Adjustment for differences in hospital volume is essential when measuring hospital-level outcomes. Earned outcomes accurately categorize hospital quality and correlate with reliability-adjustment but are easier to calculate and interpret. From 2016 to 2019, highest quality open AAA repair hospitals prevented >40 perioperative deaths compared with the average hospital, and >80 perioperative deaths compared with lowest quality hospitals.

Hospital variation of outcomes in status epilepticus

OBJECTIVE

Understanding factors driving variation in status epilepticus outcomes would be critical to improve care. We evaluated the degree to which patient and hospital characteristics explained hospital-to-hospital variability in intubation and postacute outcomes.

METHODS

This was a retrospective cohort study of Medicare beneficiaries admitted with status epilepticus between 2009 and 2019. Outcomes included intubation, discharge to a facility, and 30- and 90-day readmissions and mortality. Multilevel models calculated percent variation in each outcome due to hospital-to-hospital differences.

RESULTS

We included 29 150 beneficiaries. The median age was 68 years (interquartile range [IQR] = 57-78), and 18 084 (62%) were eligible for Medicare due to disability. The median (IQR) percentages of each outcome across hospitals were: 30-day mortality 25% (0%-38%), any 30-day readmission 14% (0%-25%), 30-day status epilepticus readmission 0% (0%-3%), 30-day facility stay 40% (25%-53%), and intubation 46% (20%-61%). However, after accounting for many hospitals with small sample size, hospital-to-hospital differences accounted for 2%-6% of variation in all unadjusted outcomes, and approximately 1%-5% (maximally 8% for 30-day readmission for status epilepticus) after adjusting for patient, hospitalization, and/or hospital characteristics. Although many characteristics significantly predicted outcomes, the largest effect size was cardiac arrest predicting death (odds ratio = 10.1, 95% confidence interval = 8.8-11.7), whereas hospital characteristics (e.g., staffing, accreditation, volume, setting, services) all had lesser effects.

SIGNIFICANCE

Hospital-to-hospital variation explained little variation in studied outcomes. Rather, certain patient characteristics (e.g., cardiac arrest) had greater effects. Interventions to improve outcomes after status epilepticus may be better focused on individual or prehospital factors, rather than at the inpatient systems level.

Association between surgical quality and long-term survival in lung cancer

OBJECTIVES

There are significant variations in both perioperative and long-term outcomes after lung cancer resection. While perioperative outcomes are often used as comparative measures of quality, they are unreliable, and their association with long-term outcomes remain unclear. In this context, we evaluated whether historical perioperative mortality after lung cancer resection is associated with 5-year survival.

PATIENTS AND METHODS

The National Cancer Database (NCDB) was queried to identify patients diagnosed with non-small cell lung cancer (NSCLC) in 2010-2016 who underwent surgical resection (n = 234200). Hospital-level reliability-adjusted 90-day mortality rate quartiles for 2010-2013 was used as the independent variable to analyze 5-year survival for patients diagnosed in 2014-2016 (n = 85396).

RESULTS

There were 85,396 patients in the 2014-2016 cohort across 1,086 hospitals. Overall observed 90-day mortality rate was 3.2% (SD 17.6%) with 2.6% (SD 16.0%) for the historically best performing quartile vs. 3.9% (SD 19.4%) for the worst performing quartile (p < 0.0001). Patients who underwent resection at hospitals with the best historical mortality rate had significantly better 5-year survival across all stages compared to those treated at hospitals in the worst performing quartile in multivariate Cox regression analysis (all stages - HR 1.21 [95% CI 1.15-1.26]; stage I - HR 1.19 [95% CI 1.12-1.25]; stage II - HR 1.20 [95% CI 1.09-1.32]; stage III - HR 1.36 [95% CI 1.20-1.54]) and Kaplan-Meier survival estimates (all stages - p < 0.0001, stage I - p < 0.0001; stage II - p = 0.0004; stage III - p < 0.0001).

CONCLUSION

With expanded lung cancer screening criteria and likely increase in early-stage detection, profiling performance is paramount to ensuring mortality benefits. We found that episodes surrounding surgical resection may be used to profile long-term outcomes that likely reflect quality across a broader context of care. Evaluating lung cancer care quality using perioperative outcomes may be useful in profiling provider performance and guiding value-based payment policies.

Stability of hospital quality indicators over time: A multi-year observational study of German hospital data

BACKGROUND

Retrospective hospital quality indicators can only be useful if they are trustworthy signals of current or future quality. Despite extensive longitudinal quality indicator data and many hospital quality public reporting initiatives, research on quality indicator stability over time is scarce and skepticism about their usefulness widespread.

OBJECTIVE

Based on aggregated, widely available hospital-level quality indicators, this paper sought to determine whether quality indicators are stable over time. Implications for health policy were drawn and the limited methodological foundation for stability assessments of hospital-level quality indicators enhanced.

METHODS

Two longitudinal datasets (self-reported and routine data), including all hospitals in Germany and covering the period from 2004 to 2017, were analysed. A logistic regression using Generalized Estimating Equations, a time-dependent, graphic quintile representation of risk-adjusted rates and Spearman's rank correlation coefficient were used.

RESULTS

For a total of eight German quality indicators significant stability over time was demonstrated. The probability of remaining in the best quality cluster in the future across all hospitals reached from 46.9% (CI: 42.4-51.6%) for hip replacement reoperations to 80.4% (CI: 76.4-83.8%) for decubitus. Furthermore, graphical descriptive analysis showed that the difference in adverse event rates for the 20% top performing compared to the 20% worst performing hospitals in the two following years is on average between 30% for stroke and AMI and 79% for decubitus. Stability over time has been shown to vary strongly between indicators and treatment areas.

CONCLUSION

Quality indicators were found to have sufficient stability over time for public reporting. Potentially, increasing case volumes per hospital, centralisation of medical services and minimum-quantity regulations may lead to more stable and reliable quality of care indicators. Finally, more robust policy interventions such as outcome-based payment, should only be applied to outcome indicators with a higher level of stability over time. This should be subject to future research.

Data-Driven Blood Transfusion Thresholds for Severely Injured Patients During Blood Shortages

INTRODUCTION

Crises like the COVID-19 pandemic create blood product shortages. Patients requiring transfusions are placed at risk and institutions may need to judiciously administer blood during massive blood transfusions protocols (MTP). The purpose of this study is to provide data-driven guidance for the modification of MTP when the blood supply is severely limited.

METHODS

This is a retrospective cohort study of 47 Level I and II trauma centers (TC) within a single healthcare system whose patients received MTP from 2017 to 2019. All TC used a unifying MTP protocol for balanced blood product transfusions. The primary outcome was mortality as a function of volume of blood transfused and age. Hemoglobin thresholds and measures of futility were also estimated. Risk-adjusted analyses were performed using multivariable and hierarchical regression to account for confounders and hospital variation.

RESULTS

Proposed MTP maximum volume thresholds for three age groupings are as follows: 60 units for ages 16-30 y, 48 units for ages 31-55 y, and 24 units for >55 y. The range of mortality under the transfusion threshold was 30%-36% but doubled to 67-77% when the threshold was exceeded. Hemoglobin concentration differences relative to survival were clinically nonsignificant. Prehospital measures of futility were prehospital cardiac arrest and nonreactive pupils. In hospital risk factors of futility were mid-line shift on brain CT and cardiopulmonary arrest.

CONCLUSIONS

Establishing MTP threshold practices under blood shortage conditions, such as the COVID pandemic, could sustain blood availability by following relative thresholds for MTP use according to age groups and key risk factors.

Health care delivery system contributions to management of newly diagnosed prostate cancer

BACKGROUND

Despite clinical guidelines advocating for use of conservative management in specific clinical scenarios for men with prostate cancer, there continues to be tremendous variation in its uptake. This variation may be amplified among men with competing health risks, for whom treatment decisions are not straightforward. The degree to which characteristics of the health care delivery system explain this variation remains unclear.

METHODS

Using national Medicare data, men with newly diagnosed prostate cancer between 2014 and 2019 were identified. Hierarchical logistic regression models were used to assess the association between use of treatment and health care delivery system determinants operating at the practice level, which included measures of financial incentives (i.e., radiation vault ownership), practice organization (i.e., single specialty vs. multispecialty groups), and the health care market (i.e., competition). Variance was partitioned to estimate the relative influence of patient and practice characteristics on the variation in use of treatment within strata of noncancer mortality risk groups.

RESULTS

Among 62,507 men with newly diagnosed prostate cancer, the largest variation in the use of treatment between practices was observed for men with high and very high-risk of noncancer mortality (range of practice-level rates of treatment for high: 57%-71% and very high: 41%-61%). Addition of health care delivery system determinants measured at the practice level explained 13% and 15% of the variation in use of treatment among men with low and intermediate risk of noncancer mortality in 10 years, respectively. Conversely, these characteristics explained a larger share of the variation in use of treatment among men with high and very high-risk of noncancer mortality (26% and 40%, respectively).

CONCLUSIONS

Variation among urology practices in use of treatment was highest for men with high and very high-risk noncancer mortality. Practice characteristics explained a large share of this variation.

Evaluations of statistical methods for outlier detection when benchmarking in clinical registries: a systematic review

OBJECTIVES

Benchmarking is common in clinical registries to support the improvement of health outcomes by identifying underperforming clinician or health service providers. Despite the rise in clinical registries and interest in publicly reporting benchmarking results, appropriate methods for benchmarking and outlier detection within clinical registries are not well established, and the current application of methods is inconsistent. The aim of this review was to determine the current statistical methods of outlier detection that have been evaluated in the context of clinical registry benchmarking.

DESIGN

A systematic search for studies evaluating the performance of methods to detect outliers when benchmarking in clinical registries was conducted in five databases: EMBASE, ProQuest, Scopus, Web of Science and Google Scholar. A modified healthcare modelling evaluation tool was used to assess quality; data extracted from each study were summarised and presented in a narrative synthesis.

RESULTS

Nineteen studies evaluating a variety of statistical methods in 20 clinical registries were included. The majority of studies conducted application studies comparing outliers without statistical performance assessment (79%), while only few studies used simulations to conduct more rigorous evaluations (21%). A common comparison was between random effects and fixed effects regression, which provided mixed results. Registry population coverage, provider case volume minimum and missing data handling were all poorly reported.

CONCLUSIONS

The optimal methods for detecting outliers when benchmarking clinical registry data remains unclear, and the use of different models may provide vastly different results. Further research is needed to address the unresolved methodological considerations and evaluate methods across a range of registry conditions.

PROSPERO REGISTRATION NUMBER

CRD42022296520.

"Failure to Rescue" following Colorectal Cancer Resection: Variation and Improvements in a National Study of Postoperative Mortality

OBJECTIVE

To examine variation in "failure to rescue" (FTR) as a driver of differences in mortality between centres and over time for patients undergoing colorectal cancer surgery.

BACKGROUND

Wide variation exists in postoperative mortality following colorectal cancer surgery. FTR has been identified as an important determinant of variation in postoperative outcomes. We hypothesized that differences in mortality both between hospitals and over time are driven by variation in FTR.

METHODS

A national population-based study of patients undergoing colorectal cancer resection from 2010 to 2019 in Aotearoa New Zealand was conducted. Rates of 90-day FTR, mortality, and complications were calculated overall, and for surgical and nonoperative complications. Twenty District Health Boards (DHBs) were ranked into quartiles using risk- and reliability-adjusted 90-day mortality rates. Variation between DHBs and trends over the 10-year period were examined.

RESULTS

Overall, 15,686 patients undergoing resection for colorectal adenocarcinoma were included. Increased postoperative mortality at high-mortality centers (OR 2.4, 95% CI 1.8-3.3) was driven by higher rates of FTR (OR 2.0, 95% CI 1.5-2.8), and postoperative complications (OR 1.4, 95% CI 1.3-1.6). These trends were consistent across operative and nonoperative complications. Over the 2010 to 2019 period, postoperative mortality halved (OR 0.5, 95% CI 0.4-0.6), associated with a greater improvement in FTR (OR 0.5, 95% CI 0.4-0.7) than complications (OR 0.8, 95% CI 0.8-0.9). Differences between centers and over time remained when only analyzing patients undergoing elective surgery.

CONCLUSION

Mortality following colorectal cancer resection has halved over the past decade, predominantly driven by improvements in "rescue" from complications. Differences in FTR also drive hospital-level variation in mortality, highlighting the central importance of "rescue" as a target for surgical quality improvement.

The Measurement Reliability of Complications and Patient Satisfaction After Common Surgical Procedures

OBJECTIVE

To evaluate the reliability of surgeon outcomes.

BACKGROUND

Surgeons' outcomes are now widely used in public reporting and value-based reimbursement, but the reliability of these measures continues to raise concerns.

METHODS

We performed a retrospective study of surgeons performing cholecystectomy, colectomy, and hernia repair on adult patients between January 1, 2017, and December 31, 2020. Outcomes were risk-adjusted rates of complications and high patient satisfaction. We estimated the reliability of each outcome, its relationship with case volume, and the number of surgeons who reached an acceptable level of reliability (≥0.70).

RESULTS

A total of 23,533 patients with a mean age of 56.8 (16.2) years and 10,191 (43.3%) females underwent operations by 333 surgeons. Risk-adjusted complication rate was 2.5% [95% confidence interval (CI): 2.2%-2.8%] and risk-adjusted high satisfaction rate was 79.9% (95% CI: 78.7%-81.0%). The reliability of the complication rate was 0.27 (95% CI: 0.25-0.29) and the reliability of the high satisfaction rate was 0.53 (95% CI: 0.50-0.55). Reliability increased with case volume; however, only 5 (1.5%) surgeons performed enough cases to reach acceptable reliability for their complication rate, while 86 (25.8%) surgeons reached acceptable reliability for their patient satisfaction rate. After adjustment for reliability, the range of complication rates decreased 29-fold from 0% to 14.3% to 2.4% to 2.9%, and the range of patient satisfaction decreased 2.6-fold from 25.3% to 100.0% to 64.9% to 92.4%.

CONCLUSIONS

Among surgeons performing common operations, complications and patient satisfaction had relatively low reliability. Although reliability increased with volume, most surgeons had insufficient case volume to achieve acceptable reliability of their outcomes. As such, these measures likely offer little to no meaningful information to inform decision-making.

Heterogeneity in Surgical Quality Improvement in Michigan

OBJECTIVE

The aim of this study was to evaluate changes in 30-day postoperative outcomes and individual hospital variation in outcomes from 2012 to 2019 in a collaborative quality improvement network.

SUMMARY BACKGROUND DATA

Collaborative quality improvement efforts have been shown to improve postoperative outcomes overall; however, heterogeneity in improvement between participating hospitals remains unclear. Understanding the distribution of individual hospital-level changes is necessary to inform resource allocation and policy design.

METHODS

We performed a retrospective cohort study of 51 hospitals in the Michigan Surgical Quality Collaborative (MSQC) from 2012 to 2019. Risk-and reliability-adjusted hospital rates of 30-day mortality, complications, serious complications, emergency department (ED) visits, readmissions, and reoperations were calculated for each year and compared between the last 2 years and the first 2 years of the study period.

RESULTS

There was a significant decrease in the rates of all 5 adverse outcomes across MSQC hospitals from 2012 to 2019. Of the 51 individual hospitals, 31 (61%) hospitals achieved a decrease in mortality (range -1.3 percentage points to +0.6 percentage points), 40 (78%) achieved a decrease in complications (range -8.5 percentage points to +2.9 percentage points), 26 (51%) achieved a decrease in serious complications (range -3.2 percentage points to +3.0 percentage points), 29 (57%) achieved a decrease in ED visits (range 5.0 percentage points to +2.2 percentage points), 46 (90%) achieved a decrease in readmissions (range -3.1 percentage points to +0.4 percentage points) and 39 (76%) achieved a decrease in reoperations (range 3.3 percentage points to +1.0 percentage points).

CONCLUSIONS

Despite overall improvement in surgical outcomes across hospitals participating in a quality improvement collaborative, there was substantial variation in improvement between hospitals, highlighting opportunities to better understand hospital-level barriers and facilitators to surgical quality improvement.

Hospital-level variation in mesh use for ventral and incisional hernia repair

BACKGROUND

Placement of prosthetic mesh during ventral and incisional hernia repair has been shown to reduce the incidence of postoperative hernia recurrence. Consequently, multiple consensus guidelines recommend the use of mesh for ventral hernias of any size. However, the extent to which real-world practice patterns reflect these recommendations is unclear.

METHODS

We performed a retrospective review of the Michigan Surgical Quality Collaborative Hernia Registry (MSQC-HR) to identify patients undergoing clean ventral or incisional hernia repair between January 1, 2020 and December 31, 2021. The primary outcome was mesh use. We used two-step hierarchical logistic regression modeling with empirical Bayes estimates to evaluate the association of hospital-level mesh use with patient, operative, and hernia characteristics.

RESULTS

A total of 5262 patients underwent ventral and incisional hernia repair at 65 hospitals with a mean age of 53.8 (14.5) years, 2292 (43.6%) females, and a mean hernia width of 3.2 (3.4) cm. Mean hospital volume was 81 (49) cases. Mesh was used in 4098 (77.9%) patients. At the patient level, hernia width and surgical approach were significantly associated with mesh use. Specifically, mesh use was 6.2% (95% CI 4.8-7.5%) more likely with each additional centimeter of hernia width and 28.0% (95% CI 26.1-29.8%) more likely for minimally invasive repair compared to open repair. At the hospital level, there was wide variation in mesh use, ranging from 38.0% (95% CI 31.5-44.9%) to 96.4% (95% CI 95.3-97.2%). Hospital-level mesh use was not associated with differences in hernia size (β =  - 0.003, P = 0.978), surgical approach (β =  - 1.109, P = 0.414), or any other patient factors.

CONCLUSIONS

Despite strong evidence supporting the use of mesh in ventral and incisional hernia repair, there is substantial variation in mesh use between hospitals that is not explained by differences in patient characteristics or operative approach. This suggests that opportunities exist to standardize surgical practice to better align with evidence supporting the use of mesh in the management of these hernias.

Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data

Background

Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.

Materials and methods

Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.

Results

Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [AUC = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; AUC = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (R ² = 0.242 and R ² = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: ρ = 0, p = 0.99; RSMR: ρ = 0.017, p = 0.56; reliability-adjusted RSMR: ρ = 0.067; p = 0.026).

Conclusion

Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.

Private Equity Acquisitions Of Ambulatory Surgical Centers Were Not Associated With Quality, Cost, Or Volume Changes

Ambulatory surgical centers (ASCs) are increasingly being acquired by private equity firms, yet the implications for patients remain understudied. In this study we employed a quasi-experimental difference-in-differences design within an event study framework to assess changes in outcomes associated with the acquisition of ASCs by private equity entities. Using a two-way fixed effects model, we assessed the baseline probability of an unplanned hospital visit, total costs, and total encounters three years preacquisition compared with three years postacquisition in ASCs acquired by private equity versus those acquired by non-private equity entities. We identified ninety-one ASCs acquired by private equity and fifty-seven ASCs acquired by non-private equity entities during the period 2011-14. There was no statistically significant observed change in the probability of an unplanned hospital visit, total costs, or total encounters after acquisition by private equity relative to acquisition by non-private equity entities. When we compared private equity-acquired ASCs with matched ASCs that were never acquired, we also found no statistically significant relative change in the probability of an unplanned hospital visit, total costs, or total encounters. Regulators should ensure that data on private equity acquisitions are transparent and that data are available to track the long-term quality and financial implications of these acquisitions.

Do all Patients Get the Same Care Across Hospitals?

There is extensive research demonstrating significant variation in the utilization of surgery and outcomes from surgery, including differences in mortality, complications, readmission, and failure to rescue. Literature suggests that these variations exist across as well as within small area geographies in the United States. There is also significant evidence of variation in access and outcomes from surgery that is attributable to race. Emerging research is demonstrating that there may be some variation attributable to a patient's social determinants of health and their lived averment. Those affected must work together to determine rate of utilization and how much variation is acceptable.

The win ratio: A novel approach to define and analyze postoperative composite outcomes to reflect patient and clinician priorities

BACKGROUND

The "win ratio" (WR) is a novel statistical technique that hierarchically weighs various postoperative outcomes (eg, mortality weighted more than complications) into a composite metric to define an overall benefit or "win." We sought to use the WR to assess the impact of social vulnerability on the likelihood of achieving a "win" after hepatopancreatic surgery.

METHODS

Individuals who underwent an elective hepatopancreatic procedure between 2013 and 2017 were identified using the Medicare database, which was merged with the Center for Disease Control and Prevention's Social Vulnerability Index. The win ratio was defined based on a hierarchy of postoperative outcomes: 90-day mortality, perioperative complications, 90-day readmissions, and length of stay. Patients matched based on procedure type, race, sex, age, and Charlson Comorbidity Index score were compared and assessed relative to win ratio.

RESULTS

Among 32,557 Medicare beneficiaries who underwent hepatectomy (n = 11,621, 35.7%) or pancreatectomy (n = 20,936, 64.3%), 16,846 (51.7%) patients were male with median age of 72 years (interquartile range 68-77) and median Charlson Comorbidity Index of 3 (interquartile range 2-8), and a small subset of patients were a racial/ethnic minority (n = 3,759, 11.6%). Adverse events associated with lack of a postoperative optimal outcome included 90-day mortality (n = 2,222, 6.8%), postoperative complication (n = 8,029, 24.7%), readmission (n = 6,349, 19.5%), and length of stay (median: 7 days, interquartile range 5-11). Overall, the patients from low Social Vulnerability Index areas were more likely to "win" with a textbook outcome (win ratio 1.07, 95% confidence interval 1.01-1.12) compared with patients from high social vulnerability counties; in contrast, there was no difference in the win ratio among patients living in average versus high Social Vulnerability Index (win ratio 1.04, 95% confidence interval 0.98-1.10). In assessing surgeon volume, patients who had a liver or pancreas procedure performed by a high-volume surgeon had a higher win ratio versus patients who were treated by a low-volume surgeon (win ratio 1.21, 95% confidence interval 1.16-1.25). In contrast, there was no difference in the win ratio (win ratio 1.01, 95% confidence interval 0.97-1.06) among patients relative to teaching hospital status.

CONCLUSION

Using a novel statistical approach, the win ratio ranked outcomes to create a composite measure to assess a postoperative "win." The WR demonstrated that social vulnerability was an important driver in explaining disparate postoperative outcomes.

Variation of ventral and incisional hernia repairs in kidney transplant recipients

INTRODUCTION

As survivorship following kidney transplant continues to improve, so does the probability of intervening on common surgical conditions, such as ventral or incisional hernia, in this population. Ventral hernia management is known to vary across institutions and this variation has an impact on patient outcomes. We sought to evaluate hospital level variation of ventral or incisional hernia repair (VIHR) in the kidney transplant population.

METHODS

We performed a retrospective review of 100% inpatient Medicare claims to identify patients who underwent kidney transplant between 2007 and 2018. The primary outcome was 1- and 3-year ventral or incisional risk- and reliability-adjusted VIHR rates. Patient and hospital characteristics were evaluated across risk- and reliability-adjusted VIHR rate tertiles. Models were adjusted for age, sex, race, and Elixhauser comorbidities.

RESULTS

Overall, 139,741 patients underwent kidney transplant during the study period with a mean age (SD) of 51.6 (13.7) years. 84,717 (60.6%) were male, and 72,657 (52.0%) were white. Median follow up time was 5.4 years. 2098 (1.50%) patients underwent VIHR. the 1 year risk- and reliability-adjusted hernia repair rates were 0.49% (95% Conf idence Interval (CI) 0.48-0.51, range 0.31-0.59) in tertile 1, 0.63% (95% CI 0.62-0.63, range 0.59-0.68) in tertile 2, and 0.98 (95% CI 0.91-1.05, range 0.68-2.94) in tertile 3. Accordingly, compared to hospitals in tertile 1, the odds of post-transplant hernia repair tertile 2 hospitals were 1.78 (95% CI 1.37-2.31) and at tertile 3 hospitals 3.53 (95% CI 2.87-4.33).

CONCLUSIONS

In a large cohort of Medicare patients undergoing kidney transplant, the overall cumulative incidence of hernia repair varied substantially across hospital tertiles. Patient and hospital characteristics varied across tertile, most notably in diabetes and obesity. Future research is needed to understand if program and surgeon level factors are contributing to the observed variation in treatment of this common disease.

Variation in Hospital Cost and 1-Year Episodes of Care after Diaphragmatic Hernia Repair

BACKGROUND

Diaphragmatic hernia repair is a common operation performed at all types of hospitals. The variation in costs and repeat episodes of care after this operation is not known.

STUDY DESIGN

The Healthcare Cost and Utilization Project State Inpatient and State Ambulatory Surgery and Services Databases for Florida were queried to identify patients undergoing diaphragmatic hernia repair between 2011 and 2018 and the associated inpatient and outpatient encounters within 12 months postoperatively. Hospitals were ranked by cost and grouped into quintiles. All costs and charges were reliability and case-mix adjusted with the use of hierarchical multivariable regression.

RESULTS

In total, 8,848 patients underwent diaphragmatic hernia operations at 158 hospitals. The most expensive hospital quintile had lower surgical volume, location in rural settings, and fewer than 100 beds. There was a wide variation in costs after diaphragmatic hernia repair. On unadjusted comparison, index costs were $23,041 more expensive in hospitals in the highest quintile than in the lowest quintile. Cost differences were persistent even after case-mix and reliability adjustment. The variation in adjusted aggregate charges for associated outpatient and inpatient encounters in the first year after the index operation was considerably lower than that of the index hospitalization.

CONCLUSION

There is nearly a 2-fold variation in the cost of a diaphragmatic hernia repair across hospitals. Most of the variation occurs during the index surgical encounter and not for repeat encounters during the first postoperative year. As bundled payment models mature, hospitals and payers will need to target this variation to ensure cost-efficiency.

Dysphagia among geriatric trauma patients: A population-based study

Objective

To determine the significance of dysphagia on clinical outcomes of geriatric trauma patients.

Methods

This is a retrospective population-based study of geriatric trauma patients 65 years and older utilizing the Florida Agency for Health Care Administration dataset from 2010 to 2019. Patients with pre-admission dysphagia were excluded. Multivariable regression was used to create statistical adjustments. Primary outcomes included mortality and the development of dysphagia. Secondary outcomes included length of stay and complications. Subgroup analyses included patients with dementia, patients who received transgastric feeding tubes (GFTs) or tracheostomies, and speech language therapy consultation.

Results

A total of 52,946 geriatric patients developed dysphagia after admission during a 9-year period out of 1,150,438 geriatric trauma admissions. In general, patients who developed dysphagia had increased mortality, length of stay, and complications. When adjusted for traumatic brain and cervical spine injuries, the addition of mechanical ventilation decreased the mortality odds. This was also observed in the subset of patients with dysphagia who had GFTs placed. Of the three primary risk factors for dysphagia investigated, mechanical ventilation was the most strongly associated with later development of dysphagia and mortality.

Conclusion

The geriatric trauma population is vulnerable to dysphagia with a large number associated with traumatic brain injury, cervical spine injury, and polytraumatic injuries that lead to mechanical ventilation. Earlier intubation/mechanical ventilation in association with GFTs was found to be associated with decreased inpatient hospital mortality. Tracheostomy placement was shown to be an independent risk factor for the development of dysphagia. The utilization of speech language therapy was found to be inconsistently utilized.

One-year Mortality Is Not a Reliable Indicator of Lung Transplant Center Performance

BACKGROUND

In the United States, the Organ Procurement and Transplant Network uses one-year mortality as the primary measure of transplant center quality. We sought to evaluate the reliability of mortality outcomes in lung transplant and compare statistical methods of program performance evaluation.

METHODS

We used the Standard Transplant Analysis and Research files from the United Network for Organ Sharing to identify lung transplant recipients from 2013-2018 in the United States. We stratified hospitals based on 30-day, 1-year and 5-year survival using risk adjustment, reliability adjustment using empirical Bayes technique, and hierarchical Bayesian mixed-effects models currently used by the OPTN. We measured variation in mortality rates and identification of performance outliers between techniques.

RESULTS

We identified 12,769 recipients in 69 centers. Reliability adjustment reduced variation in hospital outcomes and had a large impact on hospital mortality rankings. For example, with 1-year mortality, 28% (5 hospitals) of the "best" hospitals (top 25%) and 18% (3 hospitals) of the "worst" hospitals (bottom 25%) were reclassified after reliability adjustment. The overall reliability of 1-year mortality was low at 0.42. Compared to the Bayesian method used by the OPTN, reliability adjustment identified fewer outliers. 5-year survival reached a higher reliability plateau with a lower volume of cases required.

CONCLUSIONS

The reliability of 1-year mortality in lung transplantation is low, while 5-year survival estimates may be more reliable at lower case volumes. Reliability adjustment yielded more conservative measures of center performance and fewer outliers compared to current Bayesian methods.

Patient, Surgeon, or Hospital: Explaining Variation in Outcomes after Colectomy

BACKGROUND

Complication rates after colectomy remain high. Previous work has failed to establish the relative contribution of patient comorbidities, surgeon performance, and hospital systems in the development of complications after elective colectomy.

STUDY DESIGN

We identified all patients undergoing elective colectomy between 2012 and 2018 at hospitals participating in the Michigan Surgical Quality Collaborative. The primary outcome was development of a postoperative complication. We used risk- and reliability-adjusted generalized linear mixed models to estimate the degree to which variance in patient-, surgeon-, and hospital-level factors contribute to complications.

RESULTS

A total of 15,755 patients were included in the study. The mean hospital-level complication rate was 15.8% (range, 8.7% to 30.2%). The proportion of variance attributable to the patient level was 35.0%, 2.4% was attributable to the surgeon level, and 1.8% was attributable to the hospital level. The predicted probability of complication for the least comorbid patient was 1.5% (CI 0.7-3.1%) at the highest performing hospital with the highest performing surgeon, and 6.6% (CI 3.2-12.2%) at the lowest performing hospital with the lowest performing surgeon. By contrast, the most comorbid patient in the cohort had a 66.3% (CI 39.5-85.6%) or 89.4% (CI 73.7-96.2%) risk of complication.

CONCLUSIONS

This study demonstrated that variance from measured factors at the patient level contributed more than 8-fold more to the development of complications after colectomy compared with variance at the surgeon and hospital level, highlighting the impact of patient comorbidities on postoperative outcomes. These results underscore the importance of initiatives that optimize patient foundational health to improve surgical care.

Social determinants of health and patient-level mortality prediction after trauma

BACKGROUND

Social determinants of health (SDOH) impact patient outcomes in trauma. Census data are often used to account for SDOH; however, there is no consensus on which variables are most important. Social vulnerability indices offer the advantage of combining multiple constructs into a single variable. Our objective was to determine if incorporation of SDOH in patient-level risk-adjusted outcome modeling improved predictive performance.

METHODS

We evaluated two social vulnerability indices at the zip code level: Distressed Community Index (DCI) and National Risk Index (NRI). Individual variable combinations from Agency for Healthcare Research and Quality's SDOH data set were used for comparison. Patients were obtained from the Pennsylvania Trauma Outcomes Study 2000 to 2020. These measures were added to a validated base mortality prediction model with comparison of area under the curve and Bayesian information criterion. We performed center benchmarking using risk-standardized mortality ratios to evaluate change in rank and outlier status based on SDOH. Geospatial analysis identified geographic variation and autocorrelation.

RESULTS

There were 449,541 patients included. The DCI and NRI were associated with an increase in mortality (adjusted odds ratio, 1.02; 95% confidence interval, 1.01-1.03 per 10% percentile rank increase; p < 0.01, respectively). The DCI, NRI, and seven Agency for Healthcare Research and Quality variables also improved base model fit but discrimination was similar. Two thirds of centers changed mortality ranking when accounting for SDOH compared with the base model alone. Outlier status changed in 7% of centers, most representing an improvement from worse-than-expected to nonoutlier or nonoutlier to better-than-expected. There was significant geographic variation and autocorrelation of the DCI and NRI (DCI; Moran's I 0.62, p = 0.01; NRI; Moran's I 0.34, p = 0.01).

CONCLUSION

Social determinants of health are associated with an individual patient's risk of mortality after injury. Accounting for SDOH may be important in risk adjustment for trauma center benchmarking.

LEVEL OF EVIDENCE

Prognostic/Epidemiologic, level IV.

Extending Trauma Quality Improvement Beyond Trauma Centers: Hospital Variation in Outcomes Among Nontrauma Hospitals

OBJECTIVE

The American College of Surgeons (ACS) conducts a robust quality improvement program for ACS-verified trauma centers, yet many injured patients receive care at non-accredited facilities. This study tested for variation in outcomes across non-trauma hospitals and characterized hospitals associated with increased mortality.

SUMMARY BACKGROUND DATA

The study included state trauma registry data of 37,670 patients treated between January 1, 2013, and December 31, 2015. Clinical data were supplemented with data from the American Hospital Association and US Department of Agriculture, allowing comparisons among 100 nontrauma hospitals.

METHODS

Using Bayesian techniques, risk-adjusted and reliability-adjusted rates of mortality and interfacility transfer, as well as Emergency Departments length-of-stay (ED-LOS) among patients transferred from EDs were calculated for each hospital. Subgroup analyses were performed for patients ages >55 years and those with decreased Glasgow coma scores (GCS). Multiple imputation was used to address missing data.

RESULTS

Mortality varied 3-fold (0.9%-3.1%); interfacility transfer rates varied 46-fold (2.1%-95.6%); and mean ED-LOS varied 3-fold (81-231 minutes). Hospitals that were high and low statistical outliers were identified for each outcome, and subgroup analyses demonstrated comparable hospital variation. Metropolitan hospitals were associated increased mortality [odds ratio (OR) 1.7, P = 0.004], decreased likelihood of interfacility transfer (OR 0.7, P ≤ 0.001), and increased ED-LOS (coef. 0.1, P ≤ 0.001) when compared with nonmetropolitan hospitals and risk-adjusted.

CONCLUSIONS

Wide variation in trauma outcomes exists across nontrauma hospitals. Efforts to improve trauma quality should include engagement of nontrauma hospitals to reduce variation in outcomes of injured patients treated at those facilities.

Impact of hospital quality on surgical outcomes in patients with high social vulnerability: Association of textbook outcomes and social vulnerability by hospital quality

BACKGROUND

We sought to define the impact of high- versus low-quality hospitals on the risk of adverse outcomes among patients undergoing hepatopancreatic surgery relative to social vulnerability. Social vulnerability is an important factor associated with risk of adverse postoperative outcomes.

METHODS

Patients from 2013 to 2017 were identified from the Medicare Inpatient Standard Analytic File. Hospital quality was determined by calculating risk-adjusted probability to achieve a textbook outcome. The Social Vulnerability Index was used to categorize patients. Risk-adjusted probability of mortality, morbidity, and textbook outcome was examined across varying social vulnerability indices stratified by low-, average-, and high-quality hospitals.

RESULTS

Among 27,000 patients who underwent a pancreatectomy (67%) or hepatectomy (33%%), median patient age was 72 years, 48% were female, and 89% were White; mean Social Vulnerability Index was 49. Risk-adjusted 90-day mortality (odds ratio: 1.32, 95% CI: 1.20-1.59, P = .004) and postoperative complications (odds ratio: 1.12, 95% confidence interval: 1.00-1.24, P = .044) were both higher among beneficiaries from the highest social vulnerability counties versus the lowest counties. At low-quality hospitals, patients from the highest vulnerability counties had 70% higher odds of mortality (odds ratio: 1.70, 95% confidence interval: 1.16-2.48, P = .007), 31% higher odds of overall morbidity odds ratio: 1.31, 95% confidence interval: 1.05-2.63, P = .013), and 19% lower odds of achieving a textbook outcome (odds ratio: 0.81, 95% confidence interval: 0.66-0.99, P = .035)-all of which were markedly worse compared with outcomes achieved at high-quality hospitals.

CONCLUSION

Among patients with increased social vulnerability, outcomes were considerably better at high-quality hospitals. Referral of socially vulnerable patients to high-quality hospitals represents an important opportunity to ensure optimal outcomes after complex surgery.

Outcomes in Lung Cancer Surgery: Capturing Reliable Metrics

BACKGROUND

Measuring variation in perioperative outcomes to accurately discriminate performance between surgical providers may be limited by reliability. We aimed to evaluate reliability estimates of metrics associated with lung cancer resection.

METHODS

We performed a retrospective cohort study utilizing the 2015 National Cancer Database to identify patients undergoing lung cancer resection. Primary outcomes were reliability estimates for perioperative outcomes and for measures of adherence to clinical benchmarks, generated through hierarchical multi-level modeling techniques.

RESULTS

We identified 27,300 patients undergoing resection. Overall risk- and reliability-adjusted 30- and 90-day mortality rates were 1.7% and 3.3%, respectively; 61.0% and 41.1% of eligible patients received stage-appropriate adjuvant and neoadjuvant therapy. Video-assisted thoracoscopic surgery (VATS) was performed in 59.6% of cases with clinical stage I disease. The mean reliability of 30- and 90-day mortality was 0.11 (standard deviation (SD) 0.09) and 0.22 (SD 0.15), respectively; for performing VATS for stage I disease, 0.97 (SD 0.04). When stratified by hospital volume quartile, the mean reliability of 30-day mortality was 0.04 (SD 0.03) in the lowest and 0.20 (SD 0.10) in the highest quartile. Only 14% of hospitals met an established 0.7 reliability benchmark for 30- and 90-day mortality, but over 97% of hospitals exceeded these benchmarks for providing stage-appropriate systemic therapy and performing VATS for stage I disease.

CONCLUSIONS

Metrics used to compare lung cancer surgical performance between providers have varying levels of reliability. Reliability should be considered when profiling providers, which will become particularly important as lung cancer treatment under screening programs continues to expand.

Administrative Data Are Unreliable for Ranking Hospital Performance Based on Serious Complications After Spine Fusion

STUDY DESIGN

Retrospective analysis of administrative billing data.

OBJECTIVE

To evaluate the extent to which a metric of serious complications determined from administrative data can reliably profile hospital performance in spine fusion surgery.

SUMMARY OF BACKGROUND DATA

While payers are increasingly focused on implementing pay-for-performance measures, quality metrics must reliably reflect true differences in performance among the hospitals profiled.

METHODS

We used State Inpatient Databases from nine states to characterize serious complications after elective cervical and thoracolumbar fusion. Hierarchical logistic regression was used to risk-adjust differences in case mix, along with variability from low case volumes. The reliability of this risk-stratified complication rate (RSCR) was assessed as the variation between hospitals that was not due to chance alone, calculated separately by fusion type and year. Finally, we estimated the proportion of hospitals that had sufficient case volumes to obtain reliable (>0.7) complication estimates.

RESULTS

From 2010 to 2017 we identified 154,078 cervical and 213,133 thoracolumbar fusion surgeries. 4.2% of cervical fusion patients had a serious complication, and the median RSCR increased from 4.2% in 2010 to 5.5% in 2017. The reliability of the RSCR for cervical fusion was poor and varied substantially by year (range 0.04-0.28). Overall, 7.7% of thoracolumbar fusion patients experienced a serious complication, and the RSCR varied from 6.8% to 8.0% during the study period. Although still modest, the RSCR reliability was higher for thoracolumbar fusion (range 0.16-0.43). Depending on the study year, 0% to 4.5% of hospitals had sufficient cervical fusion case volume to report reliable (>0.7) estimates, whereas 15% to 36% of hospitals reached this threshold for thoracolumbar fusion.

CONCLUSION

A metric of serious complications was unreliable for benchmarking cervical fusion outcomes and only modestly reliable for thoracolumbar fusion. When assessed using administrative datasets, these measures appear inappropriate for high-stakes applications, such as public reporting or pay-for-performance.Level of Evidence: 3.

Quality and Location of the Surgical Episode Mediate a Large Proportion of Socioeconomic-Based Survival Disparities in Patients with Resected Stage I-III Colon Cancer

INTRODUCTION

Lower socioeconomic status (SES) is associated with shorter overall survival (OS) in patients with locoregional colon cancer. We aimed to estimate: (1) the proportion of SES-based OS disparities mediated by disparities in the quality and location of surgical treatment in patients with resected stage I-III colon cancer and (2) the relative importance of components of surgical quality.

PATIENTS AND METHODS

We examined patients ages 18-80 years with resected stage I-III colon adenocarcinoma using the 2010-2016 National Cancer Database. SES was defined at the zip code level. Inverse odds weighting mediation analysis was used to estimate the proportion mediated (PM) for nine treatment quality-related and facility-related factors and composite PMs in models including all nine mediators. Models compared high SES patients with each lower SES stratum.

RESULTS

Among 171,009 patients, 5-year OS increased from 70.4% in low SES patients to 78.1% in high SES. When high SES patients were compared with low, lower-middle, and upper-middle SES patients, PM ranges among lower SES strata were: minimally invasive surgery 16.0-16.6%, lymph nodes examined 7.7-9.6%, positive margins 3.8-6.5%, length of stay 16.7-28.1%, readmissions insignificant to 3.7%, treatment at > 1 CoC facility 2.7-3.1%, facility type insignificant to 7.3%, facility volume 2.9-8.2%, and adjusted facility 90-day mortality rates 33.2-42.8%. Composite PMs were 76.9% (95% CI 61.3%, 92.4%) for low SES, 68.7% (95% CI 56.4%, 81.1%) for lower-middle SES, and 60.9% (95% CI 43.1%, 78.6%) for upper-middle SES.

CONCLUSIONS

These data suggest that improving the quality of the surgical episode for disadvantaged patients undergoing resection for locoregional colon cancer could decrease SES-based survival disparities by over half.

Hospital effects drive variation in access to inpatient rehabilitation after trauma

BACKGROUND

Postacute care rehabilitation is critically important to recover after trauma, but many patients do not have access. A better understanding of the drivers behind inpatient rehabilitation facility (IRF) use has the potential for major cost-savings as well as higher-quality and more equitable patient care. We sought to quantify the variation in hospital rates of trauma patient discharge to inpatient rehabilitation and understand which factors (patient vs. injury vs. hospital level) contribute the most.

METHODS

We performed a retrospective cohort study of 668,305 adult trauma patients admitted to 900 levels I to IV trauma centers between 2011 and 2015 using the National Trauma Data Bank. Participants were included if they met the following criteria: age >18 years, Injury Severity Score of ≥9, identifiable injury type, and who had one of the Centers for Medicare & Medicaid Services preferred diagnoses for inpatient rehabilitation under the "60% rule."

RESULTS

The overall risk- and reliability-adjusted hospital rates of discharge to IRF averaged 18.8% in the nonelderly adult cohort (18-64 years old) and 23.4% in the older adult cohort (65 years or older). Despite controlling for all patient-, injury-, and hospital-level factors, hospital discharge of patients to IRF varied substantially between hospital quintiles and ranged from 9% to 30% in the nonelderly adult cohort and from 7% to 46% in the older adult cohort. Proportions of total variance ranged from 2.4% (patient insurance) to 12.1% (injury-level factors) in the nonelderly adult cohort and from 0.3% (patient-level factors) to 26.0% (unmeasured hospital-level factors) in the older adult cohort.

CONCLUSION

Among a cohort of injured patients with diagnoses that are associated with significant rehabilitation needs, the hospital at which a patient receives their care may drive a patient's likelihood of recovering at an IRF just as much, if not more, than their clinical attributes.

LEVEL OF EVIDENCE

Care management, level IV.

Using Consistently Low Performance to Identify Low-Quality Physician Groups

Importance

There has been a growth in the use of performance-based payment models in the past decade, but inherently noisy and stochastic quality measures complicate the assessment of the quality of physician groups. Examining consistently low performance across multiple measures or multiple years could potentially identify a subset of low-quality physician groups.

Objective

To identify low-performing physician groups based on consistently low performance after adjusting for patient characteristics across multiple measures or multiple years for 10 commonly used quality measures for diabetes and cardiovascular disease (CVD).

Design, Setting, and Participants

This cross-sectional study used medical and pharmacy claims and laboratory data for enrollees ages 18 to 65 years with diabetes or CVD in an Aetna health insurance plan between 2016 and 2019. Each physician group's risk-adjusted performance for a given year was estimated using mixed-effects linear probability regression models. Performance was correlated across measures and time, and the proportion of physician groups that performed in the bottom quartile was examined across multiple measures or multiple years. Data analysis was conducted between September 2020 and May 2021.

Exposures

Primary care physician groups.

Main Outcomes and Measures

Performance scores of 6 quality measures for diabetes and 4 for CVD, including hemoglobin A1c (HbA1c) testing, low-density lipoprotein testing, statin use, HbA1c control, low-density lipoprotein control, and hospital-based utilization.

Results

A total of 786 641 unique enrollees treated by 890 physician groups were included; 414 655 (52.7%) of the enrollees were men and the mean (SD) age was 53 (9.5) years. After adjusting for age, sex, and clinical and social risk variables, correlations among individual measures were weak (eg, performance-adjusted correlation between any statin use and LDL testing for patients with diabetes, r = -0.10) to moderate (correlation between LDL testing for diabetes and LDL testing for CVD, r = .43), but year-to-year correlations for all measures were moderate to strong. One percent or fewer of physician groups performed in the bottom quartile for all 6 diabetes measures or all 4 cardiovascular disease measures in any given year, while 14 (4.0%) to 39 groups (11.1%) were in the bottom quartile in all 4 years for any given measure other than hospital-based utilization for CVD (1.1%).

Conclusions and Relevance

A subset of physician groups that was consistently low performing could be identified by considering performance measures across multiple years. Considering the consistency of group performance could contribute a novel method to identify physician groups most likely to benefit from limited resources.

Improving target price calculations in Medicare bundled payment programs

OBJECTIVE

To compare the predictive accuracy of two approaches to target price calculations under Bundled Payments for Care Improvement-Advanced (BPCI-A): the traditional Centers for Medicare and Medicaid Services (CMS) methodology and an empirical Bayes approach designed to mitigate the effects of regression to the mean.

DATA SOURCES

Medicare fee-for-service claims for beneficiaries discharged from acute care hospitals between 2010 and 2016.

STUDY DESIGN

We used data from a baseline period (discharges between January 1, 2010 and September 30, 2013) to predict spending in a performance period (discharges between October 1, 2015 and June 30, 2016). For 23 clinical episode types in BPCI-A, we compared the average prediction error across hospitals associated with each statistical approach. We also calculated an average across all clinical episode types and explored differences by hospital size.

DATA COLLECTION/EXTRACTION METHODS

We used a 20% sample of Medicare claims, excluding hospitals and episode types with small numbers of observations.

PRINCIPAL FINDINGS

The empirical Bayes approach resulted in significantly more accurate episode spending predictions for 19 of 23 clinical episode types. Across all episode types, prediction error averaged $8456 for the CMS approach versus $7521 for the empirical Bayes approach. Greater improvements in accuracy were observed with increasing hospital size.

CONCLUSIONS

CMS should consider using empirical Bayes methods to calculate target prices for BPCI-A.

Peer groups for organisational learning: Clustering with practical constraints

Peer-grouping is used in many sectors for organisational learning, policy implementation, and benchmarking. Clustering provides a statistical, data-driven method for constructing meaningful peer groups, but peer groups must be compatible with business constraints such as size and stability considerations. Additionally, statistical peer groups are constructed from many different variables, and can be difficult to understand, especially for non-statistical audiences. We developed methodology to apply business constraints to clustering solutions and allow the decision-maker to choose the balance between statistical goodness-of-fit and conformity to business constraints. Several tools were utilised to identify complex distinguishing features in peer groups, and a number of visualisations are developed to explain high-dimensional clusters for non-statistical audiences. In a case study where peer group size was required to be small (≤ 100 members), we applied constrained clustering to a noisy high-dimensional data-set over two subsequent years, ensuring that the clusters were sufficiently stable between years. Our approach not only satisfied clustering constraints on the test data, but maintained an almost monotonic negative relationship between goodness-of-fit and stability between subsequent years. We demonstrated in the context of the case study how distinguishing features between clusters can be communicated clearly to different stakeholders with substantial and limited statistical knowledge.

Robotic total knee arthroplasty: A missed opportunity for cost savings in Bundled Payment for Care Improvement initiatives?

BACKGROUND

The use of robotic total knee arthroplasty has become increasingly prevalent. Proponents of robotic total knee arthroplasty tout its potential to not only improve outcomes, but also to reduce costs compared with traditional total knee arthroplasty. Despite its potential to deliver on the value proposition, whether robotic total knee arthroplasty has led to improved outcomes and cost savings within Medicare's Bundled Payment for Care Improvement initiative remains unexplored.

METHODS

Medicare beneficiaries who underwent total knee arthroplasty designated under Medicare severity diagnosis related group 469 or 470 in the year 2017 were identified using the 100% Medicare Inpatient Standard Analytic Files. Hospitals participating in the Bundled Payment for Care Improvement were identified using the Bundled Payment for Care Improvement analytic file. We calculated risk-adjusted, price-standardized payments for the surgical episode from admission through 90-days postdischarge. Outcomes, utilization, and spending were assessed relative to variation between robotic and traditional total knee arthroplasty.

RESULTS

Overall, 198,371 patients underwent total knee arthroplasty (traditional total knee arthroplasty: n= 194,020, 97.8% versus robotic total knee arthroplasty: n = 4,351, 2.2%). Among the 3,272 hospitals that performed total knee arthroplasty, only 300 (9.3%) performed robotic total knee arthroplasty. Among the 183 participating in the Bundled Payment for Care Improvement, only 40 (19%) hospitals performed robotic total knee arthroplasty. Risk-adjusted 90-day episode spending was $14,263 (95% confidence interval $14,231-$14,294) among patients who underwent traditional total knee arthroplasty versus $13,676 (95% confidence interval $13,467-$13,885) among patients who had robotic total knee arthroplasty. Patients who underwent robotic total knee arthroplasty had a shorter length of stay (traditional total knee arthroplasty: 2.3 days, 95% confidence interval: 2.3-2.3 versus robotic total knee arthroplasty: 1.9 days, 95% confidence interval: 1.9-2.0), as well as a lower incidence of complications (traditional total knee arthroplasty: 3.3%, 95% confidence interval: 3.2-3.3 versus robotic total knee arthroplasty: 2.7%, 95% confidence interval: 2.3-3.1). Of note, patients who underwent robotic total knee arthroplasty were less often discharged to a postacute care facility than patients who underwent traditional total knee arthroplasty (traditional total knee arthroplasty: 32.4%, 95% confidence interval: 32.3-32.5 versus robotic total knee arthroplasty: 16.8%, 95% confidence interval 16.1-17.6). Both Bundled Payment for Care Improvement and non-Bundled Payment for Care Improvement hospitals with greater than 50% robotic total knee arthroplasty utilization had lower spending per episode of care versus spending at hospitals with less than 50% robotic total knee arthroplasty utilization.

CONCLUSION

Overall 90-day episode spending for robotic total knee arthroplasty was lower than traditional total knee arthroplasty (Δ $-587, 95% confidence interval: $-798 to $-375). The decrease in spending was attributable to shorter length of stay, fewer complications, as well as lower utilization of postacute care facility. The cost savings associated with robotic total knee arthroplasty was only realized when robotic total knee arthroplasty volume surpassed 50% of all total knee arthroplasty volume. Hospitals participating in the Bundled Payment for Care Improvement may experience cost-saving with increased utilization of robotic total knee arthroplasty.

Inappropriate empirical antibiotic therapy for bloodstream infections based on discordant in-vitro susceptibilities: a retrospective cohort analysis of prevalence, predictors, and mortality risk in US hospitals

BACKGROUND

The prevalence and effects of inappropriate empirical antibiotic therapy for bloodstream infections are unclear. We aimed to establish the population-level burden, predictors, and mortality risk of in-vitro susceptibility-discordant empirical antibiotic therapy among patients with bloodstream infections.

METHODS

Our retrospective cohort analysis of electronic health record data from 131 hospitals in the USA included patients with suspected-and subsequently confirmed-bloodstream infections who were treated empirically with systemic antibiotics between Jan 1, 2005, and Dec 31, 2014. We included all patients with monomicrobial bacteraemia caused by common bloodstream pathogens who received at least one systemic antibiotic either on the day blood cultures were drawn or the day after, and for whom susceptibility data were available. We calculated the prevalence of discordant empirical antibiotic therapy-which was defined as receiving antibiotics on the day blood culture samples were drawn to which the cultured isolate was not susceptible in vitro-overall and by hospital type by using regression tree analysis. We used generalised estimating equations to identify predictors of receiving discordant empirical antibiotic therapy, and used logistic regression to calculate adjusted odds ratios for the relationship between in-hospital mortality and discordant empirical antibiotic therapy.

FINDINGS

21 608 patients with bloodstream infections received empirical antibiotic therapy on the day of first blood culture collection. Of these patients, 4165 (19%) received discordant empirical antibiotic therapy. Discordant empirical antibiotic therapy was independently associated with increased risk of mortality (adjusted odds ratio 1·46 [95% CI, 1·28-1·66]; p<0·0001), a relationship that was unaffected by the presence or absence of resistance or sepsis or septic shock. Infection with antibiotic-resistant species strongly predicted receiving discordant empirical therapy (adjusted odds ratio 9·09 [95% CI 7·68-10·76]; p<0·0001). Most incidences of discordant empirical antibiotic therapy and associated deaths occurred among patients with bloodstream infections caused by Staphylococcus aureus or Enterobacterales.

INTERPRETATION

Approximately one in five patients with bloodstream infections in US hospitals received discordant empirical antibiotic therapy, receipt of which was closely associated with infection with antibiotic-resistant pathogens. Receiving discordant empirical antibiotic therapy was associated with increased odds of mortality overall, even in patients without sepsis. Early identification of bloodstream pathogens and resistance will probably improve population-level outcomes.

FUNDING

US National Institutes of Health, US Centers for Disease Control and Prevention, and US Agency for Healthcare Research and Quality.

Omission of Adjuvant Chemotherapy in Rectal Cancer Patients with Pathologic Complete Response: a National Analysis

BACKGROUND

An increasing number of patients achieve a pathologic complete response (pCR) after neoadjuvant chemoradiation for locally advanced rectal cancer. Consensus guidelines continue to recommend oncologic resection followed by adjuvant chemotherapy in these patients. We hypothesize that there is significant variability in compliance with this recommendation.

METHODS

The National Cancer Database was queried from 2006 to 2015 for patients with locally advanced rectal cancer who underwent neoadjuvant chemoradiation followed by oncologic resection with a pCR (ypT0N0). Hierarchical logistic regression models were used to generate risk and reliability-adjusted rates of adjuvant chemotherapy utilization in patients with pCR at each hospital.

RESULTS

In total, 2421 pCR patients were identified. Five-year overall survival was improved in pCR patients who received adjuvant chemotherapy compared with those who did not (92 vs. 85%, p < 0.01). Multivariate analysis indicated that improvement in overall survival remained associated with adjuvant chemotherapy (HR 0.60, 95% CI 0.44-0.82, p < 0.01). The mean adjuvant chemotherapy utilization rate among hospitals was 32%. There was an upward trend in use over the past decade, but two-thirds still do not receive the recommended therapy. High chemotherapy utilizer hospitals were more likely to be academic centers (54.9 vs. 45.9%, p < 0.01) when compared with low chemotherapy utilizers.

CONCLUSION

Adjuvant chemotherapy is associated with improved survival in rectal cancer patients with pCR following neoadjuvant chemoradiation and oncologic resection. However, utilization among centers in the USA was only 32% with significant variability across centers. National efforts are needed to standardize treatment patterns according to national guidelines.

Variations in surgical spending within hospital systems for complex cancer surgery

BACKGROUND

Approximately 70% of hospitals today are part of larger health systems. Proponents of hospital consolidation tout its potential to reduce health spending and improve outcomes, but to the authors' knowledge the available evidence has suggested that this promise is unrealized. Variations in costs and outcomes within systems may highlight opportunities for collaborative quality improvement and practice standardization. To assess this potential, the authors sought to measure variations in episode spending within and across hospital systems among Medicare beneficiaries undergoing complex cancer surgery.

METHODS

Using 100% Medicare claims data, the authors identified fee-for-service Medicare patients who were undergoing elective pancreatectomy, lung resection, or colectomy for cancer from 2014 through 2016. Risk-adjusted, price-standardized payments for the surgical episode from admission through 30 days after discharge were calculated. The authors then assessed the reliability-adjusted variations at the hospital and system levels.

RESULTS

Average episode payments varied nearly as much within hospital systems for pancreatectomy ($1946 between the lowest and highest spending systems; 95% CI, $1910-$1972), lung resection ($625 between the lowest and highest spending systems; 95% CI, $621-$630), and colectomy ($813 between the lowest and highest spending systems; 95% CI, $809-$817) as they did between the lowest and highest spending hospitals (pancreatectomy: $2034; lung resection: $1789; and colectomy: $770). For pancreatectomy, this variation was driven by index hospitalization spending whereas both index hospitalization and postacute care use drove variations for lung resection and colectomy.

CONCLUSIONS

In this analysis of Medicare patients undergoing complex cancer surgery, wide variations in surgical episode spending were noted both within and across hospital systems. System leaders may seek to better understand variations in practices among their hospitals to standardize care and reduce variations in outcomes, use, and costs.

Needs assessment for novel Gram-negative antibiotics in US hospitals: a retrospective cohort study

BACKGROUND

Evidence-based needs assessments for novel antibiotics against highly-resistant Gram-negative infections (GNIs) are scarce. We aimed to use real-world data from an electronic health record repository to identify treatment opportunities in US hospitals for GNIs resistant to all first-line drugs.

METHODS

For this retrospective cohort study, population estimates with an unmet need for novel Gram-negative antibiotics were quantified using the Cerner Health Facts database (2009-15), aggregating episodes of infection in US hospitals with pathogens displaying difficult-to-treat resistance (DTR; resistance to carbapenems, other β-lactams, and fluoroquinolones) and episodes involving empirical coverage with reserve drugs (colistin or polymyxin B and aminoglycosides). Episodes displaying extended-spectrum cephalosporin resistance (ECR) were also estimated. Episodes were multiplied by site-specific and fixed 14-day treatment durations for conservative and liberal days-of-therapy (DOT) estimates and stratified by site and taxon. Hospital type-specific DOT rates were reliability adjusted to account for random variation; cluster analyses quantified contribution from outbreaks.

FINDINGS

Across 2 996 271 inpatient encounters and 134 hospitals, there were 1352 DTR-GNI episodes, 1765 episodes involving empirical therapy with colistin or polymyxin B, and 16 632 episodes involving aminoglycosides. Collectively, these yielded 39·0 (conservative estimate) to 138·2 (liberal estimate) DOT per 10 000 encounters for a novel DTR-GNI-targeted drug, whereas greater treatment opportunities were identified for ECR (six times greater) and β-lactam susceptible GNIs (70 times greater). The most common DTR-GNI site and pathogen was lower respiratory (14·3 [43·3%] of 33 DOT per 10 000 encounters) and Pseudomonas aeruginosa (522 [38·1%] of 1371 episodes), whereas Enterobacteriaceae urinary-tract infections dominated the ECR or carbapenem-sparing niche (59·0% [5589 of 9535 episodes]) equating to 210·7 DOT per 10 000 encounters. DTR Stenotrophomonas maltophilia, Burkholderia spp, and Achromobacter spp represented less than 1 DOT per 10 000 encounters each. The estimated need for DTR-GNI-targeted antibiotics saw minor contributions by outbreaks and varied from 0·5 to 73·1 DOT per 10 000 encounters by hospital type.

INTERPRETATION

Suspected or documented GNIs with no or suboptimal treatment options are relatively infrequent. Non-revenue-based strategies and innovative trial designs are probably essential to the development of antibiotics with improved effectiveness for these GNIs.

FUNDING

Center for Drug Evaluation and Research, US Food and Drug Administration; Intramural Research Program, National Institutes of Health Clinical Center and the National Institute of Allergy and Infectious Diseases and the National Cancer Institute.

The Need for Accurate Risk Assessment in a High-Risk Patient Population: A NSQIP Study Evaluating Outcomes of Cholecystectomy in the Patient With Cancer

BACKGROUND

Cholecystectomy is considered a low-risk procedure with proven safety in many high-risk patient populations. However, the risk of cholecystectomy in patients with active cancer has not been established.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database was queried to identify all patients with disseminated cancer who underwent cholecystectomy from 2005 to 2016. Postcholecystectomy outcomes were defined for patients with cancer and those without by comparing several outcomes measures. A multivariate model was used to estimate the odds of 30-d mortality.

RESULTS

We compared outcomes in 3097 patients with disseminated cancer to a matched cohort of patients without cancer. Patients with cancer had more comorbidities at baseline: dyspnea (10.5% versus 7.0%, P < 0.0001), steroid use (10.1% versus 3.0%, P < 0.0001), and loss of >10% body weight in 6-mo prior (9.3% versus 1.6%, P < 0.0001). Patients with cancer sustained higher rates of wound (2.3% versus 5.6%, P < 0.0001), respiratory (1.4% versus 3.9%, P < 0.0001), and cardiovascular (2.0% versus 6.8%, P < 0.0001) complications. In addition, patients with disseminated cancer experienced a longer length of stay and higher 30-d mortality. Multivariate modeling showed that the odds of 30-d mortality was 3.3 times greater in patients with cancer.

CONCLUSIONS

Compared to patients without cancer, those with disseminated cancer are at higher risk of complication and mortality following cholecystectomy. Traditional treatment algorithms should be used with caution and care decisions individualized based on the patient's disease status and treatment goals.

Assessment of Primary Care Clinician Concordance With Guidelines for Use of Magnetic Resonance Imaging in Patients With Nonspecific Low Back Pain in the Veterans Affairs Health System

IMPORTANCE

Magnetic responance imaging (MRI) of the lumbar spine that is not concordant with treatment guidelines for low back pain represents an unnecessary cost for US health plans and may be associated with adverse effects. Use of MRI in the US Department of Veterans Affairs (VA) primary care clinics remains unknown.

OBJECTIVE

To assess the use of MRI scans during the first 6 weeks (early MRI scans) of episodes of nonspecific low back pain in VA primary care sites and to determine if historical concordance can identify clinicians and sites that are the least concordant with guidelines.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective cohort study of electronic health records from 944 VA primary care sites from the 3 years ending in 2016. Data were analyzed between January 2017 and August 2019. Participants were patients with new episodes of nonspecific low back pain and the primary care clinicians responsible for their care.

EXPOSURES

MRI scans.

MAIN OUTCOMES AND MEASURES

The proportion of early MRI scans at VA primary care clinics was assessed. Clinician concordance with published guidelines over 2 years was used to select clinicians expected to have low concordance in a third year.

RESULTS

A total of 1 285 405 new episodes of nonspecific low back pain from 920 547 patients (mean [SD] age, 56.7 [15.8] years; 93.6% men) were attributed to 9098 clinicians (mean [SD] age, 52.1 [10.1] years; 55.7% women). An early MRI scan of the lumbar spine was performed in 31 132 of the episodes (2.42%; 95% CI, 2.40%-2.45%). Historical concordance was better than a random draw in selecting the 10% of clinicians who were subsequently the least concordant with published guidelines. For primary care clinicians, the area under the receiver operating characteristic curve was 0.683 (95% CI, 0.658-0.701). For primary care sites, the area was under this curve was 0.8035 (95% CI, 0.754-0.855). The 10% of clinicians with the least historical concordance were responsible for just 19.2% of the early MRI scans performed in the follow-up year.

CONCLUSIONS AND RELEVANCE

VA primary care clinics had low rates of use of early MRI scans. A history of low concordance with imaging guidelines was associated with subsequent low concordance but with limited potential to select clinicians most in need of interventions to implement guidelines.

National Quality Forum Guidelines for Evaluating the Scientific Acceptability of Risk-adjusted Clinical Outcome Measures: A Report From the National Quality Forum Scientific Methods Panel

: Quality measurement is at the heart of efforts to achieve high-quality surgical and medical care at a lower cost. Without accurate quality measures, it is not possible to appropriately align incentives with quality. The aim of these National Quality Forum (NQF) guidelines is to provide measure developers and other stakeholders with guidance on the standards used by the NQF to evaluate the scientific acceptability of performance measures. Using a methodologically rigorous and transparent process for evaluating health care quality measures is the best insurance that alternative payment plans will truly reward and promote higher quality care. Performance measures need to be credible in order for physicians and hospitals to willingly partner with payers in efforts to improve population outcomes. Our goal in creating this position paper is to promote the transparency of NQF evaluations, improve the quality of performance measurements, and engage surgeons and all other stakeholders to work together to advance the science of performance measurement.

Hospital-level variation in the development of persistent critical illness

PURPOSE

Patients with persistent critical illness may account for up to half of all intensive care unit (ICU) bed-days. It is unknown if there is hospital variation in the development of persistent critical illness and if hospital performance affects the incidence of persistent critical illness.

METHODS

This is a retrospective analysis of Veterans admitted to the Veterans Administration (VA) ICUs from 2015 to 2017. Hospital performance was defined by the risk- and reliability-adjusted 30-day mortality. Persistent critical illness was defined as an ICU length of stay of at least 11 days. We used 2-level multilevel logistic regression models to assess variation in risk- and reliability-adjusted probabilities in the development of persistent critical illness.

RESULTS

In the analysis of 100 hospitals which encompassed 153,512 hospitalizations, 4.9% (N = 7640/153,512) developed persistent critical illness. There was variation in the development of persistent critical illness despite controlling for patient characteristics (intraclass correlation: 0.067, 95% CI 0.049-0.091). Hospitals with higher risk- and reliability-adjusted 30-day mortality had higher probabilities of developing persistent critical illness (predicted probability: 0.057, 95% CI 0.051-0.063, p < 0.01) compared to those with lower risk- and reliability-adjusted 30-day mortality (predicted probability: 0.046, 95% CI 0.041-0.051, p < 0.01). The median odds ratio was 1.4 (95% CI 1.33-1.49) implying that, for two patients with the same physiology on admission at two different VA hospitals, the patient admitted to the hospital with higher adjusted mortality would have 40% greater odds of developing persistent critical illness.

CONCLUSION

Hospitals with higher risk- and reliability-adjusted 30-day mortality have a higher probability of developing persistent critical illness. Understanding the drivers of this variation may identify modifiable factors contributing to the development of persistent critical illness.

Variation in value among hospitals performing complex cancer operations

BACKGROUND

While variation in outcomes has driven centralization of complex cancer surgery, variation in cost and value remains unexplored. We evaluated outcomes relative to cost among hospitals performing esophageal and pancreatic resection for cancer.

METHODS

Using 100% Medicare claims data, we identified fee-for-service Medicare patients undergoing elective esophagectomy and pancreatectomy for cancer from 2014 to 2016. Risk- and reliability-adjusted, price-standardized payments for the surgical episode from admission through 30 days post discharge, as well as risk- and reliability-adjusted complication rates for each hospital, were calculated. Hospitals were separated into quintiles relative to payments and outcomes. Highest-value hospitals were defined as hospitals in the top 2 quartiles for both cost and outcomes.

RESULTS

Among 11,586 Medicare beneficiaries who underwent a complex oncologic operation between 2014 and 2016, 66% had a pancreatic neoplasm, while 33% had an esophageal neoplasm. Overall, 31.1% patients underwent an operation at a high-value hospital. Among patients who underwent pancreatectomy, the risk-adjusted postoperative complication rate was 31.4% at the lowest-value hospitals vs 22.7% at highest-value hospitals (odds ratio: 0.57, 95% confidence interval 0.47-0.70). The esophagectomy, risk-adjusted postoperative complication rate was 48.3% at lowest-value hospitals versus 29.8% at highest-value hospitals (odds ratio: 0.36, 95% confidence interval 0.27-0.47). The average difference in episode cost of care for an esophagectomy at lowest- versus highest-value hospitals was $5,617; the difference for pancreatectomy was $2,748.

CONCLUSION

There was wide variation in complication rates and average costs among lowest- versus highest-value hospitals performing esophagectomy and pancreatectomy for cancer. Even among highest quality hospitals, wide variation in average episode costs was noted. Surgeons should seek to better understand practice variation to standardize care and decrease variation in outcomes, utilization, and costs.

Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons

OBJECTIVES

Administrative claims data are commonly used for sepsis surveillance, research, and quality improvement. However, variations in diagnosis, documentation, and coding practices for sepsis and organ dysfunction may confound efforts to estimate sepsis rates, compare outcomes, and perform risk adjustment. We evaluated hospital variation in the sensitivity of claims data relative to clinical data from electronic health records and its impact on outcome comparisons.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study of 4.3 million adult encounters at 193 U.S. hospitals in 2013-2014.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Sepsis was defined using electronic health record-derived clinical indicators of presumed infection (blood culture draws and antibiotic administrations) and concurrent organ dysfunction (vasopressors, mechanical ventilation, doubling in creatinine, doubling in bilirubin to ≥ 2.0 mg/dL, decrease in platelets to < 100 cells/µL, or lactate ≥ 2.0 mmol/L). We compared claims for sepsis prevalence and mortality rates between both methods. All estimates were reliability adjusted to account for random variation using hierarchical logistic regression modeling. The sensitivity of hospitals' claims data was low and variable: median 30% (range, 5-54%) for sepsis, 66% (range, 26-84%) for acute kidney injury, 39% (range, 16-60%) for thrombocytopenia, 36% (range, 29-44%) for hepatic injury, and 66% (range, 29-84%) for shock. Correlation between claims and clinical data was moderate for sepsis prevalence (Pearson coefficient, 0.64) and mortality (0.61). Among hospitals in the lowest sepsis mortality quartile by claims, 46% shifted to higher mortality quartiles using clinical data. Using implicit sepsis criteria based on infection and organ dysfunction codes also yielded major differences versus clinical data.

CONCLUSIONS

Variation in the accuracy of claims data for identifying sepsis and organ dysfunction limits their use for comparing hospitals' sepsis rates and outcomes. Using objective clinical data may facilitate more meaningful hospital comparisons.