Ranking hospitals on surgical mortality: the importance of reliability adjustment

Hospital Variability in Use of Adjuvant Chemotherapy for Patients with Stage 2 and 3 Colon Cancer

BACKGROUND

Following oncologic resection, adjuvant chemotherapy is associated with decreased recurrence and improved survival in stage 3 colon cancer. However, there is controversy regarding its use in stage 2 colon cancer with high-risk features (tumor depth T4, poorly differentiated, positive margin, and/or inadequate lymph node retrieval). Consensus guidelines recommend no adjuvant chemotherapy in the absence of these high-risk features (low-risk stage 2).

OBJECTIVE

This study aimed to examine hospital characteristics associated with poor risk-adjusted, stage-specific guideline compliance for the use of adjuvant chemotherapy in stage 3 and low-risk stage 2 colon cancer.

DESIGN

This was a retrospective study. Stepwise logistic regression was used to identify patient and hospital factors associated with administration of adjuvant chemotherapy. Hierarchical regression models were used to calculate risk- and reliability-adjusted rates of chemotherapy use and observed-to-expected ratios in each hospital's stage 2 low-risk and stage 3 patients.

SETTINGS

Data were retrieved from the National Cancer Database.

PATIENTS

Patients selected were adults treated with oncologic resection for stage 2 to 3 colon cancer between 2004 and 2010.

MAIN OUTCOME MEASURES

The primary outcome measured was receipt of adjuvant chemotherapy.

RESULTS

A total of 167,345 patients were identified at 1395 hospitals. The mean overall risk-adjusted adjuvant chemotherapy rate was 65.3% for stage 3 and 15.2% for low-risk stage 2. Analysis of low outlier hospitals for stage 3 colon cancer, where adjuvant chemotherapy was underutilized, demonstrated that 62.8% were low-volume centers and 51.4% were community centers. Of high outlier hospitals for stage 2 low-risk disease, where adjuvant chemotherapy was overutilized, 87.2% were low-volume hospitals and 67.2% were community centers.

LIMITATIONS

Selection bias and the inability to compare specific chemotherapy regimens were limitations of this study.

CONCLUSIONS

Following oncologic resection, administration of adjuvant chemotherapy for low-risk stage 2 and stage 3 disease varies substantially among hospitals in the United States. Outlier hospitals were most likely to be low-volume community centers.

The HARM score for gastrointestinal surgery: Application and validation of a novel, reliable and simple tool to measure surgical quality and outcomes

BACKGROUND

The HospitAl length of stay, Readmissions and Mortality (HARM) score is a simple, inexpensive quality tool, linked directly to patient outcomes. We assess the HARM score for measuring surgical quality across multiple surgical populations.

METHODS

Upper gastrointestinal, hepatobiliary, and colorectal surgery cases between 2005 and 2009 were identified from the Healthcare Cost and Utilization Project California State Inpatient Database. Composite and individual HARM scores were calculated from length of stay, 30-day readmission and mortality, correlated to complication rates for each hospital and stratified by operative type.

RESULTS

71,419 admissions were analyzed. Higher HARM scores correlated with higher complication rates for all cases after risk adjustment and stratification by operation type, elective or emergent status.

CONCLUSIONS

The HARM score is a simple and valid quality measurement for upper gastrointestinal, hepatobiliary and colorectal surgery. The HARM score could facilitate benchmarking to improve patient outcomes and resource utilization, and may facilitate outcome improvement.

Variation in Hospital Risk–Adjusted Mortality Rates Following Transcatheter Aortic Valve Replacement in the United States

Background—

The use of transcatheter aortic valve replacement (TAVR) to treat aortic stenosis in the United States is growing, yet little is known about the variation in procedural outcomes in community practice. We developed a TAVR in-hospital mortality risk model and used it to quantify variation in mortality rates across United States (US) TAVR centers.

Methods and Results—

We analyzed data from 22 248 TAVR procedures performed at 318 sites participating in the Society of Thoracic Surgeons/American College of Cardiology Transcatheter Valve Therapy Registry (November 2011 to October 2014). A Bayesian hierarchical model was developed to estimate hospital-specific risk-adjusted mortality rates adjusting for 40 patient baseline factors. A total of 1130 in-hospital deaths (5.1%) were observed. Reliability-adjusted risk-adjusted mortality rate estimates ranged from 3.4% to 7.7% with an interquartile range of 4.8% to 5.4%. A patient’s predicted odds of dying was 80% higher if treated by a hospital 1 standard deviation above the mean compared with a hospital 1 standard deviation below the mean (odds ratio =1.8; 95% credible interval, 1.4%–2.2%).

Conclusions—

Risk modeling of TAVR in-hospital mortality revealed variation in risk-adjusted mortality rates during the US early commercial experience. Transcatheter Valve Therapy Registry analyses using this model will support research, feedback reporting, and the identification of factors associated with quality.

ICU Use and Quality of Care for Patients With Myocardial Infarction and Heart Failure

BACKGROUND

Quality of care for acute myocardial infarction (AMI) and heart failure (HF) varies across hospitals, but the factors driving variation are incompletely understood. We evaluated the relationship between a hospital's ICU or coronary care unit (CCU) admission rate and quality of care provided to patients with AMI or HF.

METHODS

A retrospective cohort study of Medicare beneficiaries hospitalized in 2010 with AMI or HF was performed. Hospitals were grouped into quintiles according to their risk- and reliability-adjusted ICU admission rates for AMI or HF. We examined the rates that hospitals failed to deliver standard AMI or HF processes of care (process measure failure rates), 30-day mortality, 30-day readmissions, and Medicare spending after adjusting for patient and hospital characteristics.

RESULTS

Hospitals in the lowest quintile had ICU admission rates < 29% for AMI or < 8% for HF. Hospitals in the top quintile had rates > 61% for AMI or > 24% for HF. Hospitals in the highest quintile had higher process measure failure rates for some but not all process measures. Hospitals in the top quintile had greater 30-day mortality (14.8% vs 14.0% [P = .002] for AMI; 11.4% vs 10.6% [P < .001] for HF), but no differences in 30-day readmissions or Medicare spending were seen compared with hospitals in the lowest quintile.

CONCLUSIONS

Hospitals with the highest rates of ICU admission for patients with AMI or HF delivered lower quality of care and had higher 30-day mortality for these conditions. Hospitals with high ICU use may be targets to improve care delivery.

Rising Billing for Intermediate Intensive Care among Hospitalized Medicare Beneficiaries between 1996 and 2010

RATIONALE

Intermediate care (i.e., step-down or progressive care) is an alternative to the intensive care unit (ICU) for patients with moderate severity of illness. The adoption and current use of intermediate care is unknown.

OBJECTIVES

To characterize trends in intermediate care use among U.S. hospitals.

METHODS

We examined 135 million acute care hospitalizations among elderly individuals (≥65 yr) enrolled in fee-for-service Medicare (U.S. federal health insurance program) from 1996 to 2010. We identified patients receiving intermediate care as those with intensive care or coronary care room and board charges labeled intermediate ICU.

MEASUREMENTS AND MAIN RESULTS

In 1996, a total of 960 of the 3,425 hospitals providing critical care billed for intermediate care (28%), and this increased to 1,643 of 2,783 hospitals (59%) in 2010 (P < 0.01). Only 8.2% of Medicare hospitalizations in 1996 were billed for intermediate care, but billing steadily increased to 22.8% by 2010 (P < 0.01), whereas the percentage billed for ICU care and ward-only care declined. Patients billed for intermediate care had more acute organ failures diagnoses codes compared with general ward patients (22.4% vs. 15.8%). When compared with patients billed for ICU care, those billed for intermediate care had fewer organ failures (22.4% vs. 43.4%), less mechanical ventilation (0.9% vs. 16.7%), lower mean Medicare spending ($8,514 vs. $18,150), and lower 30-day mortality (5.6% vs. 16.5%) (P < 0.01 for all comparisons).

CONCLUSIONS

Intermediate care billing increased markedly between 1996 and 2010. These findings highlight the need to better define the value, specific practices, and effective use of intermediate care for patients and hospitals.

Understanding Inpatient Cost Variation in Kidney Transplantation: Implications for Payment Reforms

OBJECTIVE

To examine the magnitude and sources of inpatient cost variation for kidney transplantation.

METHODS

We used the 2005-2009 Nationwide Inpatient Sample to identify patients who underwent kidney transplantation. We first calculated the patient-level cost of each transplantation admission and then aggregated costs to the hospital level. We fit hierarchical linear regression models to identify sources of cost variation and to estimate how much unexplained variation remained after adjusting for case-mix variables commonly found in administrative datasets.

RESULTS

We identified 8866 living donor (LDRT) and 5589 deceased donor (DDRT) renal transplantations. We found that higher costs were associated with the presence of complications (LDRT, 14%; P <.001; DDRT, 24%; P <.001), plasmapheresis (LDRT, 27%; P <.001; DDRT, 27%; P <.001), dialysis (LDRT, 4%; P <.001), and prolonged length of stay (LDRT, 84%; P <.001; DDRT, 82%; P <.001). Even after case-mix adjustment, a considerable amount of unexplained cost variation remained between transplant centers (DDRT, 52%; LDRT, 66%).

CONCLUSION

Although significant inpatient cost variation is present across transplant centers, much of the cost variation for kidney transplantation is not explained by commonly used risk-adjustment variables in administrative datasets. These findings suggest that although there is an opportunity to achieve savings through payment reforms for kidney transplantation, policymakers should seek alternative sources of information (eg, clinical registry data) to delineate sources of warranted and unwarranted cost variation.

A national evaluation of clinical and economic outcomes in open versus laparoscopic colorectal surgery

BACKGROUND

Surgical value is based on optimizing clinical and financial outcomes. The clinical benefits of laparoscopic surgery are well established; however, many patients are still not offered a laparoscopic procedure. Our objective was to compare the modern clinical and financial outcomes of laparoscopic and open colorectal surgery.

METHODS

The Premier Perspective database identified patients undergoing elective colorectal resections from January 1, 2013 to December 31, 2013. Cases were stratified by operative approach into laparoscopic and open cohorts. Groups were controlled on all demographics, diagnosis, procedural, hospital characteristics, surgeon volume, and surgeon specialty and then compared for clinical and financial outcomes. The main outcome measures were length of stay (LOS), complications, readmission rates, and cost by surgical approach.

RESULTS

A total of 6343 patients were matched and analyzed in each cohort. The most common diagnosis was diverticulitis (p = 0.0835) and the most common procedure a sigmoidectomy (p = 0.0962). The LOS was significantly shorter in laparoscopic compared to open (mean 5.78 vs. 7.80 days, p < 0.0001). The laparoscopic group had significantly lower readmission (5.82 vs. 7.68 %, p < 0.0001), complication (32.60 vs. 42.28 %, p < 0.0001), and mortality rates (0.52 vs. 1.28 %, p < 0.0001). The total cost was significantly lower in laparoscopic than in open (mean $17,269 vs. $20,552, p < 0.0001). By category, laparoscopy was significantly more cost-effective for pharmacy (p < 0.0001), room and board (p < 0.0001), recovery room (p = 0.0058), ICU (p < 0.0001), and laboratory and imaging services (both p < 0.0001). Surgical supplies (p < 0.0001), surgery (p < 0.0001), and anesthesia (p = 0.0053) were higher for the laparoscopic group.

CONCLUSIONS

Laparoscopy is more cost-effective and produces better patient outcomes than open colorectal surgery. Minimally invasive colorectal surgery is now the standard that should be offered to patients, providing value to both patient and provider.

Association between use of warfarin with common sulfonylureas and serious hypoglycemic events: retrospective cohort analysis

STUDY QUESTION

Is warfarin use associated with an increased risk of serious hypoglycemic events among older people treated with the sulfonylureas glipizide and glimepiride?

METHODS

This was a retrospective cohort analysis of pharmacy and medical claims from a 20% random sample of Medicare fee for service beneficiaries aged 65 years or older. It included 465,918 beneficiaries with diabetes who filled a prescription for glipizide or glimepiride between 2006 and 2011 (4,355,418 person quarters); 71,895 (15.4%) patients also filled a prescription for warfarin (416,479 person quarters with warfarin use). The main outcome measure was emergency department visit or hospital admission with a primary diagnosis of hypoglycemia in person quarters with concurrent fills of warfarin and glipizide/glimepiride compared with the rates in quarters with glipizide/glimepiride fills only, Multivariable logistic regression was used to adjust for individual characteristics. Secondary outcomes included fall related fracture and altered consciousness/mental status.

SUMMARY ANSWER AND LIMITATIONS

In quarters with glipizide/glimepiride use, hospital admissions or emergency department visits for hypoglycemia were more common in person quarters with concurrent warfarin use compared with quarters without warfarin use (294/416,479 v 1903/3,938,939; adjusted odds ratio 1.22, 95% confidence interval 1.05 to 1.42). The risk of hypoglycemia associated with concurrent use was higher among people using warfarin for the first time, as well as in those aged 65-74 years. Concurrent use of warfarin and glipizide/glimepiride was also associated with hospital admission or emergency department visit for fall related fractures (3919/416,479 v 20,759/3,938,939; adjusted odds ratio 1.47, 1.41 to 1.54) and altered consciousness/mental status (2490/416,479 v 14,414/3,938,939; adjusted odds ratio 1.22, 1.16 to 1.29). Unmeasured factors could be correlated with both warfarin use and serious hypoglycemic events, leading to confounding. The findings may not generalize beyond the elderly Medicare population.

WHAT THIS STUDY ADDS

A substantial positive association was seen between use of warfarin with glipizide/glimepiride and hospital admission/emergency department visits for hypoglycemia and related diagnoses, particularly in patients starting warfarin. The findings suggest the possibility of a significant drug interaction between these medications.

FUNDING, COMPETING INTERESTS, DATA SHARING

JAR and DPG receive support from the National Institute on Aging, the Commonwealth Fund, and the Leonard D. Schaeffer Center for Health Policy and Economics at the University of Southern California. ABJ receives support from the NIH Office of the Director. No additional data are available.

The relationship between historical risk-adjusted 30-day mortality and subsequent hip fracture outcomes: Retrospective cohort study

BACKGROUND

While 30-day risk-adjusted mortality is a performance measure for hip fracture care, it has not been shown to predict long-term outcomes. We assessed whether hospital rankings based on historical 30-day mortality predicted subsequent hip fracture outcomes.

METHODS

Using national Medicare data, we calculated annual hospital performance rankings based on standardized 30-day hip fracture mortality ratios. We used logistic regression to measure the association of patients' survival at 180 days with their hospital's ranking for the year prior to admission. Subgroup analyses assessed whether associations between hospital performance and 180-day outcomes were similar for community-dwelling patients as well as those living in nursing homes prior to fracture.

RESULTS

Out of 378,077 patients hospitalized with hip fractures between January 1, 2007 and June 30, 2009, 81,653 (21.6%) died by 180 days. Worse historical hospital performance was associated with a greater adjusted odds of 30 day mortality (odds ratio (OR), fourth vs. first quartile: 1.24, 95% confidence interval (CI): 1.18, 1.29, P<0.001) and 180 day mortality (OR, fourth vs. first quartile: 1.15, 95% CI 1.11, 1.18, P<0.001). Past hospital performance was associated with death or new nursing home placement among community dwellers (OR, fourth vs. first quartile: 1.09, 95% CI 1.05, 1.13, P<0.001), but was not associated with death or new dependence in locomotion among nursing home residents (OR 1.05, 95% CI 0.97, 1.15, P=0.229).

CONCLUSIONS

Better historical hospital hip fracture mortality predicts modest decreases in mortality at 180 days for subsequent patients, but is inconsistently associated with changes in functional outcomes.

LEVEL OF EVIDENCE

Level 3 (Non-randomized controlled cohort study).

Evidence and practice in spine registries

BACKGROUND AND PURPOSE

We performed a systematic review and a survey in order to (1) evaluate the evidence for the impact of spine registries on the quality of spine care, and with that, on patient-related outcomes, and (2) evaluate the methodology used to organize, analyze, and report the "quality of spine care" from spine registries.

METHODS

To study the impact, the literature on all spinal disorders was searched. To study methodology, the search was restricted to degenerative spinal disorders. The risk of bias in the studies included was assessed with the Newcastle-Ottawa scale. Additionally, a survey among registry representatives was performed to acquire information about the methodology and practice of existing registries.

RESULTS

4,273 unique references up to May 2014 were identified, and 1,210 were eligible for screening and assessment. No studies on impact were identified, but 34 studies were identified to study the methodology. Half of these studies (17 of the 34) were judged to have a high risk of bias. The survey identified 25 spine registries, representing 14 countries. The organization of these registries, methods used, analytical approaches, and dissemination of results are presented.

INTERPRETATION

We found a lack of evidence that registries have had an impact on the quality of spine care, regardless of whether intervention was non-surgical and/or surgical. To improve the quality of evidence published with registry data, we present several recommendations. Application of these recommendations could lead to registries showing trends, monitoring the quality of spine care given, and ultimately improving the value of the care given to patients with degenerative spinal disorders.

Impact of Surgical Quality Improvement on Payments in Medicare Patients

OBJECTIVE

To examine the financial impact of quality improvement using Medicare payment data.

BACKGROUND

Demonstrating a business case for quality improvement--that is, that fewer complications translates into lower costs--is essential to justify investment in quality improvement. Prior research is limited to cross-sectional studies showing that patients with complications have higher costs. We designed a study to better evaluate the relationship between payments and complications by using quality improvement itself as a measured outcome.

METHODS

We used national Medicare data for patients undergoing general (n = 1,485,667) and vascular (n = 531,951) procedures. We calculated hospitals' rates of serious complications in 2 time periods: 2003-2004 and 2009-2010. We sorted hospitals into quintiles by the change in complication rates across these time periods. Costs were assessed using price-standardized Medicare payments, and regression analyses used to determine the average change in payments over time.

RESULTS

There was significant change in serious complication rates across the 2 time periods. The top 20% of hospitals demonstrated a 38% decrease (14.3% vs 11.6%, P < 0.001) in complications; in contrast the bottom 20% demonstrated a 25% increase (11.1% vs 16.5%, P < 0.001). There was a strong relationship between quality improvement and payments. The top hospitals reduced their payments by $1544 per patient (95% confidence interval: $1334-1755), whereas the bottom of hospitals had no significant change (average $67 increase, 95% confidence interval: -$123 to $258).

CONCLUSIONS

Hospitals that reduced their complications over time had significant reductions in Medicare payments. This demonstrates that payers are clearly incentivized to invest in quality improvement.

Neighborhood Influences on Emergency Medical Services Use for Acute Stroke: A Population-Based Cross-sectional Study

STUDY OBJECTIVE

Delay to hospital arrival limits acute stroke treatment. Use of emergency medical services (EMS) is key in ensuring timely stroke care. We aim to identify neighborhoods with low EMS use and to evaluate whether neighborhood-level factors are associated with EMS use.

METHODS

We conducted a secondary analysis of data from the Brain Attack Surveillance in Corpus Christi project, a population-based stroke surveillance study of ischemic stroke and intracerebral hemorrhage cases presenting to emergency departments in Nueces County, TX. The primary outcome was arrival by EMS. The primary exposures were neighborhood resident age, poverty, and violent crime. We estimated the association of neighborhood-level factors with EMS use, using hierarchic logistic regression, controlling for individual factors (stroke severity, ethnicity, and age).

RESULTS

During 2000 to 2009 there were 4,004 identified strokes, with EMS use data available for 3,474. Nearly half (49%) of stroke cases arrived by EMS. Adjusted stroke EMS use was lower in neighborhoods with higher family income (odds ratio [OR] 0.86; 95% confidence interval [CI] 0.75 to 0.97) and a larger percentage of older adults (OR 0.70; 95% CI 0.56 to 0.89). Individual factors associated with stroke EMS use included white race (OR 1.41; 95% CI 1.13 to 1.76) and older age (OR 1.36 per 10-year age increment; 95% CI 1.27 to 1.46). The proportion of neighborhood stroke cases arriving by EMS ranged from 17% to 71%. The fully adjusted model explained only 0.3% (95% CI 0% to 1.1%) of neighborhood EMS stroke use variance, indicating that individual factors are more strongly associated with stroke EMS use than neighborhood factors.

CONCLUSION

Although some neighborhood-level factors were associated with EMS use, patient-level factors explained nearly all variability in stroke EMS use. In this community, strategies to increase EMS use should target individuals rather than specific neighborhoods.

Variation in Medicare Payments for Colorectal Cancer Surgery

PURPOSE

Colorectal cancer (CRC) is the second most expensive cancer in the United States. Episode-based bundled payments may be a strategy to decrease costs. However, it is unknown how payments are distributed across hospitals and different perioperative services.

METHODS

We extracted actual Medicare payments for patients in the fee-for-service Medicare population who underwent CRC surgery between January 2004 and December 2006 (N = 105,016 patients). Payments included all service types from the date of hospitalization up to 1 year later. Hospitals were ranked from least to most expensive and grouped into quintiles. Results were case-mix adjusted and price standardized using empirical Bayes methods. We assessed the contributions of index hospitalization, physician services, readmissions, and postacute care to the overall variation in payment.

RESULTS

There is wide variation in total payments for CRC care within the first year after CRC surgery. Actual Medicare payments were $51,345 per patient in the highest quintile and $26,441 per patient in the lowest quintile, representing a difference of Δ = $24,902. Differences were persistent after price standardization (Δ = $17,184 per patient) and case-mix adjustment (Δ = $4,790 per patient). Payments for the index surgical hospitalization accounted for the largest share (65%) of payments but only minimally varied (11.6%) across quintiles. However, readmissions and postacute care services accounted for substantial variations in total payments.

CONCLUSION

Medicare spending in the first year after CRC surgery varies across hospitals even after case-mix adjustment and price standardization. Variation is largely driven by postacute care and not the index surgical hospitalization. This has significant implications for policy decisions on how to bundle payments and define episodes of surgical CRC care.

Temporal Changes in the Influence of Hospitals and Regional Healthcare Networks on Severe Sepsis Mortality

OBJECTIVES

There is systematic variation between hospitals in their care of severe sepsis, but little information on whether this variation impacts sepsis-related mortality, or how hospitals' and health-systems' impacts have changed over time. We examined whether hospital and regional organization of severe sepsis care is associated with meaningful differences in 30-day mortality in a large integrated health care system, and the extent to which those effects are stable over time.

DESIGN

In this retrospective cohort study, we used risk- and reliability-adjusted hierarchical logistic regression to estimate hospital- and region-level random effects after controlling for severity of illness using a rich mix of administrative and clinical laboratory data.

SETTING

One hundred fourteen U.S. Department of Veterans Affairs hospitals in 21 geographic regions.

PATIENTS

Forty-three thousand seven hundred thirty-three patients with severe sepsis in 2012, compared to 33,095 such patients in 2008.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The median hospital in the worst quintile of performers had a risk-adjusted 30-day mortality of 16.7% (95% CI, 13.5%, 20.5%) in 2012 compared with the best quintile, which had a risk-adjusted mortality of 12.8% (95% CI, 10.7%, 15.3%). Hospitals and regions explained a statistically and clinically significant proportion of the variation in patient outcomes. Thirty-day mortality after severe sepsis declined from 18.3% in 2008 to 14.7% in 2012 despite very similar severity of illness between years. The proportion of the variance in sepsis-related mortality explained by hospitals and regions was stable between 2008 and 2012.

CONCLUSIONS

In this large integrated healthcare system, there is clinically significant variation in sepsis-related mortality associated with hospitals and regions. The proportion of variance explained by hospitals and regions has been stable over time, although sepsis-related mortality has declined.

Reliability of hospital cost profiles in inpatient surgery

BACKGROUND

With increased policy emphasis on shifting risk from payers to providers through mechanisms such as bundled payments and accountable care organizations, hospitals are increasingly in need of metrics to understand their costs relative to peers. However, it is unclear whether Medicare payments for surgery can reliably compare hospital costs.

METHODS

We used national Medicare data to assess patients undergoing colectomy, pancreatectomy, and open incisional hernia repair from 2009 to 2010 (n = 339,882 patients). We first calculated risk-adjusted hospital total episode payments for each procedure. We then used hierarchical modeling techniques to estimate the reliability of total episode payments for each procedure and explored the impact of hospital caseload on payment reliability. Finally, we quantified the number of hospitals meeting published reliability benchmarks.

RESULTS

Mean risk-adjusted total episode payments ranged from $13,262 (standard deviation [SD] $14,523) for incisional hernia repair to $25,055 (SD $22,549) for pancreatectomy. The reliability of hospital episode payments varied widely across procedures and depended on sample size. For example, mean episode payment reliability for colectomy (mean caseload, 157) was 0.80 (SD 0.18), whereas for pancreatectomy (mean caseload, 13) the mean reliability was 0.45 (SD 0.27). Many hospitals met published reliability benchmarks for each procedure. For example, 90% of hospitals met reliability benchmarks for colectomy, 40% for pancreatectomy, and 66% for incisional hernia repair.

CONCLUSION

Episode payments for inpatient surgery are a reliable measure of hospital costs for commonly performed procedures, but are less reliable for lower volume operations. These findings suggest that hospital cost profiles based on Medicare claims data may be used to benchmark efficiency, especially for more common procedures.

What is quality, and can we define it in lung cancer?-the case for quality improvement

Decades worth of advances in diagnostics and therapeutics are associated with only marginal improvements in survival among lung cancer patients. An obvious explanation is late stage at presentation, but gaps in the quality of care may be another reason for stifled improvements in survival rates. A framework for quality put forth by Avedis Donabedian consists of measuring structures-of-care, processes, and outcomes. Using this approach to explore for potential quality gaps, there is evidence of inexplicable variability in outcomes across patients and hospitals; variation in outcomes across differing provider types (structures-of-care); and variation in approaches to staging (processes-of-care). However, this research has limitations and incontrovertible evidence of quality gaps is challenging to obtain. Other challenges to defining quality include scientific and clinical uncertainty among providers and the fact that quality is a multi-dimensional construct that cannot be measured by a single metric. Nonetheless, two facts compel us to pursue quality improvement: (I) both empirically and anecdotally, actual care falls short of expected care; and (II) evidence of potential quality gaps is not ignorable primarily on ethical grounds.

Pancreatic Resection Results in a Statewide Surgical Collaborative

BACKGROUND

A strong relationship between hospital caseload and adverse outcomes has been demonstrated for pancreatic resections. Participation in regional surgical collaboratives may mitigate this phenomenon. This study sought to investigate changes over time in adverse outcomes after pancreatectomy across hospitals with different caseloads in a statewide surgical collaborative.

METHODS

The study investigated patients undergoing pancreatic resection from January 2008 to August 2013 at Michigan Surgical Quality Collaborative (MSQC) hospitals (1007 patients in 19 academic and community hospitals). Risk-adjusted rates of major complications, mortality, and failure to rescue were compared between hospitals based on caseloads (low, medium, and high) in early (2008-2010) and later (2011-2013) periods. Finally, the degree to which different complications explained changes in hospital outcome variation was assessed.

RESULTS

Adjusted rates of major complications and mortality decreased over time, driven largely by improvements at low-caseload hospitals. In 2008-2010, risk-adjusted major complication rates were higher for low-caseload than for high-caseload hospitals (27.8 vs. 17.8 %; p = 0.02). However, these differences were attenuated in 2011-2013 (22.2 vs. 20.0 %; p = 0.74). Similarly, adjusted mortality rates were higher in low-caseload hospitals in 2008-2010 (6.2 vs. 0.8 %; p = 0.02), but these differences were attenuated in 2011-2013 (3.3 vs. 1.1 %; p = 0.18). Variation in major complications decreased, largely due to decreased variation in "medical" complication rates, with less change in surgical-site complications.

CONCLUSION

Participation in regional quality collaboratives by lower-volume hospitals can attenuate the volume-outcome relationship for pancreatic surgery. Continued work in collaboratives with an emphasis on technical and intraoperative aspects of care may improve overall quality of care.

Quality-Cost Relationship in Congenital Heart Surgery

BACKGROUND

There is an increasing focus on optimizing health care quality and reducing costs. The care of children undergoing heart surgery requires significant investment of resources, and it remains unclear how costs of care relate to quality. We evaluated this relationship across a multicenter cohort.

METHODS

Clinical data from The Society of Thoracic Surgeons Database were merged with cost data from the Pediatric Health Information Systems Database for children undergoing heart surgery (2006 to 2010). Hospital-level costs were modeled using Bayesian hierarchical methods adjusting for case-mix, and hospitals were categorized into cost tertiles. The primary quality metric evaluated was in-hospital mortality.

RESULTS

Overall, 27 hospitals (30,670 patients) were included. Median adjusted cost per case was $82,360 and varied fivefold across hospitals, while median adjusted mortality was 3.4% and ranged from 2.4% to 5.0% across hospitals. Overall, hospitals in the lowest cost tertile had significantly lower adjusted mortality rates compared with the middle and high cost tertiles (2.5% vs 3.8% and 3.5%, respectively, both p < 0.001). When assessed at the individual hospital level, most (75%) but not all hospitals in the lowest cost tertile were also in the lowest mortality tertile. Similar relationships were seen across the spectrum of surgical complexity. Lower cost hospitals also had shorter length of stay and trends toward fewer major complications.

CONCLUSIONS

Lowest cost hospitals generally deliver the highest quality care for children undergoing heart surgery, although there is some variation in this relationship. This information is important in the design of initiatives aiming to optimize health care value in this population.

Laparoscopic versus open ventral hernia repair: longitudinal outcomes and cost analysis using statewide claims data

BACKGROUND

There is still considerable debate regarding the best operative approach to ventral hernia repair. Using two large statewide databases, this study sought to evaluate the longitudinal outcomes and associated costs of laparoscopic and open ventral hernia repair.

METHODS

All patients undergoing elective ventral hernia repair from 2007-2011 were identified from inpatient discharge data from California and New York. In-hospital morbidity, in-hospital mortality, incidence of readmission, and incidence of revisional ventral hernia repair were evaluated as a function of surgical technique. The associated costs of medical care for laparoscopic versus open ventral hernia repair were evaluate for both the index procedure and all subsequent admissions and procedures within the study period.

RESULTS

A total of 13,567 patients underwent elective ventral hernia repair with mesh; 9228 (69%) underwent OVHR and 4339 (31%) underwent LVHR. At time of the index procedure, LVHR was associated with a lower incidence of reoperation (OR 0.29, CI 0.12-0.58, p = 0.001), wound disruption (OR 0.35, CI 0.16-0.78, p = 0.01), wound infection (OR 0.50, CI 0.25-0.70, p < 0.001), blood transfusion (OR 0.47, CI 0.36-0.61, p < 0.001), ARDS (OR 0.74, CI 0.54-0.99, p < 0.05), and total index visit complications (OR 0.72, CI 0.64-0.80, p < 0.001). LVHR was associated with significantly fewer readmissions (OR 0.81, CI 0.75-0.88, p < 0.001) and a lower risk for revisional VHR (OR 0.75, CI 0.64-0.88, p < 0.001). LVHR was associated with lower total costs at 1 year ($3451, CI 1892-5011, p < 0.001).

CONCLUSIONS

Open ventral hernia repair was associated with a higher incidence of perioperative complications, postoperative readmissions and need for revisional hernia repair when compared to laparoscopic ventral hernia repair, even when controlling for patient sociodemographics. In congruence, open ventral hernia repair was associated with higher costs for both the index hernia repair and tallied over the length of follow-up for readmissions and revisional hernia repair.

Hospitals with the highest intensive care utilization provide lower quality pneumonia care to the elderly

OBJECTIVE

Quality of care for patients admitted with pneumonia varies across hospitals, but causes of this variation are poorly understood. Whether hospitals with high ICU utilization for patients with pneumonia provide better quality care is unknown. We sought to investigate the relationship between a hospital's ICU admission rate for elderly patients with pneumonia and the quality of care it provided to patients with pneumonia.

DESIGN

Retrospective cohort study.

SETTING

Two thousand eight hundred twelve U.S. hospitals.

PATIENTS

Elderly (age≥65 years) fee-for-service Medicare beneficiaries with either a (1) principal diagnosis of pneumonia or (2) principal diagnosis of sepsis or respiratory failure and secondary diagnosis of pneumonia in 2008.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We grouped hospitals into quintiles based on ICU admission rates for pneumonia. We compared rates of failure to deliver pneumonia processes of care (calculated as 100-adherence rate), 30-day mortality, hospital readmissions, and Medicare spending across hospital quintile. After controlling for other hospital characteristics, hospitals in the highest quintile more often failed to deliver pneumonia process measures, including appropriate initial antibiotics (13.0% vs 10.7%; p<0.001), and pneumococcal vaccination (15.0% vs 13.3%; p=0.03) compared with hospitals in quintiles 1-4. Hospitals in the highest quintile of ICU admission rate for pneumonia also had higher 30-day mortality, 30-day hospital readmission rates, and hospital spending per patient than other hospitals.

CONCLUSIONS

Quality of care was lower among hospitals with the highest rates of ICU admission for elderly patients with pneumonia; such hospitals were less likely to deliver pneumonia processes of care and had worse outcomes for patients with pneumonia. High pneumonia-specific ICU admission rates for elderly patients identify a group of hospitals that may deliver inefficient and poor-quality pneumonia care and may benefit from interventions to improve care delivery.

The evolving identity, capacity, and capability of the future surgeon

Technology has transformed surgery more within the last 30 years than the previous 2000 years of human history combined. These innovations have changed not only how the surgeon practices but have also altered the very essence of what it is to be a surgeon in the modern era. Beyond the industrial revolution, today's information revolution allows patients access to an abundance of easily accessible, unfiltered information which they can use to evaluate their surgical treatment, and truly participate in their personal care. We are entering yet another revolution specifically affecting surgeons, where the traditional surgical tools of our craft are becoming "smart." Intelligence in surgical tools and connectivity based on sensory data, processing, and analysis are enabling and enhancing a surgeon's capacity and capability. Given the tempo of change, within one generation the traditional role and identity of a surgeon will be fully transformed. In this article, the impact of the information revolution, technological advances combined with smart connectivity on the changing role of surgery will be considered.

Reliability of surgeon-specific reporting of complications after colectomy

OBJECTIVE

We sought to determine the reliability of surgeon-specific postoperative complication rates after colectomy.

BACKGROUND

Conventional measures of surgeon-specific performance fail to acknowledge variation attributed to statistical noise, risking unreliable assessment of quality.

METHODS

We examined all patients who underwent segmental colectomy with anastomosis from 2008 through 2010 participating in the Michigan Surgical Quality Collaborative Colectomy Project. Surgeon-specific complication rates were risk-adjusted according to patient characteristics with multiple logistic regression. Hierarchical modeling techniques were used to determine the reliability of surgeon-specific risk-adjusted complication rates. We then adjusted these rates for reliability. To evaluate the extent to which surgeon-level variation was reduced, surgeons were placed into quartiles based on performance and complication rates were compared before and after reliability adjustment.

RESULTS

A total of 5033 patients (n = 345 surgeons) undergoing partial colectomy reported a risk-adjusted complication rate of 24.5%. Approximately 86% of the variability of complication rates across surgeons was explained by measurement noise, whereas the remaining 14% represented true signal. Risk-adjusted complication rates varied from 0% to 55.1% across quartiles before adjusting for reliability. Reliability adjustment greatly diminished this variation, generating a 1.2-fold difference (21.4%-25.6%). A caseload of 168 colectomies across 3 years was required to achieve a reliability of more than 0.7, which is considered a proficient level. Only 1 surgeon surpassed this volume threshold.

CONCLUSIONS

The vast majority of surgeons do not perform enough colectomies to generate a reliable surgeon-specific complication rate. Risk-adjusted complication rates should be viewed with caution when evaluating surgeons with low operative volume, as statistical noise is a large determinant in estimating their surgeon-specific complication rates.

Monitoring Obstetric Anesthesia Safety across Hospitals through Multilevel Modeling

Background:

The rate of anesthesia-related adverse events (ARAEs) is recommended for monitoring patient safety across hospitals. To ensure comparability, it is adjusted for patients’ characteristics with logistic models (i.e., risk adjustment). The rate adjusted for patient-level characteristics and hospital affiliation through multilevel modeling is suggested as a better metric. This study aims to assess a multilevel model-based rate of ARAEs.

Methods:

Data were obtained from the State Inpatient Database for New York 2008–2011. Discharge records for labor and delivery and ARAEs were identified with International Classification of Diseases, Ninth Revision, Clinical Modification codes. The rate of ARAEs for each hospital during 2008–2009 was calculated using both the multilevel and the logistic modeling approaches. Performance of the two methods was assessed with (1) interhospital variability measured by the SD of the rates; (2) reclassification of hospitals; and (3) prediction of hospital performance in 2010–2011. Rankability of each hospital was assessed with the multilevel model.

Results:

The study involved 466,442 discharge records in 2008–2009 from 144 hospitals. The overall observed rate of ARAEs in 2008–2009 was 4.62 per 1,000 discharges [95% CI, 4.43 to 4.82]. Compared with risk adjustment, multilevel modeling decreased SD of ARAE rates from 4.7 to 1.3 across hospitals, reduced the proportion of hospitals classified as good performers from 18% to 10%, and performed similarly well in predicting future ARAE rates. Twenty-six hospitals (18%) were nonrankable due to inadequate reliability.

Conclusion:

The multilevel modeling approach could be used as an alternative to risk adjustment in monitoring obstetric anesthesia safety across hospitals.

A primer on using shrinkage to compare in-hospital mortality between centers

Outcomes of cardiothoracic surgery are usually compared among hospitals or physicians by reporting the frequency of in-hospital mortality. Although there is agreement that these frequencies should be adjusted for case mix, there remains uncertainty about the value of using a statistical model that represents hospitals as random effects as opposed to the conventional approach of fixed effects. For years, the Northern New England Cardiovascular Disease Study Group has compared in-hospital mortality after coronary artery bypass graft surgery among centers using a fixed effects approach. An alternative method using random effects has become increasingly popular, and is the method used by cardiothoracic surgery registries such as the Massachusetts Data Analysis Center. The purpose of this report is to provide a short background on fixed versus random effects modeling, describe the use of shrinkage estimators including empirical Bayes, and illustrate them using data from the Northern New England Cardiovascular Disease Study Group. We conclude that both are acceptable approaches to hospital profiling if done in combination with appropriate risk adjustment.

What can the national quality forum tell us about performance measurement in anesthesiology?

BACKGROUND

Anesthesiologists face increasing pressure to demonstrate the value of the care they provide, whether locally or nationally through public reporting and payor requirements. In this article, we describe the current state of performance measurement in anesthesia care at the national level and highlight gaps and opportunities in performance measurement for anesthesiologists.

METHODS

We evaluated all endorsed performance measures in the National Quality Forum (NQF), the clearinghouse for all federal performance measures, and classified all measures as follows: (1) anesthesia-specific; (2) surgery-specific; (3) jointly attributable; or (4) other. We used NQF-provided descriptors to characterize measures in terms of (1) structure, process, outcome, or efficiency; (2) patients, disease, and events targeted; (3) procedural specialty; (4) reporting eligibility; (5) measures stewards; and (6) timing in the care stream.

RESULTS

Of the 637 endorsed performance measures, few (6, 1.0%) were anesthesia-specific. An additional 39 measures (6.1%) were surgery-specific, and 67 others (10.5%) were jointly attributable. "Anesthesia-specific" measures addressed preoperative antibiotic timing (n = 4), normothermia (n = 1), and protocol use for the placement of central venous catheter (n = 1). Jointly attributable measures included outcome measures (n = 49/67, 73.1%), which were weighted toward mortality alone (n = 24) and cardiac surgery (n = 14). Other jointly attributable measures addressed orthopedic surgery (n = 4), general surgical oncologic resections (n = 12), or nonspecified surgeries (n = 15), but none specifically addressed anesthesia care outside the operating room such as for endoscopy. Only 4 measures were eligible for value-based purchasing. No named anesthesiology professional groups were among measure stewards, but surgical professional groups (n = 33/67, 47%) were frequent measure stewards.

CONCLUSIONS

Few NQF performance measures are specific to anesthesia practice, and none of these appears to demonstrate the value of anesthesia care or differentiate high-quality providers. To demonstrate their role in patient-centered, outcome-driven care, anesthesiologists may consider actively partnering in jointly attributable or team-based reporting. Future measures may incorporate surgical procedures not proportionally represented, as well as procedural and sedation care provided in nonoperating room settings.

Bayesian and frequentist approaches to assessing reliability and precision of health-care provider quality measures

Our purpose was to compare frequentist, empirical Bayes, and Bayesian hierarchical model approaches to estimating reliability of health care quality measures, including construction of credible intervals to quantify uncertainty in reliability estimates, using data on inpatient fall rates on hospital nursing units. Precision of reliability estimates and Bayesian approaches to estimating reliability are not well studied. We analyzed falls data from 2372 medical units; the rate of unassisted falls per 1000 inpatient days was the measure of interest. The Bayesian methods “shrunk” the observed fall rates and frequentist reliability estimates toward their posterior means. We examined the association between reliability and precision in fall rate rankings by plotting the length of a 90% credible interval for each unit’s percentile rank against the unit’s estimated reliability. Precision of rank estimates tended to increase as reliability increased but was limited even at higher reliability levels: Among units with reliability >0.8, only 5.5% had credible interval length <20; among units with reliability >0.9, only 31.9% had credible interval length <20. Thus, a high reliability estimate may not be sufficient to ensure precise differentiation among providers. Bayesian approaches allow for assessment of this precision.

Information presentation features and comprehensibility of hospital report cards: design analysis and online survey among users

Background

Improving the transparency of information about the quality of health care providers is one way to improve health care quality. It is assumed that Internet information steers patients toward better-performing health care providers and will motivate providers to improve quality. However, the effect of public reporting on hospital quality is still small. One of the reasons is that users find it difficult to understand the formats in which information is presented.

Objective

We analyzed the presentation of risk-adjusted mortality rate (RAMR) for coronary angiography in the 10 most commonly used German public report cards to analyze the impact of information presentation features on their comprehensibility. We wanted to determine which information presentation features were utilized, were preferred by users, led to better comprehension, and had similar effects to those reported in evidence-based recommendations described in the literature.

Methods

The study consisted of 5 steps: (1) identification of best-practice evidence about the presentation of information on hospital report cards; (2) selection of a single risk-adjusted quality indicator; (3) selection of a sample of designs adopted by German public report cards; (4) identification of the information presentation elements used in public reporting initiatives in Germany; and (5) an online panel completed an online questionnaire that was conducted to determine if respondents were able to identify the hospital with the lowest RAMR and if respondents’ hospital choices were associated with particular information design elements.

Results

Evidence-based recommendations were made relating to the following information presentation features relevant to report cards: evaluative table with symbols, tables without symbols, bar charts, bar charts without symbols, bar charts with symbols, symbols, evaluative word labels, highlighting, order of providers, high values to indicate good performance, explicit statements of whether high or low values indicate good performance, and incomplete data (“N/A” as a value). When investigating the RAMR in a sample of 10 hospitals’ report cards, 7 of these information presentation features were identified. Of these, 5 information presentation features improved comprehensibility in a manner reported previously in literature.

Conclusions

To our knowledge, this is the first study to systematically analyze the most commonly used public reporting card designs used in Germany. Best-practice evidence identified in international literature was in agreement with 5 findings about German report card designs: (1) avoid tables without symbols, (2) include bar charts with symbols, (3) state explicitly whether high or low values indicate good performance or provide a “good quality” range, (4) avoid incomplete data (N/A given as a value), and (5) rank hospitals by performance. However, these findings are preliminary and should be subject of further evaluation. The implementation of 4 of these recommendations should not present insurmountable obstacles. However, ranking hospitals by performance may present substantial difficulties.

Ensuring excellence in centers of excellence programs

Studies have found associations between better outcomes and a variety of structural and process criteria that help explain the wide outcome variations that occur across hospitals. In response, Centers of Excellence programs have been developed by multiple third parties. Despite this, programs have yielded disappointing results and can have unintended consequences. To outweigh potential harms, outcomes at Centers of Excellence must be clearly superior. We need to change how hospitals are designated and provide evidence that Centers of Excellence are truly excellent.

Hospital-level variation in ICU admission and critical care procedures for patients hospitalized for pulmonary embolism

BACKGROUND

Variation in the use of ICUs for low-risk conditions contributes to health system inefficiency. We sought to examine the relationship between ICU use for patients with pulmonary embolism (PE) and cost, mortality, readmission, and procedure use.

METHODS

We performed a retrospective cohort study including 61,249 adults with PE discharged from 263 hospitals in three states between 2007 and 2010. We generated hospital-specific ICU admission rate quartiles and used a series of multilevel models to evaluate relationships between admission rates and risk-adjusted in-hospital mortality, readmission, and costs and between ICU admission rates and several critical care procedures.

RESULTS

Hospital quartiles varied in unadjusted ICU admission rates for PE (range, ≤ 15% to > 31%). Among all patients, there was a small trend toward increased use of arterial catheterization (0.6%-1.1%, P < .01) in hospital quartiles with higher levels of ICU admission. However, use of invasive mechanical ventilation (14.4%-7.9%, P < .01), noninvasive ventilation (6.6%-3.0%, P < .01), central venous catheterization (14.6%-11.3%, P < .02), and thrombolytics (11.0%-4.7%, P < .01) in patients in the ICU declined across hospital quartiles. There was no relationship between ICU admission rate and risk-adjusted hospital mortality, costs, or readmission.

CONCLUSIONS

Hospitals vary widely in ICU admission rates for acute PE without a detectable impact on mortality, cost, or readmission. Patients admitted to ICUs in higher-using hospitals received many critical care procedures less often, suggesting that these patients may have had weaker indications for ICU admission. Hospitals with greater ICU admission may be appropriate targets for improving efficiency in ICU admissions.

Variation in hospital treatment patterns for metastatic colorectal cancer

BACKGROUND

There are many treatment options for metastatic colorectal cancer (CRC). However, to the authors' knowledge, national treatment patterns for metastatic CRC, and the stability of hospital treatment patterns over time, have not been well described.

METHODS

Data from the 2006 through 2011 National Cancer Data Base were used to study adults with newly diagnosed metastatic CRC (84,161 patients from 1051 hospitals). Using hierarchical models, the authors characterized hospital volume in the use of different treatment modalities (primary site resection, metastatic site resection, chemotherapy, and palliative care). The authors then assessed variation in the receipt of treatment according to the hospitals' relative volume of services used. Finally, the extent to which hospital treatment patterns changed over the past decade was examined.

RESULTS

Overall use of volume of services varied widely (5.0% in the hospitals with low volumes of service to 22.3% in the hospitals with high volumes of service). As hospitals' volumes of services increased, adjusted rates of metastatic site surgery (6.6% to 30.8%; P<.001) and multiagent chemotherapy (37.8% to 57.4%; P<.001) use increased, but primary site resection demonstrated little variation (56.8% vs 59.5%; P = .024). It is interesting to note that use of palliative care also increased (8.1% to 11.3%; P = .002). Hospital treatment patterns did not change over time, with hospitals with high volumes of service consistently using more metastatic site resection and multiagent chemotherapy than hospitals with low volumes of service.

CONCLUSIONS

There is wide variation in hospital treatment patterns for patients with metastatic CRC, and these patterns have been stable over time. It appears that much of the approach for metastatic CRC treatment depends on the hospital in which the patient presents.

Management strategies to effect change in intensive care units: lessons from the world of business. Part I. Targeting quality improvement initiatives

The business community has developed strategies to ensure the quality of the goods or services they produce and to improve the management of multidisciplinary work teams. With modification, many of these techniques can be imported into intensive care units (ICUs) to improve clinical operations and patient safety. In Part I of a three-part ATS Seminar series, we argue for adopting business management strategies in ICUs and set forth strategies for targeting selected quality improvement initiatives. These tools are relevant to health care today as focus is placed on limiting low-value care and measuring, reporting, and improving quality. In the ICU, the complexity of illness and the need to standardize processes make these tools even more appealing. Herein, we highlight four techniques to help prioritize initiatives. First, the "80/20 rule" mandates focus on the few (20%) interventions likely to drive the majority (80%) of improvement. Second, benchmarking--a process of comparison with peer units or institutions--is essential to identifying areas of strength and weakness. Third, root cause analyses, in which structured retrospective reviews of negative events are performed, can be used to identify and fix systems errors. Finally, failure mode and effects analysis--a process aimed at prospectively identifying potential sources of error--allows for systems fixes to be instituted in advance to prevent negative outcomes. These techniques originated in fields other than health care, yet adoption has and can help ICU managers prioritize issues for quality improvement.

HIV quality report cards: impact of case-mix adjustment and statistical methods

BACKGROUND

There will be increasing pressure to publicly report and rank the performance of healthcare systems on human immunodeficiency virus (HIV) quality measures. To inform discussion of public reporting, we evaluated the influence of case-mix adjustment when ranking individual care systems on the viral control quality measure.

METHODS

We used data from the Veterans Health Administration (VHA) HIV Clinical Case Registry and administrative databases to estimate case-mix adjusted viral control for 91 local systems caring for 12 368 patients. We compared results using 2 adjustment methods, the observed-to-expected estimator and the risk-standardized ratio.

RESULTS

Overall, 10 913 patients (88.2%) achieved viral control (viral load ≤400 copies/mL). Prior to case-mix adjustment, system-level viral control ranged from 51% to 100%. Seventeen (19%) systems were labeled as low outliers (performance significantly below the overall mean) and 11 (12%) as high outliers. Adjustment for case mix (patient demographics, comorbidity, CD4 nadir, time on therapy, and income from VHA administrative databases) reduced the number of low outliers by approximately one-third, but results differed by method. The adjustment model had moderate discrimination (c statistic = 0.66), suggesting potential for unadjusted risk when using administrative data to measure case mix.

CONCLUSIONS

Case-mix adjustment affects rankings of care systems on the viral control quality measure. Given the sensitivity of rankings to selection of case-mix adjustment methods-and potential for unadjusted risk when using variables limited to current administrative databases-the HIV care community should explore optimal methods for case-mix adjustment before moving forward with public reporting.

Reliability of surgical outcomes for predicting future hospital performance

BACKGROUND

Because of small sample sizes and low event rates, risk-adjusted surgical outcomes often do not meet reliability benchmarks for distinguishing hospital performance. Nonetheless, it is unclear whether these measures may still be useful for predicting future hospital surgical performance.

METHODS

We used national Medicare data to analyze patients undergoing colectomy from 2007 to 2010 (n=462,959 patients). We first quantified 2007-2008 outcome reliability (ability to differentiate quality differences) and ranked hospitals based on their 2007-2008 risk-adjusted outcome rates. To assess the ability of adjusted outcomes to predict true performance, we evaluated future (2009-2010) outcomes across quintiles of past performance. We then systematically sampled 2007-2008 cases to evaluate performance prediction when hospitals' past performance was measured with progressively lower reliability levels.

RESULTS

Outcomes in 2007-2008 were good predictors of outcomes in the next 2 years (2009-2010), but predictive strength depended upon reliability. With progressive sampling of 2007-2008 caseloads, outcome reliability and predictive strength decreased. With 100% sampling of 2007-2008 caseloads, the worst versus best hospital quintile based on past performance had 1.52 [95% confidence interval (CI), 1.44-1.60] times the odds of mortality and 1.50 (95% CI, 1.44-1.56) times the odds of complications in 2009-2010. With 10% sampling, outcome reliability was well below commonly accepted benchmarks, but the worst quintile of hospitals in 2007-2008 still had 1.12 (95% CI, 1.06-1.19) times the odds of mortality and 1.16 (95% CI, 1.11-1.21) times the odds of complications in 2009-2010 compared with the best quintile of hospitals.

CONCLUSIONS

Even at very low reliability levels, risk-adjusted outcome measures may distinguish best and worst hospitals' surgical performance. This study suggests that commonly accepted reliability thresholds may be too high, especially in the context of selective referral.

Profiling hospitals on bariatric surgery quality: which outcomes are most reliable?

BACKGROUND

Under the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program, hospitals will receive risk-adjusted outcomes feedback for peer comparisons and benchmarking. It remains uncertain whether bariatric outcomes have adequate reliability to identify outlying performance, especially for hospitals with low caseloads that will be included in the program. We explored the ability of risk-adjusted outcomes to identify outlying hospital performance with bariatric surgery for a range of hospital caseloads.

STUDY DESIGN

We used the 2010 State Inpatient Databases for 12 states (N = 31,240 patients) to assess different outcomes (eg, complications, reoperation, and mortality) after bariatric stapling procedures. We first quantified outcomes reliability on a 0 (no reliability) to 1 (perfect reliability) scale. We then assessed whether risk- and reliability-adjusted outcomes could identify outlying performance among hospitals with different annual caseloads.

RESULTS

Overall and serious complications had the highest overall reliability, but this was heavily dependent on caseload. For example, among hospitals with the lowest caseloads (mean 56 cases/year), reliability for overall complications was 0.49 and 6.0% of hospitals had outlying performance. For hospitals with the highest caseloads (mean 298 cases/year), reliability for overall complications was 0.79 and 30.3% of hospitals had outlying performance. Reoperation had adequate reliability for hospitals with caseloads higher than 120 cases/year. Mortality had unacceptably low reliability regardless of hospital caseloads.

CONCLUSIONS

Overall complications and serious complications have adequate reliability for distinguishing outlying performance with bariatric surgery, even for hospitals with low annual caseloads. Rare outcomes, such as reoperations, have inadequate reliability to inform peer-based comparisons for hospitals with low annual caseloads, and mortality has unacceptably low reliability for bariatric performance profiling.

Ranking and rankability of hospital postoperative mortality rates in colorectal cancer surgery

OBJECTIVES

To examine to what extent random variation and variation in case-mix influence hospital rankings on the basis of mortality rates and to determine the suitability of mortality for ranking hospitals in colorectal surgery.

BACKGROUND

Comparing and ranking postoperative mortality rates between hospitals becomes increasingly popular. Differences in hospital case-mix, and chance variation related to caseload, may influence rankings. The suitability of mortality for rankings remains unclear.

METHODS

Data were derived from the Dutch Surgical Colorectal Audit. Hospital rankings based on fixed- and random-effects logistic regression models, unadjusted and adjusted for case-mix were compared with the percentile based on expected ranks (the chance that a hospital performs better than a random hospital). Rankability, measuring which part of variation between hospitals is not due to chance, was calculated.

RESULTS

Some 25,591 patients undergoing colorectal resections in 92 hospitals were evaluated. Postoperative mortality rates ranged between 0% and 8.8%. Adjustment for case-mix with a fixed-effects model caused large changes in rankings. A smaller additional effect on changes in rankings occurred after adjusting with a random-effects model, with lower volume hospitals moving toward the mean. Percentile based on expected ranks ranged between 10% and 85%. Rankability was 38%, meaning that 62% of hospital variation in mortality was due to chance.

CONCLUSIONS

Hospital ranks changed after case-mix adjustment and random-effects models, compared with unadjusted analysis. A large proportion of hospital variation in mortality was due to chance. Caution should be warranted when interpreting hospital rankings on the basis of postoperative mortality. Percentiles of expected ranks may help identify hospitals with exceptional performance.

Using same-hospital readmission rates to estimate all-hospital readmission rates

BACKGROUND

Since October of 2012, Medicare's Hospital Readmissions Reduction Program has fined 2,200 hospitals a total of $500 million. Although the program penalizes readmission to any hospital, many institutions can only track readmissions to their own hospitals. We sought to determine the extent to which same-hospital readmission rates can be used to estimate all-hospital readmission rates after major surgery.

STUDY DESIGN

We evaluated 3,940 hospitals treating 741,656 Medicare fee-for-service beneficiaries undergoing CABG, hip fracture repair, or colectomy between 2006 and 2008. We used hierarchical logistic regression to calculate risk- and reliability-adjusted rates of 30-day readmission to the same hospital and to any hospital. We next evaluated the correlation between same-hospital and all-hospital rates. To analyze the impact on hospital profiling, we compared rankings based on same-hospital rates with those based on all-hospital rates.

RESULTS

The mean risk- and reliability-adjusted all-hospital readmission rate was 13.2% (SD 1.5%) and mean same-hospital readmission rate was 8.4% (SD 1.1%). Depending on the operation, between 57% (colectomy) and 63% (CABG) of hospitals were reclassified when profiling was based on same-hospital readmission rates instead of on all-hospital readmission rates. This was particularly pronounced in the middle 3 quintiles, where 66% to 73% of hospitals were reclassified.

CONCLUSIONS

In evaluating hospital profiling under Medicare's Hospital Readmissions Reduction Program, same-hospital rates provide unstable estimates of all-hospital readmission rates. To better anticipate penalties, hospitals require novel approaches for accurately tracking the totality of their postoperative readmissions.

Hospital costs and inpatient mortality among children undergoing surgery for congenital heart disease

OBJECTIVE

To determine the association between hospital costs and risk-adjusted inpatient mortality among children undergoing surgery for congenital heart disease (CHD) in U.S. acute-care hospitals.

DATA SOURCES/STUDY SETTINGS

Retrospective cohort study of 35,446 children in 2003, 2006, and 2009 Kids' Inpatient Database (KID).

STUDY DESIGN

Cross-sectional logistic regression of risk-adjusted inpatient mortality and hospital costs, adjusting for a variety of patient-, hospital-, and community-level confounders.

DATA COLLECTION/EXTRACTION METHODS

We identified relevant discharges in the KID using the AHRQ Pediatric Quality Indicator for pediatric heart surgery mortality, and linked these records to hospital characteristics from American Hospital Association Surveys and community characteristics from the Census.

PRINCIPAL FINDINGS

Children undergoing CHD surgery in higher cost hospitals had lower risk-adjusted inpatient mortality (p=.002). An increase from the 25th percentile of treatment costs to the 75th percentile was associated with a 13.6 percent reduction in risk-adjusted mortality.

CONCLUSIONS

Greater hospital costs are associated with lower risk-adjusted inpatient mortality for children undergoing CHD surgery. The specific mechanisms by which greater costs improve mortality merit further exploration.

Composite measures for profiling hospitals on bariatric surgery performance

IMPORTANCE

The optimal approach for profiling hospital performance with bariatric surgery is unclear.

OBJECTIVE

To develop a novel composite measure for profiling hospital performance with bariatric surgery.

DESIGN, SETTING, AND PARTICIPANTS

Using clinical registry data from the Michigan Bariatric Surgery Collaborative, we studied all patients undergoing bariatric surgery from January 1, 2008, through December 31, 2010. For laparoscopic gastric bypass surgery, we used empirical Bayes techniques to create a composite measure by combining several measures, including serious complications, reoperations, and readmissions; hospital and surgeon volume; and outcomes with other related procedures. Hospitals were ranked for 2008 through 2009 and placed in 1 of 3 groups: 3-star (top 20%), 2-star (middle 60%), and 1-star (bottom 20%). We assessed how well these ratings predicted outcomes in the next year (2010) compared with other widely used measures.

MAIN OUTCOMES AND MEASURES

Risk-adjusted serious complications.

RESULTS

Composite measures explained a larger proportion of hospital-level variation in serious complication rates with laparoscopic gastric bypass than other measures. For example, the composite measure explained 89% of the variation compared with only 28% for risk-adjusted complication rates alone. Composite measures also appeared better at predicting future performance compared with individual measures. When ranked on the composite measure, 1-star hospitals had 2-fold higher serious complication rates (4.6% vs 2.4%; odds ratio, 2.0; 95% CI, 1.1-3.5) compared with 3-star hospitals. Differences in serious complication rates between 1- and 3-star hospitals were much smaller when hospitals were ranked using serious complications (4.0% vs 2.7%; odds ratio, 1.6; 95% CI, 0.8-2.9) and hospital volume (3.3% vs 3.2%; odds ratio, 0.85; 95% CI, 0.4-1.7).

CONCLUSIONS AND RELEVANCE

Composite measures are much better at explaining hospital-level variation in serious complications and predicting future performance than other approaches. In this preliminary study, it appears that such composite measures may be better than existing alternatives for profiling hospital performance with bariatric surgery.

Reliability of hospital readmission rates in vascular surgery

OBJECTIVE

The Center for Medicare and Medicaid Services recently began assessing financial penalties to hospitals with high readmission rates for a narrow set of medical conditions. Because these penalties will be extended to surgical conditions in the near future, we sought to determine whether readmissions are a reliable predictor of hospital performance with vascular surgery.

METHODS

We examined 4 years of national Medicare claims data from 1576 hospitals on beneficiaries undergoing three common vascular procedures: open or endovascular abdominal aortic aneurysm repair (n = 81,520) or lower extremity arterial bypass (n = 57,190). First, we divided our population into two groups on the basis of operative date (2005-2006 and 2007-2008) and generated hospital risk- and reliability-adjusted readmission rates for each time period. We evaluated reliability through the use of the "test-retest" method; highly reliable measures will show little variation in rates over time. Specifically, we evaluated the year-to-year reliability of readmissions by calculating Spearman rank correlation and weighted κ tests for readmission rates between the two time periods.

RESULTS

The Spearman coefficient between 2005-2006 readmissions rankings and 2007-2008 readmissions rankings was 0.57 (P < .001) and weighted κ was 0.42 (P < .001), indicating a moderate correlation. However, only 32% of the variation in hospital readmission rates in 2007-2008 was explained by readmissions during the 2 prior years. There were major reclassifications of hospital rankings between years, with 63% of hospitals migrating among performance quintiles between 2005-2006 and 2007-2008.

CONCLUSIONS

Risk-adjusted readmission rates for vascular surgery vary substantially year to year; this implies that much of the observed variation in readmission rates is either random or caused by unmeasured factors and not caused by changes in hospital quality that may be captured by administrative data.

Variation in congenital heart surgery costs across hospitals

BACKGROUND

A better understanding of costs associated with common and resource-intense conditions such as congenital heart disease has become increasingly important as children's hospitals face growing pressure to both improve quality and reduce costs. We linked clinical information from a large registry with resource utilization data from an administrative data set to describe costs for common congenital cardiac operations and assess variation across hospitals.

METHODS

Using linked data from The Society of Thoracic Surgeons and Pediatric Health Information Systems Databases (2006-2010), estimated costs/case for 9 operations of varying complexity were calculated. Between-hospital variation in cost and associated factors were assessed by using Bayesian methods, adjusting for important patient characteristics.

RESULTS

Of 12,718 operations (27 hospitals) included, median cost/case increased with operation complexity (atrial septal defect repair, [$25,499] to Norwood operation, [$165,168]). Significant between-hospital variation (up to ninefold) in adjusted cost was observed across operations. Differences in length of stay (LOS) and complication rates explained an average of 28% of between-hospital cost variation. For the Norwood operation, high versus low cost hospitals had an average LOS of 50.8 vs. 31.8 days and a major complication rate of 50% vs. 25.3%. High volume hospitals had lower costs for the most complex operations.

CONCLUSIONS

This study establishes benchmarks for hospital costs for common congenital heart operations and demonstrates wide variability across hospitals related in part to differences in LOS and complication rates. These data may be useful in designing initiatives aimed at both improving quality of care and reducing cost.