Ranking hospitals on surgical mortality: the importance of reliability adjustment

Reliability of readmission rates as a hospital quality measure in cardiac surgery

BACKGROUND

Recent policy interventions have reduced payments to hospitals with higher-than-predicted risk-adjusted readmission rates. However, whether readmission rates reliably discriminate deficiencies in hospital quality is uncertain. We sought to determine the reliability of 30-day readmission rates after cardiac operations as a measure of hospital performance and evaluate the effect of hospital caseload on reliability.

METHODS

We examined national Medicare beneficiaries undergoing coronary artery bypass graft operations for 2006 to 2008 (n=244,874 patients, n=1,210 hospitals). First, we performed multivariable logistic regression examining patient factors to calculate a risk-adjusted readmission rate for each hospital. We then used hierarchical modeling to estimate the reliability of this quality measure for each hospital. Finally, we determined the proportion of total variation attributable to three factors: true signal, statistical noise, and patient factors.

RESULTS

A median of 151 (25% to 75% interquartile range, 79 to 265) coronary artery bypasses were performed per hospital during the 3-year period. The median risk-adjusted 30-day readmission rate was 17.6% (25% to 75% interquartile range, 14.4% to 20.8%). Of the variation in readmission rates, 55% was explained by measurement noise, 4% could be attributed to patient characteristics, and the remaining 41% represented true signal in readmission rates. Only 53 hospitals (4.4%) achieved a proficient level of reliability exceeding 0.70. To achieve this reliability, 599 cases were required during the 3-year period. In 33.7% of hospitals, a moderate degree of reliability exceeding 0.5 was achieved, which required 218 cases.

CONCLUSIONS

The vast majority of hospitals do not achieve a minimum acceptable level of reliability for 30-day readmission rates. Despite recent enthusiasm, readmission rates are not a reliable measure of hospital quality in cardiac surgery.

Insurer market structure and variation in commercial health care spending

OBJECTIVE

To examine the relationship between insurance market structure and health care prices, utilization, and spending.

DATA SOURCES

Claims for 37.6 million privately insured employees and their dependents from the Truven Health Market Scan Database in 2009. Measures of insurer market structure derived from Health Leaders Inter study data.

METHODS

Regression models are used to estimate the association between insurance market concentration and health care spending, utilization, and price, adjusting for differences in patient characteristics and other market-level traits.

RESULTS

Insurance market concentration is inversely related to prices and spending, but positively related to utilization. Our results imply that, after adjusting for input price differences, a market with two equal size insurers is associated with 3.9 percent lower medical care spending per capita (p = .002) and 5.0 percent lower prices for health care services relative to one with three equal size insurers (p < .001).

CONCLUSION

Greater fragmentation in the insurance market might lead to higher prices and higher spending for care, suggesting some of the gains from insurer competition may be absorbed by higher prices for health care. Greater attention to prices and utilization in the provider market may need to accompany procompetitive insurance market strategies.

Public reporting of surgeon outcomes: low numbers of procedures lead to false complacency

The English National Health Service published outcome information for individual surgeons for ten specialties in June, 2013. We looked at whether individual surgeons do sufficient numbers of procedures to be able to reliably identify those with poor performance. For some specialties, the number of procedures that a surgeon does each year is low and, as a result, the chance of identifying a surgeon with increased mortality rates is also low. Therefore, public reporting of individual surgeons' outcomes could lead to false complacency. We recommend use of outcomes that are fairly frequent, considering the hospital as the unit of reporting when numbers are low, and avoiding interpretation of no evidence of poor performance as evidence of acceptable performance.

Perioperative mechanical circulatory support in children: an analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database

OBJECTIVES

Analyses of mechanical circulatory support (MCS) in pediatric heart surgery have primarily focused on single-center outcomes or narrow applications. We describe the patterns of use, patient characteristics, and MCS-associated outcomes across a large multicenter cohort.

METHODS

Patients (aged <18 years) in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (2000-2010) were included. The characteristics and outcomes of those receiving postoperative MCS were described, and bayesian hierarchical models were used to examine variations in the adjusted MCS rates across institutions.

RESULTS

Of 96,596 operations (80 centers), MCS was used in 2.4%. The MCS patients were younger (13 vs 195 days, P < .0001) and more often had STS-defined preoperative risk factors (57.2% vs 32.7%, P < .0001). The operations with the greatest MCS rates included the Norwood procedure (17%) and complex biventricular repairs (arterial switch, ventricular septal defect, and arch repair [14%]). More than one half of the MCS patients did not survive to hospital discharge (53.2% vs 2.9% of non-MCS patients; P < .0001). MCS-associated mortality was greatest for truncus arteriosus and Ross-Konno operations (both 71%). The hospital-level MCS rates adjusted for patient characteristics and case mix varied by 15-fold across institutions, with both high- and low-volume hospitals having substantial variation in MCS rates.

CONCLUSIONS

Perioperative MCS use varied widely across centers. The MCS rates were greatest overall for the Norwood procedure and complex biventricular repairs. Although MCS can be a life-saving therapy, more than one half of MCS patients will not survive to hospital discharge, with mortality >70% for some operations. Future studies aimed at better understanding the appropriate indications, optimal timing, and management of MCS could help to reduce the variation in MCS use across hospitals and improve outcomes.

The volume effect in liver surgery--a systematic review and meta-analysis

BACKGROUND

There is an inverse relationship between hospital and surgeon volume and mortality in many types of complex surgery. The aim of this paper is to investigate the volume effect on outcomes of liver surgery.

METHODS

A systematic review and meta-analysis was performed. A literature search was conducted using Medline and EMBASE from 1995 to 2012. A random effects model was used.

RESULTS

Seventeen studies were selected for detailed analysis. Definition of a high-volume institution varied from 2 to more than 33 procedures per year. The pooled odds ratio of mortality rate in low- vs high-volume centres was 2.0 [95 % confidence interval (CI), 1.6-2.4; P < 0.001]. Some studies divided centres into more than two groups and compared the highest and lowest volume groups. The pooled odds ratio of mortality rate for this comparison type was 3.2 (95 % CI, 1.7-5.8; P < 0.001). Funnel plots suggest possible publication bias. There was inadequate data to compare morbidity. Only two of seven studies demonstrated a shorter length of stay in the high-volume centres. There was no convincing volume effect on long-term survival.

CONCLUSIONS

This study suggests a strong relationship between volume and perioperative mortality. No difference in morbidity, length of stay or survival was demonstrated.

Hospital variation in postoperative infection and outcome after congenital heart surgery

BACKGROUND

Several initiatives aim to reduce postoperative infection across a variety of surgical patients as a means to improve overall quality of care and reduce variation across centers. However, the association of infection rates with hospital-level outcomes and resource utilization has not been well described. We evaluated this association across a multicenter cohort undergoing congenital heart surgery.

METHODS

The Society of Thoracic Surgeons Congenital Heart Surgery Database was linked to resource utilization data from the Pediatric Health Information Systems Database for hospitals participating in both (2006 to 2010). Hospital-level infection rates (sepsis, wound infection, mediastinitis, endocarditis, pneumonia) adjusted for patient risk factors and case mix were calculated using Bayesian methodology, and association with hospital mortality rates, postoperative length of stay (LOS), and total costs evaluated.

RESULTS

The cohort included 32,856 patients (28 centers); 3.7% had a postoperative infection. Across hospitals, the adjusted infection rate varied from 0.9% to 9.8%. Hospitals with the highest infection rates had longer (LOS) (13.2 vs 11.7 days, p < 0.001) and increased hospital costs ($71,100 vs $65,100, p < 0.001), but similar mortality rates (odds ratio 0.99, 95% confidence interval 0.80 to 1.21, p = 0.9). The proportion of variation in costs and LOS explained by infection was 15% and 6%, respectively.

CONCLUSIONS

Infection after congenital heart surgery contributes to prolonged LOS and increased costs on a hospital level. However, given that infection rates alone explained relatively little of the variation in these outcomes across hospitals, further study is needed to identify additional factors that may be targeted in initiatives to reduce variation and improve outcomes across centers.

Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) collects detailed clinical data from participating hospitals using standardized data definitions, analyzes these data, and provides participating hospitals with reports that permit risk-adjusted comparisons with a surgical quality standard. Since its inception, the ACS NSQIP has worked to refine surgical outcomes measurements and enhance statistical methods to improve the reliability and validity of this hospital profiling. From an original focus on controlling for between-hospital differences in patient risk factors with logistic regression, ACS NSQIP has added a variable to better adjust for the complexity and risk profile of surgical procedures (procedure mix adjustment) and stabilized estimates derived from small samples by using a hierarchical model with shrinkage adjustment. New models have been developed focusing on specific surgical procedures (eg, "Procedure Targeted" models), which provide opportunities to incorporate indication and other procedure-specific variables and outcomes to improve risk adjustment. In addition, comparative benchmark reports given to participating hospitals have been expanded considerably to allow more detailed evaluations of performance. Finally, procedures have been developed to estimate surgical risk for individual patients. This article describes the development of, and justification for, these new statistical methods and reporting strategies in ACS NSQIP.

Target setting for elective infra-renal AAA surgery: A single mortality figure?

OBJECTIVES

One of the key standards set by the UK NAAASP is that centres performing elective abdominal aortic aneurysm (AAA) repair have a mortality rate of <6%. In light of this, and the current aim to reduce elective AAA repair mortality to 3.5% by 2013, we sought to investigate the statistical validity of such targets.

METHODS

The National Vascular Database (NVD) was interrogated and the degree of AAA missing data and its geographical variation is described. Utilising published data from 2006 to 2008 a funnel plot was used to illustrate NHS Trust level data for current estimates of mortality rate. A binomial distribution model was applied to calculate variation in observed mortality rates in relation to number of patients treated, based on a "true" mortality rate of 3.5%. Funnel plots were constructed using simulated data-sets for units performing 10, 30, 50, 100 or 150 procedures annually with control-limits calculated using a cumulative probability distribution. Finally the effect of case-mix on mortality was modelled and shown graphically.

RESULTS

The NVD AAA data set shows a range of data missingness across variables (median 22%, IQR 10-64%). High levels of missingness typically coincide with non-required, non-preferred variables however this is subject to geographical variation. Funnel plots of simulated data demonstrate that smaller units have greater variability in 3-year mortality (range 0.0-10.0%) than the largest units performing 150 procedures annually (1.3-5.6%). Around 20% of NVD variables are described as "preferred", these typically relate to clinical measurements and patient medications and would inform any risk model of mortality. Data missingness amongst these variables ranges from 5 to 50%.

CONCLUSIONS

There are many problems with the use of a single mortality figure to assess performance. These include the natural statistical variability and the means by which "case-mix" is taken into consideration. This article calls for further research into mortality target setting and suggests strategies which may help provide solutions nationally and facilitate international comparison.

Severe sepsis in pre-hospital emergency care: analysis of incidence, care, and outcome

RATIONALE

Severe sepsis is common and highly morbid, yet the epidemiology of severe sepsis at the frontier of the health care system-pre-hospital emergency care-is unknown.

OBJECTIVES

We examined the epidemiology of pre-hospital severe sepsis among emergency medical services (EMS) encounters, relative to acute myocardial infarction and stroke.

METHODS

Retrospective study using a community-based cohort of all nonarrest, nontrauma King County EMS encounters from 2000 to 2009 who were transported to a hospital.

MEASUREMENTS AND MAIN RESULTS

Overall incidence rate of hospitalization with severe sepsis among EMS encounters, as well as pre-hospital characteristics, admission diagnosis, and outcomes. Among 407,176 EMS encounters, we identified 13,249 hospitalizations for severe sepsis, of whom 2,596 died in the hospital (19.6%). The crude incidence rate of severe sepsis was 3.3 per 100 EMS encounters, greater than for acute myocardial infarction or stroke (2.3 per 100 and 2.2 per 100 EMS encounters, respectively). More than 40% of all severe sepsis hospitalizations arrived at the emergency department after EMS transport, and 80% of cases were diagnosed on admission. Pre-hospital care intervals, on average, exceeded 45 minutes for those hospitalized with severe sepsis. One-half or fewer of patients with severe sepsis were transported by paramedics (n = 7,114; 54%) or received pre-hospital intravenous access (n = 4,842; 37%).

CONCLUSIONS

EMS personnel care for a substantial and increasing number of patients with severe sepsis, and spend considerable time on scene and during transport. Given the emphasis on rapid diagnosis and intervention for sepsis, the pre-hospital interval may represent an important opportunity for recognition and care of sepsis.

Hospital-level variation in the use of intensive care

OBJECTIVE

To determine the extent to which hospitals vary in the use of intensive care, and the proportion of variation attributable to differences in hospital practice that is independent of known patient and hospital factors.

DATA SOURCE

Hospital discharge data in the State Inpatient Database for Maryland and Washington States in 2006.

STUDY DESIGN

Cross-sectional analysis of 90 short-term, acute care hospitals with critical care capabilities. DATA COLLECTION/METHODS: We quantified the proportion of variation in intensive care use attributable to hospitals using intraclass correlation coefficients derived from mixed-effects logistic regression models after successive adjustment for known patient and hospital factors.

PRINCIPAL FINDINGS

The proportion of hospitalized patients admitted to an intensive care unit (ICU) across hospitals ranged from 3 to 55 percent (median 12 percent; IQR: 9, 17 percent). After adjustment for patient factors, 19.7 percent (95 percent CI: 15.1, 24.4) of total variation in ICU use across hospitals was attributable to hospitals. When observed hospital characteristics were added, the proportion of total variation in intensive care use attributable to unmeasured hospital factors decreased by 26-14.6 percent (95 percent CI: 11, 18.3 percent).

CONCLUSIONS

Wide variability exists in the use of intensive care across hospitals, not attributable to known patient or hospital factors, and may be a target to improve efficiency and quality of critical care.

Variation in use of intensive care for adults with diabetic ketoacidosis*

OBJECTIVE

Intensive care unit beds are limited, yet few guidelines exist for triage of patients to the intensive care unit, especially patients at low risk for mortality. The frequency with which low-risk patients are admitted to intensive care units in different hospitals is unknown. Our objective was to assess variation in the use of intensive care for patients with diabetic ketoacidosis, a common condition with a low risk of mortality.

DESIGN

Observational study using the New York State In-patient Database (2005-2007).

SETTING

One hundred fifty-nine New York State acute care hospitals.

PATIENTS

Fifteen thousand nine hundred ninety-four adult (≥ 18) hospital admissions with a primary diagnosis of diabetic ketoacidosis (International Classification of Diseases, Ninth Revision, Clinical Modification 250.1x).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We calculated reliability- and risk-adjusted intensive care unit utilization, hospital length of stay, and mortality. We identified hospital-level factors associated with increased likelihood of intensive care unit admission after controlling patient characteristics using multilevel, mixed-effects logistic regression analyses; we assessed the amount of residual variation in intensive care unit utilization using the intraclass correlation coefficient. Use of intensive care for diabetic ketoacidosis patients varied widely across hospitals (adjusted range: 2.1% to 87.7%), but was not associated with hospital length of stay or mortality. After multilevel adjustment, hospitals with a high volume of diabetic ketoacidosis admissions admitted diabetic ketoacidosis patients to the intensive care unit less often (odds ratio 0.40, p = .002, highest quintile compared to lowest), whereas hospitals with higher rates of intensive care unit utilization for all nondiabetic ketoacidosis in-patients admitted diabetic ketoacidosis patients to the intensive care unit more frequently (odds ratio 1.31, p = .001, for each additional 10% increase). In the multilevel model, more than half (58%) of the variation in the intensive care unit admission practice attributable to hospitals remained unexplained.

CONCLUSIONS

We observed variations across hospitals in the use of intensive care for diabetic ketoacidosis patients that was not associated with differences in-hospital length of stay or mortality. Institutional practice patterns appear to impact admission decisions and represent a potential target for reduction of resource utilization in higher use institutions.

Reliability adjustment for reporting hospital outcomes with surgery

BACKGROUND

Reliability adjustment, a novel technique for quantifying and removing statistical "noise" from quality rankings, is becoming more widely used outside surgery. We sought to evaluate its impact on hospital outcomes assessed with the American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP).

METHODS

We used prospective, clinical data from the ACS-NSQIP to identify all patients undergoing colon resection in 2007 (n = 181 hospitals, n = 18,455 patients). We first used standard NSQIP techniques to generate risk-adjusted mortality and morbidity rates for each hospital. Using hierarchical logistic regression models, we then adjusted these for reliability using empirical Bayes techniques. To evaluate the impact of reliability adjustment, we first estimated the extent to which hospital-level variation was reduced. We then compared hospital mortality and morbidity rankings and outlier status before and after reliability adjustment.

RESULTS

Reliability adjustment greatly diminished apparent variation in hospital outcomes. For risk-adjusted mortality, there was a 6-fold difference before (1.4%-7.8%) and less than a 2-fold difference (3.2% to 5.7%) after reliability adjustment. For risk-adjusted morbidity, there was a 2-fold difference (18.0%-38.2%) before and a 1.5-fold difference (20.8%-34.8%) after reliability adjustment. Reliability adjustment had a large impact on hospital mortality and morbidity rankings. For example, with rankings based on mortality, 44% (16 hospitals) of the "best" hospitals (top 20%) were reclassified after reliability adjustment. Similarly, 22% (8 hospitals) of the "worst" hospitals (bottom 20%) were reclassified after reliability adjustment.

CONCLUSIONS

Reliability adjustment reduces variation due to statistical noise and results in more accurate estimates of risk-adjusted hospital outcomes. Given the risk of misclassifying hospitals and surgeons using standard approaches, this technique should be considered when reporting surgical outcomes.

Evaluation of failure to rescue as a quality metric in pediatric heart surgery: an analysis of the STS Congenital Heart Surgery Database

BACKGROUND

Failure to rescue (FTR; the probability of death after a complication) has been adopted as a quality metric in adult cardiac surgery, in which it has been shown that high-performing centers with low mortality rates do not have fewer complications, but rather lower mortality in those who experience a complication (lower FTR). It is unknown whether this holds true in pediatric heart surgery. We characterized the relationship between complications, FTR, and mortality in this population.

METHODS

Children (0 to 18 years) undergoing heart surgery at centers participating in the Society of Thoracic Surgeons Congenital Heart Surgery Database (2006 to 2009) were included. Outcomes were examined in multivariable analysis adjusting for patient characteristics, surgical risk category, and within-center clustering.

RESULTS

This study included 40,930 patients from 72 centers. Overall in-hospital mortality was 3.7%, 39.3% had a postoperative complication, and the FTR rate (number of deaths in those with a complication) was 9.1%. When hospitals were characterized by in-hospital mortality rate, there was no difference across hospital mortality tertiles in the complication rate in adjusted analysis; however, hospitals in the lowest mortality tertile had significantly lower FTR rates (6.6% versus 12.4%; p<0.0001). Similar results were seen when evaluating high-severity complications and across surgical risk groups.

CONCLUSIONS

This analysis suggests that hospitals with low mortality rates do not have fewer complications after pediatric heart surgery, but instead have lower mortality in those who experience a complication (lower FTR). Further investigation into FTR as a quality metric in pediatric heart surgery is warranted.

Composite measures for rating hospital quality with major surgery

OBJECTIVE

To assess the value of a novel composite measure for identifying the best hospitals for major procedures.

DATA SOURCE

We used national Medicare data for patients undergoing five high-risk surgical procedures between 2005 and 2008.

STUDY DESIGN

For each procedure, we used empirical Bayes techniques to create a composite measure combining hospital volume, risk-adjusted mortality with the procedure of interest, risk-adjusted mortality with other related procedures, and other variables. Hospitals were ranked based on 2005-2006 data and placed in one of three groups: 1-star (bottom 20 percent), 2-star (middle 60 percent), and 3-star (top 20 percent). We assessed how well these ratings forecasted risk-adjusted mortality rates in the next 2 years (2007-2008), compared to other measures.

PRINCIPAL FINDINGS

For all five procedures, the composite measures based on 2005-2006 data performed well in predicting future hospital performance. Compared to 1-star hospitals, risk-adjusted mortality was much lower at 3-star hospitals for esophagectomy (6.7 versus 14.4 percent), pancreatectomy (4.7 versus 9.2 percent), coronary artery bypass surgery (2.6 versus 5.0 percent), aortic valve replacement (4.5 versus 8.5 percent), and percutaneous coronary interventions (2.4 versus 4.1 percent). Compared to individual surgical quality measures, the composite measures were better at forecasting future risk-adjusted mortality. These measures also outperformed the Center for Medicare and Medicaid Services (CMS) Hospital Compare ratings.

CONCLUSION

Composite measures of surgical quality are very effective at predicting hospital mortality rates with major procedures. Such measures would be more informative than existing quality indicators in helping patients and payers identify high-quality hospitals with specific procedures.

Guided transfer of critically ill patients: where patients are transferred can be an informed choice

PURPOSE OF REVIEW

Given increasingly scarce healthcare resources and highly differentiated hospitals, with growing demand for critical care, interhospital transfer is an essential part of the care of many patients. The purpose of this review is to examine the extent to which hospital quality is considered when transferring critically ill patients, and to examine the potential benefits to patients of a strategy that incorporates objective quality data into referral patterns.

RECENT FINDINGS

Interhospital transfer of critically ill patients is now common and safe. Although extensive research has focused on which patients should be transferred and when they should be transferred, recent study has focused on where patients should be transferred. Yet, the choice of destination hospital is rarely recognized as a therapeutic choice with implications for patient outcomes. The recent public release of high-quality, risk-adjusted and reliability-adjusted outcome data for most hospitals now offers physicians an informed basis on which to choose to which destination hospital a patient should be transferred. A strategy of 'guided transfer' that integrates public quality information into critical care transfer decisions is now feasible.

SUMMARY

Although hospitals often transfer patients, there may be substantial room for improvement in transfer patterns. Guiding transfers on the basis of objective quality information may offer substantial benefits to patients, and could be incorporated into quality improvement initiatives.

Large variations in Medicare payments for surgery highlight savings potential from bundled payment programs

Payers are considering bundled payments for inpatient surgery, combining provider reimbursements into a single payment for the entire episode. We found that current Medicare episode payments for certain inpatient procedures varied by 49-130 percent across hospitals sorted into five payment groups. Intentional differences in payments attributable to such factors as geography or illness severity explained much of this variation. But after adjustment for these differences, per episode payments to the highest-cost hospitals were higher than those to the lowest-cost facilities by up to $2,549 for colectomy and $7,759 for back surgery. Postdischarge care accounted for a large proportion of the variation in payments, as did discretionary physician services, which may be driven in turn by variations in surgeons' practice styles. Our study suggests that bundled payments could yield sizable savings for payers, although the effect on individual institutions will vary because hospitals that were relatively expensive for one procedure were often relatively inexpensive for others. More broadly, our data suggest that many hospitals have considerable room to improve their cost efficiency for inpatient surgery and should look for patterns of excess utilization, particularly among surgical specialties, other inpatient specialist consultations, and various types of postdischarge care.

Hospital quality and the cost of inpatient surgery in the United States

CONTEXT

Payers, policy makers, and professional organizations have launched a variety of initiatives aimed at improving hospital quality with inpatient surgery. Despite their obvious benefits for patients, the likely impact of these efforts on health care costs is uncertain. In this context, we examined relationships between hospital outcomes and expenditures in the US Medicare population.

METHODS

Using the 100% national claims files, we identified all US hospitals performing coronary artery bypass graft, total hip replacement, abdominal aortic aneurysm repair, or colectomy procedures between 2005 and 2007. For each procedure, we ranked hospitals by their risk- and reliability-adjusted outcomes (complication and mortality rates, respectively) and sorted them into quintiles. We then examined relationships between hospital outcomes and risk-adjusted, 30-day episode payments.

RESULTS

There was a strong, positive correlation between hospital complication rates and episode payments for all procedures. With coronary artery bypass graft, for example, hospitals in the highest complication quintile had average payments that were $5353 per patient higher than at hospitals in the lowest quintile ($46,024 vs $40,671, P < 0.001). Payments to hospitals with high complication rates were also higher for colectomy ($2719 per patient), abdominal aortic aneurysm repair ($5279), and hip replacement ($2436). Higher episode payments at lower-quality hospitals were attributable in large part to higher payments for the index hospitalization, although 30-day readmissions, physician services, and postdischarge ancillary care also contributed. Despite the strong association between hospital complication rates and payments, hospital mortality was not associated with expenditures.

CONCLUSIONS

Medicare payments around episodes of inpatient surgery are substantially higher at hospitals with high complications. These findings suggest that local, regional, and national efforts aimed at improving surgical quality may ultimately reduce costs and improve outcomes.

Reliability of superficial surgical site infections as a hospital quality measure

BACKGROUND

Although rates of superficial surgical site infection (SSI) are increasingly used as measures of hospital quality, the statistical reliability of using SSI rates in this context is uncertain. We used the American College of Surgeons National Surgical Quality Improvement Program data to determine the reliability of SSI rates as a measure of hospital performance and to evaluate the effect of hospital caseload on reliability.

STUDY DESIGN

We examined all patients who underwent colon resection in hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program in 2007 (n = 18,455 patients, n = 181 hospitals). We first calculated the number of cases and the risk-adjusted rate of SSI at each hospital. We then used hierarchical modeling to estimate the reliability of this quality measure for each hospital. Finally, we quantified the proportion of hospital-level variation in SSI rates due to patient characteristics and measurement noise.

RESULTS

The average number of colon resections per hospital was 102 (SD 65). The risk-adjusted rate of superficial SSI was 10.5%, but varied from 0 to 30% across hospitals. Approximately 35% of the variation in SSI rates was explained by noise, 7% could be attributed to patient characteristics, and the remaining 58% represented true differences in SSI rates. Just more than half of the hospitals (54%) had a reliability >0.70, which is considered a minimum acceptable level. To achieve this level of reliability, 94 cases were required.

CONCLUSIONS

SSI rates are a reliable measure of hospital quality when an adequate number of cases have been reported. For hospitals with inadequate caseloads, the National Surgical Quality Improvement Program sampling strategy could be altered to provide enough cases to ensure reliability.