Hospital patient safety grades may misrepresent hospital performance

Reducing Catheter-Associated Urinary Tract Infections Across a Hospital System Through Urine Culture Stewardship

Objective

To evaluate the effectiveness and safety of an evidence-based urine culture stewardship program in reducing hospital catheter-associated urinary tract infections (CAUTIs) and the rate of CAUTIs across a 3-hospital system.

Patients and Methods

This is a prospective, 2-year quality improvement program conducted from October 1, 2018, to September 30, 2020. An evidence-based urine culture stewardship program was designed, which consisted of the following: criteria for allowing or restricting urine cultures from catheterized patients, a best practice advisory integrated into the ordering system of an electronic medical record, and a systematic provider education and feedback program to ensure compliance. The system-wide rates of CAUTIs (total CAUTIs/catheter days×1000), changes in intercepts, trends, mortality, length of stay, rates of device utilization, and rates of hospital-onset sepsis were compared for 3 years before and 2 years after the launch of the program.

Results

Catheter-associated urinary tract infections progressively decreased after the initiation of the program (B=−0.21, P=.001). When the trends before and after the initiation of the program were compared, there were no statistically significant increases in the ratio of actual to predicted hospital length of stay, intensive care unit length of stay, system-wide mortality, and intensive care unit mortality. Although the rates of hospital-acquired sepsis remained consistent after the implementation of the stewardship program through the first quarter of 2020, the rates showed an increase in the second and third quarters of 2020. However, hospital-onset sepsis events associated with the diagnosis of a urinary tract infection did not increase after the intervention.

Conclusion

Urine culture stewardship is a safe and effective way to reduce CAUTIs among patients in a large multihospital health care system. Patient safety indicators appeared unchanged after the implementation of the program, and ongoing follow-up will improve confidence in the long-term sustainability of this strategy.

Leapfrog Hospital Safety Score, Magnet Designation, and Healthcare-Associated Infections in United States Hospitals

OBJECTIVE

Healthcare-associated infections (HAIs) pose a challenge to patient safety. Although studies have explored individual level, few have focused on organizational factors such as a hospital's safety infrastructure (indicated by Leapfrog Hospital Safety Score) or workplace quality (Magnet recognition). The aim of the study was to determine whether Magnet and hospitals with better Leapfrog Hospital Safety Scores have fewer HAIs.

METHODS

Ordered probit regression analyses tested associations between Safety Score, Magnet status, and standardized infection ratios, depicting whether a hospital had a Clostridium difficile infection (CDI) and methicillin-resistant Staphylococcus aureus (MRSA) bloodstream infection standardized infection ratio that was "better," "no different," or "worse" than a National Benchmark as per Centers for Disease Control and Prevention's National Healthcare Safety Network definitions.

RESULTS

Accounting for confounders, relative to "A" hospitals, "B" and "C" hospitals had significant and negative relationships with CDI (-0.16, P < 0.01, and -0.14, P < 0.05, respectively) but not MRSA bacteremia. Magnet hospitals had a significant and positive relationship with MRSA bloodstream infections (0.74, P < 0.001) but a significant negative relationship with CDI (-0.21, P < 0.01) compared with non-Magnet.

CONCLUSIONS

A hospitals performed better on CDI but not MRSA bloodstream infections. In contrast, Magnet designation was associated with fewer than expected MRSA infections but more than expected CDIs. These mixed results indicate that hospital global assessments of safety and workplace quality differentially and imperfectly predict its level of HAIs, suggesting the need for more precise organizational measures of safety and more nuanced approaches to infection prevention and reduction.

Quality of Care Among Medicare Patients Undergoing Pancreatic Surgery: Safety Grade, Magnet Recognition, and Leapfrog Minimum Volume Standards-Which Quality Benchmark Matters?

BACKGROUND

The association of national quality benchmarking metrics with postoperative outcomes following complex surgery remains unknown. We assessed the relationship between the "quality trifactor" of Leapfrog minimum volume standards, Hospital Safety Grade A, and Magnet Recognition with outcomes of Medicare patients undergoing pancreatectomy.

METHODS

The Standard Analytic Files (SAF) merged with Leapfrog Hospital Survey and Leapfrog Safety Scores Denominator Files were reviewed to identify Medicare patients who underwent pancreatic procedures between 2013 and 2015. Primary outcomes were overall and serious complications, as well as 30- and 90-day mortality. Multivariable logistic regression analyses were conducted to evaluate possible associations among hospitals meeting the quality trifactor and short-term outcomes.

RESULTS

Among 4853 Medicare patients, 909 (18.7%) underwent pancreatectomy at hospitals meeting the quality trifactor. Among 260 hospitals, 7.3% (n = 19) met the quality trifactor. Safety Grade A (48.8%, n = 127) was the most commonly met criterion followed by Magnet Recognition (36.2%, n = 94); the Leapfrog minimum volume standards were achieved by 25% (n = 65) of hospitals. Patients undergoing surgery at hospitals that were only Safety Grade A and Magnet designated, but did not meet Leapfrog criteria, had higher odds of serious complications (OR 1.59, 95% CI 1.00-2.51). In contrast, patients undergoing treatment at hospitals having all three designations (i.e., the quality trifactor) had 40% and 39% lower odds of both serious complications (OR 0.60, 95% CI 0.37-0.97) and 90-day mortality (OR 0.61, 95% CI 0.42-0.89), respectively. In turn, patients undergoing pancreatectomy at quality trifactor hospitals had higher odds of experiencing the composite quality measure textbook outcome (OR 1.28, 95% CI 1.03-1.59) versus patients undergoing pancreatectomy at non-trifactor hospitals.

CONCLUSION

While Safety Grade A and Magnet designation alone were not associated with higher odds of an optimal composite outcome following pancreatectomy, compliance with Leapfrog criteria to achieve the "quality trifactor" metric was associated with lower odds of serious complications and mortality.

Assessment of hospital quality and safety standards among Medicare beneficiaries undergoing surgery for cancer

BACKGROUND

We sought to assess the relationship between Leapfrog minimum volume standards, Hospital Safety Grades, and Magnet recognition with outcomes among patients undergoing rectal, lung, esophageal, and pancreatic resection for cancer.

METHODS

Standard Analytical Files linked with the Leapfrog Hospital Survey and the Leapfrog Safety Scores Denominator Files were used to identify Medicare patients who underwent surgery for cancer from 2016 to 2017. Multivariable logistic regression analysis was used to examine textbook outcomes relative to Leapfrog volume, safety grades, and Magnet recognition.

RESULTS

Among 26,268 Medicare beneficiaries, 7,491 (28.5%) were treated at hospitals meeting the quality trifactor (Leapfrog, safety grade A, and Magnet recognition) vs 18,777 (71.5%) at hospitals not meeting ≥1 designation. Patients at trifactor hospitals had lower odds of complications (odds ratio = 0.83, 95% confidence interval: 0.76-0.89), prolonged duration of stay (odds ratio = 0.89, 95% confidence interval: 0.82-0.97), and higher odds of experiencing textbook outcome (odds ratio = 1.12, 95% confidence interval: 1.06-1.19). Patients undergoing surgery for lung (odds ratio = 1.19, 95% confidence interval: 1.10-1.30) and pancreatic cancer (odds ratio = 1.37, 95% confidence interval: 1.21-1.55) at trifactor hospitals had higher odds of textbook outcome, whereas this effect was not noted after esophageal (odds ratio = 1.16, 95% confidence interval: 0.90-1.48) or rectal cancer (odds ratio = 1.11, 95% confidence interval: 0.98-1.27) surgery. Leapfrog minimum volume standards mediated the effect of the quality trifactor on patient outcomes.

CONCLUSION

Quality trifactor hospitals had better short-term outcomes after lung and pancreatic cancer surgery compared with nontrifactor hospitals.

Are hospital ratings systems transparent? An examination of Consumer Reports and the Leapfrog Hospital Safety Grade

The health care industry is complex, dynamic, and large. In such uncertain environments where a great deal of revenue is at stake, competition and comparative claims flourish. One such manifestation is hospital ratings systems. This research examines two influential hospital ratings to explore whether the hospital ratings of each system was straightforward and reproducible. Regressions and structural equations models were fit to examine the relationships among the hospital ratings constructs. Both hospital ratings systems were excellent in their transparency and reproducibility. The Consumer Reports and Leapfrog ratings systems can confidently tout that their hospital scores reflect what they claim to measure. The unique aspects of each system are also noted.

Dissecting Leapfrog: How Well Do Leapfrog Safe Practices Scores Correlate With Hospital Compare Ratings and Penalties, and How Much Do They Matter?

BACKGROUND

Voluntary Leapfrog Safe Practices Score (SPS) measures were among the first public reports of hospital performance. Recently, Medicare's Hospital Compare website has reported compulsory measures. Leapfrog's Hospital Safety Score (HSS) grades incorporate SPS and Medicare measures. We evaluate associations between Leapfrog SPS and Medicare measures, and the impact of SPS on HSS grades.

METHODS

Using 2013 hospital data, we linked Leapfrog HSS data with central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) standardized infection ratios (SIRs), and Hospital Readmission and Hospital-Acquired Condition (HAC) Reduction Program penalties incorporating 2013 performance. For SPS-providing hospitals, we used linear and logistic regression models to predict CLABSI/CAUTI SIRs and penalties as a function of SPS. For hospitals not reporting SPS, we simulated change in HSS grades after imputing a range of SPS.

RESULTS

In total, 1089 hospitals reported SPS; >50% self-reported perfect scores for all but 1 measure. No SPS measures were associated with SIRs. One SPS (feedback) was associated with lower odds of HAC penalization (odds ratio, 0.86; 95% confidence interval, 0.76-0.97). Among hospitals not reporting SPS (N=1080), 98% and 54% saw grades decline by 1+ letters with first and 10th percentile SPS imputed, respectively; 49% and 54% saw grades improve by 1+ letter with median and highest SPS imputed.

CONCLUSIONS

Voluntary Leapfrog SPS measures skew toward positive self-report and bear little association with compulsory Medicare outcomes and penalties. SPS significantly impacts HSS grades, particularly when lower SPS is reported. With increasing compulsory reporting, Leapfrog SPS seems limited for comparing hospital performance.