Variation in surgical site infection monitoring and reporting by state

Validation of an Electronic Surveillance Algorithm to Identify Patients With Post-Operative Surgical Site Infections Using National Surgical Quality Improvement Program Pediatric Data

BACKGROUND

Surgical site infections (SSIs) are common, but data related to these infections maybe difficult to capture. We developed an electronic surveillance algorithm to identify patients with SSIs. Our objective was to validate our algorithm by comparing it with our institutional National Surgical Quality Improvement Program Pediatric (NSQIP Peds) data.

METHODS

We applied our algorithm to our institutional NSQIP Peds 2015-2017 cohort. The algorithm consisted of the presence of a diagnosis code for post-operative infection or the presence of 4 criteria: diagnosis code for infection, antibiotic administration, positive culture, and readmission/surgery related to infection. We compared the algorithm's SSI rate to the NSQIP Peds identified SSI. Algorithm performance was assessed using sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and Cohen's kappa. The charts of discordant patients were reviewed to understand limitations of the algorithm.

RESULTS

Of 3879 patients included, 2.5% had SSIs by NSQIP Peds definition and 1.9% had SSIs by our algorithm. Our algorithm achieved a sensitivity of 44%, specificity of 99%, NPV of 99%, PPV of 59%, and Cohen's kappa of 0.5. Of the 54 false negatives, 37% were diagnosed/treated as outpatients, 31% had tracheitis, and 17% developed SSIs during their post-operative admission. Of the 30 false positives, 33% had an infection at index surgery and 33% had SSIs related to other surgeries/procedures.

CONCLUSIONS

Our algorithm achieved high specificity and NPV compared with NSQIP Peds reported SSIs and may be useful when identifying SSIs in patient populations that are not actively monitored for SSIs.

Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates

OBJECTIVE

Surgical site infections (SSIs) are common costly hospital-acquired conditions. While statistical process control (SPC) use in healthcare has increased, limited rigorous empirical research compares and optimises these methods for SSI surveillance. We sought to determine which SPC chart types and design parameters maximise the detection of clinically relevant SSI rate increases while minimising false alarms.

DESIGN

Systematic retrospective data analysis and empirical optimisation.

METHODS

We analysed 12 years of data on 13 surgical procedures from a network of 58 community hospitals. Statistically significant SSI rate increases (signals) at individual hospitals initially were identified using 50 different SPC chart variations (Shewhart or exponentially weighted moving average, 5 baseline periods, 5 baseline types). Blinded epidemiologists evaluated the clinical significance of 2709 representative signals of potential outbreaks (out of 5536 generated), rating them as requiring 'action' or 'no action'. These ratings were used to identify which SPC approaches maximised sensitivity and specificity within a broader set of 3600 individual chart variations (additional baseline variations and chart types, including moving average (MA), and five control limit widths) and over 32 million dual-chart combinations based on different baseline periods, reference data (network-wide vs local hospital SSI rates), control limit widths and other calculation considerations. Results were validated with an additional year of data from the same hospital cohort.

RESULTS

The optimal SPC approach to detect clinically important SSI rate increases used two simultaneous MA charts calculated using lagged rolling baseline windows and 1 SD limits. The first chart used 12-month MAs with 18-month baselines and best identified small sustained increases above network-wide SSI rates. The second chart used 6-month MAs with 3-month baselines and best detected large short-term increases above individual hospital SSI rates. This combination outperformed more commonly used charts, with high sensitivity (0.90; positive predictive value=0.56) and practical specificity (0.67; negative predictive value=0.94).

CONCLUSIONS

An optimised combination of two MA charts had the best performance for identifying clinically relevant small but sustained above-network SSI rates and large short-term individual hospital increases.

What Is the Real Rate of Surgical Site Infection?

PURPOSE

Surgical site infections (SSIs) are associated with patient morbidity and increased health care costs. Although several national organizations including the University HealthSystem Consortium (UHC), the National Surgical Quality Improvement Program (NSQIP), and the National Healthcare Safety Network (NHSN) monitor SSI, there is no standard reporting methodology.

METHODS

We queried the UHC, NSQIP, and NHSN databases from July 2012 to June 2014 for SSI after gynecologic surgery at our institution. Each organization uses different definitions and inclusion and exclusion criteria for SSI. The rate of SSI was also obtained from chart review from April 1 to June 30, 2014. SSI was classified as superficial, deep, or organ space infection. The rates reported by the agencies were compared with the rates obtained by chart review using Fisher's exact test.

RESULTS

Overall SSI rates for the databases were as follows: UHC, 1.5%; NSQIP, 8.8%; and NHSN, 2.8% (P < .001). The individual databases had wide variation in the rate of deep infection (UHC, 0.7%; NSQIP, 4.7%; NHSN, 1.3%; P < .001) and organ space infection (UHC, 0.4%; NSQIP, 4.4%; NHSN, 1.4%; P < .001). In agreement with the variation in reporting methodology, only 19 cases (24.4%) were included in more than one database and only one case was included in all three databases (1.3%).

CONCLUSION

There is discordance among national reporting agencies tracking SSI. Adopting standardized metrics across agencies could improve consistency and accuracy in assessing SSI rates.

Surgical Site Infections: Volume-Outcome Relationship and Year-to-Year Stability of Performance Rankings

BACKGROUND

Surgical site infection (SSI) rates are publicly reported as quality metrics and increasingly used to determine financial reimbursement.

OBJECTIVE

To evaluate the volume-outcome relationship as well as the year-to-year stability of performance rankings following coronary artery bypass graft (CABG) surgery and hip arthroplasty.

RESEARCH DESIGN

We performed a retrospective cohort study of Medicare beneficiaries who underwent CABG surgery or hip arthroplasty at US hospitals from 2005 to 2011, with outcomes analyzed through March 2012. Nationally validated claims-based surveillance methods were used to assess for SSI within 90 days of surgery. The relationship between procedure volume and SSI rate was assessed using logistic regression and generalized additive modeling. Year-to-year stability of SSI rates was evaluated using logistic regression to assess hospitals' movement in and out of performance rankings linked to financial penalties.

RESULTS

Case-mix adjusted SSI risk based on claims was highest in hospitals performing <50 CABG/year and <200 hip arthroplasty/year compared with hospitals performing ≥200 procedures/year. At that same time, hospitals in the worst quartile in a given year based on claims had a low probability of remaining in that quartile the following year. This probability increased with volume, and when using 2 years' experience, but the highest probabilities were only 0.59 for CABG (95% confidence interval, 0.52-0.66) and 0.48 for hip arthroplasty (95% confidence interval, 0.42-0.55).

CONCLUSIONS

Aggregate SSI risk is highest in hospitals with low annual procedure volumes, yet these hospitals are currently excluded from quality reporting. Even for higher volume hospitals, year-to-year random variation makes past experience an unreliable estimator of current performance.

Surgical site infection: poor compliance with guidelines and care bundles

Surgical site infections (SSIs) are probably the most preventable of the health care-associated infections. Despite the widespread international introduction of level I evidence-based guidelines for the prevention of SSIs, such as that of the National Institute for Clinical Excellence (NICE) in the UK and the surgical care improvement project (SCIP) of the USA, SSI rates have not measurably fallen. The care bundle approach is an accepted method of packaging best, evidence-based measures into routine care for all patients and, common to many guidelines for the prevention of SSI, includes methods for preoperative removal of hair (where appropriate), rational antibiotic prophylaxis, avoidance of perioperative hypothermia, management of perioperative blood glucose and effective skin preparation. Reasons for poor compliance with care bundles are not clear and have not matched the wide uptake and perceived benefit of the WHO 'Safe Surgery Saves Lives' checklist. Recommendations include the need for further research and continuous updating of guidelines; comprehensive surveillance, using validated definitions that facilitate benchmarking of anonymised surgeon-specific SSI rates; assurance that incorporation of checklists and care bundles has taken place; the development of effective communication strategies for all health care providers and those who commission services and comprehensive information for patients.

Improving public reporting and data validation for complex surgical site infections after coronary artery bypass graft surgery and hip arthroplasty

Diagnosis codes in claims submitted for reimbursement following coronary artery bypass graft surgery and hip arthroplasty allow standardized and efficient identification of deep and organ/space surgical site infections.

Background

 Deep and organ/space surgical site infections (D/OS SSI) cause significant morbidity, mortality, and costs. Rates are publicly reported and increasingly used as quality metrics affecting hospital payment. Lack of standardized surveillance methods threaten the accuracy of reported data and decrease confidence in comparisons based upon these data.

Methods

 We analyzed data from national validation studies that used Medicare claims to trigger chart review for SSI confirmation after coronary artery bypass graft surgery (CABG) and hip arthroplasty. We evaluated code performance (sensitivity and positive predictive value) to select diagnosis codes that best identified D/OS SSI. Codes were analyzed individually and in combination.

Results

 Analysis included 143 patients with D/OS SSI after CABG and 175 patients with D/OS SSI after hip arthroplasty. For CABG, 9 International Classification of Diseases, 9th Revision (ICD-9) diagnosis codes identified 92% of D/OS SSI, with 1 D/OS SSI identified for every 4 cases with a diagnosis code. For hip arthroplasty, 6 ICD-9 diagnosis codes identified 99% of D/OS SSI, with 1 D/OS SSI identified for every 2 cases with a diagnosis code.

Conclusions

 This standardized and efficient approach for identifying D/OS SSI can be used by hospitals to improve case detection and public reporting. This method can also be used to identify potential D/OS SSI cases for review during hospital audits for data validation.

Using clinical variables to guide surgical site infection detection: a novel surveillance strategy

BACKGROUND

Surgical site infections (SSIs) are a common and expensive health care-associated infection, and are used as a health care quality benchmark. As such, SSI detection is a major focus of infection prevention programs. In an effort to improve on conventional surveillance methods, a simple algorithm for SSI detection was developed using clinical variables not traditionally included in National Healthcare Safety Network definitions.

METHODS

A case-control study was conducted among surgeries performed at the Veterans Affairs Boston Healthcare System between January 2008 and December 2009. SSI cases were matched to controls without SSI. Clinical variables (administrative, microbiological, pharmacy, radiology) were compared between the groups to determine those that best identified SSI.

RESULTS

A total of 70 SSIs were matched to 70 controls. On multivariable analysis, variables significantly associated with SSI identification were wound culture order, computed tomography scan/magnetic resonance imaging order, antibiotic order within 30 days after surgery, and application of a relevant International Classification of Disease, Ninth Revision code. Among patients with no SSI identifiers, 98% were correctly classified as having no SSI. Among patients with multiple SSI identifiers, 97.1% were correctly identified as having SSI. The area under the curve for this model was 0.87.

CONCLUSION

We have derived a novel surveillance algorithm for SSI detection with excellent operating characteristics. This algorithm could be automated to streamline infection control efforts.

Medicare claims can be used to identify US hospitals with higher rates of surgical site infection following vascular surgery

BACKGROUND

Surgical site infections (SSIs) following vascular surgery have high morbidity and costs, and are increasingly tracked as hospital quality measures.

OBJECTIVE

To assess the ability of Medicare claims to identify US hospitals with high SSI rates after vascular surgery.

RESEARCH DESIGN

Using claims from fee-for-service Medicare enrollees of age 65 years and older who underwent vascular surgery from 2005 to 2008, we derived hospital rankings using previously validated codes suggestive of SSI, with individual-level adjustment for age, sex, and comorbidities. We then obtained medical records for validation of SSI from hospitals ranked in the best and worst deciles of performance, and used logistic regression to calculate the risk-adjusted odds of developing an SSI in worst-decile versus best-decile hospitals.

RESULTS

Among 203,023 Medicare patients who underwent vascular surgery at 2512 US hospitals, a patient undergoing surgery in a hospital ranked in the worst-performing decile based on claims had 2.5 times higher odds of developing a chart-confirmed SSI relative to a patient with the same age, sex, and comorbidities in a hospital ranked in the best-performing decile (95% confidence interval, 2.0-3.1). SSI confirmation among patients with claims suggesting infection was similar across deciles, and we found similar findings in analyses limited to deep and organ/space SSIs. We report on diagnosis codes with high sensitivity for identifying deep and organ/space SSI, with one-to-one mapping to ICD-10-CM codes.

CONCLUSIONS

Claims-based surveillance offers a standardized and objective methodology that can be used to improve SSI surveillance and to validate hospitals' publicly reported data.

Building a business case for colorectal surgery quality improvement

BACKGROUND

Improving surgical quality is a priority, but building a business case for the efforts could be challenging. Bridging the gap between the clinicians and hospital leaders is the first step to align quality and financial priorities within health care.

OBJECTIVE

The aim of this study was to evaluate the financial impact of the surgical comprehensive unit-based safety program on colorectal surgery procedures.

DESIGN

This a retrospective cohort study.

SETTING

This study was conducted at a university-based tertiary care hospital.

PATIENTS

All patients undergoing colectomy or proctectomy between July 2010 and June 2012 were included.

INTERVENTION

A comprehensive unit-based safety program focused on colorectal surgical site infection reduction was implemented. Three surgeons participated in the program in year 1, and 5 surgeons participated in year 2. Patients were categorized as participating or nonparticipating based on the surgeon who performed the procedure.

MAIN OUTCOME MEASURES

Resource utilization and cost were the main outcome measures.

RESULTS

During the 2 years, there were 626 patients who met the selection criteria. Participating surgeons operated on 444 patients (70.9%), and the nonparticipating surgeons operated on 182 patients (29.1%). After adjusting for covariates, the variable direct cost was significantly lower for the participating surgeons in laboratory work by $191 (p = 0.009), operating room utilization by $149 (p = 0.05), and supplies by $615 (p = 0.003). The surgical site infection rates, need for an intensive care unit stay, and length of stay were not significantly different between the 2 groups.

LIMITATIONS

The multiple biases related to surgeon self-selection for program participation and surgeon training and clinical skills were not addressed in this study owing to the limitations in sample size and data collection.

CONCLUSION

A comprehensive unit-based safety program implementation, including dedicated frontline providers who focused on the standardization of protocols, was able to reduce the variation in resource utilization and costs in comparison with a control group.

Clinical and economic burden of surgical site infection (SSI) and predicted financial consequences of elimination of SSI from an English hospital

BACKGROUND

Although surgical site infections (SSIs) are known to be associated with increased length of stay (LOS) and additional cost, their impact on the profitability of surgical procedures is unknown.

AIM

To determine the clinical and economic burden of SSI over a two-year period and to predict the financial consequences of their elimination.

METHODS

SSI surveillance and Patient Level Information and Costing System (PLICS) datasets for patients who underwent major surgical procedures at Plymouth Hospitals NHS Trust between April 2010 and March 2012 were consolidated. The main outcome measures were the attributable postoperative length of stay (LOS), cost, and impact on the margin differential (profitability) of SSI. A secondary outcome was the predicted financial consequence of eliminating all SSIs.

FINDINGS

The median additional LOS attributable to SSI was 10 days [95% confidence interval (CI): 7-13 days] and a total of 4694 bed-days were lost over the two-year period. The median additional cost attributable to SSI was £5,239 (95% CI: 4,622-6,719) and the aggregate extra cost over the study period was £2,491,424. After calculating the opportunity cost of eliminating all SSIs that had occurred in the two-year period, the combined overall predicted financial benefit of doing so would have been only £694,007. For seven surgical categories, the hospital would have been financially worse off if it had successfully eliminated all SSIs.

CONCLUSION

SSI causes significant clinical and economic burden. Nevertheless the current system of reimbursement provided a financial disincentive to their reduction.

e-Measures: insight into the challenges and opportunities of automating publicly reported quality measures

Using electronic health records (EHR) to automate publicly reported quality measures is receiving increasing attention and is one of the promises of EHR implementation. Kaiser Permanente has fully or partly automated six of 13 the joint commission measure sets. We describe our experience with automation and the resulting time savings: a reduction by approximately 50% of abstractor time required for one measure set alone (surgical care improvement project). However, our experience illustrates the gap between the current and desired states of automated public quality reporting, which has important implications for measure developers, accrediting entities, EHR vendors, public/private payers, and government.

A benchmark too far: findings from a national survey of surgical site infection surveillance

BACKGROUND

The national surgical site infection (SSI) surveillance service in England collates and publishes SSI rates that are used for benchmarking and to identify the prevalence of SSIs. However, research studies using high-quality SSI surveillance report rates that are much higher than those published by the national surveillance service. This variance questions the validity of data collected through the national service.

AIM

To audit SSI definitions and data collection methods used by hospital trusts in England.

METHOD

All 156 hospital trusts in England were sent questionnaires that focused on aspects of SSI definitions and data collection methods.

FINDINGS

Completed questionnaires were received from 106 hospital trusts. There were considerable differences in data collection methods and data quality that caused wide variation in reported SSI rates. For example, the SSI rate for knee replacement surgery was 4.1% for trusts that used high-quality postdischarge surveillance (PDS) and 1.5% for trusts that used low-quality PDS. Contrary to national protocols and definitions, 10% of trusts did not provide data on superficial infections, 15% of trusts did not use the recommended SSI definition, and 8% of trusts used inpatient data alone. Thirty trusts did not submit a complete set of their data to the national surveillance service. Unsubmitted data included non-mandatory data, PDS data and continuous data.

CONCLUSION

The national surveillance service underestimates the prevalence of SSIs and is not appropriate for benchmarking. Hospitals that conduct high-quality SSI surveillance will be penalized within the current surveillance service.