Agreement among healthcare professionals in ten European countries in diagnosing case-vignettes of surgical-site infections

Evaluation of a multifaceted implementation strategy for semi-automated surveillance of surgical site infections after total hip or knee arthroplasty: a multicentre pilot study in the Netherlands

INTRODUCTION

To promote the nation-wide implementation of semi-automated surveillance (AS) of surgical site infection after hip and knee arthroplasty, the Dutch National Institute for Public Health and the Environment (RIVM) deployed a decentralised multifaceted implementation strategy. This strategy consisted of a protocol specifying minimum requirements for an AS system, supported by a user manual, education module, individual guidance for hospitals and user-group meetings. This study describes an effect evaluation and process evaluation of the implementation strategy for AS in five frontrunner hospitals.

METHODS

To evaluate the effect of the implementation strategy, the achieved phase of implementation was determined in each frontrunner hospital at the end of the study period. The process evaluation consisted of (1) an evaluation of the feasibility of strategy elements, (2) an evaluation of barriers and facilitators for implementation and (3) an evaluation of the workload for implementation. Interviews were performed as a basis for a subsequent survey quantifying the results regarding the feasibility as well as barriers and facilitators. Workload was self-monitored per profession. Qualitative data were analysed using a framework analysis, whereas quantitative data were analysed descriptively.

RESULTS

One hospital finished the complete implementation process in 240 person-hours. Overall, the elements of the implementation strategy were often used, positively received and overall, the strategy was rated effective and feasible. During the implementation process, participants perceived the relative advantage of AS and had sufficient knowledge about AS. However, barriers regarding complexity of AS data extraction, data-infrastructure, and validation, lack of capacity and motivation at the IT department, and difficulties with assigning roles and responsibilities were experienced.

CONCLUSION

A decentralised multifaceted implementation strategy is suitable for the implementation of AS in hospitals. Effective local project management, including clear project leadership and ownership, obtaining commitment of higher management levels, active involvement of stakeholders, and appropriate allocation of roles and responsibilities is important for successful implementation and should be facilitated by the implementation strategy. Sufficient knowledge about AS, its requirements and the implementation process should be available among stakeholders by e.g. an education module. Furthermore, exchange of knowledge and experiences between hospitals should be encouraged in user-group meetings.

The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery

BACKGROUND

In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR).

METHODS

Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated.

RESULTS

From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm.

CONCLUSIONS

The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.

Variability in Serious Safety Event Classification among Children's Hospitals: A Measure for Comparison?

Introduction

Hospitals have no standard for measuring comparative rates of serious safety events (SSE). A pediatric hospital safety collaborative has used a common definition and measurement system to classify SSE and calculate a serious safety event rate. An opportunity exists to evaluate the use of this measurement system.

Methods

A web-based survey utilizing 7 case vignettes was sent to 132 network hospitals to assess agreement in classifying the vignettes as SSEs. Respondents classified the vignettes according to the taxonomy used at their respective organizations for deviations and SSE classification.

Results

Of the 82 respondents, 67 (82%) utilized the same SSE classification system. Respondents did not assess deviations for 2 of the 7 vignettes, which had clear deviations. Of the remaining 5 vignettes, 3 had a substantial agreement of deviation (>85%, Gwet's AC ≥ 0.68), and 2 had fair agreement (<70%, Gwet's AC ≤ 0.39). Four of the 7 vignettes had a substantial agreement on SSE classification (>80%; Gwet's AC ≥ 0.80), and 3 had slight to moderate agreement (<70%, Gwet's AC ≤ 0.78).

Conclusions

Results demonstrated agreement and variability in determining deviation and SSE classification in the 7 vignettes. Although the SSE methodology and metric used by participant pediatric hospitals yields generally similar review results, one must be cautious in using the SSE rate to compare patient safety outcomes across different hospitals.

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Objective:

Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals.

Design:

Multicenter retrospective cohort study.

Methods:

From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance.

Results:

In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%–14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data.

Conclusions:

In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.

Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization

BACKGROUND

As most automated surveillance (AS) methods to detect healthcare-associated infections (HAIs) have been developed and implemented in research settings, information about the feasibility of large-scale implementation is scarce.

AIM

We aimed to describe key aspects of the design of AS systems and implementation in European institutions and hospitals.

METHODS

An online survey was distributed via email in February/March 2019 among 1) PRAISE (Providing a Roadmap for Automated Infection Surveillance in Europe) network members; 2) corresponding authors of peer-reviewed European publications on existing AS systems; and 3) the mailing list of national infection prevention and control focal points of the European Centre for Disease Prevention and Control. Three AS systems from the survey were selected, based on quintessential features, for in-depth review focusing on implementation in practice.

FINDINGS

Through the survey and the review of three selected AS systems, notable differences regarding the methods, algorithms, data sources and targeted HAIs were identified. The majority of AS systems used a classification algorithm for semi-automated surveillance and targeted HAIs were mostly surgical site infections, urinary tract infections, sepsis or other bloodstream infections. AS systems yielded a reduction of workload for hospital staff. Principal barriers of implementation were strict data security regulations as well as creating and maintaining an information technology infrastructure.

CONCLUSION

AS in Europe is characterized by heterogeneity in methods and surveillance targets. To allow for comparisons and encourage homogenization, future publications on AS systems should provide detailed information on source data, methods and the state of implementation.

Governance aspects of large-scale implementation of automated surveillance of healthcare-associated infections

OBJECTIVES

Surveillance of healthcare-associated infections (HAI) is increasingly automated by applying algorithms to routine-care data stored in electronic health records. Hitherto, initiatives have mainly been confined to single healthcare facilities and research settings, leading to heterogeneity in design. The PRAISE network - Providing a Roadmap for Automated Infection Surveillance in Europe - designed a roadmap to provide guidance on how to move automated surveillance (AS) from the research setting to large-scale implementation. Supplementary to this roadmap, we here discuss the governance aspects of automated HAI surveillance within networks, aiming to support both the coordinating centres and participating healthcare facilities as they set up governance structures and to enhance involvement of legal specialists.

METHODS

This article is based on PRAISE network discussions during two workshops. A taskforce was installed that further elaborated governance aspects for AS networks by reviewing documents and websites, consulting experts and organizing teleconferences. Finally, the article has been reviewed by an independent panel of international experts.

RESULTS

Strict governance is indispensable in surveillance networks, especially when manual decisions are replaced by algorithms and electronically stored routine-care data are reused for the purpose of surveillance. For endorsement of AS networks, governance aspects specifically related to AS networks need to be addressed. Key considerations include enabling participation and inclusion, trust in the collection, use and quality of data (including data protection), accountability and transparency.

CONCLUSIONS

This article on governance aspects can be used by coordinating centres and healthcare facilities participating in an AS network as a starting point to set up governance structures. Involvement of main stakeholders and legal specialists early in the development of an AS network is important for endorsement, inclusivity and compliance with the laws and regulations that apply.

PRAISE: providing a roadmap for automated infection surveillance in Europe

INTRODUCTION

Healthcare-associated infections (HAI) are among the most common adverse events of medical care. Surveillance of HAI is a key component of successful infection prevention programmes. Conventional surveillance - manual chart review - is resource intensive and limited by concerns regarding interrater reliability. This has led to the development and use of automated surveillance (AS). Many AS systems are the product of in-house development efforts and heterogeneous in their design and methods. With this roadmap, the PRAISE network aims to provide guidance on how to move AS from the research setting to large-scale implementation, and how to ensure the delivery of surveillance data that are uniform and useful for improvement of quality of care.

METHODS

The PRAISE network brings together 30 experts from ten European countries. This roadmap is based on the outcome of two workshops, teleconference meetings and review by an independent panel of international experts.

RESULTS

This roadmap focuses on the surveillance of HAI within networks of healthcare facilities for the purpose of comparison, prevention and quality improvement initiatives. The roadmap does the following: discusses the selection of surveillance targets, different organizational and methodologic approaches and their advantages, disadvantages and risks; defines key performance requirements of AS systems and suggestions for their design; provides guidance on successful implementation and maintenance; and discusses areas of future research and training requirements for the infection prevention and related disciplines. The roadmap is supported by accompanying documents regarding the governance and information technology aspects of implementing AS.

CONCLUSIONS

Large-scale implementation of AS requires guidance and coordination within and across surveillance networks. Transitions to large-scale AS entail redevelopment of surveillance methods and their interpretation, intensive dialogue with stakeholders and the investment of considerable resources. This roadmap can be used to guide future steps towards implementation, including designing solutions for AS and practical guidance checklists.

Validation of an algorithm for semiautomated surveillance to detect deep surgical site infections after primary total hip or knee arthroplasty-A multicenter study

OBJECTIVE

Surveillance of healthcare-associated infections is often performed by manual chart review. Semiautomated surveillance may substantially reduce workload and subjective data interpretation. We assessed the validity of a previously published algorithm for semiautomated surveillance of deep surgical site infections (SSIs) after total hip arthroplasty (THA) or total knee arthroplasty (TKA) in Dutch hospitals. In addition, we explored the ability of a hospital to automatically select the patients under surveillance.

DESIGN

Multicenter retrospective cohort study.

METHODS

Hospitals identified patients who underwent THA or TKA either by procedure codes or by conventional surveillance. For these patients, routine care data regarding microbiology results, antibiotics, (re)admissions, and surgeries within 120 days following THA or TKA were extracted from electronic health records. Patient selection was compared with conventional surveillance and patients were retrospectively classified as low or high probability of having developed deep SSI by the algorithm. Sensitivity, positive predictive value (PPV), and workload reduction were calculated and compared to conventional surveillance.

RESULTS

Of 9,554 extracted THA and TKA surgeries, 1,175 (12.3%) were revisions, and 8,378 primary surgeries remained for algorithm validation (95 deep SSIs, 1.1%). Sensitivity ranged from 93.6% to 100% and PPV ranged from 55.8% to 72.2%. Workload was reduced by ≥98%. Also, 2 SSIs (2.1%) missed by the algorithm were explained by flaws in data selection.

CONCLUSIONS

This algorithm reliably detects patients with a high probability of having developed deep SSI after THA or TKA in Dutch hospitals. Our results provide essential information for successful implementation of semiautomated surveillance for deep SSIs after THA or TKA.

Futility of Cluster Designs at Individual Hospitals to Study Surgical Site Infections and Interventions Involving the Installation of Capital Equipment in Operating Rooms

Anesthesia workspaces are integral components in the chains of many intraoperative bacterial transmission events resulting in surgical site infections (SSI). Matched cohort designs can be used to compare SSI rates among operating rooms (ORs) with or without capital equipment purchases (e.g., new anesthesia machines). Patients receiving care in intervention ORs (i.e., with installed capital equipment) are matched with similar patients receiving care in ORs lacking the intervention. We evaluate statistical power of an alternative design for clinical trials in which, instead, SSI incidences are compared directly among ORs (i.e., the ORs form the clusters) at single hospitals (e.g., the 5 ORs with bactericidal lights vs. the 5 other ORs). Data used for parameter estimates were SSI for 24 categories of procedures among 338 hospitals in the State of California, 2015. Estimated statistical power was ≅8.4% for detecting a reduction in the incidence of SSI from 3.6% to 2.4% over 1 year with 5 intervention ORs and 5 control ORs. For ≅80% statistical power, >20 such hospitals would be needed to complete a study in 1 year. Matched paired cluster designs pair similar ORs (e.g., 2 cardiac ORs, 1 to intervention and 1 to control). With 5 pairs, statistical power would be even less than the estimated 8.4%. Cluster designs (i.e., analyses by OR) are not suitable for comparing SSI among ORs at single hospitals. Even though matched cohort designs are non-randomized and thus have lesser validity, matching patients by their risk factors for SSI is more practical.

Global incidence of surgical site infection after appendectomy: a systematic review and meta-analysis

BACKGROUND

Although surgical site infection (SSI) is one of the most studied healthcare-associated infections, the global burden of SSI after appendectomy remains unknown.

OBJECTIVE

We estimated the incidence of SSI after appendectomy at global and regional levels.

DESIGN

Systematic review and meta-analysis.

PARTICIPANTS

Appendectomy patients.

DATA SOURCES

EMBASE, PubMed and Web of Science were searched, with no language restrictions, to identify observational studies and clinical trials published between 1 January 2000 and 30 December 2018 and reporting on the incidence of SSI after appendectomy. A random-effect model meta-analysis served to obtain the pooled incidence of SSI after appendectomy.

RESULTS

In total, 226 studies (729 434 participants from 49 countries) were included in the meta-analysis. With regard to methodological quality, 59 (26.1%) studies had low risk of bias, 147 (65.0%) had moderate risk of bias and 20 (8.8%) had high risk of bias. We found an overall incidence of SSI of 7.0 per 100 appendectomies (95% prediction interval: 1.0-17.6), varying from 0 to 37.4 per 100 appendectomies. A subgroup analysis to identify sources of heterogeneity showed that the incidence varied from 5.8 in Europe to 12.6 per 100 appendectomies in Africa (p<0.0001). The incidence of SSI after appendectomy increased when the level of income decreased, from 6.2 in high-income countries to 11.1 per 100 appendectomies in low-income countries (p=0.015). Open appendectomy (11.0 per 100 surgical procedures) was found to have a higher incidence of SSI compared with laparoscopy (4.6 per 100 appendectomies) (p=0.0002).

CONCLUSION

This study suggests a high burden of SSI after appendectomy in some regions (especially Africa) and in low-income countries. Strategies are needed to implement and disseminate the WHO guidelines to decrease the burden of SSI after appendectomy in these regions.

PROSPERO REGISTRATION NUMBER

CRD42017075257.

Evaluation of Wound Photography for Remote Postoperative Assessment of Surgical Site Infections

Importance

Surgeons are increasingly interested in using mobile and online applications with wound photography to monitor patients after surgery. Early work using remote care to diagnose surgical site infections (SSIs) demonstrated improved diagnostic accuracy using wound photographs to augment patients' electronic reports of symptoms, but it is unclear whether these findings are reproducible in real-world practice.

Objective

To determine how wound photography affects surgeons' abilities to diagnose SSIs in a pragmatic setting.

Design, Setting, and Participants

This prospective study compared surgeons' paired assessments of postabdominal surgery case vignettes with vs without wound photography for detection of SSIs. Data for case vignettes were collected prospectively from May 1, 2007, to January 31, 2009, at Erasmus University Medical Center, Rotterdam, the Netherlands, and from July 1, 2015, to February 29, 2016, at Vanderbilt University Medical Center, Nashville, Tennessee. The surgeons were members of the American Medical Association whose self-designated specialty is general, abdominal, colorectal, oncologic, or vascular surgery and who completed internet-based assessments from May 21 to June 10, 2016.

Intervention

Surgeons reviewed online clinical vignettes with or without wound photography.

Main Outcomes and Measures

Surgeons' diagnostic accuracy, sensitivity, specificity, confidence, and proposed management with respect to SSIs.

Results

A total of 523 surgeons (113 women and 410 men; mean [SD] age, 53 [10] years) completed a mean of 2.9 clinical vignettes. For the diagnosis of SSIs, the addition of wound photography did not change accuracy (863 of 1512 [57.1%] without and 878 of 1512 [58.1%] with photographs). Photographs decreased sensitivity (from 0.58 to 0.50) but increased specificity (from 0.56 to 0.63). In 415 of 1512 cases (27.4%), the addition of wound photography changed the surgeons' assessment (215 of 1512 [14.2%] changed from incorrect to correct and 200 of 1512 [13.2%] changed from correct to incorrect). Surgeons reported greater confidence when vignettes included a wound photograph compared with vignettes without a wound photograph, regardless of whether they correctly identified an SSI (median, 8 [interquartile range, 6-9] vs median, 8 [interquartile range, 7-9]; P < .001) but they were more likely to undertriage patients when vignettes included a wound photograph, regardless of whether they correctly identified an SSI.

Conclusions and Relevance

In a practical simulation, wound photography increased specificity and surgeon confidence, but worsened sensitivity for detection of SSIs. Remote evaluation of patient-generated wound photographs may not accurately reflect the clinical state of surgical incisions. Effective widespread implementation of remote postoperative assessment with photography may require additional development of tools, participant training, and mechanisms to verify image quality.

Highlighting the gaps in quantifying the economic burden of surgical site infections associated with antimicrobial-resistant bacteria

Antibiotics are the pillar of surgery from prophylaxis to treatment; any failure is potentially a leading cause for increased morbidity and mortality. Robust data on the burden of SSI especially those due to antimicrobial resistance (AMR) show variable rates between countries and geographical regions but accurate estimates of the incidence of surgical site infections (SSI) due to AMR and its related global economic impact are yet to be determined. Quantifying the burden of SSI treatment is an incentive to sensitize governments, healthcare systems, and the society to invest in quality improvement and sustainable development. However in the absence of a unified epidemiologically sound infection definition of SSI and a well-designed global surveillance system, the end result is a lack of accurate and reliable data that limits the comparability of estimates between countries and the possibility of tracking changes to inform healthcare professionals about the appropriateness of implemented infection prevention and control strategies. This review aims to highlight the reported gaps in surveillance methods, epidemiologic data, and evidence-based SSI prevention practices and in the methodologies undertaken for the evaluation of the economic burden of SSI associated with AMR bacteria. If efforts to tackle this problem are taken in isolation without a global alliance and data is still lacking generalizability and comparability, we may see the future as a race between the global research efforts for the advancement in surgery and the global alarming reports of the increased incidence of antimicrobial-resistant pathogens threatening to undermine any achievement.

Understanding determinants of infection control practices in surgery: the role of shared ownership and team hierarchy

Background

Despite a large literature on surgical site infection (SSI), the determinants of prevention behaviours in surgery remain poorly studied. Understanding key social and contextual components of surgical staff behaviour may help to design and implement infection control (IC) improvement interventions in surgery.

Methods

Qualitative semi-structured interviews were conducted with surgeons (n = 8), nurses (n = 5) theatre personnel (n = 3), and other healthcare professionals involved in surgery (n = 4) in a 1500-bed acute care London hospital group. Participants were approached through established mailing lists and snowball sampling. Interviews were recorded and transcribed verbatim. Transcripts were coded and analysed thematically using a constant comparative approach.

Results

IC behaviour of surgical staff was governed by factors at individual, team, and wider hospital level. IC practices were linked to the perceived risk of harm caused by an SSI more than the development of an SSI alone. Many operating room participants saw SSI prevention as a team responsibility. The sense of ownership over SSI occurence was closely tied to how preventable staff perceived infections to be, with differences observed between clean and contaminated surgery. However, senior surgeons claimed personal accountability for rates despite feeling SSIs are often not preventable. Hierarchy impacted on behaviour in different ways depending on whether it was within or between professional categories. One particular knowledge gap highlighted was the lack of awareness regarding criteria for SSI diagnosis.

Conclusions

To influence IC behaviours in surgery, interventions need to consider the social team structure and shared ownership of the clinical outcome in order to increase the awareness in specialties where SSIs are not seen as serious complications.

Electronic supplementary material

The online version of this article (10.1186/s13756-019-0565-8) contains supplementary material, which is available to authorized users.

Development of case vignettes for assessment of the inter-rater variability of national validation teams for the point prevalence survey of healthcare-associated infections and antimicrobial use in European acute care hospitals

BACKGROUND

In 2016-17 the European Centre for Disease Prevention and Control (ECDC) organized the second point prevalence survey (PPS) of healthcare-associated infections (HCAIs) and antimicrobial use in European acute care hospitals. This survey included a validation study to maximize the accuracy of case identification and classification.

AIM

ECDC developed case vignettes to assess the performance of the national validation teams.

METHODS

Case vignettes were developed by two medical doctors with experience in the management of HCAIs and antimicrobial stewardship. The case vignettes were based on actual clinical cases. The distribution of HCAIs among the case vignettes reflected the distribution of HCAIs in the previous PPS. All case vignettes were pilot-tested by three expert raters. Agreement among the expert raters was measured using kappa statistics.

FINDINGS

Sixty case vignettes were developed. Twenty-nine of them were HCAI cases and 31 were cases without an HCAI. The inter-rater reliability using kappa statistics was 0.78 for the presence of HCAI and 0.89 for the antimicrobial use, respectively.

CONCLUSION

The agreement between the expert raters was very good for antimicrobial use and good for the presence of HCAI. Case vignettes can be a tool to support standardization of surveillance, improving the validity and comparability of the data.

Retrospective analysis of the Photo at Discharge scheme and readmission for surgical site infection following coronary artery bypass graft surgery

Background

Surgical site infection (SSI) is a costly and devastating complication of surgery. Many cardiac SSIs develop after the patient leaves hospital, but evidence demonstrating the benefit of patient/carer involvement in the process of monitoring and promptly identifying SSI post-discharge is limited. This study estimates the probability of readmission for SSI for coronary artery bypass graft (CABG) patients receiving the Photo at Discharge (PaD).

Methods

Trained personnel undertook continuous, prospective SSI surveillance using Public Health England protocol between January 2013 and December 2016. Baseline covariables were collected for 1747 CABG-only procedures. As a quasi-randomised design, we adjusted for non-random PaD assignment using retrospective propensity score (PS)-matching based on 12 variables of interest, assessed whether the model had been adequately specified and performed an outcomes analysis.

Results

A total of 568 patients with PaD were PS-matched with 568 controls. The probabilities of SSI readmission were 0.352 (2/568) and 1.761 (10/568), respectively. The difference in risk of readmission for SSI was significant (relative risk = 0.2, 95% confidence interval = 0.04-0.91; P = 0.04).

Conclusion

Findings from this single-centre observation study suggest the PaD is associated with a reduction in CABG readmission for SSI and a further study is warranted to verify the efficacy of this strategy.

Structure, Process, and Outcome Quality of Surgical Site Infection Surveillance in Switzerland

OBJECTIVE

To assess the structure and quality of surveillance activities and to validate outcome detection in the Swiss national surgical site infection (SSI) surveillance program.

DESIGN

Countrywide survey of SSI surveillance quality.

SETTING

147 hospitals or hospital units with surgical activities in Switzerland.

METHODS

Site visits were conducted with on-site structured interviews and review of a random sample of 15 patient records per hospital: 10 from the entire data set and 5 from a subset of patients with originally reported infection. Process and structure were rated in 9 domains with a weighted overall validation score, and sensitivity, specificity, positive predictive value, and negative predictive value were calculated for the identification of SSI.

RESULTS

Of 50 possible points, the median validation score was 35.5 (range, 16.25–48.5). Public hospitals (P<.001), hospitals in the Italian-speaking region of Switzerland (P=.021), and hospitals with longer participation in the surveillance (P=.018) had higher scores than others. Domains that contributed most to lower scores were quality of chart review and quality of data extraction. Of 49 infections, 15 (30.6%) had been overlooked in a random sample of 1,110 patient records, accounting for a sensitivity of 69.4% (95% confidence interval [CI], 54.6%–81.7%), a specificity of 99.9% (95% CI, 99.5%–100%), a positive predictive value of 97.1% (95% CI, 85.1%–99.9%), and a negative predictive value of 98.6% (95% CI, 97.7%–99.2%).

CONCLUSIONS

Irrespective of a well-defined surveillance methodology, there is a wide variation of SSI surveillance quality. The quality of chart review and the accuracy of data collection are the main areas for improvement.

Infect Control Hosp Epidemiol 2017;38:1172–1181

Assessment of the accuracy and consistency in the application of standardized surveillance definitions: A summary of the American Journal of Infection Control and National Healthcare Safety Network case studies, 2010-2016

BACKGROUND

The Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) surveillance definitions are the most widely used criteria for health care-associated infection (HAI) surveillance. NHSN participants agree to conduct surveillance in accordance with the NHSN protocol and criteria. To assess the application of these standardized surveillance specifications and offer infection preventionists (IPs) opportunities for ongoing education, a series of case studies, with questions related to NHSN definitions and criteria were published.

METHODS

Beginning in 2010, case studies with multiple-choice questions based on standard surveillance criteria and protocols were written and published in the American Journal of Infection Control with a link to an online survey. Participants anonymously submitted their responses before receiving the correct answers.

RESULTS

The 22 case studies had 7,950 respondents who provided 27,790 responses to 75 questions during the first 6 years. Correct responses were selected 62.5% of the time (17,376 out of 27,290), but ranged widely (16%-87%). In a subset analysis, 93% of participants self-identified as IPs (3,387 out of 3,640), 4.5% were public health professionals (163 out of 3,640), and 2.5% were physicians (90 out of 3,640). IPs responded correctly (62%) more often than physicians (55%) (P = .006).

CONCLUSIONS

Among a cohort of voluntary participants, accurate application of surveillance criteria to case studies was suboptimal, highlighting the need for continuing education, competency development, and auditing.

First Results of the Swiss National Surgical Site Infection Surveillance Program: Who Seeks Shall Find

OBJECTIVES

To report on the results of the Swiss national surgical site infection (SSI) surveillance program, including temporal trends, and to describe methodological characteristics that may influence SSI rates

DESIGN

Countrywide survey of SSI over a 4-year period. Analysis of prospectively collected data including patient and procedure characteristics as well as aggregated SSI rates stratified by risk categories, type of SSI, and time of diagnosis. Temporal trends were analyzed using stepwise multivariate logistic regression models with adjustment of the effect of the duration of participation in the surveillance program for confounding factors.

SETTING

The study included 164 Swiss public and private hospitals with surgical activities.

RESULTS

From October 2011 to September 2015, a total of 187,501 operations performed in this setting were included. Cumulative SSI rates varied from 0.9% for knee arthroplasty to 14.4% for colon surgery. Postdischarge follow-up was completed in &gt;90% of patients at 1 month for surgeries without an implant and in &gt;80% of patients at 12 months for surgeries with an implant. High rates of SSIs were detected postdischarge, from 20.7% in colon surgeries to 93.3% in knee arthroplasties. Overall, the impact of the duration of surveillance was significantly and independently associated with a decrease in SSI rates in herniorraphies and C-sections but not for the other procedures. Nevertheless, some hospitals observed significant decreases in their rates for various procedures.

CONCLUSIONS

Intensive post-discharge surveillance may explain high SSI rates and cause artificial differences between programs. Surveillance per se, without structured and mandatory quality improvement efforts, may not produce the expected decrease in SSI rates.

Infect Control Hosp Epidemiol 2017;38:697–704

Evaluation of Nosocomial Infection Control Programs in health services

OBJECTIVES:

to evaluate the Nosocomial Infection Control Programs in hospital institutions regarding structure and process indicators.

METHOD:

this is a descriptive, exploratory and quantitative study conducted in 2013. The study population comprised 13 Nosocomial Infection Control Programs of health services in a Brazilian city of the state of São Paulo. Public domain instruments available in the Manual of Evaluation Indicators of Nosocomial Infection Control Practices were used.

RESULTS:

The indicators with the highest average compliance were "Evaluation of the Structure of the Nosocomial Infection Control Programs" (75%) and "Evaluation of the Epidemiological Surveillance System of Nosocomial Infection" (82%) and those with the lowest mean compliance scores were "Evaluation of Operational Guidelines" (58.97%) and "Evaluation of Activities of Control and Prevention of Nosocomial Infection" (60.29%).

CONCLUSION:

The use of indicators identified that, despite having produced knowledge about prevention and control of nosocomial infections, there is still a large gap between the practice and the recommendations.

Differences in identifying healthcare associated infections using clinical vignettes and the influence of respondent characteristics: a cross-sectional survey of Australian infection prevention staff

Background

Australia has commenced public reporting and benchmarking of healthcare associated infections (HAIs), despite not having a standardised national HAI surveillance program. Annual hospital Staphylococcus aureus bloodstream (SAB) infection rates are released online, with other HAIs likely to be reported in the future. Although there are known differences between hospitals in Australian HAI surveillance programs, the effect of these differences on reported HAI rates is not known.

Objective

To measure the agreement in HAI identification, classification, and calculation of HAI rates, and investigate the influence of differences amongst those undertaking surveillance on these outcomes.

Methods

A cross-sectional online survey exploring HAI surveillance practices was administered to infection prevention nurses who undertake HAI surveillance. Seven clinical vignettes describing HAI scenarios were included to measure agreement in HAI identification, classification, and calculation of HAI rates. Data on characteristics of respondents was also collected. Three of the vignettes were related to surgical site infection and four to bloodstream infection. Agreement levels for each of the vignettes were calculated. Using the Australian SAB definition, and the National Health and Safety Network definitions for other HAIs, we looked for an association between the proportion of correct answers and the respondents’ characteristics.

Results

Ninety-two infection prevention nurses responded to the vignettes. One vignette demonstrated 100 % agreement from responders, whilst agreement for the other vignettes varied from 53 to 75 %. Working in a hospital with more than 400 beds, working in a team, and State or Territory was associated with a correct response for two of the vignettes. Those trained in surveillance were more commonly associated with a correct response, whilst those working part-time were less likely to respond correctly.

Conclusion

These findings reveal the need for further HAI surveillance support for those working part-time and in smaller facilities. It also confirms the need to improve uniformity of HAI surveillance across Australian hospitals, and raises questions on the validity of the current comparing of national HAI SAB rates.

Matching bacteriological and medico-administrative databases is efficient for a computer-enhanced surveillance of surgical site infections: retrospective analysis of 4,400 surgical procedures in a French university hospital

OBJECTIVE

Our goal was to estimate the performance statistics of an electronic surveillance system for surgical site infections (SSIs), generally applicable in French hospitals.

METHODS

Three detection algorithms using 2 different data sources were tested retrospectively on 9 types of surgical procedures performed between January 2010 and December 2011 in the University Hospital of Nantes. The first algorithm was based on administrative codes, the second was based on bacteriological data, and the third used both data sources. For each algorithm, sensitivity, specificity, and positive and negative predictive values (PPV and NPV) were calculated. The reference method was the hospital's routine surveillance: a comprehensive review of the computerized medical charts of the patients who underwent one of the targeted procedures during the study period.

SETTING

A 3,000-bed teaching hospital in western France.

POPULATION

We analyzed 4,400 targeted surgical procedures.

RESULTS

Sensitivity results varied significantly between the three algorithms, from 25% (95% confidence interval, 17-33) when using only administrative codes to 87% (80%-93%) with the bacteriological data and 90% (85%-96%) with the combined algorithm. Fewer variations were observed for specificity (91%-98%), PPV (21%-25%), and NPV (98% to nearly 100%). Overall, performance statistics were higher for deep SSIs than for superficial infections.

CONCLUSIONS

A reliable computer-enhanced SSI surveillance can easily be implemented in French hospitals using common data sources. This should allow infection control professionals to spend more time on prevention and education duties. However, a multicenter study should be conducted to assess the generalizability of this method.