Differences in identifying healthcare associated infections using clinical vignettes and the influence of respondent characteristics: a cross-sectional survey of Australian infection prevention staff

Clinical Adjudication of Hemodialysis Catheter-Related Bloodstream Infections: Findings from the REDUCCTION Trial

Key Points

The inter-rater reliability of reporting hemodialysis catheter-related infectious events between site investigators and trial adjudicators in Australia and New Zealand was substantial. The high concordance level in reporting catheter infections improves confidence in using site-level bacteremia rates as a clinical metric for quality benchmarking and future pragmatic clinical trials. A rigorous adjudication protocol may not be needed if clearly defined criteria to ascertain catheter-associated bacteremia are used.

Background

Hemodialysis catheter-related bloodstream infection (HD-CRBSI) are a significant source of morbidity and mortality among dialysis patients, but benchmarking remains difficult because of varying definitions of HD-CRBSI. This study explored the effect of clinical adjudication process on HD-CRBSI reporting.

Methods

The REDUcing the burden of Catheter ComplicaTIOns: a National approach trial implemented an evidence-based intervention bundle using a stepped-wedge design to reduce HD-CRBSI rates in 37 Australian kidney services. Six New Zealand services participated in an observational capacity. Adult patients with a new hemodialysis catheter between December 2016 and March 2020 were included. HD-CRBSI events reported were compared with the adjudicated outcomes using the end point definition and adjudication processes of the REDUcing the burden of Catheter ComplicaTIOns: a National approach trial. The concordance level was estimated using Gwet agreement coefficient (AC1) adjusted for service-level effects and implementation tranches (Australia only), with the primary outcome being the concordance of confirmed HD-CRBSI.

Results

A total of 744 hemodialysis catheter-related infectious events were reported among 7258 patients, 12,630 catheters, and 1.3 million catheter-exposure days. The majority were confirmed HD-CRBSI, with 77.9% agreement and substantial concordance (AC1=0.77; 95% confidence interval [CI], 0.73 to 0.81). Exit site infections have the highest concordance (AC1=0.85; 95% CI, 0.78 to 0.91); the greatest discordance was in events classified as other (AC1=0.33; 95% CI, 0.16 to 0.49). The concordance of all hemodialysis catheter infectious events remained substantial (AC1=0.80; 95% CI, 0.76 to 0.83) even after adjusting for the intervention tranches in Australia and overall service-level clustering.

Conclusions

There was a substantial level of concordance in overall and service-level reporting of confirmed HD-CRBSI. A standardized end point definition of HD-CRBSI resulted in comparable hemodialysis catheter infection rates in Australian and New Zealand kidney services. Consistent end point definition could enable reliable benchmarking outside clinical trials without the need for independent clinical adjudication.

The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients

BACKGROUND

Continuous surveillance for healthcare-associated infections such as central venous catheter-related bloodstream infections (CVC-BSI) is crucial for prevention. However, traditional surveillance methods are resource-intensive and prone to bias. This study aimed to develop and validate fully-automated surveillance algorithms for CVC-BSI.

METHODS

Two algorithms were developed using electronic health record data from 1000 admissions with a positive blood culture (BCx) at Karolinska University Hospital from 2017: (1) Combining microbiological findings in BCx and CVC cultures with BSI symptoms; (2) Only using microbiological findings. These algorithms were validated in 5170 potential CVC-BSI-episodes from all admissions in 2018-2019, and results extrapolated to all potential CVC-BSI-episodes within this period (n = 181,354). The reference standard was manual record review according to ECDC's definition of microbiologically confirmed CVC-BSI (CRI3-CVC).

RESULTS

In the potential CVC-BSI-episodes, 51 fulfilled ECDC's definition and the algorithms identified 47 and 49 episodes as CVC-BSI, respectively. Both algorithms performed well in assessing CVC-BSI. Overall, algorithm 2 performed slightly better with in the total period a sensitivity of 0.880 (95%-CI 0.783-0.959), specificity of 1.000 (95%-CI 0.999-1.000), PPV of 0.918 (95%-CI 0.833-0.981) and NPV of 1.000 (95%-CI 0.999-1.000). Incidence according to the reference and algorithm 2 was 0.33 and 0.31 per 1000 in-patient hospital-days, respectively.

CONCLUSIONS

Both fully-automated surveillance algorithms for CVC-BSI performed well and could effectively replace manual surveillance. The simpler algorithm, using only microbiology data, is suitable when BCx testing adheres to recommendations, otherwise the algorithm using symptom data might be required. Further validation in other settings is necessary to assess the algorithms' generalisability.

Variability in Serious Safety Event Classification among Children's Hospitals: A Measure for Comparison?

Introduction

Hospitals have no standard for measuring comparative rates of serious safety events (SSE). A pediatric hospital safety collaborative has used a common definition and measurement system to classify SSE and calculate a serious safety event rate. An opportunity exists to evaluate the use of this measurement system.

Methods

A web-based survey utilizing 7 case vignettes was sent to 132 network hospitals to assess agreement in classifying the vignettes as SSEs. Respondents classified the vignettes according to the taxonomy used at their respective organizations for deviations and SSE classification.

Results

Of the 82 respondents, 67 (82%) utilized the same SSE classification system. Respondents did not assess deviations for 2 of the 7 vignettes, which had clear deviations. Of the remaining 5 vignettes, 3 had a substantial agreement of deviation (>85%, Gwet's AC ≥ 0.68), and 2 had fair agreement (<70%, Gwet's AC ≤ 0.39). Four of the 7 vignettes had a substantial agreement on SSE classification (>80%; Gwet's AC ≥ 0.80), and 3 had slight to moderate agreement (<70%, Gwet's AC ≤ 0.78).

Conclusions

Results demonstrated agreement and variability in determining deviation and SSE classification in the 7 vignettes. Although the SSE methodology and metric used by participant pediatric hospitals yields generally similar review results, one must be cautious in using the SSE rate to compare patient safety outcomes across different hospitals.

The accuracy of fully automated algorithms for surveillance of healthcare-associated urinary tract infections in hospitalized patients

BACKGROUND

Surveillance for healthcare-associated infections such as healthcare-associated urinary tract infections (HA-UTI) is important for directing resources and evaluating interventions. However, traditional surveillance methods are resource-intensive and subject to bias.

AIM

To develop and validate a fully automated surveillance algorithm for HA-UTI using electronic health record (EHR) data.

METHODS

Five algorithms were developed using EHR data from 2979 admissions at Karolinska University Hospital from 2010 to 2011: (1) positive urine culture (UCx); (2) positive UCx + UTI codes (International Statistical Classification of Diseases and Related Health Problems, 10^th revision); (3) positive UCx + UTI-specific antibiotics; (4) positive UCx + fever and/or UTI symptoms; (5) algorithm 4 with negation for fever without UTI symptoms. Natural language processing (NLP) was used for processing free-text medical notes. The algorithms were validated in 1258 potential UTI episodes from January to March 2012 and results extrapolated to all UTI episodes within this period (N = 16,712). The reference standard for HA-UTIs was manual record review according to the European Centre for Disease Prevention and Control (and US Centers for Disease Control and Prevention) definitions by trained healthcare personnel.

FINDINGS

Of the 1258 UTI episodes, 163 fulfilled the ECDC HA-UTI definition and the algorithms classified 391, 150, 189, 194, and 153 UTI episodes, respectively, as HA-UTI. Algorithms 1, 2, and 3 had insufficient performances. Algorithm 4 achieved better performance and algorithm 5 performed best for surveillance purposes with sensitivity 0.667 (95% confidence interval: 0.594-0.733), specificity 0.997 (0.996-0.998), positive predictive value 0.719 (0.624-0.807) and negative predictive value 0.997 (0.996-0.997).

CONCLUSION

A fully automated surveillance algorithm based on NLP to find UTI symptoms in free-text had acceptable performance to detect HA-UTI compared to manual record review. Algorithms based on administrative and microbiology data only were not sufficient.

Designing Surveillance of Healthcare-Associated Infections in the Era of Automation and Reporting Mandates

Surveillance and feedback of infection rates to clinicians and other stakeholders is a cornerstone of healthcare-associated infection (HAI) prevention programs. In addition, HAIs are increasingly included in public reporting and payment mandates. Conventional manual surveillance methods are resource intensive and lack standardization. Developments in information technology have propelled a movement toward the use of standardized and semiautomated methods.When developing automated surveillance systems, several strategies can be chosen with regard to the degree of automation and standardization and the definitions used. Yet, the advantages of highly standardized surveillance may come at the price of decreased clinical relevance and limited preventability. The choice among (automated) surveillance approaches, therefore, should be guided by the intended aim and scale of surveillance (eg, research, in-hospital quality improvement, national surveillance, or pay-for-performance mandates), as this choice dictates subsequent methods, important performance characteristics, and suitability of the data generated for the different applications.

Healthcare-associated infections in Australia: tackling the 'known unknowns'

Australia does not have a national healthcare-associated infection (HAI) surveillance program. Without national surveillance, we do not understand the burden of HAIs, nor can we accurately assess the effects of national infection prevention initiatives. Recent research has demonstrated disparity between existing jurisdictional-based HAI surveillance activity while also identifying broad key stakeholder support for the establishment of a national program. A uniform surveillance program will also address growing concerns about hospital performance measurements and enable public reporting of hospital data.

An interactive nomogram to predict healthcare-associated infections in ICU patients: A multicenter study in GuiZhou Province, China

Objective

To develop and validate an interactive nomogram to predict healthcare-associated infections (HCAIs) in the intensive care unit (ICU).

Methods

A multicenter retrospective study was conducted to review 2017 data from six hospitals in Guizhou Province, China. A total of 1,782 ICU inpatients were divided into either a training set (n = 1,189) or a validation set (n = 593). The patients’ demographic characteristics, basic clinical features from the previous admission, and their need for bacterial culture during the current admission were extracted from electronic medical records of the hospitals to predict HCAI. Univariate and multivariable analyses were used to identify independent risk factors of HCAI in the training set. The multivariable model’s performance was evaluated in both the training set and the validation set, and an interactive nomogram was constructed according to multivariable regression model. Moreover, the interactive nomogram was used to predict the possibility of a patient developing an HCAI based on their prior admission data. Finally, the clinical usefulness of the interactive nomogram was estimated by decision analysis using the entire dataset.

Results

The nomogram model included factor development (local economic development levels), length of stay (LOS; days of hospital stay), fever (days of persistent fever), diabetes (history of diabetes), cancer (history of cancer) and culture (the need for bacterial culture). The model showed good calibration and discrimination in the training set [area under the curve (AUC), 0.871; 95% confidence interval (CI), 0.848–0.894] and in the validation set (AUC, 0.862; 95% CI, 0.829–0.895). The decision curve demonstrated the clinical usefulness of our interactive nomogram.

Conclusions

The developed interactive nomogram is a simple and practical instrument for quantifying the individual risk of HCAI and promptly identifying high-risk patients.

Establishing the prevalence of healthcare-associated infections in Australian hospitals: protocol for the Comprehensive Healthcare Associated Infection National Surveillance (CHAINS) study

INTRODUCTION

A healthcare-associated infection (HAI) data point prevalence study (PPS) conducted in 1984 in Australian hospitals estimated the prevalence of HAI to be 6.3%. Since this time, there have been no further national estimates undertaken. In the absence of a coordinated national surveillance programme or regular PPS, there is a dearth of national HAI data to inform policy and practice priorities.

METHODS AND ANALYSIS

A national HAI PPS study will be undertaken based on the European Centres for Disease Control method. Nineteen public acute hospitals will participate. A standardised algorithm will be used to detect HAIs in a two-stage cluster design, random sample of adult inpatients in acute wards and all intensive care unit patients. Data from each hospital will be collected by two trained members of the research team. We will estimate the prevalence of HAIs, invasive device use, single room placement and deployment of transmission-based precautions.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the Alfred Health Human Research Ethics Committee (HREC/17/Alfred/203) via the National Mutual Assessment. A separate approval was obtained from the Tasmanian Health and Medical Human Research Committee (H0016978) for participating Tasmanian hospitals. Findings will be disseminated in individualised participating hospital reports, peer-reviewed publications and conference presentations.

Electronic surveillance and using administrative data to identify healthcare associated infections

PURPOSE OF REVIEW

Traditional surveillance of healthcare associated infections (HCAI) is time consuming and error-prone. We have analysed literature of the past year to look at new developments in this field. It is divided into three parts: new algorithms for electronic surveillance, the use of administrative data for surveillance of HCAI, and the definition of new endpoints of surveillance, in accordance with an automatic surveillance approach.

RECENT FINDINGS

Most studies investigating electronic surveillance of HCAI have concentrated on bloodstream infection or surgical site infection. However, the lack of important parameters in hospital databases can lead to misleading results. The accuracy of administrative coding data was poor at identifying HCAI. New endpoints should be defined for automatic detection, with the most crucial step being to win clinicians' acceptance.

SUMMARY

Electronic surveillance with conventional endpoints is a successful method when hospital information systems implemented key changes and enhancements. One requirement is the access to systems for hospital administration and clinical databases.Although the primary source of data for HCAI surveillance is not administrative coding data, these are important components of a hospital-wide programme of automated surveillance. The implementation of new endpoints for surveillance is an approach which needs to be discussed further.

Assessment of the accuracy and consistency in the application of standardized surveillance definitions: A summary of the American Journal of Infection Control and National Healthcare Safety Network case studies, 2010-2016

BACKGROUND

The Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) surveillance definitions are the most widely used criteria for health care-associated infection (HAI) surveillance. NHSN participants agree to conduct surveillance in accordance with the NHSN protocol and criteria. To assess the application of these standardized surveillance specifications and offer infection preventionists (IPs) opportunities for ongoing education, a series of case studies, with questions related to NHSN definitions and criteria were published.

METHODS

Beginning in 2010, case studies with multiple-choice questions based on standard surveillance criteria and protocols were written and published in the American Journal of Infection Control with a link to an online survey. Participants anonymously submitted their responses before receiving the correct answers.

RESULTS

The 22 case studies had 7,950 respondents who provided 27,790 responses to 75 questions during the first 6 years. Correct responses were selected 62.5% of the time (17,376 out of 27,290), but ranged widely (16%-87%). In a subset analysis, 93% of participants self-identified as IPs (3,387 out of 3,640), 4.5% were public health professionals (163 out of 3,640), and 2.5% were physicians (90 out of 3,640). IPs responded correctly (62%) more often than physicians (55%) (P = .006).

CONCLUSIONS

Among a cohort of voluntary participants, accurate application of surveillance criteria to case studies was suboptimal, highlighting the need for continuing education, competency development, and auditing.

Novel application of a discrete choice experiment to identify preferences for a national healthcare-associated infection surveillance programme: a cross-sectional study

OBJECTIVE

To identify key stakeholder preferences and priorities when considering a national healthcare-associated infection (HAI) surveillance programme through the use of a discrete choice experiment (DCE).

SETTING

Australia does not have a national HAI surveillance programme. An online web-based DCE was developed and made available to participants in Australia.

PARTICIPANTS

A sample of 184 purposively selected healthcare workers based on their senior leadership role in infection prevention in Australia.

PRIMARY AND SECONDARY OUTCOMES

A DCE requiring respondents to select 1 HAI surveillance programme over another based on 5 different characteristics (or attributes) in repeated hypothetical scenarios. Data were analysed using a mixed logit model to evaluate preferences and identify the relative importance of each attribute.

RESULTS

A total of 122 participants completed the survey (response rate 66%) over a 5-week period. Excluding 22 who mismatched a duplicate choice scenario, analysis was conducted on 100 responses. The key findings included: 72% of stakeholders exhibited a preference for a surveillance programme with continuous mandatory core components (mean coefficient 0.640 (p<0.01)), 65% for a standard surveillance protocol where patient-level data are collected on infected and non-infected patients (mean coefficient 0.641 (p<0.01)), and 92% for hospital-level data that are publicly reported on a website and not associated with financial penalties (mean coefficient 1.663 (p<0.01)).

CONCLUSIONS

The use of the DCE has provided a unique insight to key stakeholder priorities when considering a national HAI surveillance programme. The application of a DCE offers a meaningful method to explore and quantify preferences in this setting.