Linkage of microbiology reports and hospital discharge diagnoses for surveillance of surgical site infections

Automated Cluster Detection of Health Care-Associated Infection Based on the Multisource Surveillance of Process Data in the Area Network: Retrospective Study of Algorithm Development and Validation

BACKGROUND

The cluster detection of health care-associated infections (HAIs) is crucial for identifying HAI outbreaks in the early stages.

OBJECTIVE

We aimed to verify whether multisource surveillance based on the process data in an area network can be effective in detecting HAI clusters.

METHODS

We retrospectively analyzed the incidence of HAIs and 3 indicators of process data relative to infection, namely, antibiotic utilization rate in combination, inspection rate of bacterial specimens, and positive rate of bacterial specimens, from 4 independent high-risk units in a tertiary hospital in China. We utilized the Shewhart warning model to detect the peaks of the time-series data. Subsequently, we designed 5 surveillance strategies based on the process data for the HAI cluster detection: (1) antibiotic utilization rate in combination only, (2) inspection rate of bacterial specimens only, (3) positive rate of bacterial specimens only, (4) antibiotic utilization rate in combination + inspection rate of bacterial specimens + positive rate of bacterial specimens in parallel, and (5) antibiotic utilization rate in combination + inspection rate of bacterial specimens + positive rate of bacterial specimens in series. We used the receiver operating characteristic (ROC) curve and Youden index to evaluate the warning performance of these surveillance strategies for the detection of HAI clusters.

RESULTS

The ROC curves of the 5 surveillance strategies were located above the standard line, and the area under the curve of the ROC was larger in the parallel strategy than in the series strategy and the single-indicator strategies. The optimal Youden indexes were 0.48 (95% CI 0.29-0.67) at a threshold of 1.5 in the antibiotic utilization rate in combination-only strategy, 0.49 (95% CI 0.45-0.53) at a threshold of 0.5 in the inspection rate of bacterial specimens-only strategy, 0.50 (95% CI 0.28-0.71) at a threshold of 1.1 in the positive rate of bacterial specimens-only strategy, 0.63 (95% CI 0.49-0.77) at a threshold of 2.6 in the parallel strategy, and 0.32 (95% CI 0.00-0.65) at a threshold of 0.0 in the series strategy. The warning performance of the parallel strategy was greater than that of the single-indicator strategies when the threshold exceeded 1.5.

CONCLUSIONS

The multisource surveillance of process data in the area network is an effective method for the early detection of HAI clusters. The combination of multisource data and the threshold of the warning model are 2 important factors that influence the performance of the model.

Validation of semiautomated surgical site infection surveillance using electronic screening algorithms in 38 surgery categories

OBJECTIVE

To verify the validity of a semiautomated surgical site infection (SSI) surveillance system using electronic screening algorithms in 38 categories of surgery.

DESIGN

A cohort study for validation of semiautomated SSI surveillance system using screening algorithms.

SETTING

A 1,989-bed tertiary-care referral center in Seoul, Republic of Korea.

METHODS

A dataset of 40,516 surgical procedures in 38 categories stored in the conventional SSI surveillance registry at the Samsung Medical Center between January 2013 and December 2014 was used as the reference standard. In the semiautomated surveillance system, electronic screening algorithms flagged cases meeting at least 1 of 3 criteria: antibiotic prescription, microbial culture, and infectious disease consultation. Flagged cases were audited by infection preventionists. Analyses of sensitivity, specificity, and positive predictive value (PPV) were conducted for the semiautomated surveillance system, and its effect on reducing the workload for chart review was evaluated.

RESULTS

A total of 575 SSI events (1·42%) were identified by conventional SSI surveillance. The sensitivity of the semiautomated SSI surveillance was 96·7%, and the PPV of the screening algorithms alone was 4·1%. Semiautomated SSI surveillance reduced the chart review workload of the infection preventionists from 1,283 to 482 person hours per year (a 62·4% decrease).

CONCLUSIONS

Compared to conventional surveillance, semiautomated surveillance using electronic screening algorithms followed by chart review of selected cases can provide high-validity surveillance results and can significantly reduce the workload of infection preventionists.

Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review

OBJECTIVE

Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI.

METHODS

Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics.

RESULTS

57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances.

CONCLUSIONS

Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.

Using clinical variables to guide surgical site infection detection: a novel surveillance strategy

BACKGROUND

Surgical site infections (SSIs) are a common and expensive health care-associated infection, and are used as a health care quality benchmark. As such, SSI detection is a major focus of infection prevention programs. In an effort to improve on conventional surveillance methods, a simple algorithm for SSI detection was developed using clinical variables not traditionally included in National Healthcare Safety Network definitions.

METHODS

A case-control study was conducted among surgeries performed at the Veterans Affairs Boston Healthcare System between January 2008 and December 2009. SSI cases were matched to controls without SSI. Clinical variables (administrative, microbiological, pharmacy, radiology) were compared between the groups to determine those that best identified SSI.

RESULTS

A total of 70 SSIs were matched to 70 controls. On multivariable analysis, variables significantly associated with SSI identification were wound culture order, computed tomography scan/magnetic resonance imaging order, antibiotic order within 30 days after surgery, and application of a relevant International Classification of Disease, Ninth Revision code. Among patients with no SSI identifiers, 98% were correctly classified as having no SSI. Among patients with multiple SSI identifiers, 97.1% were correctly identified as having SSI. The area under the curve for this model was 0.87.

CONCLUSION

We have derived a novel surveillance algorithm for SSI detection with excellent operating characteristics. This algorithm could be automated to streamline infection control efforts.

Identifying colon and open reduction of fracture surgical site infections using a partially automated electronic algorithm

BACKGROUND

Algorithms leveraging electronic data may reduce manual review burden for surgical site infection (SSI) surveillance with little to no reduction in sensitivity. We developed an algorithm to identify colon and open reduction of fracture (FX) SSIs to reduce manual chart review.

METHODS

A retrospective cohort of colon and FX procedures and associated SSIs was constructed. Potential SSIs were identified by positive microbiologic cultures or administrative data for diagnosis or treatment of wound infection. Sensitivity and specificity of the algorithm were assessed. The number of charts needing review to identify 1 SSI, and the potential time-savings from the algorithm, were calculated.

RESULTS

Four hundred seventy-three colon (SSI rate = 7%) and 1081 FX (SSI rate = 3%) procedures were identified. The algorithm was 91% and 97% sensitive and 76% and 93% specific for colon and FX procedures, respectively. Overall, chart review would have been reduced by 24.3 hours per 100 procedures, decreasing the number of charts to review to identify 1 SSI from 23.9 for manual review to 3.9 with the algorithm.

CONCLUSIONS

The algorithm identified SSIs with excellent sensitivity and specificity, resulting in substantial reductions in manual chart review. This algorithm could be tailored and applied to other hospitals.

Development and validation of a simple and easy-to-employ electronic algorithm for identifying clinical methicillin-resistant Staphylococcus aureus infection

BACKGROUND

With growing demands to track and publicly report and compare infection rates, efforts to utilize automated surveillance systems are increasing. We developed and validated a simple algorithm for identifying patients with clinical methicillin-resistant Staphylococcus aureus (MRSA) infection using microbiologic and antimicrobial variables. We also estimated resource savings.

METHODS

Patients who had a culture positive for MRSA at any of 5 acute care Veterans Affairs hospitals were eligible. Clinical infection was defined on the basis of manual chart review. The electronic algorithm defined clinical MRSA infection as a positive non-sterile-site culture with receipt of MRSA-active antibiotics during the 5 days prior to or after the culture.

RESULTS

In total, 246 unique non-sterile-site cultures were included, of which 168 represented infection. The sensitivity (43.4%-95.8%) and specificity (34.6%-84.6%) of the electronic algorithm varied depending on the combination of antimicrobials included. On multivariable analysis, predictors of algorithm failure were outpatient status (odds ratio, 0.23 [95% confidence interval, 0.10-0.56]) and respiratory culture (odds ratio, 0.29 [95% confidence interval, 0.13-0.65]). The median cost was $2.43 per chart given 4.6 minutes of review time per chart.

CONCLUSIONS

Our simple electronic algorithm for detecting clinical MRSA infections has excellent sensitivity and good specificity. Implementation of this electronic system may streamline and standardize surveillance and reporting efforts.

Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review

OBJECTIVE

As more electronic health records have become available during the last decade, we aimed to uncover recent trends in use of electronically available patient data by electronic surveillance systems for healthcare associated infections (HAIs) and identify consequences for system effectiveness.

METHODS

A systematic review of published literature evaluating electronic HAI surveillance systems was performed. The PubMed service was used to retrieve publications between January 2001 and December 2011. Studies were included in the review if they accurately described what electronic data were used and if system effectiveness was evaluated using sensitivity, specificity, positive predictive value, or negative predictive value. Trends were identified by analyzing changes in the number and types of electronic data sources used.

RESULTS

26 publications comprising discussions on 27 electronic systems met the eligibility criteria. Trend analysis showed that systems use an increasing number of data sources which are either medico-administrative or clinical and laboratory-based data. Trends on the use of individual types of electronic data confirmed the paramount role of microbiology data in HAI detection, but also showed increased use of biochemistry and pharmacy data, and the limited adoption of clinical data and physician narratives. System effectiveness assessments indicate that the use of heterogeneous data sources results in higher system sensitivity at the expense of specificity.

CONCLUSIONS

Driven by the increased availability of electronic patient data, electronic HAI surveillance systems use more data, making systems more sensitive yet less specific, but also allow systems to be tailored to the needs of healthcare institutes' surveillance programs.

Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review

BACKGROUND

Traditional methodologies for healthcare-associated infection (HCAI) surveillance can be resource intensive and time consuming. As a consequence, surveillance is often limited to specific organisms or conditions. Various electronic databases exist within the healthcare setting and may be utilized to perform HCAI surveillance.

AIM

To assess the utility of electronic surveillance systems for monitoring and detecting HCAI.

METHODS

A systematic review of published literature on surveillance of HCAI was performed. Databases were searched for studies published between January 2000 and December 2011. Search terms were divided into infection, surveillance and data management terms, and combined using Boolean operators. Studies were included for review if they demonstrated or proposed the use of electronic systems for HCAI surveillance.

FINDINGS

In total, 44 studies met the inclusion criteria. For the majority of studies, emphasis was on the linkage of electronic databases to provide automated methods for monitoring infections in specific clinical settings. Twenty-one studies assessed the performance of their method with traditional surveillance methodologies or a manual reference method. Where sensitivity and specificity were calculated, these varied depending on the organism or condition being surveyed and the data sources employed.

CONCLUSIONS

The implementation of electronic surveillance was found to be feasible in many settings, with several systems fully integrated into hospital information systems and routine surveillance practices. The results of this review suggest that electronic surveillance systems should be developed to maximize the efficacy of abundant electronic data sources existing within hospitals.

Evaluation study of different strategies for detecting surgical site infections using the hospital information system at Lyon University Hospital, France

OBJECTIVE

To evaluate different strategies for detecting surgical site infections (SSIs) using different sources (notification by the surgeon, bacteriological results, antibiotic prescription, and discharge diagnosis codes).

BACKGROUND

Surveillance plays a role in reducing the risks of SSIs but the performance of case reports by surgeons is insufficient. Indirect methods of SSI detection are an alternative to increase the quality of surveillance.

METHODS

A retrospective cohort study of 446 patients operated consecutively during the first half of 2007 was set up in a 56-bed general surgery unit in Lyon University Hospital, France. Patients were followed up 30 days after intervention. Different methods of detection were established by combining different data sources. The sensitivity and specificity of these methods were calculated by using, as reference method, the manual review of the medical records.

RESULTS

The sensitivity and specificity of SSI detection were, respectively, 18.4% (95% confidence interval [CI]: 7.9-31.6) and 100% for surgeon notification; 63.2% (95% CI: 47.3-78.9) and 95.1% (95% CI: 92.9-97.1) for detection based on positive cultures; 68.4% (95% CI: 52.6-81.6) and 87.5% (95% CI: 84.3-90.7) using antibiotic prescription; 26.3% (95% CI: 13.2-42.1) and 99.5% (95% CI: 98.8-100) using discharge diagnosis codes. By combining the latter 3 sources, the sensitivity increased at 86.8% (95% CI: 76.3-97.4) and the specificity was lowered at 85.5% (95% CI: 82.1-89.0).

CONCLUSIONS

SSI detection based on the combination of data extracted automatically from the hospital information system performed well. This strategy has been implemented gradually in Lyon University Hospital.

Use of administrative data in efficient auditing of hospital-acquired surgical site infections, New York State 2009-2010

OBJECTIVE

To efficiently validate the accuracy of surgical site infection (SSI) data reported to the National Healthcare Safety Network (NHSN) by New York State (NYS) hospitals.

DESIGN

Validation study.

SETTING

176 NYS hospitals.

METHODS

NYS Department of Health staff validated the data reported to NHSN by review of a stratified sample of medical records from each hospital. The four strata were (1) SSIs reported to NHSN; (2) records with an indication of infection from diagnosis codes in administrative data but not reported to NHSN as SSIs; (3) records with discordant procedure codes in NHSN and state data sets; (4) records not in the other three strata.

RESULTS

A total of 7,059 surgical charts (6% of the procedures reported by hospitals) were reviewed. In stratum 1, 7% of reported SSIs did not meet the criteria for inclusion in NHSN and were subsequently removed. In stratum 2, 24% of records indicated missed SSIs not reported to NHSN, whereas in strata 3 and 4, only 1% of records indicated missed SSIs; these SSIs were subsequently added to NHSN. Also, in stratum 3, 75% of records were not coded for the correct NHSN procedure. Errors were highest for colon data; the NYS colon SSI rate increased by 7.5% as a result of hospital audits.

CONCLUSIONS

Audits are vital for ensuring the accuracy of hospital-acquired infection (HAI) data so that hospital HAI rates can be fairly compared. Use of administrative data increased the efficiency of identifying problems in hospitals' SSI surveillance that caused SSIs to be unreported and caused errors in denominator data.

Comparison of two computer algorithms to identify surgical site infections

BACKGROUND

Surgical site infections (SSIs), the second most common healthcare-associated infections, increase hospital stay and healthcare costs significantly. Traditional surveillance of SSIs is labor-intensive. Mandatory reporting and new non-payment policies for some SSIs increase the need for efficient and standardized surveillance methods. Computer algorithms using administrative, clinical, and laboratory data collected routinely have shown promise for complementing traditional surveillance.

METHODS

Two computer algorithms were created to identify SSIs in inpatient admissions to an urban, academic tertiary-care hospital in 2007 using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes (Rule A) and laboratory culture data (Rule B). We calculated the number of SSIs identified by each rule and both rules combined and the percent agreement between the rules. In a subset analysis, the results of the rules were compared with those of traditional surveillance in patients who had undergone coronary artery bypass graft surgery (CABG).

RESULTS

Of the 28,956 index hospital admissions, 5,918 patients (20.4%) had at least one major surgical procedure. Among those and readmissions within 30 days, the ICD-9-CM-only rule identified 235 SSIs, the culture-only rule identified 287 SSIs; combined, the rules identified 426 SSIs, of which 96 were identified by both rules. Positive and negative agreement between the rules was 36.8% and 97.1%, respectively, with a kappa of 0.34 (95% confidence interval [CI] 0.27-0.41). In the subset analysis of patients who underwent CABG, of the 22 SSIs identified by traditional surveillance, Rule A identified 19 (86.4%) and Rule B identified 13 (59.1%) cases. Positive and negative agreement between Rules A and B within these "positive controls" was 81.3% and 50.0% with a kappa of 0.37 (95% CI 0.04-0.70).

CONCLUSION

Differences in the rates of SSI identified by computer algorithms depend on sources and inherent biases in electronic data. Different algorithms may be appropriate, depending on the purpose of case identification. Further research on the reliability and validity of these algorithms and the impact of changes in reimbursement on clinician practices and electronic reporting is suggested.

Use of diagnosis codes and/or wound culture results for surveillance of surgical site infection after mastectomy and breast reconstruction

We compared surveillance of surgical site infection (SSI) after major breast surgery by using a combination of International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis codes and microbiology-based surveillance. The sensitivity of the coding algorithm for identification of SSI was 87.5%, and the sensitivity of wound culture for identification of SSI was 78.1%. Our results suggest that SSI surveillance can be reliably performed using claims data.

Surveillance of selected post-caesarean infections based on electronic registries: validation study including post-discharge infections

The importance of surveillance of post-discharge infections has increased as a consequence of shorter hospital stay after surgical procedures. This study examined the ability of a computer-based surveillance system to identify urinary tract infections (UTIs) and postoperative wound infections (PWIs) within 30 days after caesarean section. We assessed the use of data from various electronic registries to identify patients with post-caesarean UTI and PWI classified according to a reference standard. The standard was based on information from medical records and self-reported data (questionnaire) using modified Centers for Disease Control and Prevention definitions. The sensitivity of the computer system in detecting UTI diagnosed during hospital stay, readmission or at visits to hospital outpatient clinics was 80.0%; the specificity was 99.9%. For post-discharge UTIs diagnosed outside the hospital, sensitivity and specificity were 76.3% and 99.9%, respectively. For PWIs diagnosed in hospital and post-discharge outside hospital, sensitivities were 77.1% and 68.9%, and the specificities 99.5% and 98.2%. We conclude that a computer-based surveillance system may identify in-hospital infections and post-discharge infections with a relatively high sensitivity and excellent specificity.

Estimating the annual burden of health care-associated infections in Finnish adult acute care hospitals

BACKGROUND

We estimated the burden of health care-associated infections (HAIs) occurring in Finnish adult acute care hospitals using national hospitalization data and estimates of HAI based on a recent national prevalence survey.

METHODS

A total of 7531 non-HAI patients and 703 HAI patients (8.5%) identified in the prevalence survey were included in the study. Using the patients' national identity numbers and the prevalence survey date, we obtained data on hospitalizations, including discharge diagnoses from the National Hospital Discharge Registry (NHDR), and the dates and causes of death from the National Population Information System. We converted the prevalence of HAI into incidence using the Rhame-Sudderth formula, assessed the 28-day case fatality of the HAI patients, and then extrapolated the annual estimates of HAI burden from the total number of hospitalizations in adult acute care hospitals in 2005 (n = 804,456). We also assessed the sensitivity of the NHDR diagnoses in identifying HAIs.

RESULTS

The estimated incidence of HAIs was 5.7% (95% confidence interval = 5.0% to 6.5%), and the 28-day case fatality was 9.8%. Thus, >8500 hospitalizations per million population annually would result in at least 1 HAI and approximately 270 HAI-associated deaths within 28 days. The sensitivity of the NHDR diagnoses was 34% (range by infection type, 0% to 67%).

CONCLUSION

Our disease burden estimates can be used in health care planning and resource allocation. The NHDR was not a reliable source for case finding of HAIs.

Validity of electronic surveillance systems: a systematic review

Electronic surveillance that utilises information held in databases is more efficient than conventional infection surveillance methods. Validity is not well-defined, however. We systematically reviewed studies comparing the utility of electronic and conventional surveillance methods. Publications were identified using Medline (1980-2007) and bibliographic review. The sensitivity and specificity of electronic compared with conventional surveillance was reported. Twenty-four studies were included. Six studies reported that nosocomial infections could be detected utilising microbiology data alone with good overall sensitivity (range: 63-91%) and excellent specificity (range: 87 to >99%). Two studies used three laboratory-based algorithms for the detection of infection outbreaks yielding variable utility measures (sensitivity, range: 43-91%; specificity, range: 67-86%). Seven studies using only administrative data including discharge coding (International Classification of Diseases, 9th edn, Clinical Modification) and pharmacy data claimed databases had good sensitivity (range: 59-96%) and excellent specificity (range: 95 to >99%) in detecting nosocomial infections. Six studies combined both laboratory and administrative data for a range of infections, and overall had higher sensitivity (range: 71-94%) but lower specificity (range: 47 to >99%) than with use of either alone. Three studies evaluated community-acquired infections with variable results. Electronic surveillance has moderate to excellent utility compared with conventional methods for nosocomial infections. Future studies are needed to refine electronic algorithms further, especially with community-onset infections.

Economics of infection control surveillance technology: cost-effective or just cost?

BACKGROUND

Previous studies have suggested that informatics tools, such as automated alert and decision support systems, may increase the efficiency and quality of infection control surveillance. However, little is known about the cost-effectiveness of these tools.

METHODS

We focus on 2 types of economic analyses that have utility in assessing infection control interventions (cost-effectiveness analysis and business-case analysis) and review the available literature on the economics of computerized infection control surveillance systems.

RESULTS

Previous studies on the effectiveness of computerized infection control surveillance have been limited to assessments of whether these tools increase the sensitivity and specificity of surveillance over traditional methods. Furthermore, we identified only 2 studies that assessed the costs associated with computerized infection control surveillance. Thus, it remains unknown whether computerized infection control surveillance systems are cost-effective and whether use of these systems improves patient outcomes.

CONCLUSION

The existing data are insufficient to allow for a summary conclusion on the cost-effectiveness of infection control surveillance technology. All future studies of computerized infection control surveillance systems should aim to collect outcomes and economic data to inform decision making and assist hospitals with completing business-cases analyses.