Validation of automated sepsis surveillance based on the Sepsis-3 clinical criteria against physician record review in a general hospital population: observational study using electronic health records data

Development and Prospective Implementation of a Large Language Model based System for Early Sepsis Prediction

Sepsis is a dysregulated host response to infection with high mortality and morbidity. Early detection and intervention have been shown to improve patient outcomes, but existing computational models relying on structured electronic health record data often miss contextual information from unstructured clinical notes. This study introduces COMPOSER-LLM, an open-source large language model (LLM) integrated with the COMPOSER model to enhance early sepsis prediction. For high-uncertainty predictions, the LLM extracts additional context to assess sepsis-mimics, improving accuracy. Evaluated on 2,500 patient encounters, COMPOSER-LLM achieved a sensitivity of 72.1%, positive predictive value of 52.9%, F-1 score of 61.0%, and 0.0087 false alarms per patient hour, outperforming the standalone COMPOSER model. Prospective validation yielded similar results. Manual chart review found 62% of false positives had bacterial infections, demonstrating potential clinical utility. Our findings suggest that integrating LLMs with traditional models can enhance predictive performance by leveraging unstructured data, representing a significant advance in healthcare analytics.

Population Sepsis Incidence, Mortality, and Trends in Hong Kong Between 2009 and 2018 Using Clinical and Administrative Data

BACKGROUND

Sepsis surveillance using electronic health record (EHR)-based data may provide more accurate epidemiologic estimates than administrative data, but experience with this approach to estimate population-level sepsis burden is lacking.

METHODS

This was a retrospective cohort study including all adults admitted to publicly funded hospitals in Hong Kong between 2009 and 2018. Sepsis was defined as clinical evidence of presumed infection (clinical cultures and treatment with antibiotics) and concurrent acute organ dysfunction (≥2-point increase in baseline Sequential Organ Failure Assessment [SOFA] score). Trends in incidence, mortality, and case fatality risk (CFR) were modeled by exponential regression. Performance of the EHR-based definition was compared with 4 administrative definitions using 500 medical record reviews.

RESULTS

Among 13 540 945 hospital episodes during the study period, 484 541 (3.6%) had sepsis by EHR-based criteria with 22.4% CFR. In 2018, age- and sex-adjusted standardized sepsis incidence was 756 per 100 000 (relative change: +2.8%/y [95% CI: 2.0%-3.7%] between 2009 and 2018) and standardized sepsis mortality was 156 per 100 000 (relative change: +1.9%/y; 95% CI: .9%-2.8%). Despite decreasing CFR (relative change: -0.5%/y; 95% CI: -1.0%, -.1%), sepsis accounted for an increasing proportion of all deaths (relative change: +3.9%/y; 95% CI: 2.9%-4.8%). Medical record reviews demonstrated that the EHR-based definition more accurately identified sepsis than administrative definitions (area under the curve [AUC]: .91 vs .52-.55; P < .001).

CONCLUSIONS

An objective EHR-based surveillance definition demonstrated an increase in population-level standardized sepsis incidence and mortality in Hong Kong between 2009 and 2018 and was much more accurate than administrative definitions. These findings demonstrate the feasibility and advantages of an EHR-based approach for widescale sepsis surveillance.

A dynamic customized electronic health record rule based clinical decision support tool for standardized adult intensive care metrics

Objectives

Many routine patient care items should be reviewed at least daily for intensive care unit (ICU) patients. These items are often incompletely performed, and dynamic clinical decision support tools (CDSTs) may improve attention to these daily items. We sought to evaluate the accuracy of institutionalized electronic health record (EHR) based custom dynamic CDST to support 22 ICU rounding quality metrics across 7 categories (hypoglycemia, venothromboembolism prophylaxis, stress ulcer prophylaxis, mechanical ventilation, sedation, nutrition, and catheter removal).

Design

The dynamic CDST evaluates patient characteristics and patient orders, then identifies gaps between active interventions and conditions with recommendations of evidence based clinical practice guidelines across 22 areas of care for each patient. The results of the tool prompt clinicians to address any identified care gaps. We completed a confusion matrix to assess the sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) of the dynamic CDST and the individual metrics.

Setting

Tertiary academic medical center and community hospital ICUs.

Subject

Customized Clinical Decision Support Tool.

Measurements and Main Results

The metrics were evaluated 1421 times over 484 patients. The overall accuracy of the entire dynamic CDST is 0.979 with a sensitivity of 0.979, specificity of 0.978, PPV 0.969, and NPV 0.986.

Conclusions

A customized, EHR based dynamic CDST can be highly accurate. Integrating a comprehensive dynamic CDST into existing workflows could improve attention and actions related to routine ICU quality metrics.

Automated Identification of Postoperative Infections to Allow Prediction and Surveillance Based on Electronic Health Record Data: Scoping Review

BACKGROUND

Postoperative infections remain a crucial challenge in health care, resulting in high morbidity, mortality, and costs. Accurate identification and labeling of patients with postoperative bacterial infections is crucial for developing prediction models, validating biomarkers, and implementing surveillance systems in clinical practice.

OBJECTIVE

This scoping review aimed to explore methods for identifying patients with postoperative infections using electronic health record (EHR) data to go beyond the reference standard of manual chart review.

METHODS

We performed a systematic search strategy across PubMed, Embase, Web of Science (Core Collection), the Cochrane Library, and Emcare (Ovid), targeting studies addressing the prediction and fully automated surveillance (ie, without manual check) of diverse bacterial infections in the postoperative setting. For prediction modeling studies, we assessed the labeling methods used, categorizing them as either manual or automated. We evaluated the different types of EHR data needed for the surveillance and labeling of postoperative infections, as well as the performance of fully automated surveillance systems compared with manual chart review.

RESULTS

We identified 75 different methods and definitions used to identify patients with postoperative infections in studies published between 2003 and 2023. Manual labeling was the predominant method in prediction modeling research, 65% (49/75) of the identified methods use structured data, and 45% (34/75) use free text and clinical notes as one of their data sources. Fully automated surveillance systems should be used with caution because the reported positive predictive values are between 0.31 and 0.76.

CONCLUSIONS

There is currently no evidence to support fully automated labeling and identification of patients with infections based solely on structured EHR data. Future research should focus on defining uniform definitions, as well as prioritizing the development of more scalable, automated methods for infection detection using structured EHR data.

Validity of Routinely Reported Rutherford Scores Reported by Clinicians as Part of Daily Clinical Practice

Routinely reported structured data from the electronic health record (EHR) are frequently used for secondary purposes. However, it is unknown how valid routinely reported data are for reuse. This study aimed to assess the validity of routinely reported Rutherford scores by clinicians as an indicator for the validity of structured data in the EHR. This observational study compared clinician-reported Rutherford scores with medical record review Rutherford scores for all visits at the vascular surgery department between April 1, 2016 and December 31, 2018. Free-text fields with clinical information for all visits were extracted for the assignment of the medical record review Rutherford score, after which the agreement with the clinician-reported Rutherford score was assessed using Fleiss' Kappa. A total of 6,633 visits were included for medical record review. Substantial agreement was shown between clinician-reported Rutherford scores and medical record review Rutherford scores for the left ( k  = 0.62, confidence interval [CI]: 0.60-0.63) and right leg ( k  = 0.62, CI: 0.60-0.64). This increased to the almost perfect agreement for left ( k  = 0.84, CI: 0.82-0.86) and right leg ( k  = 0.85, CI: 0.83-0.87), when excluding missing clinician-reported Rutherford scores. Expert's judgment was rarely required to be the deciding factor (11 out of 6,633). Substantial agreement between clinician-reported Rutherford scores and medical record review Rutherford scores was found, which could be an indicator for the validity of routinely reported data. Depending on its purpose, the secondary use of routinely collected Rutherford scores is a viable option.

The Impact of Common Variations in Sequential Organ Failure Assessment Score Calculation on Sepsis Measurement Using Sepsis-3 Criteria: A Retrospective Analysis Using Electronic Health Record Data

OBJECTIVES

To assess the impact of different methods of calculating Sequential Organ Failure Assessment (SOFA) scores using electronic health record data on the incidence, outcomes, agreement, and predictive validity of Sepsis-3 criteria.

DESIGN

Retrospective observational study.

SETTING

Five Massachusetts hospitals.

PATIENTS

Hospitalized adults, 2015 to 2022.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We defined sepsis as a suspected infection (culture obtained and antibiotic administered) with a concurrent increase in SOFA score by greater than or equal to 2 points (Sepsis-3 criteria). Our reference SOFA implementation strategy imputed normal values for missing data, used Pa o2 /F io2 ratios for respiratory scores, and assumed normal baseline SOFA scores for community-onset sepsis. We then implemented SOFA scores using different missing data imputation strategies (averaging worst values from preceding and following days vs. carrying forward nonmissing values), imputing respiratory scores using Sp o2 /F io2 ratios, and incorporating comorbidities and prehospital laboratory data into baseline SOFA scores. Among 1,064,459 hospitalizations, 297,512 (27.9%) had suspected infection and 141,052 (13.3%) had sepsis with an in-hospital mortality rate of 10.3% using the reference SOFA method. The percentage of patients missing SOFA components for at least 1 day in the infection window was highest for Pa o2 /F io2 ratios (98.6%), followed by Sp o2 /F io2 ratios (73.5%), bilirubin (68.5%), and Glasgow Coma Scale scores (57.2%). Different missing data imputation strategies yielded near-perfect agreement in identifying sepsis (kappa 0.99). However, using Sp o2 /F io2 imputations yielded higher sepsis incidence (18.3%), lower mortality (8.1%), and slightly lower predictive validity for mortality (area under the receiver operating curves [AUROC] 0.76 vs. 0.78). For community-onset sepsis, incorporating comorbidities and historical laboratory data into baseline SOFA score estimates yielded lower sepsis incidence (6.9% vs. 11.6%), higher mortality (13.4% vs. 9.6%), and higher predictive validity (AUROC 0.79 vs. 0.75) relative to the reference SOFA implementation.

CONCLUSIONS

Common variations in calculating respiratory and baseline SOFA scores, but not in handling missing data, lead to substantial differences in observed incidence, mortality, agreement, and predictive validity of Sepsis-3 criteria.

Validation of Adult Sepsis Event and Epidemiologic Analysis of Sepsis Prevalence and Mortality Using Adult Sepsis Event's Electronic Health Records-Based Sequential Organ Failure Assessment Criteria: A Single-Center Study in South Korea

OBJECTIVES

In 2018, the Centers for Disease Control and Prevention introduced the Adult Sepsis Event (ASE) definition, using electronic health records (EHRs) data for surveillance and sepsis quality improvement. However, data regarding ASE outside the United States remain limited. We therefore aimed to validate the diagnostic accuracy of the ASE and to assess the prevalence and mortality of sepsis using ASE.

DESIGN

Retrospective cohort study.

SETTING

A single center in South Korea, with 2732 beds including 221 ICU beds.

PATIENTS

During the validation phase, adult patients who were hospitalized or visiting the emergency department between November 5 and November 11, 2019, were included. In the subsequent phase of epidemiologic analysis, we included adult patients who were admitted from January to December 2020.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

ASE had a sensitivity of 91.6%, a specificity of 98.3%, a positive predictive value (PPV) of 57.4%, and a negative predictive value of 99.8% when compared with the Sepsis-3 definition. Of 126,998 adult patient hospitalizations in 2020, 6,872 cases were diagnosed with sepsis based on the ASE (5.4% per year), and 893 patients were identified as having sepsis according to the International Classification of Diseases , 10th Edition (ICD-10) (0.7% per year). Hospital mortality rates were 16.6% (ASE) and 23.5% (ICD-10-coded sepsis). Monthly sepsis prevalence and hospital mortality exhibited less variation when diagnosed using ASE compared with ICD-10 coding (coefficient of variation [CV] for sepsis prevalence: 0.051 vs. 0.163, Miller test p < 0.001; CV for hospital mortality: 0.087 vs. 0.261, p = 0.001).

CONCLUSIONS

ASE demonstrated high sensitivity and a moderate PPV compared with the Sepsis-3 criteria in a Korean population. The prevalence of sepsis, as defined by ASE, was 5.4% per year and was similar to U.S. estimates. The prevalence of sepsis by ASE was eight times higher and exhibited less monthly variability compared with that based on the ICD-10 code.

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

Patient safety is a key quality issue for health systems. Healthcare acquired adverse events (AEs) compromise safety and quality; therefore, their reporting and monitoring is a patient safety priority. Although administrative datasets are potentially efficient tools for monitoring rates of AEs, concerns remain over the accuracy of their data. Chart review validation studies are required to explore the potential of administrative data to inform research and health policy. This review aims to present an overview of the methodological approaches and strategies used to validate rates of AEs in administrative data through chart review. This review was conducted in line with the Joanna Briggs Institute methodological framework for scoping reviews. Through database searches, 1054 sources were identified, imported into Covidence, and screened against the inclusion criteria. Articles that validated rates of AEs in administrative data through chart review were included. Data were extracted, exported to Microsoft Excel, arranged into a charting table, and presented in a tabular and descriptive format. Fifty-six studies were included. Most sources reported on surgical AEs; however, other medical specialties were also explored. Chart reviews were used in all studies; however, few agreed on terminology for the study design. Various methodological approaches and sampling strategies were used. Some studies used the Global Trigger Tool, a two-stage chart review method, whilst others used alternative single-, two-stage, or unclear approaches. The sources used samples of flagged charts (n = 24), flagged and random charts (n = 11), and random charts (n = 21). Most studies reported poor or moderate accuracy of AE rates. Some studies reported good accuracy of AE recording which highlights the potential of using administrative data for research purposes. This review highlights the potential for administrative data to provide information on AE rates and improve patient safety and healthcare quality. Nonetheless, further work is warranted to ensure that administrative data are accurate. The variation of methodological approaches taken, and sampling techniques used demonstrate a lack of consensus on best practice; therefore, further clarity and consensus are necessary to develop a more systematic approach to chart reviewing.

The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients

BACKGROUND

Continuous surveillance for healthcare-associated infections such as central venous catheter-related bloodstream infections (CVC-BSI) is crucial for prevention. However, traditional surveillance methods are resource-intensive and prone to bias. This study aimed to develop and validate fully-automated surveillance algorithms for CVC-BSI.

METHODS

Two algorithms were developed using electronic health record data from 1000 admissions with a positive blood culture (BCx) at Karolinska University Hospital from 2017: (1) Combining microbiological findings in BCx and CVC cultures with BSI symptoms; (2) Only using microbiological findings. These algorithms were validated in 5170 potential CVC-BSI-episodes from all admissions in 2018-2019, and results extrapolated to all potential CVC-BSI-episodes within this period (n = 181,354). The reference standard was manual record review according to ECDC's definition of microbiologically confirmed CVC-BSI (CRI3-CVC).

RESULTS

In the potential CVC-BSI-episodes, 51 fulfilled ECDC's definition and the algorithms identified 47 and 49 episodes as CVC-BSI, respectively. Both algorithms performed well in assessing CVC-BSI. Overall, algorithm 2 performed slightly better with in the total period a sensitivity of 0.880 (95%-CI 0.783-0.959), specificity of 1.000 (95%-CI 0.999-1.000), PPV of 0.918 (95%-CI 0.833-0.981) and NPV of 1.000 (95%-CI 0.999-1.000). Incidence according to the reference and algorithm 2 was 0.33 and 0.31 per 1000 in-patient hospital-days, respectively.

CONCLUSIONS

Both fully-automated surveillance algorithms for CVC-BSI performed well and could effectively replace manual surveillance. The simpler algorithm, using only microbiology data, is suitable when BCx testing adheres to recommendations, otherwise the algorithm using symptom data might be required. Further validation in other settings is necessary to assess the algorithms' generalisability.

Innovative Techniques for Infection Control and Surveillance in Hospital Settings and Long-Term Care Facilities: A Scoping Review

Healthcare-associated infections (HAIs) pose significant challenges in healthcare systems, with preventable surveillance playing a crucial role. Traditional surveillance, although effective, is resource-intensive. The development of new technologies, such as artificial intelligence (AI), can support traditional surveillance in analysing an increasing amount of health data or meeting patient needs. We conducted a scoping review, following the PRISMA-ScR guideline, searching for studies of new digital technologies applied to the surveillance, control, and prevention of HAIs in hospitals and LTCFs published from 2018 to 4 November 2023. The literature search yielded 1292 articles. After title/abstract screening and full-text screening, 43 articles were included. The mean study duration was 43.7 months. Surgical site infections (SSIs) were the most-investigated HAI and machine learning was the most-applied technology. Three main themes emerged from the thematic analysis: patient empowerment, workload reduction and cost reduction, and improved sensitivity and personalization. Comparative analysis between new technologies and traditional methods showed different population types, with machine learning methods examining larger populations for AI algorithm training. While digital tools show promise in HAI surveillance, especially for SSIs, challenges persist in resource distribution and interdisciplinary integration in healthcare settings, highlighting the need for ongoing development and implementation strategies.

Validation of an ICD-Based Algorithm to Identify Sepsis: A Retrospective Study

Background

Sepsis surveillance was important for resources allocation, prevention, and development of health policy.

Objective

The aim of the study was to validate a modified International Classification of Diseases (ICD)-10 based algorithm for identifying hospitalized patients with sepsis.

Methods

We retrospectively analyzed a prospective, single-center cohort of adult patients who were consecutively admitted to one medical ICU ward and ten non-ICU wards with suspected or confirmed infections during a 6-month period. A modified ICD-10 based algorithm was validated against a reference standard of Sequential Organ Failure Assessment (SOFA) score based on Sepsis-3. Sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), and areas under the receiver operating characteristic curves (AUROCs) were calculated for modified ICD-10 criteria, eSOFA criteria, Martin's criteria, and Angus's criteria.

Results

Of the 547 patients in the cohort, 332 (61%) patients met Sepsis-3 criteria and 274 (50%) met modified ICD-10 criteria. In the ICU setting, modified ICD-10 criteria had SE (84.47%), SP (88.57%), PPV (95.60), and NPV (65.96). In non-ICU settings, modified ICD-10 had SE (64.19%), SP (80.00%), PPV (80.33), and NPV (63.72). In the whole cohort, the AUROCs of modified ICD-10 criteria, eSOFA, Angus's criteria, and Martin's criteria were 0.76, 0.75, 0.62, and 0.62, respectively.

Conclusion

This study demonstrated that modified ICD-10 criteria had higher validity compared with Angus's criteria and Martin's criteria. Validity of the modified ICD-10 criteria was similar to eSOFA criteria. Modified ICD-10 algorithm can be used to provide an accurate estimate of population-based sepsis burden of China.

Design of a national patient-centred clinical pathway for sepsis in Sweden

BACKGROUND

The World Health Organization has adopted a resolution on sepsis and urged member states to develop national processes to improve sepsis care. In Sweden, sepsis was selected as one of the ten first diagnoses to be addressed, when the Swedish government in 2019 allocated funds for patient-centred clinical pathways in healthcare. A national multidisciplinary working group, including a patient representative, was appointed to develop the patient-centred clinical pathway for sepsis.

METHODS

The working group mapped challenges and needs surrounding sepsis care and included a survey sent to all emergency departments (ED) in Sweden, and then designed a patient-centred clinical pathway for sepsis.

RESULTS

The working group decided to focus on the following four areas: (1) sepsis alert for early detection and management optimisation for the most severely ill sepsis patients in the ED; (2) accurate sepsis diagnosis coding; (3) structured information to patients at discharge after sepsis care and (4) structured telephone follow-up after sepsis care. A health-economic analysis indicated that the implementation of the clinical pathway for sepsis will most likely not drive costs. An important aspect of the clinical pathway is implementing continuous monitoring of performance and process indicators. A national working group is currently building up such a system for monitoring, focusing on extraction of this information from the electronic health records systems.

CONCLUSION

A national patient-centred clinical pathway for sepsis has been developed and is currently being implemented in Swedish healthcare. We believe that the clinical pathway and the accompanying monitoring will provide a more efficient and equal sepsis care and improved possibilities to monitor and further develop sepsis care in Sweden.

Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data

Sepsis is a leading cause of mortality and early identification improves survival. With increasing digitalization of health care data automated sepsis prediction models hold promise to aid in prompt recognition. Most previous studies have focused on the intensive care unit (ICU) setting. Yet only a small proportion of sepsis develops in the ICU and there is an apparent clinical benefit to identify patients earlier in the disease trajectory. In this cohort of 82,852 hospital admissions and 8038 sepsis episodes classified according to the Sepsis-3 criteria, we demonstrate that a machine learned score can predict sepsis onset within 48 h using sparse routine electronic health record data outside the ICU. Our score was based on a causal probabilistic network model-SepsisFinder-which has similarities with clinical reasoning. A prediction was generated hourly on all admissions, providing a new variable was registered. Compared to the National Early Warning Score (NEWS2), which is an established method to identify sepsis, the SepsisFinder triggered earlier and had a higher area under receiver operating characteristic curve (AUROC) (0.950 vs. 0.872), as well as area under precision-recall curve (APR) (0.189 vs. 0.149). A machine learning comparator based on a gradient-boosting decision tree model had similar AUROC (0.949) and higher APR (0.239) than SepsisFinder but triggered later than both NEWS2 and SepsisFinder. The precision of SepsisFinder increased if screening was restricted to the earlier admission period and in episodes with bloodstream infection. Furthermore, the SepsisFinder signaled median 5.5 h prior to antibiotic administration. Identifying a high-risk population with this method could be used to tailor clinical interventions and improve patient care.

Concordance Between Initial Presumptive and Final Adjudicated Diagnoses of Infection Among Patients Meeting Sepsis-3 Criteria in the Emergency Department

BACKGROUND

Guidelines emphasize rapid antibiotic treatment for sepsis, but infection presence is often uncertain at initial presentation. We investigated the incidence and drivers of false-positive presumptive infection diagnosis among emergency department (ED) patients meeting Sepsis-3 criteria.

METHODS

For a retrospective cohort of patients hospitalized after meeting Sepsis-3 criteria (acute organ failure and suspected infection including blood cultures drawn and intravenous antimicrobials administered) in 1 of 4 EDs from 2013 to 2017, trained reviewers first identified the ED-diagnosed source of infection and adjudicated the presence and source of infection on final assessment. Reviewers subsequently adjudicated final infection probability for a randomly selected 10% subset of subjects. Risk factors for false-positive infection diagnosis and its association with 30-day mortality were evaluated using multivariable regression.

RESULTS

Of 8267 patients meeting Sepsis-3 criteria in the ED, 699 (8.5%) did not have an infection on final adjudication and 1488 (18.0%) patients with confirmed infections had a different source of infection diagnosed in the ED versus final adjudication (ie, initial/final source diagnosis discordance). Among the subset of patients whose final infection probability was adjudicated (n = 812), 79 (9.7%) had only "possible" infection and 77 (9.5%) were not infected. Factors associated with false-positive infection diagnosis included hypothermia, altered mental status, comorbidity burden, and an "unknown infection source" diagnosis in the ED (odds ratio: 6.39; 95% confidence interval: 5.14-7.94). False-positive infection diagnosis was not associated with 30-day mortality.

CONCLUSIONS

In this large multihospital study, <20% of ED patients meeting Sepsis-3 criteria had no infection or only possible infection on retrospective adjudication.

Sustained low catheter related infection (CRI) incidence in an observational follow-up study of 9924 catheters using automated data scripts as quality assurance for central venous catheter (CVC) management

Background

To maintain a low incidence of Catheter Related Infections (CRI) and Catheter Related Bloodstream Infections (CRBSI), continuous follow-up studies on catheter management are necessary. The aims of the present study were to investigate the incidence of catheter tip colonisation, CRI and CRBSI in the Region, to further explore the feasibility of automatic data collection and to investigate associations between independent variables and CRI.

Methods

Data from electronic patient charts on all documented central venous catheter (CVC) insertions from multiple hospitals in southern Sweden, between March 2019 and August 2020, were automatically extracted. Multivariable regression analyses were used to identify associated risk factors.

Results

In total, 9924 CVC insertions were included. The prevalence of CRI and CRBSI were 0.7% (n = 74) and 0.02% (n = 20) with incidences of 1.2/1000 catheter days and 0.3/1000 catheter days, respectively.

Conclusions

We found a sustained low incidence of CRI and CRBSI in the Region. Catheter tips were less likely to be colonised when the subclavian route was used compared to the internal jugular route and male sex as well as increased number of catheter lumens were associated with both catheter tip colonisation and CRI. By using automated scripts, data extraction was efficient and feasible but also demonstrated that real-time quality assurance should be recommended, since this is superior to current standard.

Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis

BACKGROUND

The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database.

METHODS

We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm's performance.

RESULTS

Agreement between the RL agent's policy and the actual documented treatment reached 59%. Our RL agent's treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians' policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians' historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia.

CONCLUSIONS

Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a 'precision-medicine' approach to future prospective controlled trials and practice.

Update on Sepsis Epidemiology in the Era of COVID-19

Timely and accurate data on the epidemiology of sepsis is essential to inform public policy, clinical practice, and research priorities. Recent studies have illuminated several ongoing questions about sepsis epidemiology, including the incidence and outcomes of sepsis in non-Western countries and in specialized populations such as surgical patients, patients with cancer, and the elderly. There have also been new insights into the limitations of current surveillance methods using administrative data and increasing experience tracking sepsis incidence and outcomes using "big data" approaches that take advantage of detailed electronic health record data. The COVID-19 pandemic, however, has fundamentally changed the landscape of sepsis epidemiology. It has increased sepsis rates, helped highlight ongoing controversies about how to define sepsis, and intensified debate about the possible unintended consequences of overly rigid sepsis care bundles. Despite these controversies, there is a growing consensus that severe COVID-19 causing organ dysfunction is appropriate to label as sepsis, even though it is treated very differently from bacterial sepsis, and that surveillance strategies need to be modified to reliably identify these cases to fully capture and delineate the current burden of sepsis. This review will summarize recent insights into the epidemiology of sepsis and highlight several urgent questions and priorities catalyzed by COVID-19.

Establishment of a Chinese critical care database from electronic healthcare records in a tertiary care medical center

The medical specialty of critical care, or intensive care, provides emergency medical care to patients suffering from life-threatening complications and injuries. The medical specialty is featured by the generation of a huge amount of high-granularity data in routine practice. Currently, these data are well archived in the hospital information system for the primary purpose of routine clinical practice. However, data scientists have noticed that in-depth mining of such big data may provide insights into the pathophysiology of underlying diseases and healthcare practices. There have been several openly accessible critical care databases being established, which have generated hundreds of scientific outputs published in scientific journals. However, such work is still in its infancy in China. China is a large country with a huge patient population, contributing to the generation of large healthcare databases in hospitals. In this data descriptor article, we report the establishment of an openly accessible critical care database generated from the hospital information system.

Thoughts on the construction of public health informatization for community health archives grass roots management system

With the development of social economy and the continuous improvement of people's living standards, people expect to receive high-level medical services, and the requirements for medical care are also getting higher and higher. However, there are still objective problems such as rising medical costs, difficulty in seeking medical treatment, uneven distribution of medical resources, low efficiency of medical services, and uneven medical quality. This paper first analyzes the significance of public health informatization construction, focuses on the elements of public health informatization construction, and expounds the status quo of health informatization construction and the existing problems in community health informatization. Then, this paper expounds the construction of public health informatization based on the grass-roots management system of community health records, and discusses the construction of a health information platform centered on the health records of community residents. Afterwards, this paper proposes and studies the functions of the community medical information archives management system from three aspects: the composition of the community medical information archives management system, the problems of system management, and the development requirements of the system, and proposes an algorithm based on a decision tree model to enhance public health informology. Finally, on the basis of experiments and investigations, Internet technology and decision tree model algorithms are introduced into the public health information system construction of the community health archives system to build a new public health information system, and the satisfaction rate can be increased by 23%.

Association Between Time to Appropriate Antimicrobial Treatment and 30-day Mortality in Patients With Bloodstream Infections: A Retrospective Cohort Study

BACKGROUND

Effective antimicrobial treatment is key for survival in bloodstream infection (BSI), but the impact of timing of treatment remains unclear. Our aim was to assess the association between time to appropriate antimicrobial treatment and 30-day mortality in BSI patients.

METHODS

This was a retrospective cohort study using electronic health record data from a large academic center in Sweden. Adult patients admitted between the years 2012 and 2019, with onset of BSI at the emergency department or general wards, were included. Pathogen-antimicrobial drug combinations were classified as appropriate or inappropriate based on reported in vitro susceptibilities. To avoid immortal time bias, the association between appropriate therapy and mortality was assessed with multivariable logistic regression analysis at pre-specified landmark times.

RESULTS

We included 10 628 BSI-episodes, occurring in 9192 unique patients. The overall 30-day mortality was 11.8%. No association in favor of a protective effect between appropriate therapy and mortality was found at the 1, 3 and 6 hours landmark after blood culture collection. At 12 hours, the risk of death increased with inappropriate treatment (adjusted odds ratio 1.17 [95% confidence interval {CI}, 1.01-1.37]) and continued to increase gradually at 24, 48, and 72 hours. Stratifying by high or low SOFA score generated similar odds ratios, with wider confidence intervals.

CONCLUSIONS

Delays in appropriate antimicrobial treatment were associated with increased 30-day mortality after 12 hours from blood culture collection, but not at 1, 3, and 6 hours, in BSI. These results indicate a benchmark for providing rapid microbiological diagnostics of blood cultures.

Epidemiology of Organ Failure Before and During COVID-19 Pandemic Surge Conditions

BACKGROUND

Understanding the distribution of organ failure before and during the COVID-19 pandemic surge can provide a deeper understanding of how the pandemic strained health care systems and affected outcomes.

OBJECTIVE

To assess the distribution of organ failure in 3 New York City hospitals during the COVID-19 pandemic.

METHODS

A retrospective cohort study of adult admissions across hospitals from February 1, 2020, through May 31, 2020, was conducted. The cohort was stratified into those admitted before March 17, 2020 (prepandemic) and those admitted on or after that date (SARS-CoV-2-positive and non-SARS-CoV-2). Sequential Organ Failure Assessment scores were computed every 2 hours for each admission.

RESULTS

A total of 1 794 975 scores were computed for 20 704 admissions. Before and during the pandemic, renal failure was the most common type of organ failure at admission and respiratory failure was the most common type of hospital-onset organ failure. The SARS-CoV-2-positive group showed a 231% increase in respiratory failure compared with the prepandemic group. More than 65% of hospital-onset organ failure in the prepandemic group and 83% of hospital-onset respiratory failure in the SARS-CoV-2-positive group occurred outside intensive care units. The SARS-CoV-2-positive group showed a 341% increase in multiorgan failure compared with the prepandemic group. Compared with the prepandemic and non-SARS-CoV-2 patients, SARS-CoV-2-positive patients had significantly higher mortality for the same admission and maximum organ failure score.

CONCLUSION

Most hospital-onset organ failure began outside intensive care units, with a marked increase in multiorgan failure during pandemic surge conditions and greater hospital mortality for the severity of organ failure.

Availability of Multiple Organ Failure Score Components in Surgical Patients

Background: Scoring systems are often used describe the degree of multi-system organ failure (MOF), however, the data used to calculate these scores are often missing. Studies utilizing these scoring systems often underreport the frequency of missing data. No study has examined the availability of clinical data needed to calculate Sequential Organ Failure Assessment (SOFA), and other organ failure scores. The primary objective of this study is to observe how often emergency general surgery and trauma patients have missing data needed to calculate MOF scores. Patients and Methods: Patients admitted between June 2017 and September 2019 were evaluated. Data to calculate SOFA, quick SOFA (qSOFA), Marshall Multiple Organ Dysfunction Score (MODS), Denver Post-Injury Multiple Organ Failure, and systemic inflammatory response syndrome (SIRS) criteria, as well as demographic and general admission and discharge data, were collected. Results: Of the 238 patients included in this study, 66.4% were emergency general surgery and 33.6% were trauma patients. For all patients, the median intensive care unit (ICU) length of stay (LOS) was seven days (range, 4-12), the median hospital LOS was 14 days (range, 10-21), and 28 patients (11.8%) did not survive to hospital discharge. Sequential Organ Failure Assessment was calculable in 21.4%-18.1%, whereas MODS was calculable in 6.3%-5.0% on days three and five, respectively. The Denver score was calculable in 32.5%-28.8% of trauma patients on these days. Of the data points needed to calculate these scores, the partial pressure of oxygen (Pao₂)/fraction of inspired oxygen (FIo₂) ratio, central venous pressure (CVP), and bilirubin were the least available components. Conclusions: Data needed to fully calculate SOFA and other common MOF scores are often not readily available highlighting the degree of imputation required to calculate these scores. We recommend better reporting of the degree of missing data in the literature.

Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization

BACKGROUND

As most automated surveillance (AS) methods to detect healthcare-associated infections (HAIs) have been developed and implemented in research settings, information about the feasibility of large-scale implementation is scarce.

AIM

We aimed to describe key aspects of the design of AS systems and implementation in European institutions and hospitals.

METHODS

An online survey was distributed via email in February/March 2019 among 1) PRAISE (Providing a Roadmap for Automated Infection Surveillance in Europe) network members; 2) corresponding authors of peer-reviewed European publications on existing AS systems; and 3) the mailing list of national infection prevention and control focal points of the European Centre for Disease Prevention and Control. Three AS systems from the survey were selected, based on quintessential features, for in-depth review focusing on implementation in practice.

FINDINGS

Through the survey and the review of three selected AS systems, notable differences regarding the methods, algorithms, data sources and targeted HAIs were identified. The majority of AS systems used a classification algorithm for semi-automated surveillance and targeted HAIs were mostly surgical site infections, urinary tract infections, sepsis or other bloodstream infections. AS systems yielded a reduction of workload for hospital staff. Principal barriers of implementation were strict data security regulations as well as creating and maintaining an information technology infrastructure.

CONCLUSION

AS in Europe is characterized by heterogeneity in methods and surveillance targets. To allow for comparisons and encourage homogenization, future publications on AS systems should provide detailed information on source data, methods and the state of implementation.

The accuracy of fully automated algorithms for surveillance of healthcare-onset Clostridioides difficile infections in hospitalized patients

We developed and validated a set of fully automated surveillance algorithms for healthcare-onset CDI using electronic health records. In a validation data set of 750 manually annotated admissions, the algorithm based on International Classification of Disease, Tenth Revision (ICD-10) code A04.7 had insufficient sensitivity. Algorithms based on microbiological test results with or without addition of symptoms performed well.

Impact of Minimally Invasive Esophagectomy in Post-Operative Atrial Fibrillation and Long-Term Mortality in Patients Among Esophageal Cancer

Aims:

Postoperative Atrial fibrillation (POAF) after esophagectomy may prolong stay in intensive care and increase risk of perioperative complications. A minimally invasive approach is becoming the preferred option for esophagectomy, yet its implications for POAF risk remains unclear. The association between POAF and minimally invasive esophagectomy (MIE) was examined in this study.

Methods:

We used a dataset of 575 patients who underwent esophagectomy. Multivariate logistic regression analysis was performed to examine the association between MIE and POAF. A cox proportional hazards model was applied to assess the long-term mortality (MIE vs open esophagectomy, OE).

Results:

Of the 575 patients with esophageal cancer, 62 developed POAF. MIE was negatively associated with the occurrence of POAF (Odds ratio: 0.163, 95%CI: 0.033-0.801). No significant difference was observed in long-term mortality (Odds ratio: 2.144, 95%CI: 0.963-4.775).

Conclusions:

MIE may reduced the incidence of POAF without compromising the survival of patients with esophageal cancer. Moreover, the specific mechanism of MIE providing this possible advantage needs to be determined by larger prospective cohort studies with specific biomarker information from laboratory tests.

Clinical phenotypes and outcomes of SARS-CoV-2, influenza, RSV and seven other respiratory viruses: a retrospective study using complete hospital data

Background

An understanding of differences in clinical phenotypes and outcomes COVID-19 compared with other respiratory viral infections is important to optimise the management of patients and plan healthcare. Herein we sought to investigate such differences in patients positive for SARS-CoV-2 compared with influenza, respiratory syncytial virus (RSV) and other respiratory viruses.

Methods

We performed a retrospective cohort study of hospitalised adults and children (≤15 years) who tested positive for SARS-CoV-2, influenza virus A/B, RSV, rhinovirus, enterovirus, parainfluenza viruses, metapneumovirus, seasonal coronaviruses, adenovirus or bocavirus in a respiratory sample at admission between 2011 and 2020.

Results

A total of 6321 adult (1721 SARS-CoV-2) and 6379 paediatric (101 SARS-CoV-2) healthcare episodes were included in the study. In adults, SARS-CoV-2 positivity was independently associated with younger age, male sex, overweight/obesity, diabetes and hypertension, tachypnoea as well as better haemodynamic measurements, white cell count, platelet count and creatinine values. Furthermore, SARS-CoV-2 was associated with higher 30-day mortality as compared with influenza (adjusted HR (aHR) 4.43, 95% CI 3.51 to 5.59), RSV (aHR 3.81, 95% CI 2.72 to 5.34) and other respiratory viruses (aHR 3.46, 95% CI 2.61 to 4.60), as well as higher 90-day mortality, ICU admission, ICU mortality and pulmonary embolism in adults. In children, patients with SARS-CoV-2 were older and had lower prevalence of chronic cardiac and respiratory diseases compared with other viruses.

Conclusions

SARS-CoV-2 is associated with more severe outcomes compared with other respiratory viruses, and although associated with specific patient and clinical characteristics at admission, a substantial overlap precludes discrimination based on these characteristics.

Information technology aspects of large-scale implementation of automated surveillance of healthcare-associated infections

INTRODUCTION

Healthcare-associated infections (HAI) are a major public health concern. Monitoring of HAI rates, with feedback, is a core component of infection prevention and control programmes. Digitalization of healthcare data has created novel opportunities for automating the HAI surveillance process to varying degrees. However, methods are not standardized and vary widely between different healthcare facilities. Most current automated surveillance (AS) systems have been confined to local settings, and practical guidance on how to implement large-scale AS is needed.

METHODS

This document was written by a task force formed in March 2019 within the PRAISE network (Providing a Roadmap for Automated Infection Surveillance in Europe), gathering experts in HAI surveillance from ten European countries.

RESULTS

The document provides an overview of the key e-health aspects of implementing an AS system of HAI in a clinical environment to support both the infection prevention and control team and information technology (IT) departments. The focus is on understanding the basic principles of storage and structure of healthcare data, as well as the general organization of IT infrastructure in surveillance networks and participating healthcare facilities. The fundamentals of data standardization, interoperability and algorithms in relation to HAI surveillance are covered. Finally, technical aspects and practical examples of accessing, storing and sharing healthcare data within a HAI surveillance network, as well as maintenance and quality control of such a system, are discussed.

CONCLUSIONS

With the guidance given in this document, along with the PRAISE roadmap and governance documents, readers will find comprehensive support to implement large-scale AS in a surveillance network.

Descriptors of Sepsis Using the Sepsis-3 Criteria: A Cohort Study in Critical Care Units Within the U.K. National Institute for Health Research Critical Care Health Informatics Collaborative

Supplemental Digital Content is available in the text.

To describe the epidemiology of sepsis in critical care by applying the Sepsis-3 criteria to electronic health records.

PRAISE: providing a roadmap for automated infection surveillance in Europe

INTRODUCTION

Healthcare-associated infections (HAI) are among the most common adverse events of medical care. Surveillance of HAI is a key component of successful infection prevention programmes. Conventional surveillance - manual chart review - is resource intensive and limited by concerns regarding interrater reliability. This has led to the development and use of automated surveillance (AS). Many AS systems are the product of in-house development efforts and heterogeneous in their design and methods. With this roadmap, the PRAISE network aims to provide guidance on how to move AS from the research setting to large-scale implementation, and how to ensure the delivery of surveillance data that are uniform and useful for improvement of quality of care.

METHODS

The PRAISE network brings together 30 experts from ten European countries. This roadmap is based on the outcome of two workshops, teleconference meetings and review by an independent panel of international experts.

RESULTS

This roadmap focuses on the surveillance of HAI within networks of healthcare facilities for the purpose of comparison, prevention and quality improvement initiatives. The roadmap does the following: discusses the selection of surveillance targets, different organizational and methodologic approaches and their advantages, disadvantages and risks; defines key performance requirements of AS systems and suggestions for their design; provides guidance on successful implementation and maintenance; and discusses areas of future research and training requirements for the infection prevention and related disciplines. The roadmap is supported by accompanying documents regarding the governance and information technology aspects of implementing AS.

CONCLUSIONS

Large-scale implementation of AS requires guidance and coordination within and across surveillance networks. Transitions to large-scale AS entail redevelopment of surveillance methods and their interpretation, intensive dialogue with stakeholders and the investment of considerable resources. This roadmap can be used to guide future steps towards implementation, including designing solutions for AS and practical guidance checklists.

Research Trends in Artificial Intelligence Applications in Human Factors Health Care: Mapping Review

BACKGROUND

Despite advancements in artificial intelligence (AI) to develop prediction and classification models, little research has been devoted to real-world translations with a user-centered design approach. AI development studies in the health care context have often ignored two critical factors of ecological validity and human cognition, creating challenges at the interface with clinicians and the clinical environment.

OBJECTIVE

The aim of this literature review was to investigate the contributions made by major human factors communities in health care AI applications. This review also discusses emerging research gaps, and provides future research directions to facilitate a safer and user-centered integration of AI into the clinical workflow.

METHODS

We performed an extensive mapping review to capture all relevant articles published within the last 10 years in the major human factors journals and conference proceedings listed in the "Human Factors and Ergonomics" category of the Scopus Master List. In each published volume, we searched for studies reporting qualitative or quantitative findings in the context of AI in health care. Studies are discussed based on the key principles such as evaluating workload, usability, trust in technology, perception, and user-centered design.

RESULTS

Forty-eight articles were included in the final review. Most of the studies emphasized user perception, the usability of AI-based devices or technologies, cognitive workload, and user's trust in AI. The review revealed a nascent but growing body of literature focusing on augmenting health care AI; however, little effort has been made to ensure ecological validity with user-centered design approaches. Moreover, few studies (n=5 against clinical/baseline standards, n=5 against clinicians) compared their AI models against a standard measure.

CONCLUSIONS

Human factors researchers should actively be part of efforts in AI design and implementation, as well as dynamic assessments of AI systems' effects on interaction, workflow, and patient outcomes. An AI system is part of a greater sociotechnical system. Investigators with human factors and ergonomics expertise are essential when defining the dynamic interaction of AI within each element, process, and result of the work system.

Measurement of Sepsis in a National Cohort Using Three Different Methods to Define Baseline Organ Function

Rationale: In 2017, the U.S. Centers for Disease Control and Prevention (CDC) developed a new surveillance definition of sepsis, the adult sepsis event (ASE), to better track sepsis epidemiology. The ASE requires evidence of acute organ dysfunction and defines baseline organ function pragmatically as the best in-hospital value. This approach may undercount sepsis if new organ dysfunction does not resolve by discharge.Objectives: To understand how sepsis identification and outcomes differ when using the best laboratory values during hospitalization versus methods that use historical lookbacks to define baseline organ function.Methods: We identified all patients hospitalized at 138 Veterans Affairs hospitals (2013-2018) admitted via the emergency department with two or more systemic inflammatory response criteria, were treated with antibiotics within 48 hours (i.e., had potential infection), and completed 4+ days of antibiotics (i.e., had suspected infection). We considered the following three approaches to defining baseline renal, hematologic, and liver function: the best values during hospitalization (as in the Centers for Disease Control and Prevention's ASE), the best values during hospitalization plus the prior 90 days (3-mo baseline), and the best values during hospitalization plus the prior 180 days (6-mo baseline). We determined how many patients met the criteria for sepsis by each approach, and then compared characteristics and outcomes of sepsis hospitalizations between the three approaches.Results: Among 608,128 hospitalizations with potential infection, 72.1%, 68.5%, and 58.4% had creatinine, platelet, and total bilirubin measured, respectively, in the prior 3 months. A total of 86.0%, 82.6%, and 74.8%, respectively, had these labs in the prior 6 months. Using the hospital baseline, 100,568 hospitalizations met criteria for community-acquired sepsis. By contrast, 111,983 and 117,435 met criteria for sepsis using the 3- and 6-month baselines, for a relative increase of 11% and 17%, respectively. Patient characteristics were similar across the three approaches. In-hospital mortality was 7.2%, 7.0%, and 6.8% for sepsis hospitalizations identified using the hospital, 3-month baseline, and 6-month baseline. The 30-day mortality was 12.5%, 12.7%, and 12.5%, respectively.Conclusions: Among veterans hospitalized with potential infection, the majority had laboratory values in the prior 6 months. Using 3- and 6-month lookbacks to define baseline organ function resulted in an 11% and 17% relative increase, respectively, in the number of sepsis hospitalizations identified.

Surveillance Strategies for Tracking Sepsis Incidence and Outcomes

Sepsis is a leading cause of death and the target of intense efforts to improve recognition, management and outcomes. Accurate sepsis surveillance is essential to properly interpreting the impact of quality improvement initiatives, making meaningful comparisons across hospitals and geographic regions, and guiding future research and resource investments. However, it is challenging to reliably track sepsis incidence and outcomes because sepsis is a heterogeneous clinical syndrome without a pathologic reference standard, allowing for subjectivity and broad discretion in assigning diagnoses. Most epidemiologic studies of sepsis to date have used hospital discharge codes and have suggested dramatic increases in sepsis incidence and decreases in mortality rates over time. However, diagnosis and coding practices vary widely between hospitals and are changing over time, complicating the interpretation of absolute rates and trends. Other surveillance approaches include death records, prospective clinical registries, retrospective medical record reviews, and analyses of the usual care arms of randomized controlled trials. Each of these strategies, however, has substantial limitations. Recently, the US Centers for Disease Control and Prevention released an "Adult Sepsis Event" definition that uses objective clinical indicators of infection and organ dysfunction that can be extracted from most hospitals' electronic health record systems. Emerging data suggest that electronic health record-based clinical surveillance, such as surveillance of Adult Sepsis Event, is accurate, can be applied uniformly across diverse hospitals, and generates more credible estimates of sepsis trends than administrative data. In this review, we discuss the advantages and limitations of different sepsis surveillance strategies and consider future directions.

Validity of "Sepsis-3" criteria in identifying patients with community-onset sepsis in Internal Medicine wards; a prospective, multicenter study

BACKGROUND

Few data are available on the validity of "Sepsis-3" criteria in identifying patients with sepsis in internal medicine wards (IMWs). Real-life data about this topic and on the prevalence of sepsis in IMWs could be useful for improving hospital organization.

OBJECTIVES

To assess the validity of "Sepsis-3" criteria in identifying patients with community-onset sepsis in IMWs. Secondary objectives: to evaluate the prevalence of these patients in IMWs and to compare "Sepsis-3" and "Sepsis-1" criteria.

METHODS

Multicenter, prospective, observational, cohort study, carried out in 22 IMWs of Tuscany (Italy). All patients admitted to each of the study centers over a period of 21-31 days were evaluated within 48 hours; those with clinical signs of infection were enrolled. The main outcome was in-hospital mortality.

RESULTS

2,839 patients were evaluated and 938 (33%) met the inclusion criteria. Patients with sepsis diagnosed according to "Sepsis-3" were 522, representing 55.6% of patients with infection and 18.4% of all patients hospitalized; they were older than those without sepsis (79.4±12.5 vs 74.6±15.2 years, p<0.001). In-hospital mortality was significantly higher in patients with sepsis compared to others (13.8% vs 4.6%; p<0.001). "Sepsis-3" criteria showed greater predictive validity for in-hospital mortality than "Sepsis-1" criteria (AUROC=0.71; 95%CI, 0.66-0.77 vs 0.60; 95%CI 0.54-0.66; p=0.0038).

CONCLUSIONS

"Sepsis-3" criteria are able to identify patients with community-onset sepsis in IMWs, whose prevalence and in-hospital mortality are remarkably high. Medical departments should adapt their organization to the needs for care of these complex patients.

The accuracy of fully automated algorithms for surveillance of healthcare-associated urinary tract infections in hospitalized patients

BACKGROUND

Surveillance for healthcare-associated infections such as healthcare-associated urinary tract infections (HA-UTI) is important for directing resources and evaluating interventions. However, traditional surveillance methods are resource-intensive and subject to bias.

AIM

To develop and validate a fully automated surveillance algorithm for HA-UTI using electronic health record (EHR) data.

METHODS

Five algorithms were developed using EHR data from 2979 admissions at Karolinska University Hospital from 2010 to 2011: (1) positive urine culture (UCx); (2) positive UCx + UTI codes (International Statistical Classification of Diseases and Related Health Problems, 10^th revision); (3) positive UCx + UTI-specific antibiotics; (4) positive UCx + fever and/or UTI symptoms; (5) algorithm 4 with negation for fever without UTI symptoms. Natural language processing (NLP) was used for processing free-text medical notes. The algorithms were validated in 1258 potential UTI episodes from January to March 2012 and results extrapolated to all UTI episodes within this period (N = 16,712). The reference standard for HA-UTIs was manual record review according to the European Centre for Disease Prevention and Control (and US Centers for Disease Control and Prevention) definitions by trained healthcare personnel.

FINDINGS

Of the 1258 UTI episodes, 163 fulfilled the ECDC HA-UTI definition and the algorithms classified 391, 150, 189, 194, and 153 UTI episodes, respectively, as HA-UTI. Algorithms 1, 2, and 3 had insufficient performances. Algorithm 4 achieved better performance and algorithm 5 performed best for surveillance purposes with sensitivity 0.667 (95% confidence interval: 0.594-0.733), specificity 0.997 (0.996-0.998), positive predictive value 0.719 (0.624-0.807) and negative predictive value 0.997 (0.996-0.997).

CONCLUSION

A fully automated surveillance algorithm based on NLP to find UTI symptoms in free-text had acceptable performance to detect HA-UTI compared to manual record review. Algorithms based on administrative and microbiology data only were not sufficient.

Survival prediction of patients with sepsis from age, sex, and septic episode number alone

Sepsis is a life-threatening condition caused by an exaggerated reaction of the body to an infection, that leads to organ failure or even death. Since sepsis can kill a patient even in just one hour, survival prediction is an urgent priority among the medical community: even if laboratory tests and hospital analyses can provide insightful information about the patient, in fact, they might not come in time to allow medical doctors to recognize an immediate death risk and treat it properly. In this context, machine learning can be useful to predict survival of patients within minutes, especially when applied to few medical features easily retrievable. In this study, we show that it is possible to achieve this goal by applying computational intelligence algorithms to three features of patients with sepsis, recorded at hospital admission: sex, age, and septic episode number. We applied several data mining methods to a cohort of 110,204 admissions of patients, and obtained high prediction scores both on this complete dataset (top precision-recall area under the curve PR AUC = 0.966) and on its subset related to the recent Sepsis-3 definition (top PR AUC = 0.860). Additionally, we tested our models on an external validation cohort of 137 patients, and achieved good results in this case too (top PR AUC = 0.863), confirming the generalizability of our approach. Our results can have a huge impact on clinical settings, allowing physicians to forecast the survival of patients by sex, age, and septic episode number alone.

Validation study of German inpatient administrative health data for epidemiological surveillance and measurement of quality of care for sepsis: the OPTIMISE study protocol

INTRODUCTION

Sepsis is a major cause of preventable deaths in hospitals. This study aims to investigate if sepsis incidence and quality of care can be assessed using inpatient administrative health data (IAHD).

METHODS AND ANALYSIS

Design: Retrospective observational validation study using routine data to assess the diagnostic accuracy of sepsis coding in IAHD regarding sepsis diagnosis based on medical record review.

PROCEDURE

A stratified sample of 10 000 patients with an age ≥15 years treated in between 2015 and 2017 in 10 German hospitals is investigated. All available information of medical records is screened by trained physicians to identify true sepsis cases ('gold standard') both according to current ('sepsis-1') definitions and new ('sepsis-3') definitions. Data from medical records are linked to IAHD on patient level using a pseudonym.

ANALYSES

Proportions of cases with sepsis according to sepsis-1 and sepsis-3 definitions are calculated and compared with estimates from coding of sepsis in IAHD. Predictive accuracy (sensitivity, specificity) of different coding abstraction strategies regarding the gold standard is estimated. Predictive accuracy of mortality risk factors obtained from IAHD regarding the respective risk factors obtained from medical records is calculated. An IAHD-based risk model for hospital mortality is compared with a record-based risk model regarding model-fit and predicted risk of death. Analyses adjust for sampling weights. The obtained estimates of sensitivity and specificity for sepsis coding in IAHD are used to estimate adjusted incidence proportions of sepsis based on German national IAHD.

ETHICS AND DISSEMINATION

The study has been approved by the ethics commission of the Jena University Hospital (No. 2018-1065-Daten). The results of the study will be discussed in an expert panel to write a memorandum on improving the utility of IAHD for epidemiological surveillance and quality management of sepsis care.

TRIAL REGISTRATION NUMBER

DRKS00017775; Pre-results.

A surveillance method to identify patients with sepsis from electronic health records in Hong Kong: a single centre retrospective study

BACKGROUND

Currently there are only two population studies on sepsis incidence in Asia. The burden of sepsis in Hong Kong is unknown. We developed a sepsis surveillance method to estimate sepsis incidence from a population electronic health record (EHR) in Hong Kong using objective clinical data. The study objective was to assess our method's performance in identifying sepsis using a retrospective cohort. We compared its accuracy to administrative sepsis surveillance methods such as Angus' and Martin's methods.

METHOD

In this single centre retrospective study we applied our sepsis surveillance method on adult patients admitted to a tertiary hospital in Hong Kong. Two clinicians independently reviewed the clinical notes to determine which patients had sepsis. Performance was assessed by sensitivity, specificity, positive predictive value, negative predictive value and area under the curve (AUC) of Angus', Martin's and our surveillance methods using clinical review as "gold standard."

RESULTS

Between January 1 and February 28, 2018, our sepsis surveillance method identified 1352 adult patients hospitalised with suspected infection. We found that 38.9% (95%CI 36.3-41.5) of these patients had sepsis. Using a 490 patient validation cohort, two clinicians had good agreement with weighted kappa of 0.75 (95% CI 0.69-0.81) before coming to consensus on diagnosis of uncomplicated infection or sepsis for all patients. Our method had sensitivity 0.93 (95%CI 0.89-0.96), specificity 0.86 (95%CI 0.82-0.90) and an AUC 0.90 (95%CI 0.87-0.92) when validated against clinician review. In contrast, Angus' and Martin's methods had AUCs 0.56 (95%CI 0.53-0.58) and 0.56 (95%CI 0.52-0.59), respectively.

CONCLUSIONS

A sepsis surveillance method based on objective data from a population EHR in Hong Kong was more accurate than administrative methods. It may be used to estimate sepsis population incidence and outcomes in Hong Kong.

TRIAL REGISTRATION

This study was retrospectively registered at clinicaltrials.gov on October 3, 2019 ( NCT04114214 ).