Predicting central line-associated bloodstream infections and mortality using supervised machine learning

What every intensivist should know about..Patient safety huddles in the ICU

Patient safety huddles are brief, multidisciplinary conversations that focus on a specific topic or event. Huddles have been shown to improve communication among healthcare providers in a variety of settings, including the intensive care unit (ICU). This paper presents key features of patient safety huddles and describes the ways in which huddle techniques may be particularly relevant to the practice of critical care.

Harnessing of Artificial Intelligence for the Diagnosis and Prevention of Hospital-Acquired Infections: A Systematic Review

Healthcare-associated infections (HAIs) are the most common adverse events in healthcare and constitute a major global public health concern. Surveillance represents the foundation for the effective prevention and control of HAIs, yet conventional surveillance is costly and labor intensive. Artificial intelligence (AI) and machine learning (ML) have the potential to support the development of HAI surveillance algorithms for the understanding of HAI risk factors, the improvement of patient risk stratification as well as the prediction and timely detection and prevention of infections. AI-supported systems have so far been explored for clinical laboratory testing and imaging diagnosis, antimicrobial resistance profiling, antibiotic discovery and prediction-based clinical decision support tools in terms of HAIs. This review aims to provide a comprehensive summary of the current literature on AI applications in the field of HAIs and discuss the future potentials of this emerging technology in infection practice. Following the PRISMA guidelines, this study examined the articles in databases including PubMed and Scopus until November 2023, which were screened based on the inclusion and exclusion criteria, resulting in 162 included articles. By elucidating the advancements in the field, we aim to highlight the potential applications of AI in the field, report related issues and shortcomings and discuss the future directions.

Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges

The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis

BACKGROUND

Sepsis is a life-threatening condition caused by an abnormal response of the body to infection and imposes a significant health and economic burden worldwide due to its high mortality rate. Early recognition of sepsis is crucial for effective treatment. This study aimed to systematically evaluate the performance of various machine learning models in predicting the onset of sepsis.

METHODS

We conducted a comprehensive search of the Cochrane Library, PubMed, Embase, and Web of Science databases, covering studies from database inception to November 14, 2022. We used the PROBAST tool to assess the risk of bias. We calculated the predictive performance for sepsis onset using the C-index and accuracy. We followed the PRISMA guidelines for this study.

RESULTS

We included 23 eligible studies with a total of 4,314,145 patients and 26 different machine learning models. The most frequently used models in the studies were random forest (n = 9), extreme gradient boost (n = 7), and logistic regression (n = 6) models. The random forest (test set n = 9, acc = 0.911) and extreme gradient boost (test set n = 7, acc = 0.957) models were the most accurate based on our analysis of the predictive performance. In terms of the C-index outcome, the random forest (n = 6, acc = 0.79) and extreme gradient boost (n = 7, acc = 0.83) models showed the highest performance.

CONCLUSION

Machine learning has proven to be an effective tool for predicting sepsis at an early stage. However, to obtain more accurate results, additional machine learning methods are needed. In our research, we discovered that the XGBoost and random forest models exhibited the best predictive performance and were most frequently utilized for predicting the onset of sepsis.

TRIAL REGISTRATION

CRD42022384015.

Systematic review finds risk of bias and applicability concerns for models predicting central line-associated bloodstream infection

OBJECTIVES

To systematically review the risk of bias and applicability of published prediction models for risk of central line-associated bloodstream infection (CLA-BSI) in hospitalized patients.

STUDY DESIGN AND SETTING

Systematic review of literature in PubMed, Embase, Web of Science Core Collection, and Scopus up to July 10, 2023. Two authors independently appraised risk models using CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) and assessed their risk of bias and applicability using Prediction model Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

Sixteen studies were included, describing 37 models. When studies presented multiple algorithms, we focused on the model that was selected as the best by the study authors. Eventually we appraised 19 models, among which 15 were regression models and four machine learning models. All models were at a high risk of bias, primarily due to inappropriate proxy outcomes, predictors that are unavailable at prediction time in clinical practice, inadequate sample size, negligence of missing data, lack of model validation, and absence of calibration assessment. 18 out of 19 models had a high concern for applicability, one model had unclear concern for applicability due to incomplete reporting.

CONCLUSION

We did not identify a prediction model of potential clinical use. There is a pressing need to develop an applicable model for CLA-BSI.

Sepsis Prediction by Using a Hybrid Metaheuristic Algorithm: A Novel Approach for Optimizing Deep Neural Networks

The early diagnosis of sepsis reduces the risk of the patient's death. Gradient-based algorithms are applied to the neural network models used in the estimation of sepsis in the literature. However, these algorithms become stuck at the local minimum in solution space. In recent years, swarm intelligence and an evolutionary approach have shown proper results. In this study, a novel hybrid metaheuristic algorithm was proposed for optimization with regard to the weights of the deep neural network and applied for the early diagnosis of sepsis. The proposed algorithm aims to reach the global minimum with a local search strategy capable of exploring and exploiting particles in Particle Swarm Optimization (PSO) and using the mental search operator of the Human Mental Search algorithm (HMS). The benchmark functions utilized to compare the performance of HMS, PSO, and HMS-PSO revealed that the proposed approach is more reliable, durable, and adjustable than other applied algorithms. HMS-PSO is integrated with a deep neural network (HMS-PSO-DNN). The study focused on predicting sepsis with HMS-PSO-DNN, utilizing a dataset of 640 patients aged 18 to 60. The HMS-PSO-DNN model gave a better mean squared error (MSE) result than other algorithms in terms of accuracy, robustness, and performance. We obtained the MSE value of 0.22 with 30 independent runs.

Graph convolutional network-based fusion model to predict risk of hospital acquired infections

OBJECTIVE

Hospital acquired infections (HAIs) are one of the top 10 leading causes of death within the United States. While current standard of HAI risk prediction utilizes only a narrow set of predefined clinical variables, we propose a graph convolutional neural network (GNN)-based model which incorporates a wide variety of clinical features.

MATERIALS AND METHODS

Our GNN-based model defines patients' similarity based on comprehensive clinical history and demographics and predicts all types of HAI rather than focusing on a single subtype. An HAI model was trained on 38 327 unique hospitalizations while a distinct model for surgical site infection (SSI) prediction was trained on 18 609 hospitalization. Both models were tested internally and externally on a geographically disparate site with varying infection rates.

RESULTS

The proposed approach outperformed all baselines (single-modality models and length-of-stay [LoS]) with achieved area under the receiver operating characteristics of 0.86 [0.84-0.88] and 0.79 [0.75-0.83] (HAI), and 0.79 [0.75-0.83] and 0.76 [0.71-0.76] (SSI) for internal and external testing. Cost-effective analysis shows that the GNN modeling dominated the standard LoS model strategy on the basis of lower mean costs ($1651 vs $1915).

DISCUSSION

The proposed HAI risk prediction model can estimate individualized risk of infection for patient by taking into account not only the patient's clinical features, but also clinical features of similar patients as indicated by edges of the patients' graph.

CONCLUSIONS

The proposed model could allow prevention or earlier detection of HAI, which in turn could decrease hospital LoS and associated mortality, and ultimately reduce the healthcare cost.

Strategies of Predictive Schemes and Clinical Diagnosis for Prognosis Using MIMIC-III: A Systematic Review

The prime purpose of the proposed study is to construct a novel predictive scheme for assisting in the prognosis of criticality using the MIMIC-III dataset. With the adoption of various analytics and advanced computing in the healthcare system, there is an increasing trend toward developing an effective prognostication mechanism. Predictive-based modeling is the best alternative to work in this direction. This paper discusses various scientific contributions using desk research methodology towards the Medical Information Mart for Intensive Care (MIMIC-III). This open-access dataset is meant to help predict patient trajectories for various purposes ranging from mortality forecasting to treatment planning. With a dominant machine learning approach in this perspective, there is a need to discover the effectiveness of existing predictive methods. The resultant outcome of this paper offers an inclusive discussion about various available predictive schemes and clinical diagnoses using MIMIC-III in order to contribute toward better information associated with its strengths and weaknesses. Therefore, the paper provides a clear visualization of existing schemes for clinical diagnosis using a systematic review approach.

Changes in Central Line–Associated Bloodstream Infection (CLABSI) Rates Following Implementation of Levofloxacin Prophylaxis for Children and Adolescents With High-Risk Leukemia

Background: Despite initiatives to reduce central line–associated bloodstream infection (CLABSI), children and adolescents with hematologic malignancies, as well as those with relapsed disease, remain at the greatest risk for infection. This single-institution project evaluated changes in CLABSI rates following implementation of antibacterial prophylaxis with levofloxacin for patients with high-risk hematologic malignancies. Methods: Positive blood culture events meeting National Health Safety Network surveillance criteria to be classified as CLABSIs from January 1, 2006, to December 31, 2019, were included. Data were organized into four time periods for comparison based on implementation of CLABSI-reduction interventions. Conditional Poisson regression models were used to evaluate the effect of time (intervention period) on CLABSI rates with post hoc Tukey pairwise comparisons between each of the four time periods. Results: From 2006 and 2019, 227 patients experienced 310 CLABSIs. Clinically important decreases in CLABSI rates from baseline (4.84 per 1,000 line days) occurred with implementation of Children's Hospital Association (CHA) bundles (3.29 per 1,000 line days); however, this difference was not significant ( p  =  .16). CLABSI rates decreased from baseline with the addition of formalized supportive cares (2.66 per 1,000 line days; incidence rate ratio [IRR]  =  0.60; p < .01), and with the use of antibacterial prophylaxis (1.66 per 1,000 line days; IRR  =  0.35; p < .01). Post hoc comparisons indicated decreased CLABSI rates with the use of antibacterial prophylaxis compared with CHA bundles alone (IRR  =  0.49; p  =  .011) and CHA bundles plus formalized supportive cares (IRR  =  0.58; p  =  .046). Discussion: Results demonstrate sustained success using a practice-based evidence approach to guide CLABSI-reduction interventions. Follow-up research, applying machine learning algorithms, may identify additional risk factors and inform future interventions.

Mapping progress in intravascular catheter quality surveillance: An Australian case study of electronic medical record data linkage

Background and significance

Intravascular (IV) catheters are the most invasive medical device in healthcare. Localized priority-setting related to IV catheter quality surveillance is a key objective of recent healthcare reform in Australia. We sought to determine the plausibility of using electronic health record (EHR) data for catheter surveillance by mapping currently available data across state-wide platforms. This work has identified barriers and facilitators to a state-wide EHR surveillance initiative.

Materials and methods

Data variables were generated and mapped from routinely used EHR sources across Queensland, Australia through a systematic search of gray literature and expert consultation with clinical information specialists. EHR systems were eligible for inclusion if they collected data related to IV catheter insertion, care, or outcomes of hospitalized patients. Generated variables were mapped against international recommendations for IV catheter surveillance, with data linkage and data export capacity narratively summarized.

Results

We identified five EHR systems, namely, iEMR, MetaVision ICU®, Multiprac, RiskMan, and the Nephrology Registry. Systems were used across jurisdictions and hospital wards. Data linkage was not evident across systems. Extraction processes for catheter data were not standardized, lacking clear and reliable extraction techniques. In combination, EHR systems collected 43/50 international variables recommended for catheter surveillance, however, individual systems collected a median of 24/50 (IQR 22, 30) variables. We did not identify integrated clinical analytic systems (incorporating machine learning) to support clinical decision making or for risk stratification (e.g., catheter-related infection).

Conclusion

Current data linkage across EHR systems limits the development of an IV catheter quality surveillance system to provide timely data related to catheter complications and harm. To facilitate reliable and timely surveillance of catheter outcomes using clinical informatics, substantial work is needed to overcome existing barriers and transform health surveillance.

Development and validation of BLOOMY prediction scores for 14-day and 6-month mortality in hospitalised adults with bloodstream infections: a multicentre, prospective, cohort study

BACKGROUND

The burden of bloodstream infections remains high worldwide and cannot be confined to short-term in-hospital mortality. We aimed to develop scores to predict short-term and long-term mortality in patients with bloodstream infections.

METHODS

The Bloodstream Infection due to Multidrug-resistant Organisms: Multicenter Study on Risk Factors and Clinical Outcomes (BLOOMY) study is a prospective, multicentre cohort study at six German tertiary care university hospitals to develop and validate two scores assessing 14-day and 6-month mortality in patients with bloodstream infections. We excluded patients younger than 18 years or who were admitted to an ophthalmology or psychiatry ward. Microbiological, clinical, laboratory, treatment, and survival data were prospectively collected on day 0 and day 3 and then from day 7 onwards, weekly. Participants were followed up for 6 months. All patients in the derivation cohort who were alive on day 3 were included in the analysis. Predictive scores were developed using logistic regression and Cox proportional hazards models with a machine-learning approach. Validation was completed using the C statistic and predictive accuracy was assessed using sensitivity, specificity, and predictive values.

FINDINGS

Between Feb 1, 2017, and Jan 31, 2019, 2568 (61·5%) of 4179 eligible patients were recruited into the derivation cohort. The in-hospital mortality rate was 23·75% (95% CI 22·15-25·44; 610 of 2568 patients) and the 6-month mortality rate was 41·55% (39·54-43·59; 949 of 2284). The model predictors for 14-day mortality (C statistic 0·873, 95% CI 0·849-0·896) and 6-month mortality (0·807, 0·784-0·831) included age, body-mass index, platelet and leukocyte counts, C-reactive protein concentrations, malignancy (ie, comorbidity), in-hospital acquisition, and pathogen. Additional predictors were, for 14-day mortality, mental status, hypotension, and the need for mechanical ventilation on day 3 and, for 6-month mortality, focus of infection, in-hospital complications, and glomerular filtration rate at the end of treatment. The scores were validated in a cohort of 1023 patients with bloodstream infections, recruited between Oct 9, 2019, and Dec 31, 2020. The BLOOMY 14-day score showed a sensitivity of 61·32% (95% CI 51·81-70·04), a specificity of 86·36% (83·80-88·58), a positive predictive value (PPV) of 37·57% (30·70-44·99), and a negative predictive value (NPV) of 94·35% (92·42-95·80). The BLOOMY 6-month score showed a sensitivity of 69·93% (61·97-76·84), a specificity of 66·44% (61·86-70·73), a PPV of 40·82% (34·85-47·07), and a NPV of 86·97% (82·91-90·18).

INTERPRETATION

The BLOOMY scores showed good discrimination and predictive values and could support the development of protocols to manage bloodstream infections and also help to estimate the short-term and long-term burdens of bloodstream infections.

FUNDING

DZIF German Center for Infection Research.

TRANSLATION

For the German translation of the abstract see Supplementary Materials section.

Early prediction of central line associated bloodstream infection using machine learning

BACKGROUND

Central line-associated bloodstream infections (CLABSIs) are associated with significant morbidity, mortality, and increased healthcare costs. Despite the high prevalence of CLABSIs in the U.S., there are currently no tools to stratify a patient's risk of developing an infection as the result of central line placement. To this end, we have developed and validated a machine learning algorithm (MLA) that can predict a patient's likelihood of developing CLABSI using only electronic health record data in order to provide clinical decision support.

METHODS

We created three machine learning models to retrospectively analyze electronic health record data from 27,619 patient encounters. The models were trained and validated using an 80:20 split for the train and test data. Patients designated as having a central line procedure based on International Statistical Classification of Diseases and Related Health Problems 10 codes were included.

RESULTS

XGBoost was the highest performing MLA out of the three models, obtaining an AUROC of 0.762 for CLABSI risk prediction at 48 hours after the recorded time for central line placement.

CONCLUSIONS

Our results demonstrate that MLAs may be effective clinical decision support tools for assessment of CLABSI risk and should be explored further for this purpose.

Bacteremia detection from complete blood count and differential leukocyte count with machine learning: complementary and competitive with C-reactive protein and procalcitonin tests

Background

Biomarkers, such as leukocyte count, C-reactive protein (CRP), and procalcitonin (PCT), have been commonly used to predict the occurrence of life-threatening bacteremia and provide prognostic information, given the need for prompt intervention. However, such diagnosis methods require much time and money. Therefore, we propose a method with a high prediction capability using machine learning (ML) models based on complete blood count (CBC) and differential leukocyte count (DC) and compare its performance with traditional CRP or PCT biomarker methods and those of models incorporating CRP or PCT biomarkers.

Methods

We collected 366,586 daily blood culture (BC) results, of which 350,775 (93.2%), 308,803 (82.1%), and 23,912 (6.4%) cases were issued CBC/DC (CBC/DC group), CRP with CBC/DC (CRP&CBC/DC group), and PCT with CBC/DC (PCT&CBC/DC group), respectively. For the ML methods, conventional logistic regression and random forest models were selected, trained, applied, and validated for each group. Fivefold validation and prediction capability were also evaluated and reported.

Results

Overall, the ML methods, such as the random forest model, demonstrated promising performances. When trained with CBC/DC data, it achieved an area under the ROC curve (AUC) of 0.802, which is superior to the prediction conventionally made with CRP/PCT levels (0.699/0.731). Upon evaluating the performance enhanced by incorporating CRP or PCT biomarkers, it reported no substantial AUC increase with the addition of either CRP or PCT to CBC/DC data, which suggests the predicting power and applicability of using only CBC/DC data. Moreover, it showed competitive prognostic capability compared to the PCT test with similar all-cause in-hospital mortality (45.10% vs. 47.40%) and overall median survival time (27 vs. 25 days).

Conclusions

The ML models using only CBC/DC data yielded more accurate bacteremia predictions compared to those by methods using CRP and PCT data and reached similar prognostic performance as by PCT data. Thus, such models are potentially complementary and competitive with traditional CRP and PCT biomarkers for conducting and guiding antibiotic usage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-022-07223-7.

Artificial Intelligence in Infection Management in the ICU

This article is one of ten reviews selected from the Annual Update in Intensive Care and Emergency Medicine 2022. Other selected articles can be found online at https://www.biomedcentral.com/collections/annualupdate2022. Further information about the Annual Update in Intensive Care and Emergency Medicine is available from https://link.springer.com/bookseries/8901.

Deep Learning Model to Predict Serious Infection Among Children With Central Venous Lines

Objective: Predict the onset of presumed serious infection, defined as a positive blood culture drawn and new antibiotic course of at least 4 days (PSI*), among pediatric patients with Central Venous Lines (CVLs). Design: Retrospective cohort study. Setting: Single academic children's hospital. Patients: All hospital encounters from January 2013 to December 2018, excluding the ones without a CVL or with a length-of-stay shorter than 24 h. Measurements and Main Results: Clinical features including demographics, laboratory results, vital signs, characteristics of the CVLs and medications used were extracted retrospectively from electronic medical records. Data were aggregated across all hospitals within a single pediatric health system and used to train a deep learning model to predict the occurrence of PSI* during the next 48 h of hospitalization. The proposed model prediction was compared to prediction of PSI* by a marker of illness severity (PELOD-2). The baseline prevalence of line infections was 0.34% over all segmented 48-h time windows. Events were identified among cases using onset time. All data from admission till the onset was used for cases and among controls we used all data from admission till discharge. The benchmarks were aggregated over all 48 h time windows [N=748,380 associated with 27,137 patient encounters]. The model achieved an area under the receiver operating characteristic curve of 0.993 (95% CI = [0.990, 0.996]), the enriched positive predictive value (PPV) was 23 times greater than the base prevalence. Conversely, prediction by PELOD-2 achieved a lower PPV of 1.5% [0.9%, 2.1%] which was 5 times the baseline prevalence. Conclusion: A deep learning model that employs common clinical features in the electronic health record can help predict the onset of CLABSI in hospitalized children with central venous line 48 hours prior to the time of specimen collection.

Using Unsupervised Machine Learning to Identify Age- and Sex-Independent Severity Subgroups Among Patients with COVID-19: Observational Longitudinal Study

Background

Early detection and intervention are the key factors for improving outcomes in patients with COVID-19.

Objective

The objective of this observational longitudinal study was to identify nonoverlapping severity subgroups (ie, clusters) among patients with COVID-19, based exclusively on clinical data and standard laboratory tests obtained during patient assessment in the emergency department.

Methods

We applied unsupervised machine learning to a data set of 853 patients with COVID-19 from the HM group of hospitals (HM Hospitales) in Madrid, Spain. Age and sex were not considered while building the clusters, as these variables could introduce biases in machine learning algorithms and raise ethical implications or enable discrimination in triage protocols.

Results

From 850 clinical and laboratory variables, four tests—the serum levels of aspartate transaminase (AST), lactate dehydrogenase (LDH), C-reactive protein (CRP), and the number of neutrophils—were enough to segregate the entire patient pool into three separate clusters. Further, the percentage of monocytes and lymphocytes and the levels of alanine transaminase (ALT) distinguished cluster 3 patients from the other two clusters. The highest proportion of deceased patients; the highest levels of AST, ALT, LDH, and CRP; the highest number of neutrophils; and the lowest percentages of monocytes and lymphocytes characterized cluster 1. Cluster 2 included a lower proportion of deceased patients and intermediate levels of the previous laboratory tests. The lowest proportion of deceased patients; the lowest levels of AST, ALT, LDH, and CRP; the lowest number of neutrophils; and the highest percentages of monocytes and lymphocytes characterized cluster 3.

Conclusions

A few standard laboratory tests, deemed available in all emergency departments, have shown good discriminative power for the characterization of severity subgroups among patients with COVID-19.

How does innovative technology impact nursing in infectious diseases and infection control? A scoping review

Aim

Considering the increasing number of emerging infectious diseases, innovative approaches are strongly in demand. Additionally, research in this field has expanded exponentially. Thus, faced with this diverse information, we aim to clarify key concepts and knowledge gaps of technology in nursing and the field of infectious diseases.

Design

This scoping review followed the methodology of scoping review guidance from Arksey and O’Malley.

Methods

Six databases were searched systematically (PubMed, Web of Science, IEEE Explore, EBSCOhost, Cochrane Library and Summon). After the removal of duplicates, 532 citations were retrieved and 77 were included in the analysis.

Results

We identified five major trends in technology for nursing and infectious diseases: artificial intelligence, the Internet of things, information and communications technology, simulation technology and e‐learning. Our findings indicate that the most promising trend is the IoT because of the many positive effects validated in most of the reviewed studies.

A machine learning approach to predict healthcare-associated infections at intensive care unit admission: findings from the SPIN-UTI project

BACKGROUND

Identifying patients at higher risk of healthcare-associated infections (HAIs) in intensive care units (ICUs) represents a major challenge for public health. Machine learning could improve patient risk stratification and lead to targeted infection prevention and control interventions.

AIM

To evaluate the performance of the Simplified Acute Physiology Score (SAPS) II for HAI risk prediction in ICUs, using both traditional statistical and machine learning approaches.

METHODS

Data for 7827 patients from the 'Italian Nosocomial Infections Surveillance in Intensive Care Units' project were used in this study. The Support Vector Machines (SVM) algorithm was applied to classify patients according to sex, patient origin, non-surgical treatment for acute coronary disease, surgical intervention, SAPS II at admission, presence of invasive devices, trauma, impaired immunity, and antibiotic therapy in 48 h preceding ICU admission.

FINDINGS

The performance of SAPS II for predicting HAI risk provides a receiver operating characteristic curve with an area under the curve of 0.612 (P<0.001) and accuracy of 56%. Considering SAPS II along with other characteristics at ICU admission, the SVM classifier was found to have accuracy of 88% and an AUC of 0.90 (P<0.001) for the test set. The predictive ability was lower when considering the same SVM model but with the SAPS II variable removed (accuracy 78%, AUC 0.66).

CONCLUSIONS

This study suggested that the SVM model is a useful tool for early prediction of patients at higher risk of HAIs at ICU admission.

Early Prediction of Seven-Day Mortality in Intensive Care Unit Using a Machine Learning Model: Results from the SPIN-UTI Project

Patients in intensive care units (ICUs) were at higher risk of worsen prognosis and mortality. Here, we aimed to evaluate the ability of the Simplified Acute Physiology Score (SAPS II) to predict the risk of 7-day mortality, and to test a machine learning algorithm which combines the SAPS II with additional patients’ characteristics at ICU admission. We used data from the “Italian Nosocomial Infections Surveillance in Intensive Care Units” network. Support Vector Machines (SVM) algorithm was used to classify 3782 patients according to sex, patient’s origin, type of ICU admission, non-surgical treatment for acute coronary disease, surgical intervention, SAPS II, presence of invasive devices, trauma, impaired immunity, antibiotic therapy and onset of HAI. The accuracy of SAPS II for predicting patients who died from those who did not was 69.3%, with an Area Under the Curve (AUC) of 0.678. Using the SVM algorithm, instead, we achieved an accuracy of 83.5% and AUC of 0.896. Notably, SAPS II was the variable that weighted more on the model and its removal resulted in an AUC of 0.653 and an accuracy of 68.4%. Overall, these findings suggest the present SVM model as a useful tool to early predict patients at higher risk of death at ICU admission.

Identifying Urinary Tract Infection-Related Information in Home Care Nursing Notes

Objectives:

Urinary tract infection (UTI) is common in home care but not easily captured with standard assessment. This study aimed to examine the value of nursing notes in detecting UTI signs and symptoms in home care.

Design:

The study developed a natural language processing (NLP) algorithm to automatically identify UTI-related information in nursing notes.

Setting and Participants:

Home care visit notes (n = 1,149,586) and care coordination notes (n = 1,461,171) for 89,459 patients treated in the largest nonprofit home care agency in the United States during 2014.

Measures:

We generated 6 categories of UTI-related information from literature and used the Unified Medical Language System (UMLS) to identify a preliminary list of terms. The NLP algorithm was tested on a gold standard set of 300 clinical notes annotated by clinical experts. We used structured Outcome and Assessment Information Set data to extract the frequency of UTI-related emergency department (ED) visits or hospitalizations and explored time-patterns in documentation of UTI-related information.

Results:

The NLP system achieved very good overall performance (F measure = 0.9, 95% CI: 0.87–0.93) based on the test results obtained by using the notes for patients admitted to the ED or hospital due to UTI. UTI-related information was significantly more prevalent (P < .01 for all the tests) in home care episodes with UTI-related ED admission or hospitalization vs the general patient population; 81% of home care episodes with UTI-related hospitalization or ED admission had at least 1 category of UTI-related information vs 21.6% among episodes without UTI-related hospitalization or ED admission. Frequency of UTI-related information documentation increased in advance of UTI-related hospitalization or ED admission, peaking within a few days before the event.

Conclusions and Implications:

Information in nursing notes is often overlooked by stakeholders and not integrated into predictive modeling for decision-making support, but our findings highlight their value in early risk identification and care guidance. Health care administrators should consider using NLP to extract clinical data from nursing notes to improve early detection and treatment, which may lead to quality improvement and cost reduction.

Physician and Nurse Research in Multidisciplinary Intensive Care Units

BACKGROUND

Although clinical care is multidisciplinary, intensive care unit research commonly focuses on single-discipline themes. We sought to characterize intensive care unit research conducted by physicians and nurses.

METHODS

One hundred randomly selected reports of clinical studies published in critical care medical and nursing journals were reviewed.

RESULTS

Of the 100 articles reviewed, 50 were published in medical journals and 50 were published in nursing journals. Only 1 medical study (2%) used qualitative methods, compared with 9 nursing studies (18%) (P = .02). The distribution of quantitative study designs differed between medical and nursing journals (P < .001), with medical journals having a predominance of cohort studies (29 articles [58%]). Compared with medical journal articles, nursing journal articles had significantly fewer authors (median [interquartile range], 5 [3-6] vs 8 [6-10]; P < .001) and study participants (94 [51-237] vs 375 [86-4183]; P < .001) and a significantly lower proportion of male study participants (55% [26%-65%] vs 60% [51%-65%]; P = .02). Studies published in medical journals were much more likely than those published in nursing journals to exclusively involve patients as participants (47 [94%] vs 25 [50%]; P < .001). Coauthorship between physicians and nurses was evident in 14 articles (14%), with infrequent inclusion of authors from other health care disciplines.

CONCLUSIONS

Physician research and nurse research differ in several important aspects and tend to occur within silos. Increased interprofessional collaboration is possible and worthwhile.

Comparing Machine Learning Algorithms for Predicting Acute Kidney Injury

Prior studies have used vital signs and laboratory measurements with conventional modeling techniques to predict acute kidney injury (AKI). The purpose of this study was to use the trend in vital signs and laboratory measurements with machine learning algorithms for predicting AKI in ICU patients. The eICU Collaborative Research Database was queried for five consecutive days of laboratory measurements per patient. Patients with AKI were identified and trends in vital signs and laboratory values were determined by calculating the slope of the least-squares-fit linear equation using three days for each value. Different machine learning classifiers (gradient boosted trees [GBT], logistic regression, and deep learning) were trained to predict AKI using the laboratory values, vital signs, and slopes. There were 151,098 ICU stays identified and the rate of AKI was 5.6 per cent. The best performing algorithm was GBT with an AUC of 0.834 ± 0.006 and an F-measure of 42.96 per cent ± 1.26 per cent. Logistic regression performed with an AUC of 0.827 ± 0.004 and an F-measure of 28.29 per cent ± 1.01 per cent. Deep learning performed with an AUC of 0.817 ± 0.005 and an F-measure of 42.89 per cent ± 0.91 per cent. The most important variable for GBT was the slope of the minimum creatinine (30.32%). This study identifies the best performing machine learning algorithms for predicting AKI using trends in laboratory values in ICU patients. Early identification of these patients using readily available data indicates that incorporating machine learning predictive models into electronic medical record systems is an inevitable requisite for improving patient outcomes.

The development an artificial intelligence algorithm for early sepsis diagnosis in the intensive care unit

BACKGROUND

Severe sepsis and septic shock are still the leading causes of death in Intensive Care Units (ICUs), and timely diagnosis is crucial for treatment outcomes. The progression of electronic medical records (EMR) offers the possibility of storing a large quantity of clinical data that can facilitate the development of artificial intelligence (AI) in medicine. However, several difficulties, such as poor structure and heterogenicity of the raw EMR data, are encountered when introducing AI with ICU data. Labor-intensive work, including manual data entry, personal medical records sorting, and laboratory results interpretation may hinder the progress of AI. In this article, we introduce the developing of an AI algorithm designed for sepsis diagnosis using pre-selected features; and compare the performance of the AI algorithm with SOFA score based diagnostic method.

MATERIALS AND METHODS

This is a prospective open-label cohort study. A specialized EMR, named TED_ICU, was implemented for continuous data recording. One hundred six clinical features relevant to sepsis diagnosis were selected prospectively. A labeling work to allocate SEPSIS or NON_SEPSIS status for each ICU patient was performed by the in-charge intensivist according to SEPSIS-3 criteria, along with the automatic recording of selected features every day by TED_ICU. Afterward, we use de-identified data to develop the AI algorithm. Several machine learning methods were evaluated using 5-fold cross-validation, and XGBoost, a decision-tree based algorithm was adopted for our AI algorithm development due to best performance.

RESULTS

The study was conducted between August 2018 and December 2018 for the first stage of analysis. We collected 1588 instances, including 444 SEPSIS and 1144 NON-SEPSIS, from 434 patients. The 434 patients included 259 (59.6%) male patients and 175 female patients. The mean age was 67.6-year-old, and the mean APACHE II score was 13.8. The SEPSIS cohort had a higher SOFA score and increased use of organ support treatment. The AI algorithm was developed with a shuffle method using 80% of the instances for training and 20% for testing. The established AI algorithm achieved the following: accuracy = 82% ± 1%; sensitivity = 65% ± 5%; specificity = 88% ± 2%; precision = 67% ± 3%; and F1 = 0.66 ± 0.02. The area under the receiver operating characteristic curve (AUROC) was approximately 0.89. The SOFA score was used on the same 1588 instances for sepsis diagnosis, and the result was inferior to our AI algorithm (AUROC = 0.596).

CONCLUSION

Using real-time data, collected by EMR, from the ICU daily practice, our AI algorithm established with pre-selected features and XGBoost can provide a timely diagnosis of sepsis with an accuracy greater than 80%. AI algorithm also outperforms the SOFA score in sepsis diagnosis and exhibits practicality as clinicians can deploy appropriate treatment earlier. The early and precise response of this AI algorithm will result in cost reduction, outcome improvement, and benefit for healthcare systems, medical staff, and patients as well.

Analysis of adult disease characteristics and mortality on MIMIC-III

Purpose

To deeply analyze the basic information and disease information of adult patients in the MIMIC-III (Medical Information Mart for Intensive Care III) database, and provide data reference for clinicians and researchers.

Materials and methods

Tableau2019.1.0 and Navicat12.0.29 were used for data analysis and extraction of disease distribution of adult patients in the MIMIC-III database.

Result

A total of 38,163 adult patients were included in the MIMIC-III database. Only 38,156 patients with the first diagnosis were selected. Among them, 21,598 were males accounting for 56.6% the median age was 66 years (Q1-Q3: 53–78), the median length of a hospital stay was 7 days (Q1-Q3: 4–12), and the median length of an ICU stay was 2.1 days (Q1-Q3: 1.2–4.1). Septicemia was the disease with the highest mortality rate among patients and the total mortality rate was 48.9%. The disease with the largest number of patients at the last time was other forms of chronic ischemic heart disease.

Conclusion

By analyzing the patients’ basic information, the admission spectrum and the disease morbidity and mortality can help more researchers understand the MIMIC-III database and facilitate further research.

Machine learning in infection management using routine electronic health records: tools, techniques, and reporting of future technologies

BACKGROUND

Machine learning (ML) is increasingly being used in many areas of health care. Its use in infection management is catching up as identified in a recent review in this journal. We present here a complementary review to this work.

OBJECTIVES

To support clinicians and researchers in navigating through the methodological aspects of ML approaches in the field of infection management.

SOURCES

A Medline search was performed with the keywords artificial intelligence, machine learning, infection∗, and infectious disease∗ for the years 2014-2019. Studies using routinely available electronic hospital record data from an inpatient setting with a focus on bacterial and fungal infections were included.

CONTENT

Fifty-two studies were included and divided into six groups based on their focus. These studies covered detection/prediction of sepsis (n = 19), hospital-acquired infections (n = 11), surgical site infections and other postoperative infections (n = 11), microbiological test results (n = 4), infections in general (n = 2), musculoskeletal infections (n = 2), and other topics (urinary tract infections, deep fungal infections, antimicrobial prescriptions; n = 1 each). In total, 35 different ML techniques were used. Logistic regression was applied in 18 studies followed by random forest, support vector machines, and artificial neural networks in 18, 12, and seven studies, respectively. Overall, the studies were very heterogeneous in their approach and their reporting. Detailed information on data handling and software code was often missing. Validation on new datasets and/or in other institutions was rarely done. Clinical studies on the impact of ML in infection management were lacking.

IMPLICATIONS

Promising approaches for ML use in infectious diseases were identified. But building trust in these new technologies will require improved reporting. Explainability and interpretability of the models used were rarely addressed and should be further explored. Independent model validation and clinical studies evaluating the added value of ML approaches are needed.

Evidence-based Clinical Decision Support Systems for the prediction and detection of three disease states in critical care: A systematic literature review

Background: Clinical decision support (CDS) systems have emerged as tools providing intelligent decision making to address challenges of critical care. CDS systems can be based on existing guidelines or best practices; and can also utilize machine learning to provide a diagnosis, recommendation, or therapy course. Methods: This research aimed to identify evidence-based study designs and outcome measures to determine the clinical effectiveness of clinical decision support systems in the detection and prediction of hemodynamic instability, respiratory distress, and infection within critical care settings. PubMed, ClinicalTrials.gov and Cochrane Database of Systematic Reviews were systematically searched to identify primary research published in English between 2013 and 2018. Studies conducted in the USA, Canada, UK, Germany and France with more than 10 participants per arm were included. Results: In studies on hemodynamic instability, the prediction and management of septic shock were the most researched topics followed by the early prediction of heart failure. For respiratory distress, the most popular topics were pneumonia detection and prediction followed by pulmonary embolisms. Given the importance of imaging and clinical notes, this area combined Machine Learning with image analysis and natural language processing. In studies on infection, the most researched areas were the detection, prediction, and management of sepsis, surgical site infections, as well as acute kidney injury. Overall, a variety of Machine Learning algorithms were utilized frequently, particularly support vector machines, boosting techniques, random forest classifiers and neural networks. Sensitivity, specificity, and ROC AUC were the most frequently reported performance measures. Conclusion: This review showed an increasing use of Machine Learning for CDS in all three areas. Large datasets are required for training these algorithms; making it imperative to appropriately address, challenges such as class imbalance, correct labelling of data and missing data. Recommendations are formulated for the development and successful adoption of CDS systems.

Evidence-based Clinical Decision Support Systems for the prediction and detection of three disease states in critical care: A systematic literature review

Background: Clinical decision support (CDS) systems have emerged as tools providing intelligent decision making to address challenges of critical care. CDS systems can be based on existing guidelines or best practices; and can also utilize machine learning to provide a diagnosis, recommendation, or therapy course. Methods: This research aimed to identify evidence-based study designs and outcome measures to determine the clinical effectiveness of clinical decision support systems in the detection and prediction of hemodynamic instability, respiratory distress, and infection within critical care settings. PubMed, ClinicalTrials.gov and Cochrane Database of Systematic Reviews were systematically searched to identify primary research published in English between 2013 and 2018. Studies conducted in the USA, Canada, UK, Germany and France with more than 10 participants per arm were included. Results: In studies on hemodynamic instability, the prediction and management of septic shock were the most researched topics followed by the early prediction of heart failure. For respiratory distress, the most popular topics were pneumonia detection and prediction followed by pulmonary embolisms. Given the importance of imaging and clinical notes, this area combined Machine Learning with image analysis and natural language processing. In studies on infection, the most researched areas were the detection, prediction, and management of sepsis, surgical site infections, as well as acute kidney injury. Overall, a variety of Machine Learning algorithms were utilized frequently, particularly support vector machines, boosting techniques, random forest classifiers and neural networks. Sensitivity, specificity, and ROC AUC were the most frequently reported performance measures. Conclusion: This review showed an increasing use of Machine Learning for CDS in all three areas. Large datasets are required for training these algorithms; making it imperative to appropriately address, challenges such as class imbalance, correct labelling of data and missing data. Recommendations are formulated for the development and successful adoption of CDS systems.

Inhospital death is a biased measure of fatal outcome from bloodstream infection

Purpose

Inhospital death is commonly used as an outcome measure. However, it may be a biased measure of overall fatal outcome. The objective of this study was to evaluate inhospital death as a measure of all-cause 30-day case fatality in patients with bloodstream infection (BSI).

Patients and methods

A population-based surveillance cohort study was conducted, and patients who died in hospital within 30 days (30-day inhospital death) were compared with those who died in any location by day 30 post BSI diagnosis (30-day all-cause case fatality).

Results

A total of 1,773 residents had first incident episodes of BSI. Overall, 299 patients died for a 30-day all-cause case fatality rate of 16.9%. Most (1,587; 89.5%) of the patients were admitted to hospital, and ten (5.4%) of the 186 patients not admitted to hospital died. Of the 1,587 admitted patients, 242 died for a 30-day inhospital death rate of 15.2%. A further 47 patients admitted to hospital died after discharge but within 30 days of BSI diagnosis for a 30-day case fatality rate among admitted patients of 18.2%. Patients who died following discharge within 30 days were older and more likely to have dementia.

Conclusion

The use of inhospital death is a biased measure of true case fatality.

Introduction to Machine Learning in Digital Healthcare Epidemiology

To exploit the full potential of big routine data in healthcare and to efficiently communicate and collaborate with information technology specialists and data analysts, healthcare epidemiologists should have some knowledge of large-scale analysis techniques, particularly about machine learning. This review focuses on the broad area of machine learning and its first applications in the emerging field of digital healthcare epidemiology.