Evidence-based Clinical Decision Support Systems for the prediction and detection of three disease states in critical care: A systematic literature review

Neural networks for the prediction of bacterial and fungal infections: current evidence and implications

In the present narrative review, we discuss the use of artificial neural networks (ANNs) for predicting bacterial and fungal infections based on commonly available clinical and laboratory data, focusing on promises and challenges of these machine learning models. For predicting different bacterial or fungal infections from data commonly found in electronical medical records, ANN models may reach, based on current literature, an acceptable performance for discriminating between infected and non-infected patients, and outperformed other machine learning (ML)-based models in 38.3% of the retrieved studies evaluating at least another ML approach. In the near future, as for other ML models, the use of ANNs could be leveraged to provide real-time support to clinicians in clinical decision-making processes, although further research is needed in terms of quality of data and explainability of ANN model predictions to better understand whether and how these techniques can be safely adopted in everyday clinical practice.

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

Artificial intelligence in oral health surveillance among under-served communities

A sizable percentage of the population in India still does not have easy access to dental facilities. Therefore, it is of interest to document the role of artificial intelligence (AI) in oral surveillance of underserved communities. Available data shows that AI makes it possible to screen, diagnose, track, prioritize, and monitor dental patients remotely via smart devices. As a result, dentists won't have to deal with simple situations that only require standard treatments; freeing them up to focus on more complicated cases. Additionally, this would allow dentists to reach a broader, more underprivileged population in difficult-to-reach places. AI fracture recognition and categorization performance has shown promise in preliminary testing. Methods for detecting aberrations are frequently employed in public health practise and research continues to be focused on them.

Critical risk and success factors for sustainability of an electronic health data capture, processing and dissemination platform for Uganda

Several studies have investigated challenges that have marred success or even caused the failure of eHealth implementations in Uganda; however, none has focused on the risks and success factors of their sustainability. This study explored critical risk and success factors for the sustainability of an electronic health data capture, processing and dissemination platform for Uganda. A mixed-method research design was followed involving collecting empirical data from all four regions of Uganda. A purposive sampling strategy was used to select the study districts per region, health facilities per district, and respondents/participants per facility or district. Findings revealed several risks and success factors for sustainability, including; bad leadership, corruption, lack of sustainable maintenance programs, lack of suitable sustainability plans, lack of ICT infrastructure investment, poor management systems, funds, stakeholder buy-ins, data sharing and access rights. The success factors included reinvestments as a partial sustainability plan for ICT infrastructure. These factors can be leveraged to ensure the continued operation of eHealth implementations in Uganda. Every electronic health project aiming at success should always make due consideration/sustainability plan at the onset of project conceptualisation; as lack of such a plan has often resulted in failed projects after the initial funds have been withdrawn.

Artificial intelligence and its impact on the domains of universal health coverage, health emergencies and health promotion: An overview of systematic reviews

BACKGROUND

Artificial intelligence is fueling a new revolution in medicine and in the healthcare sector. Despite the growing evidence on the benefits of artificial intelligence there are several aspects that limit the measure of its impact in people's health. It is necessary to assess the current status on the application of AI towards the improvement of people's health in the domains defined by WHO's Thirteenth General Programme of Work (GPW13) and the European Programme of Work (EPW), to inform about trends, gaps, opportunities, and challenges.

OBJECTIVE

To perform a systematic overview of systematic reviews on the application of artificial intelligence in the people's health domains as defined in the GPW13 and provide a comprehensive and updated map on the application specialties of artificial intelligence in terms of methodologies, algorithms, data sources, outcomes, predictors, performance, and methodological quality.

METHODS

A systematic search in MEDLINE, EMBASE, Cochrane and IEEEXplore was conducted between January 2015 and June 2021 to collect systematic reviews using a combination of keywords related to the domains of universal health coverage, health emergencies protection, and better health and wellbeing as defined by the WHO's PGW13 and EPW. Eligibility criteria was based on methodological quality and the inclusion of practical implementation of artificial intelligence. Records were classified and labeled using ICD-11 categories into the domains of the GPW13. Descriptors related to the area of implementation, type of modeling, data entities, outcomes and implementation on care delivery were extracted using a structured form and methodological aspects of the included reviews studies was assessed using the AMSTAR checklist.

RESULTS

The search strategy resulted in the screening of 815 systematic reviews from which 203 were assessed for eligibility and 129 were included in the review. The most predominant domain for artificial intelligence applications was Universal Health Coverage (N = 98) followed by Health Emergencies (N = 16) and Better Health and Wellbeing (N = 15). Neoplasms area on Universal Health Coverage was the disease area featuring most of the applications (21.7 %, N = 28). The reviews featured analytics primarily over both public and private data sources (67.44 %, N = 87). The most used type of data was medical imaging (31.8 %, N = 41) and predictors based on regions of interest and clinical data. The most prominent subdomain of Artificial Intelligence was Machine Learning (43.4 %, N = 56), in which Support Vector Machine method was predominant (20.9 %, N = 27). Regarding the purpose, the application of Artificial Intelligence I is focused on the prediction of the diseases (36.4 %, N = 47). With respect to the validation, more than a half of the reviews (54.3 %, N = 70) did not report a validation procedure and, whenever available, the main performance indicator was the accuracy (28.7 %, N = 37). According to the methodological quality assessment, a third of the reviews (34.9 %, N = 45) implemented methods for analysis the risk of bias and the overall AMSTAR score below was 5 (4.01 ± 1.93) on all the included systematic reviews.

CONCLUSION

Artificial intelligence is being used for disease modelling, diagnose, classification and prediction in the three domains of GPW13. However, the evidence is often limited to laboratory and the level of adoption is largely unbalanced between ICD-11 categoriesand diseases. Data availability is a determinant factor on the developmental stage of artificial intelligence applications. Most of the reviewed studies show a poor methodological quality and are at high risk of bias, which limits the reproducibility of the results and the reliability of translating these applications to real clinical scenarios. The analyzed papers show results only in laboratory and testing scenarios and not in clinical trials nor case studies, limiting the supporting evidence to transfer artificial intelligence to actual care delivery.

Machine Learning Models for Early Prediction of Sepsis on Large Healthcare Datasets

Sepsis is a highly lethal syndrome with heterogeneous clinical manifestation that can be hard to identify and treat. Early diagnosis and appropriate treatment are critical to reduce mortality and promote survival in suspected cases and improve the outcomes. Several screening prediction systems have been proposed for evaluating the early detection of patient deterioration, but the efficacy is still limited at individual level. The increasing amount and the versatility of healthcare data suggest implementing machine learning techniques to develop models for predicting sepsis. This work presents an experimental study of some machine-learning-based models for sepsis prediction considering vital signs, laboratory test results, and demographics using Medical Information Mart for Intensive Care III (MIMIC-III) (v1.4), a publicly available dataset. The experimental results demonstrate an overall higher performance of machine learning models over the commonly used Sequential Organ Failure Assessment (SOFA) and Quick SOFA (qSOFA) scoring systems at the time of sepsis onset.

Ambiguous definitions for baseline serum creatinine affect acute kidney diagnosis at the emergency department

Background

Acute kidney injury (AKI) incidence is increasing, however AKI is often missed at the emergency department (ED). AKI diagnosis depends on changes in kidney function by comparing a serum creatinine (SCr) measurement to a baseline value. However, it remains unclear to what extent different baseline values may affect AKI diagnosis at ED.

Methods

Routine care data from ED visits between 2012 and 2019 were extracted from the Utrecht Patient Oriented Database. We evaluated baseline definitions with criteria from the RIFLE, AKIN and KDIGO guidelines. We evaluated four baseline SCr definitions (lowest, most recent, mean, median), as well as five different time windows (up to 365 days prior to ED visit) to select a baseline and compared this to the first measured SCr at ED. As an outcome, we assessed AKI prevalence at ED.

Results

We included 47,373 ED visits with both SCr-ED and SCr-BL available. Of these, 46,100 visits had a SCr-BL from the − 365/− 7 days time window. Apart from the lowest value, AKI prevalence remained similar for the other definitions when varying the time window. The lowest value with the − 365/− 7 time window resulted in the highest prevalence (21.4%). Importantly, applying the guidelines with all criteria resulted in major differences in prevalence ranging from 5.9 to 24.0%.

Conclusions

AKI prevalence varies with the use of different baseline definitions in ED patients. Clinicians, as well as researchers and developers of automatic diagnostic tools should take these considerations into account when aiming to diagnose AKI in clinical and research settings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12882-021-02581-x.

Risk Stratifying Indeterminate Thyroid Nodules With Machine Learning

BACKGROUND

Up to 30% of thyroid nodules are classified as indeterminate after fine needle aspiration biopsy. These indeterminate thyroid nodules (ITNs) require surgical pathology for definitive diagnosis. Molecular testing provides additional pre-operative cancer risk stratification but adds expense and invasive testing. The purpose of this study is to utilize a machine learning (ML) algorithm to predict malignancy of ITNs using data available from less invasive tests.

MATERIALS AND METHODS

We conducted a retrospective study using medical records from one academic and one community center. Thyroid nodules with an indeterminate diagnosis on fine needle aspiration biopsy and completed diagnostic pathology were included. Linear, non-linear, and non-linear-ensemble ML methods were tested for accuracy when predicting malignancy using 10-fold cross-validation. Classifiers were evaluated using area under the receiver operating characteristics curve (AUROC).

RESULTS

A total of 355 nodules met inclusion criteria. Of these, 171 (48.2%) were diagnosed with cancer. A Random Forest classifier performed the best, producing an accuracy of 79.1%, a sensitivity of 75.5%, specificity of 82.4%, positive predicative value of 80.3%, negative predictive value of 79.0%, and an AUROC of 0.859.

CONCLUSIONS

ML methods accurately risk stratify ITNs using data gathered from existing, non-invasive, and inexpensive diagnostic tests. Applying an ML model with existing data can become a cost-effective alternative to molecular testing. Future studies will prospectively evaluate the performance of this ML approach when combined with expert judgment.

Cognitive biases, environmental, patient and personal factors associated with critical care decision making: A scoping review

PURPOSE

Cognitive biases and factors affecting decision making in critical care can potentially lead to life-threatening errors. We aimed to examine the existing evidence on the influence of cognitive biases and factors on decision making in critical care.

MATERIALS AND METHODS

We conducted a scoping review by searching MEDLINE for articles from 2004 to November 2020. We included studies conducted in physicians that described cognitive biases or factors associated with decision making. During the study process we decided on the method to summarize the evidence, and based on the obtained studies a descriptive summary of findings was the best fit.

RESULTS

Thirty heterogenous studies were included. Four main biases or factors were observed, e.g. cognitive biases, personal factors, environmental factors, and patient factors. Six (20%) studies reported biases associated with decision making comprising omission-, status quo-, implicit-, explicit-, outcome-, and overconfidence bias. Nineteen (63%) studies described personal factors, twenty-two (73%) studies described environmental factors, and sixteen (53%) studies described patient factors.

CONCLUSIONS

The current evidence on cognitive biases and factors is heterogenous, but shows they influence clinical decision. Future studies should investigate the prevalence of cognitive biases and factors in clinical practice and their impact on clinical outcomes.

Predictive modeling for peri-implantitis by using machine learning techniques

The purpose of this retrospective cohort study was to create a model for predicting the onset of peri-implantitis by using machine learning methods and to clarify interactions between risk indicators. This study evaluated 254 implants, 127 with and 127 without peri-implantitis, from among 1408 implants with at least 4 years in function. Demographic data and parameters known to be risk factors for the development of peri-implantitis were analyzed with three models: logistic regression, support vector machines, and random forests (RF). As the results, RF had the highest performance in predicting the onset of peri-implantitis (AUC: 0.71, accuracy: 0.70, precision: 0.72, recall: 0.66, and f1-score: 0.69). The factor that had the most influence on prediction was implant functional time, followed by oral hygiene. In addition, PCR of more than 50% to 60%, smoking more than 3 cigarettes/day, KMW less than 2 mm, and the presence of less than two occlusal supports tended to be associated with an increased risk of peri-implantitis. Moreover, these risk indicators were not independent and had complex effects on each other. The results of this study suggest that peri-implantitis onset was predicted in 70% of cases, by RF which allows consideration of nonlinear relational data with complex interactions.

Routine Health Information Systems in the European Context: A Systematic Review of Systematic Reviews

(1) Background: The aim of this study is to provide a better understanding of the requirements to improve routine health information systems (RHISs) for the management of health systems, including the identification of best practices, opportunities, and challenges in the 53 countries and territories of the WHO European region. (2) Methods: We conducted an overview of systematics reviews and searched the literature in the databases MEDLINE/PubMed, Cochrane, EMBASE, and Web of Science electronic databases. After a meticulous screening, we identified 20 that met the inclusion criteria, and RHIS evaluation results were presented according to the Performance of Routine Information System Management (PRISM) framework. (3) Results: The reviews were published between 2007 and 2020, focusing on the use of different systems or technologies and aimed to analyze interventions on professionals, centers, or patients' outcomes. All reviews examined showed variability in results in accordance with the variability of interventions and target populations. We have found different areas for improvement for RHISs according to the three determinants of the PRISM framework that influence the configuration of RHISs: technical, organizational, or behavioral elements. (4) Conclusions: RHIS interventions in the European region are promising. However, new global and international strategies and the development of tools and mechanisms should be promoted to highly integrate platforms among European countries.

Three ways of knowing: the integration of clinical expertise, evidence-based medicine, and artificial intelligence in assisted reproductive technologies

Decision-making in fertility care is on the cusp of a significant frameshift. Online tools to integrate artificial intelligence into the decision-making process across all aspects of ART are rapidly emerging. These tools have the potential to improve outcomes and transition decision-making from one based on traditional provider centric assessments toward a hybrid triad of expertise, evidence, and algorithmic data analytics using AI. We can look forward to a time when AI will be the third part of a provider's tool box to complement expertise and medical literature to enable ever more accurate predictions and outcomes in ART. In their fully integrated format, these tools will be part of a digital fertility ecosystem of analytics embedded within an EMR. To date, the impact of AI on ART outcomes is inconclusive. No prospective studies have shown clear cut benefit or cost reductions over current practices, but we are very early in the process of developing and evaluating these tools. We owe it to ourselves to begin to examine these AI-driven analytics and develop a very clear idea about where we can and should go before we roll these tools into clinical care. Thoughtful scrutiny is essential lest we find ourselves in a position of trying to modulate and modify after entry of these tools into our clinics and patient care. The purpose of this commentary is to highlight the evolution and impact AI has had in other fields relevant to the fertility sector and describe a vision for applications within ART that could improve outcomes, reduce costs, and positively impact clinical care.

Real-Time Clinical Decision Support Based on Recurrent Neural Networks for In-Hospital Acute Kidney Injury: External Validation and Model Interpretation

Background

Acute kidney injury (AKI) is commonly encountered in clinical practice and is associated with poor patient outcomes and increased health care costs. Despite it posing significant challenges for clinicians, effective measures for AKI prediction and prevention are lacking. Previously published AKI prediction models mostly have a simple design without external validation. Furthermore, little is known about the process of linking model output and clinical decisions due to the black-box nature of neural network models.

Objective

We aimed to present an externally validated recurrent neural network (RNN)–based continuous prediction model for in-hospital AKI and show applicable model interpretations in relation to clinical decision support.

Methods

Study populations were all patients aged 18 years or older who were hospitalized for more than 48 hours between 2013 and 2017 in 2 tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographic data, laboratory values, vital signs, and clinical conditions of patients were obtained from electronic health records of each hospital. We developed 2-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicted the future trajectory of creatinine values up to 72 hours. The performance of each developed model was evaluated using the internal and external validation data sets. For the explainability of our models, different model-agnostic interpretation methods were used, including Shapley Additive Explanations, partial dependence plots, individual conditional expectation, and accumulated local effects plots.

Results

We included 69,081 patients in the training, 7675 in the internal validation, and 72,352 in the external validation cohorts for model development after excluding cases with missing data and those with an estimated glomerular filtration rate less than 15 mL/min/1.73 m2 or end-stage kidney disease. Model 1 predicted any AKI development with an area under the receiver operating characteristic curve (AUC) of 0.88 (internal validation) and 0.84 (external validation), and stage 2 or higher AKI development with an AUC of 0.93 (internal validation) and 0.90 (external validation). Model 2 predicted the future creatinine values within 3 days with mean-squared errors of 0.04-0.09 for patients with higher risks of AKI and 0.03-0.08 for those with lower risks. Based on the developed models, we showed AKI probability according to feature values in total patients and each individual with partial dependence, accumulated local effects, and individual conditional expectation plots. We also estimated the effects of feature modifications such as nephrotoxic drug discontinuation on future creatinine levels.

Conclusions

We developed and externally validated a continuous AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts; thus, we suggest approaches to support clinical decisions based on prediction models for in-hospital AKI.

Risk of acute deterioration and care complexity individual factors associated with health outcomes in hospitalised patients with COVID-19: a multicentre cohort study

BACKGROUND

Evidence about the impact of systematic nursing surveillance on risk of acute deterioration of patients with COVID-19 and the effects of care complexity factors on inpatient outcomes is scarce. The aim of this study was to determine the association between acute deterioration risk, care complexity factors and unfavourable outcomes in hospitalised patients with COVID-19.

METHODS

A multicentre cohort study was conducted from 1 to 31 March 2020 at seven hospitals in Catalonia. All adult patients with COVID-19 admitted to hospitals and with a complete minimum data set were recruited retrospectively. Patients were classified based on the presence or absence of a composite unfavourable outcome (in-hospital mortality and adverse events). The main measures included risk of acute deterioration (as measured using the VIDA early warning system) and care complexity factors. All data were obtained blinded from electronic health records. Multivariate logistic analysis was performed to identify the VIDA score and complexity factors associated with unfavourable outcomes.

RESULTS

Out of a total of 1176 patients with COVID-19, 506 (43%) experienced an unfavourable outcome during hospitalisation. The frequency of unfavourable outcomes rose with increasing risk of acute deterioration as measured by the VIDA score. Risk factors independently associated with unfavourable outcomes were chronic underlying disease (OR: 1.90, 95% CI 1.32 to 2.72; p<0.001), mental status impairment (OR: 2.31, 95% CI 1.45 to 23.66; p<0.001), length of hospital stay (OR: 1.16, 95% CI 1.11 to 1.21; p<0.001) and high risk of acute deterioration (OR: 4.32, 95% CI 2.83 to 6.60; p<0.001). High-tech hospital admission was a protective factor against unfavourable outcomes (OR: 0.57, 95% CI 0.36 to 0.89; p=0.01).

CONCLUSION

The systematic nursing surveillance of the status and evolution of COVID-19 inpatients, including the careful monitoring of acute deterioration risk and care complexity factors, may help reduce deleterious health outcomes in COVID-19 inpatients.

Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective

Sepsis is a major cause of death worldwide. Over the past years, prediction of clinically relevant events through machine learning models has gained particular attention. In the present perspective, we provide a brief, clinician-oriented vision on the following relevant aspects concerning the use of machine learning predictive models for the early detection of sepsis in the daily practice: (i) the controversy of sepsis definition and its influence on the development of prediction models; (ii) the choice and availability of input features; (iii) the measure of the model performance, the output, and their usefulness in the clinical practice. The increasing involvement of artificial intelligence and machine learning in health care cannot be disregarded, despite important pitfalls that should be always carefully taken into consideration. In the long run, a rigorous multidisciplinary approach to enrich our understanding in the application of machine learning techniques for the early recognition of sepsis may show potential to augment medical decision-making when facing this heterogeneous and complex syndrome.

This is your toolkit in hemodynamic monitoring

PURPOSE OF REVIEW

To appraise the basic and more advanced methods available for hemodynamic monitoring, and describe the definitions and criteria for the use of hemodynamic variables.

RECENT FINDINGS

The hemodynamic assessment in critically ill patients suspected of circulatory shock follows a step-by-step algorithm to help determine diagnosis and prognosis. Determination of accurate diagnosis and prognosis in turn is crucial for clinical decision-making. Basic monitoring involving clinical examination in combination with hemodynamic variables obtained with an arterial catheter and a central venous catheter may be sufficient for the majority of patients with circulatory shock. In case of uncertainty of the underlying cause or to guide treatment in severe shock may require additional advanced hemodynamic technologies, and each is utilized for different indications and has specific limitations. Future developments include refining the clinical examination and performing studies that demonstrate better patient outcomes by targeting hemodynamic variables using advanced hemodynamic monitoring.

SUMMARY

Determination of accurate diagnosis and prognosis for patients suspected of circulatory shock is essential for optimal decision-making. Numerous techniques are available, and each has its specific indications and value.