Probabilistic Machine Learning for Healthcare

AI-driven multi-agent reinforcement learning framework for real-time monitoring of physiological signals in stress and depression contexts

PURPOSE

Effective patient monitoring is crucial for timely healthcare interventions and improved outcomes, especially in managing conditions influenced by stress and depression, which can manifest through physiological changes. Traditional monitoring systems often struggle with the complexity and dynamic nature of such conditions, leading to delays in identifying critical scenarios. This study proposes a novel multi-agent deep reinforcement learning (DRL) framework to address these challenges by monitoring vital signs and providing real-time decision-making capabilities.

METHODS

Our framework deploys multiple learning agents, each dedicated to monitoring specific physiological features such as heart rate, respiration, and temperature. These agents interact with a generic healthcare monitoring environment, learn patients' behavior patterns, and estimate the level of emergency to alert Medical Emergency Teams (METs) accordingly. The study evaluates the proposed system using two real-world datasets-PPG-DaLiA and WESAD-designed to capture physiological and stress-related data. The performance is compared with baseline models, including Q-Learning, PPO, Actor-Critic, Double DQN, and DDPG, as well as existing monitoring frameworks like WISEML and CA-MAQL. Hyperparameter optimization is also performed to fine-tune learning rates and discount factors.

RESULTS

Experimental results demonstrate that the proposed multi-agent DRL framework outperforms baseline models in accurately monitoring patients' vital signs under stress and varying conditions. The optimized agents adapt effectively to dynamic environments, ensuring timely detection of critical health deviations. Comparative evaluations reveal superior performance in metrics related to decision-making accuracy and response efficiency, highlighting the robustness of the framework.

CONCLUSIONS

The proposed AI-driven monitoring system offers significant advancements over traditional methods by handling complex and uncertain environments, adapting to varying patient conditions influenced by stress and depression, and making autonomous, real-time decisions. While the framework demonstrates high accuracy and adaptability, challenges related to data scale and future vital sign prediction remain. Future research will focus on extending predictive capabilities to further enhance proactive healthcare interventions.

A novel predictive method for URS and laser lithotripsy using machine learning and explainable AI: results from the FLEXOR international database

PURPOSE

We developed Machine learning (ML) algorithms to predict ureteroscopy (URS) outcomes, offering insights into diagnosis and treatment planning, personalised care and improved clinical decision-making.

METHODS

FLEXOR is a large international multicentric database including 6669 patients treated with URS for urolithiasis from 2015 to 2023. Preoperative and postoperative(PO) correlations were investigated through 15 ML-trained algorithms. Outcomes included stone free status (SFS, at 3-month imaging follow up), intraoperative (PCS bleeding, ureteric/PCS injury, need for postoperative drainage) and PO complications (fever, sepsis, need for reintervention). ML was applied for the prediction, correlation and logistic regression analysis. Explainable AI emphasizes key features and their contributions to the output.

RESULTS

Extra Tree Classifier achieved the best accuracy (81%) in predicting SFS. PCS bleed was negatively linked with 'positive urine culture'(-0.08), 'tamsulosin'(-0.08), 'stone location'(-0.10), 'fibre optic scope'(-0.19), 'Moses Fibre'(-0.09), and 'TFL'(-0.09), and positively with 'elevated creatine'(0.25), 'fever'(0.11), and 'stone diameter'(0.21). 'PCS injury' and 'ureteric injury' both showed moderate correlation with 'elevated creatinine'(0.11), 'fever'(0.10), and 'lower pole stone'(0.09). 'Tamsulosin'(0.23) use, presence of 'multiple'(0.25) or 'lower pole'(0.25) stones, 'reusable scope'(0.17) and 'Moses Fibre'(0.2546) increased the risk for PO stent, while 'digital scope'(-0.13) or 'TFL'(-0.29) reduced it. 'Preoperative fever'(0.10), 'positive urine culture'(0.16), and 'stone diameter'(0.10) may play a role in 'PO fever' and 'sepsis'. SFS was mainly influenced by 'age'(0.12), 'preoperative fever'(0.09), 'multiple stones'(0.15), 'stone diameter'(0.17), 'Moses Fibre"(0.15) and 'TFL'(-0.28).

CONCLUSION

ML is valuable tool for accurately predicting outcomes by analysing pre-existing datasets. Our model demonstrated strong performance in outcomes and risks prediction, laying the groundwork for development of accessible predictive models.

Predictive analytics for early detection of hospital-acquired complications: An artificial intelligence approach

BACKGROUND

Hospital-acquired complications (HACs) have an adverse impact on patient recovery by impeding their path to full recovery and increasing healthcare costs.

OBJECTIVE

The aim of this study was to create a HAC risk prediction machine learning (ML) framework using hospital administrative data collections within North Metropolitan Health Service (NMHS), Western Australia.

METHOD

A retrospective cohort study was performed among 64,315 patients between July 2020 to June 2022 to develop an automated ML framework by inputting HAC and the healthcare site to obtain site-specific predictive algorithms for patients admitted to the hospital in NMHS. Univariate analysis was used for initial feature screening for 270 variables. Of these, 77 variables had significant relationship with any HAC. After excluding non-contemporaneous data, 37 variables were included in developing the ML framework based on logistic regression (LR), decision tree (DT) and random forest (RF) models to predict occurrence of four specific HACs: delirium, aspiration pneumonia, pneumonia and urinary tract infection.

RESULTS

All models exhibited similar performance with area under the curve scores around 0.90 for both training and testing datasets. For sensitivity, DT and RF exceeded LR performance while on average, false positives were lowest for LR-based models. Patient's length of stay, Charlson Index, operation length and intensive care unit stay were common predictors.

CONCLUSION

Integrating ML-based risk detection systems into clinical workflows can potentially enhance patient safety and optimise resource allocation. LR-based models exhibited best performance.

IMPLICATIONS

We have successfully developed a "real-time" risk prediction model, where patient risk scores are calculated and reviewed daily.

AI as an intervention: improving clinical outcomes relies on a causal approach to AI development and validation

The primary practice of healthcare artificial intelligence (AI) starts with model development, often using state-of-the-art AI, retrospectively evaluated using metrics lifted from the AI literature like AUROC and DICE score. However, good performance on these metrics may not translate to improved clinical outcomes. Instead, we argue for a better development pipeline constructed by working backward from the end goal of positively impacting clinically relevant outcomes using AI, leading to considerations of causality in model development and validation, and subsequently a better development pipeline. Healthcare AI should be "actionable," and the change in actions induced by AI should improve outcomes. Quantifying the effect of changes in actions on outcomes is causal inference. The development, evaluation, and validation of healthcare AI should therefore account for the causal effect of intervening with the AI on clinically relevant outcomes. Using a causal lens, we make recommendations for key stakeholders at various stages of the healthcare AI pipeline. Our recommendations aim to increase the positive impact of AI on clinical outcomes.

Individualized post-crisis monitoring of psychiatric patients via Hidden Markov models

Introduction

Individuals in the midst of a mental health crisis frequently exhibit instability and face an elevated risk of recurring crises in the subsequent weeks, which underscores the importance of timely intervention in mental healthcare. This work presents a data-driven method to infer the mental state of a patient during the weeks following a mental health crisis by leveraging their historical data. Additionally, we propose a policy that determines the necessary duration for closely monitoring a patient after a mental health crisis before considering them stable.

Methods

We model the patient's mental state as a Hidden Markov Process, partially observed through mental health crisis events. We introduce a closed-form solution that leverages the model parameters to optimally estimate the risk of future mental health crises. Our policy determines a patient should be closely monitored when their estimated risk of crisis exceeds a predefined threshold. The method's performance is evaluated using both simulated data and a real-world dataset comprising 162 anonymized psychiatric patients.

Results

In the simulations, 96.2 % of the patients identified by the policy were in an unstable state, achieving a F1 score of 0.74 . In the real-world dataset, the policy yielded an F1 score of 0.79, with a sensitivity of 79.8 % and specificity of 88.9 % . Under this policy, 67.3 % of the patients should undergo close monitoring for one week, 21.6 % during 2 weeks or more, while 11.1 % do not need close monitoring.

Discussion

The simulation results provide compelling evidence that the method is effective under the specified assumptions. When applied to actual psychiatric patients, the proposed policy showed significant potential for providing an individualized assessment of the required duration for close and automatic monitoring after a mental health crisis to reduce the relapse risks.

Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunities

In recent years, machine learning (ML) and deep learning (DL) have been the leading approaches to solving various challenges, such as disease predictions, drug discovery, medical image analysis, etc., in intelligent healthcare applications. Further, given the current progress in the fields of ML and DL, there exists the promising potential for both to provide support in the realm of healthcare. This study offered an exhaustive survey on ML and DL for the healthcare system, concentrating on vital state of the art features, integration benefits, applications, prospects and future guidelines. To conduct the research, we found the most prominent journal and conference databases using distinct keywords to discover scholarly consequences. First, we furnished the most current along with cutting-edge progress in ML-DL-based analysis in smart healthcare in a compendious manner. Next, we integrated the advancement of various services for ML and DL, including ML-healthcare, DL-healthcare, and ML-DL-healthcare. We then offered ML and DL-based applications in the healthcare industry. Eventually, we emphasized the research disputes and recommendations for further studies based on our observations.

Scalable and interpretable alternative to chart review for phenotype evaluation using standardized structured data from electronic health records

OBJECTIVES

Chart review as the current gold standard for phenotype evaluation cannot support observational research on electronic health records and claims data sources at scale. We aimed to evaluate the ability of structured data to support efficient and interpretable phenotype evaluation as an alternative to chart review.

MATERIALS AND METHODS

We developed Knowledge-Enhanced Electronic Profile Review (KEEPER) as a phenotype evaluation tool that extracts patient's structured data elements relevant to a phenotype and presents them in a standardized fashion following clinical reasoning principles. We evaluated its performance (interrater agreement, intermethod agreement, accuracy, and review time) compared to manual chart review for 4 conditions using randomized 2-period, 2-sequence crossover design.

RESULTS

Case ascertainment with KEEPER was twice as fast compared to manual chart review. 88.1% of the patients were classified concordantly using charts and KEEPER, but agreement varied depending on the condition. Missing data and differences in interpretation accounted for most of the discrepancies. Pairs of clinicians agreed in case ascertainment in 91.2% of the cases when using KEEPER compared to 76.3% when using charts. Patient classification aligned with the gold standard in 88.1% and 86.9% of the cases respectively.

CONCLUSION

Structured data can be used for efficient and interpretable phenotype evaluation if they are limited to relevant subset and organized according to the clinical reasoning principles. A system that implements these principles can achieve noninferior performance compared to chart review at a fraction of time.

Predictive Models for Health Deterioration: Understanding Disease Pathways for Personalized Medicine

Artificial intelligence (AI) and machine learning (ML) methods are currently widely employed in medicine and healthcare. A PubMed search returns more than 100,000 articles on these topics published between 2018 and 2022 alone. Notwithstanding several recent reviews in various subfields of AI and ML in medicine, we have yet to see a comprehensive review around the methods' use in longitudinal analysis and prediction of an individual patient's health status within a personalized disease pathway. This review seeks to fill that gap. After an overview of the AI and ML methods employed in this field and of specific medical applications of models of this type, the review discusses the strengths and limitations of current studies and looks ahead to future strands of research in this field. We aim to enable interested readers to gain a detailed impression of the research currently available and accordingly plan future work around predictive models for deterioration in health status.

Diabetic Foot Ulcer Identification: A Review

Diabetes is a chronic condition caused by an uncontrolled blood sugar levels in the human body. Its early diagnosis may prevent severe complications such as diabetic foot ulcers (DFUs). A DFU is a critical condition that can lead to the amputation of a diabetic patient's lower limb. The diagnosis of DFU is very complicated for the medical professional as it often goes through several costly and time-consuming clinical procedures. In the age of data deluge, the application of deep learning, machine learning, and computer vision techniques have provided various solutions for assisting clinicians in making more reliable and faster diagnostic decisions. Therefore, the automatic identification of DFU has recently received more attention from the research community. The wound characteristics and visual perceptions with respect to computer vision and deep learning, especially convolutional neural network (CNN) approaches, have provided potential solutions for DFU diagnosis. These approaches have the potential to be quite helpful in current medical practices. Therefore, a detailed comprehensive study of such existing approaches was required. The article aimed to provide researchers with a detailed current status of automatic DFU identification tasks. Multiple observations have been made from existing works, such as the use of traditional ML and advanced DL techniques being necessary to help clinicians make faster and more reliable diagnostic decisions. In traditional ML approaches, image features provide signification information about DFU wounds and help with accurate identification. However, advanced DL approaches have proven to be more promising than ML approaches. The CNN-based solutions proposed by various authors have dominated the problem domain. An interested researcher will successfully be able identify the overall idea in the DFU identification task, and this article will help them finalize the future research goal.

Interpretable clinical phenotypes among patients hospitalized with COVID-19 using cluster analysis

Background

Multiple clinical phenotypes have been proposed for coronavirus disease (COVID-19), but few have used multimodal data. Using clinical and imaging data, we aimed to identify distinct clinical phenotypes in patients admitted with COVID-19 and to assess their clinical outcomes. Our secondary objective was to demonstrate the clinical applicability of this method by developing an interpretable model for phenotype assignment.

Methods

We analyzed data from 547 patients hospitalized with COVID-19 at a Canadian academic hospital. We processed the data by applying a factor analysis of mixed data (FAMD) and compared four clustering algorithms: k-means, partitioning around medoids (PAM), and divisive and agglomerative hierarchical clustering. We used imaging data and 34 clinical variables collected within the first 24 h of admission to train our algorithm. We conducted a survival analysis to compare the clinical outcomes across phenotypes. With the data split into training and validation sets (75/25 ratio), we developed a decision-tree-based model to facilitate the interpretation and assignment of the observed phenotypes.

Results

Agglomerative hierarchical clustering was the most robust algorithm. We identified three clinical phenotypes: 79 patients (14%) in Cluster 1, 275 patients (50%) in Cluster 2, and 203 (37%) in Cluster 3. Cluster 2 and Cluster 3 were both characterized by a low-risk respiratory and inflammatory profile but differed in terms of demographics. Compared with Cluster 3, Cluster 2 comprised older patients with more comorbidities. Cluster 1 represented the group with the most severe clinical presentation, as inferred by the highest rate of hypoxemia and the highest radiological burden. Intensive care unit (ICU) admission and mechanical ventilation risks were the highest in Cluster 1. Using only two to four decision rules, the classification and regression tree (CART) phenotype assignment model achieved an AUC of 84% (81.5-86.5%, 95 CI) on the validation set.

Conclusions

We conducted a multidimensional phenotypic analysis of adult inpatients with COVID-19 and identified three distinct phenotypes associated with different clinical outcomes. We also demonstrated the clinical usability of this approach, as phenotypes can be accurately assigned using a simple decision tree. Further research is still needed to properly incorporate these phenotypes in the management of patients with COVID-19.

Quantifying deep neural network uncertainty for atrial fibrillation detection with limited labels

Atrial fibrillation (AF) is the most common arrhythmia found in the intensive care unit (ICU), and is associated with many adverse outcomes. Effective handling of AF and similar arrhythmias is a vital part of modern critical care, but obtaining knowledge about both disease burden and effective interventions often requires costly clinical trials. A wealth of continuous, high frequency physiological data such as the waveforms derived from electrocardiogram telemetry are promising sources for enriching clinical research. Automated detection using machine learning and in particular deep learning has been explored as a solution for processing these data. However, a lack of labels, increased presence of noise, and inability to assess the quality and trustworthiness of many machine learning model predictions pose challenges to interpretation. In this work, we propose an approach for training deep AF models on limited, noisy data and report uncertainty in their predictions. Using techniques from the fields of weakly supervised learning, we leverage a surrogate model trained on non-ICU data to create imperfect labels for a large ICU telemetry dataset. We combine these weak labels with techniques to estimate model uncertainty without the need for extensive human data annotation. AF detection models trained using this process demonstrated higher classification performance (0.64-0.67 F1 score) and improved calibration (0.05-0.07 expected calibration error).

Beyond "sex prediction": Estimating and interpreting multivariate sex differences and similarities in the brain

Previous studies have shown that machine-learning (ML) algorithms can "predict" sex based on brain anatomical/ functional features. The high classification accuracy achieved by ML algorithms is often interpreted as revealing large differences between the brains of males and females and as confirming the existence of "male/female brains". However, classification and estimation are different concepts, and using classification metrics as surrogate estimates of between-group differences may result in major statistical and interpretative distortions. The present study avoids these distortions and provides a novel and detailed assessment of multivariate sex differences in gray matter volume (GMVOL) that does not rely on classification metrics. Moreover, appropriate regression methods were used to identify the brain areas that contribute the most to these multivariate differences, and clustering techniques and analyses of similarities (ANOSIM) were employed to empirically assess whether they assemble into two sex-typical profiles. Results revealed that multivariate sex differences in GMVOL: (1) are "large" if not adjusted for total intracranial volume (TIV) variation, but "small" when controlling for this variable; (2) differ in size between individuals and also depends on the ML algorithm used for their calculation (3) do not stem from two sex-typical profiles, and so describing them in terms of "male/female brains" is misleading.