A Multimodal Ensemble Driven by Multiobjective Optimisation to Predict Overall Survival in Non-Small-Cell Lung Cancer

Hierarchical embedding attention for overall survival prediction in lung cancer from unstructured EHRs

The automated processing of Electronic Health Records (EHRs) poses a significant challenge due to their unstructured nature, rich in valuable, yet disorganized information. Natural Language Processing (NLP), particularly Named Entity Recognition (NER), has been instrumental in extracting structured information from EHR data. However, existing literature primarly focuses on extracting handcrafted clinical features through NLP and NER methods without delving into their learned representations. In this work, we explore the untapped potential of these representations by considering their contextual richness and entity-specific information. Our proposed methodology extracts representations generated by a transformer-based NER model on EHRs data, combines them using a hierarchical attention mechanism, and employs the obtained enriched representation as input for a clinical prediction model. Specifically, this study addresses Overall Survival (OS) in Non-Small Cell Lung Cancer (NSCLC) using unstructured EHRs data collected from an Italian clinical centre encompassing 838 records from 231 lung cancer patients. Whilst our study is applied on EHRs written in Italian, it serves as use case to prove the effectiveness of extracting and employing high level textual representations that capture relevant information as named entities. Our methodology is interpretable because the hierarchical attention mechanism highlights the information in EHRs that the model considers the most crucial during the decision-making process. We validated this interpretability by measuring the agreement of domain experts on the importance assigned by the hierarchical attention mechanism to EHRs information through a questionnaire. Results demonstrate the effectiveness of our method, showcasing statistically significant improvements over traditional manually extracted clinical features.

Class balancing diversity multimodal ensemble for Alzheimer's disease diagnosis and early detection

Alzheimer's disease (AD) poses significant global health challenges due to its increasing prevalence and associated societal costs. Early detection and diagnosis of AD are critical for delaying progression and improving patient outcomes. Traditional diagnostic methods and single-modality data often fall short in identifying early-stage AD and distinguishing it from Mild Cognitive Impairment (MCI). This study addresses these challenges by introducing a novel approach: multImodal enseMble via class BALancing diversity for iMbalancEd Data (IMBALMED). IMBALMED integrates multimodal data from the Alzheimer's Disease Neuroimaging Initiative database, including clinical assessments, neuroimaging phenotypes, biospecimen, and subject characteristics data. It employs a new ensemble of model classifiers, designed specifically for this framework, which combines eight distinct families of learning paradigms trained with diverse class balancing techniques to overcome class imbalance and enhance model accuracy. We evaluate IMBALMED on two diagnostic tasks (binary and ternary classification) and four binary early detection tasks (at 12, 24, 36, and 48 months), comparing its performance with state-of-the-art algorithms and an unbalanced dataset method. To further validate the proposed model and ensure genuine generalization to real-world scenarios, we conducted an external validation experiment using data from the most recent phase of the ADNI dataset. IMBALMED demonstrates superior diagnostic accuracy and predictive performance in both binary and ternary classification tasks, significantly improving early detection of MCI at a 48-month time point and showing excellent generalizability in the 12-month task during external validation. The method shows improved classification performance and robustness, offering a promising solution for early detection and management of AD.

A graph neural network-based model with out-of-distribution robustness for enhancing antiretroviral therapy outcome prediction for HIV-1

Predicting the outcome of antiretroviral therapies (ART) for HIV-1 is a pressing clinical challenge, especially when the ART includes drugs with limited effectiveness data. This scarcity of data can arise either due to the introduction of a new drug to the market or due to limited use in clinical settings, resulting in clinical dataset with highly unbalanced therapy representation. To tackle this issue, we introduce a novel joint fusion model, which combines features from a Fully Connected (FC) Neural Network and a Graph Neural Network (GNN) in a multi-modality fashion. Our model uses both tabular data about genetic sequences and a knowledge base derived from Stanford drug-resistance mutation tables, which serve as benchmark references for deducing in-vivo treatment efficacy based on the viral genetic sequence. By leveraging this knowledge base structured as a graph, the GNN component enables our model to adapt to imbalanced data distributions and account for Out-of-Distribution (OoD) drugs. We evaluated these models' robustness against OoD drugs in the test set. Our comprehensive analysis demonstrates that the proposed model consistently outperforms the FC model. These results underscore the advantage of integrating Stanford scores in the model, thereby enhancing its generalizability and robustness, but also extending its utility in contributing in more informed clinical decisions with limited data availability. The source code is available at https://github.com/federicosiciliano/graph-ood-hiv.

An optimized ensemble search approach for classification of higher-level gait disorder using brain magnetic resonance images

Higher-Level Gait Disorder (HLGD) is a type of gait disorder estimated to affect up to 6% of the older population. By definition, its symptoms originate from the higher-level nervous system, yet its association with brain morphology remains unclear. This study hypothesizes that there are patterns in brain morphology linked to HLGD. For the first time in the literature, this work investigates whether deep learning, in the form of convolutional neural networks, can capture patterns in magnetic resonance images to identify individuals affected by HLGD. To handle this new classification task, we propose setting up an ensemble of models. This leverages the benefits of combining classifiers instead of determining which network is the most suitable, developing a new architecture, or customizing an existing one. We introduce a computationally cost-effective search algorithm to find the optimal ensemble by leveraging a cost function of both traditional performance scores and the diversity among the models. Using a unique dataset from a large population-based cohort (VESPR), the ensemble identified by our algorithm demonstrated superior performance compared to single networks, other ensemble fusion techniques, and the best linear radiological measure. This emphasizes the importance of implementing diversity into the cost function. Furthermore, the results indicate significant morphological differences in brain structure between HLGD-affected individuals and controls, motivating research about which areas the networks base their classifications on, to get a better understanding of the pathophysiology of HLGD.

Machine learning predicts pulmonary Long Covid sequelae using clinical data

Long COVID is a multi-systemic disease characterized by the persistence or occurrence of many symptoms that in many cases affect the pulmonary system. These, in turn, may deteriorate the patient's quality of life making it easier to develop severe complications. Being able to predict this syndrome is therefore important as this enables early treatment. In this work, we investigated three machine learning approaches that use clinical data collected at the time of hospitalization to this goal. The first works with all the descriptors feeding a traditional shallow learner, the second exploits the benefits of an ensemble of classifiers, and the third is driven by the intrinsic multimodality of the data so that different models learn complementary information. The experiments on a new cohort of data from 152 patients show that it is possible to predict pulmonary Long Covid sequelae with an accuracy of up to 94 % . As a further contribution, this work also publicly discloses the related data repository to foster research in this field.

Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications

With the increasing availability of diverse healthcare data sources, such as medical images and electronic health records, there is a growing need to effectively integrate and fuse this multimodal data for comprehensive analysis and decision-making. However, despite its potential, multimodal data fusion in healthcare remains limited. This review paper provides an overview of existing literature on multimodal data fusion in healthcare, covering 69 relevant works published between 2018 and 2024. It focuses on methodologies that integrate different data types to enhance medical analysis, including techniques for integrating medical images with structured and unstructured data, combining multiple image modalities, and other features. Additionally, the paper reviews various approaches to multimodal data fusion, such as early, intermediate, and late fusion methods, and examines the challenges and limitations associated with these techniques. The potential benefits and applications of multimodal data fusion in various diseases are highlighted, illustrating specific strategies employed in healthcare artificial intelligence (AI) model development. This research synthesizes existing information to facilitate progress in using multimodal data for improved medical diagnosis and treatment planning.

Effects of Ward Noise Reduction Administration on Mental Health and Lung Function of Patients with Lung Cancer

OBJECTIVE

This study aimed to analyze the effects of ward noise reduction administration on the lung function and mental health of patients with lung cancer.

METHODS

A total of 195 patients who underwent lung cancer surgery in PLA Northern Theater Command Air Force Hospital from November 2020 to November 2022 were selected to be divided into a control group (routine nursing) and an observation group (routine nursing and ward noise reduction administration) in accordance with the medical record system. The general demographic data, noise level, lung function (forced expiratory volume in 1 s (FEV1), forced vital capacity (FVC) and FEV1/FVC)), and complications of patients in the two groups were collected. Propensity score matching (PSM) was used to balance the baseline data of the two groups, and t-test and chi-square test were used to analyze the data.

RESULTS

After PSM was conducted, 50 patients were enrolled in each group. No statistical difference was found in the baseline data, preadministration noise levels, and FEV1, FVC, FEV1/FVC, state-anxiety inventory (S-AI), and trait anxiety inventory scale (T-AI) scores between the two groups (P > 0.05). After ward noise reduction was administered, the noise level in the observation group was lower than that in the control group (P < 0.05). The FEV1, FVC, and FEV1/FVC scores of the observation group were higher than those of the control group but were not statistically significant (P > 0.05). The S-AI and T-AI scores of the observation group were lower than those of the control group (P < 0.05). No differences were found in the complications between the two groups (P > 0.05).

CONCLUSION

Administering ward noise reduction in patients with lung cancer can alleviate their negative emotions, thus worthy of clinical adoption.

Multi-objective optimization determines when, which and how to fuse deep networks: An application to predict COVID-19 outcomes

The COVID-19 pandemic has caused millions of cases and deaths and the AI-related scientific community, after being involved with detecting COVID-19 signs in medical images, has been now directing the efforts towards the development of methods that can predict the progression of the disease. This task is multimodal by its very nature and, recently, baseline results achieved on the publicly available AIforCOVID dataset have shown that chest X-ray scans and clinical information are useful to identify patients at risk of severe outcomes. While deep learning has shown superior performance in several medical fields, in most of the cases it considers unimodal data only. In this respect, when, which and how to fuse the different modalities is an open challenge in multimodal deep learning. To cope with these three questions here we present a novel approach optimizing the setup of a multimodal end-to-end model. It exploits Pareto multi-objective optimization working with a performance metric and the diversity score of multiple candidate unimodal neural networks to be fused. We test our method on the AIforCOVID dataset, attaining state-of-the-art results, not only outperforming the baseline performance but also being robust to external validation. Moreover, exploiting XAI algorithms we figure out a hierarchy among the modalities and we extract the features' intra-modality importance, enriching the trust on the predictions made by the model.