Nested cross-validation with ensemble feature selection and classification model for high-dimensional biological data

Radiomic analysis of cohort-specific diagnostic errors in reading dense mammograms using artificial intelligence

OBJECTIVES

This study aims to investigate radiologists' interpretation errors when reading dense screening mammograms using a radiomics-based artificial intelligence approach.

METHODS

Thirty-six radiologists from China and Australia read 60 dense mammograms. For each cohort, we identified normal areas that looked suspicious of cancer and the malignant areas containing cancers. Then radiomic features were extracted from these identified areas and random forest models were trained to recognize the areas that were most frequently linked to diagnostic errors within each cohort. The performance of the model and discriminatory power of significant radiomic features were assessed.

RESULTS

We found that in the Chinese cohort, the AUC values for predicting false positives were 0.864 (CC) and 0.829 (MLO), while in the Australian cohort, they were 0.652 (CC) and 0.747 (MLO). For false negatives, the AUC values in the Chinese cohort were 0.677 (CC) and 0.673 (MLO), and in the Australian cohort, they were 0.600 (CC) and 0.505 (MLO). In both cohorts, regions with higher Gabor and maximum response filter outputs were more prone to false positives, while areas with significant intensity changes and coarse textures were more likely to yield false negatives.

CONCLUSIONS

This cohort-based pipeline proves effective in identifying common errors for specific reader cohorts based on image-derived radiomic features.

ADVANCES IN KNOWLEDGE

This study demonstrates that radiomics-based AI can effectively identify and predict radiologists' interpretation errors in dense mammograms, with distinct radiomic features linked to false positives and false negatives in Chinese and Australian cohorts.

Chronic aquatic toxicity assessment of diverse chemicals on Daphnia magna using QSAR and chemical read-across

We have modeled here chronic Daphnia toxicity taking pNOEC (negative logarithm of no observed effect concentration in mM) and pEC50 (negative logarithm of half-maximal effective concentration in mM) as endpoints using QSAR and chemical read-across approaches. The QSAR models were developed by strictly obeying the OECD guidelines and were found to be reliable, predictive, accurate, and robust. From the selected features in the developed models, we have found that an increase in lipophilicity and saturation, the presence of electrophilic or electronegative or heavy atoms, the presence of sulphur, amine, and their related functionality, an increase in mean atomic polarizability, and higher number of (thio-) carbamates (aromatic) groups are responsible for chronic toxicity. Therefore, this information might be useful for the development of environmentally friendly and safer chemicals and data-gap filling as well as reducing the use of identified toxic chemicals which have chronic toxic effects on aquatic ecosystems. Approved classes of drugs from DrugBank databases and diverse groups of chemicals from the Chemical and Product Categories (CPDat) database were also assessed through the developed models.

Prediction of malaria positivity using patients' demographic and environmental features and clinical symptoms to complement parasitological confirmation before treatment

BACKGROUND

Current malaria diagnosis methods that rely on microscopy and Histidine Rich Protein-2 (HRP2)-based rapid diagnostic tests (RDT) have drawbacks that necessitate the development of improved and complementary malaria diagnostic methods to overcome some or all these limitations. Consequently, the addition of automated detection and classification of malaria using laboratory methods can provide patients with more accurate and faster diagnosis. Therefore, this study used a machine-learning model to predict Plasmodium falciparum (Pf) antigen positivity (presence of malaria) based on sociodemographic behaviour, environment, and clinical features.

METHOD

Data from 200 Nigerian patients were used to develop predictive models using nested cross-validation and sequential backward feature selection (SBFS), with 80% of the dataset randomly selected for training and optimisation and the remaining 20% for testing the models. Outcomes were classified as Pf-positive or Pf-negative, corresponding to the presence or absence of malaria, respectively.

RESULTS

Among the three machine learning models examined, the penalised logistic regression model had the best area under the receiver operating characteristic curve for the training set (AUC = 84%; 95% confidence interval [CI]: 75-93%) and test set (AUC = 83%; 95% CI: 63-100%). Increased odds of malaria were associated with higher body weight (adjusted odds ratio (AOR) = 4.50, 95% CI: 2.27 to 8.01, p < 0.0001). Even though the association between the odds of having malaria and body temperature was not significant, patients with high body temperature had higher odds of testing positive for the Pf antigen than those who did not have high body temperature (AOR = 1.40, 95% CI: 0.99 to 1.91, p = 0.068). In addition, patients who had bushes in their surroundings (AOR = 2.60, 95% CI: 1.30 to 4.66, p = 0.006) or experienced fever (AOR = 2.10, 95% CI: 0.88 to 4.24, p = 0.099), headache (AOR = 2.07; 95% CI: 0.95 to 3.95, p = 0.068), muscle pain (AOR = 1.49; 95% CI: 0.66 to 3.39, p = 0.333), and vomiting (AOR = 2.32; 95% CI: 0.85 to 6.82, p = 0.097) were more likely to experience malaria. In contrast, decreased odds of malaria were associated with age (AOR = 0.62, 95% CI: 0.41 to 0.90, p = 0.012) and BMI (AOR = 0.47, 95% CI: 0.26 to 0.80, p = 0.006).

CONCLUSION

Newly developed routinely collected baseline sociodemographic, environmental, and clinical features to predict Pf antigen positivity may be a valuable tool for clinical decision-making.

Personalized Risk Analysis to Improve the Psychological Resilience of Women Undergoing Treatment for Breast Cancer: Development of a Machine Learning-Driven Clinical Decision Support Tool

BACKGROUND

Health professionals are often faced with the need to identify women at risk of manifesting poor psychological resilience following the diagnosis and treatment of breast cancer. Machine learning algorithms are increasingly used to support clinical decision support (CDS) tools in helping health professionals identify women who are at risk of adverse well-being outcomes and plan customized psychological interventions for women at risk. Clinical flexibility, cross-validated performance accuracy, and model explainability permitting person-specific identification of risk factors are highly desirable features of such tools.

OBJECTIVE

This study aimed to develop and cross-validate machine learning models designed to identify breast cancer survivors at risk of poor overall mental health and global quality of life and identify potential targets of personalized psychological interventions according to an extensive set of clinical recommendations.

METHODS

A set of 12 alternative models was developed to improve the clinical flexibility of the CDS tool. All models were validated using longitudinal data from a prospective, multicenter clinical pilot at 5 major oncology centers in 4 countries (Italy, Finland, Israel, and Portugal; the Predicting Effective Adaptation to Breast Cancer to Help Women to BOUNCE Back [BOUNCE] project). A total of 706 patients with highly treatable breast cancer were enrolled shortly after diagnosis and before the onset of oncological treatments and were followed up for 18 months. An extensive set of demographic, lifestyle, clinical, psychological, and biological variables measured within 3 months after enrollment served as predictors. Rigorous feature selection isolated key psychological resilience outcomes that could be incorporated into future clinical practice.

RESULTS

Balanced random forest classifiers were successful at predicting well-being outcomes, with accuracies ranging between 78% and 82% (for 12-month end points after diagnosis) and between 74% and 83% (for 18-month end points after diagnosis). Explainability and interpretability analyses built on the best-performing models were used to identify potentially modifiable psychological and lifestyle characteristics that, if addressed systematically in the context of personalized psychological interventions, would be most likely to promote resilience for a given patient.

CONCLUSIONS

Our results highlight the clinical utility of the BOUNCE modeling approach by focusing on resilience predictors that can be readily available to practicing clinicians at major oncology centers. The BOUNCE CDS tool paves the way for personalized risk assessment methods to identify patients at high risk of adverse well-being outcomes and direct valuable resources toward those most in need of specialized psychological interventions.

Personalized screening and risk profiles for Mild Cognitive Impairment via a Machine Learning Framework: Implications for general practice

OBJECTIVES

Diagnosis of Mild Cognitive Impairment (MCI) requires lengthy diagnostic procedures, typically available at tertiary Health Care Centers (HCC). This prospective study evaluated a flexible Machine Learning (ML) framework toward identifying persons with MCI or dementia based on information that can be readily available in a primary HC setting.

METHODS

Demographic and clinical data, informant ratings of recent behavioral changes, self-reported anxiety and depression symptoms, subjective cognitive complaints, and Mini Mental State Examination (MMSE) scores were pooled from two aging cohorts from the island of Crete, Greece (N = 763 aged 60-93 years) comprising persons diagnosed with MCI (n = 277) or dementia (n = 153), and cognitively non-impaired persons (CNI, n = 333). A Balanced Random Forest Classifier was used for classification and variable importance-based feature selection in nested cross-validation schemes (CNI vs MCI, CNI vs Dementia, MCI vs Dementia). Global-level model-agnostic analyses identified predictors displaying nonlinear behavior. Local level agnostic analyses pinpointed key predictor variables for a given classification result after statistically controlling for all other predictors in the model.

RESULTS

Classification of MCI vs CNI was achieved with improved sensitivity (74 %) and comparable specificity (73 %) compared to MMSE alone (37.2 % and 94.3 %, respectively). Additional high-ranking features included age, education, behavioral changes, multicomorbidity and polypharmacy. Higher classification accuracy was achieved for MCI vs Dementia (sensitivity/specificity = 87 %) and CNI vs Dementia (sensitivity/specificity = 94 %) using the same set of variables. Model agnostic analyses revealed notable individual variability in the contribution of specific variables toward a given classification result.

CONCLUSIONS

Improved capacity to identify elderly with MCI can be achieved by combining demographic and medical information readily available at the PHC setting with MMSE scores, and informant ratings of behavioral changes. Explainability at the patient level may help clinicians identify specific predictor variables and patient scores to a given prediction outcome toward personalized risk assessment.

Hemodynamic MRI parameters to predict asymptomatic unilateral carotid artery stenosis with random forest machine learning

BACKGROUND

Internal carotid artery stenosis (ICAS) can cause stroke and cognitive decline. Associated hemodynamic impairments, which are most pronounced within individual watershed areas (iWSA) between vascular territories, can be assessed with hemodynamic-oxygenation-sensitive MRI and may help to detect severely affected patients. We aimed to identify the most sensitive parameters and volumes of interest (VOI) to predict high-grade ICAS with random forest machine learning. We hypothesized an increased predictive ability considering iWSAs and a decreased cognitive performance in correctly classified patients.

MATERIALS AND METHODS

Twenty-four patients with asymptomatic, unilateral, high-grade carotid artery stenosis and 24 age-matched healthy controls underwent MRI comprising pseudo-continuous arterial spin labeling (pCASL), breath-holding functional MRI (BH-fMRI), dynamic susceptibility contrast (DSC), T2 and T2* mapping, MPRAGE and FLAIR. Quantitative maps of eight perfusion, oxygenation and microvascular parameters were obtained. Mean values of respective parameters within and outside of iWSAs split into gray (GM) and white matter (WM) were calculated for both hemispheres and for interhemispheric differences resulting in 96 features. Random forest classifiers were trained on whole GM/WM VOIs, VOIs considering iWSAs and with additional feature selection, respectively.

RESULTS

The most sensitive features in decreasing order were time-to-peak (TTP), cerebral blood flow (CBF) and cerebral vascular reactivity (CVR), all of these inside of iWSAs. Applying iWSAs combined with feature selection yielded significantly higher receiver operating characteristics areas under the curve (AUC) than whole GM/WM VOIs (AUC: 0.84 vs. 0.90, p = 0.039). Correctly predicted patients presented with worse cognitive performances than frequently misclassified patients (Trail-making-test B: 152.5s vs. 94.4s, p = 0.034).

CONCLUSION

Random forest classifiers trained on multiparametric MRI data allow identification of the most relevant parameters and VOIs to predict ICAS, which may improve personalized treatments.

Machine learning for exploring neurophysiological functionality in multiple sclerosis based on trigeminal and hand blink reflexes

Brainstem dysfunctions are very common in Multiple Sclerosis (MS) and are a critical predictive factor for future disability. Brainstem functionality can be explored with blink reflexes, subcortical responses consisting in a blink following a peripheral stimulation. Some reflexes are already employed in clinical practice, such as Trigeminal Blink Reflex (TBR). Here we propose for the first time in MS the exploration of Hand Blink Reflex (HBR), which size is modulated by the proximity of the stimulated hand to the face, reflecting the extension of the peripersonal space. The aim of this work is to test whether Machine Learning (ML) techniques could be used in combination with neurophysiological measurements such as TBR and HBR to improve their clinical information and potentially favour the early detection of brainstem dysfunctionality. HBR and TBR were recorded from a group of People with MS (PwMS) with Relapsing-Remitting form and from a healthy control group. Two AdaBoost classifiers were trained with TBR and HBR features each, for a binary classification task between PwMS and Controls. Both classifiers were able to identify PwMS with an accuracy comparable and even higher than clinicians. Our results indicate that ML techniques could represent a tool for clinicians for investigating brainstem functionality in MS. Also, HBR could be promising when applied in clinical practice, providing additional information about the integrity of brainstem circuits potentially favouring early diagnosis.

Chronic Mild Traumatic Brain Injury: Aberrant Static and Dynamic Connectomic Features Identified Through Machine Learning Model Fusion

Traumatic Brain Injury (TBI) is a frequently occurring condition and approximately 90% of TBI cases are classified as mild (mTBI). However, conventional MRI has limited diagnostic and prognostic value, thus warranting the utilization of additional imaging modalities and analysis procedures. The functional connectomic approach using resting-state functional MRI (rs-fMRI) has shown great potential and promising diagnostic capabilities across multiple clinical scenarios, including mTBI. Additionally, there is increasing recognition of a fundamental role of brain dynamics in healthy and pathological cognition. Here, we undertake an in-depth investigation of mTBI-related connectomic disturbances and their emotional and cognitive correlates. We leveraged machine learning and graph theory to combine static and dynamic functional connectivity (FC) with regional entropy values, achieving classification accuracy up to 75% (77, 74 and 76% precision, sensitivity and specificity, respectively). As compared to healthy controls, the mTBI group displayed hypoconnectivity in the temporal poles, which correlated positively with semantic (r = 0.43, p < 0.008) and phonemic verbal fluency (r = 0.46, p < 0.004), while hypoconnectivity in the right dorsal posterior cingulate correlated positively with depression symptom severity (r = 0.54, p < 0.0006). These results highlight the importance of residual FC in these regions for preserved cognitive and emotional function in mTBI. Conversely, hyperconnectivity was observed in the right precentral and supramarginal gyri, which correlated negatively with semantic verbal fluency (r=-0.47, p < 0.003), indicating a potential ineffective compensatory mechanism. These novel results are promising toward understanding the pathophysiology of mTBI and explaining some of its most lingering emotional and cognitive symptoms.

A multi-expert ensemble system for predicting Alzheimer transition using clinical features

Alzheimer’s disease (AD) diagnosis often requires invasive examinations (e.g., liquor analyses), expensive tools (e.g., brain imaging) and highly specialized personnel. The diagnosis commonly is established when the disorder has already caused severe brain damage, and the clinical signs begin to be apparent. Instead, accessible and low-cost approaches for early identification of subjects at high risk for developing AD years before they show overt symptoms are fundamental to provide a critical time window for more effective clinical management, treatment, and care planning. This article proposes an ensemble-based machine learning algorithm for predicting AD development within 9 years from first overt signs and using just five clinical features that are easily detectable with neuropsychological tests. The validation of the system involved both healthy individuals and mild cognitive impairment (MCI) patients drawn from the ADNI open dataset, at variance with previous studies that considered only MCI. The system shows higher levels of balanced accuracy, negative predictive value, and specificity than other similar solutions. These results represent a further important step to build a preventive fast-screening machine-learning-based tool to be used as a part of routine healthcare screenings.

A novel ensemble feature selection method for pixel-level segmentation of HER2 overexpression

Classifying histopathology images on a pixel-level requires sets of features able to capture the complex characteristics of the images, like the irregular cell morphology and the color heterogeneity on the tissue aspect. In this context, feature selection becomes a crucial step in the classification process such that it reduces model complexity and computational costs, avoids overfitting, and thereby it improves the model performance. In this study, we propose a new ensemble feature selection method by combining a set of base selectors, classifiers, and rank aggregation methods, aiming to determine from any initial set of handcrafted features, a smaller set of relevant color and texture pixel-level features, subsequently used for segmenting HER2 overexpression on a pixel-level, in breast cancer tissue images. We have been able to significantly reduce the set of initial features, using the proposed ensemble feature selection method. The best results are obtained using $$\chi ^2$$ χ 2 , Random Forest, and Runoff as the based selector, classifier, and aggregation method, respectively. The classification performance of the best model trained on the selected features set results in 0.939 recall, 0.866 specificity, 0.903 accuracy, 0.875 precision, and 0.906 F1-score.