Comparison among random forest, logistic regression, and existing clinical risk scores for predicting outcomes in patients with atrial fibrillation: A report from the J-RHYTHM registry

Machine Learning Predicts Bleeding Risk in Atrial Fibrillation Patients on Direct Oral Anticoagulant

Predicting major bleeding in nonvalvular atrial fibrillation (AF) patients on direct oral anticoagulants (DOACs) is crucial for personalized care. Alternatives like left atrial appendage closure devices lower stroke risk with fewer nonprocedural bleeds. This study compares machine learning (ML) models with conventional bleeding risk scores (HAS-BLED, ORBIT, and ATRIA) for predicting bleeding events requiring hospitalization in AF patients on DOACs at their index cardiologist visit. This retrospective cohort study used electronic health records from 2010 to 2022 at the University of Pittsburgh Medical Center. It included 24,468 nonvalvular AF patients (age ≥18) on DOACs, excluding those with prior significant bleeding or warfarin use. The primary outcome was hospitalization for bleeding within one year, with follow-up at one, two, and five years. ML algorithms (logistic regression, classification trees, random forest, XGBoost, k-nearest neighbor, naïve Bayes) were compared for performance. Of 24,468 patients, 553 (2.3%) had bleeding within one year, 829 (3.5%) within two years, and 1,292 (5.8%) within five years. ML models outperformed HAS-BLED, ATRIA, and ORBIT in 1-year predictions. The random forest model achieved an AUC of 0.76 (0.70 to 0.81), G-Mean of 0.67, and net reclassification index of 0.14 compared to HAS-BLED's AUC of 0.57 (p < 0.001). ML models showed superior results across all timepoints and for hemorrhagic stroke. SHAP analysis identified new risk factors, including BMI, cholesterol profile, and insurance type. In conclusion, ML models demonstrated improved performance to conventional bleeding risk scores and uncovered novel risk factors, offering potential for more personalized bleeding risk assessment in AF patients on DOACs.

Reevaluating feature importances in machine learning models for schizophrenia and bipolar disorder: The need for true associations

Skorobogatov et al. developed supervised machine learning models to predict diagnoses and illness states in schizophrenia and bipolar disorder. However, their reliance on bootstrap forests and generalized regressions introduces significant biases in feature importance assessments. This paper highlights the critical distinction between feature importances generated by machine learning and actual associations, which are often model-specific and context-dependent. We underscore the limitations of biased feature importances and advocate for the use of robust statistical methods, such as Chi-squared tests and Spearman's correlation, to reveal true associations. Reassessing findings with these methods will enable more accurate interpretations and reinforce the importance of understanding the limitations inherent in machine learning methodologies.

Machine learning for outcome prediction in patients with non-valvular atrial fibrillation from the GLORIA-AF registry

Clinical risk scores that predict outcomes in patients with atrial fibrillation (AF) have modest predictive value. Machine learning (ML) may achieve greater results when predicting adverse outcomes in patients with recently diagnosed AF. Several ML models were tested and compared with current clinical risk scores on a cohort of 26,183 patients (mean age 70.13 (standard deviation 10.13); 44.8% female) with non-valvular AF. Inputted into the ML models were 23 demographic variables alongside comorbidities and current treatments. For one-year stroke prediction, ML achieved an area under the curve (AUC) of 0.653 (95% confidence interval 0.576-0.730), compared to the CHADS2 and CHA2DS2-VASc scores performance of 0.587 (95% CI 0.559-0.615) and 0.535 (95% CI 0.521-0.550), respectively. Using ML for one-year major bleed prediction increased the AUC from 0.537 (95% CI 0.518-0.557) generated by the HAS-BLED score to 0.677 (95% CI 0.619-0.724). ML was able to predict one-year and three-year all-cause mortality with an AUC of 0.734 (95% CI 0.696-0.771) and 0.742 (95% CI 0.718-0.766). In this study a significant improvement in performance was observed when transitioning from clinical risk scores to machine learning-based approaches across all applications tested. Obtaining precise prediction tools is desirable for increased interventions to reduce event rates.Trial Registry https://www.clinicaltrials.gov ; Unique identifier: NCT01468701, NCT01671007, NCT01937377.

Machine Learning Identifies Arrhythmogenic Features of QRS Fragmentation in Human Cardiomyopathy: Implications for Improving Risk Stratification

BACKGROUND

Heterogeneous ventricular activation can provide the substrate for ventricular arrhythmias (VA), but its manifestation on the electrocardiogram (ECG) as a risk stratifier is not well-defined.

OBJECTIVE

To characterize the spatiotemporal features of QRS peaks that best predict VA in patients with cardiomyopathy (CM) using machine learning (ML).

METHODS

Prospectively enrolled CM patients with prophylactic defibrillators (n=95) underwent digital, high-resolution ECG recordings during intrinsic rhythm and ventricular pacing at 100 to 120 beats/min. Intra QRS peaks in the signal-averaged precordial leads were identified and their characteristics (amplitude, width, and timing within the QRS) were transformed into 4-bin histograms. Random forest models of these characteristics in each lead alongside clinical data were trained on 76 patients and tested on 19 patients with cross-validation to determine the features that predicted VA.

RESULTS

Patients were followed up for 45 (22-48) months, and 21% had VA. The individual machine learning (ML) models determined (area under the receiver operating characteristic curve [AUROC]) intrinsic QRS peak amplitude (0.88), width (0.78), and location (0.84) to all predict VA. In a combined model including all QRS peak characteristics, peaks with amplitude < 31 μV in V1, a width of 4 to 8 ms in V1, and location in the final quarter of the QRS of V1 were most predictive. Neither clinical data nor QRS peak characteristics assessed during ventricular pacing improved VA prediction when combined with intrinsic QRS peak characteristics.

CONCLUSIONS

Arrhythmogenic QRS fragmentation is characterized by narrow, low-amplitude peaks in the terminal QRS of lead V1. These features alone without clinical variables or ventricular pacing are sufficient to accurately risk stratify CM patients.

Machine Learning - Based Bleeding Risk Predictions in Atrial Fibrillation Patients on Direct Oral Anticoagulants

Importance

Accurately predicting major bleeding events in non-valvular atrial fibrillation (AF) patients on direct oral anticoagulants (DOACs) is crucial for personalized treatment and improving patient outcomes, especially with emerging alternatives like left atrial appendage closure devices. The left atrial appendage closure devices reduce stroke risk comparably but with significantly fewer non-procedural bleeding events.

Objective

To evaluate the performance of machine learning (ML) risk models in predicting clinically significant bleeding events requiring hospitalization and hemorrhagic stroke in non-valvular AF patients on DOACs compared to conventional bleeding risk scores (HAS-BLED, ORBIT, and ATRIA) at the index visit to a cardiologist for AF management.

Design

Prognostic modeling with retrospective cohort study design using electronic health record (EHR) data, with clinical follow-up at one-, two-, and five-years.

Setting

University of Pittsburgh Medical Center (UPMC) system.

Participants

24,468 non-valvular AF patients aged ≥18 years treated with DOACs, excluding those with prior history of significant bleeding, other indications for DOACs, on warfarin or contraindicated to DOACs.

Exposures

DOAC therapy for non-valvular AF.

Main Outcomes and Measures

The primary endpoint was clinically significant bleeding requiring hospitalization within one year of index visit. The models incorporated demographic, clinical, and laboratory variables available in the EHR at the index visit.

Results

Among 24,468 patients, 553 (2.3%) had bleeding events within one year, 829 (3.5%) within two years, and 1,292 (5.8%) within five years of index visit. We evaluated multivariate logistic regression and ML models including random forest, classification trees, k-nearest neighbor, naive Bayes, and extreme gradient boosting (XGBoost) which modestly outperformed HAS-BLED, ATRIA, and ORBIT scores in predicting clinically significant bleeding at 1-year follow-up. The best performing model (random forest) showed area under the curve (AUC-ROC) 0.76 (0.70-0.81), G-Mean score of 0.67, net reclassification index 0.14 compared to 0.57 (0.50-0.63), G-Mean score of 0.57 for HASBLED score, p-value for difference <0.001. The ML models had improved performance compared to conventional risk across time-points of 2-year and 5-years and within the subgroup of hemorrhagic stroke. SHAP analysis identified novel risk factors including measures from body mass index, cholesterol profile, and insurance type beyond those used in conventional risk scores.

Conclusions and Relevance

Our findings demonstrate the superior performance of ML models compared to conventional bleeding risk scores and identify novel risk factors highlighting the potential for personalized bleeding risk assessment in AF patients on DOACs.

Sepsis-induced coagulopathy is associated with new episodes of atrial fibrillation in patients admitted to critical care in sinus rhythm

Background

Sepsis is a life-threatening disease commonly complicated by activation of coagulation and immune pathways. Sepsis-induced coagulopathy (SIC) is associated with micro- and macrothrombosis, but its relation to other cardiovascular complications remains less clear. In this study we explored associations between SIC and the occurrence of atrial fibrillation (AF) in patients admitted to the Intensive Care Unit (ICU) in sinus rhythm. We also aimed to identify predictive factors for the development of AF in patients with and without SIC.

Methods

Data were extracted from the publicly available AmsterdamUMCdb database. Patients with sepsis and documented sinus rhythm on admission to ICU were included. Patients were stratified into those who fulfilled the criteria for SIC and those who did not. Following univariate analysis, logistic regression models were developed to describe the association between routinely documented demographics and blood results and the development of at least one episode of AF. Machine learning methods (gradient boosting machines and random forest) were applied to define the predictive importance of factors contributing to the development of AF.

Results

Age was the strongest predictor for the development of AF in patients with and without SIC. Routine coagulation tests activated Partial Thromboplastin Time (aPTT) and International Normalized Ratio (INR) and C-reactive protein (CRP) as a marker of inflammation were also associated with AF occurrence in SIC-positive and SIC-negative patients. Cardiorespiratory parameters (oxygen requirements and heart rate) showed predictive potential.

Conclusion

Higher INR, elevated CRP, increased heart rate and more severe respiratory failure are risk factors for occurrence of AF in critical illness, suggesting an association between cardiac, respiratory and immune and coagulation pathways. However, age was the most dominant factor to predict the first episodes of AF in patients admitted in sinus rhythm with and without SIC.

HAS-BLED vs. ORBIT scores in anticoagulated patients with atrial fibrillation: A systematic review and meta-analysis

Background

The 2021 UK National Institute for Health and Care Excellence guidelines tend to recommend the ORBIT score for predicting bleeding risk in patients with atrial fibrillation (AF) with anticoagulants. Herein, we comprehensively re-assessed the predicted abilities of the HAS-BLED vs. ORBIT score since several newly published data showed different findings.

Methods

We comprehensively searched the PubMed electronic database until December 2021 to identify relevant studies reporting the ORBIT vs. HAS-BLED scores in anticoagulated patients with AF. Their predicted abilities were assessed using the C-index, reclassification, and calibration analysis.

Results

Finally, 17 studies were included in this review. In the pooled analysis, the ORBIT score had a C-index of 0.63 (0.60-0.66), 0.59 (0.53-0.66), and 0.57 (0.48-0.67) for major bleeding, any clinically relevant bleeding, and intracranial bleeding, respectively, while the HAS-BLED score had a C-index of 0.61 (0.59-0.63), 0.59 (0.56-0.63), and 0.57 (0.51-0.64) for major bleeding, any clinically relevant bleeding, and intracranial bleeding, respectively. There were no statistical differences in the accuracy of predicting these bleeding events between the two scoring systems. For the outcome of major bleeding, the subgroup analyses based on vitamin K antagonists vs. direct oral anticoagulants suggested no differences in the discrimination ability between the HAS-BLED and ORBIT scores. Reclassification and calibration analyses of HAS-BLED vs. ORBIT should be further assessed due to the limited and conflicting data.

Conclusion

Our current findings suggested that the HAS-BLED and ORBIT scores at least had similar predictive abilities for major bleeding risk in anticoagulated (vitamin K antagonists or direct oral anticoagulants) patients with AF, supporting the use of the HAS-BLED score in clinical practice.

Machine learning and statistic analysis to predict drug treatment outcome in pediatric epilepsy patients with tuberous sclerosis complex

OBJECTIVES

We aimed to investigate the association between multi-modality features and epilepsy drug treatment outcomes and propose a machine learning model to predict epilepsy drug treatment outcomes with multi-modality features.

METHODS

This retrospective study consecutively enrolled 103 epilepsy children with rare TSC. Multi-modality data were used to characterize risk factors for epilepsy drug treatment outcome of TSC, including clinical data, TSC1, and TSC2 genes test results, magnetic resonance imaging (MRI), computerized tomography (CT), and electroencephalogram (EEG). Three common feature selection methods and six common machine learning models were used to find the best combination of feature selection and machine learning model for epilepsy drug treatment outcomes prediction with multi-modality features for TSC clinical application.

RESULTS

The analysis of variance based on selected 35 features combined with multilayer perceptron (MLP) model achieved the best area-under-curve score (AUC) of 0.812 (±0.005). Infantile spasms, EEG discharge type, epileptiform discharge in the right frontal area of EEG, drug-resistant epilepsy, gene mutation type, and type II lesions were positively correlated with drug treatment outcome. Age of onset and age of visiting doctors were negatively correlated with drug treatment outcome (p < 0.05). Our machine learning results found that among MRI features, lesion type is the most important in the outcome prediction, followed by location and quantity.

CONCLUSION

We developed and validated an effective prediction model for epilepsy drug treatment outcomes of TSC. Our results suggested that multi-modality features analysis and MLP-based machine learning can predict epilepsy drug treatment outcomes of TSC.

Artificial intelligence for the detection, prediction, and management of atrial fibrillation

The present article reviews the state of the art of machine learning algorithms for the detection, prediction, and management of atrial fibrillation (AF), as well as of the development and evaluation of artificial intelligence (AI) in cardiology and beyond. Today, AI detects AF with a high accuracy using 12-lead or single-lead electrocardiograms or photoplethysmography. The prediction of paroxysmal or future AF currently operates at a level of precision that is too low for clinical use. Further studies are needed to determine whether patient selection for interventions may be possible with machine learning.

Applying Machine Learning to Carotid Sonographic Features for Recurrent Stroke in Patients With Acute Stroke

BACKGROUND

Although carotid sonographic features have been used as predictors of recurrent stroke, few large-scale studies have explored the use of machine learning analysis of carotid sonographic features for the prediction of recurrent stroke.

METHODS

We retrospectively collected electronic medical records of enrolled patients from the data warehouse of China Medical University Hospital, a tertiary medical center in central Taiwan, from January 2012 to November 2018. We included patients who underwent a documented carotid ultrasound within 30 days of experiencing an acute first stroke during the study period. We classified these participants into two groups: those with non-recurrent stroke (those who has not been diagnosed with acute stroke again during the study period) and those with recurrent stoke (those who has been diagnosed with acute stroke during the study period). A total of 1,235 carotid sonographic parameters were analyzed. Data on the patients' demographic characteristics and comorbidities were also collected. Python 3.7 was used as the programming language, and the scikit-learn toolkit was used to complete the derivation and verification of the machine learning methods.

RESULTS

In total, 2,411 patients were enrolled in this study, of whom 1,896 and 515 had non-recurrent and recurrent stroke, respectively. After extraction, 43 features of carotid sonography (36 carotid sonographic parameters and seven transcranial color Doppler sonographic parameter) were analyzed. For predicting recurrent stroke, CatBoost achieved the highest area under the curve (0.844, CIs 95% 0.824-0.868), followed by the Light Gradient Boosting Machine (0.832, CIs 95% 0.813-0.851), random forest (0.819, CIs 95% 0.802-0.846), support-vector machine (0.759, CIs 95% 0.739-0.781), logistic regression (0.781, CIs 95% 0.764-0.800), and decision tree (0.735, CIs 95% 0.717-0.755) models.

CONCLUSION

When using the CatBoost model, the top three features for predicting recurrent stroke were determined to be the use of anticoagulation medications, the use of NSAID medications, and the resistive index of the left subclavian artery. The CatBoost model demonstrated efficiency and achieved optimal performance in the predictive classification of non-recurrent and recurrent stroke.

Machine Learning-Based Prediction Models for Depression Symptoms Among Chinese Healthcare Workers During the Early COVID-19 Outbreak in 2020: A Cross-Sectional Study

BACKGROUND

The 2019 novel coronavirus (COVID-19)-related depression symptoms of healthcare workers have received worldwide recognition. Although many studies identified risk exposures associated with depression symptoms among healthcare workers, few have focused on a predictive model using machine learning methods. As a society, governments, and organizations are concerned about the need for immediate interventions and alert systems for healthcare workers who are mentally at-risk. This study aims to develop and validate machine learning-based models for predicting depression symptoms using survey data collected during the COVID-19 outbreak in China.

METHOD

Surveys were conducted of 2,574 healthcare workers in hospitals designated to care for COVID-19 patients between 20 January and 11 February 2020. The patient health questionnaire (PHQ)-9 was used to measure the depression symptoms and quantify the severity, a score of ≥5 on the PHQ-9 represented depression symptoms positive, respectively. Four machine learning approaches were trained (75% of data) and tested (25% of data). Cross-validation with 100 repetitions was applied to the training dataset for hyperparameter tuning. Finally, all models were compared to evaluate their predictive performances and screening utility: decision tree, logistics regression with least absolute shrinkage and selection operator (LASSO), random forest, and gradient-boosting tree.

RESULTS

Important risk predictors identified and ranked by the machine learning models were highly consistent: self-perceived health status factors always occupied the top five most important predictors, followed by worried about infection, working on the frontline, a very high level of uncertainty, having received any form of psychological support material and having COVID-19-like symptoms. The area under the curve [95% CI] of machine learning models were as follows: LASSO model, 0.824 [0.792-0.856]; random forest, 0.828 [0.797-0.859]; gradient-boosting tree, 0.829 [0.798-0.861]; and decision tree, 0.785 [0.752-0.819]. The calibration plot indicated that the LASSO model, random forest, and gradient-boosting tree fit the data well. Decision curve analysis showed that all models obtained net benefits for predicting depression symptoms.

CONCLUSIONS

This study shows that machine learning prediction models are suitable for making predictions about mentally at-risk healthcare workers predictions in a public health emergency setting. The application of multidimensional machine learning models could support hospitals' and healthcare workers' decision-making on possible psychological interventions and proper mental health management.