Uncertainty estimation for deep learning-based automated analysis of 12-lead electrocardiograms

Adopting artificial intelligence in cardiovascular medicine: a scoping review

Recent years have witnessed significant transformations in cardiovascular medicine, driven by the rapid evolution of artificial intelligence (AI). This scoping review was conducted to capture the breadth of AI applications within cardiovascular science. Employing a structured approach, we sourced relevant articles from PubMed, with an emphasis on journals encompassing general cardiology and digital medicine. We applied filters to highlight cardiovascular articles published in journals focusing on general internal medicine, cardiology and digital medicine, thereby identifying the prevailing trends in the field. Following a comprehensive full-text screening, a total of 140 studies were identified. Over the preceding 5 years, cardiovascular medicine's interplay with AI has seen an over tenfold augmentation. This expansive growth encompasses multiple cardiovascular subspecialties, including but not limited to, general cardiology, ischemic heart disease, heart failure, and arrhythmia. Deep learning emerged as the predominant methodology. The majority of AI endeavors in this domain have been channeled toward enhancing diagnostic and prognostic capabilities, utilizing resources such as hospital datasets, electrocardiograms, and echocardiography. A significant uptrend was observed in AI's application for omics data analysis. However, a clear gap persists in AI's full-scale integration into the clinical decision-making framework. AI, particularly deep learning, has demonstrated robust applications across cardiovascular subspecialties, indicating its transformative potential in this field. As we continue on this trajectory, ensuring the alignment of technological progress with medical ethics becomes crucial. The abundant digital health data today further accentuates the need for meticulous systematic reviews, tailoring them to each cardiovascular subspecialty.

Automatic triage of twelve-lead electrocardiograms using deep convolutional neural networks: a first implementation study

Aims

Expert knowledge to correctly interpret electrocardiograms (ECGs) is not always readily available. An artificial intelligence (AI)-based triage algorithm (DELTAnet), able to support physicians in ECG prioritization, could help reduce current logistic burden of overreading ECGs and improve time to treatment for acute and life-threatening disorders. However, the effect of clinical implementation of such AI algorithms is rarely investigated.

Methods and results

Adult patients at non-cardiology departments who underwent ECG testing as a part of routine clinical care were included in this prospective cohort study. DELTAnet was used to classify 12-lead ECGs into one of the following triage classes: normal, abnormal not acute, subacute, and acute. Performance was compared with triage classes based on the final clinical diagnosis. Moreover, the associations between predicted classes and clinical outcomes were investigated. A total of 1061 patients and ECGs were included. Performance was good with a mean concordance statistic of 0.96 (95% confidence interval 0.95-0.97) when comparing DELTAnet with the clinical triage classes. Moreover, zero ECGs that required a change in policy or referral to the cardiologist were missed and there was a limited number of cases predicted as acute that did not require follow-up (2.6%).

Conclusion

This study is the first to prospectively investigate the impact of clinical implementation of an ECG-based AI triage algorithm. It shows that DELTAnet is efficacious and safe to be used in clinical practice for triage of 12-lead ECGs in non-cardiology hospital departments.

[Artificial intelligence in wearable electrocardiogram monitoring]

Electrocardiogram (ECG) monitoring owns important clinical value in diagnosis, prevention and rehabilitation of cardiovascular disease (CVD). With the rapid development of Internet of Things (IoT), big data, cloud computing, artificial intelligence (AI) and other advanced technologies, wearable ECG is playing an increasingly important role. With the aging process of the population, it is more and more urgent to upgrade the diagnostic mode of CVD. Using AI technology to assist the clinical analysis of long-term ECGs, and thus to improve the ability of early detection and prediction of CVD has become an important direction. Intelligent wearable ECG monitoring needs the collaboration between edge and cloud computing. Meanwhile, the clarity of medical scene is conducive for the precise implementation of wearable ECG monitoring. This paper first summarized the progress of AI-related ECG studies and the current technical orientation. Then three cases were depicted to illustrate how the AI in wearable ECG cooperate with the clinic. Finally, we demonstrated the two core issues-the reliability and worth of AI-related ECG technology and prospected the future opportunities and challenges.

Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023)

Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare. A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging. Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals. Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.

Quantifying deep neural network uncertainty for atrial fibrillation detection with limited labels

Atrial fibrillation (AF) is the most common arrhythmia found in the intensive care unit (ICU), and is associated with many adverse outcomes. Effective handling of AF and similar arrhythmias is a vital part of modern critical care, but obtaining knowledge about both disease burden and effective interventions often requires costly clinical trials. A wealth of continuous, high frequency physiological data such as the waveforms derived from electrocardiogram telemetry are promising sources for enriching clinical research. Automated detection using machine learning and in particular deep learning has been explored as a solution for processing these data. However, a lack of labels, increased presence of noise, and inability to assess the quality and trustworthiness of many machine learning model predictions pose challenges to interpretation. In this work, we propose an approach for training deep AF models on limited, noisy data and report uncertainty in their predictions. Using techniques from the fields of weakly supervised learning, we leverage a surrogate model trained on non-ICU data to create imperfect labels for a large ICU telemetry dataset. We combine these weak labels with techniques to estimate model uncertainty without the need for extensive human data annotation. AF detection models trained using this process demonstrated higher classification performance (0.64-0.67 F1 score) and improved calibration (0.05-0.07 expected calibration error).

Uncertainty estimation for margin detection in cancer surgery using mass spectrometry

PURPOSE

Rapid evaporative ionization mass spectrometry (REIMS) is an emerging technology for clinical margin detection. Deployment of REIMS depends on construction of reliable deep learning models that can categorize tissue according to its metabolomic signature. Challenges associated with developing these models include the presence of noise during data acquisition and the variance in tissue signatures between patients. In this study, we propose integration of uncertainty estimation in deep models to factor predictive confidence into margin detection in cancer surgery.

METHODS

iKnife is used to collect 693 spectra of cancer and healthy samples acquired from 91 patients during basal cell carcinoma resection. A Bayesian neural network and two baseline models are trained on these data to perform classification as well as uncertainty estimation. The samples with high estimated uncertainty are then removed, and new models are trained using the clean data. The performance of proposed and baseline models, with different ratios of filtered data, is then compared.

RESULTS

The data filtering does not improve the performance of the baseline models as they cannot provide reliable estimations of uncertainty. In comparison, the proposed model demonstrates a statistically significant improvement in average balanced accuracy (75.2%), sensitivity (74.1%) and AUC (82.1%) after removing uncertain training samples. We also demonstrate that if highly uncertain samples are predicted and removed from the test data, sensitivity further improves to 88.2%.

CONCLUSIONS

This is the first study that applies uncertainty estimation to inform model training and deployment for tissue recognition in cancer surgery. Uncertainty estimation is leveraged in two ways: by factoring a measure of input noise in training the models and by including predictive confidence in reporting the outputs. We empirically show that considering uncertainty for model development can help improve the overall accuracy of a margin detection system using REIMS.

Practical Lessons on 12-Lead ECG Classification: Meta-Analysis of Methods From PhysioNet/Computing in Cardiology Challenge 2020

Cardiovascular diseases (CVDs) are one of the most fatal disease groups worldwide. Electrocardiogram (ECG) is a widely used tool for automatically detecting cardiac abnormalities, thereby helping to control and manage CVDs. To encourage more multidisciplinary researches, PhysioNet/Computing in Cardiology Challenge 2020 (Challenge 2020) provided a public platform involving multi-center databases and automatic evaluations for ECG classification tasks. As a result, 41 teams successfully submitted their solutions and were qualified for rankings. Although Challenge 2020 was a success, there has been no in-depth methodological meta-analysis of these solutions, making it difficult for researchers to benefit from the solutions and results. In this study, we aim to systematically review the 41 solutions in terms of data processing, feature engineering, model architecture, and training strategy. For each perspective, we visualize and statistically analyze the effectiveness of the common techniques, and discuss the methodological advantages and disadvantages. Finally, we summarize five practical lessons based on the aforementioned analysis: (1) Data augmentation should be employed and adapted to specific scenarios; (2) Combining different features can improve performance; (3) A hybrid design of different types of deep neural networks (DNNs) is better than using a single type; (4) The use of end-to-end architectures should depend on the task being solved; (5) Multiple models are better than one. We expect that our meta-analysis will help accelerate the research related to ECG classification based on machine-learning models.

Application of artificial intelligence to the electrocardiogram

Artificial intelligence (AI) has given the electrocardiogram (ECG) and clinicians reading them super-human diagnostic abilities. Trained without hard-coded rules by finding often subclinical patterns in huge datasets, AI transforms the ECG, a ubiquitous, non-invasive cardiac test that is integrated into practice workflows, into a screening tool and predictor of cardiac and non-cardiac diseases, often in asymptomatic individuals. This review describes the mathematical background behind supervised AI algorithms, and discusses selected AI ECG cardiac screening algorithms including those for the detection of left ventricular dysfunction, episodic atrial fibrillation from a tracing recorded during normal sinus rhythm, and other structural and valvular diseases. The ability to learn from big data sets, without the need to understand the biological mechanism, has created opportunities for detecting non-cardiac diseases as COVID-19 and introduced challenges with regards to data privacy. Like all medical tests, the AI ECG must be carefully vetted and validated in real-world clinical environments. Finally, with mobile form factors that allow acquisition of medical-grade ECGs from smartphones and wearables, the use of AI may enable massive scalability to democratize healthcare.