Diagnostic Accuracy of Machine Learning Models to Identify Congenital Heart Disease: A Meta-Analysis

Prediction of cyanotic and acyanotic congenital heart disease using machine learning models

BACKGROUND

Congenital heart disease is most commonly seen in neonates and it is a major cause of pediatric illness and childhood morbidity and mortality.

AIM

To identify and build the best predictive model for predicting cyanotic and acyanotic congenital heart disease in children during pregnancy and identify their potential risk factors.

METHODS

The data were collected from the Pediatric Cardiology Department at Chaudhry Pervaiz Elahi Institute of Cardiology Multan, Pakistan from December 2017 to October 2019. A sample of 3900 mothers whose children were diagnosed with cyanotic or acyanotic congenital heart disease was taken. Multivariate outlier detection methods were used to identify the potential outliers. Different machine learning models were compared, and the best-fitted model was selected using the area under the curve, sensitivity, and specificity of the models.

RESULTS

Out of 3900 patients included, about 69.5% had acyanotic and 30.5% had cyanotic congenital heart disease. Males had more cases of acyanotic (53.6%) and cyanotic (54.5%) congenital heart disease as compared to females. The odds of having cyanotic was 1.28 times higher for children whose mothers used more fast food frequently during pregnancy. The artificial neural network model was selected as the best predictive model with an area under the curve of 0.9012, sensitivity of 65.76%, and specificity of 97.23%.

CONCLUSION

Children having a positive family history are at very high risk of having cyanotic and acyanotic congenital heart disease. Males are more at risk and their mothers need more care, good food, and physical activity during pregnancy. The best-fitted model for predicting cyanotic and acyanotic congenital heart disease is the artificial neural network. The results obtained and the best model identified will be useful for medical practitioners and public health scientists for an informed decision-making process about the earlier diagnosis and improve the health condition of children in Pakistan.

Accurately assessing congenital heart disease using artificial intelligence

Congenital heart disease (CHD) remains a significant global health challenge, particularly contributing to newborn mortality, with the highest rates observed in middle- and low-income countries due to limited healthcare resources. Machine learning (ML) presents a promising solution by developing predictive models that more accurately assess the risk of mortality associated with CHD. These ML-based models can help healthcare professionals identify high-risk infants and ensure timely and appropriate care. In addition, ML algorithms excel at detecting and analyzing complex patterns that can be overlooked by human clinicians, thereby enhancing diagnostic accuracy. Despite notable advancements, ongoing research continues to explore the full potential of ML in the identification of CHD. The proposed article provides a comprehensive analysis of the ML methods for the diagnosis of CHD in the last eight years. The study also describes different data sets available for CHD research, discussing their characteristics, collection methods, and relevance to ML applications. In addition, the article also evaluates the strengths and weaknesses of existing algorithms, offering a critical review of their performance and limitations. Finally, the article proposes several promising directions for future research, with the aim of further improving the efficacy of ML in the diagnosis and treatment of CHD.

Does machine learning have a high performance to predict obesity among adults and older adults? A systematic review and meta-analysis

AIM

Machine learning may be a tool with the potential for obesity prediction. This study aims to review the literature on the performance of machine learning models in predicting obesity and to quantify the pooled results through a meta-analysis.

DATA SYNTHESIS

A systematic review and meta-analysis were conducted, including studies that used machine learning to predict obesity. Searches were conducted in October 2023 across databases including LILACS, Web of Science, Scopus, Embase, and CINAHL. We included studies that utilized classification models and reported results in the Area Under the ROC Curve (AUC) (PROSPERO registration: CRD42022306940), without imposing restrictions on the year of publication. The risk of bias was assessed using an adapted version of the Transparent Reporting of a multivariable prediction model for individual Prognosis or Diagnosis (TRIPOD). Meta-analysis was conducted using MedCalc software. A total of 14 studies were included, with the majority demonstrating satisfactory performance for obesity prediction, with AUCs exceeding 0.70. The random forest algorithm emerged as the top performer in obesity prediction, achieving an AUC of 0.86 (95%CI: 0.76-0.96; I2: 99.8%), closely followed by logistic regression with an AUC of 0.85 (95%CI: 0.75-0.95; I2: 99.6%). The least effective model was gradient boosting, with an AUC of 0.77 (95%CI: 0.71-0.82; I2: 98.1%).

CONCLUSION

Machine learning models demonstrated satisfactory predictive performance for obesity. However, future research should utilize more comparable data, larger databases, and a broader range of machine learning models.

Enhancing Clinical Validation for Early Cardiovascular Disease Prediction through Simulation, AI, and Web Technology

Cardiovascular diseases (CVDs) remain a major global health challenge and a leading cause of mortality, highlighting the need for improved predictive models. We introduce an innovative agent-based dynamic simulation technique that enhances our AI models' capacity to predict CVD progression. This method simulates individual patient responses to various cardiovascular risk factors, improving prediction accuracy and detail. Also, by incorporating an ensemble learning model and interface of web application in the context of CVD prediction, we developed an AI dashboard-based model to enhance the accuracy of disease prediction and provide a user-friendly app. The performance of traditional algorithms was notable, with Ensemble learning and XGBoost achieving accuracies of 91% and 95%, respectively. A significant aspect of our research was the integration of these models into a streamlit-based interface, enhancing user accessibility and experience. The streamlit application achieved a predictive accuracy of 97%, demonstrating the efficacy of combining advanced AI techniques with user-centered web applications in medical prediction scenarios. This 97% confidence level was evaluated by Brier score and calibration curve. The design of the streamlit application facilitates seamless interaction between complex ML models and end-users, including clinicians and patients, supporting its use in real-time clinical settings. While the study offers new insights into AI-driven CVD prediction, we acknowledge limitations such as the dataset size. In our research, we have successfully validated our predictive proposed methodology against an external clinical setting, demonstrating its robustness and accuracy in a real-world fixture. The validation process confirmed the model's efficacy in the early detection of CVDs, reinforcing its potential for integration into clinical workflows to aid in proactive patient care and management. Future research directions include expanding the dataset, exploring additional algorithms, and conducting clinical trials to validate our findings. This research provides a valuable foundation for future studies, aiming to make significant strides against CVDs.

Identification of Congenital Valvular Murmurs in Young Patients Using Deep Learning-Based Attention Transformers and Phonocardiograms

One in every four newborns suffers from congenital heart disease (CHD) that causes defects in the heart structure. The current gold-standard assessment technique, echocardiography, causes delays in the diagnosis owing to the need for experts who vary markedly in their ability to detect and interpret pathological patterns. Moreover, echo is still causing cost difficulties for low- and middle-income countries. Here, we developed a deep learning-based attention transformer model to automate the detection of heart murmurs caused by CHD at an early stage of life using cost-effective and widely available phonocardiography (PCG). PCG recordings were obtained from 942 young patients at four major auscultation locations, including the aortic valve (AV), mitral valve (MV), pulmonary valve (PV), and tricuspid valve (TV), and they were annotated by experts as absent, present, or unknown murmurs. A transformation to wavelet features was performed to reduce the dimensionality before the deep learning stage for inferring the medical condition. The performance was validated through 10-fold cross-validation and yielded an average accuracy and sensitivity of 90.23 % and 72.41 %, respectively. The accuracy of discriminating between murmurs' absence and presence reached 76.10 % when evaluated on unseen data. The model had accuracies of 70 %, 88 %, and 86 % in predicting murmur presence in infants, children, and adolescents, respectively. The interpretation of the model revealed proper discrimination between the learned attributes, and AV channel was found important (score 0.75) for the murmur absence predictions while MV and TV were more important for murmur presence predictions. The findings potentiate deep learning as a powerful front-line tool for inferring CHD status in PCG recordings leveraging early detection of heart anomalies in young people. It is suggested as a tool that can be used independently from high-cost machinery or expert assessment.

Congenital heart disease detection by pediatric electrocardiogram based deep learning integrated with human concepts

Early detection is critical to achieving improved treatment outcomes for child patients with congenital heart diseases (CHDs). Therefore, developing effective CHD detection techniques using low-cost and non-invasive pediatric electrocardiogram are highly desirable. We propose a deep learning approach for CHD detection, CHDdECG, which automatically extracts features from pediatric electrocardiogram and wavelet transformation characteristics, and integrates them with key human-concept features. Developed on 65,869 cases, CHDdECG achieved ROC-AUC of 0.915 and specificity of 0.881 on a real-world test set covering 12,000 cases. Additionally, on two external test sets with 7137 and 8121 cases, the overall ROC-AUC were 0.917 and 0.907 while specificities were 0.937 and 0.907. Notably, CHDdECG surpassed cardiologists in CHD detection performance comparison, and feature importance scores suggested greater influence of automatically extracted electrocardiogram features on CHD detection compared with human-concept features, implying that CHDdECG may grasp some knowledge beyond human cognition. Our study directly impacts CHD detection with pediatric electrocardiogram and demonstrates the potential of pediatric electrocardiogram for broader benefits.

The Performance of Wearable AI in Detecting Stress Among Students: Systematic Review and Meta-Analysis

BACKGROUND

Students usually encounter stress throughout their academic path. Ongoing stressors may lead to chronic stress, adversely affecting their physical and mental well-being. Thus, early detection and monitoring of stress among students are crucial. Wearable artificial intelligence (AI) has emerged as a valuable tool for this purpose. It offers an objective, noninvasive, nonobtrusive, automated approach to continuously monitor biomarkers in real time, thereby addressing the limitations of traditional approaches such as self-reported questionnaires.

OBJECTIVE

This systematic review and meta-analysis aim to assess the performance of wearable AI in detecting and predicting stress among students.

METHODS

Search sources in this review included 7 electronic databases (MEDLINE, Embase, PsycINFO, ACM Digital Library, Scopus, IEEE Xplore, and Google Scholar). We also checked the reference lists of the included studies and checked studies that cited the included studies. The search was conducted on June 12, 2023. This review included research articles centered on the creation or application of AI algorithms for the detection or prediction of stress among students using data from wearable devices. In total, 2 independent reviewers performed study selection, data extraction, and risk-of-bias assessment. The Quality Assessment of Diagnostic Accuracy Studies-Revised tool was adapted and used to examine the risk of bias in the included studies. Evidence synthesis was conducted using narrative and statistical techniques.

RESULTS

This review included 5.8% (19/327) of the studies retrieved from the search sources. A meta-analysis of 37 accuracy estimates derived from 32% (6/19) of the studies revealed a pooled mean accuracy of 0.856 (95% CI 0.70-0.93). Subgroup analyses demonstrated that the accuracy of wearable AI was moderated by the number of stress classes (P=.02), type of wearable device (P=.049), location of the wearable device (P=.02), data set size (P=.009), and ground truth (P=.001). The average estimates of sensitivity, specificity, and F1-score were 0.755 (SD 0.181), 0.744 (SD 0.147), and 0.759 (SD 0.139), respectively.

CONCLUSIONS

Wearable AI shows promise in detecting student stress but currently has suboptimal performance. The results of the subgroup analyses should be carefully interpreted given that many of these findings may be due to other confounding factors rather than the underlying grouping characteristics. Thus, wearable AI should be used alongside other assessments (eg, clinical questionnaires) until further evidence is available. Future research should explore the ability of wearable AI to differentiate types of stress, distinguish stress from other mental health issues, predict future occurrences of stress, consider factors such as the placement of the wearable device and the methods used to assess the ground truth, and report detailed results to facilitate the conduct of meta-analyses.

TRIAL REGISTRATION

PROSPERO CRD42023435051; http://tinyurl.com/3fzb5rnp.

Global Access to Comprehensive Care for Paediatric and Congenital Heart Disease

Paediatric and congenital heart disease (PCHD) is common but remains forgotten on the global health agenda. Congenital heart disease is the most frequent major congenital anomaly, affecting approximately 1 in every 100 live births. In high-income countries, most children now live into adulthood, whereas in low- and middle-income countries, over 90% of patients do not get the care they need. Rheumatic heart disease is the most common acquired cardiovascular disease in children and adolescents. While almost completely eradicated in high-income countries, over 30-40 million people live with rheumatic heart disease in low- and middle-income countries. Challenges exist in the care for PCHD and, increasingly, adult congenital heart disease (ACHD) worldwide. In this review, we summarize the current status of PCHD and ACHD care through the health systems lens of workforce, infrastructure, financing, service delivery, information management and technology, and governance. We further highlight gaps in knowledge and opportunities moving forward to improve access to care for all those living with PCHD or ACHD worldwide.

Fetal electrocardiography and artificial intelligence for prenatal detection of congenital heart disease

INTRODUCTION

This study aims to investigate non-invasive electrocardiography as a method for the detection of congenital heart disease (CHD) with the help of artificial intelligence.

MATERIAL AND METHODS

An artificial neural network was trained for the identification of CHD using non-invasively obtained fetal electrocardiograms. With the help of a Bayesian updating rule, multiple electrocardiographs were used to increase the algorithm's performance.

RESULTS

Using 122 measurements containing 65 healthy and 57 CHD cases, the accuracy, sensitivity, and specificity were found to be 71%, 63%, and 77%, respectively. The sensitivity was however 75% and 69% for CHD cases requiring an intervention in the neonatal period and first year of life, respectively. Furthermore, a positive effect of measurement length on the detection performance was observed, reaching optimal performance when using 14 electrocardiography segments (37.5 min) or more. A small negative trend between gestational age and accuracy was found.

CONCLUSIONS

The proposed method combining recent advances in obtaining non-invasive fetal electrocardiography with artificial intelligence for the automatic detection of CHD achieved a detection rate of 63% for all CHD and 75% for critical CHD. This feasibility study shows that detection rates of CHD might improve by using electrocardiography-based screening complementary to the standard ultrasound-based screening. More research is required to improve performance and determine the benefits to clinical practice.

The Role of Artificial Intelligence in Prediction, Risk Stratification, and Personalized Treatment Planning for Congenital Heart Diseases

This narrative review delves into the potential of artificial intelligence (AI) in predicting, stratifying risk, and personalizing treatment planning for congenital heart disease (CHD). CHD is a complex condition that affects individuals across various age groups. The review highlights the challenges in predicting risks, planning treatments, and prognosticating long-term outcomes due to CHD's multifaceted nature, limited data, ethical concerns, and individual variabilities. AI, with its ability to analyze extensive data sets, presents a promising solution. The review emphasizes the need for larger, diverse datasets, the integration of various data sources, and the analysis of longitudinal data. Prospective validation in real-world clinical settings, interpretability, and the importance of human clinical expertise are also underscored. The ethical considerations surrounding privacy, consent, bias, monitoring, and human oversight are examined. AI's implications include improved patient outcomes, cost-effectiveness, and real-time decision support. The review aims to provide a comprehensive understanding of AI's potential for revolutionizing CHD management and highlights the significance of collaboration and transparency to address challenges and limitations.

Identifying the Factors Affecting the Incidence of Congenital Heart Disease Using Support Vector Machine and Particle Swarm Optimization

Background

Congenital malformations are defined as "any defect in the structure of a person that exists from birth". Among them, congenital heart malformations have the highest prevalence in the world. This study focuses on the development of a predictive model for congenital heart disease in Isfahan using support vector machine (SVM) and particle swarm intelligence.

Materials and Methods

It consists of four parts: data collection, preprocessing, identify target features, and technique. The proposed technique is a combination of the SVM method and particle swarm optimization (PSO).

Results

The data set includes 1389 patients and 399 features. The best performance in terms of accuracy, with 81.57%, is related to the PSO-SVM technique and the worst performance, with 78.62%, is related to the random forest technique. Congenital extra cardiac anomalies are considered as the most important factor with averages of 0.655.

Conclusion

Congenital extra cardiac anomalies are considered as the most important factor. Detecting more important feature affecting congenital heart disease allows physicians to treat the variable risk factors associated with congenital heart disease progression. The use of a machine learning approach provides the ability to predict the presence of congenital heart disease with high accuracy and sensitivity.

Diagnostic accuracy of artificial intelligence for detecting gastrointestinal luminal pathologies: A systematic review and meta-analysis

Background

Artificial Intelligence (AI) holds considerable promise for diagnostics in the field of gastroenterology. This systematic review and meta-analysis aims to assess the diagnostic accuracy of AI models compared with the gold standard of experts and histopathology for the diagnosis of various gastrointestinal (GI) luminal pathologies including polyps, neoplasms, and inflammatory bowel disease.

Methods

We searched PubMed, CINAHL, Wiley Cochrane Library, and Web of Science electronic databases to identify studies assessing the diagnostic performance of AI models for GI luminal pathologies. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. We performed a meta-analysis and hierarchical summary receiver operating characteristic curves (HSROC). The risk of bias was assessed using Quality Assessment for Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Subgroup analyses were conducted based on the type of GI luminal disease, AI model, reference standard, and type of data used for analysis. This study is registered with PROSPERO (CRD42021288360).

Findings

We included 73 studies, of which 31 were externally validated and provided sufficient information for inclusion in the meta-analysis. The overall sensitivity of AI for detecting GI luminal pathologies was 91.9% (95% CI: 89.0–94.1) and specificity was 91.7% (95% CI: 87.4–94.7). Deep learning models (sensitivity: 89.8%, specificity: 91.9%) and ensemble methods (sensitivity: 95.4%, specificity: 90.9%) were the most commonly used models in the included studies. Majority of studies (n = 56, 76.7%) had a high risk of selection bias while 74% (n = 54) studies were low risk on reference standard and 67% (n = 49) were low risk for flow and timing bias.

Interpretation

The review suggests high sensitivity and specificity of AI models for the detection of GI luminal pathologies. There is a need for large, multi-center trials in both high income countries and low- and middle- income countries to assess the performance of these AI models in real clinical settings and its impact on diagnosis and prognosis.

Systematic review registration

[https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=288360], identifier [CRD42021288360].

Using Innovative Machine Learning Methods to Screen and Identify Predictors of Congenital Heart Diseases

Objective: Congenital heart diseases (CHDs) are associated with an extremely heavy global disease burden as the most common category of birth defects. Genetic and environmental factors have been identified as risk factors of CHDs previously. However, high volume clinical indicators have never been considered when predicting CHDs. This study aimed to predict the occurrence of CHDs by considering thousands of variables from self-reported questionnaires and routinely collected clinical laboratory data using machine learning algorithms.

Methods: We conducted a birth cohort study at one of the largest cardiac centers in China from 2011 to 2017. All fetuses were screened for CHDs using ultrasound and cases were confirmed by at least two pediatric cardiologists using echocardiogram. A total of 1,127 potential predictors were included to predict CHDs. We used the Explainable Boosting Machine (EBM) for prediction and evaluated the model performance using area under the Receive Operating Characteristics (ROC) curves (AUC). The top predictors were selected according to their contributions and predictive values. Thresholds were calculated for the most significant predictors.

Results: Overall, 5,390 mother-child pairs were recruited. Our prediction model achieved an AUC of 76% (69-83%) from out-of-sample predictions. Among the top 35 predictors of CHDs we identified, 34 were from clinical laboratory tests and only one was from the questionnaire (abortion history). Total accuracy, sensitivity, and specificity were 0.65, 0.74, and 0.65, respectively. Maternal serum uric acid (UA), glucose, and coagulation levels were the most consistent and significant predictors of CHDs. According to the thresholds of the predictors identified in our study, which did not reach the current clinical diagnosis criteria, elevated UA (>4.38 mg/dl), shortened activated partial thromboplastin time (<33.33 s), and elevated glucose levels were the most important predictors and were associated with ranges of 1.17-1.54 relative risks of CHDs. We have developed an online predictive tool for CHDs based on our findings that may help screening and prevention of CHDs.

Conclusions: Maternal UA, glucose, and coagulation levels were the most consistent and significant predictors of CHDs. Thresholds below the current clinical definition of “abnormal” for these predictors could be used to help develop CHD screening and prevention strategies.