Fetal Congenital Heart Disease Echocardiogram Screening Based on DGACNN: Adversarial One-Class Classification Combined with Video Transfer Learning

A comprehensive scoping review on machine learning-based fetal echocardiography analysis

Fetal echocardiography (ultrasound of the fetal heart) plays a vital role in identifying heart defects, allowing clinicians to establish prenatal and postnatal management plans. Machine learning-based methods are emerging to support the automation of fetal echocardiographic analysis; this review presents the findings from a literature review in this area. Searches were queried at leading indexing platforms ACM, IEEE Xplore, PubMed, Scopus, and Web of Science, including papers published until July 2023. In total, 343 papers were found, where 48 papers were selected to compose the detailed review. The reviewed literature presents research on neural network-based methods to identify fetal heart anatomy in classification and segmentation modelling. The reviewed literature uses five categorical technical analysis terms: attention and saliency, coarse to fine, dilated convolution, generative adversarial networks, and spatio-temporal. This review offers a technical overview for those already working in the field and an introduction to those new to the topic.

FHD deep learning prognosis approach: Early detection of fetal heart disease (FHD) using ultrasonography image-based IROI combined multiresolution DCNN

Fetal Heart Disease (FHD) is the most prevalent root cause of infant demise which accounts for 21% of all congenital abnormalities, with most instances being catastrophic, thereby rendering the need for early prognosis. Ultrasonography is the forefront imaging modality for assessing fetal growth in four-chamber and blood vessel malformation. Clinically diagnosing the abnormality is time-consuming and requires the skill of a radiologist. In subsequent, numerous preceding research strategies ideal to meta-heuristic and deep learning's Faster Artificial Neural Network (FANN), Dense Recurrent Neural Network (DRNN), Mask-Regional Convolution Neural Network (M RCNN) and Enhanced Deep Learning-assisted CNN aid in the identification of FHD. However, the prediction models have encountered multiple challenges owing to imprecise hinders and irrelevant adhesion. Hence, we propose the automated hierarchical network-driven findings of FHD in four-chamber and blood vessels using ultrasonic 2D imaging which undergoes 3 consequential processes of Enhanced-Adaptive Median Filtering (EAMF) pre-process concerning noise variations i.e., test for SNR distortion and image enhancement i.e., visual quality, Intensified Region of Interest (IROI) segmentation for exploiting feature selection via spatial mask-labeling and Multiresolution Deep Convolutional Neural Network (MDCNN) classification in the detection of diseased pattern via confusion metrics (CM). The lesion findings of CM is determined using MATLAB R2023b with an overall substantial efficiency of 99.79% in both normal and abnormal conditions with a significant potential to assist cardiologists in the prognosis of FHD.

Accurately assessing congenital heart disease using artificial intelligence

Congenital heart disease (CHD) remains a significant global health challenge, particularly contributing to newborn mortality, with the highest rates observed in middle- and low-income countries due to limited healthcare resources. Machine learning (ML) presents a promising solution by developing predictive models that more accurately assess the risk of mortality associated with CHD. These ML-based models can help healthcare professionals identify high-risk infants and ensure timely and appropriate care. In addition, ML algorithms excel at detecting and analyzing complex patterns that can be overlooked by human clinicians, thereby enhancing diagnostic accuracy. Despite notable advancements, ongoing research continues to explore the full potential of ML in the identification of CHD. The proposed article provides a comprehensive analysis of the ML methods for the diagnosis of CHD in the last eight years. The study also describes different data sets available for CHD research, discussing their characteristics, collection methods, and relevance to ML applications. In addition, the article also evaluates the strengths and weaknesses of existing algorithms, offering a critical review of their performance and limitations. Finally, the article proposes several promising directions for future research, with the aim of further improving the efficacy of ML in the diagnosis and treatment of CHD.

SKGC: A General Semantic-Level Knowledge Guided Classification Framework for Fetal Congenital Heart Disease

Congenital heart disease (CHD) is the most common congenital disability affecting healthy development and growth, even resulting in pregnancy termination or fetal death. Recently, deep learning techniques have made remarkable progress to assist in diagnosing CHD. One very popular method is directly classifying fetal ultrasound images, recognized as abnormal and normal, which tends to focus more on global features and neglects semantic knowledge of anatomical structures. The other approach is segmentation-based diagnosis, which requires a large number of pixel-level annotation masks for training. However, the detailed pixel-level segmentation annotation is costly or even unavailable. Based on the above analysis, we propose SKGC, a universal framework to identify normal or abnormal four-chamber heart (4CH) images, guided by a few annotation masks, while improving accuracy remarkably. SKGC consists of a semantic-level knowledge extraction module (SKEM), a multi-knowledge fusion module (MFM), and a classification module (CM). SKEM is responsible for obtaining high-level semantic knowledge, serving as an abstract representation of the anatomical structures that obstetricians focus on. MFM is a lightweight but efficient module that fuses semantic-level knowledge with the original specific knowledge in ultrasound images. CM classifies the fused knowledge and can be replaced by any advanced classifier. Moreover, we design a new loss function that enhances the constraint between the foreground and background predictions, improving the quality of the semantic-level knowledge. Experimental results on the collected real-world NA-4CH and the publicly FEST datasets show that SKGC achieves impressive performance with the best accuracy of 99.68% and 95.40%, respectively. Notably, the accuracy improves from 74.68% to 88.14% using only 10 labeled masks.

FetalBrainAwareNet: Bridging GANs with anatomical insight for fetal ultrasound brain plane synthesis

Over the past decade, deep-learning (DL) algorithms have become a promising tool to aid clinicians in identifying fetal head standard planes (FHSPs) during ultrasound (US) examination. However, the adoption of these algorithms in clinical settings is still hindered by the lack of large annotated datasets. To overcome this barrier, we introduce FetalBrainAwareNet, an innovative framework designed to synthesize anatomically accurate images of FHSPs. FetalBrainAwareNet introduces a cutting-edge approach that utilizes class activation maps as a prior in its conditional adversarial training process. This approach fosters the presence of the specific anatomical landmarks in the synthesized images. Additionally, we investigate specialized regularization terms within the adversarial training loss function to control the morphology of the fetal skull and foster the differentiation between the standard planes, ensuring that the synthetic images faithfully represent real US scans in both structure and overall appearance. The versatility of our FetalBrainAwareNet framework is highlighted by its ability to generate high-quality images of three predominant FHSPs using a singular, integrated framework. Quantitative (Fréchet inception distance of 88.52) and qualitative (t-SNE) results suggest that our framework generates US images with greater variability compared to state-of-the-art methods. By using the synthetic images generated with our framework, we increase the accuracy of FHSP classifiers by 3.2% compared to training the same classifiers solely with real acquisitions. These achievements suggest that using our synthetic images to increase the training set could provide benefits to enhance the performance of DL algorithms for FHSPs classification that could be integrated in real clinical scenarios.

Artificial intelligence in fetal echocardiography: Recent advances and future prospects

In the past few decades, great progress has been made in prenatal diagnosis of congenital heart disease (CHD). Fetal echocardiography is recognized as the main prenatal screening and diagnostic tool that can accurately detect approximately 85 % of fetal cardiac abnormalities. Evaluation of the fetal heart remains a major challenge in prenatal ultrasound screening and diagnosis due to fetal position, involuntary movement, small and complex fetal cardiac anatomy, maternal abdominal wall conditions, and lack of expertise in fetal echocardiography by some physicians engaged in obstetric ultrasound. Artificial intelligence (AI) can automate and standardize the display of each diagnostic section of the fetal heart and thus contribute to accurate diagnosis, which significantly optimizes the clinical application of fetal echocardiography. In this review, we not only clarify the role of AI but also highlight its significance and future solutions in the field of fetal echocardiography.

SPReCHD: Four-Chamber Semantic Parsing Network for Recognizing Fetal Congenital Heart Disease in Medical Metaverse

Echocardiography is essential for evaluating cardiac anatomy and function during early recognition and screening for congenital heart disease (CHD), a widespread and complex congenital malformation. However, fetal CHD recognition still faces many difficulties due to instinctive fetal movements, artifacts in ultrasound images, and distinctive fetal cardiac structures. These factors hinder capturing robust and discriminative representations from ultrasound images, resulting in CHD's low prenatal detection rate. Hence, we propose a multi-scale gated axial-transformer network (MSGATNet) to capture fetal four-chamber semantic information. Then, we propose a SPReCHD: four-chamber semantic parsing network for recognizing fetal CHD in the clinical treatment of the medical metaverse, integrating MSGATNet to segment and locate four-chamber arbitrary contours, further capturing distinguished representations for the fetal heart. Comprehensive experiments indicate that our SPReCHD is sufficient in recognizing fetal CHD, achieving a precision of 95.92%, a recall of 94%, an accuracy of 95%, and a F1 score of 94.95% on the test set, dramatically improving the fetal CHD's prenatal detection rate.

Investigation on ultrasound images for detection of fetal congenital heart defects

Congenital heart defects (CHD) are one of the serious problems that arise during pregnancy. Early CHD detection reduces death rates and morbidity but is hampered by the relatively low detection rates (i.e., 60%) of current screening technology. The detection rate could be increased by supplementing ultrasound imaging with fetal ultrasound image evaluation (FUSI) using deep learning techniques. As a result, the non-invasive foetal ultrasound image has clear potential in the diagnosis of CHD and should be considered in addition to foetal echocardiography. This review paper highlights cutting-edge technologies for detecting CHD using ultrasound images, which involve pre-processing, localization, segmentation, and classification. Existing technique of preprocessing includes spatial domain filter, non-linear mean filter, transform domain filter, and denoising methods based on Convolutional Neural Network (CNN); segmentation includes thresholding-based techniques, region growing-based techniques, edge detection techniques, Artificial Neural Network (ANN) based segmentation methods, non-deep learning approaches and deep learning approaches. The paper also suggests future research directions for improving current methodologies.

A Theoretical Exploration of Artificial Intelligence's Impact on Feto-Maternal Health from Conception to Delivery

The implementation of Artificial Intelligence (AI) in healthcare is enhancing diagnostic accuracy in clinical setups. The use of AI in healthcare is steadily increasing with advancing technology, extending beyond disease diagnosis to encompass roles in feto-maternal health. AI harnesses Machine Learning (ML), Natural Language Processing (NLP), Artificial Neural Networks (ANN), and computer vision to analyze data and draw conclusions. Considering maternal health, ML analyzes vast datasets to predict maternal and fetal health outcomes, while NLP interprets medical texts and patient records to assist in diagnosis and treatment decisions. ANN models identify patterns in complex feto-maternal medical data, aiding in risk assessment and intervention planning whereas, computer vision enables the analysis of medical images for early detection of feto-maternal complications. AI facilitates early pregnancy detection, genetic screening, and continuous monitoring of maternal health parameters, providing real-time alerts for deviations, while also playing a crucial role in the early detection of fetal abnormalities through enhanced ultrasound imaging, contributing to informed decision-making. This review investigates into the application of AI, particularly through predictive models, in addressing the monitoring of feto-maternal health. Additionally, it examines potential future directions and challenges associated with these applications.

Ultrasound imaging based recognition of prenatal anomalies: a systematic clinical engineering review

For prenatal screening, ultrasound (US) imaging allows for real-time observation of developing fetal anatomy. Understanding normal and aberrant forms through extensive fetal structural assessment enables for early detection and intervention. However, the reliability of anomaly diagnosis varies depending on operator expertise and device limits. First trimester scans in conjunction with circulating biochemical markers are critical in identifying high-risk pregnancies, but they also pose technical challenges. Recent engineering advancements in automated diagnosis, such as artificial intelligence (AI)-based US image processing and multimodal data fusion, are developing to improve screening efficiency, accuracy, and consistency. Still, creating trust in these data-driven solutions is necessary for integration and acceptability in clinical settings. Transparency can be promoted by explainable AI (XAI) technologies that provide visual interpretations and illustrate the underlying diagnostic decision making process. An explanatory framework based on deep learning is suggested to construct charts depicting anomaly screening results from US video feeds. AI modelling can then be applied to these charts to connect defects with probable deformations. Overall, engineering approaches that increase imaging, automation, and interpretability hold enormous promise for altering traditional workflows and expanding diagnostic capabilities for better prenatal care.

HFSCCD: A Hybrid Neural Network for Fetal Standard Cardiac Cycle Detection in Ultrasound Videos

In the fetal cardiac ultrasound examination, standard cardiac cycle (SCC) recognition is the essential foundation for diagnosing congenital heart disease. Previous studies have mostly focused on the detection of adult CCs, which may not be applicable to the fetus. In clinical practice, localization of SCCs needs to recognize end-systole (ES) and end-diastole (ED) frames accurately, ensuring that every frame in the cycle is a standard view. Most existing methods are not based on the detection of key anatomical structures, which may not recognize irrelevant views and background frames, results containing non-standard frames, or even it does not work in clinical practice. We propose an end-to-end hybrid neural network based on an object detector to detect SCCs from fetal ultrasound videos efficiently, which consists of 3 modules, namely Anatomical Structure Detection (ASD), Cardiac Cycle Localization (CCL), and Standard Plane Recognition (SPR). Specifically, ASD uses an object detector to identify 9 key anatomical structures, 3 cardiac motion phases, and the corresponding confidence scores from fetal ultrasound videos. On this basis, we propose a joint probability method in the CCL to learn the cardiac motion cycle based on the 3 cardiac motion phases. In SPR, to reduce the impact of structure detection errors on the accuracy of the standard plane recognition, we use XGBoost algorithm to learn the relation knowledge of the detected anatomical structures. We evaluate our method on the test fetal ultrasound video datasets and clinical examination cases and achieve remarkable results. This study may pave the way for clinical practices.

Advances in the Application of Artificial Intelligence in Fetal Echocardiography

Congenital heart disease is a severe health risk for newborns. Early detection of abnormalities in fetal cardiac structure and function during pregnancy can help patients seek timely diagnostic and therapeutic advice, and early intervention planning can significantly improve fetal survival rates. Echocardiography is one of the most accessible and widely used diagnostic tools in the diagnosis of fetal congenital heart disease. However, traditional fetal echocardiography has limitations due to fetal, maternal, and ultrasound equipment factors and is highly dependent on the skill level of the operator. Artificial intelligence (AI) technology, with its rapid development utilizing advanced computer algorithms, has great potential to empower sonographers in time-saving and accurate diagnosis and to bridge the skill gap in different regions. In recent years, AI-assisted fetal echocardiography has been successfully applied to a wide range of ultrasound diagnoses. This review systematically reviews the applications of AI in the field of fetal echocardiography over the years in terms of image processing, biometrics, and disease diagnosis and provides an outlook for future research.

Applications of artificial intelligence-powered prenatal diagnosis for congenital heart disease

Artificial intelligence (AI) has made significant progress in the medical field in the last decade. The AI-powered analysis methods of medical images and clinical records can now match the abilities of clinical physicians. Due to the challenges posed by the unique group of fetuses and the dynamic organ of the heart, research into the application of AI in the prenatal diagnosis of congenital heart disease (CHD) is particularly active. In this review, we discuss the clinical questions and research methods involved in using AI to address prenatal diagnosis of CHD, including imaging, genetic diagnosis, and risk prediction. Representative examples are provided for each method discussed. Finally, we discuss the current limitations of AI in prenatal diagnosis of CHD, namely Volatility, Insufficiency and Independence (VII), and propose possible solutions.

The Lifelong Impact of Artificial Intelligence and Clinical Prediction Models on Patients With Tetralogy of Fallot

Medical advancements in the diagnosis, surgical techniques, perioperative care, and continued care throughout childhood have transformed the outlook for individuals with tetralogy of Fallot (TOF), improving survival and shifting the perspective towards lifelong care. However, with a growing population of survivors, longstanding challenges have been accentuated, and new challenges have surfaced, necessitating a re-evaluation of TOF care. Availability of prenatal diagnostics, insufficient information from traditional imaging techniques, previously unforeseen medical complications, and debates surrounding optimal timing and indications for reintervention are among the emerging issues. To address these challenges, the integration of artificial intelligence and machine learning holds great promise as they have the potential to revolutionize patient management and positively impact lifelong outcomes for individuals with TOF. Innovative applications of artificial intelligence and machine learning have spanned across multiple domains of TOF care, including screening and diagnosis, automated image processing and interpretation, clinical risk stratification, and planning and performing cardiac interventions. By embracing these advancements and incorporating them into routine clinical practice, personalized medicine could be delivered, leading to the best possible outcomes for patients. In this review, we provide an overview of these evolving applications and emphasize the challenges, limitations, and future potential for integrating them into clinical care.

Enhancing Fetal Anomaly Detection in Ultrasonography Images: A Review of Machine Learning-Based Approaches

Fetal development is a critical phase in prenatal care, demanding the timely identification of anomalies in ultrasound images to safeguard the well-being of both the unborn child and the mother. Medical imaging has played a pivotal role in detecting fetal abnormalities and malformations. However, despite significant advances in ultrasound technology, the accurate identification of irregularities in prenatal images continues to pose considerable challenges, often necessitating substantial time and expertise from medical professionals. In this review, we go through recent developments in machine learning (ML) methods applied to fetal ultrasound images. Specifically, we focus on a range of ML algorithms employed in the context of fetal ultrasound, encompassing tasks such as image classification, object recognition, and segmentation. We highlight how these innovative approaches can enhance ultrasound-based fetal anomaly detection and provide insights for future research and clinical implementations. Furthermore, we emphasize the need for further research in this domain where future investigations can contribute to more effective ultrasound-based fetal anomaly detection.

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

Artificial intelligence (AI) has gained prominence in medical imaging, particularly in obstetrics and gynecology (OB/GYN), where ultrasound (US) is the preferred method. It is considered cost effective and easily accessible but is time consuming and hindered by the need for specialized training. To overcome these limitations, AI models have been proposed for automated plane acquisition, anatomical measurements, and pathology detection. This study aims to overview recent literature on AI applications in OB/GYN US imaging, highlighting their benefits and limitations. For the methodology, a systematic literature search was performed in the PubMed and Cochrane Library databases. Matching abstracts were screened based on the PICOS (Participants, Intervention or Exposure, Comparison, Outcome, Study type) scheme. Articles with full text copies were distributed to the sections of OB/GYN and their research topics. As a result, this review includes 189 articles published from 1994 to 2023. Among these, 148 focus on obstetrics and 41 on gynecology. AI-assisted US applications span fetal biometry, echocardiography, or neurosonography, as well as the identification of adnexal and breast masses, and assessment of the endometrium and pelvic floor. To conclude, the applications for AI-assisted US in OB/GYN are abundant, especially in the subspecialty of obstetrics. However, while most studies focus on common application fields such as fetal biometry, this review outlines emerging and still experimental fields to promote further research.

Automatic echocardiographic anomalies interpretation using a stacked residual-dense network model

Echocardiographic interpretation during the prenatal or postnatal period is important for diagnosing cardiac septal abnormalities. However, manual interpretation can be time consuming and subject to human error. Automatic segmentation of echocardiogram can support cardiologists in making an initial interpretation. However, such a process does not always provide straightforward information to make a complete interpretation. The segmentation process only identifies the region of cardiac septal abnormality, whereas complete interpretation should determine based on the position of defect. In this study, we proposed a stacked residual-dense network model to segment the entire region of cardiac and classifying their defect positions to generate automatic echocardiographic interpretation. We proposed the generalization model with incorporated two modalities: prenatal and postnatal echocardiography. To further evaluate the effectiveness of our model, its performance was verified by five cardiologists. We develop a pipeline process using 1345 echocardiograms for training data and 181 echocardiograms for unseen data from prospective patients acquired during standard clinical practice at Muhammad Hoesin General Hospital in Indonesia. As a result, the proposed model produced of 58.17% intersection over union (IoU), 75.75% dice similarity coefficient (DSC), and 76.36% mean average precision (mAP) for the validation data. Using unseen data, we achieved 42.39% IoU, 55.72% DSC, and 51.04% mAP. Further, the classification of defect positions using unseen data had approximately 92.27% accuracy, 94.33% specificity, and 92.05% sensitivity. Finally, our proposed model is validated with human expert with varying Kappa value. On average, these results hold promise of increasing suitability in clinical practice as a supporting diagnostic tool for establishing the diagnosis.

Coronary heart disease prediction method fusing domain-adaptive transfer learning with graph convolutional networks (GCN)

Graph convolutional networks (GCNs) have achieved impressive results in many medical scenarios involving graph node classification tasks. However, there are difficulties in transfer learning for graph representation learning and graph network models. Most GNNs work only in a single domain and cannot transfer the learned knowledge to other domains. Coronary Heart Disease (CHD) is a high-mortality disease, and there are non-public and significant differences in CHD datasets for current research, which makes it difficult to perform unified transfer learning. Therefore, in this paper, we propose a novel adversarial domain-adaptive multichannel graph convolutional network (DAMGCN) that can perform graph transfer learning on cross-domain tasks to achieve cross-domain medical knowledge transfer on different CHD datasets. First, we use a two-channel GCN model for feature aggregation using local consistency and global consistency. Then, a uniform node representation is generated for different graphs using an attention mechanism. Finally, we provide a domain adversarial module to decrease the discrepancies between the source and target domain classifiers and optimize the three loss functions in order to accomplish source and target domain knowledge transfer. The experimental findings demonstrate that our model performs best on three CHD datasets, and its performance is greatly enhanced by graph transfer learning.

A multicenter study on two-stage transfer learning model for duct-dependent CHDs screening in fetal echocardiography

Duct-dependent congenital heart diseases (CHDs) are a serious form of CHD with a low detection rate, especially in underdeveloped countries and areas. Although existing studies have developed models for fetal heart structure identification, there is a lack of comprehensive evaluation of the long axis of the aorta. In this study, a total of 6698 images and 48 videos are collected to develop and test a two-stage deep transfer learning model named DDCHD-DenseNet for screening critical duct-dependent CHDs. The model achieves a sensitivity of 0.973, 0.843, 0.769, and 0.759, and a specificity of 0.985, 0.967, 0.956, and 0.759, respectively, on the four multicenter test sets. It is expected to be employed as a potential automatic screening tool for hierarchical care and computer-aided diagnosis. Our two-stage strategy effectively improves the robustness of the model and can be extended to screen for other fetal heart development defects.

Use of artificial intelligence and deep learning in fetal ultrasound imaging

Deep learning is considered the leading artificial intelligence tool in image analysis in general. Deep-learning algorithms excel at image recognition, which makes them valuable in medical imaging. Obstetric ultrasound has become the gold standard imaging modality for detection and diagnosis of fetal malformations. However, ultrasound relies heavily on the operator's experience, making it unreliable in inexperienced hands. Several studies have proposed the use of deep-learning models as a tool to support sonographers, in an attempt to overcome these problems inherent to ultrasound. Deep learning has many clinical applications in the field of fetal imaging, including identification of normal and abnormal fetal anatomy and measurement of fetal biometry. In this Review, we provide a comprehensive explanation of the fundamentals of deep learning in fetal imaging, with particular focus on its clinical applicability. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

Current status and challenges in prenatal and neonatal screening, diagnosis, and management of congenital heart disease in China

Congenital heart disease (CHD), a wide spectrum of diseases with varied outcomes, is the most common congenital malformation worldwide. In this Series of three papers, we describe the burden of CHD in China; the development of screening, diagnosis, treatment, and follow-up strategies; and challenges associated with the disease. We also propose solutions and recommendations for policies and actions to improve the outcomes of CHD. In the first paper in this Series, we focus on prenatal and neonatal screening, diagnosis, and management of CHD. Based on advanced international knowledge, the Chinese Government has developed a network system comprising prenatal screening, diagnosis of CHD subtypes, specialist consultation appointments, and treatment centres for CHD. A new professional discipline, fetal cardiology, has been formed and rapidly developed. Consequently, the overall coverage of prenatal and neonatal screening and the accuracy of CHD diagnoses have gradually improved, and the neonatal CHD mortality rate has decreased substantially. However, China still faces several challenges in the prevention and treatment of CHD, such as insufficient diagnostic capabilities and unqualified consultation services in some regions and rural areas. TRANSLATION: For the Chinese translation of the abstract see Supplementary Materials section.

Improved Multi-Head Self-Attention Classification Network for Multi-View Fetal Echocardiography Recognition

The accurate acquisition of multiview fetal cardiac ultrasound images is very important for the diagnosis of fetal congenital heart disease (FCHD). However, these manual clinical procedures have drawbacks, e.g., varying technical capabilities and inefficiency. Therefore, exploring automatic recognition method for multiview images of fetal heart ultrasound scans is highly desirable to improve prenatal diagnosis efficiency and accuracy. In this work, we propose an improved multi-head self-attention mechanism called IMSA combined with residual networks to stably solve the problem of multiview identification and anatomical structure localization. In details, IMSA can capture short- and long-range dependencies from different subspaces and merge them to extract more precise features, thus making use of the correlation between fetal heart structures to make view recognition more focused on anatomical structures rather than disturbing regions, such as artifacts and speckle noises. We validate our proposed method on fetal cardiac ultrasound imaging datasets from a single center and 38 multicenter studies and the results outperform other state-of-the-art networks by 3%-15% of F1 scores in fetal heart six standard view recognition.Clinical Relevance- This technology has great potential in assisting cardiologists to complete the automatic acquisition of multi-section fetal echocardiography images.

A deep learning-based method for pediatric congenital heart disease detection with seven standard views in echocardiography

Background

With the aggregation of clinical data and the evolution of computational resources, artificial intelligence-based methods have become possible to facilitate clinical diagnosis. For congenital heart disease (CHD) detection, recent deep learning-based methods tend to achieve classification with few views or even a single view. Due to the complexity of CHD, the input images for the deep learning model should cover as many anatomical structures of the heart as possible to enhance the accuracy and robustness of the algorithm. In this paper, we first propose a deep learning method based on seven views for CHD classification and then validate it with clinical data, the results of which show the competitiveness of our approach.

Methods

A total of 1411 children admitted to the Children's Hospital of Zhejiang University School of Medicine were selected, and their echocardiographic videos were obtained. Then, seven standard views were selected from each video, which were used as the input to the deep learning model to obtain the final result after training, validation and testing.

Results

In the test set, when a reasonable type of image was input, the area under the curve (AUC) value could reach 0.91, and the accuracy could reach 92.3%. During the experiment, shear transformation was used as interference to test the infection resistance of our method. As long as appropriate data were input, the above experimental results would not fluctuate obviously even if artificial interference was applied.

Conclusions

These results indicate that the deep learning model based on the seven standard echocardiographic views can effectively detect CHD in children, and this approach has considerable value in practical application.

Application and Progress of Artificial Intelligence in Fetal Ultrasound

Prenatal ultrasonography is the most crucial imaging modality during pregnancy. However, problems such as high fetal mobility, excessive maternal abdominal wall thickness, and inter-observer variability limit the development of traditional ultrasound in clinical applications. The combination of artificial intelligence (AI) and obstetric ultrasound may help optimize fetal ultrasound examination by shortening the examination time, reducing the physician's workload, and improving diagnostic accuracy. AI has been successfully applied to automatic fetal ultrasound standard plane detection, biometric parameter measurement, and disease diagnosis to facilitate conventional imaging approaches. In this review, we attempt to thoroughly review the applications and advantages of AI in prenatal fetal ultrasound and discuss the challenges and promises of this new field.

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

Data augmentation for medical imaging: A systematic literature review

Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.

A review on deep-learning algorithms for fetal ultrasound-image analysis

Deep-learning (DL) algorithms are becoming the standard for processing ultrasound (US) fetal images. A number of survey papers in the field is today available, but most of them are focusing on a broader area of medical-image analysis or not covering all fetal US DL applications. This paper surveys the most recent work in the field, with a total of 153 research papers published after 2017. Papers are analyzed and commented from both the methodology and the application perspective. We categorized the papers into (i) fetal standard-plane detection, (ii) anatomical structure analysis and (iii) biometry parameter estimation. For each category, main limitations and open issues are presented. Summary tables are included to facilitate the comparison among the different approaches. In addition, emerging applications are also outlined. Publicly-available datasets and performance metrics commonly used to assess algorithm performance are summarized, too. This paper ends with a critical summary of the current state of the art on DL algorithms for fetal US image analysis and a discussion on current challenges that have to be tackled by researchers working in the field to translate the research methodology into actual clinical practice.

Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases

Early prenatal screening with an ultrasound (US) can significantly lower newborn mortality caused by congenital heart diseases (CHDs). However, the need for expertise in fetal cardiologists and the high volume of screening cases limit the practically achievable detection rates. Hence, automated prenatal screening to support clinicians is desirable. This paper presents and analyses potential deep learning (DL) techniques to diagnose CHDs in fetal USs. Four convolutional neural network architectures were compared to select the best classifier with satisfactory results. Hence, dense convolutional network (DenseNet) 201 architecture was selected for the classification of seven CHDs, such as ventricular septal defect, atrial septal defect, atrioventricular septal defect, Ebstein's anomaly, tetralogy of Fallot, transposition of great arteries, hypoplastic left heart syndrome, and a normal control. The sensitivity, specificity, and accuracy of the DenseNet201 model were 100%, 100%, and 100%, respectively, for the intra-patient scenario and 99%, 97%, and 98%, respectively, for the inter-patient scenario. We used the intra-patient DL prediction model to validate our proposed model against the prediction results of three expert fetal cardiologists. The proposed model produces a satisfactory result, which means that our model can support expert fetal cardiologists to interpret the decision to improve CHD diagnostics. This work represents a step toward the goal of assisting front-line sonographers with CHD diagnoses at the population level.

Tunicate swarm-based grey wolf algorithm for fetal heart chamber segmentation and classification: a heuristic-based optimal feature selection concept

Ultrasound image quality management and assessment are an important stage in clinical diagnosis. This operation is often carried out manually, which has several issues, including reliance on the operator’s experience, lengthy labor, and considerable intra-observer variance. As a result, automatic quality evaluation of Ultrasound images is particularly desirable in medical applications. This research work plans to perform the fetal heart chamber segmentation and classification using the novel intelligent technology named as hybrid optimization algorithm Tunicate Swarm-based Grey Wolf Algorithm (TS-GWA). Initially, the US fetal images data is collected and data undergoes the preprocessing using the total variation technique. From the preprocessed images, the optimal features are extracted using the TF-IDF approach. Then, Segmentation is processed on optimally selected features using Spatially Regularized Discriminative Correlation Filters (SRDCF) method. In the final step, the classification of fetal images is done using the Modified Long Short-Term Memory (MLSTM) Network. The fitness function behind the optimal feature selection as well as the hidden neuron optimization of MLSTM is the maximization of PSNR and minimization of MSE. The PSNR value is improved from 3.1 to 9.8 in the proposed method and accuracy of the proposed classification algorithm is improved from 1.9 to 12.13 compared to other existing techniques. The generalization ability and the adaptability of proposed TS-GWA method are described by conducting the various performance analysis. Extensive performance result shows that proposed intelligent techniques performs better than the existing segmentation methods.

Diagnosis of fetal total anomalous pulmonary venous connection based on the post-left atrium space ratio using artificial intelligence

OBJECTIVE

To explore whether the post-left atrium space (PLAS) ratio would be useful for prenatal diagnosis of total anomalous pulmonary venous connection (TAPVC) using echocardiography and artificial intelligence.

METHODS

We retrospectively included 642 frames of four-chamber view from 319 fetuses (32 with TAPVC and 287 without TAPVC) in end-systolic and end-diastolic periods with multiple apex directions. The average gestational age was 25.6±2.7 weeks. No other cardiac or extracardiac malformations were observed. The dataset was divided into a training set (n=540; 48 with TAPVC and 492 without TAPVC) and test set (n=102; 20 with TAPVC and 82 without TAPVC). The PLAS ratio was defined as the ratio of the epicardium-descending aortic distance to the center of the heart-descending aortic distance. Supervised learning was used in DeepLabv3+, FastFCN, PSPNet, and DenseASPP segmentation models. The area under the curve (AUC) was used on the test set.

RESULTS

Expert annotations showed that this ratio was not related to the period or apex direction. It was higher in the TAPVC group than in the control group detected by expert and the four models. The AUC of expert annotations, DeepLabv3+, FastFCN, PSPNet, and DenseASPP were 0.977, 0.941, 0.925, 0.856, and 0.887, respectively.

CONCLUSION

Segmentation models achieve good diagnostic accuracy for TAPVC based on the PLAS ratio. This article is protected by copyright. All rights reserved.

Towards deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via Ultrasound Images

Several reviews have been conducted regarding artificial intelligence (AI) techniques to improve pregnancy outcomes. But they are not focusing on ultrasound images. This survey aims to explore how AI can assist with fetal growth monitoring via ultrasound image. We reported our findings using the guidelines for PRISMA. We conducted a comprehensive search of eight bibliographic databases. Out of 1269 studies 107 are included. We found that 2D ultrasound images were more popular (88) than 3D and 4D ultrasound images (19). Classification is the most used method (42), followed by segmentation (31), classification integrated with segmentation (16) and other miscellaneous methods such as object-detection, regression, and reinforcement learning (18). The most common areas that gained traction within the pregnancy domain were the fetus head (43), fetus body (31), fetus heart (13), fetus abdomen (10), and the fetus face (10). This survey will promote the development of improved AI models for fetal clinical applications.

•

Artificial intelligence studies to monitor fetal development via ultrasound images

•

Fetal issues categorized based on four categories — general, head, heart, face, abdomen

•

The most used AI techniques are classification, segmentation, object detection, and RL

•

The research and practical implications are included.

Artificial intelligence in perinatal diagnosis and management of congenital heart disease

Prenatal diagnosis and management of congenital heart disease (CHD) has progressed substantially in the past few decades. Fetal echocardiography can accurately detect and diagnose approximately 85% of cardiac anomalies. The prenatal diagnosis of CHD results in improved care, with improved risk stratification, perioperative status and survival. However, there is much work to be done. A minority of CHD is actually identified prenatally. This seemingly incongruous gap is due, in part, to diminished recognition of an anomaly even when present in the images and the need for increased training to obtain specialized cardiac views. Artificial intelligence (AI) is a field within computer science that focuses on the development of algorithms that "learn, reason, and self-correct" in a human-like fashion. When applied to fetal echocardiography, AI has the potential to improve image acquisition, image optimization, automated measurements, identification of outliers, classification of diagnoses, and prediction of outcomes. Adoption of AI in the field has been thus far limited by a paucity of data, limited resources to implement new technologies, and legal and ethical concerns. Despite these barriers, recognition of the potential benefits will push us to a future in which AI will become a routine part of clinical practice.

Adoption of Compound Echocardiography under Artificial Intelligence Algorithm in Fetal Congenial Heart Disease Screening during Gestation

This research was aimed at exploring the diagnostic and screening effect of composite echocardiography based on the artificial intelligence (AI) segmentation algorithm on fetal congenital heart disease (CHD) during pregnancy, so as to reduce the birth rate of newborns with CHD. A total of 204 fetuses with abnormal heart conditions were divided into group II, group C (optimized with the AI algorithm), and group W (not optimized with the AI algorithm). In addition, 9,453 fetuses with normal heart conditions were included in group I. The abnormal distribution of fetal heart and the difference of cardiac Z score between group II and group I were analyzed, and the diagnostic value of group C and group W for CHD was compared. The results showed that the segmentation details of the proposed algorithm were better than those of the convolutional neural network (CNN), and the Dice coefficient, precision, and recall values were higher than those of the CNN. In fetal CHD, the incidence of abnormal ultrasonic manifestations was ventricular septal defect (98/48.04%), abnormal right subclavian artery (29/14.22%), and persistent left superior vena cava (25/12.25%). The diagnostic sensitivity (75.0% vs. 51.5%), specificity (99.6% vs. 99.2%), accuracy (99.0% vs. 98.2%), negative predictive value (88.5% vs. 78.5%), and positive predictive value (99% vs. 57.7%) of echocardiography segmentation in group C were significantly higher than those in group W. To sum up, echocardiography segmented by the AI algorithm could obviously improve the diagnostic efficiency of fetal CHD during gestation. Cardiac ultrasound parameters of children with CHD changed greatly.

Role of Four-Chamber Heart Ultrasound Images in Automatic Assessment of Fetal Heart: A Systematic Understanding

The fetal echocardiogram is useful for monitoring and diagnosing cardiovascular diseases in the fetus in utero. Importantly, it can be used for assessing prenatal congenital heart disease, for which timely intervention can improve the unborn child’s outcomes. In this regard, artificial intelligence (AI) can be used for the automatic analysis of fetal heart ultrasound images. This study reviews nondeep and deep learning approaches for assessing the fetal heart using standard four-chamber ultrasound images. The state-of-the-art techniques in the field are described and discussed. The compendium demonstrates the capability of automatic assessment of the fetal heart using AI technology. This work can serve as a resource for research in the field.

Use of real-time artificial intelligence in detection of abnormal image patterns in standard sonographic reference planes in screening for fetal intracranial malformations

OBJECTIVES

To develop and validate an artificial intelligence system, the Prenatal ultrasound diagnosis Artificial Intelligence Conduct System (PAICS), to detect different patterns of fetal intracranial abnormality in standard sonographic reference planes for screening for congenital central nervous system (CNS) malformations.

METHODS

Neurosonographic images from normal fetuses and fetuses with CNS malformations at 18-40 gestational weeks were retrieved from the databases of two tertiary hospitals in China and assigned randomly (ratio, 8:1:1) to training, fine-tuning and internal validation datasets to develop and evaluate the PAICS. The system was built based on a real-time convolutional neural network (CNN) algorithm, You Only Look Once, version 3 (YOLOv3). An image dataset from a third tertiary hospital was used to further validate, externally, the performance of the PAICS and to compare its performance with that of sonologists with different levels of expertise. Furthermore, a prospective video dataset was employed to evaluate the performance of the PAICS in a real-time scan scenario. The diagnostic accuracy, sensitivity, specificity and area under the receiver-operating-characteristics curve (AUC) were calculated to assess the performance of the PAICS and to compare this with the performance of sonologists with different levels of experience.

RESULTS

In total, 43 890 images from 16 297 pregnancies and 169 videos from 166 pregnancies were used to develop and validate the PAICS. The system achieved excellent performance in identifying 10 types of intracranial image pattern, with macro- and microaverage AUCs, respectively, of 0.933 (95% CI, 0.798-1.000) and 0.977 (95% CI, 0.970-0.985) for the internal validation image dataset, 0.902 (95% CI, 0.816-0.989) and 0.898 (95% CI, 0.885-0.911) for the external validation image dataset and 0.969 (95% CI, 0.886-1.000) and 0.981 (95% CI, 0.974-0.988) in the real-time scan setting. The performance of the PAICS was comparable to that of expert sonologists in terms of macro- and microaverage accuracy (P = 0.863 and P = 0.775, respectively), sensitivity (P = 0.883, P = 0.846) and AUC (P = 0.891, P = 0.788), but required significantly less time (0.025 s per image for PAICS vs 4.4 s for experts, P < 0.001).

CONCLUSIONS

Both in the image dataset and in the real-time scan setting, the PAICS achieved excellent diagnostic performance for various fetal CNS abnormalities. Its performance was comparable to that of experts, but it required less time. A CNN algorithm can be trained to detect fetal CNS abnormalities. The PAICS has the potential to be an effective and efficient tool in screening for fetal CNS malformations in clinical practice. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.

Artificial intelligence in pediatric cardiology: taking baby steps in the big world of data

PURPOSE OF REVIEW

Artificial intelligence (AI) has changed virtually every aspect of modern life, and medicine is no exception. Pediatric cardiology is both a perceptual and a cognitive subspecialty that involves complex decision-making, so AI is a particularly attractive tool for this medical discipline. This review summarizes the foundational work and incremental progress made as AI applications have emerged in pediatric cardiology since 2020.

RECENT FINDINGS

AI-based algorithms can be useful for pediatric cardiology in many areas, including: (1) clinical examination and diagnosis, (2) image processing, (3) planning and management of cardiac interventions, (4) prognosis and risk stratification, (5) omics and precision medicine, and (6) fetal cardiology. Most AI initiatives showcased in medical journals seem to work well in silico, but progress toward implementation in actual clinical practice has been more limited. Several barriers to implementation are identified, some encountered throughout medicine generally, and others specific to pediatric cardiology.

SUMMARY

Despite barriers to acceptance in clinical practice, AI is already establishing a durable role in pediatric cardiology. Its potential remains great, but to fully realize its benefits, substantial investment to develop and refine AI for pediatric cardiology applications will be necessary to overcome the challenges of implementation.

A Novel Intelligent Fault Diagnosis Method for Rolling Bearings Based on Wasserstein Generative Adversarial Network and Convolutional Neural Network under Unbalanced Dataset

Rolling bearings are widely used in industrial manufacturing, and ensuring their stable and effective fault detection is a core requirement in the manufacturing process. However, it is a great challenge to achieve a highly accurate rolling bearing fault diagnosis because of the severe imbalance and distribution differences in fault data due to weak early fault features and interference from environmental noise. An intelligent fault diagnosis strategy for rolling bearings based on grayscale image transformation, a generative adversative network, and a convolutional neural network was proposed to solve this problem. First, the original vibration signal is converted into a grayscale image. Then more training samples are generated using GANs to solve severe imbalance and distribution differences in fault data. Finally, the rolling bearing condition detection and fault identification are carried out by using SECNN. The availability of the method is substantiated by experiments on datasets with different data imbalance ratios. In addition, the superiority of this diagnosis strategy is verified by comparing it with other mainstream intelligent diagnosis techniques. The experimental result demonstrates that this strategy can reach more than 99.6% recognition accuracy even under substantial environmental noise interference or changing working conditions and has good stability in the presence of a severe imbalance in fault data.

A Progressive Generative Adversarial Method for Structurally Inadequate Medical Image Data Augmentation

The generation-based data augmentation method can overcome the challenge caused by the imbalance of medical image data to a certain extent. However, most of the current research focus on images with unified structure which are easy to learn. What is different is that ultrasound images are structurally inadequate, making it difficult for the structure to be captured by the generative network, resulting in the generated image lacks structural legitimacy. Therefore, a Progressive Generative Adversarial Method for Structurally Inadequate Medical Image Data Augmentation is proposed in this paper, including a network and a strategy. Our Progressive Texture Generative Adversarial Network alleviates the adverse effect of completely truncating the reconstruction of structure and texture during the generation process and enhances the implicit association between structure and texture. The Image Data Augmentation Strategy based on Mask-Reconstruction overcomes data imbalance from a novel perspective, maintains the legitimacy of the structure in the generated data, as well as increases the diversity of disease data interpretably. The experiments prove the effectiveness of our method on data augmentation and image reconstruction on Structurally Inadequate Medical Image both qualitatively and quantitatively. Finally, the weakly supervised segmentation of the lesion is the additional contribution of our method.