Clinically Feasible and Accurate View Classification of Echocardiographic Images Using Deep Learning

Deep learning supported echocardiogram analysis: A comprehensive review

An echocardiogram is a sophisticated ultrasound imaging technique employed to diagnose heart conditions. The transthoracic echocardiogram, one of the most prevalent types, is instrumental in evaluating significant cardiac diseases. However, interpreting its results heavily relies on the clinician's expertise. In this context, artificial intelligence has emerged as a vital tool for helping clinicians. This study critically analyzes key state-of-the-art research that uses deep learning techniques to automate transthoracic echocardiogram analysis and support clinical judgments. We have systematically organized and categorized articles that proffer solutions for view classification, enhancement of image quality and dataset, segmentation and identification of cardiac structures, detection of cardiac function abnormalities, and quantification of cardiac functions. We compared the performance of various deep learning approaches within each category, identifying the most promising methods. Additionally, we highlight limitations in current research and explore promising avenues for future exploration. These include addressing generalizability issues, incorporating novel AI approaches, and tackling the analysis of rare cardiac diseases.

Artificial intelligence-based classification of echocardiographic views

Aims

Augmenting echocardiography with artificial intelligence would allow for automated assessment of routine parameters and identification of disease patterns not easily recognized otherwise. View classification is an essential first step before deep learning can be applied to the echocardiogram.

Methods and results

We trained two- and three-dimensional convolutional neural networks (CNNs) using transthoracic echocardiographic (TTE) studies obtained from 909 patients to classify nine view categories (10 269 videos). Transthoracic echocardiographic studies from 229 patients were used in internal validation (2582 videos). Convolutional neural networks were tested on 100 patients with comprehensive TTE studies (where the two examples chosen by CNNs as most likely to represent a view were evaluated) and 408 patients with five view categories obtained via point-of-care ultrasound (POCUS). The overall accuracy of the two-dimensional CNN was 96.8%, and the averaged area under the curve (AUC) was 0.997 on the comprehensive TTE testing set; these numbers were 98.4% and 0.998, respectively, on the POCUS set. For the three-dimensional CNN, the accuracy and AUC were 96.3% and 0.998 for full TTE studies and 95.0% and 0.996 on POCUS videos, respectively. The positive predictive value, which defined correctly identified predicted views, was higher with two-dimensional rather than three-dimensional networks, exceeding 93% in apical, short-axis aortic valve, and parasternal long-axis left ventricle views.

Conclusion

An automated view classifier utilizing CNNs was able to classify cardiac views obtained using TTE and POCUS with high accuracy. The view classifier will facilitate the application of deep learning to echocardiography.

Harnessing the Power of AI: A Comprehensive Review of Left Ventricular Ejection Fraction Assessment With Echocardiography

The quantification of left ventricular ejection fraction (LVEF) has important clinical utility in the assessment of cardiac function and is vital for the diagnosis of cardiovascular diseases. A transthoracic echocardiogram serves as the most commonly used tool for LVEF assessment for several reasons, including, its noninvasive nature, great safety profile, real-time image processing ability, portability, and cost-effectiveness. However, transthoracic echocardiogram is highly dependent on the clinical skill of the sonographer and interpreting physician. Moreover, even amongst well-trained clinicians, significant interobserver variability exists in the quantification of LVEF. In search of possible solutions, the usage of artificial intelligence (AI) has been increasingly tested in the clinical setting. While AI-derived ejection fraction is in the preliminary stages of development, it has shown promise in its ability to rapidly quantify LVEF, decrease variability, increase accuracy, and utilize higher-order processing capabilities. This review will delineate the latest advancements of AI in evaluating LVEF through echocardiography and explore the challenges and future trajectory of this emerging domain.

Development and preliminary testing of a prior knowledge-based visual navigation system for cardiac ultrasound scanning

Purpose

Ultrasound is widely used to diagnose disease and guide surgery because it is versatile, inexpensive and radiation-free. However, image acquisition is dependent on the operation of a professional sonographer, which is a difficult skill to learn for a wider range of non-sonographers.

Methods

We propose a prior knowledge-based visual navigation method to obtain three important standard ultrasound views of the heart, based on the sonographer's skill learning and augmented reality prompts. The key information about the probe movement was captured using vision-based tracking and normalisation methods on 14 volunteers, based on a professional sonographer's practice. An augmented reality-based navigation method was then proposed to guide operators with no ultrasound experience to find standard views of the heart in a second set of three volunteers.

Results

Through quantitative analysis and qualitative scoring, the results showed that the proposed method can effectively guide non-sonographers to obtain standard views with diagnostic value.

Conclusion

It is believed that the method proposed in this paper has clear application value in primary care, and expansion of the data will allow the accuracy of the navigation to be further improved.

Reducing echocardiographic examination time through routine use of fully automated software: a comparative study of measurement and report creation time

BACKGROUND

Manual interpretation of echocardiographic data is time-consuming and operator-dependent. With the advent of artificial intelligence (AI), there is a growing interest in its potential to streamline echocardiographic interpretation and reduce variability. This study aimed to compare the time taken for measurements by AI to that by human experts after converting the acquired dynamic images into DICOM data.

METHODS

Twenty-three consecutive patients were examined by a single operator, with varying image quality and different medical conditions. Echocardiographic parameters were independently evaluated by human expert using the manual method and the fully automated US2.ai software. The automated processes facilitated by the US2.ai software encompass real-time processing of 2D and Doppler data, measurement of clinically important variables (such as LV function and geometry), automated parameter assessment, and report generation with findings and comments aligned with guidelines. We assessed the duration required for echocardiographic measurements and report creation.

RESULTS

The AI significantly reduced the measurement time compared to the manual method (159 ± 66 vs. 325 ± 94 s, p < 0.01). In the report creation step, AI was also significantly faster compared to the manual method (71 ± 39 vs. 429 ± 128 s, p < 0.01). The incorporation of AI into echocardiographic analysis led to a 70% reduction in measurement and report creation time compared to manual methods. In cases with fair or poor image quality, AI required more corrections and extended measurement time than in cases of good image quality. Report creation time was longer in cases with increased report complexity due to human confirmation of AI-generated findings.

CONCLUSIONS

This fully automated software has the potential to serve as an efficient tool for echocardiographic analysis, offering results that enhance clinical workflow by providing rapid, zero-click reports, thereby adding significant value.

Artificial Intelligence in Heart Failure: Friend or Foe?

In recent times, there have been notable changes in cardiovascular medicine, propelled by the swift advancements in artificial intelligence (AI). The present work provides an overview of the current applications and challenges of AI in the field of heart failure. It emphasizes the "garbage in, garbage out" issue, where AI systems can produce inaccurate results with skewed data. The discussion covers issues in heart failure diagnostic algorithms, particularly discrepancies between existing models. Concerns about the reliance on the left ventricular ejection fraction (LVEF) for classification and treatment are highlighted, showcasing differences in current scientific perceptions. This review also delves into challenges in implementing AI, including variable considerations and biases in training data. It underscores the limitations of current AI models in real-world scenarios and the difficulty in interpreting their predictions, contributing to limited physician trust in AI-based models. The overarching suggestion is that AI can be a valuable tool in clinicians' hands for treating heart failure patients, as far as existing medical inaccuracies have been addressed before integrating AI into these frameworks.

Revolution of echocardiographic reporting: the new era of artificial intelligence and natural language processing

Artificial intelligence (AI) has been making a significant impact on cardiovascular imaging, transforming everything from data capture to report generation. In the field of echocardiography, AI offers the potential to enhance accuracy, speed up reporting, and reduce the workload of physicians. This is an advantage because, compared to computed tomography and magnetic resonance imaging, echocardiograms tend to exhibit higher observer variability in interpretation. This review takes a comprehensive viewpoint at AI-based reporting systems and their application in echocardiography, emphasizing the need for automated diagnoses. The integration of natural language processing (NLP) technologies, including ChatGPT, could provide revolutionary advancements. One of the exciting prospects of AI integration is its potential to accelerate reporting, thereby improving patient outcomes and access to treatment, while also mitigating physician burnout. However, AI introduces new challenges like ensuring data quality, managing potential over-reliance on AI, addressing legal and ethical concerns, and balancing significant costs against benefits. As we navigate these complexities, it's important for cardiologists to stay updated with AI advancements and learn to utilize them effectively. AI has the potential to be integrated into daily clinical practice, becoming a valuable tool for healthcare professionals dealing with heart diseases, provided it's approached with careful consideration.

Artificial intelligence and heart failure: A state-of-the-art review

Heart failure (HF) is a heterogeneous syndrome affecting more than 60 million individuals globally. Despite recent advancements in understanding of the pathophysiology of HF, many issues remain including residual risk despite therapy, understanding the pathophysiology and phenotypes of patients with HF and preserved ejection fraction, and the challenges related to integrating a large amount of disparate information available for risk stratification and management of these patients. Risk prediction algorithms based on artificial intelligence (AI) may have superior predictive ability compared to traditional methods in certain instances. AI algorithms can play a pivotal role in the evolution of HF care by facilitating clinical decision making to overcome various challenges such as allocation of treatment to patients who are at highest risk or are more likely to benefit from therapies, prediction of adverse outcomes, and early identification of patients with subclinical disease or worsening HF. With the ability to integrate and synthesize large amounts of data with multidimensional interactions, AI algorithms can supply information with which physicians can improve their ability to make timely and better decisions. In this review, we provide an overview of the AI algorithms that have been developed for establishing early diagnosis of HF, phenotyping HF with preserved ejection fraction, and stratifying HF disease severity. This review also discusses the challenges in clinical deployment of AI algorithms in HF, and the potential path forward for developing future novel learning-based algorithms to improve HF care.

Evaluation of convolutional neural networks for the detection of inter-breath-hold motion from a stack of cardiac short axis slice images

PURPOSE

This study aimed to develop and validate a deep learning-based method that detects inter-breath-hold motion from an estimated cardiac long axis image reconstructed from a stack of short axis cardiac cine images.

METHODS

Cardiac cine magnetic resonance image data from all short axis slices and 2-/3-/4-chamber long axis slices were considered for the study. Data from 740 subjects were used for model development, and data from 491 subjects were used for testing. The method utilized the slice orientation information to calculate the intersection line of a short axis plane and a long axis plane. An estimated long axis image is shown along with a long axis image as a motion-free reference image, which enables visual assessment of the inter-breath-hold motion from the estimated long axis image. The estimated long axis image was labeled as either a motion-corrupted or a motion-free image. Deep convolutional neural network (CNN) models were developed and validated using the labeled data.

RESULTS

The method was fully automatic in obtaining long axis images reformatted from a 3D stack of short axis slices and predicting the presence/absence of inter-breath-hold motion. The deep CNN model with EfficientNet-B0 as a feature extractor was effective at motion detection with an area under the receiver operating characteristic (AUC) curve of 0.87 for the testing data.

CONCLUSION

The proposed method can automatically assess inter-breath-hold motion in a stack of cardiac cine short axis slices. The method can help prospectively reacquire problematic short axis slices or retrospectively correct motion.

Assessment of Right Ventricle Function and Tricuspid Regurgitation in Heart Failure: Current Advances in Diagnosis and Imaging

Right ventricular (RV) systolic dysfunction increases mortality among heart failure patients, and therefore, accurate diagnosis and monitoring is paramount. RV anatomy and function are complex, usually requiring a combination of imaging modalities to completely quantitate volumes and function. Tricuspid regurgitation usually occurs with RV dysfunction, and quantifying this valvular lesion also may require multiple imaging modalities. Echocardiography is the first-line imaging tool for identifying RV dysfunction, with cardiac MRI and cardiac computed tomography adding valuable additional information.

Echocardiography-based AI for detection and quantification of atrial septal defect

Objectives

We developed and tested a deep learning (DL) framework applicable to color Doppler echocardiography for automatic detection and quantification of atrial septal defects (ASDs).

Background

Color Doppler echocardiography is the most commonly used non-invasive imaging tool for detection of ASDs. While prior studies have used DL to detect the presence of ASDs from standard 2D echocardiographic views, no study has yet reported automatic interpretation of color Doppler videos for detection and quantification of ASD.

Methods

A total of 821 examinations from two tertiary care hospitals were collected as the training and external testing dataset. We developed DL models to automatically process color Doppler echocardiograms, including view selection, ASD detection and identification of the endpoints of the atrial septum and of the defect to quantify the size of defect and the residual rim.

Results

The view selection model achieved an average accuracy of 99% in identifying four standard views required for evaluating ASD. In the external testing dataset, the ASD detection model achieved an area under the curve (AUC) of 0.92 with 88% sensitivity and 89% specificity. The final model automatically measured the size of defect and residual rim, with the mean biases of 1.9 mm and 2.2 mm, respectively.

Conclusion

We demonstrated the feasibility of using a deep learning model for automated detection and quantification of ASD from color Doppler echocardiography. This model has the potential to improve the accuracy and efficiency of using color Doppler in clinical practice for screening and quantification of ASDs, that are required for clinical decision making.

Assisted probe guidance in cardiac ultrasound: A review

Echocardiography is the most frequently used imaging modality in cardiology. However, its acquisition is affected by inter-observer variability and largely dependent on the operator's experience. In this context, artificial intelligence techniques could reduce these variabilities and provide a user independent system. In recent years, machine learning (ML) algorithms have been used in echocardiography to automate echocardiographic acquisition. This review focuses on the state-of-the-art studies that use ML to automate tasks regarding the acquisition of echocardiograms, including quality assessment (QA), recognition of cardiac views and assisted probe guidance during the scanning process. The results indicate that performance of automated acquisition was overall good, but most studies lack variability in their datasets. From our comprehensive review, we believe automated acquisition has the potential not only to improve accuracy of diagnosis, but also help novice operators build expertise and facilitate point of care healthcare in medically underserved areas.

Machine Learning for Pediatric Echocardiographic Mitral Regurgitation Detection

BACKGROUND

Echocardiography-based screening for valvular disease in at-risk asymptomatic children can result in early diagnosis. These screening programs, however, are resource intensive and may not be feasible in many resource-limited settings. Automated echocardiographic diagnosis may enable more widespread echocardiographic screening, early diagnosis, and improved outcomes. In this feasibility study, the authors sought to build a machine learning model capable of identifying mitral regurgitation (MR) on echocardiography.

METHODS

Echocardiograms were labeled by clip for view and by frame for the presence of MR. The labeled data were used to build two convolutional neural networks to perform the stepwise tasks of classifying the clips (1) by view and (2) by the presence of any MR, including physiologic, in parasternal long-axis color Doppler views. The view classification model was developed using 66,330 frames, and model performance was evaluated using a hold-out testing data set with 45 echocardiograms (11,730 frames). The MR detection model was developed using 938 frames, and model performance was evaluated using a hold-out testing data set with 42 echocardiograms (182 frames). Metrics to evaluate model performance included accuracy, precision, recall, F1 score (average of precision and recall, ranging from 0 to 1, with 1 suggesting perfect precision and recall), and receiver operating characteristic analysis.

RESULTS

For the parasternal long-axis view with color Doppler, the view classification convolutional neural network achieved an F1 score of 0.97. The MR detection convolutional neural network achieved testing accuracy of 0.86 and an area under the receiver operating characteristic curve of 0.91.

CONCLUSIONS

A machine learning model is capable of discerning MR on transthoracic echocardiography. This is an encouraging step toward machine learning-based diagnosis of valvular heart disease on pediatric echocardiography.

A formal validation of a deep learning-based automated workflow for the interpretation of the echocardiogram

This study compares a deep learning interpretation of 23 echocardiographic parameters-including cardiac volumes, ejection fraction, and Doppler measurements-with three repeated measurements by core lab sonographers. The primary outcome metric, the individual equivalence coefficient (IEC), compares the disagreement between deep learning and human readers relative to the disagreement among human readers. The pre-determined non-inferiority criterion is 0.25 for the upper bound of the 95% confidence interval. Among 602 anonymised echocardiographic studies from 600 people (421 with heart failure, 179 controls, 69% women), the point estimates of IEC are all <0 and the upper bound of the 95% confidence intervals below 0.25, indicating that the disagreement between the deep learning and human measures is lower than the disagreement among three core lab readers. These results highlight the potential of deep learning algorithms to improve efficiency and reduce the costs of echocardiography.

Automatic view classification of contrast and non-contrast echocardiography

Background

Contrast and non-contrast echocardiography are crucial for cardiovascular diagnoses and treatments. Correct view classification is a foundational step for the analysis of cardiac structure and function. View classification from all sequences of a patient is laborious and depends heavily on the sonographer’s experience. In addition, the intra-view variability and the inter-view similarity increase the difficulty in identifying critical views in contrast and non-contrast echocardiography. This study aims to develop a deep residual convolutional neural network (CNN) to automatically identify multiple views of contrast and non-contrast echocardiography, including parasternal left ventricular short axis, apical two, three, and four-chamber views.

Methods

The study retrospectively analyzed a cohort of 855 patients who had undergone left ventricular opacification at the Department of Ultrasound Medicine, Wuhan Union Medical College Hospital from 2013 to 2021, including 70.3% men and 29.7% women aged from 41 to 62 (median age, 53). All datasets were preprocessed to remove sensitive information and 10 frames with equivalent intervals were sampled from each of the original videos. The number of frames in the training, validation, and test datasets were, respectively, 19,370, 2,370, and 2,620 from 9 views, corresponding to 688, 84, and 83 patients. We presented the CNN model to classify echocardiographic views with an initial learning rate of 0.001, and a batch size of 4 for 30 epochs. The learning rate was decayed by a factor of 0.9 per epoch.

Results

On the test dataset, the overall classification accuracy is 99.1 and 99.5% for contrast and non-contrast echocardiographic views. The average precision, recall, specificity, and F1 score are 96.9, 96.9, 100, and 96.9% for the 9 echocardiographic views.

Conclusions

This study highlights the potential of CNN in the view classification of echocardiograms with and without contrast. It shows promise in improving the workflow of clinical analysis of echocardiography.

Echocardiography-based AI detection of regional wall motion abnormalities and quantification of cardiac function in myocardial infarction

Objective

To compare the performance of a newly developed deep learning (DL) framework for automatic detection of regional wall motion abnormalities (RWMAs) for patients presenting with the suspicion of myocardial infarction from echocardiograms obtained with portable bedside equipment versus standard equipment.

Background

Bedside echocardiography is increasingly used by emergency department setting for rapid triage of patients presenting with chest pain. However, compared to images obtained with standard equipment, lower image quality from bedside equipment can lead to improper diagnosis. To overcome these limitations, we developed an automatic workflow to process echocardiograms, including view selection, segmentation, detection of RWMAs and quantification of cardiac function that was trained and validated on image obtained from bedside and standard equipment.

Methods

We collected 4,142 examinations from one hospital as training and internal testing dataset and 2,811 examinations from other hospital as the external test dataset. For data pre-processing, we adopted DL model to automatically recognize three apical views and segment the left ventricle. Detection of RWMAs was achieved with 3D convolutional neural networks (CNN). Finally, DL model automatically measured the size of cardiac chambers and left ventricular ejection fraction.

Results

The view selection model identified the three apical views with an average accuracy of 96%. The segmentation model provided good agreement with manual segmentation, achieving an average Dice of 0.89. In the internal test dataset, the model detected RWMAs with AUC of 0.91 and 0.88 respectively for standard and bedside ultrasound. In the external test dataset, the AUC were 0.90 and 0.85. The automatic cardiac function measurements agreed with echocardiographic report values (e. g., mean bias is 4% for left ventricular ejection fraction).

Conclusion

We present a fully automated echocardiography pipeline applicable to both standard and bedside ultrasound with various functions, including view selection, quality control, segmentation, detection of the region of wall motion abnormalities and quantification of cardiac function.

Semi-supervised learning with natural language processing for right ventricle classification in echocardiography-a scalable approach

We created a deep learning model, trained on text classified by natural language processing (NLP), to assess right ventricular (RV) size and function from echocardiographic images. We included 12,684 examinations with corresponding written reports for text classification. After manual annotation of 1489 reports, we trained an NLP model to classify the remaining 10,651 reports. A view classifier was developed to select the 4-chamber or RV-focused view from an echocardiographic examination (n = 539). The final models were two image classification models trained on the predicted labels from the combined manual annotation and NLP models and the corresponding echocardiographic view to assess RV function (training set n = 11,008) and size (training set n = 9951. The text classifier identified impaired RV function with 99% sensitivity and 98% specificity and RV enlargement with 98% sensitivity and 98% specificity. The view classification model identified the 4-chamber view with 92% accuracy and the RV-focused view with 73% accuracy. The image classification models identified impaired RV function with 93% sensitivity and 72% specificity and an enlarged RV with 80% sensitivity and 85% specificity; agreement with the written reports was substantial (both κ = 0.65). Our findings show that models for automatic image assessment can be trained to classify RV size and function by using model-annotated data from written echocardiography reports. This pipeline for auto-annotation of the echocardiographic images, using a NLP model with medical reports as input, can be used to train an image-assessment model without manual annotation of images and enables fast and inexpensive expansion of the training dataset when needed.

Automated Echocardiographic Detection of Severe Coronary Artery Disease Using Artificial Intelligence

OBJECTIVES

The purpose of this study was to establish whether an artificially intelligent (AI) system can be developed to automate stress echocardiography analysis and support clinician interpretation.

BACKGROUND

Coronary artery disease is the leading global cause of mortality and morbidity and stress echocardiography remains one of the most commonly used diagnostic imaging tests.

METHODS

An automated image processing pipeline was developed to extract novel geometric and kinematic features from stress echocardiograms collected as part of a large, United Kingdom-based prospective, multicenter, multivendor study. An ensemble machine learning classifier was trained, using the extracted features, to identify patients with severe coronary artery disease on invasive coronary angiography. The model was tested in an independent U.S.

STUDY

How availability of an AI classification might impact clinical interpretation of stress echocardiograms was evaluated in a randomized crossover reader study.

RESULTS

Acceptable classification accuracy for identification of patients with severe coronary artery disease in the training data set was achieved on cross-fold validation based on 31 unique geometric and kinematic features, with a specificity of 92.7% and a sensitivity of 84.4%. This accuracy was maintained in the independent validation data set. The use of the AI classification tool by clinicians increased inter-reader agreement and confidence as well as sensitivity for detection of disease by 10% to achieve an area under the receiver-operating characteristic curve of 0.93.

CONCLUSION

Automated analysis of stress echocardiograms is possible using AI and provision of automated classifications to clinicians when reading stress echocardiograms could improve accuracy, inter-reader agreement, and reader confidence.

Automated interpretation of systolic and diastolic function on the echocardiogram: a multicohort study

BACKGROUND

Echocardiography is the diagnostic modality for assessing cardiac systolic and diastolic function to diagnose and manage heart failure. However, manual interpretation of echocardiograms can be time consuming and subject to human error. Therefore, we developed a fully automated deep learning workflow to classify, segment, and annotate two-dimensional (2D) videos and Doppler modalities in echocardiograms.

METHODS

We developed the workflow using a training dataset of 1145 echocardiograms and an internal test set of 406 echocardiograms from the prospective heart failure research platform (Asian Network for Translational Research and Cardiovascular Trials; ATTRaCT) in Asia, with previous manual tracings by expert sonographers. We validated the workflow against manual measurements in a curated dataset from Canada (Alberta Heart Failure Etiology and Analysis Research Team; HEART; n=1029 echocardiograms), a real-world dataset from Taiwan (n=31 241), the US-based EchoNet-Dynamic dataset (n=10 030), and in an independent prospective assessment of the Asian (ATTRaCT) and Canadian (Alberta HEART) datasets (n=142) with repeated independent measurements by two expert sonographers.

FINDINGS

In the ATTRaCT test set, the automated workflow classified 2D videos and Doppler modalities with accuracies (number of correct predictions divided by the total number of predictions) ranging from 0·91 to 0·99. Segmentations of the left ventricle and left atrium were accurate, with a mean Dice similarity coefficient greater than 93% for all. In the external datasets (n=1029 to 10 030 echocardiograms used as input), automated measurements showed good agreement with locally measured values, with a mean absolute error range of 9-25 mL for left ventricular volumes, 6-10% for left ventricular ejection fraction (LVEF), and 1·8-2·2 for the ratio of the mitral inflow E wave to the tissue Doppler e' wave (E/e' ratio); and reliably classified systolic dysfunction (LVEF <40%, area under the receiver operating characteristic curve [AUC] range 0·90-0·92) and diastolic dysfunction (E/e' ratio ≥13, AUC range 0·91-0·91), with narrow 95% CIs for AUC values. Independent prospective evaluation confirmed less variance of automated compared with human expert measurements, with all individual equivalence coefficients being less than 0 for all measurements.

INTERPRETATION

Deep learning algorithms can automatically annotate 2D videos and Doppler modalities with similar accuracy to manual measurements by expert sonographers. Use of an automated workflow might accelerate access, improve quality, and reduce costs in diagnosing and managing heart failure globally.

FUNDING

A*STAR Biomedical Research Council and A*STAR Exploit Technologies.

Automated Analysis of Doppler Echocardiographic Videos as a Screening Tool for Valvular Heart Diseases

OBJECTIVES

This study sought to develop a deep learning (DL) framework to automatically analyze echocardiographic videos for the presence of valvular heart diseases (VHDs).

BACKGROUND

Although advances in DL have been applied to the interpretation of echocardiograms, such techniques have not been reported for interpretation of color Doppler videos for diagnosing VHDs.

METHODS

We developed a 3-stage DL framework for automatic screening of echocardiographic videos for mitral stenosis (MS), mitral regurgitation (MR), aortic stenosis (AS), and aortic regurgitation (AR) that classifies echocardiographic views, detects the presence of VHDs, and, when present, quantifies key metrics related to VHD severities. The algorithm was trained (n = 1,335), validated (n = 311), and tested (n = 434) using retrospectively selected studies from 5 hospitals. A prospectively collected set of 1,374 consecutive echocardiograms served as a real-world test data set.

RESULTS

Disease classification accuracy was high, with areas under the curve of 0.99 (95% CI: 0.97-0.99) for MS; 0.88 [95% CI: 0.86-0.90] for MR; 0.97 [95% CI: 0.95-0.99] for AS; and 0.90 [95% CI: 0.88-0.92]) for AR in the prospective test data set. The limits of agreement (LOA) between the DL algorithm and physician estimates of metrics of valve lesion severities compared to the LOAs between 2 experienced physicians spanned from -0.60 to 0.77 cm² vs -0.48 to 0.44 cm² for MV area; from -0.27 to 0.25 vs -0.23 to 0.08 for MR jet area/left atrial area; from -0.86 to 0.52 m/s vs -0.48 to 0.54 m/s for peak aortic valve blood flow velocity (Vmax); from -10.6 to 9.5 mm Hg vs -10.2 to 4.9 mm Hg for average peak aortic valve gradient; and from -0.39 to 0.32 vs -0.31 to 0.32 for AR jet width/left ventricular outflow tract diameter.

CONCLUSIONS

The proposed deep learning algorithm has the potential to automate and increase efficiency of the clinical workflow for screening echocardiographic images for the presence of VHDs and for quantifying metrics of disease severity.

Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: A systematic review

The echocardiogram is a test that is widely used in Heart Disease Diagnoses. However, its analysis is largely dependent on the physician's experience. In this regard, artificial intelligence has become an essential technology to assist physicians. This study is a Systematic Literature Review (SLR) of primary state-of-the-art studies that used Artificial Intelligence (AI) techniques to automate echocardiogram analyses. Searches on the leading scientific article indexing platforms using a search string returned approximately 1400 articles. After applying the inclusion and exclusion criteria, 118 articles were selected to compose the detailed SLR. This SLR presents a thorough investigation of AI applied to support medical decisions for the main types of echocardiogram (Transthoracic, Transesophageal, Doppler, Stress, and Fetal). The article's data extraction indicated that the primary research interest of the studies comprised four groups: 1) Improvement of image quality; 2) identification of the cardiac window vision plane; 3) quantification and analysis of cardiac functions, and; 4) detection and classification of cardiac diseases. The articles were categorized and grouped to show the main contributions of the literature to each type of ECHO. The results indicate that the Deep Learning (DL) methods presented the best results for the detection and segmentation of the heart walls, right and left atrium and ventricles, and classification of heart diseases using images/videos obtained by echocardiography. The models that used Convolutional Neural Network (CNN) and its variations showed the best results for all groups. The evidence produced by the results presented in the tabulation of the studies indicates that the DL contributed significantly to advances in echocardiogram automated analysis processes. Although several solutions were presented regarding the automated analysis of ECHO, this area of research still has great potential for further studies to improve the accuracy of results already known in the literature.

Application of Artificial Intelligence in Cardiovascular Medicine

The advent of advances in machine learning (ML)-based techniques has popularized wide applications of artificial intelligence (AI) in various fields ranging from robotics to medicine. In recent years, there has been a surge in the application of AI to research in cardiovascular medicine, which is largely driven by the availability of large-scale clinical and multi-omics datasets. Such applications are providing a new perspective for a better understanding of cardiovascular disease (CVD), which could be used to develop novel diagnostic and therapeutic strategies. For example, studies have shown that ML has a substantial potential for early diagnosis of different types of CVD, prediction of adverse disease outcomes such as heart failure, and development of newer and personalized treatments. In this article, we provide an overview and discuss the current status of a wide range of AI applications, including machine learning, reinforcement learning, and deep learning, in cardiovascular medicine. © 2021 American Physiological Society. Compr Physiol 11:1-12, 2021.

Deep Learning Analysis of Echocardiographic Images to Predict Positive Genotype in Patients With Hypertrophic Cardiomyopathy

Genetic testing provides valuable insights into family screening strategies, diagnosis, and prognosis in patients with hypertrophic cardiomyopathy (HCM). On the other hand, genetic testing carries socio-economical and psychological burdens. It is therefore important to identify patients with HCM who are more likely to have positive genotype. However, conventional prediction models based on clinical and echocardiographic parameters offer only modest accuracy and are subject to intra- and inter-observer variability. We therefore hypothesized that deep convolutional neural network (DCNN, a type of deep learning) analysis of echocardiographic images improves the predictive accuracy of positive genotype in patients with HCM. In each case, we obtained parasternal short- and long-axis as well as apical 2-, 3-, 4-, and 5-chamber views. We employed DCNN algorithm to predict positive genotype based on the input echocardiographic images. We performed 5-fold cross-validations. We used 2 reference models—the Mayo HCM Genotype Predictor score (Mayo score) and the Toronto HCM Genotype score (Toronto score). We compared the area under the receiver-operating-characteristic curve (AUC) between a combined model using the reference model plus DCNN-derived probability and the reference model. We calculated the p-value by performing 1,000 bootstrapping. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). In addition, we examined the net reclassification improvement. We included 99 adults with HCM who underwent genetic testing. Overall, 45 patients (45%) had positive genotype. The new model combining Mayo score and DCNN-derived probability significantly outperformed Mayo score (AUC 0.86 [95% CI 0.79–0.93] vs. 0.72 [0.61–0.82]; p < 0.001). Similarly, the new model combining Toronto score and DCNN-derived probability exhibited a higher AUC compared to Toronto score alone (AUC 0.84 [0.76–0.92] vs. 0.75 [0.65–0.85]; p = 0.03). An improvement in the sensitivity, specificity, PPV, and NPV was also achieved, along with significant net reclassification improvement. In conclusion, compared to the conventional models, our new model combining the conventional and DCNN-derived models demonstrated superior accuracy to predict positive genotype in patients with HCM.

How to standardize the measurement of left ventricular ejection fraction

Despite recent advances in imaging for myocardial deformation, left ventricular ejection fraction (LVEF) is still the most important index for systolic function in daily practice. Its role in multiple fields (e.g., valvular heart disease, myocardial infarction, cancer therapy-related cardiac dysfunction) has been a mainstay in guidelines. In addition, assessment of LVEF is vital to clinical decision-making in patients with heart failure. However, notable limitations to LVEF include poor inter-observer reproducibility dependent on observer skill, poor acoustic windows, and variations in measurement techniques. To solve these problems, methods for standardization of LVEF by sharing reference images among observers and artificial intelligence for accurate measurements have been developed. In this review, we focus on the standardization of LVEF using reference images and automated LVEF using artificial intelligence.

Automated Recognition of Ultrasound Cardiac Views Based on Deep Learning with Graph Constraint

In transthoracic echocardiographic (TTE) examination, it is essential to identify the cardiac views accurately. Computer-aided recognition is expected to improve the accuracy of cardiac views of the TTE examination, particularly when obtained by non-trained providers. A new method for automatic recognition of cardiac views is proposed consisting of three processes. First, a spatial transform network is performed to learn cardiac shape changes during a cardiac cycle, which reduces intra-class variability. Second, a channel attention mechanism is introduced to adaptively recalibrate channel-wise feature responses. Finally, the structured signals by the similarities among cardiac views are transformed into the graph-based image embedding, which acts as unsupervised regularization constraints to improve the generalization accuracy. The proposed method is trained and tested in 171792 cardiac images from 584 subjects. The overall accuracy of the proposed method on cardiac image classification is 99.10%, and the mean AUC is 99.36%, better than known methods. Moreover, the overall accuracy is 97.73%, and the mean AUC is 98.59% on an independent test set with 37,883 images from 100 subjects. The proposed automated recognition model achieved comparable accuracy with true cardiac views, and thus can be applied clinically to help find standard cardiac views.

Artificial Intelligence and Echocardiography

Artificial intelligence (AI) is evolving in the field of diagnostic medical imaging, including echocardiography. Although the dynamic nature of echocardiography presents challenges beyond those of static images from X-ray, computed tomography, magnetic resonance, and radioisotope imaging, AI has influenced all steps of echocardiography, from image acquisition to automatic measurement and interpretation. Considering that echocardiography often is affected by inter-observer variability and shows a strong dependence on the level of experience, AI could be extremely advantageous in minimizing observer variation and providing reproducible measures, enabling accurate diagnosis. Currently, most reported AI applications in echocardiographic measurement have focused on improved image acquisition and automation of repetitive and tedious tasks; however, the role of AI applications should not be limited to conventional processes. Rather, AI could provide clinically important insights from subtle and non-specific data, such as changes in myocardial texture in patients with myocardial disease. Recent initiatives to develop large echocardiographic databases can facilitate development of AI applications. The ultimate goal of applying AI to echocardiography is automation of the entire process of echocardiogram analysis. Once automatic analysis becomes reliable, workflows in clinical echocardiographic will change radically. The human expert will remain the master controlling the overall diagnostic process, will not be replaced by AI, and will obtain significant support from AI systems to guide acquisition, perform measurements, and integrate and compare data on request.

Steps to use artificial intelligence in echocardiography

Artificial intelligence (AI) has influenced every field of cardiovascular imaging in all phases from acquisition to reporting. Compared with computed tomography and magnetic resonance imaging, there is an issue of high observer variation in the interpretation of echocardiograms. Therefore, AI can help minimize the observer variation and provide accurate diagnosis in the field of echocardiography. In this review, we summarize the necessity for automated diagnosis in the echocardiographic field, and discuss the results of AI application to echocardiography and future perspectives. Currently, there are two roles for AI in cardiovascular imaging. One is the automation of tasks performed by humans, such as image segmentation, measurement of cardiac structural and functional parameters. The other is the discovery of clinically important insights. Most reported applications were focused on the automation of tasks. Moreover, algorithms that can obtain cardiac measurements are also being reported. In the next stage, AI can be expected to expand and enrich existing knowledge. With the continual evolution of technology, cardiologists should become well versed in this new knowledge of AI and be able to harness it as a tool. AI can be incorporated into everyday clinical practice and become a valuable aid for many healthcare professionals dealing with cardiovascular diseases.

Radiomics in Echocardiography: Deep Learning and Echocardiographic Analysis

PURPOSE OF REVIEW

Recent development in artificial intelligence (AI) for cardiovascular imaging analysis, involving deep learning, is the start of a new phase in the research field. We review the current state of AI in cardiovascular field and discuss about its potential to improve clinical workflows and accuracy of diagnosis.

RECENT FINDINGS

In the AI cardiovascular imaging field, there are many applications involving efficient image reconstruction, patient triage, and support for clinical decisions. These tools have a role to support repetitive clinical tasks. Although they will be powerful in some situations, these applications may have new potential in the hands of echo cardiologists, assisting but not replacing the human observer. We believe AI has the potential to improve the quality of echocardiography. Someday AI may be incorporated into the daily clinical setting, being an instrumental tool for cardiologists dealing with cardiovascular diseases.