Right ventricle segmentation from cardiac MRI: A collation study

Multi-modality multiorgan image segmentation using continual learning with enhanced hard attention to the task

BACKGROUND

Enabling a deep neural network (DNN) to learn multiple tasks using the concept of continual learning potentially better mimics human brain functions. However, current continual learning studies for medical image segmentation are mostly limited to single-modality images at identical anatomical locations.

PURPOSE

To propose and evaluate a continual learning method termed eHAT (enhanced hard attention to the task) for performing multi-modality, multiorgan segmentation tasks using a DNN.

METHODS

Four public datasets covering the lumbar spine, heart, and brain acquired by magnetic resonance imaging (MRI) and computed tomography (CT) were included to segment the vertebral bodies, the right ventricle, and brain tumors, respectively. Three-task (spine CT, heart MRI, and brain MRI) and four-task (spine CT, heart MRI, brain MRI, and spine MRI) models were tested for eHAT, with the three-task results compared with state-of-the-art continual learning methods. The effectiveness of multitask performance was measured using the forgetting rate, defined as the average difference in Dice coefficients and Hausdorff distances between multiple-task and single-task models. The ability to transfer knowledge to different tasks was evaluated using backward transfer (BWT).

RESULTS

The forgetting rates were -2.51% to -0.60% for the three-task eHAT models with varying task orders, substantially better than the -18.13% to -3.59% using original hard attention to the task (HAT), while those in four-task models were -2.54% to -1.59%. In addition, four-task U-net models with eHAT using only half the number of channels (1/4 parameters) yielded nearly equal performance with or without regularization. A retrospective model comparison showed that eHAT with fixed or automatic regularization had significantly superior BWT (-3% to 0%) compared to HAT (-22% to -4%).

CONCLUSION

We demonstrate for the first time that eHAT effectively achieves continual learning of multi-modality, multiorgan segmentation tasks using a single DNN, with improved forgetting rates compared with HAT.

The state-of-the-art in cardiac MRI reconstruction: Results of the CMRxRecon challenge in MICCAI 2023

Cardiac magnetic resonance imaging (MRI) provides detailed and quantitative evaluation of the heart's structure, function, and tissue characteristics with high-resolution spatial-temporal imaging. However, its slow imaging speed and motion artifacts are notable limitations. Undersampling reconstruction, especially data-driven algorithms, has emerged as a promising solution to accelerate scans and enhance imaging performance using highly under-sampled data. Nevertheless, the scarcity of publicly available cardiac k-space datasets and evaluation platform hinder the development of data-driven reconstruction algorithms. To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). CMRxRecon presented an extensive k-space dataset comprising cine and mapping raw data, accompanied by detailed annotations of cardiac anatomical structures. With overwhelming participation, the challenge attracted more than 285 teams and over 600 participants. Among them, 22 teams successfully submitted Docker containers for the testing phase, with 7 teams submitted for both cine and mapping tasks. All teams use deep learning based approaches, indicating that deep learning has predominately become a promising solution for the problem. The first-place winner of both tasks utilizes the E2E-VarNet architecture as backbones. In contrast, U-Net is still the most popular backbone for both multi-coil and single-coil reconstructions. This paper provides a comprehensive overview of the challenge design, presents a summary of the submitted results, reviews the employed methods, and offers an in-depth discussion that aims to inspire future advancements in cardiac MRI reconstruction models. The summary emphasizes the effective strategies observed in Cardiac MRI reconstruction, including backbone architecture, loss function, pre-processing techniques, physical modeling, and model complexity, thereby providing valuable insights for further developments in this field.

Evaluation of state-of-the-art deep learning models in the segmentation of the left and right ventricles in parasternal short-axis echocardiograms

Purpose

Previous studies on echocardiogram segmentation are focused on the left ventricle in parasternal long-axis views. Deep-learning models were evaluated on the segmentation of the ventricles in parasternal short-axis echocardiograms (PSAX-echo). Segmentation of the ventricles in complementary echocardiogram views will allow the computation of important metrics with the potential to aid in diagnosing cardio-pulmonary diseases and other cardiomyopathies. Evaluating state-of-the-art models with small datasets can reveal if they improve performance on limited data.

Approach

PSAX-echo was performed on 33 volunteer women. An experienced cardiologist identified end-diastole and end-systole frames from 387 scans, and expert observers manually traced the contours of the cardiac structures. Traced frames were pre-processed and used to create labels to train two domain-specific (Unet-Resnet101 and Unet-ResNet50), and four general-domain [three segment anything (SAM) variants, and the Detectron2] deep-learning models. The performance of the models was evaluated using the Dice similarity coefficient (DSC), Hausdorff distance (HD), and difference in cross-sectional area (DCSA).

Results

The Unet-Resnet101 model provided superior performance in the segmentation of the ventricles with 0.83, 4.93 pixels, and 106    pixel 2 on average for DSC, HD, and DCSA, respectively. A fine-tuned MedSAM model provided a performance of 0.82, 6.66 pixels, and 1252    pixel 2 , whereas the Detectron2 model provided 0.78, 2.12 pixels, and 116    pixel 2 for the same metrics, respectively.

Conclusions

Deep-learning models are suitable for the segmentation of the left and right ventricles in PSAX-echo. We demonstrated that domain-specific trained models such as Unet-ResNet provide higher accuracy for echo segmentation than general-domain segmentation models when working with small and locally acquired datasets.

Cardiovascular magnetic resonance imaging traits associated with adverse right ventricular remodeling in repaired tetralogy of Fallot: A Single Center Outcomes Using cardiovascular magnetic resonance in Tetralogy of Fallot study

BACKGROUND

Deterioration of right ventricular (RV) function in repaired tetralogy of Fallot (rToF) is poorly understood. Cardiovascular magnetic resonance (CMR) is used for monitoring, but its analysis is user-dependent and time-consuming. We sought to automate the analysis of CMR using machine learning and to identify imaging traits associated with adverse RV remodeling in the natural history of rToF.

METHODS

A longitudinal cohort of rToF patients underwent CMR at the Children's Hospital of Philadelphia. The nnU-Net method was used to train a machine learning model to segment the left ventricular (LV) blood pool, LV myocardium, and RV blood pool from two-dimensional short-axis CMR images. Conventional and novel measures were calculated and studied in association with remodeling rates using multivariable linear regression. Remodeling rates were calculated as ((Variablescan2 - Variablescan1)/years between scans) for the variables end-diastolic volume index (EDVi), end-systolic volume index (ESVi), stroke volume index (SVi), ejection fraction (EF), and peak systolic dV/dt.

RESULTS

The cohort was comprised of 758 patients, of whom 152 had 2 analyzable scans. Thirty-six patients underwent pulmonic valve replacement (PVR) between scans. Compared to patients with no intervention (representing the natural history of rToF), patients with PVR had significantly lower remodeling rates for RVEDVi, RVESVi, RVSVi, and absolute peak systolic RV dV/dt, while RVEF and left-sided metrics did not differ between groups. In 116 patients without PVR between scans, RV remodeling rates were negatively associated with baseline LV mass index, LVEDVi, LVSVi, and absolute peak systolic LV dV/dt.

CONCLUSION

We demonstrated that rToF patients with two CMR scans and PVR have significant differences in and opposite directions of RV remodeling rates compared to those with no intervention. We also showed that several left-sided measures of structure and function were associated with RV remodeling rates, indicating the importance of baseline LV measurements in characterizing future risk of adverse RV remodeling.

Accurate Reconstruction of Right Heart Shape and Motion From Cine-MRI for Image-Driven Computational Hemodynamics

Accurate reconstruction of the right heart geometry and motion from time-resolved medical images is crucial for diagnostic enhancement and computational analysis of cardiac blood dynamics. Commonly used segmentation and/or reconstruction techniques, exclusively relying on short-axis cine-MRI, lack precision in critical regions of the right heart, such as the ventricular base and the outflow tract, due to its unique morphology and motion. Furthermore, the reconstruction procedure is time-consuming and necessitates significant manual intervention for generating computational domains. This study introduces an end-to-end hybrid reconstruction method specifically designed for computational simulations. Integrating information from various cine-MRI series (short/long-axis and 2/3/4 chambers views) with minimal user contribution, our method leverages registration- and morphing-based algorithms to accurately reconstruct crucial cardiac features and complete cardiac motion. The reconstructed data enable the creation of patient-specific computational fluid dynamics models, facilitating the analysis of the hemodynamics in healthy and clinically relevant scenarios. We assessed the accuracy of our reconstruction method against ground truth and a standard method. We also evaluated volumetric clinical parameters and compared them with the literature values. The method's adaptability was investigated by reducing the number of cine-MRI views, highlighting its robustness with varying imaging data. Numerical findings supported the reliability of the approach for simulating hemodynamics. Combining registration- and morphing-based algorithms, our method offers accurate reconstructions of the right heart chambers' morphology and motion. These reconstructions can serve as valuable tools as domain and boundary conditions for computational fluid dynamics simulations, ensuring seamless and effective analysis.

Automated biventricular quantification in patients with repaired tetralogy of Fallot using a three-dimensional deep learning segmentation model

BACKGROUND

Deep learning is the state-of-the-art approach for automated segmentation of the left ventricle (LV) and right ventricle (RV) in cardiovascular magnetic resonance (CMR) images. However, these models have been mostly trained and validated using CMR datasets of structurally normal hearts or cases with acquired cardiac disease, and are therefore not well-suited to handle cases with congenital cardiac disease such as tetralogy of Fallot (TOF). We aimed to develop and validate a dedicated model with improved performance for LV and RV cavity and myocardium quantification in patients with repaired TOF.

METHODS

We trained a three-dimensional (3D) convolutional neural network (CNN) with 5-fold cross-validation using manually delineated end-diastolic (ED) and end-systolic (ES) short-axis image stacks obtained from either a public dataset containing patients with no or acquired cardiac pathology (n = 100), an institutional dataset of TOF patients (n = 96), or both datasets mixed. Our method allows for missing labels in the training images to accommodate for different ED and ES phases for LV and RV as is commonly the case in TOF. The best performing model was applied to all frames of a separate test set of TOF cases (n = 36) and ED and ES phases were automatically determined for LV and RV separately. The model was evaluated against the performance of a commercial software (suiteHEART®, NeoSoft, Pewaukee, Wisconsin, US).

RESULTS

Training on the mixture of both datasets yielded the best agreement with the manual ground truth for the TOF cases, achieving a median Dice similarity coefficient of (93.8%, 89.8%) for LV cavity and of (92.9%, 90.9%) for RV cavity at (ED, ES) respectively, and of 80.9% and 61.8% for LV and RV myocardium at ED. The offset in automated ED and ES frame selection was 0.56 and 0.89 frames on average for LV and RV respectively. No statistically significant differences were found between our model and the commercial software for LV quantification (two-sided Wilcoxon signed rank test, p<5%), while RV quantification was significantly improved with our model achieving a mean absolute error of 12 ml for RV cavity compared to 36 ml for the commercial software.

CONCLUSION

We developed and validated a fully automatic segmentation and quantification approach for LV and RV, including RV mass, in patients with repaired TOF. Compared to a commercial software, our approach is superior for RV quantification indicating its potential in clinical practice.

Artificial intelligence: The future for multimodality imaging of right ventricle

The crucial pathophysiological and prognostic roles of the right ventricle in various diseases have been well-established. Nonetheless, conventional cardiovascular imaging modalities are frequently associated with intrinsic limitations when evaluating right ventricular (RV) morphology and function. The integration of artificial intelligence (AI) in multimodality imaging presents a promising avenue to circumvent these obstacles, paving the way for future fully automated imaging paradigms. This review aimed to address the current challenges faced by clinicians and researchers in integrating RV imaging and AI technology, to provide a comprehensive overview of the current applications of AI in RV imaging, and to offer insights into future directions, opportunities, and potential challenges in this rapidly advancing field.

Computing thickness of irregularly-shaped thin walls using a locally semi-implicit scheme with extrapolation to solve the Laplace equation: Application to the right ventricle

Cardiac Magnetic Resonance (CMR) Imaging is currently considered the gold standard imaging modality in cardiology. However, it is accompanied by a tradeoff between spatial resolution and acquisition time. Providing accurate measures of thin walls relative to the image resolution may prove challenging. One such anatomical structure is the cardiac right ventricle. Methods for measuring thickness of wall-like anatomical structures often rely on the Laplace equation to provide point-to-point correspondences between both boundaries. This work presents limex, a novel method to solve the Laplace equation using ghost nodes and providing extrapolated values, which is tested on three different datasets: a mathematical phantom, a set of biventricular segmentations from CMR images of ten pigs and the database used at the RV Segmentation Challenge held at MICCAI'12. Thickness measurements using the proposed methodology are more accurate than state-of-the-art methods, especially with the coarsest image resolutions, yielding mean L1 norms of the error between 43.28% and 86.52% lower than the second-best methods on the different test datasets. It is also computationally affordable. Limex has outperformed other state-of-the-art methods in classifying RV myocardial segments by their thickness.

Clair N, Miller GF, Dorfman AL, Fogel MA, Ghelani S, Krishnamurthy R, Lam CZ, Quail M, Robinson JD, Schidlow D, Slesnick TC, Weigand J, Steeden JA, Rathod RH, Muthurangu V. A Deep Learning Pipeline for Assessing Ventricular Volumes from a Cardiac MRI Registry of Patients with Single Ventricle Physiology

Purpose To develop an end-to-end deep learning (DL) pipeline for automated ventricular segmentation of cardiac MRI data from a multicenter registry of patients with Fontan circulation (Fontan Outcomes Registry Using CMR Examinations [FORCE]). Materials and Methods This retrospective study used 250 cardiac MRI examinations (November 2007-December 2022) from 13 institutions for training, validation, and testing. The pipeline contained three DL models: a classifier to identify short-axis cine stacks and two U-Net 3+ models for image cropping and segmentation. The automated segmentations were evaluated on the test set (n = 50) by using the Dice score. Volumetric and functional metrics derived from DL and ground truth manual segmentations were compared using Bland-Altman and intraclass correlation analysis. The pipeline was further qualitatively evaluated on 475 unseen examinations. Results There were acceptable limits of agreement (LOA) and minimal biases between the ground truth and DL end-diastolic volume (EDV) (bias: -0.6 mL/m2, LOA: -20.6 to 19.5 mL/m2) and end-systolic volume (ESV) (bias: -1.1 mL/m2, LOA: -18.1 to 15.9 mL/m2), with high intraclass correlation coefficients (ICCs > 0.97) and Dice scores (EDV, 0.91 and ESV, 0.86). There was moderate agreement for ventricular mass (bias: -1.9 g/m2, LOA: -17.3 to 13.5 g/m2) and an ICC of 0.94. There was also acceptable agreement for stroke volume (bias: 0.6 mL/m2, LOA: -17.2 to 18.3 mL/m2) and ejection fraction (bias: 0.6%, LOA: -12.2% to 13.4%), with high ICCs (>0.81). The pipeline achieved satisfactory segmentation in 68% of the 475 unseen examinations, while 26% needed minor adjustments, 5% needed major adjustments, and in 0.4%, the cropping model failed. Conclusion The DL pipeline can provide fast standardized segmentation for patients with single ventricle physiology across multiple centers. This pipeline can be applied to all cardiac MRI examinations in the FORCE registry. Keywords: Cardiac, Adults and Pediatrics, MR Imaging, Congenital, Volume Analysis, Segmentation, Quantification Supplemental material is available for this article. © RSNA, 2023.

Automatic right ventricular segmentation for cine cardiac magnetic resonance images based on a new deep atlas network

BACKGROUND

The high morbidity and mortality of heart disease present a significant threat to human health. The development of methods for the quick and accurate diagnosis of heart diseases, enabling their effective treatment, has become a key issue of concern. Right ventricular (RV) segmentation from cine cardiac magnetic resonance (CMR) images plays a significant role in evaluating cardiac function for clinical diagnosis and prognosis. However, due to the complex structure of the RV, traditional segmentation methods are ineffective for RV segmentation.

PURPOSE

In this paper, we propose a new deep atlas network to improve the learning efficiency and segmentation accuracy of a deep learning network by integrating multi-atlas.

METHODS

First, a dense multi-scale U-net (DMU-net) is presented to acquire transformation parameters from atlas images to target images. The transformation parameters map the atlas image labels to the target image labels. Second, using a spatial transformation layer, the atlas images are deformed based on these parameters. Finally, the network is optimized by backpropagation with two loss functions where the mean squared error function (MSE) is used to measure the similarity of the input images and transformed images. Further, the Dice metric (DM) is used to quantify the overlap between the predicted contours and the ground truth. In our experiments, 15 datasets are used in testing, and 20 cine CMR images are selected as atlas.

RESULTS

The mean values and standard deviations for the DM and Hausdorff distance are 0.871 and 4.67 mm, 0.104 and 2.528 mm, respectively. The correlation coefficients of endo-diastolic volume, endo-systolic volume, ejection fraction, and stroke volume are 0.984, 0.926, 0.980, and 0.991, respectively, and the mean differences between all of the mentioned parameters are 3.2, -1.7, 0.02, and 4.9, respectively. Most of these differences are within the allowable range of 95%, indicating that the results are acceptable and show good consistency. The segmentation results obtained in this method are compared with those obtained by other methods that provide satisfactory performance. The other methods provide better segmentation effects at the base, but either no segmentation or the wrong segmentation at the top, which demonstrate that the deep atlas network can improve top-area segmentation accuracy.

CONCLUSION

Our results indicate that the proposed method can achieve better segmentation results than the previous methods, with both high relevance and consistency, and has the potential for clinical application.

Machine learning in cardiology: Clinical application and basic research

Machine learning is a subfield of artificial intelligence. The quality and versatility of machine learning have been rapidly improving and playing a critical role in many aspects of social life. This trend is also observed in the medical field. Generally, there are three main types of machine learning: supervised, unsupervised, and reinforcement learning. Each type of learning is adequately selected for the purpose and type of data. In the field of medicine, various types of information are collected and used, and research using machine learning is becoming increasingly relevant. Many clinical studies are conducted using electronic health and medical records, including in the cardiovascular area. Machine learning has also been applied in basic research. Machine learning has been widely used for several types of data analysis, such as clustering of microarray analysis and RNA sequence analysis. Machine learning is essential for genome and multi-omics analyses. This review summarizes the recent advancements in the use of machine learning in clinical applications and basic cardiovascular research.

Deep Learning Segmentation of the Right Ventricle in Cardiac MRI: The M&Ms Challenge

In recent years, several deep learning models have been proposed to accurately quantify and diagnose cardiac pathologies. These automated tools heavily rely on the accurate segmentation of cardiac structures in MRI images. However, segmentation of the right ventricle is challenging due to its highly complex shape and ill-defined borders. Hence, there is a need for new methods to handle such structure's geometrical and textural complexities, notably in the presence of pathologies such as Dilated Right Ventricle, Tricuspid Regurgitation, Arrhythmogenesis, Tetralogy of Fallot, and Inter-atrial Communication. The last MICCAI challenge on right ventricle segmentation was held in 2012 and included only 48 cases from a single clinical center. As part of the 12th Workshop on Statistical Atlases and Computational Models of the Heart (STACOM 2021), the M&Ms-2 challenge was organized to promote the interest of the research community around right ventricle segmentation in multi-disease, multi-view, and multi-center cardiac MRI. Three hundred sixty CMR cases, including short-axis and long-axis 4-chamber views, were collected from three Spanish hospitals using nine different scanners from three different vendors, and included a diverse set of right and left ventricle pathologies. The solutions provided by the participants show that nnU-Net achieved the best results overall. However, multi-view approaches were able to capture additional information, highlighting the need to integrate multiple cardiac diseases, views, scanners, and acquisition protocols to produce reliable automatic cardiac segmentation algorithms.

Artificial intelligence in cardiovascular imaging: enhancing image analysis and risk stratification

In this review, we summarize state-of-the-art artificial intelligence applications for non-invasive cardiovascular imaging modalities including CT, MRI, echocardiography, and nuclear myocardial perfusion imaging.

Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.

Reducing segmentation failures in cardiac MRI via late feature fusion and GAN-based augmentation

Cardiac magnetic resonance (CMR) image segmentation is an integral step in the analysis of cardiac function and diagnosis of heart related diseases. While recent deep learning-based approaches in automatic segmentation have shown great promise to alleviate the need for manual segmentation, most of these are not applicable to realistic clinical scenarios. This is largely due to training on mainly homogeneous datasets, without variation in acquisition, which typically occurs in multi-vendor and multi-site settings, as well as pathological data. Such approaches frequently exhibit a degradation in prediction performance, particularly on outlier cases commonly associated with difficult pathologies, artifacts and extensive changes in tissue shape and appearance. In this work, we present a model aimed at segmenting all three cardiac structures in a multi-center, multi-disease and multi-view scenario. We propose a pipeline, addressing different challenges with segmentation of such heterogeneous data, consisting of heart region detection, augmentation through image synthesis and a late-fusion segmentation approach. Extensive experiments and analysis demonstrate the ability of the proposed approach to tackle the presence of outlier cases during both training and testing, allowing for better adaptation to unseen and difficult examples. Overall, we show that the effective reduction of segmentation failures on outlier cases has a positive impact on not only the average segmentation performance, but also on the estimation of clinical parameters, leading to a better consistency in derived metrics.

A new method incorporating deep learning with shape priors for left ventricular segmentation in myocardial perfusion SPECT images

Accurate segmentation of the left ventricle (LV) is crucial for evaluating myocardial perfusion SPECT (MPS) and assessing LV functions. In this study, a novel method combining deep learning with shape priors was developed and validated to extract the LV myocardium and automatically measure LV functional parameters. The method integrates a three-dimensional (3D) V-Net with a shape deformation module that incorporates shape priors generated by a dynamic programming (DP) algorithm to guide its output during training. A retrospective analysis was performed on an MPS dataset comprising 31 subjects without or with mild ischemia, 32 subjects with moderate ischemia, and 12 subjects with severe ischemia. Myocardial contours were manually annotated as the ground truth. A 5-fold stratified cross-validation was used to train and validate the models. The clinical performance was evaluated by measuring LV end-systolic volume (ESV), end-diastolic volume (EDV), left ventricular ejection fraction (LVEF), and scar burden from the extracted myocardial contours. There were excellent agreements between segmentation results by our proposed model and those from the ground truth, with a Dice similarity coefficient (DSC) of 0.9573 ± 0.0244, 0.9821 ± 0.0137, and 0.9903 ± 0.0041, as well as Hausdorff distances (HD) of 6.7529 ± 2.7334 mm, 7.2507 ± 3.1952 mm, and 7.6121 ± 3.0134 mm in extracting the LV endocardium, myocardium, and epicardium, respectively. Furthermore, the correlation coefficients between LVEF, ESV, EDV, stress scar burden, and rest scar burden measured from our model results and the ground truth were 0.92, 0.958, 0.952, 0.972, and 0.958, respectively. The proposed method achieved a high accuracy in extracting LV myocardial contours and assessing LV functions.

MyoPS: A benchmark of myocardial pathology segmentation combining three-sequence cardiac magnetic resonance images

Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on the myocardium is the key to this assessment. This work defines a new task of medical image analysis, i.e., to perform myocardial pathology segmentation (MyoPS) combining three-sequence cardiac magnetic resonance (CMR) images, which was first proposed in the MyoPS challenge, in conjunction with MICCAI 2020. Note that MyoPS refers to both myocardial pathology segmentation and the challenge in this paper. The challenge provided 45 paired and pre-aligned CMR images, allowing algorithms to combine the complementary information from the three CMR sequences for pathology segmentation. In this article, we provide details of the challenge, survey the works from fifteen participants and interpret their methods according to five aspects, i.e., preprocessing, data augmentation, learning strategy, model architecture and post-processing. In addition, we analyze the results with respect to different factors, in order to examine the key obstacles and explore the potential of solutions, as well as to provide a benchmark for future research. The average Dice scores of submitted algorithms were 0.614±0.231 and 0.644±0.153 for myocardial scars and edema, respectively. We conclude that while promising results have been reported, the research is still in the early stage, and more in-depth exploration is needed before a successful application to the clinics. MyoPS data and evaluation tool continue to be publicly available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/).

Deep-active-learning approach towards accurate right ventricular segmentation using a two-level uncertainty estimation

The Right Ventricle (RV) is currently recognised to be a significant and important prognostic factor for various pathologies. Its assessment is made possible using Magnetic Resonance Imaging (CMRI) short-axis slices. Yet, due to the challenging issues of this cavity, radiologists still perform its delineation manually, which is tedious, laborious, and time-consuming. Therefore, to automatically tackle the RV segmentation issues, Deep-Learning (DL) techniques seem to be the axis of the most recent promising approaches. Along with its potential at dealing with shape variations, DL techniques highly requires a large number of labelled images to avoid over-fitting. Subsequently, with the produced large amounts of data in the medical industry, preparing annotated datasets manually is still time-consuming, and requires high skills to be accomplished. To benefit from a significant number of labelled and unlabelled CMRI images, a Deep-Active-Learning (DAL) approach is proposed in this paper to segment the RV. Thus, three main steps are distinguished. First, a personalised labelled dataset is gathered and augmented to allow initial learning. Secondly, a U-Net based architecture is modified towards efficient initial accuracy. Finally, a two-level uncertainty estimation technique is settled to enable the selection of complementary unlabelled data. The proposed pipeline is enhanced with a customised postprocessing step, in which epistemic uncertainty and Dense Conditional Random Fields are used. The proposed approach is tested on 791 images gathered from 32 public patients and 1230 images of 50 custom subjects. The obtained results show an increased dice coefficient from 0.86 to 0.91 with a decreased Hausdorff distance from 7.55 to 7.45.

Deep learning can yield clinically useful right ventricular segmentations faster than fully manual analysis

Right ventricular (RV) volumes are commonly obtained through time-consuming manual delineations of cardiac magnetic resonance (CMR) images. Deep learning-based methods can generate RV delineations, but few studies have assessed their ability to accelerate clinical practice. Therefore, we aimed to develop a clinical pipeline for deep learning-based RV delineations and validate its ability to reduce the manual delineation time. Quality-controlled delineations in short-axis CMR scans from 1114 subjects were used for development. Time reduction was assessed by two observers using 50 additional clinical scans. Automated delineations were subjectively rated as (A) sufficient for clinical use, or as needing (B) minor or (C) major corrections. Times were measured for manual corrections of delineations rated as B or C, and for fully manual delineations on all 50 scans. Fifty-eight % of automated delineations were rated as A, 42% as B, and none as C. The average time was 6 min for a fully manual delineation, 2 s for an automated delineation, and 2 min for a minor correction, yielding a time reduction of 87%. The deep learning-based pipeline could substantially reduce the time needed to manually obtain clinically applicable delineations, indicating ability to yield right ventricular assessments faster than fully manual analysis in clinical practice. However, these results may not generalize to clinics using other RV delineation guidelines.

Validation of Artificial Intelligence Cardiac MRI Measurements: Relationship to Heart Catheterization and Mortality Prediction

Background Cardiac MRI measurements have diagnostic and prognostic value in the evaluation of cardiopulmonary disease. Artificial intelligence approaches to automate cardiac MRI segmentation are emerging but require clinical testing. Purpose To develop and evaluate a deep learning tool for quantitative evaluation of cardiac MRI functional studies and assess its use for prognosis in patients suspected of having pulmonary hypertension. Materials and Methods A retrospective multicenter and multivendor data set was used to develop a deep learning-based cardiac MRI contouring model using a cohort of patients suspected of having cardiopulmonary disease from multiple pathologic causes. Correlation with same-day right heart catheterization (RHC) and scan-rescan repeatability was assessed in prospectively recruited participants. Prognostic impact was assessed using Cox proportional hazard regression analysis of 3487 patients from the ASPIRE (Assessing the Severity of Pulmonary Hypertension In a Pulmonary Hypertension Referral Centre) registry, including a subset of 920 patients with pulmonary arterial hypertension. The generalizability of the automatic assessment was evaluated in 40 multivendor studies from 32 centers. Results The training data set included 539 patients (mean age, 54 years ± 20 [SD]; 315 women). Automatic cardiac MRI measurements were better correlated with RHC parameters than were manual measurements, including left ventricular stroke volume (r = 0.72 vs 0.68; P = .03). Interstudy repeatability of cardiac MRI measurements was high for all automatic measurements (intraclass correlation coefficient range, 0.79-0.99) and similarly repeatable to manual measurements (all paired t test P > .05). Automated right ventricle and left ventricle cardiac MRI measurements were associated with mortality in patients suspected of having pulmonary hypertension. Conclusion An automatic cardiac MRI measurement approach was developed and tested in a large cohort of patients, including a broad spectrum of right ventricular and left ventricular conditions, with internal and external testing. Fully automatic cardiac MRI assessment correlated strongly with invasive hemodynamics, had prognostic value, were highly repeatable, and showed excellent generalizability. Clinical trial registration no. NCT03841344 Published under a CC BY 4.0 license. Online supplemental material is available for this article. See also the editorial by Ambale-Venkatesh and Lima in this issue. An earlier incorrect version appeared online. This article was corrected on June 27, 2022.

Cardiac segmentation on late gadolinium enhancement MRI: A benchmark study from multi-sequence cardiac MR segmentation challenge

Accurate computing, analysis and modeling of the ventricles and myocardium from medical images are important, especially in the diagnosis and treatment management for patients suffering from myocardial infarction (MI). Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) provides an important protocol to visualize MI. However, compared with the other sequences LGE CMR images with gold standard labels are particularly limited. This paper presents the selective results from the Multi-Sequence Cardiac MR (MS-CMR) Segmentation challenge, in conjunction with MICCAI 2019. The challenge offered a data set of paired MS-CMR images, including auxiliary CMR sequences as well as LGE CMR, from 45 patients who underwent cardiomyopathy. It was aimed to develop new algorithms, as well as benchmark existing ones for LGE CMR segmentation focusing on myocardial wall of the left ventricle and blood cavity of the two ventricles. In addition, the paired MS-CMR images could enable algorithms to combine the complementary information from the other sequences for the ventricle segmentation of LGE CMR. Nine representative works were selected for evaluation and comparisons, among which three methods are unsupervised domain adaptation (UDA) methods and the other six are supervised. The results showed that the average performance of the nine methods was comparable to the inter-observer variations. Particularly, the top-ranking algorithms from both the supervised and UDA methods could generate reliable and robust segmentation results. The success of these methods was mainly attributed to the inclusion of the auxiliary sequences from the MS-CMR images, which provide important label information for the training of deep neural networks. The challenge continues as an ongoing resource, and the gold standard segmentation as well as the MS-CMR images of both the training and test data are available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mscmrseg/).

A fast right ventricle segmentation in cine-MRI from a dense hough representation

Segmentation of the right ventricle (RV) is essential for the diagnosis of multiple cardiac pathologies and conditions. However, its manual delineation is a tedious task and computational support is complex due to geometric and dynamic variability.  This work introduces a dense Hough transform and representation (HT) that allows a nonparametric characterization of the shape, encoding each voxel by its curvature and orientation. This representation is integrated into a bayesian tracking approach, which efficiently segments the RV structure throughout the cardiac cycle. The proposed approach was evaluated on a public dataset, with 16 patients, achieving a Sørensen-Dice coefficient of 0.87 and 0.92, for complete volumes and basal structures, respectively. These results evidence an adequate fit of the proposed model with respect to RV shape throughout the entire cardiac cycle. La segmentación del Ventrículo Derecho (VD) es esencial para el diagnóstico de múltiples patologías y condiciones cardiacas. Sin embargo, su delineación manual es una tarea tediosa y el soporte computacional resulta complejo debido a la variabilidad geométrica y dinámica.  Este trabajo introduce una transformación y representación densa de Hough (TH) que permite una caracterización no paramétrica de la forma, codificando cada vóxel por su curvatura y orientación. Esta representación es integrada en un enfoque de seguimiento bayesiano, que logra de forma eficiente segmentar la estructura del VD, a lo largo del ciclo cardíaco. El enfoque propuesto fue evaluado en un conjunto de datos públicos, con 16 pacientes, logrando un coeficiente Sørensen-Dice de 0,87 y 0,92, para volúmenes completos y estructuras basales, respectivamente. Estos resultados evidencian una adecuada adaptación del modelo propuesto respecto a la forma del VD a lo largo de todo el ciclo cardíaco.

Novel User-Friendly Application for MRI Segmentation of Brain Resection following Epilepsy Surgery

Delineation of resected brain cavities on magnetic resonance images (MRIs) of epilepsy surgery patients is essential for neuroimaging/neurophysiology studies investigating biomarkers of the epileptogenic zone. The gold standard to delineate the resection on MRI remains manual slice-by-slice tracing by experts. Here, we proposed and validated a semiautomated MRI segmentation pipeline, generating an accurate model of the resection and its anatomical labeling, and developed a graphical user interface (GUI) for user-friendly usage. We retrieved pre- and postoperative MRIs from 35 patients who had focal epilepsy surgery, implemented a region-growing algorithm to delineate the resection on postoperative MRIs and tested its performance while varying different tuning parameters. Similarity between our output and hand-drawn gold standards was evaluated via dice similarity coefficient (DSC; range: 0-1). Additionally, the best segmentation pipeline was trained to provide an automated anatomical report of the resection (based on presurgical brain atlas). We found that the best-performing set of parameters presented DSC of 0.83 (0.72-0.85), high robustness to seed-selection variability and anatomical accuracy of 90% to the clinical postoperative MRI report. We presented a novel user-friendly open-source GUI that implements a semiautomated segmentation pipeline specifically optimized to generate resection models and their anatomical reports from epilepsy surgery patients, while minimizing user interaction.

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Since the rise of deep learning (DL) in the mid-2010s, cardiac magnetic resonance (CMR) image segmentation has achieved state-of-the-art performance. Despite achieving inter-observer variability in terms of different accuracy performance measures, visual inspections reveal errors in most segmentation results, indicating a lack of reliability and robustness of DL segmentation models, which can be critical if a model was to be deployed into clinical practice. In this work, we aim to bring attention to reliability and robustness, two unmet needs of cardiac image segmentation methods, which are hampering their translation into practice. To this end, we first study the performance accuracy evolution of CMR segmentation, illustrate the improvements brought by DL algorithms and highlight the symptoms of performance stagnation. Afterwards, we provide formal definitions of reliability and robustness. Based on the two definitions, we identify the factors that limit the reliability and robustness of state-of-the-art deep learning CMR segmentation techniques. Finally, we give an overview of the current set of works that focus on improving the reliability and robustness of CMR segmentation, and we categorize them into two families of methods: quality control methods and model improvement techniques. The first category corresponds to simpler strategies that only aim to flag situations where a model may be incurring poor reliability or robustness. The second one, instead, directly tackles the problem by bringing improvements into different aspects of the CMR segmentation model development process. We aim to bring the attention of more researchers towards these emerging trends regarding the development of reliable and robust CMR segmentation frameworks, which can guarantee the safe use of DL in clinical routines and studies.

A U-snake based deep learning network for right ventricle segmentation

PURPOSE

Ventricular segmentation is of great importance for the heart condition monitoring. However, manual segmentation is time-consuming, cumbersome and subjective. Many segmentation methods perform poorly due to the complex structure and uncertain shape of the right ventricle, so we combine deep learning to achieve automatic segmentation.

METHOD

This paper proposed a method named U-Snake network which is based on the improvement of deep snake⁵ together with level set⁸ to segment the right ventricular in the MR images. U-snake aggregates the information of each receptive field which is learned by circular convolution of multiple different dilation rates. At the same time, we also added dice loss functions and transferred the result of U-Snake to the level set so as to further enhance the effect of small object segmentation. our method is tested on the test1 and test2 datasets in the Right Ventricular Segmentation Challenge, which shows the effectiveness.

RESULTS

The experiment showed that we have obtained good result in the right ventricle segmentation challenge(RVSC). The highest segmentation accuracy on the right ventricular test set 2 reached a dice coefficient of 0.911, and the segmentation speed reached 5fps.

CONCLUSIONS

Our method, a new deep learning network named U-snake, has surpassed the previous excellent ventricular segmentation method based on mathematical theory and other classical deep learning methods, such as Residual U-net²⁷ , Inception cnn³³ , Dilated cnn²⁹ , etc. However, it can only be used as an auxiliary tool instead of replacing the work of human beings. This article is protected by copyright. All rights reserved.

Medical image segmentation with generative adversarial semi-supervised network

Recent medical image segmentation methods heavily rely on large-scale training data and high-quality annotations. However, these resources are hard to obtain due to the limitation of medical images and professional annotators. How to utilize limited annotations and maintain the performance is an essential yet challenging problem. In this paper, we try to tackle this problem in a self-learning manner by proposing a generative adversarial semi-supervised network. We use limited annotated images as main supervision signals, and the unlabeled images are manipulated as extra auxiliary information to improve the performance. More specifically, we modulate a segmentation network as a generator to produce pseudo labels for unlabeled images. To make the generator robust, we train an uncertainty discriminator with generative adversarial learning to determine the reliability of the pseudo labels. To further ensure dependability, we apply feature mapping loss to obtain statistic distribution consistency between the generated labels and the real labels. Then the verified pseudo labels are used to optimize the generator in a self-learning manner. We validate the effectiveness of the proposed method on right ventricle dataset, Sunnybrook dataset, STACOM, ISIC dataset, and Kaggle lung dataset. We obtain 0.8402-0.9121, 0.8103-0.9094, 0.9435-0.9724, 0.8635-0.886, and 0.9697-0.9885 dice coefficient with 1/8 to 1/2 proportion of densely annotated labels, respectively. The improvements are up to 28.6 points higher than the corresponding fully supervised baseline.

Cardiac MRI-Derived Myocardial Deformation Parameters Correlate with Pulmonary Valve Replacement Indications in Repaired Tetralogy of Fallot

Right ventricular (RV) volumetric cardiac magnetic resonance (CMR) criteria serve as indicators for pulmonary valve replacement (PVR) in repaired tetralogy of Fallot (rTOF). Myocardial deformation and tricuspid valve displacement parameters may be more sensitive measures of RV dysfunction. This study's aim was to describe rTOF RV deformation and tricuspid displacement patterns using novel CMR semi-automated software and determine associations with standard CMR measures. Retrospective study of 78 pediatric rTOF patients was compared to 44 normal controls. Global RV longitudinal and circumferential strain and strain rate (SR) and tricuspid valve (TV) displacement were measured. Correlation analysis between strain, SR, TV displacement, and volumes was performed between and within subgroups. The sensitivity and specificity of strain parameters in predicting CMR criteria for PVR was determined. Deformation variables were reduced in rTOF compared to controls. Decreased RV strain and TV shortening were associated with increased RV volumes and decreased RVEF. Longitudinal and circumferential parameters were predictive of RVESVi (> 80 ml/m2) and RVEF (< 47%), with circumferential strain (> - 15.88%) and SR (> - 0.62) being most sensitive. Longitudinal strain was unchanged between rTOF subgroups, while circumferential strain trended abnormal in those meeting PVR criteria compared to controls. RV deformation and TV displacement are abnormal in rTOF, and RV circumferential strain variation may reflect an adaptive response to chronic volume or pressure load. This coupled with associations of ventricular deformation with traditional PVR indications suggest importance of this analysis in the evolution of rTOF RV assessment.

Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge

The emergence of deep learning has considerably advanced the state-of-the-art in cardiac magnetic resonance (CMR) segmentation. Many techniques have been proposed over the last few years, bringing the accuracy of automated segmentation close to human performance. However, these models have been all too often trained and validated using cardiac imaging samples from single clinical centres or homogeneous imaging protocols. This has prevented the development and validation of models that are generalizable across different clinical centres, imaging conditions or scanner vendors. To promote further research and scientific benchmarking in the field of generalizable deep learning for cardiac segmentation, this paper presents the results of the Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation (M&Ms) Challenge, which was recently organized as part of the MICCAI 2020 Conference. A total of 14 teams submitted different solutions to the problem, combining various baseline models, data augmentation strategies, and domain adaptation techniques. The obtained results indicate the importance of intensity-driven data augmentation, as well as the need for further research to improve generalizability towards unseen scanner vendors or new imaging protocols. Furthermore, we present a new resource of 375 heterogeneous CMR datasets acquired by using four different scanner vendors in six hospitals and three different countries (Spain, Canada and Germany), which we provide as open-access for the community to enable future research in the field.

Automated segmentation of biventricular contours in tissue phase mapping using deep learning

Tissue phase mapping (TPM) is an MRI technique for quantification of regional biventricular myocardial velocities. Despite its potential, clinical use is limited due to the requisite labor-intensive manual segmentation of cardiac contours for all time frames. The purpose of this study was to develop a deep learning (DL) network for automated segmentation of TPM images, without significant loss in segmentation and myocardial velocity quantification accuracy compared with manual segmentation. We implemented a multi-channel 3D (three dimensional; 2D + time) dense U-Net that trained on magnitude and phase images and combined cross-entropy, Dice, and Hausdorff distance loss terms to improve the segmentation accuracy and suppress unnatural boundaries. The dense U-Net was trained and tested with 150 multi-slice, multi-phase TPM scans (114 scans for training, 36 for testing) from 99 heart transplant patients (44 females, 1-4 scans/patient), where the magnitude and velocity-encoded (V_x , V_y , V_z ) images were used as input and the corresponding manual segmentation masks were used as reference. The accuracy of DL segmentation was evaluated using quantitative metrics (Dice scores, Hausdorff distance) and linear regression and Bland-Altman analyses on the resulting peak radial and longitudinal velocities (V_r and V_z ). The mean segmentation time was about 2 h per patient for manual and 1.9 ± 0.3 s for DL. Our network produced good accuracy (median Dice = 0.85 for left ventricle (LV), 0.64 for right ventricle (RV), Hausdorff distance = 3.17 pixels) compared with manual segmentation. Peak V_r and V_z measured from manual and DL segmentations were strongly correlated (R ≥ 0.88) and in good agreement with manual analysis (mean difference and limits of agreement for V_z and V_r were -0.05 ± 0.98 cm/s and -0.06 ± 1.18 cm/s for LV, and -0.21 ± 2.33 cm/s and 0.46 ± 4.00 cm/s for RV, respectively). The proposed multi-channel 3D dense U-Net was capable of reducing the segmentation time by 3,600-fold, without significant loss in accuracy in tissue velocity measurements.

A Novel Framework with Weighted Decision Map Based on Convolutional Neural Network for Cardiac MR Segmentation

For diagnosing cardiovascular disease, an accurate segmentation method is needed. There are several unre-solved issues in the complex field of cardiac magnetic resonance imaging, some of which have been partially addressed by using deep neural networks. To solve two problems of over-segmentation and under-segmentation of anatomical shapes in the short-axis view from different cardiac magnetic resonance sequences, we propose a novel two-stage framework with a weighted decision map based on convolutional neural networks to segment the myocardium (Myo), left ventricle (LV), and right ventricle (RV) simultaneously. The framework comprises a deci-sion map extractor and a cardiac segmenter. A cascaded U-Net++ is used as a decision map extractor to acquire the decision map that decides the category of each pixel. Cardiac segmenter is a multiscale dual-path feature ag-gregation network (MDFA-Net) which consists of a densely connected network and an asymmetric encoding and decoding network. The input to the cardiac seg-menter is derived from processed original images weighted by the output of the decision map extractor. We conducted experiments on two datasets of mul-ti-sequence cardiac magnetic resonance segmentation challenge 2019 (MS-CMRSeg 2019) and myocardial pa-thology segmentation challenge 2020 (MyoPS 2020). Test results obtained on MyoPS 2020 show that proposed method with average Dice coefficient of 84.70%, 86.00% and 86.31% in the segmentation task of Myo, LV, and RV, respectively.

Detection of Cardiac Tissues using K-means Analysis Methods in Nuclear Medicine Images

BACKGROUND: Nuclear cardiology uses to diagnose the cardiac disorders such as ischemic and inflammation disorders. In cardiac scintigraphy, unraveling closely adjacent tissues in the image are challenging issue. AIM: The aim of the study is to detect of cardiac tissues using K-means analysis methods in nuclear medicine images. This study also aimed to reduce the existence of fleck noise that disturbs the contrast and make its analysis more difficult. METHODS: Thus, digital image processing uses to increase the detection rate of myocardium easily using its color-based algorithms. In this study, color-based K-means was used. The scintographs were converted into color space presentation. Then, each pixel in the image was segmented using color analysis algorithms. RESULTS: The segmented scintograph was displayed in distinct fresh image. The proposed technique defines the myocardial tissues and borders precisely. Both exactness rate and recall reckoning were calculated. The results were 97.3 + 8.46 (p > 0.05). CONCLUSION: The proposed technique offered recognition of the heart tissue with high exactness amount.

3D Attention M-net for Short-axis Left Ventricular Myocardium Segmentation in Mice MR cardiac Images

Small rodent cardiac magnetic resonance imaging (MRI) plays an important role in preclinical models of cardiac disease. Accurate myocardial boundaries delineation is crucial to most morphological and functional analysis in rodent cardiac MRIs. However, rodent cardiac MRIs, due to animal's small cardiac volume and high heart rate, are usually acquired with sub-optimal resolution and low signal-to-noise ratio (SNR). These rodent cardiac MRIs can also suffer from signal loss due to the intra-voxel dephasing. These factors make automatic myocardial segmentation challenging. Manual contouring could be applied to label myocardial boundaries but it is usually laborious, time consuming, and not systematically objective. In this study, we present a deep learning approach based on 3D attention M-net to perform automatic segmentation of left ventricular myocardium. In the deep learning architecture, we use dual spatial-channel attention gates between encoder and decoder along with multi-scale feature fusion path after decoder. Attention gates enable networks to focus on relevant spatial information and channel features to improve segmentation performance. A distance derived loss term, besides general dice loss and binary cross entropy loss, was also introduced to our hybrid loss functions to refine segmentation contours. The proposed model outperforms other generic models, like U-Net and FCN, in major segmentation metrics including the dice score (0.9072), Jaccard index (0.8307) and Hausdorff distance (3.1754 pixels), which are comparable to the results achieved by state-of-the-art models on human cardiac ACDC17 datasets.Clinical relevance Small rodent cardiac MRI is routinely used to probe the effect of individual genes or groups of genes on the etiology of a large number of cardiovascular diseases. An automatic myocardium segmentation algorithm specifically designed for these data can enhance accuracy and reproducibility of cardiac structure and function analysis.

Anatomical knowledge based level set segmentation of cardiac ventricles from MRI

This paper represents a novel level set framework for segmentation of cardiac left ventricle (LV) and right ventricle (RV) from magnetic resonance images based on anatomical structures of the heart. We first propose a level set approach to recover the endocardium and epicardium of LV by using a bi-layer level set (BILLS) formulation, in which the endocardium and epicardium are represented by the 0-level set and k-level set of a level set function. Furthermore, the recovery of LV endocardium and epicardium is achieved by a level set evolution process, called convexity preserving bi-layer level set (CP-BILLS). During the CP-BILLS evolution, the 0-level set and k-level set simultaneously evolve and move toward the true endocardium and epicardium under the guidance of image information and the impact of the convexity preserving mechanism as well. To eliminate the manual selection of the k-level, we develop an algorithm for automatic selection of an optimal k-level. As a result, the obtained endocardial and epicardial contours are convex and consistent with the anatomy of cardiac ventricles. For segmentation of the whole ventricle, we extend this method to the segmentation of RV and myocardium of both left and right ventricles by using a convex shape decomposition (CSD) structure of cardiac ventricles based on anatomical knowledge. Experimental results demonstrate promising performance of our method. Compared with some traditional methods, our method exhibits superior performance in terms of segmentation accuracy and algorithm stability. Our method is comparable with the state-of-the-art deep learning-based method in terms of segmentation accuracy and algorithm stability, but our method has no need for training and the manual segmentation of the training data.

MVSGAN: Spatial-aware Multi-view CMR Fusion for accurate 3D Left Ventricular Myocardium Segmentation

The accurate 3D left ventricular (LV) myocardium segmentation in short-axis (SAX) view of cardiac magnetic resonance (CMR) is challenged by the sparse spatial structure of CMR. The strategy of multi-view CMR fusion can provide fine-grained spatial structure for accurate segmentation. However, the large information misalignment & lack of dense 3D CMR as fusion target in multi-view CMR fusion, and the different spatial resolution between the fused cardiac model and the ground truth of segmentation in segmentation limit the strategy. In this study, we propose a multi-view spatial-aware adversarial network (MVSGAN). It studies the perception of fine-grained cardiac structure for accurate segmentation by the spatially consistent fusion of multi-view CMR. It consists of three modules: (1) A residual adversarial fusion (RAF) module takes inter-slices deep correlation and anatomical prior of multi-view CMR to refine the spatial structures by residual supplement and adversarial optimization. (2) A structural perception-aggregation (SPA) module establishes the spatial correlation between the spatially dense cardiac model and sparse segmentation label for accurate 3D CMR LV myocardium segmentation. (3) A joint training strategy utilizes the spatial dense SAX volume as explicit and implicit goals to jointly optimize the framework. The experiments are applied on a public dataset and a clinical dataset to evaluate the performance of MVSGAN. The average Dice and Jaccard score of LV myocardium segmentation obtained by MVSGAN are highest among seven existing state-of-the-art methods, which are up to 0.92 and 0.75. It is concluded that the spatial-aware multiview CMR fusion can provide meaningful spatial correlation for accurate 3D SAX LV myocardium segmentation.

Multiple two-dimensional active shape model framework for right ventricular segmentation

Segmentation of the right ventricle (RV) in MRI short axis images is very challenging due to its complex shape and various appearance among the different subjects and cross-sections. Active shape models (ASM) have shown potential for segmenting the complex structures, including the RV, through two formulations: two- and three-dimensional modeling with a reported trade-off between accuracy and complexity of each formulation. In this work, we propose a new framework for modeling the RV surface using multiple 2D contours, where information from multiple cross-sectional images are incorporated into the same model. The proposed method was tested using cardiac MRI images from 56 human subjects. Compared to a golden reference of manually delineated RV contours, the proposed method resulted in significantly lower error than (almost one half) that of the conventional 2D ASM especially at the apical slices. The mean absolute distance of the proposed method was 2.9 ± 2 mm while the conventional 2D ASM resulted in an error of 6.6 ± 4.5 mm. In addition, the computation time of the proposed method was 5 s compared to 4 ± 1 min previously reported for the 3D ASM formulation.

Unsupervised Domain Adaptation From Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks

Anisotropic multi-slice Cardiac Magnetic Resonance (CMR) Images are conventionally acquired in patient-specific short-axis (SAX) orientation. In specific cardiovascular diseases that affect right ventricular (RV) morphology, acquisitions in standard axial (AX) orientation are preferred by some investigators, due to potential superiority in RV volume measurement for treatment planning. Unfortunately, due to the rare occurrence of these diseases, data in this domain is scarce. Recent research in deep learning-based methods mainly focused on SAX CMR images and they had proven to be very successful. In this work, we show that there is a considerable domain shift between AX and SAX images, and therefore, direct application of existing models yield sub-optimal results on AX samples. We propose a novel unsupervised domain adaptation approach, which uses task-related probabilities in an attention mechanism. Beyond that, cycle consistency is imposed on the learned patient-individual 3D rigid transformation to improve stability when automatically re-sampling the AX images to SAX orientations. The network was trained on 122 registered 3D AX-SAX CMR volume pairs from a multi-centric patient cohort. A mean 3D Dice of 0.86 ± 0.06 for the left ventricle, 0.65 ± 0.08 for the myocardium, and 0.77 ± 0.10 for the right ventricle could be achieved. This is an improvement of 25% in Dice for RV in comparison to direct application on axial slices. To conclude, our pre-trained task module has neither seen CMR images nor labels from the target domain, but is able to segment them after the domain gap is reduced. Code: https://github.com/Cardio-AI/3d-mri-domain-adaptation.

TSU-net: Two-stage multi-scale cascade and multi-field fusion U-net for right ventricular segmentation

Accurate segmentation of the right ventricle from cardiac magnetic resonance images (MRI) is a critical step in cardiac function analysis and disease diagnosis. It is still an open problem due to some difficulties, such as a large variety of object sizes and ill-defined borders. In this paper, we present a TSU-net network that grips deeper features and captures targets of different sizes with multi-scale cascade and multi-field fusion in the right ventricle. TSU-net mainly contains two major components: Dilated-Convolution Block (DB) and Multi-Layer-Pool Block (MB). DB extracts and aggregates multi-scale features for the right ventricle. MB mainly relies on multiple effective field-of-views to detect objects at different sizes and fill boundary features. Different from previous networks, we used DB and MB to replace the convolution layer in the encoding layer, thus, we can gather multi-scale information of right ventricle, detect different size targets and fill boundary information in each encoding layer. In addition, in the decoding layer, we used DB to replace the convolution layer, so that we can aggregate the multi-scale features of the right ventricle in each decoding layer. Furthermore, the two-stage U-net structure is used to further improve the utilization of DB and MB through a two-layer encoding/decoding layer. Our method is validated on the RVSC, a public right ventricular data set. The results demonstrated that TSU-net achieved an average Dice coefficient of 0.86 on endocardium and 0.90 on the epicardium, thereby outperforming other models. It effectively assists doctors to diagnose the disease and promotes the development of medical images. In addition, we also provide an intuitive explanation of our network, which fully explain MB and TSU-net's ability to detect targets of different sizes and fill in boundary features.

A Survey on Adversarial Deep Learning Robustness in Medical Image Analysis

In the past years, deep neural networks (DNN) have become popular in many disciplines such as computer vision (CV), natural language processing (NLP), etc. The evolution of hardware has helped researchers to develop many powerful Deep Learning (DL) models to face numerous challenging problems. One of the most important challenges in the CV area is Medical Image Analysis in which DL models process medical images—such as magnetic resonance imaging (MRI), X-ray, computed tomography (CT), etc.—using convolutional neural networks (CNN) for diagnosis or detection of several diseases. The proper function of these models can significantly upgrade the health systems. However, recent studies have shown that CNN models are vulnerable under adversarial attacks with imperceptible perturbations. In this paper, we summarize existing methods for adversarial attacks, detections and defenses on medical imaging. Finally, we show that many attacks, which are undetectable by the human eye, can degrade the performance of the models, significantly. Nevertheless, some effective defense and attack detection methods keep the models safe to an extent. We end with a discussion on the current state-of-the-art and future challenges.

Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences

Obtaining manual labels is time-consuming and labor-intensive on cardiac image sequences. Few-shot segmentation can utilize limited labels to learn new tasks. However, it suffers from two challenges: spatial-temporal distribution bias and long-term information bias. These challenges derive from the impact of the time dimension on cardiac image sequences, resulting in serious over-adaptation. In this paper, we propose the multi-level semantic adaptation (MSA) for few-shot segmentation on cardiac image sequences. The MSA addresses the two biases by exploring the domain adaptation and the weight adaptation on the semantic features in multiple levels, including sequence-level, frame-level, and pixel-level. First, the MSA proposes the dual-level feature adjustment for domain adaptation in spatial and temporal directions. This adjustment explicitly aligns the frame-level feature and the sequence-level feature to improve the model adaptation on diverse modalities. Second, the MSA explores the hierarchical attention metric for weight adaptation in the frame-level feature and the pixel-level feature. This metric focuses on the similar frame and the target region to promote the model discrimination on the border features. The extensive experiments demonstrate that our MSA is effective in few-shot segmentation on cardiac image sequences with three modalities, i.e. MR, CT, and Echo (e.g. the average Dice is 0.9243), as well as superior to the ten state-of-the-art methods.

Multiscale attention guided U-Net architecture for cardiac segmentation in short-axis MRI images

BACKGROUND AND OBJECTIVE

Automatic cardiac segmentation plays an utmost role in the diagnosis and quantification of cardiovascular diseases.

METHODS

This paper proposes a new cardiac segmentation method in short-axis Magnetic Resonance Imaging (MRI) images, called attention U-Net architecture with input image pyramid and deep supervised output layers (AID), which can fully-automatically learn to pay attention to target structures of various sizes and shapes. During each training process, the model continues to learn how to emphasize the desired features and suppress irrelevant areas in the original images, effectively improving the accuracy of cardiac segmentation. At the same time, we introduce the Focal Tversky Loss (FTL), which can effectively solve the problem of high imbalance in the amount of data between the target class and the background class during cardiac image segmentation. In order to obtain a better representation of intermediate features, we add a multi-scale input pyramid to the attention network.

RESULTS

The proposed cardiac segmentation technique is tested on the public Left Ventricle Segmentation Challenge (LVSC) dataset, which is shown to achieve 0.75, 0.87 and 0.92 for Jaccard Index, Sensitivity and Specificity, respectively. Experimental results demonstrate that the proposed method is able to improve the segmentation accuracy compared with the standard U-Net, and achieves comparable performance to the most advanced fully-automated methods.

CONCLUSIONS

Given its effectiveness and advantages, the proposed method can facilitate cardiac segmentation in short-axis MRI images in clinical practice.

Automated left atrial time-resolved segmentation in MRI long-axis cine images using active contours

BACKGROUND

Segmentation of the left atrium (LA) is required to evaluate atrial size and function, which are important imaging biomarkers for a wide range of cardiovascular conditions, such as atrial fibrillation, stroke, and diastolic dysfunction. LA segmentations are currently being performed manually, which is time-consuming and observer-dependent.

METHODS

This study presents an automated image processing algorithm for time-resolved LA segmentation in cardiac magnetic resonance imaging (MRI) long-axis cine images of the 2-chamber (2ch) and 4-chamber (4ch) views using active contours. The proposed algorithm combines mitral valve tracking, automated threshold calculation, edge detection on a radially resampled image, edge tracking based on Dijkstra's algorithm, and post-processing involving smoothing and interpolation. The algorithm was evaluated in 37 patients diagnosed mainly with paroxysmal atrial fibrillation. Segmentation accuracy was assessed using the Dice similarity coefficient (DSC) and Hausdorff distance (HD), with manual segmentations in all time frames as the reference standard. For inter-observer variability analysis, a second observer performed manual segmentations at end-diastole and end-systole on all subjects.

RESULTS

The proposed automated method achieved high performance in segmenting the LA in long-axis cine sequences, with a DSC of 0.96 for 2ch and 0.95 for 4ch, and an HD of 5.5 mm for 2ch and 6.4 mm for 4ch. The manual inter-observer variability analysis had an average DSC of 0.95 and an average HD of 4.9 mm.

CONCLUSION

The proposed automated method achieved performance on par with human experts analyzing MRI images for evaluation of atrial size and function. Video Abstract.

DBAN: Adversarial Network With Multi-Scale Features for Cardiac MRI Segmentation

With the development of medical artificial intelligence, automatic magnetic resonance image (MRI) segmentation method is quite desirable. Inspired by the power of deep neural networks, a novel deep adversarial network, dilated block adversarial network (DBAN), is proposed to perform left ventricle, right ventricle, and myocardium segmentation in short-axis cardiac MRI. DBAN contains a segmentor along with a discriminator. In the segmentor, the dilated block (DB) is proposed to capture, and aggregate multi-scale features. The segmentor can produce segmentation probability maps while the discriminator can differentiate the segmentation probability map, and the ground truth at the pixel level. In addition, confidence probability maps generated by the discriminator can guide the segmentor to modify segmentation probability maps. Extensive experiments demonstrate that DBAN has achieved the state-of-the-art performance on the ACDC dataset. Quantitative analyses indicate that cardiac function indices from DBAN are similar to those from clinical experts. Therefore, DBAN can be a potential candidate for short-axis cardiac MRI segmentation in clinical applications.

Automated left and right ventricular chamber segmentation in cardiac magnetic resonance images using dense fully convolutional neural network

BACKGROUND AND OBJECTIVE

Segmentation of the left ventricular (LV) myocardium (Myo) and RV endocardium on cine cardiac magnetic resonance (CMR) images represents an essential step for cardiac-function evaluation and diagnosis. In order to have a common reference for comparing segmentation algorithms, several CMR image datasets were made available, but in general they do not include the most apical and basal slices, and/or gold standard tracing is limited to only one of the two ventricles, thus not fully corresponding to real clinical practice. Our aim was to develop a deep learning (DL) approach for automated segmentation of both RV and LV chambers from short-axis (SAX) CMR images, reporting separately the performance for basal slices, together with the applied criterion of choice.

METHOD

A retrospectively selected database (DB1) of 210 cine sequences (3 pathology groups) was considered: images (GE, 1.5 T) were acquired at Centro Cardiologico Monzino (Milan, Italy), and end-diastolic (ED) and end-systolic frames (ES) were manually segmented (gold standard, GS). Automatic ED and ES RV and LV segmentation were performed with a U-Net inspired architecture, where skip connections were redesigned introducing dense blocks to alleviate the semantic gap between the U-Net encoder and decoder. The proposed architecture was trained including: A) the basal slices where the Myo surrounded the LV for at least the 50% and all the other slice; B) all the slices where the Myo completely surrounded the LV. To evaluate the clinical relevance of the proposed architecture in a practical use case scenario, a graphical user interface was developed to allow clinicians to revise, and correct when needed, the automatic segmentation. Additionally, to assess generalizability, analysis of CMR images obtained in 12 healthy volunteers (DB2) with different equipment (Siemens, 3T) and settings was performed.

RESULTS

The proposed architecture outperformed the original U-Net. Comparing the performance on DB1 between the two criteria, no significant differences were measured when considering all slices together, but were present when only basal slices were examined. Automatic and manually-adjusted segmentation performed similarly compared to the GS (bias±95%LoA): LVEDV -1±12 ml, LVESV -1±14 ml, RVEDV 6±12 ml, RVESV 6±14 ml, ED LV mass 6±26 g, ES LV mass 5±26 g). Also, generalizability showed very similar performance, with Dice scores of 0.944 (LV), 0.908 (RV) and 0.852 (Myo) on DB1, and 0.940 (LV), 0.880 (RV), and 0.856 (Myo) on DB2.

CONCLUSIONS

Our results support the potential of DL methods for accurate LV and RV contours segmentation and the advantages of dense skip connections in alleviating the semantic gap generated when high level features are concatenated with lower level feature. The evaluation on our dataset, considering separately the performance on basal and apical slices, reveals the potential of DL approaches for fast, accurate and reliable automated cardiac segmentation in a real clinical setting.

Deep learning based fully automatic segmentation of the left ventricular endocardium and epicardium from cardiac cine MRI

Background

The segmentation of cardiac medical images is a crucial step for calculating clinical indices such as wall thickness, ventricular volume, and ejection fraction.

Methods

In this study, we introduce a method named LsUnet that combines multi-channel, fully convolutional neural network, and annular shape level-set methods for efficiently segmenting cardiac cine magnetic resonance (MR) images. In this method, the multi-channel deep learning algorithm is applied to train the segmentation task to extract the left ventricle (LV) endocardial and epicardial contours. Next, the segmentation contours from the multi-channel deep learning method are incorporated into a level-set formulation, which is dedicated explicitly to detecting annular shapes to assure the segmentation's accuracy and robustness.

Results

The proposed automatic approach was evaluated on 95 volumes (total 1,076 slices, ~80% as for training datasets, ~20% 2D as for testing datasets). This combined multi-channel deep learning and annular shape level-set segmentation method achieved high accuracy with average Dice values reaching 92.15% and 95.42% for LV endocardium and epicardium delineation, respectively, in comparison to the reference standard (the manual segmentation).

Conclusions

A novel method for fully automatic segmentation of the LV endocardium and epicardium from different MRI datasets is presented. The proposed workflow is accurate and robust compared to the reference and other state-of-the-art methods.

A cascaded FC-DenseNet and level set method (FCDL) for fully automatic segmentation of the right ventricle in cardiac MRI

Accurate segmentation of the right ventricle (RV) from cardiac magnetic resonance imaging (MRI) images is an essential step in estimating clinical indices such as stroke volume and ejection fraction. Recently, image segmentation methods based on fully convolutional neural networks (FCN) have drawn much attention and shown promising results. In this paper, a new fully automatic RV segmentation method combining the FC-DenseNet and the level set method (FCDL) is proposed. The FC-DenseNet is efficiently trained end-to-end, using RV images and ground truth masks to make a per-pixel semantic inference. As a result, probability images are produced, followed by the level set method responsible for smoothing and converging contours to improve accuracy. It is noted that the iteration times of the level set method is only 4 times, which is due to the semantic segmentation of the FC-DenseNet for RV. Finally, multi-object detection algorithm is applied to locate the RV. Experimental results (including 45 cases, 15 cases for training, 30 cases for testing) show that the FCDL method outperforms the U-net + level set (UL) and the level set methods that use the same dataset and the cardiac functional parameters are computed robustly by the FCDL method. The results validate the FCDL method as an efficient and satisfactory approach to RV segmentation.

Analysis of right ventricular mass from magnetic resonance imaging data: a simple post-processing algorithm for correction of partial-volume effects

Magnetic resonance imaging (MRI) of the right ventricle (RV) offers important diagnostic information, but the accuracy of this information is hampered by the complex geometry of the RV. Here, we propose a novel postprocessing algorithm that corrects for partial-volume effects in the analysis of standard MRI cine images of RV mass (RVm) and evaluate the method in clinical and preclinical data. Self-corrected RVm measurement was compared with conventionally measured RVm in 16 patients who showed different clinical indications for cardiac MRI and in 17 Wistar rats with different degrees of pulmonary congestion. The rats were studied under isoflurane anaesthesia. To evaluate the reliability of the proposed method, the measured end-systolic and end-diastolic RVm were compared. Accuracy was evaluated by comparing preclinical RVm to ex vivo RV weight (RVw). We found that use of the self-correcting algorithm improved reliability compared with conventional segmentation. For clinical data, the limits of agreement (LOAs) were -1.8 ± 8.6g (self-correcting) vs. 5.8 ± 7.8g (conventional), and coefficients of variation (CoVs) were 7.0% (self-correcting) vs. 14.3% (conventional). For preclinical data, LOAs were 21 ± 46 mg (self-correcting) vs. 64 ± 89 mg (conventional), and CoVs were 9.0% (self-correcting) and 17.4% (conventional). Self-corrected RVm also showed better correspondence with the ex vivo RVw: LOAs were -5 ± 80 mg (self-correcting) vs. 94 ± 116 mg (conventional) in end-diastole and -26 ± 74 mg (self-correcting) vs. 31 ± 98 mg (conventional) in end-systole. The new self-correcting algorithm improves the reliability and accuracy of RVm measurements in both clinical and preclinical MRI. It is simple and easy to implement and does not require any additional MRI data.NEW & NOTEWORTHY Magnetic resonance imaging (MRI) of the right ventricle (RV) offers important diagnostic information, but the accuracy of this information is hampered by the complex geometry of the RV. In particular, the crescent shape of the RV renders it particularly vulnerable to partial-volume effects. We present a new, simple, self-correcting algorithm that can be applied to correct partial-volume effects in MRI-based RV mass estimation. The self-correcting algorithm offers improved reliability and accuracy compared with the conventional approach.