Automatic model-based segmentation of the heart in CT images

An anthropomorphic phantom for atrial transseptal puncture simulation training

BACKGROUND

Transseptal puncture (TSP) is a critical prerequisite for left-sided cardiac interventions, such as atrial fibrillation (AF) ablation and left atrial appendage closure. Despite its routine nature, TSP can be technically demanding and carries a risk of complications. This study presents a novel, patient-specific, anthropomorphic phantom for TSP simulation training that can be used with X-ray fluoroscopy and ultrasound imaging.

METHODS

The TSP phantom was developed using additive manufacturing techniques and features a replaceable fossa ovalis (FO) component to allow for multiple punctures without replacing the entire model. Four cardiologists and one cardiology trainee performed TSP on the simulator, and their performance was assessed using four metrics: global isotropy index, distance from the centroid, time taken to perform TSP, and a set of 5-point Likert scale questions to evaluate the clinicians' perception of the phantom's realism and utility.

RESULTS

The results demonstrate the simulator's potential as a training tool for interventional cardiology, providing a realistic and controllable environment for clinicians to refine their TSP skills. Experienced cardiologists tended to cluster their puncture points closer to regions of the FO associated with higher global isotropy index scores, indicating a relationship between experience and optimal puncture localization. The questionnaire analysis revealed that participants generally agreed on the phantom's realistic anatomical representation and ability to accurately visualize the TSP site under fluoroscopic guidance.

CONCLUSIONS

The TSP simulator can be incorporated into training programs, offering trainees the opportunity to improve tool handling, spatial coordination, and manual dexterity prior to performing the procedure on patients. Further studies with larger sample sizes and longitudinal assessments are needed to establish the simulator's impact on TSP performance and patient outcomes.

Interactive segmentation with curve-based template deformation for spatiotemporal computed tomography of swallowing motion

Repeating X-ray computed tomography (CT) measurements over a short period of time allows for obtaining a spatiotemporal four-dimensional (4D) volume image. This study presents an interactive method for segmenting a 4DCT image by fitting a template model to a target organ. The template consists of a three-dimensional (3D) mesh model and free-form-deformation (FFD) cage enclosing the mesh. The user deforms the template by placing multiple curve constraints that specify the boundary shape of the template in 3D space. We also present curve constraints shared over all time frames and interpolated along the time axis to facilitate efficient curve specification. Our method formulates the template deformation using the FFD cage modification, allowing the user to switch between our curve-based method and traditional FFD at any time. To illustrate the feasibility of our method, we show segmentation results in which we could accurately segment three organs from a 4DCT image capturing a swallowing motion. To evaluate the usability of our method, we conducted a user study comparing our curve-based method with the cage-based FFD. We found that the participants finished segmentation in approximately 20% interaction time periods on average with our method.

Evaluation of a deep learning-enabled automated computational heart modelling workflow for personalized assessment of ventricular arrhythmias

Personalized, image-based computational heart modelling is a powerful technology that can be used to improve patient-specific arrhythmia risk stratification and ventricular tachycardia (VT) ablation targeting. However, most state-of-the-art methods still require manual interactions by expert users. The goal of this study is to evaluate the feasibility of an automated, deep learning-based workflow for reconstructing personalized computational electrophysiological heart models to guide patient-specific treatment of VT. Contrast-enhanced computed tomography (CE-CT) images with expert ventricular myocardium segmentations were acquired from 111 patients across five cohorts from three different institutions. A deep convolutional neural network (CNN) for segmenting left ventricular myocardium from CE-CT was developed, trained and evaluated. From both CNN-based and expert segmentations in a subset of patients, personalized electrophysiological heart models were reconstructed and rapid pacing was used to induce VTs. CNN-based and expert segmentations were more concordant in the middle myocardium than in the heart's base or apex. Wavefront propagation during pacing was similar between CNN-based and original heart models. Between most sets of heart models, VT inducibility was the same, the number of induced VTs was strongly correlated, and VT circuits co-localized. Our results demonstrate that personalized computational heart models reconstructed from deep learning-based segmentations even with a small training set size can predict similar VT inducibility and circuit locations as those from expertly-derived heart models. Hence, a user-independent, automated framework for simulating arrhythmias in personalized heart models could feasibly be used in clinical settings to aid VT risk stratification and guide VT ablation therapy. KEY POINTS: Personalized electrophysiological heart modelling can aid in patient-specific ventricular tachycardia (VT) risk stratification and VT ablation targeting. Current state-of-the-art, image-based heart models for VT prediction require expert-dependent, manual interactions that may not be accessible across clinical settings. In this study, we develop an automated, deep learning-based workflow for reconstructing personalized heart models capable of simulating arrhythmias and compare its predictions with that of expert-generated heart models. The number and location of VTs was similar between heart models generated from the deep learning-based workflow and expert-generated heart models. These results demonstrate the feasibility of using an automated computational heart modelling workflow to aid in VT therapeutics and has implications for generalizing personalized computational heart technology to a broad range of clinical centres.

HVSMR-2.0: A 3D cardiovascular MR dataset for whole-heart segmentation in congenital heart disease

Patients with congenital heart disease often have cardiac anatomy that deviates significantly from normal, frequently requiring multiple heart surgeries. Image segmentation from a preoperative cardiovascular magnetic resonance (CMR) scan would enable creation of patient-specific 3D surface models of the heart, which have potential to improve surgical planning, enable surgical simulation, and allow automatic computation of quantitative metrics of heart function. However, there is no publicly available CMR dataset for whole-heart segmentation in patients with congenital heart disease. Here, we release the HVSMR-2.0 dataset, comprising 60 CMR scans alongside manual segmentation masks of the 4 cardiac chambers and 4 great vessels. The images showcase a wide range of heart defects and prior surgical interventions. The dataset also includes masks of required and optional extents of the great vessels, enabling fairer comparisons across algorithms. Detailed diagnoses for each subject are also provided. By releasing HVSMR-2.0, we aim to encourage development of robust segmentation algorithms and clinically relevant tools for congenital heart disease.

Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentation

Accurate segmentation of multiple organs in the head, neck, chest, and abdomen from medical images is an essential step in computer-aided diagnosis, surgical navigation, and radiation therapy. In the past few years, with a data-driven feature extraction approach and end-to-end training, automatic deep learning-based multi-organ segmentation methods have far outperformed traditional methods and become a new research topic. This review systematically summarizes the latest research in this field. We searched Google Scholar for papers published from January 1, 2016 to December 31, 2023, using keywords "multi-organ segmentation" and "deep learning", resulting in 327 papers. We followed the PRISMA guidelines for paper selection, and 195 studies were deemed to be within the scope of this review. We summarized the two main aspects involved in multi-organ segmentation: datasets and methods. Regarding datasets, we provided an overview of existing public datasets and conducted an in-depth analysis. Concerning methods, we categorized existing approaches into three major classes: fully supervised, weakly supervised and semi-supervised, based on whether they require complete label information. We summarized the achievements of these methods in terms of segmentation accuracy. In the discussion and conclusion section, we outlined and summarized the current trends in multi-organ segmentation.

Shape-constrained deformable brain segmentation: Methods and quantitative validation

MRI-guided neuro interventions require rapid, accurate, and reproducible segmentation of anatomical brain structures for identification of targets during surgical procedures and post-surgical evaluation of intervention efficiency. Segmentation algorithms must be validated and cleared for clinical use. This work introduces a methodology for shape-constrained deformable brain segmentation, describes the quantitative validation used for its clinical clearance, and presents a comparison with manual expert segmentation and FreeSurfer, an open source software for neuroimaging data analysis. ClearPoint Maestro is software for fully-automatic brain segmentation from T1-weighted MRI that combines a shape-constrained deformable brain model with voxel-wise tissue segmentation within the cerebral hemispheres and the cerebellum. The performance of the segmentation was validated in terms of accuracy and reproducibility. Segmentation accuracy was evaluated with respect to training data and independently traced ground truth. Segmentation reproducibility was quantified and compared with manual expert segmentation and FreeSurfer. Quantitative reproducibility analysis indicates superior performance compared to both manual expert segmentation and FreeSurfer. The shape-constrained methodology results in accurate and highly reproducible segmentation. Inherent point based-correspondence provides consistent target identification ideal for MRI-guided neuro interventions.

Semi-supervised auto-segmentation method for pelvic organ-at-risk in magnetic resonance images based on deep-learning

BACKGROUND AND PURPOSE

In radiotherapy, magnetic resonance (MR) imaging has higher contrast for soft tissues compared to computed tomography (CT) scanning and does not emit radiation. However, manual annotation of the deep learning-based automatic organ-at-risk (OAR) delineation algorithms is expensive, making the collection of large-high-quality annotated datasets a challenge. Therefore, we proposed the low-cost semi-supervised OAR segmentation method using small pelvic MR image annotations.

METHODS

We trained a deep learning-based segmentation model using 116 sets of MR images from 116 patients. The bladder, femoral heads, rectum, and small intestine were selected as OAR regions. To generate the training set, we utilized a semi-supervised method and ensemble learning techniques. Additionally, we employed a post-processing algorithm to correct the self-annotation data. Both 2D and 3D auto-segmentation networks were evaluated for their performance. Furthermore, we evaluated the performance of semi-supervised method for 50 labeled data and only 10 labeled data.

RESULTS

The Dice similarity coefficient (DSC) of the bladder, femoral heads, rectum and small intestine between segmentation results and reference masks is 0.954, 0.984, 0.908, 0.852 only using self-annotation and post-processing methods of 2D segmentation model. The DSC of corresponding OARs is 0.871, 0.975, 0.975, 0.783, 0.724 using 3D segmentation network, 0.896, 0.984, 0.890, 0.828 using 2D segmentation network and common supervised method.

CONCLUSION

The outcomes of our study demonstrate that it is possible to train a multi-OAR segmentation model using small annotation samples and additional unlabeled data. To effectively annotate the dataset, ensemble learning and post-processing methods were employed. Additionally, when dealing with anisotropy and limited sample sizes, the 2D model outperformed the 3D model in terms of performance.

MWG-UNet: Hybrid Deep Learning Framework for Lung Fields and Heart Segmentation in Chest X-ray Images

Deep learning technology has achieved breakthrough research results in the fields of medical computer vision and image processing. Generative adversarial networks (GANs) have demonstrated a capacity for image generation and expression ability. This paper proposes a new method called MWG-UNet (multiple tasking Wasserstein generative adversarial network U-shape network) as a lung field and heart segmentation model, which takes advantages of the attention mechanism to enhance the segmentation accuracy of the generator so as to improve the performance. In particular, the Dice similarity, precision, and F1 score of the proposed method outperform other models, reaching 95.28%, 96.41%, and 95.90%, respectively, and the specificity surpasses the sub-optimal models by 0.28%, 0.90%, 0.24%, and 0.90%. However, the value of the IoU is inferior to the optimal model by 0.69%. The results show the proposed method has considerable ability in lung field segmentation. Our multi-organ segmentation results for the heart achieve Dice similarity and IoU values of 71.16% and 74.56%. The segmentation results on lung fields achieve Dice similarity and IoU values of 85.18% and 81.36%.

Left atrium volume and ventricular volume ratio algorithm as indication of pulmonary hypertension etiology

BACKGROUND

Pressure overload of the right heart (pulmonary hypertension [PH]) can be an acute or a chronic process with various pathophysiologic changes affecting the dimensions of the heart chambers. The automatic four-chamber volumetric analysis tool is now available to measure the volume of the cardiac chambers in patients undergoing a computed tomography pulmonary angiogram (CTPA).

PURPOSE

To characterize the volumetric changes that occurred in response to increased systolic pulmonary arterial pressures (sPAP) in acute events, such as acute pulmonary embolism (APE), compared with other etiologies.

MATERIAL AND METHODS

Consecutive patients who underwent CTPA and echocardiography within 24 h between 2011 and 2015 were included. Differences in cardiac chamber volumes were investigated in correlation to the patients' sPAP.

RESULTS

The final cohort of 961 patients included 221 (23%) patients diagnosed with APE. The right (RV) to left (LV) ventricular volume ratio (VVR) was higher, while the left atrial (LA) volume index was smaller (P < 0.001) in the patients with APE. A decision tree for the prediction of APE showed that an RV to left VVR >2.8 was characteristic of APE, whereas an LA volume index >37.5 mL/m² was more compatible with PH due to other etiologies (P < 0.001).

CONCLUSION

The combination of VVR and LA volume index may help in differentiating between APE and chronic PH. CTPA-based volumetric information may be used to help clarify the underlying etiology of the dyspnea.

Automatic Segmentation of Organs-at-Risk in Thoracic Computed Tomography Images Using Ensembled U-Net InceptionV3 Model

The objective of this article is to automatically segment organs at risk (OARs) for thoracic radiology in computed tomography (CT) scan images. The OARs in the thoracic anatomical region during the radiotherapy treatment are mainly the neighbouring organs such as the esophagus, heart, trachea, and aorta. The dataset of 40 patients was used in the proposed work by splitting it into three parts: training, validation, and test sets. The implementation was performed on the Google Colab Pro+ framework with 52 GB of RAM and 265 GB of storage space. An ensemble model was evolved for the automatic segmentation of four OARs in thoracic CT images. U-Net with InceptionV3 as the backbone was used, and different hyperparameters were used during the training of the model. The proposed model achieved precise accuracy for OARs segmentation with an average dice coefficient of 0.9413, Hausdorff value of 0.1838, sensitivity of 0.9783, and specificity of 0.9895 on the Test dataset. An ensembled U-Net InceptionV3 model has been proposed, improving the segmentation results compared with the state-of-the-art techniques such as U-Net, ResNet, Vgg16, etc. The results of the experiments revealed that the proposed model effectively improved the performance of the segmentation of the esophagus, heart, trachea, and aorta.

ST-Unet: Swin Transformer boosted U-Net with Cross-Layer Feature Enhancement for medical image segmentation

Medical image segmentation is an essential task in clinical diagnosis and case analysis. Most of the existing methods are based on U-shaped convolutional neural networks (CNNs), and one of disadvantages is that the long-term dependencies and global contextual connections cannot be effectively established, which results in inaccuracy segmentation. For fully using low-level features to enhance global features and reduce the semantic gap between encoding and decoding stages, we propose a novel Swin Transformer boosted U-Net (ST-Unet) for medical image processing in this paper, in which Swin Transformer and CNNs are used as encoder and decoder respectively. Then a novel Cross-Layer Feature Enhancement (CLFE) module is proposed to realize cross-layer feature learning, and a Spatial and Channel Squeeze & Excitation module is adopted to highlight the saliency of specific regions. Finally, we learn the features fused by the CLFE module through CNNs to recover low-level features and localize local features for realizing more accurate semantic segmentation. Experiments on widely used public datasets Synapse and ISIC 2018 prove that our proposed ST-Unet can achieve 78.86 of dice and 0.9243 of recall performance, outperforming most current medical image segmentation methods.

Weakly supervised inference of personalized heart meshes based on echocardiography videos

Echocardiography provides recordings of the heart chamber size and function and is a central tool for non-invasive diagnosis of heart diseases. It produces high-dimensional video data with substantial stochasticity in the measurements, which frequently prove difficult to interpret. To address this challenge, we propose an automated framework to enable the inference of a high resolution personalized 4D (3D plus time) surface mesh of the cardiac structures from 2D echocardiography video data. Inferring such shape models arises as a key step towards accurate personalized simulation that enables an automated assessment of the cardiac chamber morphology and function. The proposed method is trained using only unpaired echocardiography and heart mesh videos to find a mapping between these distinct visual domains in a self-supervised manner. The resulting model produces personalized 4D heart meshes, which exhibit a high correspondence with the input echocardiography videos. Furthermore, the 4D heart meshes enable the automatic extraction of echocardiographic variables, such as ejection fraction, myocardial muscle mass, and volumetric changes of chamber volumes over time with high temporal resolution.

Male pelvic multi-organ segmentation using token-based transformer Vnet

Objective. This work aims to develop an automated segmentation method for the prostate and its surrounding organs-at-risk in pelvic computed tomography to facilitate prostate radiation treatment planning.Approach. In this work, we propose a novel deep learning algorithm combining a U-shaped convolutional neural network (CNN) and vision transformer (VIT) for multi-organ (i.e. bladder, prostate, rectum, left and right femoral heads) segmentation in male pelvic CT images. The U-shaped model consists of three components: a CNN-based encoder for local feature extraction, a token-based VIT for capturing global dependencies from the CNN features, and a CNN-based decoder for predicting the segmentation outcome from the VIT's output. The novelty of our network is a token-based multi-head self-attention mechanism used in the transformer, which encourages long-range dependencies and forwards informative high-resolution feature maps from the encoder to the decoder. In addition, a knowledge distillation strategy is deployed to further enhance the learning capability of the proposed network.Main results. We evaluated the network using: (1) a dataset collected from 94 patients with prostate cancer; (2) and a public dataset CT-ORG. A quantitative evaluation of the proposed network's performance was performed on each organ based on (1) volume similarity between the segmented contours and ground truth using Dice score, segmentation sensitivity, and precision, (2) surface similarity evaluated by Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS), (3) and percentage volume difference (PVD). The performance was then compared against other state-of-art methods. Average volume similarity measures obtained by the network overall organs were Dice score = 0.91, sensitivity = 0.90, precision = 0.92, average surface similarities were HD = 3.78 mm, MSD = 1.24 mm, RMS = 2.03 mm; average percentage volume difference was PVD = 9.9% on the first dataset. The network also obtained Dice score = 0.93, sensitivity = 0.93, precision = 0.93, average surface similarities were HD = 5.82 mm, MSD = 1.16 mm, RMS = 1.24 mm; average percentage volume difference was PVD = 6.6% on the CT-ORG dataset.Significance. In summary, we propose a token-based transformer network with knowledge distillation for multi-organ segmentation using CT images. This method provides accurate and reliable segmentation results for each organ using CT imaging, facilitating the prostate radiation clinical workflow.

CAN3D: Fast 3D medical image segmentation via compact context aggregation

Direct automatic segmentation of objects in 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying multiple individual structures with complex geometries within a large volume under investigation. Most deep learning approaches address these challenges by enhancing their learning capability through a substantial increase in trainable parameters within their models. An increased model complexity will incur high computational costs and large memory requirements unsuitable for real-time implementation on standard clinical workstations, as clinical imaging systems typically have low-end computer hardware with limited memory and CPU resources only. This paper presents a compact convolutional neural network (CAN3D) designed specifically for clinical workstations and allows the segmentation of large 3D Magnetic Resonance (MR) images in real-time. The proposed CAN3D has a shallow memory footprint to reduce the number of model parameters and computer memory required for state-of-the-art performance and maintain data integrity by directly processing large full-size 3D image input volumes with no patches required. The proposed architecture significantly reduces computational costs, especially for inference using the CPU. We also develop a novel loss function with extra shape constraints to improve segmentation accuracy for imbalanced classes in 3D MR images. Compared to state-of-the-art approaches (U-Net3D, improved U-Net3D and V-Net), CAN3D reduced the number of parameters up to two orders of magnitude and achieved much faster inference, up to 5 times when predicting with a standard commercial CPU (instead of GPU). For the open-access OAI-ZIB knee MR dataset, in comparison with manual segmentation, CAN3D achieved Dice coefficient values of (mean = 0.87 ± 0.02 and 0.85 ± 0.04) with mean surface distance errors (mean = 0.36 ± 0.32 mm and 0.29 ± 0.10 mm) for imbalanced classes such as (femoral and tibial) cartilage volumes respectively when training volume-wise under only 12G video memory. Similarly, CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 examinations with expert manual semantic labels (bladder, body, bone, rectum, prostate) now released publicly for scientific use as part of this work.

Learned iterative segmentation of highly variable anatomy from limited data: Applications to whole heart segmentation for congenital heart disease

Training deep learning models that segment an image in one step typically requires a large collection of manually annotated images that captures the anatomical variability in a cohort. This poses challenges when anatomical variability is extreme but training data is limited, as when segmenting cardiac structures in patients with congenital heart disease (CHD). In this paper, we propose an iterative segmentation model and show that it can be accurately learned from a small dataset. Implemented as a recurrent neural network, the model evolves a segmentation over multiple steps, from a single user click until reaching an automatically determined stopping point. We develop a novel loss function that evaluates the entire sequence of output segmentations, and use it to learn model parameters. Segmentations evolve predictably according to growth dynamics encapsulated by training data, which consists of images, partially completed segmentations, and the recommended next step. The user can easily refine the final segmentation by examining those that are earlier or later in the output sequence. Using a dataset of 3D cardiac MR scans from patients with a wide range of CHD types, we show that our iterative model offers better generalization to patients with the most severe heart malformations.

Diagnostic accuracy of automated 3D volumetry of cardiac chambers by CT pulmonary angiography for identification of pulmonary hypertension due to left heart disease

OBJECTIVES

To assess diagnostic accuracy of automated 3D volumetry of cardiac chambers based on computed tomography pulmonary angiography (CTPA) for the differentiation of pulmonary hypertension due to left heart disease (group 2 PH) from non-group 2 PH compared to manual diameter measurements.

METHODS

Patients with confirmed PH undergoing right heart catheterisation and CTPA within 100 days for diagnostic workup of PH between August 2013 and February 2016 were included in this retrospective, single-centre study. Automated 3D segmentation of left atrium, left ventricle, right atrium and right ventricle (LA/LV/RA/RV) was performed by two independent and blinded radiologists using commercial software. For comparison, axial diameters were manually measured. The ability to differentiate group 2 PH from non-group 2 PH was assessed by means of logistic regression.

RESULTS

Ninety-one patients (median 67.5 years, 44 women) were included, thereof 19 patients (20.9%) classified as group 2 PH. After adjustment for age, sex and mean pulmonary arterial pressure, group 2 PH was significantly associated with larger LA volume (p < 0.001), larger LV volume (p = 0.001), lower RV/LV volume ratio (p = 0.04) and lower RV/LA volume ratio (p = 0.003). LA volume demonstrated the highest discriminatory ability to identify group 2 PH (AUC, 0.908; 95% confidence interval, 0.835-0.981) and was significantly superior to LA diameter (p = 0.009). Intraobserver and interobserver agreements were excellent for all volume measurements (intraclass correlation coefficients 0.926-0.999, all p < 0.001).

CONCLUSIONS

LA volume quantified by automated, CTPA-based 3D volumetry can differentiate group 2 PH from other PH groups with good diagnostic accuracy and yields significantly higher diagnostic accuracy than left atrial diameter.

KEY POINTS

• Automated cardiac chamber volumetry using non-gated CT pulmonary angiography can differentiate pulmonary hypertension due to left heart disease from other causes with good diagnostic accuracy. • Left atrial volume yields significantly higher diagnostic accuracy than left atrial axial diameter for identification of pulmonary hypertension due to left heart disease without time-consuming manual processing.

Utility of Automated Cardiac Chamber Volumetry by Non-Gated CT Pulmonary Angiography for Detection of Pulmonary Hypertension Using the 2018 Updated Hemodynamic Definition

BACKGROUND: Noninvasive tests for pulmonary hypertension (PH) are needed to help select patients for diagnostic right heart catheterization (RHC). CT pulmonary angiography (CTPA) is commonly performed for suspected PH. OBJECTIVE: To assess the utility of CTPA-based cardiac chamber volumetric measurements for diagnosis of PH in comparison with echocardiographic and conventional CTPA parameters, using as reference the 2018 updated hemodynamic definition. METHODS: This retrospective study included 109 patients (median age, 68 years; 72 women, 37 men) who underwent non-gated CTPA, echocardiography, and RHC for workup of suspected PH between August 2013 and February 2016. Two radiologists independently used automated 3D segmentation software to determine volumes of the right ventricle (RV), right atrium (RA), left ventricle (LV), and left atrium (LA), and measured axial diameters of cardiac chambers, main pulmonary artery, and ascending aorta. Interobserver agreement was assessed, and mean values were obtained; one observer repeated volumetric measurements to assess intraobserver agreement. ROC analysis was used to assess diagnostic performance for detection of PH. A multivariable binary logistic regression model was established. RESULTS: A total of 60/109 patients had PH. Intra- and interobserver agreement were excellent for all volume measurements (intraclass correlation coefficients, 0.935-0.999). In patients with, versus without, PH, RV volume was 172.6 versus 118.1 ml, and RA volume was 130.2 versus 77.0 ml (both p<.05). Cardiac chamber measurements with highest AUC for PH were RV/LV volume ratio and RA volume (both 0.791). Significant predictors of PH after adjustment for age, sex, and body surface area included RV volume per 10 ml [odds ratio (OR)=1.21], RA volume per 10 ml (OR=1.27), RV/LV volume ratio (OR=2.91), and RA/LA volume ratio (OR=11.22). Regression analysis yielded a predictive model for PH containing two independent predictors, echocardiographic pulmonary arterial systolic pressure and CTPA-based RA volume; the model had AUC 0.898, sensitivity 83.3%, and specificity 85.7%. CONCLUSION: Automated cardiac chamber volumetry using non-gated CTPA, particularly of the RA, provides incremental utility relative to echocardiographic and conventional CTPA parameters for diagnosis of PH. CLINICAL IMPACT: Automated cardiac chamber volumetry on CTPA may facilitate early nonvinvasive detection of PH, identifying patients warranting further evaluation by RHC.

Deep learning-based whole-heart segmentation in 4D contrast-enhanced cardiac CT

Automatic cardiac chamber and left ventricular (LV) myocardium segmentation over the cardiac cycle significantly extends the utilization of contrast-enhanced cardiac CT, potentially enabling in-depth assessment of cardiac function. Therefore, we evaluate an automatic method for cardiac chamber and LV myocardium segmentation in 4D cardiac CT. In this study, 4D contrast-enhanced cardiac CT scans of 1509 patients selected for transcatheter aortic valve implantation with 21,605 3D images, were divided into development (N = 12) and test set (N = 1497). 3D convolutional neural networks were trained with end-systolic (ES) and end-diastolic (ED) images. Dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) were computed for 3D segmentations at ES and ED in the development set via cross-validation, and for 2D segmentations in four cardiac phases for 81 test set patients. Segmentation quality in the full test set of 1497 patients was assessed visually on a three-point scale per structure based on estimated overlap with the ground truth. Automatic segmentation resulted in a mean DSC of 0.89 ± 0.10 and ASSD of 1.43 ± 1.45 mm in 12 patients in 3D, and a DSC of 0.89 ± 0.08 and ASSD of 1.86 ± 1.20 mm in 81 patients in 2D. The qualitative evaluation in the whole test set of 1497 patients showed that automatic segmentations were assigned grade 1 (clinically useful) in 98.5%, 92.2%, 83.1%, 96.3%, and 91.6% of cases for LV cavity and myocardium, right ventricle, left atrium, and right atrium. Our automatic method using convolutional neural networks performed clinically useful segmentation across the cardiac cycle in a large set of 4D cardiac CT images, potentially enabling in-depth assessment of cardiac function.

A deep-learning approach for direct whole-heart mesh reconstruction

Automated construction of surface geometries of cardiac structures from volumetric medical images is important for a number of clinical applications. While deep-learning-based approaches have demonstrated promising reconstruction precision, these approaches have mostly focused on voxel-wise segmentation followed by surface reconstruction and post-processing techniques. However, such approaches suffer from a number of limitations including disconnected regions or incorrect surface topology due to erroneous segmentation and stair-case artifacts due to limited segmentation resolution. We propose a novel deep-learning-based approach that directly predicts whole heart surface meshes from volumetric CT and MR image data. Our approach leverages a graph convolutional neural network to predict deformation on mesh vertices from a pre-defined mesh template to reconstruct multiple anatomical structures in a 3D image volume. Our method demonstrated promising performance of generating whole heart reconstructions with as good or better accuracy than prior deep-learning-based methods on both CT and MR data. Furthermore, by deforming a template mesh, our method can generate whole heart geometries with better anatomical consistency and produce high-resolution geometries from lower resolution input image data. Our method was also able to produce temporally-consistent surface mesh predictions for heart motion from CT or MR cine sequences, and therefore can potentially be applied for efficiently constructing 4D whole heart dynamics. Our code and pre-trained networks are available at https://github.com/fkong7/MeshDeformNet.

Heart Region Segmentation using Dense VNet from Multimodality Images

Cardiovascular diseases (CVD) have been identified as one of the most common causes of death in the world. Advanced development of imaging techniques is allowing timely detection of CVD and helping physicians in providing correct treatment plans in saving lives. Segmentation and Identification of various substructures of the heart are very important in modeling a digital twin of the patient-specific heart. Manual delineation of various substructures of the heart is tedious and time-consuming. Here we have implemented Dense VNet for detecting substructures of the heart from both CT and MRI multimodality data. Due to the limited availability of data we have implemented an on-the-fly elastic deformation data augmentation technique. The result of the proposed has been shown to outperform other methods reported in the literature on both CT and MRI datasets.

MA-SOCRATIS: An automatic pipeline for robust segmentation of the left ventricle and scar

Multi-atlas segmentation of cardiac regions and total infarct scar (MA-SOCRATIS) is an unsupervised automatic pipeline to segment left ventricular myocardium and scar from late gadolinium enhanced MR images (LGE-MRI) of the heart. We implement two different pipelines for myocardial and scar segmentation from short axis LGE-MRI. Myocardial segmentation has two steps; initial segmentation and re-estimation. The initial segmentation step makes a first estimate of myocardium boundaries by using multi-atlas segmentation techniques. The re-estimation step refines the myocardial segmentation by a combination of k-means clustering and a geometric median shape variation technique. An active contour technique determines the unhealthy and healthy myocardial wall. The scar segmentation pipeline is a combination of a Rician-Gaussian mixture model and full width at half maximum (FWHM) thresholding, to determine the intensity pixels in scar regions. Following this step a watershed method with an automatic seed-points framework segments the final scar region. MA-SOCRATIS was evaluated using two different datasets. In both datasets ground truths were based on manual segmentation of short axis images from LGE-MRI scans. The first dataset included 40 patients from the MS-CMRSeg 2019 challenge dataset (STACOM at MICCAI 2019). The second is a collection of 20 patients with scar regions that are challenging to segment. MA-SOCRATIS achieved robust and accurate performance in automatic segmentation of myocardium and scar regions without the need of training or tuning in both cohorts, compared with state-of-the-art techniques (intra-observer and inter observer myocardium segmentation: 81.9% and 70% average Dice value, and scar (intra-observer and inter observer segmentation: 70.5% and 70.5% average Dice value).

The impact of shape uncertainty on aortic-valve pressure-drop computations

Patient-specific image-based computational fluid dynamics (CFD) is widely adopted in the cardiovascular research community to study hemodynamics, and will become increasingly important for personalized medicine. However, segmentation of the flow domain is not exact and geometric uncertainty can be expected which propagates through the computational model, leading to uncertainty in model output. Seventy-four aortic-valves were segmented from computed tomography images at peak systole. Statistical shape modeling was used to obtain an approximate parameterization of the original segmentations. This parameterization was used to train a meta-model that related the first five shape mode coefficients and flowrate to the CFD-computed transvalvular pressure-drop. Consequently, shape uncertainty in the order of 0.5 and 1.0 mm was emulated by introducing uncertainty in the shape mode coefficients. A global variance-based sensitivity analysis was performed to quantify output uncertainty and to determine relative importance of the shape modes. The first shape mode captured the opening/closing behavior of the valve and uncertainty in this mode coefficient accounted for more than 90% of the output variance. However, sensitivity to shape uncertainty is patient-specific, and the relative importance of the fourth shape mode coefficient tended to increase with increases in valvular area. These results show that geometric uncertainty in the order of image voxel size may lead to substantial uncertainty in CFD-computed transvalvular pressure-drops. Moreover, this illustrates that it is essential to assess the impact of geometric uncertainty on model output, and that this should be thoroughly quantified for applications that wish to use image-based CFD models.

A bi-atrial statistical shape model for large-scale in silico studies of human atria: Model development and application to ECG simulations

Large-scale electrophysiological simulations to obtain electrocardiograms (ECG) carry the potential to produce extensive datasets for training of machine learning classifiers to, e.g., discriminate between different cardiac pathologies. The adoption of simulations for these purposes is limited due to a lack of ready-to-use models covering atrial anatomical variability. We built a bi-atrial statistical shape model (SSM) of the endocardial wall based on 47 segmented human CT and MRI datasets using Gaussian process morphable models. Generalization, specificity, and compactness metrics were evaluated. The SSM was applied to simulate atrial ECGs in 100 random volumetric instances. The first eigenmode of our SSM reflects a change of the total volume of both atria, the second the asymmetry between left vs. right atrial volume, the third a change in the prominence of the atrial appendages. The SSM is capable of generalizing well to unseen geometries and 95% of the total shape variance is covered by its first 24 eigenvectors. The P waves in the 12-lead ECG of 100 random instances showed a duration of 109.7±12.2 ms in accordance with large cohort studies. The novel bi-atrial SSM itself as well as 100 exemplary instances with rule-based augmentation of atrial wall thickness, fiber orientation, inter-atrial bridges and tags for anatomical structures have been made publicly available. This novel, openly available bi-atrial SSM can in future be employed to generate large sets of realistic atrial geometries as a basis for in silico big data approaches.

A Tensor-based Catheter and Wire Detection and Tracking Framework and Its Clinical Applications

OBJECTIVE

Catheters and wires are used extensively in cardiac catheterization procedures. Detecting their positions in fluoroscopic X-ray images is important for several clinical applications such as motion compensation and co-registration between 2D and 3D imaging modalities. Detecting the complete length of a catheter or wire object as well as electrode positions on the catheter or wire is a challenging task.

METHOD

In this paper, an automatic detection framework for catheters and wires is developed. It is based on path reconstruction from image tensors, which are eigen direction vectors generated from a multiscale vessel enhancement filter. A catheter or a wire object is detected as the smooth path along those eigen direction vectors. Furthermore, a real-time tracking method based on a template generated from the detection method was developed.

RESULTS

The proposed framework was tested on a total of 7,754 X-ray images. Detection errors for catheters and guidewires are 0.56 0.28 mm and 0.68 0.33 mm, respectively. The proposed framework was also tested and validated in two clinical applications. For motion compensation using catheter tracking, the 2D target registration errors (TRE) of 1.8 mm 0.9 mm was achieved. For co-registration between 2D X-ray images and 3D models from MRI images, a TRE of 2.3 0.9 mm was achieved.

CONCLUSION

A novel and fully automatic detection framework and its clinical applications are developed.

SIGNIFICANCE

The proposed framework can be applied to improve the accuracy of image-guidance systems for cardiac catheterization procedures.

Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation

BACKGROUND

A novel Generative Adversarial Networks (GAN) based bidirectional cross-modality unsupervised domain adaptation (GBCUDA) framework is developed for cardiac image segmentation, which can effectively tackle the problem of network's segmentation performance degradation when adapting to the target domain without ground truth labels.

METHOD

GBCUDA uses GAN for image alignment, applies adversarial learning to extract image features, and gradually enhances the domain invariance of extracted features. The shared encoder performs an end-to-end learning task in which features that differ between the two domains complement each other. The self-attention mechanism is incorporated to the GAN network, which can generate details based on the prompts of all feature positions. Furthermore, spectrum normalization is implemented to stabilize the training of GAN, and knowledge distillation loss is introduced to process high-level feature-maps in order to better complete the cross-mode segmentation task.

RESULTS

The effectiveness of our proposed unsupervised domain adaptation framework is tested over the Multi-Modality Whole Heart Segmentation (MM-WHS) Challenge 2017 dataset. The proposed method is able to improve the average Dice from 74.1% to 81.5% for the four cardiac substructures, and reduce the average symmetric surface distance (ASD) from 7.0 to 5.8 over CT images. For MRI images, our proposed framework trained on CT images gives the average Dice of 59.2% and reduces the average ASD from 5.7 to 4.9.

CONCLUSIONS

The evaluation results demonstrate our method's effectiveness on domain adaptation and the superiority to the current state-of-the-art domain adaptation methods.

Correlation between CT-derived cardiac chamber volume, myocardial injury and mortality in acute pulmonary embolism

INTRODUCTION

The release of troponin in patients with acute pulmonary embolism (PE) is assumed to be secondary to elevated intracardiac chamber pressure. Since right ventricular (RV) hypertrophy does not develop in the acute setting, pressure overload correlates with chamber dilatation, causing myocardial injury. The aim of the present study was to investigate correlations between cardiac chamber volume, troponin and subsequent early mortality in patients with acute PE and refine risk stratification.

MATERIALS AND METHODS

Patients who underwent a computerized tomographic pulmonary angiogram (CTPA) and a troponin test within 24 h of the CTPA were included. Automated software calculated the volumes of the four cardiac chambers indexed to body surface area (BSA) and correlated them to troponin and early all-cause mortality.

RESULTS

The final cohort consisted of 370 patients (56% females) with acute PE. RV volume and right to left ventricular volume ratio (VVR) were the most significant indicators for elevated troponin (receiving operating characteristic [ROC] 0.796, confidence interval [CI]: 0.749-0.843, p < 0.001, and ROC 0.802, CI: 0.753-0.851, p < 0.001, respectively). VVR cutoff values, which are predictive of elevated troponins, correlated with higher 30-day mortality (odds ratio = 3.1, CI 1.5-6.7, p = 0.003) for a VVR >3 compared to a VVR <2.

CONCLUSION

Cardiac chamber volume correlates to elevated troponin in patients with acute PE. A higher VVR reflects an increased likelihood for myocardial ischemia, as well as an increased short-term mortality risk. These data are available seconds after CTPA performance and may contribute to refining patients' risk stratification.

Review of Deep Learning Based Automatic Segmentation for Lung Cancer Radiotherapy

Lung cancer is the leading cause of cancer-related mortality for males and females. Radiation therapy (RT) is one of the primary treatment modalities for lung cancer. While delivering the prescribed dose to tumor targets, it is essential to spare the tissues near the targets-the so-called organs-at-risk (OARs). An optimal RT planning benefits from the accurate segmentation of the gross tumor volume and surrounding OARs. Manual segmentation is a time-consuming and tedious task for radiation oncologists. Therefore, it is crucial to develop automatic image segmentation to relieve radiation oncologists of the tedious contouring work. Currently, the atlas-based automatic segmentation technique is commonly used in clinical routines. However, this technique depends heavily on the similarity between the atlas and the image segmented. With significant advances made in computer vision, deep learning as a part of artificial intelligence attracts increasing attention in medical image automatic segmentation. In this article, we reviewed deep learning based automatic segmentation techniques related to lung cancer and compared them with the atlas-based automatic segmentation technique. At present, the auto-segmentation of OARs with relatively large volume such as lung and heart etc. outperforms the organs with small volume such as esophagus. The average Dice similarity coefficient (DSC) of lung, heart and liver are over 0.9, and the best DSC of spinal cord reaches 0.9. However, the DSC of esophagus ranges between 0.71 and 0.87 with a ragged performance. In terms of the gross tumor volume, the average DSC is below 0.8. Although deep learning based automatic segmentation techniques indicate significant superiority in many aspects compared to manual segmentation, various issues still need to be solved. We discussed the potential issues in deep learning based automatic segmentation including low contrast, dataset size, consensus guidelines, and network design. Clinical limitations and future research directions of deep learning based automatic segmentation were discussed as well.

Head and neck multi-organ segmentation on dual-energy CT using dual pyramid convolutional neural networks

Organ delineation is crucial to diagnosis and therapy, while it is also labor-intensive and observer-dependent. Dual energy CT (DECT) provides additional image contrast than conventional single energy CT (SECT), which may facilitate automatic organ segmentation. This work aims to develop an automatic multi-organ segmentation approach using deep learning for head-and-neck region on DECT. We proposed a mask scoring regional convolutional neural network (R-CNN) where comprehensive features are firstly learnt from two independent pyramid networks and are then combined via deep attention strategy to highlight the informative ones extracted from both two channels of low and high energy CT. To perform multi-organ segmentation and avoid misclassification, a mask scoring subnetwork was integrated into the Mask R-CNN framework to build the correlation between the class of potential detected organ's region-of-interest (ROI) and the shape of that organ's segmentation within that ROI. We evaluated our model on DECT images from 127 head-and-neck cancer patients (66 training, 61 testing) with manual contours of 19 organs as training target and ground truth. For large- and mid-sized organs such as brain and parotid, the proposed method successfully achieved average Dice similarity coefficient (DSC) larger than 0.8. For small-sized organs with very low contrast such as chiasm, cochlea, lens and optic nerves, the DSCs ranged between around 0.5 and 0.8. With the proposed method, using DECT images outperforms using SECT in almost all 19 organs with statistical significance in DSC (p<0.05). Meanwhile, by using the DECT, the proposed method is also significantly superior to a recently developed FCN-based method in most of organs in terms of DSC and the 95th percentile Hausdorff distance. Quantitative results demonstrated the feasibility of the proposed method, the superiority of using DECT to SECT, and the advantage of the proposed R-CNN over FCN on the head-and-neck patient study. The proposed method has the potential to facilitate the current head-and-neck cancer radiation therapy workflow in treatment planning.

A review of deep learning based methods for medical image multi-organ segmentation

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.

Male pelvic CT multi-organ segmentation using synthetic MRI-aided dual pyramid networks

The delineation of the prostate and organs-at-risk (OARs) is fundamental to prostate radiation treatment planning, but is currently labor-intensive and observer-dependent. We aimed to develop an automated computed tomography (CT)-based multi-organ (bladder, prostate, rectum, left and right femoral heads (RFHs)) segmentation method for prostate radiation therapy treatment planning. The proposed method uses synthetic MRIs (sMRIs) to offer superior soft-tissue information for male pelvic CT images. Cycle-consistent adversarial networks (CycleGAN) were used to generate CT-based sMRIs. Dual pyramid networks (DPNs) extracted features from both CTs and sMRIs. A deep attention strategy was integrated into the DPNs to select the most relevant features from both CTs and sMRIs to identify organ boundaries. The CT-based sMRI generated from our previously trained CycleGAN and its corresponding CT images were inputted to the proposed DPNs to provide complementary information for pelvic multi-organ segmentation. The proposed method was trained and evaluated using datasets from 140 patients with prostate cancer, and were then compared against state-of-art methods. The Dice similarity coefficients and mean surface distances between our results and ground truth were 0.95 ± 0.05, 1.16 ± 0.70 mm; 0.88 ± 0.08, 1.64 ± 1.26 mm; 0.90 ± 0.04, 1.27 ± 0.48 mm; 0.95 ± 0.04, 1.08 ± 1.29 mm; and 0.95 ± 0.04, 1.11 ± 1.49 mm for bladder, prostate, rectum, left and RFHs, respectively. Mean center of mass distances was within 3 mm for all organs. Our results performed significantly better than those of competing methods in most evaluation metrics. We demonstrated the feasibility of sMRI-aided DPNs for multi-organ segmentation on pelvic CT images, and its superiority over other networks. The proposed method could be used in routine prostate cancer radiotherapy treatment planning to rapidly segment the prostate and standard OARs.

Detection of severe pulmonary hypertension based on computed tomography pulmonary angiography

Pulmonary hypertension (PH) is often diagnosed late in the disease course. As many patients may undergo computed tomography pulmonary angiography (CTPA) for exclusion of pulmonary embolism (PE), we aimed to create a model that can detect the existence of PH and grade its severity. Consecutive patients who underwent CTPA which was negative for PE, and echocardiography study within 24 h, were included. The CT parameters evaluated to assess PH were: the diameters of the main pulmonary artery (MPA), ascending aorta (AA), calculation of each heart chamber volume, and the severity of reflux of contrast material. Randomly, 70% of patients were included in the model creation group, and 30% were used to validate the model. The final study group included 740 patients, 268 male patients, median age 72 years. 374 patients (51%) had PH, of them 94 (13%) had severe PH on the echocardiography. Right atrium (RA) and Left atrium (LA) volume indices were the strongest parameter to indicate PH (area under the curve, AUC = 0.738 and 0.736, respectively), while Right ventricle (RV) and RA volume indices were the strongest parameter to identify severe PH (AUC = 0.735 and 0.715, respectively) with MPA diameter being the least influential indicator (AUC = 0.623). Using the patients age, gender, and multiple CTPA parameters, we created a model for predicting the existence of severe PH. After validation, the model demonstrated 91% sensitivity and a negative predictive value of 97%. Applying our models, CTPA can be used to identify severe PH immediately after the completion of CTPA exam.

Automatic Fetal Middle Sagittal Plane Detection in Ultrasound Using Generative Adversarial Network

Background and Objective: In the first trimester of pregnancy, fetal growth, and abnormalities can be assessed using the exact middle sagittal plane (MSP) of the fetus. However, the ultrasound (US) image quality and operator experience affect the accuracy. We present an automatic system that enables precise fetal MSP detection from three-dimensional (3D) US and provides an evaluation of its performance using a generative adversarial network (GAN) framework. Method: The neural network is designed as a filter and generates masks to obtain the MSP, learning the features and MSP location in 3D space. Using the proposed image analysis system, a seed point was obtained from 218 first-trimester fetal 3D US volumes using deep learning and the MSP was automatically extracted. Results: The experimental results reveal the feasibility and excellent performance of the proposed approach between the automatically and manually detected MSPs. There was no significant difference between the semi-automatic and automatic systems. Further, the inference time in the automatic system was up to two times faster than the semi-automatic approach. Conclusion: The proposed system offers precise fetal MSP measurements. Therefore, this automatic fetal MSP detection and measurement approach is anticipated to be useful clinically. The proposed system can also be applied to other relevant clinical fields in the future.

Fully automated left atrium segmentation from anatomical cine long-axis MRI sequences using deep convolutional neural network with unscented Kalman filter

This study proposes a fully automated approach for the left atrial segmentation from routine cine long-axis cardiac magnetic resonance image sequences using deep convolutional neural networks and Bayesian filtering. The proposed approach consists of a classification network that automatically detects the type of long-axis sequence and three different convolutional neural network models followed by unscented Kalman filtering (UKF) that delineates the left atrium. Instead of training and predicting all long-axis sequence types together, the proposed approach first identifies the image sequence type as to 2, 3 and 4 chamber views, and then performs prediction based on neural nets trained for that particular sequence type. The datasets were acquired retrospectively and ground truth manual segmentation was provided by an expert radiologist. In addition to neural net based classification and segmentation, another neural net is trained and utilized to select image sequences for further processing using UKF to impose temporal consistency over cardiac cycle. A cyclic dynamic model with time-varying angular frequency is introduced in UKF to characterize the variations in cardiac motion during image scanning. The proposed approach was trained and evaluated separately with varying amount of training data with images acquired from 20, 40, 60 and 80 patients. Evaluations over 1515 images with equal number of images from each chamber group acquired from an additional 20 patients demonstrated that the proposed model outperformed state-of-the-art and yielded a mean Dice coefficient value of 94.1%, 93.7% and 90.1% for 2, 3 and 4-chamber sequences, respectively, when trained with datasets from 80 patients.

Combining statistical shape modeling, CFD, and meta-modeling to approximate the patient-specific pressure-drop across the aortic valve in real-time

BACKGROUND

Advances in medical imaging, segmentation techniques, and high performance computing have stimulated the use of complex, patient-specific, three-dimensional Computational Fluid Dynamics (CFD) simulations. Patient-specific, CFD-compatible geometries of the aortic valve are readily obtained. CFD can then be used to obtain the patient-specific pressure-flow relationship of the aortic valve. However, such CFD simulations are computationally expensive, and real-time alternatives are desired.

AIM

The aim of this work is to evaluate the performance of a meta-model with respect to high-fidelity, three-dimensional CFD simulations of the aortic valve.

METHODS

Principal component analysis was used to build a statistical shape model (SSM) from a population of 74 iso-topological meshes of the aortic valve. Synthetic meshes were created with the SSM, and steady-state CFD simulations at flow-rates between 50 and 650 mL/s were performed to build a meta-model. The meta-model related the statistical shape variance, and flow-rate to the pressure-drop.

RESULTS

Even though the first three shape modes account for only 46% of shape variance, the features relevant for the pressure-drop seem to be captured. The three-mode shape-model approximates the pressure-drop with an average error of 8.8% to 10.6% for aortic valves with a geometric orifice area below 150 mm² . The proposed methodology was least accurate for aortic valve areas above 150 mm² . Further reduction to a meta-model introduces an additional 3% error.

CONCLUSIONS

Statistical shape modeling can be used to capture shape variation of the aortic valve. Meta-models trained by SSM-based CFD simulations can provide an estimate of the pressure-flow relationship in real-time.

Automatic estimation of aortic and mitral valve displacements in dynamic CTA with 4D graph-cuts

The location of the mitral and aortic valves in dynamic cardiac imaging is useful for extracting functional derived parameters such as ejection fraction, valve excursions, and global longitudinal strain, and when performing anatomical structures tracking using slice following or valve intervention's planning. Completely automatic segmentation methods are still challenging tasks because of their fast movements and the different positions that prevent good visibility of the leaflets along the full cardiac cycle. In this article, we propose a processing pipeline to track the displacement of the aortic and mitral valve annuli from high-resolution cardiac four-dimensional computed tomographic angiography (4D-CTA). The proposed method is based on the dynamic separation of left ventricle, left atrium and aorta using statistical shape modeling and an energy minimization algorithm based on graph-cuts and has been evaluated on a set of 15 electrocardiography-gated 4D-CTAs. We report a mean agreement distance between manual annotations and our proposed method of 2.52±1.06 mm for the mitral annulus and 2.00±0.69 mm for the aortic valve annulus based on valve locations detected from manual anatomical landmarks. In addition, we show the effect of detecting the valvular planes on derived functional parameters (ejection fraction, global longitudinal strain, and excursions of the mitral and aortic valves).

Cardiac Gated Computed Tomography Angiography Discloses a Correlation Between the Volumes of All Four Cardiac Chambers and Heart Rate in Men But Not in Women

Background: Currently, normal values of the cardiac chambers' volumes are adjusted only for gender and body surface area (BSA). We aim to investigate the association between the heart rate and the volume of each of the four cardiac chambers using cardiac-gated computed tomography angiography (CCTA).

Methods: A total of 350 consecutive patients without known cardiac diseases or significant (>50%) stenosis undergoing CCTA between January 2009 and June 2014 for suspected coronary artery disease were included. Cardiac chamber volumes adjusted to BSA were calculated using automated model-based segmentation analysis software of the CCTA data and correlated with patients' mean heart rate during the scan.

Results: There were 240 men and 110 women, median interquartile range age was 55 years (47–61). Women were older 59.0 years (53.7–64) versus 52.0 years (45.0–59.0), had higher prevalence of hyperlipidemia, diabetes mellitus, anemia, and hypothyroidism, and higher median heart rates 64.0 (59.7–66.0) versus 60.0 (55.0–65.0) (p < 0.001). Men had a negative correlation between the volume of each cardiac chamber and the heart rate [r_{age_adj} = (−0.4)–(−0.27), p < 0.001 for all], whereas such a correlation was not found in women. The multivariate analysis showed that a decrease of five beats per minute was associated with an increase of 4%–5% in volume of each chamber in men. There was no such association among females.

Conclusions: Lower heart rate is associated with an increase of each cardiac chamber volume by CCTA in men. This association is not found in women. More extensive studies are required to further elaborate on these gender differences.

Impact of right ventricular volumes on the outcomes of TAVR: a volumetric analysis of preprocedural computed tomography

AIMS

The aim of this study was to assess the prognostic implications of increased right ventricle volume index (RVVI) using cardiac-gated computed tomography angiography (CCTA) data among patients undergoing transcatheter valve replacement (TAVR).

METHODS AND RESULTS

CCTA of 323 patients who underwent TAVR at Stanford University Medical Center (CA, USA) and Tel Aviv Medical Center (Israel) between 2013 and 2016 was analysed by an automatic four-chamber volumetric software and grouped into quartiles according to RVVI. Higher one-year mortality rates were noted for the upper quartiles - 5%, 4.9%, 8.6%, and 16% (p=0.039), in Q1 <59 ml/m2, Q2 59-69 ml/m2, Q3 69-86 ml/m2, and Q4 >86 ml/m2, respectively. However, the differences were not significant after propensity score adjustments. Sub-analyses of Q1 demonstrated an escalating risk for one-year mortality in concordance to RVVI: HR 2.28, HR 2.76, and HR 4.7, for the upper 25th, 15th, and 5th percentiles, respectively (p<0.05 for all comparisons). After propensity score adjustments for clinical and echocardiographic characteristics, only the upper 5th percentiles (RVVI >120 ml/m2) retained statistical significance (HR 2.82, 95% CI: 1.02-7.78, p=0.045). Notably, 68.7% of patients from this group were considered low-intermediate risk for surgery.

CONCLUSIONS

Cardiac volumetric data by CCTA performed for procedural planning may help to predict outcome in patients undergoing TAVR.

Nextmed: Automatic Imaging Segmentation, 3D Reconstruction, and 3D Model Visualization Platform Using Augmented and Virtual Reality

The visualization of medical images with advanced techniques, such as augmented reality and virtual reality, represent a breakthrough for medical professionals. In contrast to more traditional visualization tools lacking 3D capabilities, these systems use the three available dimensions. To visualize medical images in 3D, the anatomical areas of interest must be segmented. Currently, manual segmentation, which is the most commonly used technique, and semi-automatic approaches can be time consuming because a doctor is required, making segmentation for each individual case unfeasible. Using new technologies, such as computer vision and artificial intelligence for segmentation algorithms and augmented and virtual reality for visualization techniques implementation, we designed a complete platform to solve this problem and allow medical professionals to work more frequently with anatomical 3D models obtained from medical imaging. As a result, the Nextmed project, due to the different implemented software applications, permits the importation of digital imaging and communication on medicine (dicom) images on a secure cloud platform and the automatic segmentation of certain anatomical structures with new algorithms that improve upon the current research results. A 3D mesh of the segmented structure is then automatically generated that can be printed in 3D or visualized using both augmented and virtual reality, with the designed software systems. The Nextmed project is unique, as it covers the whole process from uploading dicom images to automatic segmentation, 3D reconstruction, 3D visualization, and manipulation using augmented and virtual reality. There are many researches about application of augmented and virtual reality for medical image 3D visualization; however, they are not automated platforms. Although some other anatomical structures can be studied, we focused on one case: a lung study. Analyzing the application of the platform to more than 1000 dicom images and studying the results with medical specialists, we concluded that the installation of this system in hospitals would provide a considerable improvement as a tool for medical image visualization.

Automated segmentation of abdominal organs from contrast-enhanced computed tomography using analysis of texture features

Generation of three-dimensional personalized geometric models of anatomical structures is an important process for many practical tasks: computer-aided diagnosis, treatment planning and numerical modeling in biomedical applications. Despite many efforts done by different research groups, automatic segmentation of organs still does not have any general solution. The main difficulties are caused by peculiarities of different medical imaging modalities, image variability (for the same modality) resulting from the wide range of imaging devices, noise and artifacts, large patient anatomical variability and overlapping of intensity ranges of neighboring anatomical structures. In this article, we propose segmentation method based on analysis of texture features and developed specially for segmentation of abdominal organs. Its main advantage is robustness to interpatient gray level and anatomical variability. The proposed method was validated on the patient data. The method implementation was accelerated using graphics processing unit (GPU).

Simultaneous Multi-Structure Segmentation of the Heart and Peripheral Tissues in Contrast Enhanced Cardiac Computed Tomography Angiography

Contrast enhanced cardiac computed tomography angiography (CTA) is a prominent imaging modality for diagnosing cardiovascular diseases non-invasively. It assists the evaluation of the coronary artery patency and provides a comprehensive assessment of structural features of the heart and great vessels. However, physicians are often required to evaluate different cardiac structures and measure their size manually. Such task is very time-consuming and tedious due to the large number of image slices in 3D data. We present a fully automatic method based on a combined multi-atlas and corrective segmentation approach to label the heart and its associated cardiovascular structures. This method also automatically separates other surrounding intrathoracic structures from CTA images. Quantitative assessment of the proposed method is performed on 36 studies with a reference standard obtained from expert manual segmentation of various cardiac structures. Qualitative evaluation is also performed by expert readers to score 120 studies of the automatic segmentation. The quantitative results showed an overall Dice of 0.93, Hausdorff distance of 7.94 mm, and mean surface distance of 1.03 mm between automatically and manually segmented cardiac structures. The visual assessment also attained an excellent score for the automatic segmentation. The average processing time was 2.79 minutes. Our results indicate the proposed automatic framework significantly improves accuracy and computational speed in conventional multi-atlas based approach, and it provides comprehensive and reliable multi-structural segmentation of CTA images that is valuable for clinical application.

Anatomically consistent CNN-based segmentation of organs-at-risk in cranial radiotherapy

Planning of radiotherapy involves accurate segmentation of a large number of organs at risk (OAR), i.e., organs for which irradiation doses should be minimized to avoid important side effects of the therapy. We propose a deep learning method for segmentation of OAR inside the head, from magnetic resonance images (MRIs). Our system performs segmentation of eight structures: eye, lens, optic nerve, optic chiasm, pituitary gland, hippocampus, brainstem, and brain. We propose an efficient algorithm to train neural networks for an end-to-end segmentation of multiple and nonexclusive classes, addressing problems related to computational costs and missing ground truth segmentations for a subset of classes. We enforce anatomical consistency of the result in a postprocessing step. In particular, we introduce a graph-based algorithm for segmentation of the optic nerves, enforcing the connectivity between the eyes and the optic chiasm. We report cross-validated quantitative results on a database of 44 contrast-enhanced T1-weighted MRIs with provided segmentations of the considered OAR, which were originally used for radiotherapy planning. In addition, the segmentations produced by our model on an independent test set of 50 MRIs were evaluated by an experienced radiotherapist in order to qualitatively assess their accuracy. The mean distances between produced segmentations and the ground truth ranged from 0.1 to 0.7 mm across different organs. A vast majority (96%) of the produced segmentations were found acceptable for radiotherapy planning.

Automatic segmentation of cardiac structures for breast cancer radiotherapy

•

We developed an automatic method to segment cardiac sub-structures for radiotherapy planning CTs.

•

The Dice Similarity Coefficients and Average Surface Distance were up to 97% and < 11 mm, respectively.

•

The whole heart showed the absolute dose difference < 0.3 Gy whereas the coronary arteries showed < 2.3 Gy in breast radiotherapy simulations.

•

No notable improvement in our method beyond 10 atlases and using the manual guide points.

Background and purpose

We developed an automatic method to segment cardiac substructures given a radiotherapy planning CT images to support epidemiological studies or clinical trials looking at cardiac disease endpoints after radiotherapy.

Material and methods

We used a most-similar atlas selection algorithm and 3D deformation combined with 30 detailed cardiac atlases. We cross-validated our method within the atlas library by evaluating geometric comparison metrics and by comparing cardiac doses for simulated breast radiotherapy between manual and automatic contours. We analyzed the impact of the number of cardiac atlas in the library and the use of manual guide points on the performance of our method.

Results

The Dice Similarity Coefficients from the cross-validation reached up to 97% (whole heart) and 80% (chambers). The Average Surface Distance for the coronary arteries was less than 10.3 mm on average, with the best agreement (7.3 mm) in the left anterior descending artery (LAD). The dose comparison for simulated breast radiotherapy showed differences less than 0.06 Gy for the whole heart and atria, and 0.3 Gy for the ventricles. For the coronary arteries, the dose differences were 2.3 Gy (LAD) and 0.3 Gy (other arteries). The sensitivity analysis showed no notable improvement beyond ten atlases and the manual guide points does not significantly improve performance.

Conclusion

We developed an automated method to contour cardiac substructures for radiotherapy CTs. When combined with accurate dose calculation techniques, our method should be useful for cardiac dose reconstruction of a large number of patients in epidemiological studies or clinical trials.

Synthetic MRI-aided multi-organ segmentation on male pelvic CT using cycle consistent deep attention network

BACKGROUND AND PURPOSE

Manual contouring is labor intensive, and subject to variations in operator knowledge, experience and technique. This work aims to develop an automated computed tomography (CT) multi-organ segmentation method for prostate cancer treatment planning.

METHODS AND MATERIALS

The proposed method exploits the superior soft-tissue information provided by synthetic MRI (sMRI) to aid the multi-organ segmentation on pelvic CT images. A cycle generative adversarial network (CycleGAN) was used to estimate sMRIs from CT images. A deep attention U-Net (DAUnet) was trained on sMRI and corresponding multi-organ contours for auto-segmentation. The deep attention strategy was introduced to identify the most relevant features to differentiate different organs. Deep supervision was incorporated into the DAUnet to enhance the features' discriminative ability. Segmented contours of a patient were obtained by feeding CT image into the trained CycleGAN to generate sMRI, which was then fed to the trained DAUnet to generate organ contours. We trained and evaluated our model with 140 datasets from prostate patients.

RESULTS

The Dice similarity coefficient and mean surface distance between our segmented and bladder, prostate, and rectum manual contours were 0.95 ± 0.03, 0.52 ± 0.22 mm; 0.87 ± 0.04, 0.93 ± 0.51 mm; and 0.89 ± 0.04, 0.92 ± 1.03 mm, respectively.

CONCLUSION

We proposed a sMRI-aided multi-organ automatic segmentation method on pelvic CT images. By integrating deep attention and deep supervision strategy, the proposed network provides accurate and consistent prostate, bladder and rectum segmentation, and has the potential to facilitate routine prostate-cancer radiotherapy treatment planning.