Learned iterative segmentation of highly variable anatomy from limited data: Applications to whole heart segmentation for congenital heart disease

Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization

Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.

Automated segmentation of 3D cine cardiovascular magnetic resonance imaging

Introduction

As the life expectancy of children with congenital heart disease (CHD) is rapidly increasing and the adult population with CHD is growing, there is an unmet need to improve clinical workflow and efficiency of analysis. Cardiovascular magnetic resonance (CMR) is a noninvasive imaging modality for monitoring patients with CHD. CMR exam is based on multiple breath-hold 2-dimensional (2D) cine acquisitions that should be precisely prescribed and is expert and institution dependent. Moreover, 2D cine images have relatively thick slices, which does not allow for isotropic delineation of ventricular structures. Thus, development of an isotropic 3D cine acquisition and automatic segmentation method is worthwhile to make CMR workflow straightforward and efficient, as the present work aims to establish.

Methods

Ninety-nine patients with many types of CHD were imaged using a non-angulated 3D cine CMR sequence covering the whole-heart and great vessels. Automatic supervised and semi-supervised deep-learning-based methods were developed for whole-heart segmentation of 3D cine images to separately delineate the cardiac structures, including both atria, both ventricles, aorta, pulmonary arteries, and superior and inferior vena cavae. The segmentation results derived from the two methods were compared with the manual segmentation in terms of Dice score, a degree of overlap agreement, and atrial and ventricular volume measurements.

Results

The semi-supervised method resulted in a better overlap agreement with the manual segmentation than the supervised method for all 8 structures (Dice score 83.23 ± 16.76% vs. 77.98 ± 19.64%; P-value ≤0.001). The mean difference error in atrial and ventricular volumetric measurements between manual segmentation and semi-supervised method was lower (bias ≤ 5.2 ml) than the supervised method (bias ≤ 10.1 ml).

Discussion

The proposed semi-supervised method is capable of cardiac segmentation and chamber volume quantification in a CHD population with wide anatomical variability. It accurately delineates the heart chambers and great vessels and can be used to accurately calculate ventricular and atrial volumes throughout the cardiac cycle. Such a segmentation method can reduce inter- and intra- observer variability and make CMR exams more standardized and efficient.

Graph matching and deep neural networks based whole heart and great vessel segmentation in congenital heart disease

Congenital heart disease (CHD) is one of the leading causes of mortality among birth defects, and due to significant variations in the whole heart and great vessel, automatic CHD segmentation using CT images has been always under-researched. Even though some segmentation algorithms have been developed in the literature, none perform very well under the complex structure of CHD. To deal with the challenges, we take advantage of deep learning in processing regular structures and graph algorithms in dealing with large variations and propose a framework combining both the whole heart and great vessel segmentation in complex CHD. We benefit from deep learning in segmenting the four chambers and myocardium based on the blood pool, and then we extract the connection information and apply graph matching to determine the categories of all the vessels. Experimental results on 68 3D CT images covering 14 types of CHD illustrate our framework can increase the Dice score by 12% on average compared with the state-of-the-art whole heart and great vessel segmentation method in normal anatomy. We further introduce two cardiovascular imaging specialists to evaluate our results in the standard of the Van Praagh classification system, and achieves well performance in clinical evaluation. All these results may pave the way for the clinical use of our method in the incoming future.

Non-contrast free-breathing whole-heart 3D cine cardiovascular magnetic resonance with a novel 3D radial leaf trajectory

PURPOSE

To develop and validate a non-contrast free-breathing whole-heart 3D cine steady-state free precession (SSFP) sequence with a novel 3D radial leaf trajectory.

METHODS

We used a respiratory navigator to trigger acquisition of 3D cine data at end-expiration to minimize respiratory motion in our 3D cine SSFP sequence. We developed a novel 3D radial leaf trajectory to reduce gradient jumps and associated eddy-current artifacts. We then reconstructed the 3D cine images with a resolution of 2.0mm³ using an iterative nonlinear optimization algorithm. Prospective validation was performed by comparing ventricular volumetric measurements from a conventional breath-hold 2D cine ventricular short-axis stack against the non-contrast free-breathing whole-heart 3D cine dataset in each patient (n = 13).

RESULTS

All 3D cine SSFP acquisitions were successful and mean scan time was 07:09 ± 01:31 min. End-diastolic ventricular volumes for left ventricle (LV) and right ventricle (RV) measured from the 3D datasets were smaller than those from 2D (LV: 159.99 ± 42.99 vs. 173.16 ± 47.42; RV: 180.35 ± 46.08 vs. 193.13 ± 49.38; p-value≤0.044; bias<8%), whereas ventricular end-systolic volumes were more comparable (LV: 79.12 ± 26.78 vs. 78.46 ± 25.35; RV: 97.18 ± 32.35 vs. 102.42 ± 32.53; p-value≥0.190, bias<6%). The 3D cine data had a lower subjective image quality score.

CONCLUSION

Our non-contrast free-breathing whole-heart 3D cine sequence with novel leaf trajectory was robust and yielded smaller ventricular end-diastolic volumes compared to 2D cine imaging. It has the potential to make examinations easier and more comfortable for patients.