Patient-specific mitral leaflet segmentation from 4D ultrasound

Segmentation of Tricuspid Valve Leaflets From Transthoracic 3D Echocardiograms of Children With Hypoplastic Left Heart Syndrome Using Deep Learning

Hypoplastic left heart syndrome (HLHS) is a severe congenital heart defect in which the right ventricle and associated tricuspid valve (TV) alone support the circulation. TV failure is thus associated with heart failure, and the outcome of TV valve repair are currently poor. 3D echocardiography (3DE) can generate high-quality images of the valve, but segmentation is necessary for precise modeling and quantification. There is currently no robust methodology for rapid TV segmentation, limiting the clinical application of these technologies to this challenging population. We utilized a Fully Convolutional Network (FCN) to segment tricuspid valves from transthoracic 3DE. We trained on 133 3DE image-segmentation pairs and validated on 28 images. We then assessed the effect of varying inputs to the FCN using Mean Boundary Distance (MBD) and Dice Similarity Coefficient (DSC). The FCN with the input of an annular curve achieved a median DSC of 0.86 [IQR: 0.81-0.88] and MBD of 0.35 [0.23-0.4] mm for the merged segmentation and an average DSC of 0.77 [0.73-0.81] and MBD of 0.6 [0.44-0.74] mm for individual TV leaflet segmentation. The addition of commissural landmarks improved individual leaflet segmentation accuracy to an MBD of 0.38 [0.3-0.46] mm. FCN-based segmentation of the tricuspid valve from transthoracic 3DE is feasible and accurate. The addition of an annular curve and commissural landmarks improved the quality of the segmentations with MBD and DSC within the range of human inter-user variability. Fast and accurate FCN-based segmentation of the tricuspid valve in HLHS may enable rapid modeling and quantification, which in the future may inform surgical planning. We are now working to deploy this network for public use.

In Vivo Image-Based 4D Modeling of Competent and Regurgitant Mitral Valve Dynamics

BACKGROUND

In vivo characterization of mitral valve dynamics relies on image analysis algorithms that accurately reconstruct valve morphology and motion from clinical images. The goal of such algorithms is to provide patient-specific descriptions of both competent and regurgitant mitral valves, which can be used as input to biomechanical analyses and provide insights into the pathophysiology of diseases like ischemic mitral regurgitation (IMR).

OBJECTIVE

The goal is to generate accurate image-based representations of valve dynamics that visually and quantitatively capture normal and pathological valve function.

METHODS

We present a novel framework for 4D segmentation and geometric modeling of the mitral valve in real-time 3D echocardiography (rt-3DE), an imaging modality used for pre-operative surgical planning of mitral interventions. The framework integrates groupwise multi-atlas label fusion and template-based medial modeling with Kalman filtering to generate quantitatively descriptive and temporally consistent models of valve dynamics.

RESULTS

The algorithm is evaluated on rt-3DE data series from 28 patients: 14 with normal mitral valve morphology and 14 with severe IMR. In these 28 data series that total 613 individual 3DE images, each 3D mitral valve segmentation is validated against manual tracing, and temporal consistency between segmentations is demonstrated.

CONCLUSIONS

Automated 4D image analysis allows for reliable non-invasive modeling of the mitral valve over the cardiac cycle for comparison of annular and leaflet dynamics in pathological and normal mitral valves. Future studies can apply this algorithm to cardiovascular mechanics applications, including patient-specific strain estimation, fluid dynamics simulation, inverse finite element analysis, and risk stratification for surgical treatment.

Temporal Views of Flattened Mitral Valve Geometries

The mitral valve, one of the four valves in the human heart, controls the bloodflow between the left atrium and ventricle and may suffer from various pathologies. Malfunctioning valves can be treated by reconstructive surgeries, which have to be carefully planned and evaluated. While current research focuses on the modeling and segmentation of the valve, we base our work on existing segmentations of patient-specific mitral valves, that are also time-resolved ( 3D+t) over the cardiac cycle. The interpretation of the data can be ambiguous, due to the complex surface of the valve and multiple time steps. We therefore propose a software prototype to analyze such 3D+t data, by extracting pathophysiological parameters and presenting them via dimensionally reduced visualizations. For this, we rely on an existing algorithm to unroll the convoluted valve surface towards a flattened 2D representation. In this paper, we show that the 3D+t data can be transferred to 3D or 2D representations in a way that allows the domain expert to faithfully grasp important aspects of the cardiac cycle. In this course, we not only consider common pathophysiological parameters, but also introduce new observations that are derived from landmarks within the segmentation model. Our analysis techniques were developed in collaboration with domain experts and a survey showed that the insights have the potential to support mitral valve diagnosis and the comparison of the pre- and post-operative condition of a patient.

Combining position-based dynamics and gradient vector flow for 4D mitral valve segmentation in TEE sequences

PURPOSE

For planning and guidance of minimally invasive mitral valve repair procedures, 3D+t transesophageal echocardiography (TEE) sequences are acquired before and after the intervention. The valve is then visually and quantitatively assessed in selected phases. To enable a quantitative assessment of valve geometry and pathological properties in all heart phases, as well as the changes achieved through surgery, we aim to provide a new 4D segmentation method.

METHODS

We propose a tracking-based approach combining gradient vector flow (GVF) and position-based dynamics (PBD). An open-state surface model of the valve is propagated through time to the closed state, attracted by the GVF field of the leaflet area. The PBD method ensures topological consistency during deformation. For evaluation, one expert in cardiac surgery annotated the closed-state leaflets in 10 TEE sequences of patients with normal and abnormal mitral valves, and defined the corresponding open-state models.

RESULTS

The average point-to-surface distance between the manual annotations and the final tracked model was [Formula: see text]. Qualitatively, four cases were satisfactory, five passable and one unsatisfactory. Each sequence could be segmented in 2-6 min.

CONCLUSION

Our approach enables to segment the mitral valve in 4D TEE image data with normal and pathological valve closing behavior. With this method, in addition to the quantification of the remaining orifice area, shape and dimensions of the coaptation zone can be analyzed and considered for planning and surgical result assessment.

Signal dropout correction-based ultrasound segmentation for diastolic mitral valve modeling

Three-dimensional ultrasound segmentation of mitral valve (MV) at diastole is helpful for duplicating geometry and pathology in a patient-specific dynamic phantom. The major challenge is the signal dropout at leaflet regions in transesophageal echocardiography image data. Conventional segmentation approaches suffer from missing sonographic data leading to inaccurate MV modeling at leaflet regions. This paper proposes a signal dropout correction-based ultrasound segmentation method for diastolic MV modeling. The proposed method combines signal dropout correction, image fusion, continuous max-flow segmentation, and active contour segmentation techniques. The signal dropout correction approach is developed to recover the missing segmentation information. Once the signal dropout regions of TEE image data are recovered, the MV model can be accurately duplicated. Compared with other methods in current literature, the proposed algorithm exhibits lower computational cost. The experimental results show that the proposed algorithm gives competitive results for diastolic MV modeling compared with conventional segmentation algorithms, evaluated in terms of accuracy and efficiency.

Spatiotemporal Segmentation and Modeling of the Mitral Valve in Real-Time 3D Echocardiographic Images

Transesophageal echocardiography is the primary imaging modality for preoperative assessment of mitral valves with ischemic mitral regurgitation (IMR). While there are well known echocardiographic insights into the 3D morphology of mitral valves with IMR, such as annular dilation and leaflet tethering, less is understood about how quantification of valve dynamics can inform surgical treatment of IMR or predict short-term recurrence of the disease. As a step towards filling this knowledge gap, we present a novel framework for 4D segmentation and geometric modeling of the mitral valve in real-time 3D echocardiography (rt-3DE). The framework integrates multi-atlas label fusion and template-based medial modeling to generate quantitatively descriptive models of valve dynamics. The novelty of this work is that temporal consistency in the rt-3DE segmentations is enforced during both the segmentation and modeling stages with the use of groupwise label fusion and Kalman filtering. The algorithm is evaluated on rt-3DE data series from 10 patients: five with normal mitral valve morphology and five with severe IMR. In these 10 data series that total 207 individual 3DE images, each 3DE segmentation is validated against manual tracing and temporal consistency between segmentations is demonstrated. The ultimate goal is to generate accurate and consistent representations of valve dynamics that can both visually and quantitatively provide insight into normal and pathological valve function.

Modelling mitral valvular dynamics-current trend and future directions

Dysfunction of mitral valve causes morbidity and premature mortality and remains a leading medical problem worldwide. Computational modelling aims to understand the biomechanics of human mitral valve and could lead to the development of new treatment, prevention and diagnosis of mitral valve diseases. Compared with the aortic valve, the mitral valve has been much less studied owing to its highly complex structure and strong interaction with the blood flow and the ventricles. However, the interest in mitral valve modelling is growing, and the sophistication level is increasing with the advanced development of computational technology and imaging tools. This review summarises the state-of-the-art modelling of the mitral valve, including static and dynamics models, models with fluid-structure interaction, and models with the left ventricle interaction. Challenges and future directions are also discussed.

The geometric model of the human mitral valve

The mitral valve, which lies between the left atrium and the left ventricle, plays an important role in controlling the uniflux of blood from the left atrium to the left ventricle as one of the four human heart valves. A precise description of the shape of human mitral valve has vital significance in studying its physiological structure and periodic movement. Unsatisfyingly, there is almost no unified mathematical description of the same shape of human mitral valve in literature. In this paper, we present a geometric model for human mitral valve, as an elastic shell with a special shape. Parametric equations for the shape of human mitral valve are provided, including the anterior and the posterior parts, which can be thought as portions of two interfacing semi-elliptic cylindrical shells. The minor axis of one ellipse is equal to the major axis of the other. All the parameters are determined from the statistical data. Comparison of fitting results with existing examples validates the accuracy of our geometric model. Based on the fitting shape, one can further simulate the physiological function of the mitral valve using a suitable dynamic physical equation.

Image Segmentation and Modeling of the Pediatric Tricuspid Valve in Hypoplastic Left Heart Syndrome

Hypoplastic left heart syndrome (HLHS) is a single-ventricle congenital heart disease that is fatal if left unpalliated. In HLHS patients, the tricuspid valve is the only functioning atrioventricular valve, and its competence is therefore critical. This work demonstrates the first automated strategy for segmentation, modeling, and morphometry of the tricuspid valve in transthoracic 3D echocardiographic (3DE) images of pediatric patients with HLHS. After initial landmark placement, the automated segmentation step uses multi-atlas label fusion and the modeling approach uses deformable modeling with medial axis representation to produce patient-specific models of the tricuspid valve that can be comprehensively and quantitatively assessed. In a group of 16 pediatric patients, valve segmentation and modeling attains an accuracy (mean boundary displacement) of 0.8 ± 0.2 mm relative to manual tracing and shows consistency in annular and leaflet measurements. In the future, such image-based tools have the potential to improve understanding and evaluation of tricuspid valve morphology in HLHS and guide strategies for patient care.

Modeling the Myxomatous Mitral Valve With Three-Dimensional Echocardiography

BACKGROUND

Degenerative mitral valve disease is associated with variable and complex defects in valve morphology. Three-dimensional echocardiography (3DE) has shown promise in aiding preoperative planning for patients with this disease but to date has not been as transformative as initially predicted. The clinical usefulness of 3DE has been limited by the laborious methods currently required to extract quantitative data from the images.

METHODS

To maximize the utility of 3DE for preoperative valve evaluation, this work describes an automated 3DE image analysis method for generating models of the mitral valve that are well suited for both qualitative and quantitative assessment. The method is unique in that it captures detailed alterations in mitral leaflet and annular morphology and produces image-derived models with locally varying leaflet thickness. The method is evaluated on midsystolic transesophageal 3DE images acquired from 22 subjects with myxomatous degeneration and from 22 subjects with normal mitral valve morphology.

RESULTS

Relative to manual image analysis, the automated method accurately represents both normal and complex leaflet geometries with a mean boundary displacement error on the order of one image voxel. A detailed quantitative analysis of the valves is presented and reveals statistically significant differences between normal and myxomatous valves with respect to numerous aspects of annular and leaflet geometry.

CONCLUSIONS

This work demonstrates a successful methodology for the relatively rapid quantitative description of the complex mitral valve distortions associated with myxomatous degeneration. The methodology has the potential to significantly improve surgical planning for patients with complex mitral valve disease.

Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.

Medially constrained deformable modeling for segmentation of branching medial structures: Application to aortic valve segmentation and morphometry

Deformable modeling with medial axis representation is a useful means of segmenting and parametrically describing the shape of anatomical structures in medical images. Continuous medial representation (cm-rep) is a "skeleton-first" approach to deformable medial modeling that explicitly parameterizes an object's medial axis and derives the object's boundary algorithmically. Although cm-rep has effectively been used to segment and model a number of anatomical structures with non-branching medial topologies, the framework is challenging to apply to objects with branching medial geometries since branch curves in the medial axis are difficult to parameterize. In this work, we demonstrate the first clinical application of a new "boundary-first" deformable medial modeling paradigm, wherein an object's boundary is explicitly described and constraints are imposed on boundary geometry to preserve the branching configuration of the medial axis during model deformation. This "boundary-first" framework is leveraged to segment and morphologically analyze the aortic valve apparatus in 3D echocardiographic images. Relative to manual tracing, segmentation with deformable medial modeling achieves a mean boundary error of 0.41 ± 0.10 mm (approximately one voxel) in 22 3DE images of normal aortic valves at systole. Deformable medial modeling is additionally demonstrated on pathological cases, including aortic stenosis, Marfan syndrome, and bicuspid aortic valve disease. This study demonstrates a promising approach for quantitative 3DE analysis of aortic valve morphology.

Automatic mitral annulus tracking in volumetric ultrasound using non-rigid image registration

Analysis of mitral annular dynamics plays an important role in the diagnosis and selection of optimal valve repair strategies, but remains cumbersome and time-consuming if performed manually. In this paper we propose non-rigid image registration to automatically track the annulus in 3D ultrasound images for both normal and pathological valves, and compare the performance against manual tracing. Relevant clinical properties such as annular area, circumference and excursion could be extracted reliably by the tracking algorithm. The root-mean-square error, calculated as the difference between the manually traced landmarks (18 in total) and the automatic tracking, was 1.96 ± 0.46 mm over 10 valves (5 healthy and 5 diseased) which is within the clinically acceptable error range.

Fully automatic segmentation of the mitral leaflets in 3D transesophageal echocardiographic images using multi-atlas joint label fusion and deformable medial modeling

Comprehensive visual and quantitative analysis of in vivo human mitral valve morphology is central to the diagnosis and surgical treatment of mitral valve disease. Real-time 3D transesophageal echocardiography (3D TEE) is a practical, highly informative imaging modality for examining the mitral valve in a clinical setting. To facilitate visual and quantitative 3D TEE image analysis, we describe a fully automated method for segmenting the mitral leaflets in 3D TEE image data. The algorithm integrates complementary probabilistic segmentation and shape modeling techniques (multi-atlas joint label fusion and deformable modeling with continuous medial representation) to automatically generate 3D geometric models of the mitral leaflets from 3D TEE image data. These models are unique in that they establish a shape-based coordinate system on the valves of different subjects and represent the leaflets volumetrically, as structures with locally varying thickness. In this work, expert image analysis is the gold standard for evaluating automatic segmentation. Without any user interaction, we demonstrate that the automatic segmentation method accurately captures patient-specific leaflet geometry at both systole and diastole in 3D TEE data acquired from a mixed population of subjects with normal valve morphology and mitral valve disease.

Fast Simulation of Mitral Annuloplasty for Surgical Planning

Mitral valve repair is a complex procedure that requires the ability to predict closed valve shape through the examination of an unpressurized, accid valve. These procedures typically include the remodeling of the mitral annulus through the insertion of an annuloplasty ring. While simulations could facilitate the planning of the procedure, traditional finite-element models of mitral annuloplasty are too slow to be clinically feasible and have never been validated in tissue. This work presents a fast method for simulating valve closure post-annuloplasty using a mass-spring tissue model and subject-specific valve geometry. Closed valve shape is predicted in less than one second. The results are validated by implanting an annuloplasty ring in an excised porcine heart and comparing simulated to imaged results. Results indicate that not only can mitral annuloplasty be simulated quickly, but also with submillimeter accuracy.