Mitral annulus segmentation from four-dimensional ultrasound using a valve state predictor and constrained optical flow

Automated estimation of mitral annular plane systolic excursion by artificial intelligence from 3D ultrasound recordings

Perioperative monitoring of cardiac function is beneficial for early detection of cardiovascular complications. The standard of care for cardiac monitoring performed by trained cardiologists and anesthesiologists involves a manual and qualitative evaluation of ultrasound imaging, which is a time-demanding and resource-intensive process with intraobserver- and interobserver variability. In practice, such measures can only be performed a limited number of times during the intervention. To overcome these difficulties, this study presents a robust method for automatic and quantitative monitoring of cardiac function based on 3D transesophageal echocardiography (TEE) B-mode ultrasound recordings of the left ventricle (LV). Such an assessment obtains consistent measurements and can produce a near real-time evaluation of ultrasound imagery. Hence, the presented method is time-saving and results in increased accessibility. The mitral annular plane systolic excursion (MAPSE), characterizing global LV function, is estimated by landmark detection and cardiac view classification of two-dimensional images extracted along the long-axis of the ultrasound volume. MAPSE estimation directly from 3D TEE recordings is beneficial since it removes the need for manual acquisition of cardiac views, hence decreasing the need for interference by physicians. Two convolutional neural networks (CNNs) were trained and tested on acquired ultrasound data of 107 patients, and MAPSE estimates were compared to clinically obtained references in a blinded study including 31 patients. The proposed method for automatic MAPSE estimation had low bias and low variability in comparison to clinical reference measures. The method accomplished a mean difference for MAPSE estimates of (-0.16±1.06) mm. Thus, the results did not show significant systematic errors. The obtained bias and variance of the method were comparable to inter-observer variability of clinically obtained MAPSE measures on 2D TTE echocardiography. The novel pipeline proposed in this study has the potential to enhance cardiac monitoring in perioperative- and intensive care settings.

Aortic Annulus Detection Based on Deep Learning for Transcatheter Aortic Valve Replacement Using Cardiac Computed Tomography

BACKGROUND

To propose a deep learning architecture for automatically detecting the complex structure of the aortic annulus plane using cardiac computed tomography (CT) for transcatheter aortic valve replacement (TAVR).

METHODS

This study retrospectively reviewed consecutive patients who underwent TAVR between January 2017 and July 2020 at a tertiary medical center. Annulus Detection Permuted AdaIN network (ADPANet) based on a three-dimensional (3D) U-net architecture was developed to detect and localize the aortic annulus plane using cardiac CT. Patients (N = 72) who underwent TAVR between January 2017 and July 2020 at a tertiary medical center were enrolled. Ground truth using a limited dataset was delineated manually by three cardiac radiologists. Training, tuning, and testing sets (70:10:20) were used to build the deep learning model. The performance of ADPANet for detecting the aortic annulus plane was analyzed using the root mean square error (RMSE) and dice similarity coefficient (DSC).

RESULTS

In this study, the total dataset consisted of 72 selected scans from patients who underwent TAVR. The RMSE and DSC values for the aortic annulus plane using ADPANet were 55.078 ± 35.794 and 0.496 ± 0.217, respectively.

CONCLUSION

Our deep learning framework was feasible to detect the 3D complex structure of the aortic annulus plane using cardiac CT for TAVR. The performance of our algorithms was higher than other convolutional neural networks.

Mitral Annulus Segmentation Using Deep Learning in 3-D Transesophageal Echocardiography

3D Transesophageal Echocardiography is an excellent tool for evaluating the mitral valve and is also well suited for guiding cardiac interventions. We introduce a fully automatic method for mitral annulus segmentation in 3D Transesophageal Echocardiography, which requires no manual input. One hundred eleven multi-frame 3D transesophageal echocardiography recordings were split into training, validation, and test sets. Each 3D recording was decomposed into a set of 2D planes, exploiting the symmetry around the centerline of the left ventricle. A deep 2D convolutional neural network was trained to predict the mitral annulus coordinates, and the predictions from neighboring planes were regularized by enforcing continuity around the annulus. Applying the final model and post-processing to the test set data gave a mean error of 2.0 mm - with a standard deviation of 1.9 mm. Fully automatic segmentation of the mitral annulus can alleviate the need for manual interaction in the quantification of an array of mitral annular parameters and has the potential to eliminate inter-observer variability.

Combining position-based dynamics and gradient vector flow for 4D mitral valve segmentation in TEE sequences

PURPOSE

For planning and guidance of minimally invasive mitral valve repair procedures, 3D+t transesophageal echocardiography (TEE) sequences are acquired before and after the intervention. The valve is then visually and quantitatively assessed in selected phases. To enable a quantitative assessment of valve geometry and pathological properties in all heart phases, as well as the changes achieved through surgery, we aim to provide a new 4D segmentation method.

METHODS

We propose a tracking-based approach combining gradient vector flow (GVF) and position-based dynamics (PBD). An open-state surface model of the valve is propagated through time to the closed state, attracted by the GVF field of the leaflet area. The PBD method ensures topological consistency during deformation. For evaluation, one expert in cardiac surgery annotated the closed-state leaflets in 10 TEE sequences of patients with normal and abnormal mitral valves, and defined the corresponding open-state models.

RESULTS

The average point-to-surface distance between the manual annotations and the final tracked model was [Formula: see text]. Qualitatively, four cases were satisfactory, five passable and one unsatisfactory. Each sequence could be segmented in 2-6 min.

CONCLUSION

Our approach enables to segment the mitral valve in 4D TEE image data with normal and pathological valve closing behavior. With this method, in addition to the quantification of the remaining orifice area, shape and dimensions of the coaptation zone can be analyzed and considered for planning and surgical result assessment.

Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects

Fetal imaging is a burgeoning topic. New advancements in both magnetic resonance imaging and (3D) ultrasound currently allow doctors to diagnose fetal structural abnormalities such as those involved in twin-to-twin transfusion syndrome, gestational diabetes mellitus, pulmonary sequestration and hypoplasia, congenital heart disease, diaphragmatic hernia, ventriculomegaly, etc. Considering the continued breakthroughs in utero image analysis and (3D) reconstruction models, it is now possible to gain more insight into the ongoing development of the fetus. Best prenatal diagnosis performances rely on the conscious preparation of the clinicians in terms of fetal anatomy knowledge. Therefore, fetal imaging will likely span and increase its prevalence in the forthcoming years. This review covers state-of-the-art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time. Potential applications of the aforementioned methods into clinical settings are also inspected. Finally, improvements in existing approaches as well as most promising avenues to new areas of research are briefly outlined.

Machine learning-based 3-D geometry reconstruction and modeling of aortic valve deformation using 3-D computed tomography images

To conduct a patient-specific computational modeling of the aortic valve, 3-D aortic valve anatomic geometries of an individual patient need to be reconstructed from clinical 3-D cardiac images. Currently, most of computational studies involve manual heart valve geometry reconstruction and manual finite element (FE) model generation, which is both time-consuming and prone to human errors. A seamless computational modeling framework, which can automate this process based on machine learning algorithms, is desirable, as it can not only eliminate human errors and ensure the consistency of the modeling results but also allow fast feedback to clinicians and permits a future population-based probabilistic analysis of large patient cohorts. In this study, we developed a novel computational modeling method to automatically reconstruct the 3-D geometries of the aortic valve from computed tomographic images. The reconstructed valve geometries have built-in mesh correspondence, which bridges harmonically for the consequent FE modeling. The proposed method was evaluated by comparing the reconstructed geometries from 10 patients with those manually created by human experts, and a mean discrepancy of 0.69 mm was obtained. Based on these reconstructed geometries, FE models of valve leaflets were developed, and aortic valve closure from end systole to middiastole was simulated for 7 patients and validated by comparing the deformed geometries with those manually created by human experts, and a mean discrepancy of 1.57 mm was obtained. The proposed method offers great potential to streamline the computational modeling process and enables the development of a preoperative planning system for aortic valve disease diagnosis and treatment.

Reflections on ultrasound image analysis

Ultrasound (US) image analysis has advanced considerably in twenty years. Progress in ultrasound image analysis has always been fundamental to the advancement of image-guided interventions research due to the real-time acquisition capability of ultrasound and this has remained true over the two decades. But in quantitative ultrasound image analysis - which takes US images and turns them into more meaningful clinical information - thinking has perhaps more fundamentally changed. From roots as a poor cousin to Computed Tomography (CT) and Magnetic Resonance (MR) image analysis, both of which have richer anatomical definition and thus were better suited to the earlier eras of medical image analysis which were dominated by model-based methods, ultrasound image analysis has now entered an exciting new era, assisted by advances in machine learning and the growing clinical and commercial interest in employing low-cost portable ultrasound devices outside traditional hospital-based clinical settings. This short article provides a perspective on this change, and highlights some challenges ahead and potential opportunities in ultrasound image analysis which may both have high impact on healthcare delivery worldwide in the future but may also, perhaps, take the subject further away from CT and MR image analysis research with time.

Ultrasound guidance for beating heart mitral valve repair augmented by synthetic dynamic CT

Minimally invasive valvular intervention commonly requires intra-procedural navigation to provide spatial and temporal information of relevant cardiac structures and device components. Recently intra-procedural trans-esophageal echocardiography (TEE) has been exploited for this purpose due to its accessibility, low cost, ease of use, and real-time imaging capacity. However, the position and orientation of tissue targets relative to surgical tools can be challenging to perceive, particularly using 2D imaging planes. In this paper, we propose the use of CT images to provide a high-quality 3D context to enhance ultrasound images through image registration, providing an augmented guidance system with minimal impact on standard clinical workflow. We also describe an approach to generate synthetic 4D CT images through non-rigid registration of available ultrasound. This can be employed to avoid a requirement for higher radiation. Synthetic CT images were validated through direct comparison of synthetic and real multi-phase CT images. Validation of CT and ultrasound image registration was performed for both dynamic and synthetic CT image datasets. Our results demonstrated that the synthetically generated dynamic CT images provide similar anatomical representation for relevant cardiac anatomy relative to real dynamic CT images, and similar high registration accuracy that can be achieved for intra-procedural TEE to this versus real dynamic CT images.

Identification of Mitral Annulus Hinge Point Based on Local Context Feature and Additive SVM Classifier

The position of the hinge point of mitral annulus (MA) is important for segmentation, modeling and multimodalities registration of cardiac structures. The main difficulties in identifying the hinge point of MA are the inherent noisy, low resolution of echocardiography, and so on. This work aims to automatically detect the hinge point of MA by combining local context feature with additive support vector machines (SVM) classifier. The innovations are as follows: (1) designing a local context feature for MA in cardiac ultrasound image; (2) applying the additive kernel SVM classifier to identify the candidates of the hinge point of MA; (3) designing a weighted density field of candidates which represents the blocks of candidates; and (4) estimating an adaptive threshold on the weighted density field to get the position of the hinge point of MA and exclude the error from SVM classifier. The proposed algorithm is tested on echocardiographic four-chamber image sequence of 10 pediatric patients. Compared with the manual selected hinge points of MA which are selected by professional doctors, the mean error is in 0.96 ± 1.04 mm. Additive SVM classifier can fast and accurately identify the MA hinge point.

A mitral annulus tracking approach for navigation of off-pump beating heart mitral valve repair

PURPOSE

To develop and validate a real-time mitral valve annulus (MVA) tracking approach based on biplane transesophageal echocardiogram (TEE) data and magnetic tracking systems (MTS) to be used in minimally invasive off-pump beating heart mitral valve repair (MVR).

METHODS

The authors' guidance system consists of three major components: TEE, magnetic tracking system, and an image guidance software platform. TEE provides real-time intraoperative images to show the cardiac motion and intracardiac surgical tools. The magnetic tracking system tracks the TEE probe and the surgical tools. The software platform integrates the TEE image planes and the virtual model of the tools and the MVA model on the screen. The authors' MVA tracking approach, which aims to update the MVA model in near real-time, comprises of three steps: image based gating, predictive reinitialization, and registration based MVA tracking. The image based gating step uses a small patch centered at each MVA point in the TEE images to identify images at optimal cardiac phases for updating the position of the MVA. The predictive reinitialization step uses the position and orientation of the TEE probe provided by the magnetic tracking system to predict the position of the MVA points in the TEE images and uses them for the initialization of the registration component. The registration based MVA tracking step aims to locate the MVA points in the images selected by the image based gating component by performing image based registration.

RESULTS

The validation of the MVA tracking approach was performed in a phantom study and a retrospective study on porcine data. In the phantom study, controlled translations were applied to the phantom and the tracked MVA was compared to its "true" position estimated based on a magnetic sensor attached to the phantom. The MVA tracking accuracy was 1.29 ± 0.58 mm when the translation distance is about 1 cm, and increased to 2.85 ± 1.19 mm when the translation distance is about 3 cm. In the study on porcine data, the authors compared the tracked MVA to a manually segmented MVA. The overall accuracy is 2.37 ± 1.67 mm for single plane images and 2.35 ± 1.55 mm for biplane images. The interoperator variation in manual segmentation was 2.32 ± 1.24 mm for single plane images and 1.73 ± 1.18 mm for biplane images. The computational efficiency of the algorithm on a desktop computer with an Intel(®) Xeon(®) CPU @3.47 GHz and an NVIDIA GeForce 690 graphic card is such that the time required for registering four MVA points was about 60 ms.

CONCLUSIONS

The authors developed a rapid MVA tracking algorithm for use in the guidance of off-pump beating heart transapical mitral valve repair. This approach uses 2D biplane TEE images and was tested on a dynamic heart phantom and interventional porcine image data. Results regarding the accuracy and efficiency of the authors' MVA tracking algorithm are promising, and fulfill the requirements for surgical navigation.

Semi-automated segmentation and quantification of mitral annulus and leaflets from transesophageal 3-D echocardiographic images

Quantification of three-dimensional (3-D) morphology of the mitral valve (MV) using real-time 3-D transesophageal echocardiography (RT3-D TEE) has proved to be a valuable tool for the assessment of MV pathologies, but of limited use in clinical practice because it relies on user-intensive approaches. This study presents a new algorithm for the segmentation and morphologic quantification of the mitral annulus (MA) and mitral leaflets (ML) in closed valve configuration from RT3-D TEE volumes. Following initialization, the MA and the ML and the coaptation line (CL) are automatically obtained in 3-D. Validation with manual tracings was performed on 33 patients, resulting in segmentation errors in the order of 0.7 mm and 0.6 mm for the MA and ML segmentation, in addition to good intra- and inter-observer reproducibility (coefficients of variation below 12% and 15%, respectively). The ability of the algorithm to assess different MV pathologies as well as repaired valves with implanted annular rings was also explored. The reported performance of the proposed fast, semi-automated MA and ML quantification makes it promising for future applications in clinical settings such as the operating room, where obtaining results in short time is important.

Using a shape prior for robust modeling of the mitral annulus on 4D ultrasound data

PURPOSE

Over 40,000 annuloplasty rings are implanted each year in the USA to treat mitral regurgitation. However, the used measuring techniques to select a suitable annuloplasty ring are imprecise and highly depending on the expert's experience. This can cause a re-occurrence of the mitral regurgitation or an annuloplasty ring dehiscence, and thus the necessity of a re-operation. We propose a method to create a 4D model of the mitral annulus from ultrasound data to enable precise measurement and patient-specific implant planning.

METHODS

An initial mitral annulus model is placed interactively in the 4D image data by defining commissure points and the annulus plane for one time step in diastole and systole. The model is automatically optimized using distinct image features. A shape and pose prior of the mitral annulus is used to compensate for artifacts and to enforce a plausible anatomical morphology, while a temporal alignment ensures a natural motion of the 4D model.

RESULTS

Ground truth data were created for 4D images of 42 patients with varying image quality. A parameter and shape prior training was performed on a third of the ground truth data, while the rest was used to validate the method. The average error of the resulting mitral annulus models was computed as 2.25 ( +/-0.38 ) mm. The average expert standard deviation was determined as 1.86 (+/-0.32 ) mm.

CONCLUSION

The proposed method enables the 4D modeling of mitral annuli based on ultrasound data in less than 2 min. The resulting models are comparable to manually delineated models and can be used for measurements of annular geometries and patient-specific annuloplasty treatment planning.

Fractional-order variational optical flow model for motion estimation

A new class of fractional-order variational optical flow models, which generalizes the differential of optical flow from integer order to fractional order, is proposed for motion estimation in this paper. The corresponding Euler-Lagrange equations are derived by solving a typical fractional variational problem, and the numerical implementation based on the Grünwald-Letnikov fractional derivative definition is proposed to solve these complicated fractional partial differential equations. Theoretical analysis reveals that the proposed fractional-order variational optical flow model is the generalization of the typical Horn and Schunck (first-order) variational optical flow model and the second-order variational optical flow model, which provides a new idea for us to study the optical flow model and has an important theoretical implication in optical flow model research. The experiments demonstrate the validity of the generalization of differential order.

Toward patient-specific simulations of cardiac valves: state-of-the-art and future directions

Recent computational methods enabling patient-specific simulations of native and prosthetic heart valves are reviewed. Emphasis is placed on two critical components of such methods: (1) anatomically realistic finite element models for simulating the structural dynamics of heart valves; and (2) fluid structure interaction methods for simulating the performance of heart valves in a patient-specific beating left ventricle. It is shown that the significant progress achieved in both fronts paves the way toward clinically relevant computational models that can simulate the performance of a range of heart valves, native and prosthetic, in a patient-specific left heart environment. The significant algorithmic and model validation challenges that need to be tackled in the future to realize this goal are also discussed.

An integrated framework for finite-element modeling of mitral valve biomechanics from medical images: application to MitralClip intervention planning

Treatment of mitral valve (MV) diseases requires comprehensive clinical evaluation and therapy personalization to optimize outcomes. Finite-element models (FEMs) of MV physiology have been proposed to study the biomechanical impact of MV repair, but their translation into the clinics remains challenging. As a step towards this goal, we present an integrated framework for finite-element modeling of the MV closure based on patient-specific anatomies and boundary conditions. Starting from temporal medical images, we estimate a comprehensive model of the MV apparatus dynamics, including papillary tips, using a machine-learning approach. A detailed model of the open MV at end-diastole is then computed, which is finally closed according to a FEM of MV biomechanics. The motion of the mitral annulus and papillary tips are constrained from the image data for increased accuracy. A sensitivity analysis of our system shows that chordae rest length and boundary conditions have a significant influence upon the simulation results. We quantitatively test the generalization of our framework on 25 consecutive patients. Comparisons between the simulated closed valve and ground truth show encouraging results (average point-to-mesh distance: 1.49 ± 0.62 mm) but also the need for personalization of tissue properties, as illustrated in three patients. Finally, the predictive power of our model is tested on one patient who underwent MitralClip by comparing the simulated intervention with the real outcome in terms of MV closure, yielding promising prediction. By providing an integrated way to perform MV simulation, our framework may constitute a surrogate tool for model validation and therapy planning.