Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features

Artificial intelligence: improving the efficiency of cardiovascular imaging

INTRODUCTION

Artificial intelligence (AI) describes the use of computational techniques to mimic human intelligence. In healthcare, this typically involves large medical datasets being used to predict a diagnosis, identify new disease genotypes or phenotypes, or guide treatment strategies. Noninvasive imaging remains a cornerstone for the diagnosis, risk stratification, and management of patients with cardiovascular disease. AI can facilitate every stage of the imaging process, from acquisition and reconstruction, to segmentation, measurement, interpretation, and subsequent clinical pathways.

AREAS COVERED

In this paper, we review state-of-the-art AI techniques and their current applications in cardiac imaging, and discuss the future role of AI as a precision medicine tool.

EXPERT OPINION

Cardiovascular medicine is primed for scalable AI applications which can interpret vast amounts of clinical and imaging data in greater depth than ever before. AI-augmented medical systems have the potential to improve workflow and provide reproducible and objective quantitative results which can inform clinical decisions. In the foreseeable future, AI may work in the background of cardiac image analysis software and routine clinical reporting, automatically collecting data and enabling real-time diagnosis and risk stratification.

A Generic Approach to Lung Field Segmentation From Chest Radiographs Using Deep Space and Shape Learning

Computer-aided diagnosis (CAD) techniques for lung field segmentation from chest radiographs (CXR) have been proposed for adult cohorts, but rarely for pediatric subjects. Statistical shape models (SSMs), the workhorse of most state-of-the-art CXR-based lung field segmentation methods, do not efficiently accommodate shape variation of the lung field during the pediatric developmental stages. The main contributions of our work are: 1) a generic lung field segmentation framework from CXR accommodating large shape variation for adult and pediatric cohorts; 2) a deep representation learning detection mechanism, ensemble space learning, for robust object localization; and 3) marginal shape deep learning for the shape deformation parameter estimation. Unlike the iterative approach of conventional SSMs, the proposed shape learning mechanism transforms the parameter space into marginal subspaces that are solvable efficiently using the recursive representation learning mechanism. Furthermore, our method is the first to include the challenging retro-cardiac region in the CXR-based lung segmentation for accurate lung capacity estimation. The framework is evaluated on 668 CXRs of patients between 3 month to 89 year of age. We obtain a mean Dice similarity coefficient of 0.96 ±0.03 (including the retro-cardiac region). For a given accuracy, the proposed approach is also found to be faster than conventional SSM-based iterative segmentation methods. The computational simplicity of the proposed generic framework could be similarly applied to the fast segmentation of other deformable objects.

Deep Learning for Cardiac Image Segmentation: A Review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound and major anatomical structures of interest (ventricles, atria, and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

Simultaneous Multi-Structure Segmentation of the Heart and Peripheral Tissues in Contrast Enhanced Cardiac Computed Tomography Angiography

Contrast enhanced cardiac computed tomography angiography (CTA) is a prominent imaging modality for diagnosing cardiovascular diseases non-invasively. It assists the evaluation of the coronary artery patency and provides a comprehensive assessment of structural features of the heart and great vessels. However, physicians are often required to evaluate different cardiac structures and measure their size manually. Such task is very time-consuming and tedious due to the large number of image slices in 3D data. We present a fully automatic method based on a combined multi-atlas and corrective segmentation approach to label the heart and its associated cardiovascular structures. This method also automatically separates other surrounding intrathoracic structures from CTA images. Quantitative assessment of the proposed method is performed on 36 studies with a reference standard obtained from expert manual segmentation of various cardiac structures. Qualitative evaluation is also performed by expert readers to score 120 studies of the automatic segmentation. The quantitative results showed an overall Dice of 0.93, Hausdorff distance of 7.94 mm, and mean surface distance of 1.03 mm between automatically and manually segmented cardiac structures. The visual assessment also attained an excellent score for the automatic segmentation. The average processing time was 2.79 minutes. Our results indicate the proposed automatic framework significantly improves accuracy and computational speed in conventional multi-atlas based approach, and it provides comprehensive and reliable multi-structural segmentation of CTA images that is valuable for clinical application.

DW-Net: A cascaded convolutional neural network for apical four-chamber view segmentation in fetal echocardiography

Fetal echocardiography (FE) is a widely used medical examination for early diagnosis of congenital heart disease (CHD). The apical four-chamber view (A4C) is an important view among early FE images. Accurate segmentation of crucial anatomical structures in the A4C view is a useful and important step for early diagnosis and timely treatment of CHDs. However, it is a challenging task due to several unfavorable factors: (a) artifacts and speckle noise produced by ultrasound imaging. (b) category confusion caused by the similarity of anatomical structures and variations of scanning angles. (c) missing boundaries. In this paper, we propose an end-to-end DW-Net for accurate segmentation of seven important anatomical structures in the A4C view. The network comprises two components: 1) a Dilated Convolutional Chain (DCC) for "gridding issue" reduction, multi-scale contextual information aggregation and accurate localization of cardiac chambers. 2) a W-Net for gaining more precise boundaries and yielding refined segmentation results. Extensive experiments of the proposed method on a dataset of 895 A4C views have demonstrated that DW-Net can achieve good segmentation results, including the Dice Similarity Coefficient (DSC) of 0.827, the Pixel Accuracy (PA) of 0.933, the AUC of 0.990 and it substantially outperformed some well-known segmentation methods. Our work was highly valued by experienced clinicians. The accurate and automatic segmentation of the A4C view using the proposed DW-Net can benefit further extractions of useful clinical indicators in early FE and improve the prenatal diagnostic accuracy and efficiency of CHDs.

Semi-automated Image Segmentation of the Midsystolic Left Ventricular Mitral Valve Complex in Ischemic Mitral Regurgitation

Ischemic mitral regurgitation (IMR) is primarily a left ventricular disease in which the mitral valve is dysfunctional due to ventricular remodeling after myocardial infarction. Current automated methods have focused on analyzing the mitral valve and left ventricle independently. While these methods have allowed for valuable insights into mechanisms of IMR, they do not fully integrate pathological features of the left ventricle and mitral valve. Thus, there is an unmet need to develop an automated segmentation algorithm for the left ventricular mitral valve complex, in order to allow for a more comprehensive study of this disease. The objective of this study is to generate and evaluate segmentations of the left ventricular mitral valve complex in pre-operative 3D transesophageal echocardiography using multi-atlas label fusion. These patient-specific segmentations could enable future statistical shape analysis for clinical outcome prediction and surgical risk stratification. In this study, we demonstrate a preliminary segmentation pipeline that achieves an average Dice coefficient of 0.78 ± 0.06.

Generation of a cohort of whole-torso cardiac models for assessing the utility of a novel computed shock vector efficiency metric for ICD optimisation

Implanted cardiac defibrillators (ICDs) seek to automatically detect and terminate potentially lethal ventricular arrhythmias by applying strong internal electric shocks across the heart. However, the optimisation of the specific electrode design and configurations represents an intensive area of research in the pursuit of reduced shock strengths and fewer device complications and risks. Computational whole-torso simulations play an important role in this endeavour, although knowing which specific metric should be used to assess configuration efficacy and assessing the impact of different patient anatomies and pathologies, and the corresponding effect this may have on different metrics has not been investigated. We constructed a cohort of CT-derived high-resolution whole torso-cardiac computational models, including variants of cardiomyopathies and patients with differing torso dimensions. Simulations of electric shock application between electrode configurations corresponding to transveneous (TV-ICD) and subcutaneous (S-ICD) ICDs were modelled and conventional metrics such as defibrillation threshold (DFT) and impedance computed. In addition, we computed a novel metric termed the shock vector efficiency (η), which quantifies the fraction of electrical energy dissipated in the heart relative to the rest of the torso. Across the cohort, S-ICD configurations showed higher DFTs and impedances than TV-ICDs, as expected, although little consistent difference was seen between healthy and cardiomyopathy variants. η was consistently <2% for S-ICD configurations, becoming as high as 13% for TV-ICD setups. Simulations also suggested that a total torso height of approximately 20 cm is required for convergence in η. Overall, η was seen to be approximately negatively correlated with both DFT and impedance. However, important scenarios were identified in which certain values of DFT (or impedance) were associated with a range of η values, and vice-versa, highlighting the heterogeneity introduced by the different torsos and pathologies modelled. In conclusion, the shock vector efficiency represents a useful additional metric to be considered alongside DFT and impedance in the optimisation of ICD electrode configurations, particularly in the context of differing torso anatomies and cardiac pathologies, which can induce significant heterogeneity in conventional metrics of ICD efficacy.

A deep learning approach to estimate stress distribution: a fast and accurate surrogate of finite-element analysis

Structural finite-element analysis (FEA) has been widely used to study the biomechanics of human tissues and organs, as well as tissue-medical device interactions, and treatment strategies. However, patient-specific FEA models usually require complex procedures to set up and long computing times to obtain final simulation results, preventing prompt feedback to clinicians in time-sensitive clinical applications. In this study, by using machine learning techniques, we developed a deep learning (DL) model to directly estimate the stress distributions of the aorta. The DL model was designed and trained to take the input of FEA and directly output the aortic wall stress distributions, bypassing the FEA calculation process. The trained DL model is capable of predicting the stress distributions with average errors of 0.492% and 0.891% in the Von Mises stress distribution and peak Von Mises stress, respectively. This study marks, to our knowledge, the first study that demonstrates the feasibility and great potential of using the DL technique as a fast and accurate surrogate of FEA for stress analysis.

The importance of side branches in modeling 3D hemodynamics from angiograms for patients with coronary artery disease

Genesis of atherosclerotic lesions in the human arterial system is critically influenced by the fluid mechanics. Applying computational fluid dynamic tools based on accurate coronary physiology derived from conventional biplane angiogram data may be useful in guiding percutaneous coronary interventions. The primary objective of this study is to build and validate a computational framework for accurate personalized 3-dimensional hemodynamic simulation across the complete coronary arterial tree and demonstrate the influence of side branches on coronary hemodynamics by comparing shear stress between coronary models with and without these included. The proposed novel computational framework based on biplane angiography enables significant arterial circulation analysis. This study shows that models that take into account flow through all side branches are required for precise computation of shear stress and pressure gradient whereas models that have only a subset of side branches are inadequate for biomechanical studies as they may overestimate volumetric outflow and shear stress. This study extends the ongoing computational efforts and demonstrates that models based on accurate coronary physiology can improve overall fidelity of biomechanical studies to compute hemodynamic risk-factors.

Artificial Intelligence in Cardiovascular Medicine

PURPOSE OF REVIEW

The ripples of artificial intelligence are being felt in various sectors of human life. Machine learning, a subset of artificial intelligence, extracts information from large databases of information and is gaining traction in various fields of cardiology. In this review, we highlight noteworthy examples of machine learning utilization in echocardiography, nuclear cardiology, computed tomography, and magnetic resonance imaging over the past year.

RECENT FINDINGS

In the past year, machine learning (ML) has expanded its boundaries in cardiology with several positive results. Some studies have integrated clinical and imaging information to further augment the accuracy of these ML algorithms. All the studies mentioned in this review have clearly demonstrated superior results of ML in relation to conventional approaches for identifying obstructions or predicting major adverse events in reference to conventional approaches. As the influx of data arriving from gradually evolving technologies in health care and wearable devices continues to be more complex, ML may serve as the bridge to transcend the gap between health care and patients in the future. In order to facilitate a seamless transition between both, a few issues must be resolved for a successful implementation of ML in health care.

Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views

PURPOSE

Minimally invasive alternatives are now available for many complex surgeries. These approaches are enabled by the increasing availability of intra-operative image guidance. Yet, fluoroscopic X-rays suffer from projective transformation and thus cannot provide direct views onto anatomy. Surgeons could highly benefit from additional information, such as the anatomical landmark locations in the projections, to support intra-operative decision making. However, detecting landmarks is challenging since the viewing direction changes substantially between views leading to varying appearance of the same landmark. Therefore, and to the best of our knowledge, view-independent anatomical landmark detection has not been investigated yet.

METHODS

In this work, we propose a novel approach to detect multiple anatomical landmarks in X-ray images from arbitrary viewing directions. To this end, a sequential prediction framework based on convolutional neural networks is employed to simultaneously regress all landmark locations. For training, synthetic X-rays are generated with a physically accurate forward model that allows direct application of the trained model to real X-ray images of the pelvis. View invariance is achieved via data augmentation by sampling viewing angles on a spherical segment of [Formula: see text].

RESULTS

On synthetic data, a mean prediction error of 5.6 ± 4.5 mm is achieved. Further, we demonstrate that the trained model can be directly applied to real X-rays and show that these detections define correspondences to a respective CT volume, which allows for analytic estimation of the 11 degree of freedom projective mapping.

CONCLUSION

We present the first tool to detect anatomical landmarks in X-ray images independent of their viewing direction. Access to this information during surgery may benefit decision making and constitutes a first step toward global initialization of 2D/3D registration without the need of calibration. As such, the proposed concept has a strong prospect to facilitate and enhance applications and methods in the realm of image-guided surgery.

Artificial Intelligence in Cardiovascular Imaging: JACC State-of-the-Art Review

Data science is likely to lead to major changes in cardiovascular imaging. Problems with timing, efficiency, and missed diagnoses occur at all stages of the imaging chain. The application of artificial intelligence (AI) is dependent on robust data; the application of appropriate computational approaches and tools; and validation of its clinical application to image segmentation, automated measurements, and eventually, automated diagnosis. AI may reduce cost and improve value at the stages of image acquisition, interpretation, and decision-making. Moreover, the precision now possible with cardiovascular imaging, combined with "big data" from the electronic health record and pathology, is likely to better characterize disease and personalize therapy. This review summarizes recent promising applications of AI in cardiology and cardiac imaging, which potentially add value to patient care.

Integrated 3D Anatomical Model for Automatic Myocardial Segmentation in Cardiac CT Imagery

Segmentation of epicardial and endocardial boundaries is a critical step in diagnosing cardiovascular function in heart patients. The manual tracing of organ contours in Computed Tomography Angiography (CTA) slices is subjective, time-consuming and impractical in clinical setting. We propose a novel multi-dimensional automatic edge detection algorithm based on shape priors and principal component analysis (PCA). We have developed a highly customized parametric model for implicit representations of segmenting curves (3D) for Left Ventricle (LV), Right Ventricle (RV), and Epicardium (Epi) used simultaneously to achieve myocardial segmentation. We have combined these representations in a region-based image modeling framework with high level constraints enabling the modeling of complex cardiac anatomical structures to automatically guide the segmentation of endo/epicardial boundaries. Test results on 30 short-axis CTA datasets show robust segmentation with error (mean ± std mm) of (1.46 ± 0.41), (2.06 ± 0.65), (2.88 ± 0.59) for LV, RV and Epi respectively.

Artificial intelligence and machine learning in spine research

Artificial intelligence (AI) and machine learning (ML) techniques are revolutionizing several industrial and research fields like computer vision, autonomous driving, natural language processing, and speech recognition. These novel tools are already having a major impact in radiology, diagnostics, and many other fields in which the availability of automated solution may benefit the accuracy and repeatability of the execution of critical tasks. In this narrative review, we first present a brief description of the various techniques that are being developed nowadays, with special focus on those used in spine research. Then, we describe the applications of AI and ML to problems related to the spine which have been published so far, including the localization of vertebrae and discs in radiological images, image segmentation, computer-aided diagnosis, prediction of clinical outcomes and complications, decision support systems, content-based image retrieval, biomechanics, and motion analysis. Finally, we briefly discuss major ethical issues related to the use of AI in healthcare, namely, accountability, risk of biased decisions as well as data privacy and security, which are nowadays being debated in the scientific community and by regulatory agencies.

Efficient Multiple Organ Localization in CT Image using 3D Region Proposal Network

Organ localization is an essential preprocessing step for many medical image analysis tasks such as image registration, organ segmentation and lesion detection. In this work, we propose an efficient method for multiple organ localization in CT image using 3D region proposal network. Compared with other convolutional neural network based methods that successively detect the target organs in all slices to assemble the final 3D bounding box, our method is fully implemented in 3D manner, thus can take full advantages of the spatial context information in CT image to perform efficient organ localization with only one prediction. We also propose a novel backbone network architecture that generates high-resolution feature maps to further improve the localization performance on small organs. We evaluate our method on two clinical datasets, where 11 body organs and 12 head organs (or anatomical structures) are included. As our results shown, the proposed method achieves higher detection precision and localization accuracy than the current state-of-theart methods with approximate 4 to 18 times faster processing speed. Additionally, we have established a public dataset dedicated for organ localization on http://dx. doi.org/10.21227/df8g-pq27. The full implementation of the proposed method have also been made publicly available on https://github.com/superxuang/caffe_3d_faster_rcnn.

Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans

Robust and fast detection of anatomical structures is a prerequisite for both diagnostic and interventional medical image analysis. Current solutions for anatomy detection are typically based on machine learning techniques that exploit large annotated image databases in order to learn the appearance of the captured anatomy. These solutions are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes for anatomy detection. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluated our approach on 1487 3D-CT volumes from 532 patients, totaling over 500,000 image slices and show that it significantly outperforms state-of-the-art solutions on detecting several anatomical structures with no failed cases from a clinical acceptance perspective, while also achieving a 20-30 percent higher detection accuracy. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.

A lightweight rapid application development framework for biomedical image analysis

Biomedical imaging analysis typically comprises a variety of complex tasks requiring sophisticated algorithms and visualising high dimensional data. The successful integration and deployment of the enabling software to clinical (research) partners, for rigorous evaluation and testing, is a crucial step to facilitate adoption of research innovations within medical settings. In this paper, we introduce the Simple Medical Imaging Library Interface (SMILI), an object oriented open-source framework with a compact suite of objects geared for rapid biomedical imaging (cross-platform) application development and deployment. SMILI supports the development of both command-line (shell and Python scripting) and graphical applications utilising the same set of processing algorithms. It provides a substantial subset of features when compared to more complex packages, yet it is small enough to ship with clinical applications with limited overhead and has a license suitable for commercial use. After describing where SMILI fits within the existing biomedical imaging software ecosystem, by comparing it to other state-of-the-art offerings, we demonstrate its capabilities in creating a clinical application for manual measurement of cam-type lesions of the femoral head-neck region for the investigation of femoro-acetabular impingement (FAI) from three dimensional (3D) magnetic resonance (MR) images of the hip. This application for the investigation of FAI proved to be convenient for radiological analyses and resulted in high intra (ICC=0.97) and inter-observer (ICC=0.95) reliabilities for measurement of α-angles of the femoral head-neck region. We believe that SMILI is particularly well suited for prototyping biomedical imaging applications requiring user interaction and/or visualisation of 3D mesh, scalar, vector or tensor data.

Ultrafast Volumetric Optoacoustic Imaging of Whole Isolated Beating Mouse Heart

The Langendorff-perfused heart technique has become the model of choice for multiparametric optical mapping of cardiac function and electrophysiology. However, photon scattering in tissues represents a significant drawback of the optical imaging approach, fundamentally limiting its mapping capacity to the heart surface. This work presents the first implementation of the optoacoustic approach for 4D imaging of the entire beating isolated mouse heart. The method combines optical excitation and acoustic detection to simultaneously render rich optical contrast and high spatio-temporal resolution at centimeter-scale depths. We demonstrate volumetric imaging of deeply located cardiac features, including the interventricular septum, chordae tendineae, and papillary muscles while further tracking the heart beat cycle and the motion of the pulmonary, mitral, and tricuspid valves in real time. The technique possesses a powerful combination between high imaging depth, fast volumetric imaging speed, functional and molecular imaging capacities not available with other imaging modalities currently used in cardiac research.

3D/2D model-to-image registration by imitation learning for cardiac procedures

PURPOSE

In cardiac interventions, such as cardiac resynchronization therapy (CRT), image guidance can be enhanced by involving preoperative models. Multimodality 3D/2D registration for image guidance, however, remains a significant research challenge for fundamentally different image data, i.e., MR to X-ray. Registration methods must account for differences in intensity, contrast levels, resolution, dimensionality, field of view. Furthermore, same anatomical structures may not be visible in both modalities. Current approaches have focused on developing modality-specific solutions for individual clinical use cases, by introducing constraints, or identifying cross-modality information manually. Machine learning approaches have the potential to create more general registration platforms. However, training image to image methods would require large multimodal datasets and ground truth for each target application.

METHODS

This paper proposes a model-to-image registration approach instead, because it is common in image-guided interventions to create anatomical models for diagnosis, planning or guidance prior to procedures. An imitation learning-based method, trained on 702 datasets, is used to register preoperative models to intraoperative X-ray images.

RESULTS

Accuracy is demonstrated on cardiac models and artificial X-rays generated from CTs. The registration error was [Formula: see text] on 1000 test cases, superior to that of manual ([Formula: see text]) and gradient-based ([Formula: see text]) registration. High robustness is shown in 19 clinical CRT cases.

CONCLUSION

Besides the proposed methods feasibility in a clinical environment, evaluation has shown good accuracy and high robustness indicating that it could be applied in image-guided interventions.

Automatic aortic valve landmark localization in coronary CT angiography using colonial walk

The minimally invasive transcatheter aortic valve implantation (TAVI) is the most prevalent method to treat aortic valve stenosis. For pre-operative surgical planning, contrast-enhanced coronary CT angiography (CCTA) is used as the imaging technique to acquire 3-D measurements of the valve. Accurate localization of the eight aortic valve landmarks in CT images plays a vital role in the TAVI workflow because a small error risks blocking the coronary circulation. In order to examine the valve and mark the landmarks, physicians prefer a view parallel to the hinge plane, instead of using the conventional axial, coronal or sagittal view. However, customizing the view is a difficult and time-consuming task because of unclear aorta pose and different artifacts of CCTA. Therefore, automatic localization of landmarks can serve as a useful guide to the physicians customizing the viewpoint. In this paper, we present an automatic method to localize the aortic valve landmarks using colonial walk, a regression tree-based machine-learning algorithm. For efficient learning from the training set, we propose a two-phase optimized search space learning model in which a representative point inside the valvular area is first learned from the whole CT volume. All eight landmarks are then learned from a smaller area around that point. Experiment with preprocedural CCTA images of TAVI undergoing patients showed that our method is robust under high stenotic variation and notably efficient, as it requires only 12 milliseconds to localize all eight landmarks, as tested on a 3.60 GHz single-core CPU.

Automated segmentation of the atrial region and fossa ovalis towards computer-aided planning of inter-atrial wall interventions

BACKGROUND AND OBJECTIVE

Image-fusion strategies have been applied to improve inter-atrial septal (IAS) wall minimally-invasive interventions. Hereto, several landmarks are initially identified on richly-detailed datasets throughout the planning stage and then combined with intra-operative images, enhancing the relevant structures and easing the procedure. Nevertheless, such planning is still performed manually, which is time-consuming and not necessarily reproducible, hampering its regular application. In this article, we present a novel automatic strategy to segment the atrial region (left/right atrium and aortic tract) and the fossa ovalis (FO).

METHODS

The method starts by initializing multiple 3D contours based on an atlas-based approach with global transforms only and refining them to the desired anatomy using a competitive segmentation strategy. The obtained contours are then applied to estimate the FO by evaluating both IAS wall thickness and the expected FO spatial location.

RESULTS

The proposed method was evaluated in 41 computed tomography datasets, by comparing the atrial region segmentation and FO estimation results against manually delineated contours. The automatic segmentation method presented a performance similar to the state-of-the-art techniques and a high feasibility, failing only in the segmentation of one aortic tract and of one right atrium. The FO estimation method presented an acceptable result in all the patients with a performance comparable to the inter-observer variability. Moreover, it was faster and fully user-interaction free.

CONCLUSIONS

Hence, the proposed method proved to be feasible to automatically segment the anatomical models for the planning of IAS wall interventions, making it exceptionally attractive for use in the clinical practice.

Towards intelligent robust detection of anatomical structures in incomplete volumetric data

Robust and fast detection of anatomical structures represents an important component of medical image analysis technologies. Current solutions for anatomy detection are based on machine learning, and are generally driven by suboptimal and exhaustive search strategies. In particular, these techniques do not effectively address cases of incomplete data, i.e., scans acquired with a partial field-of-view. We address these challenges by following a new paradigm, which reformulates the detection task to teaching an intelligent artificial agent how to actively search for an anatomical structure. Using the principles of deep reinforcement learning with multi-scale image analysis, artificial agents are taught optimal navigation paths in the scale-space representation of an image, while accounting for structures that are missing from the field-of-view. The spatial coherence of the observed anatomical landmarks is ensured using elements from statistical shape modeling and robust estimation theory. Experiments show that our solution outperforms marginal space deep learning, a powerful deep learning method, at detecting different anatomical structures without any failure. The dataset contains 5043 3D-CT volumes from over 2000 patients, totaling over 2,500,000 image slices. In particular, our solution achieves 0% false-positive and 0% false-negative rates at detecting whether the landmarks are captured in the field-of-view of the scan (excluding all border cases), with an average detection accuracy of 2.78 mm. In terms of runtime, we reduce the detection-time of the marginal space deep learning method by 20-30 times to under 40 ms, an unmatched performance for high resolution incomplete 3D-CT data.

Efficient organ localization using multi-label convolutional neural networks in thorax-abdomen CT scans

Automatic localization of organs and other structures in medical images is an important preprocessing step that can improve and speed up other algorithms such as organ segmentation, lesion detection, and registration. This work presents an efficient method for simultaneous localization of multiple structures in 3D thorax-abdomen CT scans. Our approach predicts the location of multiple structures using a single multi-label convolutional neural network for each orthogonal view. Each network takes extra slices around the current slice as input to provide extra context. A sigmoid layer is used to perform multi-label classification. The output of the three networks is subsequently combined to compute a 3D bounding box for each structure. We used our approach to locate 11 structures of interest. The neural network was trained and evaluated on a large set of 1884 thorax-abdomen CT scans from patients undergoing oncological workup. Reference bounding boxes were annotated by human observers. The performance of our method was evaluated by computing the wall distance to the reference bounding boxes. The bounding boxes annotated by the first human observer were used as the reference standard for the test set. Using the best configuration, we obtained an average wall distance of [Formula: see text] mm in the test set. The second human observer achieved [Formula: see text] mm. For all structures, the results were better than those reported in previously published studies. In conclusion, we proposed an efficient method for the accurate localization of multiple organs. Our method uses multiple slices as input to provide more context around the slice under analysis, and we have shown that this improves performance. This method can easily be adapted to handle more organs.

Heart Chamber Segmentation from CT Using Convolutional Neural Networks

CT is routinely used for radiotherapy planning with organs and regions of interest being segmented for diagnostic evaluation and parameter optimization. For cardiac segmentation, many methods have been proposed for left ventricular segmentation, but few for simultaneous segmentation of the entire heart. In this work, we present a convolutional neural networks (CNN)-based cardiac chamber segmentation method for 3D CT with 5 classes: left ventricle, right ventricle, left atrium, right atrium, and background. We achieved an overall accuracy of 87.2% ± 3.3% and an overall chamber accuracy of 85.6 ± 6.1%. The deep learning based segmentation method may provide an automatic tool for cardiac segmentation on CT images.

Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis

In patients with coronary artery stenoses of intermediate severity, the functional significance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identification of patients with functionally significant coronary artery stenoses, employing deep learning analysis of the left ventricle (LV) myocardium in rest coronary CT angiography (CCTA). The study includes consecutively acquired CCTA scans of 166 patients who underwent invasive FFR measurements. To identify patients with a functionally significant coronary artery stenosis, analysis is performed in several stages. First, the LV myocardium is segmented using a multiscale convolutional neural network (CNN). To characterize the segmented LV myocardium, it is subsequently encoded using unsupervised convolutional autoencoder (CAE). As ischemic changes are expected to appear locally, the LV myocardium is divided into a number of spatially connected clusters, and statistics of the encodings are computed as features. Thereafter, patients are classified according to the presence of functionally significant stenosis using an SVM classifier based on the extracted features. Quantitative evaluation of LV myocardium segmentation in 20 images resulted in an average Dice coefficient of 0.91 and an average mean absolute distance between the segmented and reference LV boundaries of 0.7 mm. Twenty CCTA images were used to train the LV myocardium encoder. Classification of patients was evaluated in the remaining 126 CCTA scans in 50 10-fold cross-validation experiments and resulted in an area under the receiver operating characteristic curve of 0.74 ± 0.02. At sensitivity levels 0.60, 0.70 and 0.80, the corresponding specificity was 0.77, 0.71 and 0.59, respectively. The results demonstrate that automatic analysis of the LV myocardium in a single CCTA scan acquired at rest, without assessment of the anatomy of the coronary arteries, can be used to identify patients with functionally significant coronary artery stenosis. This might reduce the number of patients undergoing unnecessary invasive FFR measurements.

Fast left ventricle tracking using localized anatomical affine optical flow

In daily clinical cardiology practice, left ventricle (LV) global and regional function assessment is crucial for disease diagnosis, therapy selection, and patient follow-up. Currently, this is still a time-consuming task, spending valuable human resources. In this work, a novel fast methodology for automatic LV tracking is proposed based on localized anatomically constrained affine optical flow. This novel method can be combined to previously proposed segmentation frameworks or manually delineated surfaces at an initial frame to obtain fully delineated datasets and, thus, assess both global and regional myocardial function. Its feasibility and accuracy were investigated in 3 distinct public databases, namely in realistically simulated 3D ultrasound, clinical 3D echocardiography, and clinical cine cardiac magnetic resonance images. The method showed accurate tracking results in all databases, proving its applicability and accuracy for myocardial function assessment. Moreover, when combined to previous state-of-the-art segmentation frameworks, it outperformed previous tracking strategies in both 3D ultrasound and cardiac magnetic resonance data, automatically computing relevant cardiac indices with smaller biases and narrower limits of agreement compared to reference indices. Simultaneously, the proposed localized tracking method showed to be suitable for online processing, even for 3D motion assessment. Importantly, although here evaluated for LV tracking only, this novel methodology is applicable for tracking of other target structures with minimal adaptations.

Constrained Deep Weak Supervision for Histopathology Image Segmentation

In this paper, we develop a new weakly supervised learning algorithm to learn to segment cancerous regions in histopathology images. This paper is under a multiple instance learning (MIL) framework with a new formulation, deep weak supervision (DWS); we also propose an effective way to introduce constraints to our neural networks to assist the learning process. The contributions of our algorithm are threefold: 1) we build an end-to-end learning system that segments cancerous regions with fully convolutional networks (FCNs) in which image-to-image weakly-supervised learning is performed; 2) we develop a DWS formulation to exploit multi-scale learning under weak supervision within FCNs; and 3) constraints about positive instances are introduced in our approach to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. The proposed algorithm, abbreviated as DWS-MIL, is easy to implement and can be trained efficiently. Our system demonstrates the state-of-the-art results on large-scale histopathology image data sets and can be applied to various applications in medical imaging beyond histopathology images, such as MRI, CT, and ultrasound images.

Deep Learning on Sparse Manifolds for Faster Object Segmentation

We propose a new combination of deep belief networks and sparse manifold learning strategies for the 2D segmentation of non-rigid visual objects. With this novel combination, we aim to reduce the training and inference complexities while maintaining the accuracy of machine learning-based non-rigid segmentation methodologies. Typical non-rigid object segmentation methodologies divide the problem into a rigid detection followed by a non-rigid segmentation, where the low dimensionality of the rigid detection allows for a robust training (i.e., a training that does not require a vast amount of annotated images to estimate robust appearance and shape models) and a fast search process during inference. Therefore, it is desirable that the dimensionality of this rigid transformation space is as small as possible in order to enhance the advantages brought by the aforementioned division of the problem. In this paper, we propose the use of sparse manifolds to reduce the dimensionality of the rigid detection space. Furthermore, we propose the use of deep belief networks to allow for a training process that can produce robust appearance models without the need of large annotated training sets. We test our approach in the segmentation of the left ventricle of the heart from ultrasound images and lips from frontal face images. Our experiments show that the use of sparse manifolds and deep belief networks for the rigid detection stage leads to segmentation results that are as accurate as the current state of the art, but with lower search complexity and training processes that require a small amount of annotated training data.

A competitive strategy for atrial and aortic tract segmentation based on deformable models

Multiple strategies have previously been described for atrial region (i.e. atrial bodies and aortic tract) segmentation. Although these techniques have proven their accuracy, inadequate results in the mid atrial walls are common, restricting their application for specific cardiac interventions. In this work, we introduce a novel competitive strategy to perform atrial region segmentation with correct delineation of the thin mid walls, and integrated it into the B-spline Explicit Active Surfaces framework. A double-stage segmentation process is used, which starts with a fast contour growing followed by a refinement stage with local descriptors. Independent functions are used to define each region, being afterward combined to compete for the optimal boundary. The competition locally constrains the surface evolution, prevents overlaps and allows refinement to the walls. Three different scenarios were used to demonstrate the advantages of the proposed approach, through the evaluation of its segmentation accuracy, and its performance for heterogeneous mid walls. Both computed tomography and magnetic resonance imaging datasets were used, presenting results similar to the state-of-the-art methods for both atria and aorta. The competitive strategy showed its superior performance with statistically significant differences against the traditional free-evolution approach in cases with bad image quality or missed atrial/aortic walls. Moreover, only the competitive approach was able to accurately segment the atrial/aortic wall. Overall, the proposed strategy showed to be suitable for atrial region segmentation with a correct segmentation of the mid thin walls, demonstrating its added value with respect to the traditional techniques.

Dynamic Computed Tomography Myocardial Perfusion Imaging: Comparison of Clinical Analysis Methods for the Detection of Vessel-Specific Ischemia

BACKGROUND

The clinical analysis of myocardial dynamic computed tomography myocardial perfusion imaging lacks standardization. The objective of this prospective study was to compare different analysis approaches to diagnose ischemia in patients with stable angina referred for invasive coronary angiography.

METHODS AND RESULTS

Patients referred for evaluation of stable angina symptoms underwent adenosine-stress dynamic computed tomography myocardial perfusion imaging with a second-generation dual-source scanner. Quantitative perfusion parameters, such as blood flow, were calculated by parametric deconvolution for each myocardial voxel. Initially, perfusion parameters were extracted according to standard 17-segment model of the left ventricle (fully automatic analysis). These were then manually sampled by an operator (semiautomatic analysis). Areas under the receiver-operating characteristic curves of the 2 different approaches were compared. Invasive fractional flow reserve ≤0.80 or diameter stenosis ≥80% on quantitative coronary angiography was used as reference standard to define ischemia. We enrolled 115 patients (88 men; age 57±9 years). There were 72 of 286 (25%) vessels causing ischemia in 52 of 115 (45%) patients. The semiautomatic analysis method was better than the fully automatic method at predicting ischemia (areas under the receiver-operating characteristic curves, 0.87 versus 0.69; P<0.001) with readings obtained in the endocardial myocardium performing better than those in the epicardial myocardium (areas under the receiver-operating characteristic curves, 0.87 versus 0.72; P<0.001). The difference in performance between blood flow, expressed as relative to remote myocardium, and absolute blood flow was not statistically significant (areas under the receiver-operating characteristic curves, 0.90 versus 0.87; P=ns).

CONCLUSIONS

Endocardial perfusion parameters obtained by semiautomatic analysis of dynamic computed tomography myocardial perfusion imaging may permit robust discrimination between coronary vessels causing ischemia versus not causing ischemia.

Machine learning-based 3-D geometry reconstruction and modeling of aortic valve deformation using 3-D computed tomography images

To conduct a patient-specific computational modeling of the aortic valve, 3-D aortic valve anatomic geometries of an individual patient need to be reconstructed from clinical 3-D cardiac images. Currently, most of computational studies involve manual heart valve geometry reconstruction and manual finite element (FE) model generation, which is both time-consuming and prone to human errors. A seamless computational modeling framework, which can automate this process based on machine learning algorithms, is desirable, as it can not only eliminate human errors and ensure the consistency of the modeling results but also allow fast feedback to clinicians and permits a future population-based probabilistic analysis of large patient cohorts. In this study, we developed a novel computational modeling method to automatically reconstruct the 3-D geometries of the aortic valve from computed tomographic images. The reconstructed valve geometries have built-in mesh correspondence, which bridges harmonically for the consequent FE modeling. The proposed method was evaluated by comparing the reconstructed geometries from 10 patients with those manually created by human experts, and a mean discrepancy of 0.69 mm was obtained. Based on these reconstructed geometries, FE models of valve leaflets were developed, and aortic valve closure from end systole to middiastole was simulated for 7 patients and validated by comparing the deformed geometries with those manually created by human experts, and a mean discrepancy of 1.57 mm was obtained. The proposed method offers great potential to streamline the computational modeling process and enables the development of a preoperative planning system for aortic valve disease diagnosis and treatment.

Assessing cardiac function from total-variation-regularized 4D C-arm CT in the presence of angular undersampling

Time-resolved tomographic cardiac imaging using an angiographic C-arm device may support clinicians during minimally invasive therapy by enabling a thorough analysis of the heart function directly in the catheter laboratory. However, clinically feasible acquisition protocols entail a highly challenging reconstruction problem which suffers from sparse angular sampling of the trajectory. Compressed sensing theory promises that useful images can be recovered despite massive undersampling by means of sparsity-based regularization. For a multitude of reasons-most notably the desired reduction of scan time, dose and contrast agent required-it is of great interest to know just how little data is actually sufficient for a certain task. In this work, we apply a convex optimization approach based on primal-dual splitting to 4D cardiac C-arm computed tomography. We examine how the quality of spatially and temporally total-variation-regularized reconstruction degrades when using as few as [Formula: see text] projection views per heart phase. First, feasible regularization weights are determined in a numerical phantom study, demonstrating the individual benefits of both regularizers. Secondly, a task-based evaluation is performed in eight clinical patients. Semi-automatic segmentation-based volume measurements of the left ventricular blood pool performed on strongly undersampled images show a correlation of close to 99% with measurements obtained from less sparsely sampled data.

Interactive segmentation in MRI for orthopedic surgery planning: bone tissue

PURPOSE

Planning orthopedic surgeries is commonly performed in computed tomography (CT) images due to the higher contrast of bony structure. However, soft tissues such as muscles and ligaments that may determine the functional outcome of a procedure are not easy to identify in CT, for which fast and accurate segmentation in MRI would be desirable. To be usable in daily practice, such method should provide convenient means of interaction for modifications and corrections, e.g., during perusal by the surgeon or the planning physician for quality control.

METHODS

We propose an interactive segmentation framework for MR images and evaluate the outcome for segmentation of bones. We use a random forest classification and a random walker-based spatial regularization. The latter enables the incorporation of user input as well as enforcing a single connected anatomical structures, thanks to which a selective sampling strategy is proposed to substantially improve the supervised learning performance.

RESULTS

We evaluated our segmentation framework on 10 patient humerus MRI as well as 4 high-resolution MRI from volunteers. Interactive humerus segmentations for patients took on average 150 s with over 3.5 times time-gain compared to manual segmentations, with accuracies comparable (converging) to that of much longer interactions. For high-resolution data, a novel multi-resolution random walker strategy further reduced the run time over 20 times of the manual segmentation, allowing for a feasible interactive segmentation framework.

CONCLUSIONS

We present a segmentation framework that allows iterative corrections leading to substantial speed gains in bone annotation in MRI. This will allow us to pursue semi-automatic segmentations of other musculoskeletal anatomy first in a user-in-the-loop manner, where later less user interactions or perhaps only few for quality control will be necessary as our annotation suggestions improve.

Automatic segmentation and quantification of the cardiac structures from non-contrast-enhanced cardiac CT scans

Early structural changes to the heart, including the chambers and the coronary arteries, provide important information on pre-clinical heart disease like cardiac failure. Currently, contrast-enhanced cardiac computed tomography angiography (CCTA) is the preferred modality for the visualization of the cardiac chambers and the coronaries. In clinical practice not every patient undergoes a CCTA scan; many patients receive only a non-contrast-enhanced calcium scoring CT scan (CTCS), which has less radiation dose and does not require the administration of contrast agent. Quantifying cardiac structures in such images is challenging, as they lack the contrast present in CCTA scans. Such quantification would however be relevant, as it enables population based studies with only a CTCS scan. The purpose of this work is therefore to investigate the feasibility of automatic segmentation and quantification of cardiac structures viz whole heart, left atrium, left ventricle, right atrium, right ventricle and aortic root from CTCS scans. A fully automatic multi-atlas-based segmentation approach is used to segment the cardiac structures. Results show that the segmentation overlap between the automatic method and that of the reference standard have a Dice similarity coefficient of 0.91 on average for the cardiac chambers. The mean surface-to-surface distance error over all the cardiac structures is [Formula: see text] mm. The automatically obtained cardiac chamber volumes using the CTCS scans have an excellent correlation when compared to the volumes in corresponding CCTA scans, a Pearson correlation coefficient (R) of 0.95 is obtained. Our fully automatic method enables large-scale assessment of cardiac structures on non-contrast-enhanced CT scans.