Improving cardiac MRI convolutional neural network segmentation on small training datasets and dataset shift: A continuous kernel cut approach

Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023)

Uncertainty estimation in healthcare involves quantifying and understanding the inherent uncertainty or variability associated with medical predictions, diagnoses, and treatment outcomes. In this era of Artificial Intelligence (AI) models, uncertainty estimation becomes vital to ensure safe decision-making in the medical field. Therefore, this review focuses on the application of uncertainty techniques to machine and deep learning models in healthcare. A systematic literature review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our analysis revealed that Bayesian methods were the predominant technique for uncertainty quantification in machine learning models, with Fuzzy systems being the second most used approach. Regarding deep learning models, Bayesian methods emerged as the most prevalent approach, finding application in nearly all aspects of medical imaging. Most of the studies reported in this paper focused on medical images, highlighting the prevalent application of uncertainty quantification techniques using deep learning models compared to machine learning models. Interestingly, we observed a scarcity of studies applying uncertainty quantification to physiological signals. Thus, future research on uncertainty quantification should prioritize investigating the application of these techniques to physiological signals. Overall, our review highlights the significance of integrating uncertainty techniques in healthcare applications of machine learning and deep learning models. This can provide valuable insights and practical solutions to manage uncertainty in real-world medical data, ultimately improving the accuracy and reliability of medical diagnoses and treatment recommendations.

Attractive deep morphology-aware active contour network for vertebral body contour extraction with extensions to heterogeneous and semi-supervised scenarios

Automatic vertebral body contour extraction (AVBCE) from heterogeneous spinal MRI is indispensable for the comprehensive diagnosis and treatment of spinal diseases. However, AVBCE is challenging due to data heterogeneity, image characteristics complexity, and vertebral body morphology variations, which may cause morphology errors in semantic segmentation. Deep active contour-based (deep ACM-based) methods provide a promising complement for tackling morphology errors by directly parameterizing the contour coordinates. Extending the target contours' capture range and providing morphology-aware parameter maps are crucial for deep ACM-based methods. For this purpose, we propose a novel Attractive Deep Morphology-aware actIve contouR nEtwork (ADMIRE) that embeds an elaborated contour attraction term (CAT) and a comprehensive contour quality (CCQ) loss into the deep ACM-based framework. The CAT adaptively extends the target contours' capture range by designing an all-to-all force field to enable the target contours' energy to contribute to farther locations. Furthermore, the CCQ loss is carefully designed to generate morphology-aware active contour parameters by simultaneously supervising the contour shape, tension, and smoothness. These designs, in cooperation with the deep ACM-based framework, enable robustness to data heterogeneity, image characteristics complexity, and target contour morphology variations. Furthermore, the deep ACM-based ADMIRE is able to cooperate well with semi-supervised strategies such as mean teacher, which enables its function in semi-supervised scenarios. ADMIRE is trained and evaluated on four challenging datasets, including three spinal datasets with more than 1000 heterogeneous images and more than 10000 vertebrae bodies, as well as a cardiac dataset with both normal and pathological cases. Results show ADMIRE achieves state-of-the-art performance on all datasets, which proves ADMIRE's accuracy, robustness, and generalization ability.

SW-UNet: a U-Net fusing sliding window transformer block with CNN for segmentation of lung nodules

Medical images are information carriers that visually reflect and record the anatomical structure of the human body, and play an important role in clinical diagnosis, teaching and research, etc. Modern medicine has become increasingly inseparable from the intelligent processing of medical images. In recent years, there have been more and more attempts to apply deep learning theory to medical image segmentation tasks, and it is imperative to explore a simple and efficient deep learning algorithm for medical image segmentation. In this paper, we investigate the segmentation of lung nodule images. We address the above-mentioned problems of medical image segmentation algorithms and conduct research on medical image fusion algorithms based on a hybrid channel-space attention mechanism and medical image segmentation algorithms with a hybrid architecture of Convolutional Neural Networks (CNN) and Visual Transformer. To the problem that medical image segmentation algorithms are difficult to capture long-range feature dependencies, this paper proposes a medical image segmentation model SW-UNet based on a hybrid CNN and Vision Transformer (ViT) framework. Self-attention mechanism and sliding window design of Visual Transformer are used to capture global feature associations and break the perceptual field limitation of convolutional operations due to inductive bias. At the same time, a widened self-attentive vector is used to streamline the number of modules and compress the model size so as to fit the characteristics of a small amount of medical data, which makes the model easy to be overfitted. Experiments on the LUNA16 lung nodule image dataset validate the algorithm and show that the proposed network can achieve efficient medical image segmentation on a lightweight scale. In addition, to validate the migratability of the model, we performed additional validation on other tumor datasets with desirable results. Our research addresses the crucial need for improved medical image segmentation algorithms. By introducing the SW-UNet model, which combines CNN and ViT, we successfully capture long-range feature dependencies and break the perceptual field limitations of traditional convolutional operations. This approach not only enhances the efficiency of medical image segmentation but also maintains model scalability and adaptability to small medical datasets. The positive outcomes on various tumor datasets emphasize the potential migratability and broad applicability of our proposed model in the field of medical image analysis.

Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.

Explainable Artificial Intelligence and Cardiac Imaging: Toward More Interpretable Models

Artificial intelligence applications have shown success in different medical and health care domains, and cardiac imaging is no exception. However, some machine learning models, especially deep learning, are considered black box as they do not provide an explanation or rationale for model outcomes. Complexity and vagueness in these models necessitate a transition to explainable artificial intelligence (XAI) methods to ensure that model results are both transparent and understandable to end users. In cardiac imaging studies, there are a limited number of papers that use XAI methodologies. This article provides a comprehensive literature review of state-of-the-art works using XAI methods for cardiac imaging. Moreover, it provides simple and comprehensive guidelines on XAI. Finally, open issues and directions for XAI in cardiac imaging are discussed.

An Overview of Deep Learning Methods for Left Ventricle Segmentation

Cardiac health diseases are one of the key causes of death around the globe. The number of heart patients has considerably increased during the pandemic. Therefore, it is crucial to assess and analyze the medical and cardiac images. Deep learning architectures, specifically convolutional neural networks have profoundly become the primary choice for the assessment of cardiac medical images. The left ventricle is a vital part of the cardiovascular system where the boundary and size perform a significant role in the evaluation of cardiac function. Due to automatic segmentation and good promising results, the left ventricle segmentation using deep learning has attracted a lot of attention. This article presents a critical review of deep learning methods used for the left ventricle segmentation from frequently used imaging modalities including magnetic resonance images, ultrasound, and computer tomography. This study also demonstrates the details of the network architecture, software, and hardware used for training along with publicly available cardiac image datasets and self-prepared dataset details incorporated. The summary of the evaluation matrices with results used by different researchers is also presented in this study. Finally, all this information is summarized and comprehended in order to assist the readers to understand the motivation and methodology of various deep learning models, as well as exploring potential solutions to future challenges in LV segmentation.

Cardiac MRI segmentation with sparse annotations: Ensembling deep learning uncertainty and shape priors

The performance of deep learning for cardiac magnetic resonance imaging (MRI) segmentation is oftentimes degraded when using small datasets and sparse annotations for training or adapting a pre-trained model to previously unseen datasets. Here, we developed and evaluated an approach to addressing some of these issues to facilitate broader use of deep learning for short-axis cardiac MRI segmentation. We developed a globally optimal label fusion (GOLF) algorithm that enforced spatial smoothness to generate consensus segmentation from segmentation predictions provided by a deep learning ensemble algorithm. The GOLF consensus was entered into an uncertainty-guided coupled continuous kernel cut (ugCCKC) algorithm that employed normalized cut, image-grid continuous regularization, and "nesting" and circular shape priors of the left ventricular myocardium (LVM) and cavity (LVC). In addition, the uncertainty measurements derived from the segmentation predictions were used to constrain the similarity of GOLF and final segmentation. We optimized ugCCKC through upper bound relaxation, for which we developed an efficient coupled continuous max-flow algorithm implemented in an iterative manner. We showed that GOLF yielded average symmetric surface distance (ASSD) 0.2-0.8 mm lower than an averaging method with higher or similar Dice similarity coefficient (DSC). We also demonstrated that ugCCKC incorporating the shape priors improved DSC by 0.01-0.05 and reduced ASSD by 0.1-0.9 mm. In addition, we integrated GOLF and ugCCKC into a deep learning ensemble algorithm by refining the segmentation of an unannotated dataset and using the refined segmentation to update the trained models. With the proposed framework, we demonstrated the capability of using relatively small datasets (5-10 subjects) with sparse (5-25% slices labeled) annotations to train a deep learning algorithm, while achieving DSC of 0.871-0.893 for LVM and 0.933-0.959 for LVC on the LVQuan dataset, and these were 0.844-0.871 for LVM and 0.923-0.931 for LVC on the ACDC dataset. Furthermore, we showed that the proposed approach can be adapted to substantially alleviate the domain shift issue. Moreover, we calculated a number of commonly used LV function measurements using the derived segmentation and observed strong correlations (Pearson r=0.77-1.00, p<0.001) between algorithm and manual LV function analyses. These results suggest that the developed approaches can be used to facilitate broader application of deep learning in research and clinical cardiac MR imaging workflow.

Deep learning-based automatic segmentation of images in cardiac radiography: A promising challenge

BACKGROUND

Due to the advancement of medical imaging and computer technology, machine intelligence to analyze clinical image data increases the probability of disease prevention and successful treatment. When diagnosing and detecting heart disease, medical imaging can provide high-resolution scans of every organ or tissue in the heart. The diagnostic results obtained by the imaging method are less susceptible to human interference. They can process numerous patient information, assist doctors in early detection of heart disease, intervene and treat patients, and improve the understanding of heart disease symptoms and clinical diagnosis of great significance. In a computer-aided diagnosis system, accurate segmentation of cardiac scan images is the basis and premise of subsequent thoracic function analysis and 3D image reconstruction.

EXISTING TECHNIQUES

This paper systematically reviews automatic methods and some difficulties for cardiac segmentation in radiographic images. Combined with recent advanced deep learning techniques, the feasibility of using deep learning network models for image segmentation is discussed, and the commonly used deep learning frameworks are compared.

DEVELOPED INSIGHTS

There are many standard methods for medical image segmentation, such as traditional methods based on regions and edges and methods based on deep learning. Because of characteristics of non-uniform grayscale, individual differences, artifacts and noise of medical images, the above image segmentation methods have certain limitations. It is tough to obtain the needed results sensitivity and accuracy when performing heart segmentation. The deep learning model proposed has achieved good results in image segmentation. Accurate segmentation improves the accuracy of disease diagnosis and reduces subsequent irrelevant computations.

SUMMARY

There are two requirements for accurate segmentation of radiological images. One is to use image segmentation to improve the development of computer-aided diagnosis. The other is to achieve complete segmentation of the heart. When there are lesions or deformities in the heart, there will be some abnormalities in the radiographic images, and the segmentation algorithm needs to segment the heart altogether. The quantity of processing inside a certain range will no longer be a restriction for real-time detection with the advancement of deep learning and the enhancement of hardware device performance.

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Since the rise of deep learning (DL) in the mid-2010s, cardiac magnetic resonance (CMR) image segmentation has achieved state-of-the-art performance. Despite achieving inter-observer variability in terms of different accuracy performance measures, visual inspections reveal errors in most segmentation results, indicating a lack of reliability and robustness of DL segmentation models, which can be critical if a model was to be deployed into clinical practice. In this work, we aim to bring attention to reliability and robustness, two unmet needs of cardiac image segmentation methods, which are hampering their translation into practice. To this end, we first study the performance accuracy evolution of CMR segmentation, illustrate the improvements brought by DL algorithms and highlight the symptoms of performance stagnation. Afterwards, we provide formal definitions of reliability and robustness. Based on the two definitions, we identify the factors that limit the reliability and robustness of state-of-the-art deep learning CMR segmentation techniques. Finally, we give an overview of the current set of works that focus on improving the reliability and robustness of CMR segmentation, and we categorize them into two families of methods: quality control methods and model improvement techniques. The first category corresponds to simpler strategies that only aim to flag situations where a model may be incurring poor reliability or robustness. The second one, instead, directly tackles the problem by bringing improvements into different aspects of the CMR segmentation model development process. We aim to bring the attention of more researchers towards these emerging trends regarding the development of reliable and robust CMR segmentation frameworks, which can guarantee the safe use of DL in clinical routines and studies.

Portal Vein and Hepatic Vein Segmentation in Multi-Phase MR Images Using Flow-Guided Change Detection

Segmenting portal vein (PV) and hepatic vein (HV) from magnetic resonance imaging (MRI) scans is important for hepatic tumor surgery. Compared with single phase-based methods, multiple phases-based methods have better scalability in distinguishing HV and PV by exploiting multi-phase information. However, these methods just coarsely extract HV and PV from different phase images. In this paper, we propose a unified framework to automatically and robustly segment 3D HV and PV from multi-phase MR images, which considers both the change and appearance caused by the vascular flow event to improve segmentation performance. Firstly, inspired by change detection, flow-guided change detection (FGCD) is designed to detect the changed voxels related to hepatic venous flow by generating hepatic venous phase map and clustering the map. The FGCD uniformly deals with HV and PV clustering by the proposed shared clustering, thus making the appearance correlated with portal venous flow robustly delineate without increasing framework complexity. Then, to refine vascular segmentation results produced by both HV and PV clustering, interclass decision making (IDM) is proposed by combining the overlapping region discrimination and neighborhood direction consistency. Finally, our framework is evaluated on multi-phase clinical MR images of the public dataset (TCGA) and local hospital dataset. The quantitative and qualitative evaluations show that our framework outperforms the existing methods.

Cardiac Magnetic Resonance Left Ventricle Segmentation and Function Evaluation Using a Trained Deep-Learning Model

Cardiac MRI is the gold standard for evaluating left ventricular myocardial mass (LVMM), end-systolic volume (LVESV), end-diastolic volume (LVEDV), stroke volume (LVSV), and ejection fraction (LVEF). Deep convolutional neural networks (CNNs) can provide automatic segmentation of LV myocardium (LVF) and blood cavity (LVC) and quantification of LV function; however, the performance is typically degraded when applied to new datasets. A 2D U-net with Monte-Carlo dropout was trained on 45 cine MR images and the model was used to segment 10 subjects from the ACDC dataset. The initial segmentations were post-processed using a continuous kernel-cut method. The refined segmentations were employed to update the trained model. This procedure was iterated several times and the final updated U-net model was used to segment the remaining 90 ACDC subjects. Algorithm and manual segmentations were compared using Dice coefficient (DSC) and average surface distance in a symmetric manner (ASSD). The relationships between algorithm and manual LV indices were evaluated using Pearson correlation coefficient (r), Bland-Altman analyses, and paired t-tests. Direct application of the pre-trained model yielded DSC of 0.74 ± 0.12 for LVM and 0.87 ± 0.12 for LVC. After fine-tuning, DSC was 0.81 ± 0.09 for LVM and 0.90 ± 0.09 for LVC. Algorithm LV function measurements were strongly correlated with manual analyses (r = 0.86–0.99, p < 0.0001) with minimal biases of −8.8 g for LVMM, −0.9 mL for LVEDV, −0.2 mL for LVESV, −0.7 mL for LVSV, and −0.6% for LVEF. The procedure required ∼12 min for fine-tuning and approximately 1 s to contour a new image on a Linux (Ubuntu 14.02) desktop (Inter(R) CPU i7-7770, 4.2 GHz, 16 GB RAM) with a GPU (GeForce, GTX TITAN X, 12 GB Memory). This approach provides a way to incorporate a trained CNN to segment and quantify previously unseen cardiac MR datasets without needing manual annotation of the unseen datasets.

Dual Convolutional Neural Networks for Breast Mass Segmentation and Diagnosis in Mammography

Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis systems (CADs) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel end-to-end deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predicts diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional consideration of important shape and boundary knowledge. One path, called the Locality Preserving Learner (LPL), is devoted to hierarchically extracting and exploiting intrinsic features of the input. Whereas the other path, called the Conditional Graph Learner (CGL), focuses on generating geometrical features via modeling pixel-wise image to mask correlations. By integrating the two learners, both the cancer semantics and cancer representations are well learned, and the component learning paths in return complement each other, contributing an improvement to the mass segmentation and cancer classification problem at the same time. In addition, by integrating an automatic detection set-up, the DualCoreNet achieves fully automatic breast cancer diagnosis practically. Experimental results show that in benchmark DDSM dataset, DualCoreNet has outperformed other related works in both segmentation and classification tasks, achieving 92.27% DI coefficient and 0.85 AUC score. In another benchmark INbreast dataset, DualCoreNet achieves the best mammography segmentation (93.69% DI coefficient) and competitive classification performance (0.93 AUC score).

Machine learning on small size samples: A synthetic knowledge synthesis

Machine Learning is an increasingly important technology dealing with the growing complexity of the digitalised world. Despite the fact, that we live in a 'Big data' world where, almost 'everything' is digitally stored, there are many real-world situations, where researchers are still faced with small data samples. The present bibliometric knowledge synthesis study aims to answer the research question 'What is the small data problem in machine learning and how it is solved?' The analysis a positive trend in the number of research publications and substantial growth of the research community, indicating that the research field is reaching maturity. Most productive countries are China, United States and United Kingdom. Despite notable international cooperation, the regional concentration of research literature production in economically more developed countries was observed. Thematic analysis identified four research themes. The themes are concerned with to dimension reduction in complex big data analysis, data augmentation techniques in deep learning, data mining and statistical learning on small datasets.

Whole-body Composition Profiling Using a Deep Learning Algorithm: Influence of Different Acquisition Parameters on Algorithm Performance and Robustness

OBJECTIVES

To develop, test, and validate a body composition profiling algorithm for automated segmentation of body compartments in whole-body magnetic resonance imaging (wbMRI) and to investigate the influence of different acquisition parameters on performance and robustness.

MATERIALS AND METHODS

A segmentation algorithm for subcutaneous and visceral adipose tissue (SCAT and VAT) and total muscle mass (TMM) was designed using a deep learning U-net architecture convolutional neuronal network. Twenty clinical wbMRI scans were manually segmented and used as training, validation, and test datasets. Segmentation performance was then tested on different data, including different magnetic resonance imaging protocols and scanners with and without use of contrast media. Test-retest reliability on 2 consecutive scans of 16 healthy volunteers each as well as impact of parameters slice thickness, matrix resolution, and different coil settings were investigated. Sorensen-Dice coefficient (DSC) was used to measure the algorithms' performance with manual segmentations as reference standards. Test-retest reliability and parameter effects were investigated comparing respective compartment volumes. Abdominal volumes were compared with published normative values.

RESULTS

Algorithm performance measured by DSC was 0.93 (SCAT) to 0.77 (VAT) using the test dataset. Dependent from the respective compartment, similar or slightly reduced performance was seen for other scanners and scan protocols (DSC ranging from 0.69-0.72 for VAT to 0.83-0.91 for SCAT). No significant differences in body composition profiling was seen on repetitive volunteer scans (P = 0.88-1) or after variation of protocol parameters (P = 0.07-1).

CONCLUSIONS

Body composition profiling from wbMRI by using a deep learning-based convolutional neuronal network algorithm for automated segmentation of body compartments is generally possible. First results indicate that robust and reproducible segmentations equally accurate to a manual expert may be expected also for a range of different acquisition parameters.

Neural network-based left ventricle geometry prediction from CMR images with application in biomechanics

Combining biomechanical modelling of left ventricular (LV) function and dysfunction with cardiac magnetic resonance (CMR) imaging has the potential to improve the prognosis of patient-specific cardiovascular disease risks. Biomechanical studies of LV function in three dimensions usually rely on a computerized representation of the LV geometry based on finite element discretization, which is essential for numerically simulating in vivo cardiac dynamics. Detailed knowledge of the LV geometry is also relevant for various other clinical applications, such as assessing the LV cavity volume and wall thickness. Accurately and automatically reconstructing personalized LV geometries from conventional CMR images with minimal manual intervention is still a challenging task, which is a pre-requisite for any subsequent automated biomechanical analysis. We propose a deep learning-based automatic pipeline for predicting the three-dimensional LV geometry directly from routinely-available CMR cine images, without the need to manually annotate the ventricular wall. Our framework takes advantage of a low-dimensional representation of the high-dimensional LV geometry based on principal component analysis. We analyze how the inference of myocardial passive stiffness is affected by using our automatically generated LV geometries instead of manually generated ones. These insights will inform the development of statistical emulators of LV dynamics to avoid computationally expensive biomechanical simulations. Our proposed framework enables accurate LV geometry reconstruction, outperforming previous approaches by delivering a reconstruction error 50% lower than reported in the literature. We further demonstrate that for a nonlinear cardiac mechanics model, using our reconstructed LV geometries instead of manually extracted ones only moderately affects the inference of passive myocardial stiffness described by an anisotropic hyperelastic constitutive law. The developed methodological framework has the potential to make an important step towards personalized medicine by eliminating the need for time consuming and costly manual operations. In addition, our method automatically maps the CMR scan into a low-dimensional representation of the LV geometry, which constitutes an important stepping stone towards the development of an LV geometry-heterogeneous emulator.

Deep Learning-Based Measurement of Total Plaque Area in B-Mode Ultrasound Images

Measurement of total-plaque-area (TPA) is important for determining long term risk for stroke and monitoring carotid plaque progression. Since delineation of carotid plaques is required, a deep learning method can provide automatic plaque segmentations and TPA measurements; however, it requires large datasets and manual annotations for training with unknown performance on new datasets. A UNet++ ensemble algorithm was proposed to segment plaques from 2D carotid ultrasound images, trained on three small datasets (n = 33, 33, 34 subjects) and tested on 44 subjects from the SPARC dataset (n = 144, London, Canada). The ensemble was also trained on the entire SPARC dataset and tested with a different dataset (n = 497, Zhongnan Hospital, China). Algorithm and manual segmentations were compared using Dice-similarity-coefficient (DSC), and TPAs were compared using the difference ( ∆TPA), Pearson correlation coefficient (r) and Bland-Altman analyses. Segmentation variability was determined using the intra-class correlation coefficient (ICC) and coefficient-of-variation (CoV). For 44 SPARC subjects, algorithm DSC was 83.3-85.7%, and algorithm TPAs were strongly correlated (r = 0.985-0.988; p < 0.001) with manual results with marginal biases (0.73-6.75) mm ² using the three training datasets. Algorithm ICC for TPAs (ICC = 0.996) was similar to intra- and inter-observer manual results (ICC = 0.977, 0.995). Algorithm CoV = 6.98% for plaque areas was smaller than the inter-observer manual CoV (7.54%). For the Zhongnan dataset, DSC was 88.6% algorithm and manual TPAs were strongly correlated (r = 0.972, p < 0.001) with ∆TPA = -0.44 ±4.05 mm ² and ICC = 0.985. The proposed algorithm trained on small datasets and segmented a different dataset without retraining with accuracy and precision that may be useful clinically and for research.

Ultra-short echo-time magnetic resonance imaging lung segmentation with under-Annotations and domain shift

Ultra-short echo-time (UTE) magnetic resonance imaging (MRI) provides enhanced visualization of pulmonary structural and functional abnormalities and has shown promise in phenotyping lung disease. Here, we describe the development and evaluation of a lung segmentation approach to facilitate UTE MRI methods for patient-based imaging. The proposed approach employs a k-means algorithm in kernel space for pair-wise feature clustering and imposes image domain continuous regularization, coined as continuous kernel k-means (CKKM). The high-order CKKM algorithm was simplified through upper bound relaxation and solved within an iterative continuous max-flow framework. We combined the CKKM with U-net and atlas-based approaches and comprehensively evaluated the performance on 100 images from 25 patients with asthma and bronchial pulmonary dysplasia enrolled at Robarts Research Institute (Western University, London, Canada) and Centre Hospitalier Universitaire (Sainte-Justine, Montreal, Canada). For U-net, we trained the network five times on a mixture of five different images with under-annotations and applied the model to 64 images from the two centres. We also trained a U-net on five images with full and brush annotations from one centre, and tested the model on 32 images from the other centre. For an atlas-based approach, we employed three atlas images to segment 64 target images from the two centres through straightforward atlas registration and label fusion. We applied the CKKM algorithm to the baseline U-net and atlas outputs and refined the initial segmentation through multi-volume image fusion. The integration of CKKM substantially improved baseline results and yielded, with minimal computational cost, segmentation accuracy, and precision that were greater than some state-of-the-art deep learning models and similar to experienced observer manual segmentation. This suggests that deep learning and atlas-based approaches may be utilized to segment UTE MRI datasets using relatively small training datasets with under-annotations.

Potential Role of Artificial Intelligence in Cardiac Magnetic Resonance Imaging: Can It Help Clinicians in Making a Diagnosis?

In the era of modern medicine, artificial intelligence (AI) is a growing field of interest which is experiencing a steady development. Several applications of AI have been applied to various aspects of cardiac magnetic resonance to assist clinicians and engineers in reducing the costs of exams and, at the same time, to improve image acquisition and reconstruction, thus simplifying their analysis, interpretation, and decision-making process as well. In fact, the role of AI and machine learning in cardiovascular imaging relies on evaluating images more quickly, improving their quality, nulling intraobserver and interobserver variability in their interpretation, upgrading the understanding of the stage of the disease, and providing with a personalized approach to cardiovascular care. In addition, AI algorithm could be directed toward workflow management. This article presents an overview of the existing AI literature in cardiac magnetic resonance, with its strengths and limitations, recent applications, and promising developments. We conclude that AI is very likely be used in all the various process of diagnosis routine mode for cardiac care of patients.

Technical Note: Fully automatic segmental relaxometry (FASTR) for cardiac magnetic resonance T1 mapping

PURPOSE

Cardiac relaxometry techniques, particularly T1 mapping, have recently gained clinical importance in various cardiac pathologies. Myocardial T1 and extracellular volume are usually calculated from manual identification of left ventricular epicardial and endocardial regions. This is a laborious process, particularly for large volume studies. Here we present a fully automated relaxometry framework (FASTR) for segmental analysis of T1 maps (both native and postcontrast) and partition coefficient (λ).

METHODS

Patients (N = 11) were imaged postacute myocardial infarction on a 1.5T clinical scanner. The scan protocol involved CINE-SSFP imaging, native, and post-contrast T1 mapping using the Modified Look-Locker Inversion (MOLLI) recovery sequence. FASTR consisted of automatic myocardial segmentation of spatio-temporally coregistered CINE images as an initial guess, followed by refinement of the contours on the T1 maps to derive segmental T1 and λ. T1 and λ were then compared to those obtained from two trained expert observers.

RESULTS

Robust endocardial and epicardial contours were achieved on T1 maps despite the presence of infarcted tissue. Relative to experts, FASTR resulted in myocardial Dice coefficients (native T1: 0.752 ± 0.041; postcontrast T1: 0.751 ± 0.057) that were comparable to interobserver Dice (native T1: 0.803 ± 0.045; postcontrast T1: 0.799 ± 0.054). There were strong correlations observed for T1 and λ derived from experts and FASTR (native T1: r = 0.83; postcontrast T1: r = 0.87; λ: r = 0.78; P < 0.0001), which were comparable to inter-expert correlation coefficients (native T1: r = 0.90; postcontrast T1: r = 0.93; λ: r = 0.80; P < 0.0001).

CONCLUSIONS

Our fully automated framework, FASTR, can generate accurate myocardial segmentations for native and postcontrast MOLLI T1 analysis without the need for manual intervention. Such a design is appealing for high volume clinical protocols.

Sensitivity analysis for interpretation of machine learning based segmentation models in cardiac MRI

BACKGROUND

Image segmentation is a common task in medical imaging e.g., for volumetry analysis in cardiac MRI. Artificial neural networks are used to automate this task with performance similar to manual operators. However, this performance is only achieved in the narrow tasks networks are trained on. Performance drops dramatically when data characteristics differ from the training set properties. Moreover, neural networks are commonly considered black boxes, because it is hard to understand how they make decisions and why they fail. Therefore, it is also hard to predict whether they will generalize and work well with new data. Here we present a generic method for segmentation model interpretation. Sensitivity analysis is an approach where model input is modified in a controlled manner and the effect of these modifications on the model output is evaluated. This method yields insights into the sensitivity of the model to these alterations and therefore to the importance of certain features on segmentation performance.

RESULTS

We present an open-source Python library (misas), that facilitates the use of sensitivity analysis with arbitrary data and models. We show that this method is a suitable approach to answer practical questions regarding use and functionality of segmentation models. We demonstrate this in two case studies on cardiac magnetic resonance imaging. The first case study explores the suitability of a published network for use on a public dataset the network has not been trained on. The second case study demonstrates how sensitivity analysis can be used to evaluate the robustness of a newly trained model.

CONCLUSIONS

Sensitivity analysis is a useful tool for deep learning developers as well as users such as clinicians. It extends their toolbox, enabling and improving interpretability of segmentation models. Enhancing our understanding of neural networks through sensitivity analysis also assists in decision making. Although demonstrated only on cardiac magnetic resonance images this approach and software are much more broadly applicable.

Machine Learning and the Future of Cardiovascular Care: JACC State-of-the-Art Review

The role of physicians has always been to synthesize the data available to them to identify diagnostic patterns that guide treatment and follow response. Today, increasingly sophisticated machine learning algorithms may grow to support clinical experts in some of these tasks. Machine learning has the potential to benefit patients and cardiologists, but only if clinicians take an active role in bringing these new algorithms into practice. The aim of this review is to introduce clinicians who are not data science experts to key concepts in machine learning that will allow them to better understand the field and evaluate new literature and developments. The current published data in machine learning for cardiovascular disease is then summarized, using both a bibliometric survey, with code publicly available to enable similar analysis for any research topic of interest, and select case studies. Finally, several ways that clinicians can and must be involved in this emerging field are presented.

Cine and late gadolinium enhancement MRI registration and automated myocardial infarct heterogeneity quantification

PURPOSE

To develop an approach for automated quantification of myocardial infarct heterogeneity in late gadolinium enhancement (LGE) cardiac MRI.

METHODS

We acquired 2D short-axis cine and 3D LGE in 10 pigs with myocardial infarct. The 2D cine myocardium was segmented and registered to the LGE images. LGE image signal intensities within the warped cine myocardium masks were analyzed to determine the thresholds of infarct core (IC) and gray zone (GZ) for the standard-deviation (SD) and full-width-at-halfmaximum (FWHM) methods. The initial IC, GZ, and IC + GZ segmentations were postprocessed using a normalized cut approach. Cine segmentation and cine-LGE registration accuracies were evaluated using dice similarity coefficient and average symmetric surface distance. Automated IC, GZ, and IC + GZ volumes were compared with manual results using Pearson correlation coefficient (r), Bland-Altman analyses, and intraclass correlation coefficient.

RESULTS

For n = 87 slices containing scar, we achieved cine segmentation dice similarity coefficient = 0.87 ± 0.12, average symmetric surface distance = 0.94 ± 0.74 mm (epicardium), and 1.03 ± 0.82 mm (endocardium) in the scar region. For cine-LGE registration, dice similarity coefficient was 0.90 ± 0.06 and average symmetric surface distance was 0.72 ± 0.39 mm (epicardium) and 0.86 ± 0.53 mm (endocardium) in the scar region. For both SD and FWHM methods, automated IC, GZ, and IC + GZ volumes were strongly (r > 0.70) correlated with manual measurements, and the correlations were not significantly different from interobserver correlations (P > .05). The agreement between automated and manual scar volumes (intraclass correlation coefficient = 0.85-0.96) was similar to that between two observers (intraclass correlation coefficient = 0.81-0.99); automated scar segmentation errors were not significantly different from interobserver segmentation differences (P > .05).

CONCLUSIONS

Our approach provides fully automated cine-LGE MRI registration and LGE myocardial infarct heterogeneity quantification in preclinical studies.

Automated Bone Age Assessment with Image Registration Using Hand X-ray Images

One of the methods for identifying growth disorder is by assessing the skeletal bone age. A child with a healthy growth rate will have approximately the same chronological and bone ages. It is important to detect any growth disorder as early as possible, so that mitigation treatment can be administered with less negative consequences. Recently, the most popular approach in assessing the discrepancy between bone and chronological ages is through the subjective protocol of Tanner–Whitehouse that assesses selected regions in the hand X-ray images. This approach relies heavily on the medical personnel experience, which produces a high intra-observer bias. Therefore, an automated bone age prediction system with image registration using hand X-ray images is proposed in order to complement the inexperienced doctors by providing the second opinion. The system relies on an optimized regression network using a novel residual separable convolution model. The regressor network requires an input image to be 299 × 299 pixels, which will be mapped to the predicted bone age through three modules of the Xception network. Moreover, the images will be pre-processed or registered first to a standardized and normalized pose using separable convolutional neural networks. Three steps image registration are performed by segmenting the hand regions, which will be rotated using angle calculated from four keypoints of interest, before positional alignment is applied to ensure the region of interest is located in the middle. The hand segmentation is based on DeepLab V3 plus architecture, while keypoints regressor for angle alignment is based on MobileNet V1 architecture, where both of them use separable convolution as the core operators. To avoid the pitfall of underfitting, synthetic data are generated while using various rotation angles, zooming factors, and shearing images in order to augment the training dataset. The experimental results show that the proposed method returns the lowest mean absolute error and mean squared error of 8.200 months and 121.902 months2, respectively. Hence, an error of less than one year is acceptable in predicting the bone age, which can serve as a good supplement tool for providing the second expert opinion. This work does not consider gender information, which is crucial in making a better prediction, as the male and female bone structures are naturally different.

Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem

BACKGROUND

Automated segmentation of anatomical structures is a crucial step in image analysis. For lung segmentation in computed tomography, a variety of approaches exists, involving sophisticated pipelines trained and validated on different datasets. However, the clinical applicability of these approaches across diseases remains limited.

METHODS

We compared four generic deep learning approaches trained on various datasets and two readily available lung segmentation algorithms. We performed evaluation on routine imaging data with more than six different disease patterns and three published data sets.

RESULTS

Using different deep learning approaches, mean Dice similarity coefficients (DSCs) on test datasets varied not over 0.02. When trained on a diverse routine dataset (n = 36), a standard approach (U-net) yields a higher DSC (0.97 ± 0.05) compared to training on public datasets such as the Lung Tissue Research Consortium (0.94 ± 0.13, p = 0.024) or Anatomy 3 (0.92 ± 0.15, p = 0.001). Trained on routine data (n = 231) covering multiple diseases, U-net compared to reference methods yields a DSC of 0.98 ± 0.03 versus 0.94 ± 0.12 (p = 0.024).

CONCLUSIONS

The accuracy and reliability of lung segmentation algorithms on demanding cases primarily relies on the diversity of the training data, highlighting the importance of data diversity compared to model choice. Efforts in developing new datasets and providing trained models to the public are critical. By releasing the trained model under General Public License 3.0, we aim to foster research on lung diseases by providing a readily available tool for segmentation of pathological lungs.

Ranking Information Extracted from Uncertainty Quantification of the Prediction of a Deep Learning Model on Medical Time Series Data

Uncertainty quantification in deep learning models is especially important for the medical applications of this complex and successful type of neural architectures. One popular technique is Monte Carlo dropout that gives a sample output for a record, which can be measured statistically in terms of average probability and variance for each diagnostic class of the problem. The current paper puts forward a convolutional–long short-term memory network model with a Monte Carlo dropout layer for obtaining information regarding the model uncertainty for saccadic records of all patients. These are next used in assessing the uncertainty of the learning model at the higher level of sets of multiple records (i.e., registers) that are gathered for one patient case by the examining physician towards an accurate diagnosis. Means and standard deviations are additionally calculated for the Monte Carlo uncertainty estimates of groups of predictions. These serve as a new collection where a random forest model can perform both classification and ranking of variable importance. The approach is validated on a real-world problem of classifying electrooculography time series for an early detection of spinocerebellar ataxia 2 and reaches an accuracy of 88.59% in distinguishing between the three classes of patients.

Automated Detection of Presymptomatic Conditions in Spinocerebellar Ataxia Type 2 Using Monte Carlo Dropout and Deep Neural Network Techniques with Electrooculogram Signals

Application of deep learning (DL) to the field of healthcare is aiding clinicians to make an accurate diagnosis. DL provides reliable results for image processing and sensor interpretation problems most of the time. However, model uncertainty should also be thoroughly quantified. This paper therefore addresses the employment of Monte Carlo dropout within the DL structure to automatically discriminate presymptomatic signs of spinocerebellar ataxia type 2 in saccadic samples obtained from electrooculograms. The current work goes beyond the common incorporation of this special type of dropout into deep neural networks and uses the uncertainty derived from the validation samples to construct a decision tree at the register level of the patients. The decision tree built from the uncertainty estimates obtained a classification accuracy of 81.18% in automatically discriminating control, presymptomatic and sick classes. This paper proposes a novel method to address both uncertainty quantification and explainability to develop reliable healthcare support systems.