Markov random field segmentation of brain MR images

ETUNet:Exploring efficient transformer enhanced UNet for 3D brain tumor segmentation

Medical image segmentation is a crucial topic in medical image processing. Accurately segmenting brain tumor regions from multimodal MRI scans is essential for clinical diagnosis and survival prediction. However, similar intensity distributions, variable tumor shapes, and fuzzy boundaries pose severe challenges for brain tumor segmentation. Traditional segmentation networks based on UNet struggle to establish explicit long-range dependencies from the feature space due to the limitations of the CNN receptive field. This is particularly crucial for dense prediction tasks such as brain tumor segmentation. Recent works have incorporated the powerful global modeling capability of Transformer into UNet to achieve more precise segmentation results. Nevertheless, these methods encounter some issues: (1) the global information is often modeled by simply stacking Transformer layers for a specific module, resulting in high computational complexity and underutilization of the potential of the UNet architecture; (2) the rich boundary information of tumor subregions in multi-scale features is often overlooked. Motivated by these challenges, we propose an advanced fusion of Transformer with UNet by reexamining the core three parts (encoder, bottleneck, and skip connections). Firstly, we introduce a CNN-Transformer module in the encoder to replace the traditional CNN module, enabling the capture of deep spatial dependencies from input images. To address high-level semantic information, we incorporate a computationally efficient spatial-channel attention layer in the bottleneck for global interaction, highlighting important semantic features from the encoder path output. For irregular lesions, we fuse the multi-scale features from the encoder output and the decoder features in the skip connections by calculating cross-attention. This adaptive querying of valuable information from multi-scale features enhances the boundary localization ability of the decoder path and suppresses redundant features with low correlation. Compared to existing methods, our model further enhances the learning capacity of the overall UNet architecture while maintaining low computational complexity. Experimental results on the BraTS2018 and BraTS2020 datasets for brain tumor segmentation tasks demonstrate that our model achieves comparable or superior results compared to recent CNN or Transformer-based models. The average DSC and HD95 on the two datasets are 0.854, 6.688, and 0.862, 5.455 respectively. At the same time, our model achieves optimal segmentation of Enhancing tumors, showcasing the effectiveness of our method. Our code will be made publicly available at https://github.com/wzhangck/ETUnet.

Graphical Modeling for Multi-Source Domain Adaptation

Multi-Source Domain Adaptation (MSDA) focuses on transferring the knowledge from multiple source domains to the target domain, which is a more practical and challenging problem compared to the conventional single-source domain adaptation. In this problem, it is essential to model multiple source domains and target domain jointly, and an effective domain combination scheme is also highly required. The graphical structure among different domains is useful to tackle these challenges, in which the interdependency among various instances/categories can be effectively modeled. In this work, we propose two types of graphical models, i.e. Conditional Random Field for MSDA (CRF-MSDA) and Markov Random Field for MSDA (MRF-MSDA), for cross-domain joint modeling and learnable domain combination. In a nutshell, given an observation set composed of a query sample and the semantic prototypes (i.e. representative category embeddings) on various domains, the CRF-MSDA model seeks to learn the joint distribution of labels conditioned on the observations. We attain this goal by constructing a relational graph over all observations and conducting local message passing on it. By comparison, MRF-MSDA aims to model the joint distribution of observations over different Markov networks via an energy-based formulation, and it can naturally perform label prediction by summing the joint likelihoods over several specific networks. Compared to the CRF-MSDA counterpart, the MRF-MSDA model is more expressive and possesses lower computational cost. We evaluate these two models on four standard benchmark data sets of MSDA with distinct domain shift and data complexity, and both models achieve superior performance over existing methods on all benchmarks. In addition, the analytical studies illustrate the effect of different model components and provide insights about how the cross-domain joint modeling performs.

Target area distillation and section attention segmentation network for accurate 3D medical image segmentation

3D medical image segmentation has an essential role in medical image analysis, while attention mechanism has improved the performance by a large margin. However, existing methods obtained the attention coefficient in a small receptive field, resulting in possible performance limitations. Radiologists usually scan all the slices first to have an overall idea of the target, and then analyze regions of interest in multiple 2D views in clinic practice. We simulate radiologists' recognition process and propose to exploit the 3D context information in a deeper manner for accurate 3D medical images segmentation. Due to the similarity of human body structure, medical images of different populations have highly similar shape and location information, so we use target region distillation to extract the common segmented region information. Particularly, we proposed two optimizations including Target Area Distillation and Section Attention. Target Area Distillation adds positions information to the original input to let the network has an initial attention of the target, while section attention performs attention extraction in three 2D sections thus with large range of receptive field. We compare our method against several popular networks in two public datasets including ImageCHD and COVID-19. Experimental results show that our proposed method improves the segmentation Dice score by 2-4% over the state-of-the-art methods. Our code has been released to the public (Anonymous link).

DSKCA-UNet: Dynamic selective kernel channel attention for medical image segmentation

U-Net has attained immense popularity owing to its performance in medical image segmentation. However, it cannot be modeled explicitly over remote dependencies. By contrast, the transformer can effectively capture remote dependencies by leveraging the self-attention (SA) of the encoder. Although SA, an important characteristic of the transformer, can find correlations between them based on the original data, secondary computational complexity might retard the processing rate of high-dimensional data (such as medical images). Furthermore, SA is limited because the correlation between samples is overlooked; thus, there is considerable scope for improvement. To this end, based on Swin-UNet, we introduce a dynamic selective attention mechanism for the convolution kernels. The weight of each convolution kernel is calculated to fuse the results dynamically. This attention mechanism permits each neuron to adaptively modify its receptive field size in response to multiscale input information. A local cross-channel interaction strategy without dimensionality reduction was introduced, which effectively eliminated the influence of downscaling on learning channel attention. Through suitable cross-channel interactions, model complexity can be significantly reduced while maintaining its performance. Subsequently, the global interaction between the encoder features is used to extract more fine-grained features. Simultaneously, the mixed loss function of the weighted cross-entropy loss and Dice loss is used to alleviate category imbalances and achieve better results when the sample number is unbalanced. We evaluated our proposed method on abdominal multiorgan segmentation and cardiac segmentation datasets, achieving Dice similarity coefficient and 95% Hausdorff distance metrics of 80.30 and 14.55%, respectively, on the Synapse dataset and Dice similarity coefficient metrics of 90.80 on the ACDC dataset. The experimental results show that our proposed method has good generalization ability and robustness, and it is a powerful tool for medical image segmentation.

Survey of methods and principles in three-dimensional reconstruction from two-dimensional medical images

Three-dimensional (3D) reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units. In the coming years, most patient care will shift toward this new paradigm. However, development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved, most of which are dependent on human expertise. In this review, a survey of pre-processing steps was conducted, and reconstruction techniques for several organs in medical diagnosis were studied. Various methods and principles related to 3D reconstruction were highlighted. The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.

Impact of Training Data, Ground Truth and Shape Variability in the Deep Learning-Based Semantic Segmentation of HeLa Cells Observed with Electron Microscopy

This paper investigates the impact of the amount of training data and the shape variability on the segmentation provided by the deep learning architecture U-Net. Further, the correctness of ground truth (GT) was also evaluated. The input data consisted of a three-dimensional set of images of HeLa cells observed with an electron microscope with dimensions 8192×8192×517. From there, a smaller region of interest (ROI) of 2000×2000×300 was cropped and manually delineated to obtain the ground truth necessary for a quantitative evaluation. A qualitative evaluation was performed on the 8192×8192 slices due to the lack of ground truth. Pairs of patches of data and labels for the classes nucleus, nuclear envelope, cell and background were generated to train U-Net architectures from scratch. Several training strategies were followed, and the results were compared against a traditional image processing algorithm. The correctness of GT, that is, the inclusion of one or more nuclei within the region of interest was also evaluated. The impact of the extent of training data was evaluated by comparing results from 36,000 pairs of data and label patches extracted from the odd slices in the central region, to 135,000 patches obtained from every other slice in the set. Then, 135,000 patches from several cells from the 8192×8192 slices were generated automatically using the image processing algorithm. Finally, the two sets of 135,000 pairs were combined to train once more with 270,000 pairs. As would be expected, the accuracy and Jaccard similarity index improved as the number of pairs increased for the ROI. This was also observed qualitatively for the 8192×8192 slices. When the 8192×8192 slices were segmented with U-Nets trained with 135,000 pairs, the architecture trained with automatically generated pairs provided better results than the architecture trained with the pairs from the manually segmented ground truths. This suggests that the pairs that were extracted automatically from many cells provided a better representation of the four classes of the various cells in the 8192×8192 slice than those pairs that were manually segmented from a single cell. Finally, the two sets of 135,000 pairs were combined, and the U-Net trained with these provided the best results.

ATTransUNet: An enhanced hybrid transformer architecture for ultrasound and histopathology image segmentation

Recently, researchers have introduced Transformer into medical image segmentation networks to encode long-range dependency, which makes up for the deficiencies of convolutional neural networks (CNNs) in global context modeling, and thus improves segmentation performance. However, in Transformer, due to the heavy computational burden of paired attention modeling between redundant visual tokens, the efficiency of Transformer needs to be further improved. Therefore, in this paper, we propose ATTransUNet, a Transformer enhanced hybrid architecture based on the adaptive token for ultrasound and histopathology image segmentation. In the encoding stage of the ATTransUNet, we introduced an Adaptive Token Extraction Module (ATEM), which can mine a few important visual tokens in the image for self-attention modeling, thus reducing the complexity of the model and improving the segmentation accuracy. In addition, in the decoding stage, we introduce a Selective Feature Reinforcement Module (SFRM) to reinforce the representation of and attention to key tissues or pathological features. The proposed ATTransUNet is evaluated on the basis of three medical image segmentation datasets. The results show that ATTransUNet achieves the best segmentation performance compared with the previous state-of-the-art models, and the proposed method is also competitive in terms of the network parameters and computation.

Swin Transformer Assisted Prior Attention Network for Medical Image Segmentation

Transformer complements convolutional neural network (CNN) has achieved better performance than improved CNN-based methods. Specially, Transformer is utilized to be combined with U-shaped structure, skip-connections, encoder, and even them all together. However, the intermediate supervision network based on the coarse-to-fine strategy has not been combined with Transformer to improve the generalization of CNN-based methods. In this paper, we propose Swin-PANet, which is applying a window-based self-attention mechanism by Swin Transformer in the intermediate supervision network, called prior attention network. A new enhanced attention block based on CCA is also proposed to aggregate the features from skip-connections and prior attention network, and further refine details of boundaries. Swin-PANet can address the dilemma that traditional Transformer network has poor interpretability in the process of attention calculation and Swin-PANet can insert its attention predictions into prior attention network for intermediate supervision learning which is humanly interpretable and controllable. Hence, the intermediate supervision network assisted by Swin Transformer provides better attention learning and interpretability in network for accurate and automatic medical image segmentation. The experimental results evaluate the effectiveness of Swin-PANet which outperforms state-of-the-art methods in some famous medical segmentation tasks including cell and skin lesion segmentation.

A cascaded nested network for 3T brain MR image segmentation guided by 7T labeling

Accurate segmentation of the brain into gray matter, white matter, and cerebrospinal fluid using magnetic resonance (MR) imaging is critical for visualization and quantification of brain anatomy. Compared to 3T MR images, 7T MR images exhibit higher tissue contrast that is contributive to accurate tissue delineation for training segmentation models. In this paper, we propose a cascaded nested network (CaNes-Net) for segmentation of 3T brain MR images, trained by tissue labels delineated from the corresponding 7T images. We first train a nested network (Nes-Net) for a rough segmentation. The second Nes-Net uses tissue-specific geodesic distance maps as contextual information to refine the segmentation. This process is iterated to build CaNes-Net with a cascade of Nes-Net modules to gradually refine the segmentation. To alleviate the misalignment between 3T and corresponding 7T MR images, we incorporate a correlation coefficient map to allow well-aligned voxels to play a more important role in supervising the training process. We compared CaNes-Net with SPM and FSL tools, as well as four deep learning models on 18 adult subjects and the ADNI dataset. Our results indicate that CaNes-Net reduces segmentation errors caused by the misalignment and improves segmentation accuracy substantially over the competing methods.

Frontal Cortex Segmentation of Brain PET Imaging Using Deep Neural Networks

¹⁸F-FDG positron emission tomography (PET) imaging of brain glucose use and amyloid accumulation is a research criteria for Alzheimer's disease (AD) diagnosis. Several PET studies have shown widespread metabolic deficits in the frontal cortex for AD patients. Therefore, studying frontal cortex changes is of great importance for AD research. This paper aims to segment frontal cortex from brain PET imaging using deep neural networks. The learning framework called Frontal cortex Segmentation model of brain PET imaging (FSPET) is proposed to tackle this problem. It combines the anatomical prior to frontal cortex into the segmentation model, which is based on conditional generative adversarial network and convolutional auto-encoder. The FSPET method is evaluated on a dataset of 30 brain PET imaging with ground truth annotated by a radiologist. Results that outperform other baselines demonstrate the effectiveness of the FSPET framework.

AugMS-Net:Augmented multiscale network for small cervical tumor segmentation from MRI volumes

Cervical cancer is one of the leading causes of female-specific cancer death. Tumor region segmentation plays a pivotal role in both the clinical analysis and treatment planning of cervical cancer. Due to the heterogeneity and low contrast of biomedical images, current state-of-the-art tumor segmentation approaches are facing the challenge of the insensitive detection of small lesion regions. To tackle this problem, this paper proposes an augmented multiscale network (AugMS-Net) based on 3D U-Net to automatically segment cervical Magnetic Resonance Imaging (MRI) volumes. Since a multiscale strategy is considered one of the most promising algorithms to tackle small object recognition, we introduce a novel 3D module to explore more granular multiscale representations. Besides, we employ a deep multiscale supervision strategy to doubly supervise the side outputs hierarchically. To demonstrate the generalization of our model, we evaluated AugMS-Net on both a cervical dataset from MRI volumes and a liver dataset from Computerized Tomography (CT) volumes. Our proposed AugMS-Net shows superior performance over baseline models, yielding high accuracy while reducing the number of model parameters by nearly 20%. The source code and trained models are available at https://github.com/Cassieyy/AugMS-Net.

Weaving attention U-net: A novel hybrid CNN and attention-based method for organs-at-risk segmentation in head and neck CT images

PURPOSE

In radiotherapy planning, manual contouring is labor-intensive and time-consuming. Accurate and robust automated segmentation models improve the efficiency and treatment outcome. We aim to develop a novel hybrid deep learning approach, combining convolutional neural networks (CNNs) and the self-attention mechanism, for rapid and accurate multi-organ segmentation on head and neck computed tomography (CT) images.

METHODS

Head and neck CT images with manual contours of 115 patients were retrospectively collected and used. We set the training/validation/testing ratio to 81/9/25 and used the 10-fold cross-validation strategy to select the best model parameters. The proposed hybrid model segmented 10 organs-at-risk (OARs) altogether for each case. The performance of the model was evaluated by three metrics, that is, the Dice Similarity Coefficient (DSC), Hausdorff distance 95% (HD95), and mean surface distance (MSD). We also tested the performance of the model on the head and neck 2015 challenge dataset and compared it against several state-of-the-art automated segmentation algorithms.

RESULTS

The proposed method generated contours that closely resemble the ground truth for 10 OARs. On the head and neck 2015 challenge dataset, the DSC scores of these OARs were 0.91 ± 0.02, 0.73 ± 0.10, 0.95 ± 0.03, 0.76 ± 0.08, 0.79 ± 0.05, 0.87 ± 0.05, 0.86 ± 0.08, 0.87 ± 0.03, and 0.87 ± 0.07 for brain stem, chiasm, mandible, left/right optic nerve, left/right submandibular, and left/right parotid, respectively. Our results of the new weaving attention U-net (WAU-net) demonstrate superior or similar performance on the segmentation of head and neck CT images.

CONCLUSIONS

We developed a deep learning approach that integrates the merits of CNNs and the self-attention mechanism. The proposed WAU-net can efficiently capture local and global dependencies and achieves state-of-the-art performance on the head and neck multi-organ segmentation task.

Spatio-temporal multi-task network cascade for accurate assessment of cardiac CT perfusion

The assessment of myocardial perfusion has become increasingly important in the early diagnosis of coronary artery disease. Currently, the process of perfusion assessment is time-consuming and subjective. Although automated methods by threshold processing have been proposed, they cannot obtain an accurate perfusion assessment. Thus, there is a great clinical demand to obtain a rapid and accurate assessment of myocardial perfusion through a standard procedure using an automated algorithm. In this work, we present a spatio-temporal multi-task network cascade (ST-MNC) to provide an accurate and robust assessment of myocardial perfusion. The proposed network captures patch-based spatio-temporal representations for each pixel through a spatio-temporal encoder-decoder network. Then the multi-task network cascade uses spatio-temporal representations as shared features to predict various perfusion parameters and myocardial ischemic regions. Extensive experiments on CT images of 232 subjects demonstrate ST-MNC could produce a good approximation for perfusion parameters and an accurate classification for ischemic regions. These results show that our proposed method can provide a fast and accurate assessment of myocardial perfusion.

A metric learning method for estimating myelin content based on T2-weighted MRI from a de- and re-myelination model of multiple sclerosis

Myelin plays a critical role in the pathogenesis of neurological disorders but is difficult to characterize in vivo using standard analysis methods. Our goal was to develop a novel analytical framework for estimating myelin content using T2-weighted magnetic resonance imaging (MRI) based on a de- and re-myelination model of multiple sclerosis. We examined 18 mice with lysolecithin induced demyelination and spontaneous remyelination in the ventral white matter of thoracic spinal cord. Cohorts of 6 mice underwent 9.4T MRI at days 7 (peak demyelination), 14 (ongoing recovery), and 28 (near complete recovery), as well as histological analysis of myelin and the associated cellularity at corresponding timepoints. Our MRI framework took an unsupervised learning approach, including tissue segmentation using a Gaussian Markov random field (GMRF), and myelin and cellularity feature estimation based on the Mahalanobis distance. For comparison, we also investigated 2 regression-based supervised learning approaches, one using our GMRF results, and another using a freely available generalized additive model (GAM). Results showed that GMRF segmentation was 73.2% accurate, and our unsupervised learning method achieved a correlation coefficient of 0.67 (top quartile: 0.78) with histological myelin, similar to 0.70 (top quartile: 0.78) obtained using supervised analyses. Further, the area under the receiver operator characteristic curve of our unsupervised myelin feature (0.883, 95% CI: 0.874-0.891) was significantly better than any of the supervised models in detecting white matter myelin as compared to histology. Collectively, metric learning using standard MRI may prove to be a new alternative method for estimating myelin content, which ultimately can improve our disease monitoring ability in a clinical setting.

Machine learning techniques for biomedical image segmentation: An overview of technical aspects and introduction to state-of-art applications

In recent years, significant progress has been made in developing more accurate and efficient machine learning algorithms for segmentation of medical and natural images. In this review article, we highlight the imperative role of machine learning algorithms in enabling efficient and accurate segmentation in the field of medical imaging. We specifically focus on several key studies pertaining to the application of machine learning methods to biomedical image segmentation. We review classical machine learning algorithms such as Markov random fields, k-means clustering, random forest, etc. Although such classical learning models are often less accurate compared to the deep-learning techniques, they are often more sample efficient and have a less complex structure. We also review different deep-learning architectures, such as the artificial neural networks (ANNs), the convolutional neural networks (CNNs), and the recurrent neural networks (RNNs), and present the segmentation results attained by those learning models that were published in the past 3 yr. We highlight the successes and limitations of each machine learning paradigm. In addition, we discuss several challenges related to the training of different machine learning models, and we present some heuristics to address those challenges.

Left ventricle segmentation using a Bayesian approach with distance dependent shape priors

We propose a method for segmentation of the left ventricle in magnetic resonance cardiac images. The framework consists of an initial Bayesian segmentation of the central slice of the volume. This segmentation is used to locate a shape prior for the LV myocardial tissue. This shape prior is determined using the fact that the myocardium is approximately annular as seen in the short-axis. Then a second Bayesian segmentation is performed to obtain the final result. This procedure is repeated for the rest of the slices. An extrapolation of the area of the LV is used to determine a stopping criterion. The method was evaluated on the databases of the Cardiac Atlas project. Our results demonstrate a suitable accuracy for myocardial segmentation (≈0.8 Dice's coefficient). For the endocardium and the epicardium the Dice's coefficients are 0.94 and 0.9 respectively. The accuracy was also evaluated in terms of the Hausdorff distance and the average distance. For the myocardium we obtain 8 mm and 2 mm respectively. Our results demonstrate the capability and merits of the proposed method to estimate the structure of the LV. The method requires minimal user input and generates results with quality comparable to more complex approaches. This paper suggests a new efficient approach for automatic LV quantification based on a Bayesian technique with shape priors with errors comparable to state-of-the-art techniques.

Relative rate of change in cognitive score network dynamics via Bayesian hierarchical models reveal spatial patterns of neurodegeneration

The degeneration of the human brain is a complex process, which often affects certain brain regions due to healthy aging or disease. This degeneration can be evaluated on regions of interest (ROI) in the brain through probabilistic networks and morphological estimates. Current approaches for finding such networks are limited to analyses at discrete neuropsychological stages, which cannot appropriately account for connectivity dynamics over the onset of cognitive deterioration, and morphological changes are seldom unified with connectivity networks, despite known dependencies. To overcome these limitations, a probabilistic wombling model is proposed to simultaneously estimate ROI cortical thickness and covariance networks contingent on rates of change in cognitive decline. This proposed model was applied to analyze longitudinal data from healthy control (HC) and Alzheimer's disease (AD) groups and found connection differences pertaining to regions, which play a crucial role in lasting cognitive impairment, such as the entorhinal area and temporal regions. Moreover, HC cortical thickness estimates were significantly higher than those in the AD group across all ROIs. The analyses presented in this work will help practitioners jointly analyze brain tissue atrophy at the ROI-level conditional on neuropsychological networks, which could potentially allow for more targeted therapeutic interventions.

PECLIDES Neuro: A Personalisable Clinical Decision Support System for Neurological Diseases

Neurodegenerative diseases such as Alzheimer's and Parkinson's impact millions of people worldwide. Early diagnosis has proven to greatly increase the chances of slowing down the diseases' progression. Correct diagnosis often relies on the analysis of large amounts of patient data, and thus lends itself well to support from machine learning algorithms, which are able to learn from past diagnosis and see clearly through the complex interactions of a patient's symptoms and data. Unfortunately, many contemporary machine learning techniques fail to reveal details about how they reach their conclusions, a property considered fundamental when providing a diagnosis. Here we introduce our Personalisable Clinical Decision Support System (PECLIDES), an algorithmic process formulated to address this specific fault in diagnosis detection. PECLIDES provides a clear insight into the decision-making process leading to a diagnosis, making it a gray box model. Our algorithm enriches the fundamental work of Masheyekhi and Gras in data integration, personal medicine, usability, visualization, and interactivity. Our decision support system is an operation of translational medicine. It is based on random forests, is personalisable and allows a clear insight into the decision-making process. A well-structured rule set is created and every rule of the decision-making process can be observed by the user (physician). Furthermore, the user has an impact on the creation of the final rule set and the algorithm allows the comparison of different diseases as well as regional differences in the same disease. The algorithm is applicable to various decision problems. In this paper we will evaluate it on diagnosing neurological diseases and therefore refer to the algorithm as PECLIDES Neuro.

A Spatially Constrained Probabilistic Model for Robust Image Segmentation

In general, the hidden Markov random field (HMRF) represents the class label distribution of an image in probabilistic model based segmentation. The class label distributions provided by existing HMRF models consider either the number of neighboring pixels with similar class labels or the spatial distance of neighboring pixels with dissimilar class labels. Also, this spatial information is only considered for estimation of class labels of the image pixels, while its contribution in parameter estimation is completely ignored. This, in turn, deteriorates the parameter estimation, resulting in sub-optimal segmentation performance. Moreover, the existing models assign equal weightage to the spatial information for class label estimation of all pixels throughout the image, which, create significant misclassification for the pixels in boundary region of image classes. In this regard, the paper develops a new clique potential function and a new class label distribution, incorporating the information of image class parameters. Unlike existing HMRF model based segmentation techniques, the proposed framework introduces a new scaling parameter that adaptively measures the contribution of spatial information for class label estimation of image pixels. The importance of the proposed framework is depicted by modifying the HMRF based segmentation methods. The advantage of proposed class label distribution is also demonstrated irrespective of the underlying intensity distributions. The comparative performance of the proposed and existing class label distributions in HMRF model is demonstrated both qualitatively and quantitatively for brain MR image segmentation, HEp-2 cell delineation, natural image and object segmentation.

Knee Meniscus Segmentation and Tear Detection from MRI: A Review

Background:

Automatic diagnostic systems in medical imaging provide useful information to support radiologists and other relevant experts. The systems that help radiologists in their analysis and diagnosis appear to be increasing.

Discussion:

Knee joints are intensively studied structures, as well. In this review, studies that automatically segment meniscal structures from the knee joint MR images and detect tears have been investigated. Some of the studies in the literature merely perform meniscus segmentation, while others include classification procedures that detect both meniscus segmentation and anomalies on menisci. The studies performed on the meniscus were categorized according to the methods they used. The methods used and the results obtained from such studies were analyzed along with their drawbacks, and the aspects to be developed were also emphasized.

Conclusion:

The work that has been done in this area can effectively support the decisions that will be made by radiology and orthopedics specialists. Furthermore, these operations, which were performed manually on MR images, can be performed in a shorter time with the help of computeraided systems, which enables early diagnosis and treatment.

Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images

The author proposes a method for simultaneous registration and segmentation of multi-source images, using the multivariate mixture model (MvMM) and maximum of log-likelihood (LL) framework. Specifically, the method is applied to the problem of myocardial segmentation combining the complementary information from multi-sequence (MS) cardiac magnetic resonance (CMR) images. For the image misalignment and incongruent data, the MvMM is formulated with transformations and is further generalized for dealing with the hetero-coverage multi-modality images (HC-MMIs). The segmentation of MvMM is performed in a virtual common space, to which all the images and misaligned slices are simultaneously registered. Furthermore, this common space can be divided into a number of sub-regions, each of which contains congruent data, thus the HC-MMIs can be modeled using a set of conventional MvMMs. Results show that MvMM obtained significantly better performance compared to the conventional approaches and demonstrated good potential for scar quantification as well as myocardial segmentation. The generalized MvMM has also demonstrated better robustness in the incongruent data, where some images may not fully cover the region of interest, and the full coverage can only be reconstructed combining the images from multiple sources.

A potential field segmentation based method for tumor segmentation on multi-parametric MRI of glioma cancer patients

Background

Accurate segmentation of brain tumors is vital for the gross tumor volume (GTV) definition in radiotherapy. Functional MR images like apparent diffusion constant (ADC) and fractional anisotropy (FA) images can provide more comprehensive information for sensitive detection of the GTV. We synthesize anatomical and functional MRI for accurate and semi-automatic segmentation of GTVs and improvement of clinical efficiency.

Methods

Four MR image sets including T1-weighted contrast-enhanced (T1C), T2-weighted (T2), apparent diffusion constant (ADC) and fractional anisotropy (FA) images of 5 glioma patients were acquired and registered. A new potential field segmentation (PFS) method was proposed based on the concept of potential field in physics. For T1C, T2 and ADC images, global potential field segmentation (global-PFS) was used on user defined region of interest (ROI) for rough segmentation and then morphologically processed for accurate delineation of the GTV. For FA images, white matter (WM) was removed using local potential field segmentation (local-PFS), and then tumor extent was delineated with region growing and morphological methods. The individual segmentations of multi-parametric images were ensembled into a fused segmentation, considered as final GTV. GTVs were compared with manually delineated ground truth and evaluated with segmentation quality measure (Q), Dice’s similarity coefficient (DSC) and Sensitivity and Specificity.

Results

Experimental study with the five patients’ data and new method showed that, the mean values of Q, DSC, Sensitivity and Specificity were 0.80 (±0.07), 0.88 (±0.04), 0.92 (±0.01) and 0.88 (±0.05) respectively. The global-PFS used on ROIs of T1C, T2 and ADC images can avoid interferences from skull and other non-tumor areas. Similarity to local-PFS on FA images, it can also reduce the time complexity as compared with the global-PFS on whole image sets.

Conclusions

Efficient and semi-automatic segmentation of the GTV can be achieved with the new method. Combination of anatomical and functional MR images has the potential to provide new methods and ideas for target definition in radiotherapy.

The Adaptive Fractional Order Differential Model for Image Enhancement Based on Segmentation

For image enhancement method based on the fractional order differential, it is difficult to artificially give the optimal order of the fractional differential which can make the image enhancement effect better, and it is hard to ensure the enhancement of the target object while preserving the information of background pixels if the entire image is filtered by a fixed differential order. In order to solve this problem, the image is segmented into the object area and the background area according to the Otsu threshold algorithm based on Markov Random Field firstly. On the basis of the principle of the fractional differential for image enhancement, a piecewise function is established by combining with the different characteristics of pixels in each area, then the best order of fractional differential in the two areas can be determined adaptively. Thus, a novel adaptive fractional order differential algorithm for image enhancement on the basis of segmentation is put forward. Several fog-degraded traffic images are selected for experiments and processed by three other algorithms. The results of comparison exhibit the superiority of our algorithm.

Tumor image-derived texture features are associated with CD3 T-cell infiltration status in glioblastoma

This study analyzed magnetic resonance imaging (MRI) scans of Glioblastoma (GB) patients to develop an imaging-derived predictive model for assessing the extent of intratumoral CD3 T-cell infiltration. Pre-surgical T1-weighted post-contrast and T2-weighted Fluid-Attenuated-Inversion-Recovery (FLAIR) MRI scans, with corresponding mRNA expression of CD3D/E/G were obtained through The Cancer Genome Atlas (TCGA) for 79 GB patients. The tumor region was contoured and 86 image-derived features were extracted across the T1-post contrast and FLAIR images. Six imaging features—kurtosis, contrast, small zone size emphasis, low gray level zone size emphasis, high gray level zone size emphasis, small zone high gray level emphasis—were found associated with CD3 activity and used to build a predictive model for CD3 infiltration in an independent data set of 69 GB patients (using a 50-50 split for training and testing). For the training set, the image-based prediction model for CD3 infiltration achieved accuracy of 97.1% and area under the curve (AUC) of 0.993. For the test set, the model achieved accuracy of 76.5% and AUC of 0.847. This suggests a relationship between image-derived textural features and CD3 T-cell infiltration enabling the non-invasive inference of intratumoral CD3 T-cell infiltration in GB patients, with potential value for the radiological assessment of response to immune therapeutics.

ACTIVE MEAN FIELDS FOR PROBABILISTIC IMAGE SEGMENTATION: CONNECTIONS WITH CHAN-VESE AND RUDIN-OSHER-FATEMI MODELS

Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image-noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating the uncertainty in label assignment is important in multiple application domains, such as segmenting tumors from medical images for radiation treatment planning. One way to estimate these uncertainties is through the computation of posteriors of Bayesian models, which is computationally prohibitive for many practical applications. On the other hand, most computationally efficient methods fail to estimate label uncertainty. We therefore propose in this paper the Active Mean Fields (AMF) approach, a technique based on Bayesian modeling that uses a mean-field approximation to efficiently compute a segmentation and its corresponding uncertainty. Based on a variational formulation, the resulting convex model combines any label-likelihood measure with a prior on the length of the segmentation boundary. A specific implementation of that model is the Chan-Vese segmentation model (CV), in which the binary segmentation task is defined by a Gaussian likelihood and a prior regularizing the length of the segmentation boundary. Furthermore, the Euler-Lagrange equations derived from the AMF model are equivalent to those of the popular Rudin-Osher-Fatemi (ROF) model for image denoising. Solutions to the AMF model can thus be implemented by directly utilizing highly-efficient ROF solvers on log-likelihood ratio fields. We qualitatively assess the approach on synthetic data as well as on real natural and medical images. For a quantitative evaluation, we apply our approach to the icgbench dataset.

A novel approach to segmentation and measurement of medical image using level set methods

The study proposes a novel approach for segmentation and visualization plus value-added surface area and volume measurements for brain medical image analysis. The proposed method contains edge detection and Bayesian based level set segmentation, surface and volume rendering, and surface area and volume measurements for 3D objects of interest (i.e., brain tumor, brain tissue, or whole brain). Two extensions based on edge detection and Bayesian level set are first used to segment 3D objects. Ray casting and a modified marching cubes algorithm are then adopted to facilitate volume and surface visualization of medical-image dataset. To provide physicians with more useful information for diagnosis, the surface area and volume of an examined 3D object are calculated by the techniques of linear algebra and surface integration. Experiment results are finally reported in terms of 3D object extraction, surface and volume rendering, and surface area and volume measurements for medical image analysis.

Subconcussive Head Impact Exposure and White Matter Tract Changes over a Single Season of Youth Football

Purpose To examine the effects of subconcussive impacts resulting from a single season of youth (age range, 8-13 years) football on changes in specific white matter (WM) tracts as detected with diffusion-tensor imaging in the absence of clinically diagnosed concussions. Materials and Methods Head impact data were recorded by using the Head Impact Telemetry system and quantified as the combined-probability risk-weighted cumulative exposure (RWE_CP). Twenty-five male participants were evaluated for seasonal fractional anisotropy (FA) changes in specific WM tracts: the inferior fronto-occipital fasciculus (IFOF), inferior longitudinal fasciculus, and superior longitudinal fasciculus (SLF). Fiber tracts were segmented into a central core and two fiber terminals. The relationship between seasonal FA change in the whole fiber, central core, and the fiber terminals with RWE_CP was also investigated. Linear regression analysis was conducted to determine the association between RWE_CP and change in fiber tract FA during the season. Results There were statistically significant linear relationships between RWE_cp and decreased FA in the whole (R² = 0.433; P = .003), core (R² = 0.3649; P = .007), and terminals (R² = 0.5666; P < .001) of left IFOF. A trend toward statistical significance (P = .08) in right SLF was observed. A statistically significant correlation between decrease in FA of the right SLF terminal and RWE_CP was also observed (R² = 0.2893; P = .028). Conclusion This study found a statistically significant relationship between head impact exposure and change of FA fractional anisotropy value of whole, core, and terminals of left IFOF and right SLF's terminals where WM and gray matter intersect, in the absence of a clinically diagnosed concussion. ^© RSNA, 2016.

Sparse reconstruction of compressive sensing MRI using cross-domain stochastically fully connected conditional random fields

Background

Magnetic Resonance Imaging (MRI) is a crucial medical imaging technology for the screening and diagnosis of frequently occurring cancers. However, image quality may suffer from long acquisition times for MRIs due to patient motion, which also leads to patient discomfort. Reducing MRI acquisition times can reduce patient discomfort leading to reduced motion artifacts from the acquisition process. Compressive sensing strategies applied to MRI have been demonstrated to be effective in decreasing acquisition times significantly by sparsely sampling the k-space during the acquisition process. However, such a strategy requires advanced reconstruction algorithms to produce high quality and reliable images from compressive sensing MRI.

Methods

This paper proposes a new reconstruction approach based on cross-domain stochastically fully connected conditional random fields (CD-SFCRF) for compressive sensing MRI. The CD-SFCRF introduces constraints in both k-space and spatial domains within a stochastically fully connected graphical model to produce improved MRI reconstruction.

Results

Experimental results using T2-weighted (T2w) imaging and diffusion-weighted imaging (DWI) of the prostate show strong performance in preserving fine details and tissue structures in the reconstructed images when compared to other tested methods even at low sampling rates.

Conclusions

The ability to better utilize a limited amount of information to reconstruct T2w and DWI images in a short amount of time while preserving the important details in the images demonstrates the potential of the proposed CD-SFCRF framework as a viable reconstruction algorithm for compressive sensing MRI.

Tumor or abnormality identification from magnetic resonance images using statistical region fusion based segmentation

In this article, a statistical fusion based segmentation technique is proposed to identify different abnormality in magnetic resonance images (MRI). The proposed scheme follows seed selection, region growing-merging and fusion of multiple image segments. In this process initially, an image is divided into a number of blocks and for each block we compute the phase component of the Fourier transform. The phase component of each block reflects the gray level variation among the block but contains a large correlation among them. Hence a singular value decomposition (SVD) technique is adhered to generate a singular value of each block. Then a thresholding procedure is applied on these singular values to identify edgy and smooth regions and some seed points are selected for segmentation. By considering each seed point we perform a binary segmentation of the complete MRI and hence with all seed points we get an equal number of binary images. A parcel based statistical fusion process is used to fuse all the binary images into multiple segments. Effectiveness of the proposed scheme is tested on identifying different abnormalities: prostatic carcinoma detection, tuberculous granulomas identification and intracranial neoplasm or brain tumor detection. The proposed technique is established by comparing its results against seven state-of-the-art techniques with six performance evaluation measures.

Automatic Registration between Real-Time Ultrasonography and Pre-Procedural Magnetic Resonance Images: A Prospective Comparison between Two Registration Methods by Liver Surface and Vessel and by Liver Surface Only

The aim of this study was to compare the accuracy of and the time required for image fusion between real-time ultrasonography (US) and pre-procedural magnetic resonance (MR) images using automatic registration by a liver surface only method and automatic registration by a liver surface and vessel method. This study consisted of 20 patients referred for planning US to assess the feasibility of percutaneous radiofrequency ablation or biopsy for focal hepatic lesions. The first 10 consecutive patients were evaluated by an experienced radiologist using the automatic registration by liver surface and vessel method, whereas the remaining 10 patients were evaluated using the automatic registration by liver surface only method. For all 20 patients, image fusion was automatically executed after following the protocols and fused real-time US and MR images moved synchronously. The accuracy of each method was evaluated by measuring the registration error, and the time required for image fusion was assessed by evaluating the recorded data using in-house software. The results obtained using the two automatic registration methods were compared using the Mann-Whitney U-test. Image fusion was successful in all 20 patients, and the time required for image fusion was significantly shorter with the automatic registration by liver surface only method than with the automatic registration by liver surface and vessel method (median: 43.0 s, range: 29-74 s vs. median: 83.0 s, range: 46-101 s; p = 0.002). The registration error did not significantly differ between the two methods (median: 4.0 mm, range: 2.1-9.9 mm vs. median: 3.7 mm, range: 1.8-5.2 mm; p = 0.496). The automatic registration by liver surface only method offers faster image fusion between real-time US and pre-procedural MR images than does the automatic registration by liver surface and vessel method. However, the degree of accuracy was similar for the two methods.

DEMARCATE: Density-based magnetic resonance image clustering for assessing tumor heterogeneity in cancer

Tumor heterogeneity is a crucial area of cancer research wherein inter- and intra-tumor differences are investigated to assess and monitor disease development and progression, especially in cancer. The proliferation of imaging and linked genomic data has enabled us to evaluate tumor heterogeneity on multiple levels. In this work, we examine magnetic resonance imaging (MRI) in patients with brain cancer to assess image-based tumor heterogeneity. Standard approaches to this problem use scalar summary measures (e.g., intensity-based histogram statistics) that do not adequately capture the complete and finer scale information in the voxel-level data. In this paper, we introduce a novel technique, DEMARCATE (DEnsity-based MAgnetic Resonance image Clustering for Assessing Tumor hEterogeneity) to explore the entire tumor heterogeneity density profiles (THDPs) obtained from the full tumor voxel space. THDPs are smoothed representations of the probability density function of the tumor images. We develop tools for analyzing such objects under the Fisher-Rao Riemannian framework that allows us to construct metrics for THDP comparisons across patients, which can be used in conjunction with standard clustering approaches. Our analyses of The Cancer Genome Atlas (TCGA) based Glioblastoma dataset reveal two significant clusters of patients with marked differences in tumor morphology, genomic characteristics and prognostic clinical outcomes. In addition, we see enrichment of image-based clusters with known molecular subtypes of glioblastoma multiforme, which further validates our representation of tumor heterogeneity and subsequent clustering techniques.

Automatic Detection of White Matter Hyperintensities in Healthy Aging and Pathology Using Magnetic Resonance Imaging: A Review

White matter hyperintensities (WMH) are commonly seen in the brain of healthy elderly subjects and patients with several neurological and vascular disorders. A truly reliable and fully automated method for quantitative assessment of WMH on magnetic resonance imaging (MRI) has not yet been identified. In this paper, we review and compare the large number of automated approaches proposed for segmentation of WMH in the elderly and in patients with vascular risk factors. We conclude that, in order to avoid artifacts and exclude the several sources of bias that may influence the analysis, an optimal method should comprise a careful preprocessing of the images, be based on multimodal, complementary data, take into account spatial information about the lesions and correct for false positives. All these features should not exclude computational leanness and adaptability to available data.

Spatial Habitat Features Derived from Multiparametric Magnetic Resonance Imaging Data Are Associated with Molecular Subtype and 12-Month Survival Status in Glioblastoma Multiforme

One of the most common and aggressive malignant brain tumors is Glioblastoma multiforme. Despite the multimodality treatment such as radiation therapy and chemotherapy (temozolomide: TMZ), the median survival rate of glioblastoma patient is less than 15 months. In this study, we investigated the association between measures of spatial diversity derived from spatial point pattern analysis of multiparametric magnetic resonance imaging (MRI) data with molecular status as well as 12-month survival in glioblastoma. We obtained 27 measures of spatial proximity (diversity) via spatial point pattern analysis of multiparametric T1 post-contrast and T2 fluid-attenuated inversion recovery MRI data. These measures were used to predict 12-month survival status (≤12 or >12 months) in 74 glioblastoma patients. Kaplan-Meier with receiver operating characteristic analyses was used to assess the relationship between derived spatial features and 12-month survival status as well as molecular subtype status in patients with glioblastoma. Kaplan-Meier survival analysis revealed that 14 spatial features were capable of stratifying overall survival in a statistically significant manner. For prediction of 12-month survival status based on these diversity indices, sensitivity and specificity were 0.86 and 0.64, respectively. The area under the receiver operating characteristic curve and the accuracy were 0.76 and 0.75, respectively. For prediction of molecular subtype status, proneural subtype shows highest accuracy of 0.93 among all molecular subtypes based on receiver operating characteristic analysis. We find that measures of spatial diversity from point pattern analysis of intensity habitats from T1 post-contrast and T2 fluid-attenuated inversion recovery images are associated with both tumor subtype status and 12-month survival status and may therefore be useful indicators of patient prognosis, in addition to providing potential guidance for molecularly-targeted therapies in Glioblastoma multiforme.

Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation

We analyzed the spatial diversity of tumor habitats, regions with distinctly different intensity characteristics of a tumor, using various measurements of habitat diversity within tumor regions. These features were then used for investigating the association with a 12-month survival status in glioblastoma (GBM) patients and for the identification of epidermal growth factor receptor (EGFR)-driven tumors. T1 postcontrast and T2 fluid attenuated inversion recovery images from 65 GBM patients were analyzed in this study. A total of 36 spatial diversity features were obtained based on pixel abundances within regions of interest. Performance in both the classification tasks was assessed using receiver operating characteristic (ROC) analysis. For association with 12-month overall survival, area under the ROC curve was 0.74 with confidence intervals [0.630 to 0.858]. The sensitivity and specificity at the optimal operating point ([Formula: see text]) on the ROC were 0.59 and 0.75, respectively. For the identification of EGFR-driven tumors, the area under the ROC curve (AUC) was 0.85 with confidence intervals [0.750 to 0.945]. The sensitivity and specificity at the optimal operating point ([Formula: see text]) on the ROC were 0.76 and 0.83, respectively. Our findings suggest that these spatial habitat diversity features are associated with these clinical characteristics and could be a useful prognostic tool for magnetic resonance imaging studies of patients with GBM.

Fractal dimension analysis of cerebellum in Chiari Malformation type I

Chiari Malformation type I (CM-I) is a serious neurological disorder that is characterized by hindbrain herniation. Our aim was to evaluate the usefulness of fractal analysis in CM-I patients. To examine the morphological complexity features of this disorder, fractal dimension (FD) of cerebellar regions were estimated from magnetic resonance images (MRI) of 17 patients with CM-I and 16 healthy control subjects in this study. The areas of white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF) were calculated and the corresponding FD values were computed using a 2D box-counting method in both groups. The results indicated that CM-I patients had significantly higher (p<0.05) FD values of GM, WM and CSF tissues compared to control group. According to the results of correlation analysis between FD values and the corresponding area values, FD and area values of GM tissues in the patients group were found to be correlated. The results of the present study suggest that FD values of cerebellar regions may be a discriminative feature and a useful marker for investigation of abnormalities in the cerebellum of CM-I patients. Further studies to explore the changes in cerebellar regions with the help of 3D FD analysis and volumetric calculations should be performed as a future work.

Fully automated whole-head segmentation with improved smoothness and continuity, with theory reviewed

Individualized current-flow models are needed for precise targeting of brain structures using transcranial electrical or magnetic stimulation (TES/TMS). The same is true for current-source reconstruction in electroencephalography and magnetoencephalography (EEG/MEG). The first step in generating such models is to obtain an accurate segmentation of individual head anatomy, including not only brain but also cerebrospinal fluid (CSF), skull and soft tissues, with a field of view (FOV) that covers the whole head. Currently available automated segmentation tools only provide results for brain tissues, have a limited FOV, and do not guarantee continuity and smoothness of tissues, which is crucially important for accurate current-flow estimates. Here we present a tool that addresses these needs. It is based on a rigorous Bayesian inference framework that combines image intensity model, anatomical prior (atlas) and morphological constraints using Markov random fields (MRF). The method is evaluated on 20 simulated and 8 real head volumes acquired with magnetic resonance imaging (MRI) at 1 mm3 resolution. We find improved surface smoothness and continuity as compared to the segmentation algorithms currently implemented in Statistical Parametric Mapping (SPM). With this tool, accurate and morphologically correct modeling of the whole-head anatomy for individual subjects may now be feasible on a routine basis. Code and data are fully integrated into SPM software tool and are made publicly available. In addition, a review on the MRI segmentation using atlas and the MRF over the last 20 years is also provided, with the general mathematical framework clearly derived.

A Unified Framework for Brain Segmentation in MR Images

Brain MRI segmentation is an important issue for discovering the brain structure and diagnosis of subtle anatomical changes in different brain diseases. However, due to several artifacts brain tissue segmentation remains a challenging task. The aim of this paper is to improve the automatic segmentation of brain into gray matter, white matter, and cerebrospinal fluid in magnetic resonance images (MRI). We proposed an automatic hybrid image segmentation method that integrates the modified statistical expectation-maximization (EM) method and the spatial information combined with support vector machine (SVM). The combined method has more accurate results than what can be achieved with its individual techniques that is demonstrated through experiments on both real data and simulated images. Experiments are carried out on both synthetic and real MRI. The results of proposed technique are evaluated against manual segmentation results and other methods based on real T1-weighted scans from Internet Brain Segmentation Repository (IBSR) and simulated images from BrainWeb. The Kappa index is calculated to assess the performance of the proposed framework relative to the ground truth and expert segmentations. The results demonstrate that the proposed combined method has satisfactory results on both simulated MRI and real brain datasets.