Thin structure segmentation and visualization in three-dimensional biomedical images: a shape-based approach

VSFormer: Mining Correlations in Flexible View Set for Multi-View 3D Shape Understanding

View-based methods have demonstrated promising performance in 3D shape understanding. However, they tend to make strong assumptions about the relations between views or learn the multi-view correlations indirectly, which limits the flexibility of exploring inter-view correlations and the effectiveness of target tasks. To overcome the above problems, this article investigates flexible organization and explicit correlation learning for multiple views. In particular, we propose to incorporate different views of a 3D shape into a permutation-invariant set, referred to as View Set, which removes rigid relation assumptions and facilitates adequate information exchange and fusion among views. Based on that, we devise a nimble Transformer model, named VSFormer, to explicitly capture pairwise and higher-order correlations of all elements in the set. Meanwhile, we theoretically reveal a natural correspondence between the Cartesian product of a view set and the correlation matrix in the attention mechanism, which supports our model design. Comprehensive experiments suggest that VSFormer has better flexibility, efficient inference efficiency and superior performance. Notably, VSFormer reaches state-of-the-art results on various 3 d recognition datasets, including ModelNet40, ScanObjectNN and RGBD. It also establishes new records on the SHREC'17 retrieval benchmark.

Distinguishing metastatic triple-negative breast cancer from nonmetastatic breast cancer using second harmonic generation imaging and resonance Raman spectroscopy

Triple-negative breast cancer (TNBC) is an aggressive subset of breast cancer that is more common in African-American and Hispanic women. Early detection followed by intensive treatment is critical to improving poor survival rates. The current standard to diagnose TNBC from histopathology of biopsy samples is invasive and time-consuming. Imaging methods such as mammography and magnetic resonance (MR) imaging, while covering the entire breast, lack the spatial resolution and specificity to capture the molecular features that identify TNBC. Two nonlinear optical modalities of second harmonic generation (SHG) imaging of collagen, and resonance Raman spectroscopy (RRS) potentially offer novel rapid, label-free detection of molecular and morphological features that characterize cancerous breast tissue at subcellular resolution. In this study, we first applied MR methods to measure the whole-tumor characteristics of metastatic TNBC (4T1) and nonmetastatic estrogen receptor positive breast cancer (67NR) models, including tumor lactate concentration and vascularity. Subsequently, we employed for the first time in vivo SHG imaging of collagen and ex vivo RRS of biomolecules to detect different microenvironmental features of these two tumor models. We achieved high sensitivity and accuracy for discrimination between these two cancer types by quantitative morphometric analysis and nonnegative matrix factorization along with support vector machine. Our study proposes a new method to combine SHG and RRS together as a promising novel photonic and optical method for early detection of TNBC.

Vascular tree segmentation in medical images using Hessian-based multiscale filtering and level set method

Vascular segmentation plays an important role in medical image analysis. A novel technique for the automatic extraction of vascular trees from 2D medical images is presented, which combines Hessian-based multiscale filtering and a modified level set method. In the proposed algorithm, the morphological top-hat transformation is firstly adopted to attenuate background. Then Hessian-based multiscale filtering is used to enhance vascular structures by combining Hessian matrix with Gaussian convolution to tune the filtering response to the specific scales. Because Gaussian convolution tends to blur vessel boundaries, which makes scale selection inaccurate, an improved level set method is finally proposed to extract vascular structures by introducing an external constrained term related to the standard deviation of Gaussian function into the traditional level set. Our approach was tested on synthetic images with vascular-like structures and 2D slices extracted from real 3D abdomen magnetic resonance angiography (MRA) images along the coronal plane. The segmentation rates for synthetic images are above 95%. The results for MRA images demonstrate that the proposed method can extract most of the vascular structures successfully and accurately in visualization. Therefore, the proposed method is effective for the vascular tree extraction in medical images.

Mitral valve closure prediction with 3-D personalized anatomical models and anisotropic hyperelastic tissue assumptions

This study is concerned with the development of patient-specific simulations of the mitral valve that use personalized anatomical models derived from 3-D transesophageal echocardiography (3-D TEE). The proposed method predicts the closed configuration of the mitral valve by solving for an equilibrium solution that balances various forces including blood pressure, tissue collision, valve tethering, and tissue elasticity. The model also incorporates realistic hyperelastic and anisotropic properties for the valve leaflets. This study compares hyperelastic tissue laws with a quasi-elastic law under various physiological parameters, and provides insights into error sensitivity to chordal placement, allowing for a preliminary comparison of the influence of the two factors (chords and models) on error. Predictive errors show the promise of the method, yielding aggregate median errors of the order of 1 mm, and computed strains and stresses show good correspondence with those reported in prior studies.

Patient-specific mitral valve closure prediction using 3D echocardiography

This article presents an approach to modeling the closure of the mitral valve using patient-specific anatomical information derived from 3D transesophageal echocardiography (TEE). Our approach uses physics-based modeling to solve for the stationary configuration of the closed valve structure from the patient-specific open valve structure, which is recovered using a user-in-the-loop, thin-tissue detector segmentation. The method uses a tensile shape-finding approach based on energy minimization. This method is employed to predict the aptitude of the mitral valve leaflets to coapt. We tested the method using 10 intraoperative 3D TEE sequences by comparing the closed valve configuration predicted from the segmented open valve with the segmented closed valve, taken as ground truth. Experiments show promising results, with prediction errors on par with 3D TEE resolution and with good potential for applications in pre-operative planning.

Two-stage multishape segmentation of brain structures using image intensity, tissue type, and location information

PURPOSE

The authors propose a fast, robust, nonparametric, entropy-based, coupled, multishape approach to segment subcortical brain structures from magnetic resonance images (MRIs).

METHODS

The proposed method uses three types of information: Image intensity, tissue types, and locations of structures. The image intensity information is captured by estimating the probability density function (pdf) of the image intensities in each structure. The tissue type information is captured by applying an unsupervised tissue segmentation method to the image and estimating a probability mass function (pmf) for the tissue type of each structure. The location information is captured by estimating pdf of the location of each structure from the training datasets. The resulting pmf's and pdf's are used to define an entropy function whose minimum corresponds to a desirable segmentation of the structures. The authors propose a three-step optimization strategy for the segmentation method. In the first step, a powerful automatic initialization method is developed based on tissue type and location information of the structures. In the second step, a quasi-Newton method is used to optimize the parameters of the energy function. To speed up the iterations, derivatives of the energy function with respect to its parameters are analytically derived and used in the optimization process. In the last step, the limitations related to the prior shape model are removed and a level-set method is applied for the fine tuning of the segmentation results.

RESULTS

The proposed method is applied to two different datasets and the results are compared to those of previous methods in literature. Experimental results are presented for lateral ventricles, caudate, thalamus, putamen, pallidum, hippocampus, and amygdala.

CONCLUSIONS

The results illustrate superior performance of the proposed segmentation method compared to other methods in literature. The execution time of the algorithm is a few minutes, suitable for a variety of applications.

Robust adaptive 3-D segmentation of vessel laminae from fluorescence confocal microscope images and parallel GPU implementation

This paper presents robust 3-D algorithms to segment vasculature that is imaged by labeling laminae, rather than the lumenal volume. The signal is weak, sparse, noisy, nonuniform, low-contrast, and exhibits gaps and spectral artifacts, so adaptive thresholding and Hessian filtering based methods are not effective. The structure deviates from a tubular geometry, so tracing algorithms are not effective. We propose a four step approach. The first step detects candidate voxels using a robust hypothesis test based on a model that assumes Poisson noise and locally planar geometry. The second step performs an adaptive region growth to extract weakly labeled and fine vessels while rejecting spectral artifacts. To enable interactive visualization and estimation of features such as statistical confidence, local curvature, local thickness, and local normal, we perform the third step. In the third step, we construct an accurate mesh representation using marching tetrahedra, volume-preserving smoothing, and adaptive decimation algorithms. To enable topological analysis and efficient validation, we describe a method to estimate vessel centerlines using a ray casting and vote accumulation algorithm which forms the final step of our algorithm. Our algorithm lends itself to parallel processing, and yielded an 8 x speedup on a graphics processor (GPU). On synthetic data, our meshes had average error per face (EPF) values of (0.1-1.6) voxels per mesh face for peak signal-to-noise ratios from (110-28 dB). Separately, the error from decimating the mesh to less than 1% of its original size, the EPF was less than 1 voxel/face. When validated on real datasets, the average recall and precision values were found to be 94.66% and 94.84%, respectively.

Three-dimensional coupled-object segmentation using symmetry and tissue type information

This paper presents an automatic method for segmentation of brain structures using their symmetry and tissue type information. The proposed method generates segmented structures that have homogenous tissues. It benefits from general symmetry of the brain structures in the two hemispheres. It also benefits from the tissue regions generated by fuzzy c-means clustering. All in all, the proposed method can be described as a dynamic knowledge-based method that eliminates the need for statistical shape models of the structures while generating accurate segmentation results. The proposed approach is implemented in MATLAB and tested on the Internet Brain Segmentation Repository (IBSR) datasets. To this end, it is applied to the segmentation of caudate and ventricles three-dimensionally in magnetic resonance images (MRI) of the brain. Impacts of each of the steps of the proposed approach are demonstrated through experiments. It is shown that the proposed method generates accurate segmentation results that are insensitive to initialization and parameter selection. The proposed method is compared to four previous methods illustrating advantages and limitations of each method.

Epifluorescence-based quantitative microvasculature remodeling using geodesic level-sets and shape-based evolution

Accurate vessel segmentation is the first step in analysis of microvascular networks for reliable feature extraction and quantitative characterization. Segmentation of epifluorescent imagery of microvasculature presents a unique set of challenges and opportunities compared to traditional angiogram-based vessel imagery. This paper presents a novel system that combines methods from mathematical morphology, differential geometry, and active contours to reliably detect and segment microvasculature under varying background fluorescence conditions. The system consists of three main modules: vessel enhancement, shape-based initialization, and level-set based segmentation. Vessel enhancement deals with image noise and uneven background fluorescence using anisotropic diffusion and mathematical morphology techniques. Shape-based initialization uses features from the second-order derivatives of the enhanced vessel image and produces a coarse ridge (vessel) mask. Geodesic level-set based active contours refine the coarse ridge map and fix possible discontinuities or leakage of the level set contours that may arise from complex topology or high background fluorescence. The proposed system is tested on epifluorescence-based high resolution images of porcine dura mater microvasculature. Preliminary experiments show promising results.

Tensor classification of N-point correlation function features for histology tissue segmentation

In this paper, we utilize the N-point correlation functions (N-pcfs) to construct an appropriate feature space for achieving tissue segmentation in histology-stained microscopic images. The N-pcfs estimate microstructural constituent packing densities and their spatial distribution in a tissue sample. We represent the multi-phase properties estimated by the N-pcfs in a tensor structure. Using a variant of higher-order singular value decomposition (HOSVD) algorithm, we realize a robust classifier that provides a multi-linear description of the tensor feature space. Validated results of the segmentation are presented in a case-study that focuses on understanding the genetic phenotyping differences in mouse placentae.

Semi-Automatic Integrated Segmentation Approaches and Contour Extraction Applied to Computed Tomography Scan Images

We propose to segment two-dimensional CT scans traumatic brain injuries with various methods. These methods are hybrid, feature extraction, level sets, region growing, and watershed which are analysed based upon their parametric and nonparametric arguments. The pixel intensities, gradient magnitude, affinity map, and catchment basins of these methods are validated based upon various constraints evaluations. In this article, we also develop a new methodology for a computational pipeline that uses bilateral filtering, diffusion properties, watershed, and filtering with mathematical morphology operators for the contour extraction of the lesion in the feature available based mainly on the gradient function. The evaluations of the classification of these lesions are very briefly outlined in this context and are being undertaken by pattern recognition in another paper work.

Context-preserving rendering of medical segmentation data

We present a context-preserving visualisation method for segmented volumetric medical images. A segmented volumetric image contains a number of anatomical objects which are important features to be visualised. Our context-preserving rendering utilises the curvature at the surfaces of the segmentation objects to modulate the opacity contribution during rendering. This results in the areas of high curvature, typically the most important features, being opaque and visible while everything else being transparent.

Virtual Contrast for Coronary Vessels Based on Level Set Generated Subvoxel Accurate Centerlines

We present a tool for tracking coronary vessels in MRI scans of the human heart to aid in the screening of heart diseases. The vessels are identified through a single click inside each vessel present in a standard orthogonal view. The vessel identification results from a series of computational steps including eigenvalue analysis of the Hessian of the MRI image followed by a level set-based extraction of the vessel centerline. All identified vessels are highlighted using a virtual contrast agent and displayed simultaneously in a spherical curved reformation view. In cases of over segmentation, the vessel traces can be shortened by a click on each vessel end point. Intermediate analysis results of the vessel computation steps can be displayed as well. We successfully validated the tool on 40 MRI scans demonstrating accuracy and significant time savings over manual vessel tracing.