Bayesian Polytrees With Learned Deep Features for Multi-Class Cell Segmentation

MDF-Net: A Multi-Scale Dynamic Fusion Network for Breast Tumor Segmentation of Ultrasound Images

Breast tumor segmentation of ultrasound images provides valuable information of tumors for early detection and diagnosis. Accurate segmentation is challenging due to low image contrast between areas of interest; speckle noises, and large inter-subject variations in tumor shape and size. This paper proposes a novel Multi-scale Dynamic Fusion Network (MDF-Net) for breast ultrasound tumor segmentation. It employs a two-stage end-to-end architecture with a trunk sub-network for multiscale feature selection and a structurally optimized refinement sub-network for mitigating impairments such as noise and inter-subject variation via better feature exploration and fusion. The trunk network is extended from UNet++ with a simplified skip pathway structure to connect the features between adjacent scales. Moreover, deep supervision at all scales, instead of at the finest scale in UNet++, is proposed to extract more discriminative features and mitigate errors from speckle noise via a hybrid loss function. Unlike previous works, the first stage is linked to a loss function of the second stage so that both the preliminary segmentations and refinement subnetworks can be refined together at training. The refinement sub-network utilizes a structurally optimized MDF mechanism to integrate preliminary segmentation information (capturing general tumor shape and size) at coarse scales and explores inter-subject variation information at finer scales. Experimental results from two public datasets show that the proposed method achieves better Dice and other scores over state-of-the-art methods. Qualitative analysis also indicates that our proposed network is more robust to tumor size/shapes, speckle noise and heavy posterior shadows along tumor boundaries. An optional post-processing step is also proposed to facilitate users in mitigating segmentation artifacts. The efficiency of the proposed network is also illustrated on the "Electron Microscopy neural structures segmentation dataset". It outperforms a state-of-the-art algorithm based on UNet-2022 with simpler settings. This indicates the advantages of our MDF-Nets in other challenging image segmentation tasks with small to medium data sizes.

Superadditivity and Convex Optimization for Globally Optimal Cell Segmentation Using Deformable Shape Models

Cell nuclei segmentation is challenging due to shape variation and closely clustered or partially overlapping objects. Most previous methods are not globally optimal, limited to elliptical models, or are computationally expensive. In this work, we introduce a globally optimal approach based on deformable shape models and global energy minimization for cell nuclei segmentation and cluster splitting. We propose an implicit parameterization of deformable shape models and show that it leads to a convex energy. Convex energy minimization yields the global solution independently of the initialization, is fast, and robust. To jointly perform cell nuclei segmentation and cluster splitting, we developed a novel iterative global energy minimization method, which leverages the inherent property of superadditivity of the convex energy. This property exploits the lower bound of the energy of the union of the models and improves the computational efficiency. Our method provably determines a solution close to global optimality. In addition, we derive a closed-form solution of the proposed global minimization based on the superadditivity property for non-clustered cell nuclei. We evaluated our method using fluorescence microscopy images of five different cell types comprising various challenges, and performed a quantitative comparison with previous methods. Our method achieved state-of-the-art or improved performance.

CPP-Net: Context-Aware Polygon Proposal Network for Nucleus Segmentation

Nucleus segmentation is a challenging task due to the crowded distribution and blurry boundaries of nuclei. Recent approaches represent nuclei by means of polygons to differentiate between touching and overlapping nuclei and have accordingly achieved promising performance. Each polygon is represented by a set of centroid-to-boundary distances, which are in turn predicted by features of the centroid pixel for a single nucleus. However, using the centroid pixel alone does not provide sufficient contextual information for robust prediction and thus degrades the segmentation accuracy. To handle this problem, we propose a Context-aware Polygon Proposal Network (CPP-Net) for nucleus segmentation. First, we sample a point set rather than one single pixel within each cell for distance prediction. This strategy substantially enhances contextual information and thereby improves the robustness of the prediction. Second, we propose a Confidence-based Weighting Module, which adaptively fuses the predictions from the sampled point set. Third, we introduce a novel Shape-Aware Perceptual (SAP) loss that constrains the shape of the predicted polygons. Here, the SAP loss is based on an additional network that is pre-trained by means of mapping the centroid probability map and the pixel-to-boundary distance maps to a different nucleus representation. Extensive experiments justify the effectiveness of each component in the proposed CPP-Net. Finally, CPP-Net is found to achieve state-of-the-art performance on three publicly available databases, namely DSB2018, BBBC06, and PanNuke. Code of this paper is available at https://github.com/csccsccsccsc/cpp-net.

H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation

Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of "optimized" candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover's distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover's distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.

CANCOL, a Computer-Assisted Annotation Tool to Facilitate Colocalization and Tracking of Immune Cells in Intravital Microscopy

Two-photon intravital microscopy (2P-IVM) has become a widely used technique to study cell-to-cell interactions in living organisms. Four-dimensional imaging data obtained via 2P-IVM are classically analyzed by performing automated cell tracking, a procedure that computes the trajectories followed by each cell. However, technical artifacts, such as brightness shifts, the presence of autofluorescent objects, and channel crosstalking, affect the specificity of imaging channels for the cells of interest, thus hampering cell detection. Recently, machine learning has been applied to overcome a variety of obstacles in biomedical imaging. However, existing methods are not tailored for the specific problems of intravital imaging of immune cells. Moreover, results are highly dependent on the quality of the annotations provided by the user. In this study, we developed CANCOL, a tool that facilitates the application of machine learning for automated tracking of immune cells in 2P-IVM. CANCOL guides the user during the annotation of specific objects that are problematic for cell tracking when not properly annotated. Then, it computes a virtual colocalization channel that is specific for the cells of interest. We validated the use of CANCOL on challenging 2P-IVM videos from murine organs, obtaining a significant improvement in the accuracy of automated tracking while reducing the time required for manual track curation.

ECSU-Net: An Embedded Clustering Sliced U-Net Coupled With Fusing Strategy for Efficient Intervertebral Disc Segmentation and Classification

Automatic vertebra segmentation from computed tomography (CT) image is the very first and a decisive stage in vertebra analysis for computer-based spinal diagnosis and therapy support system. However, automatic segmentation of vertebra remains challenging due to several reasons, including anatomic complexity of spine, unclear boundaries of the vertebrae associated with spongy and soft bones. Based on 2D U-Net, we have proposed an Embedded Clustering Sliced U-Net (ECSU-Net). ECSU-Net comprises of three modules named segmentation, intervertebral disc extraction (IDE) and fusion. The segmentation module follows an instance embedding clustering approach, where our three sliced sub-nets use axis of CT images to generate a coarse 2D segmentation along with embedding space with the same size of the input slices. Our IDE module is designed to classify vertebra and find the inter-space between two slices of segmented spine. Our fusion module takes the coarse segmentation (2D) and outputs the refined 3D results of vertebra. A novel adaptive discriminative loss (ADL) function is introduced to train the embedding space for clustering. In the fusion strategy, three modules are integrated via a learnable weight control component, which adaptively sets their contribution. We have evaluated classical and deep learning methods on Spineweb dataset-2. ECSU-Net has provided comparable performance to previous neural network based algorithms achieving the best segmentation dice score of 95.60% and classification accuracy of 96.20%, while taking less time and computation resources.

Magnetic Resonance Imaging Image Feature Analysis Algorithm under Convolutional Neural Network in the Diagnosis and Risk Stratification of Prostate Cancer

This work aimed to explore the accuracy of magnetic resonance imaging (MRI) images based on the convolutional neural network (CNN) algorithm in the diagnosis of prostate cancer patients and tumor risk grading. A total of 89 patients with prostate cancer and benign prostatic hyperplasia diagnosed by MRI examination and pathological examination in hospital were selected as the research objects in this study (they passed the exclusion criteria). The MRI images of these patients were collected in two groups and divided into two groups before and after treatment according to whether the CNN algorithm was used to process them. The number of diagnosed diseases and the number of cases of risk level inferred based on the tumor grading were compared to observe which group was closer to the diagnosis of pathological biopsy. Through comparative analysis, compared with the positive rate of pathological diagnosis (44%), the positive rate after the treatment of the CNN algorithm (42%) was more similar to that before the treatment (34%), and the comparison was statistically marked (P < 0.05). In terms of risk stratification, the grading results after treatment (37 cases) were closer to the results of pathological grading (39 cases) than those before treatment (30 cases), and the comparison was statistically obvious (P < 0.05). In addition, it was obvious that the MRT images would be clearer after treatment through the observation of the MRT images before and after treatment. In conclusion, MRI image segmentation algorithm based on CNN was more accurate in the diagnosis and risk stratification of prostate cancer than routine MRI. According to the evaluation of Dice similarity coefficient (DSC) and Hausdorff I distance (HD), the CNN segmentation method used in this study was more perfect than other segmentation methods.

Marker controlled superpixel nuclei segmentation and automatic counting on immunohistochemistry staining images

MOTIVATION

For the diagnosis of cancer, manually counting nuclei on massive histopathological images is tedious and the counting results might vary due to the subjective nature of the operation.

RESULTS

This paper presents a new segmentation and counting method for nuclei, which can automatically provide nucleus counting results. This method segments nuclei with detected nuclei seed markers through a modified simple one-pass superpixel segmentation method. Rather than using a single pixel as a seed, we created a superseed for each nucleus to involve more information for improved segmentation results. Nucleus pixels are extracted by a newly proposed fusing method to reduce stain variations and preserve nucleus contour information. By evaluating segmentation results, the proposed method was compared to five existing methods on a dataset with 52 immunohistochemically (IHC) stained images. Our proposed method produced the highest mean F1-score of 0.668. By evaluating the counting results, another dataset with more than 30 000 IHC stained nuclei in 88 images were prepared. The correlation between automatically generated nucleus counting results and manual nucleus counting results was up to R2 = 0.901 (P < 0.001). By evaluating segmentation results of proposed method-based tool, we tested on a 2018 Data Science Bowl (DSB) competition dataset, three users obtained DSB score of 0.331 ± 0.006.

AVAILABILITY AND IMPLEMENTATION

The proposed method has been implemented as a plugin tool in ImageJ and the source code can be freely downloaded.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation

Class imbalance has emerged as one of the major challenges for medical image segmentation. The model cascade (MC) strategy, a popular scheme, significantly alleviates the class imbalance issue via running a set of individual deep models for coarse-to-fine segmentation. Despite its outstanding performance, however, this method leads to undesired system complexity and also ignores the correlation among the models. To handle these flaws in the MC approach, we propose in this paper a light-weight deep model, i.e., the One-pass Multi-task Network (OM-Net) to solve class imbalance better than MC does, while requiring only one-pass computation for brain tumor segmentation. First, OM-Net integrates the separate segmentation tasks into one deep model, which consists of shared parameters to learn joint features, as well as task-specific parameters to learn discriminative features. Second, to more effectively optimize OM-Net, we take advantage of the correlation among tasks to design both an online training data transfer strategy and a curriculum learning-based training strategy. Third, we further propose sharing prediction results between tasks, which enables us to design a cross-task guided attention (CGA) module. By following the guidance of the prediction results provided by the previous task, CGA can adaptively recalibrate channel-wise feature responses based on the category-specific statistics. Finally, a simple yet effective post-processing method is introduced to refine the segmentation results of the proposed attention network. Extensive experiments are conducted to demonstrate the effectiveness of the proposed techniques. Most impressively, we achieve state-of-the-art performance on the BraTS 2015 testing set and BraTS 2017 online validation set. Using these proposed approaches, we also won joint third place in the BraTS 2018 challenge among 64 participating teams.The code will be made publicly available at https://github.com/chenhong-zhou/OM-Net.

3D Flow Entropy Contour Fitting Segmentation Algorithm Based on Multi-Scale Transform Contour Constraint

Image segmentation is a crucial topic in image analysis and understanding, and the foundation of target detection and recognition. Image segmentation, essentially, can be considered as classifying the image according to the consistency of the region and the inconsistency between regions, it is widely used in medical and criminal investigation, cultural relic identification, monitoring and so forth. There are two outstanding common problems in the existing segmentation algorithm, one is the lack of accuracy, and the other is that it is not widely applicable. The main contribution of this paper is to present a novel segmentation method based on the information entropy theory and multi-scale transform contour constraint. Firstly, the target contour is initially obtained by means of a multi-scale sample top-hat and bottom-hat transform and an improved watershed method. Subsequently, in terms of this initial contour, the interesting areas can be finely segmented out with an innovative 3D flow entropy method. Finally, the sufficient synthetic and real experiments proved that the proposed algorithm can greatly improve the segmentation effect. In addition, it is widely applicable.