CM-SegNet: A deep learning-based automatic segmentation approach for medical images by combining convolution and multilayer perceptron

VENet: Variational energy network for gland segmentation of pathological images and early gastric cancer diagnosis of whole slide images

BACKGROUND AND OBJECTIVE

Gland segmentation of pathological images is an essential but challenging step for adenocarcinoma diagnosis. Although deep learning methods have recently made tremendous progress in gland segmentation, they have not given satisfactory boundary and region segmentation results of adjacent glands. These glands usually have a large difference in glandular appearance, and the statistical distribution between the training and test sets in deep learning is inconsistent. These problems make networks not generalize well in the test dataset, bringing difficulties to gland segmentation and early cancer diagnosis.

METHODS

To address these problems, we propose a Variational Energy Network named VENet with a traditional variational energy Lv loss for gland segmentation of pathological images and early gastric cancer detection in whole slide images (WSIs). It effectively integrates the variational mathematical model and the data-adaptability of deep learning methods to balance boundary and region segmentation. Furthermore, it can effectively segment and classify glands in large-size WSIs with reliable nucleus width and nucleus-to-cytoplasm ratio features.

RESULTS

The VENet was evaluated on the 2015 MICCAI Gland Segmentation challenge (GlaS) dataset, the Colorectal Adenocarcinoma Glands (CRAG) dataset, and the self-collected Nanfang Hospital dataset. Compared with state-of-the-art methods, our method achieved excellent performance for GlaS Test A (object dice 0.9562, object F1 0.9271, object Hausdorff distance 73.13), GlaS Test B (object dice 94.95, object F1 95.60, object Hausdorff distance 59.63), and CRAG (object dice 95.08, object F1 92.94, object Hausdorff distance 28.01). For the Nanfang Hospital dataset, our method achieved a kappa of 0.78, an accuracy of 0.9, a sensitivity of 0.98, and a specificity of 0.80 on the classification task of test 69 WSIs.

CONCLUSIONS

The experimental results show that the proposed model accurately predicts boundaries and outperforms state-of-the-art methods. It can be applied to the early diagnosis of gastric cancer by detecting regions of high-grade gastric intraepithelial neoplasia in WSI, which can assist pathologists in analyzing large WSI and making accurate diagnostic decisions.

Weakly-Supervised Segmentation-Based Quantitative Characterization of Pulmonary Cavity Lesions in CT Scans

OBJECTIVE

Pulmonary cavity lesion is one of the commonly seen lesions in lung caused by a variety of malignant and non-malignant diseases. Diagnosis of a cavity lesion is commonly based on accurate recognition of the typical morphological characteristics. A deep learning-based model to automatically detect, segment, and quantify the region of cavity lesion on CT scans has potential in clinical diagnosis, monitoring, and treatment efficacy assessment.

METHODS

A weakly-supervised deep learning-based method named CSA2-ResNet was proposed to quantitatively characterize cavity lesions in this paper. The lung parenchyma was firstly segmented using a pretrained 2D segmentation model, and then the output with or without cavity lesions was fed into the developed deep neural network containing hybrid attention modules. Next, the visualized lesion was generated from the activation region of the classification network using gradient-weighted class activation mapping, and image processing was applied for post-processing to obtain the expected segmentation results of cavity lesions. Finally, the automatic characteristic measurement of cavity lesions (e.g., area and thickness) was developed and verified.

RESULTS

the proposed weakly-supervised segmentation method achieved an accuracy, precision, specificity, recall, and F1-score of 98.48%, 96.80%, 97.20%, 100%, and 98.36%, respectively. There is a significant improvement (P < 0.05) compared to other methods. Quantitative characterization of morphology also obtained good analysis effects.

CONCLUSIONS

The proposed easily-trained and high-performance deep learning model provides a fast and effective way for the diagnosis and dynamic monitoring of pulmonary cavity lesions in clinic. Clinical and Translational Impact Statement: This model used artificial intelligence to achieve the detection and quantitative analysis of pulmonary cavity lesions in CT scans. The morphological features revealed in experiments can be utilized as potential indicators for diagnosis and dynamic monitoring of patients with cavity lesions.

Combining quantitative and qualitative analysis for scoring pleural line in lung ultrasound

Objective.Accurate assessment of pleural line is crucial for the application of lung ultrasound (LUS) in monitoring lung diseases, thereby aim of this study is to develop a quantitative and qualitative analysis method for pleural line.Approach.The novel cascaded deep learning model based on convolution and multilayer perceptron was proposed to locate and segment the pleural line in LUS images, whose results were applied for quantitative analysis of textural and morphological features, respectively. By using gray-level co-occurrence matrix and self-designed statistical methods, eight textural and three morphological features were generated to characterize the pleural lines. Furthermore, the machine learning-based classifiers were employed to qualitatively evaluate the lesion degree of pleural line in LUS images.Main results.We prospectively evaluated 3770 LUS images acquired from 31 pneumonia patients. Experimental results demonstrated that the proposed pleural line extraction and evaluation methods all have good performance, with dice and accuracy of 0.87 and 94.47%, respectively, and the comparison with previous methods found statistical significance (P< 0.001 for all). Meanwhile, the generalization verification proved the feasibility of the proposed method in multiple data scenarios.Significance.The proposed method has great application potential for assessment of pleural line in LUS images and aiding lung disease diagnosis and treatment.

CSAP-UNet: Convolution and self-attention paralleling network for medical image segmentation with edge enhancement

Convolution operation is performed within a local window of the input image. Therefore, convolutional neural network (CNN) is skilled in obtaining local information. Meanwhile, the self-attention (SA) mechanism extracts features by calculating the correlation between tokens from all positions in the image, which has advantage in obtaining global information. Therefore, the two modules can complement each other to improve feature extraction ability. An effective fusion method is a problem worthy of further study. In this paper, we propose a CNN and SA paralleling network CSAP-UNet with U-Net as backbone. The encoder consists of two parallel branches of CNN and Transformer to extract the feature from the input image, which takes into account both the global dependencies and the local information. Because medical images come from certain frequency bands within the spectrum, their color channels are not as uniform as natural images. Meanwhile, medical segmentation pays more attention to lesion regions in the image. Attention fusion module (AFM) integrates channel attention and spatial attention in series to fuse the output features of the two branches. The medical image segmentation task is essentially to locate the boundary of the object in the image. The boundary enhancement module (BEM) is designed in the shallow layer of the proposed network to focus more specifically on pixel-level edge details. Experimental results on three public datasets validate that CSAP-UNet outperforms state-of-the-art networks, particularly on the ISIC 2017 dataset. The cross-dataset evaluation on Kvasir and CVC-ClinicDB shows that CSAP-UNet has strong generalization ability. Ablation experiments also indicate the effectiveness of the designed modules. The code for training and test is available at https://github.com/zhouzhou1201/CSAP-UNet.git.

MSMCNet: Differential context drives accurate localization and edge smoothing of lesions for medical image segmentation

Medical image segmentation plays a crucial role in clinical assistance for diagnosis. The UNet-based network architecture has achieved tremendous success in the field of medical image segmentation. However, most methods commonly employ element-wise addition or channel merging to fuse features, resulting in smaller differentiation of feature information and excessive redundancy. Consequently, this leads to issues such as inaccurate lesion localization and blurred boundaries in segmentation. To alleviate these problems, the Multi-scale Subtraction and Multi-key Context Conversion Networks (MSMCNet) are proposed for medical image segmentation. Through the construction of differentiated contextual representations, MSMCNet emphasizes vital information and achieves precise medical image segmentation by accurately localizing lesions and enhancing boundary perception. Specifically, the construction of differentiated contextual representations is accomplished through the proposed Multi-scale Non-crossover Subtraction (MSNS) module and Multi-key Context Conversion Module (MCCM). The MSNS module utilizes the context of MCCM coding and redistribute the value of feature map pixels. Extensive experiments were conducted on widely used public datasets, including the ISIC-2018 dataset, COVID-19-CT-Seg dataset, Kvasir dataset, as well as a privately constructed traumatic brain injury dataset. The experimental results demonstrated that our proposed MSMCNet outperforms state-of-the-art medical image segmentation methods across different evaluation metrics.

TGDAUNet: Transformer and GCNN based dual-branch attention UNet for medical image segmentation

Accurate and automatic segmentation of medical images is a key step in clinical diagnosis and analysis. Currently, the successful application of Transformers' model in the field of computer vision, researchers have begun to gradually explore the application of Transformers in medical segmentation of images, especially in combination with convolutional neural networks with coding-decoding structure, which have achieved remarkable results in the field of medical segmentation. However, most studies have combined Transformers with CNNs at a single scale or processed only the highest-level semantic feature information, ignoring the rich location information in the lower-level semantic feature information. At the same time, for problems such as blurred structural boundaries and heterogeneous textures in images, most existing methods usually simply connect contour information to capture the boundaries of the target. However, these methods cannot capture the precise outline of the target and ignore the potential relationship between the boundary and the region. In this paper, we propose the TGDAUNet, which consists of a dual-branch backbone network of CNNs and Transformers and a parallel attention mechanism, to achieve accurate segmentation of lesions in medical images. Firstly, high-level semantic feature information of the CNN backbone branches is fused at multiple scales, and the high-level and low-level feature information complement each other's location and spatial information. We further use the polarised self-attentive (PSA) module to reduce the impact of redundant information caused by multiple scales, to better couple with the feature information extracted from the Transformers backbone branch, and to establish global contextual long-range dependencies at multiple scales. In addition, we have designed the Reverse Graph-reasoned Fusion (RGF) module and the Feature Aggregation (FA) module to jointly guide the global context. The FA module aggregates high-level semantic feature information to generate an original global predictive segmentation map. The RGF module captures non-significant features of the boundaries in the original or secondary global prediction segmentation graph through a reverse attention mechanism, establishing a graph reasoning module to explore the potential semantic relationships between boundaries and regions, further refining the target boundaries. Finally, to validate the effectiveness of our proposed method, we compare our proposed method with the current popular methods in the CVC-ClinicDB, Kvasir-SEG, ETIS, CVC-ColonDB, CVC-300,datasets as well as the skin cancer segmentation datasets ISIC-2016 and ISIC-2017. The large number of experimental results show that our method outperforms the currently popular methods. Source code is released at https://github.com/sd-spf/TGDAUNet.

A new classification method for diagnosing COVID-19 pneumonia based on joint CNN features of chest X-ray images and parallel pyramid MLP-mixer module

During the past three years, the coronavirus disease 2019 (COVID-19) has swept the world. The rapid and accurate recognition of covid-19 pneumonia are ,therefore, of great importance. To handle this problem, we propose a new pipeline of deep learning framework for diagnosing COVID-19 pneumonia via chest X-ray images from normal, COVID-19, and other pneumonia patients. In detail, the self-trained YOLO-v4 network was first used to locate and segment the thoracic region, and the output images were scaled to the same size. Subsequently, the pre-trained convolutional neural network was adopted to extract the features of X-ray images from 13 convolutional layers, which were fused with the original image to form a 14-dimensional image matrix. It was then put into three parallel pyramid multi-layer perceptron (MLP)-Mixer modules for comprehensive feature extraction through spatial fusion and channel fusion based on different scales so as to grasp more extensive feature correlation. Finally, by combining all image features from the 14-channel output, the classification task was achieved using two fully connected layers as well as Softmax classifier for classification. Extensive simulations based on a total of 4099 chest X-ray images were conducted to verify the effectiveness of the proposed method. Experimental results indicated that our proposed method can achieve the best performance in almost all cases, which is good for auxiliary diagnosis of COVID-19 and has great clinical application potential.

TransCUNet: UNet cross fused transformer for medical image segmentation

Accurate segmentation of medical images is crucial for clinical diagnosis and evaluation. However, medical images have complex shapes, the structures of different objects are very different, and most medical datasets are small in scale, making it difficult to train effectively. These problems increase the difficulty of automatic segmentation. To further improve the segmentation performance of the model, we propose a multi-branch network model, called TransCUNet, for segmenting medical images of different modalities. The model contains three structures: cross residual fusion block (CRFB), pyramidal pooling module (PPM) and gated axial-attention, which achieve effective extraction of high-level and low-level features of images, while showing high robustness to different size segmentation objects and different scale datasets. In our experiments, we use four datasets to train, validate and test the models. The experimental results show that TransCUNet has better segmentation performance compared to the current mainstream segmentation methods, and the model has a smaller size and number of parameters, which has great potential for clinical applications.