Two-stage multitask U-Net construction for pulmonary nodule segmentation and malignancy risk prediction

Predicting branch retinal vein occlusion development using multimodal deep learning and pre-onset fundus hemisection images

Branch retinal vein occlusion (BRVO) is a leading cause of visual impairment in working-age individuals, though predicting its occurrence from retinal vascular features alone remains challenging. We developed a deep learning model to predict BRVO based on pre-onset, metadata-matched fundus hemisection images. This retrospective cohort study included patients diagnosed with unilateral BRVO from two Korean tertiary centers (2005-2023), using hemisection fundus images from 27 BRVO-affected eyes paired with 81 unaffected hemisections (27 counter and 54 contralateral) for training. A U-net model segmented retinal optic discs and blood vessels (BVs), dividing them into upper and lower halves labeled for BRVO occurrence. Both unimodal models (using either fundus or BV images) and a BV-enhanced multimodal model were constructed to predict future BRVO. The multimodal model outperformed the unimodal models achieving an area under the receiver operating characteristic curve of 0.76 (95% confidence interval [CI], 0.66-0.83) and accuracy of 68.5% (95% CI 58.9-77.1%), with predictions focusing on arteriovenous crossing regions in the retinal vascular arcade. These findings demonstrate the potential of the BV-enhanced multimodal approach for BRVO prediction and highlight the need for larger, multicenter datasets to improve its clinical utility and predictive accuracy.

BreasTDLUSeg: A coarse-to-fine framework for segmentation of breast terminal duct lobular units on histopathological whole-slide images

Automatic segmentation of breast terminal duct lobular units (TDLUs) on histopathological whole-slide images (WSIs) is crucial for the quantitative evaluation of TDLUs in the diagnostic and prognostic analysis of breast cancer. However, TDLU segmentation remains a great challenge due to its highly heterogeneous sizes, structures, and morphologies as well as the small areas on WSIs. In this study, we propose BreasTDLUSeg, an efficient coarse-to-fine two-stage framework based on multi-scale attention to achieve localization and precise segmentation of TDLUs on hematoxylin and eosin (H&E)-stained WSIs. BreasTDLUSeg consists of two networks: a superpatch-based patch-level classification network (SPPC-Net) and a patch-based pixel-level segmentation network (PPS-Net). SPPC-Net takes a superpatch as input and adopts a sub-region classification head to classify each patch within the superpatch as TDLU positive or negative. PPS-Net takes the TDLU positive patches derived from SPPC-Net as input. PPS-Net deploys a multi-scale CNN-Transformer as an encoder to learn enhanced multi-scale morphological representations and an upsampler to generate pixel-wise segmentation masks for the TDLU positive patches. We also constructed two breast cancer TDLU datasets containing a total of 530 superpatch images with patch-level annotations and 2322 patch images with pixel-level annotations to enable the development of TDLU segmentation methods. Experiments on the two datasets demonstrate that BreasTDLUSeg outperforms other state-of-the-art methods with the highest Dice similarity coefficients of 79.97% and 92.93%, respectively. The proposed method shows great potential to assist pathologists in the pathological analysis of breast cancer. An open-source implementation of our approach can be found at https://github.com/Dian-kai/BreasTDLUSeg.

A Multi-Task Model for Pulmonary Nodule Segmentation and Classification

In the computer-aided diagnosis of lung cancer, the automatic segmentation of pulmonary nodules and the classification of benign and malignant tumors are two fundamental tasks. However, deep learning models often overlook the potential benefits of task correlations in improving their respective performances, as they are typically designed for a single task only. Therefore, we propose a multi-task network (MT-Net) that integrates shared backbone architecture and a prediction distillation structure for the simultaneous segmentation and classification of pulmonary nodules. The model comprises a coarse segmentation subnetwork (Coarse Seg-net), a cooperative classification subnetwork (Class-net), and a cooperative segmentation subnetwork (Fine Seg-net). Coarse Seg-net and Fine Seg-net share identical structure, where Coarse Seg-net provides prior location information for the subsequent Fine Seg-net and Class-net, thereby boosting pulmonary nodule segmentation and classification performance. We quantitatively and qualitatively analyzed the performance of the model by using the public dataset LIDC-IDRI. Our results show that the model achieves a Dice similarity coefficient (DI) index of 83.2% for pulmonary nodule segmentation, as well as an accuracy (ACC) of 91.9% for benign and malignant pulmonary nodule classification, which is competitive with other state-of-the-art methods. The experimental results demonstrate that the performance of pulmonary nodule segmentation and classification can be improved by a unified model that leverages the potential correlation between tasks.

Dual-branch feature fusion S3D V-Net network for lung nodules segmentation

BACKGROUND

Accurate segmentation of lung nodules can help doctors get more accurate results and protocols in early lung cancer diagnosis and treatment planning, so that patients can be better detected and treated at an early stage, and the mortality rate of lung cancer can be reduced.

PURPOSE

Currently, the improvement of lung nodule segmentation accuracy has been limited by his heterogeneous performance in the lungs, the imbalance between segmentation targets and background pixels, and other factors. We propose a new 2.5D lung nodule segmentation network model for lung nodule segmentation. This network model can well improve the extraction of edge information of lung nodules, and fuses intra-slice and inter-slice features, which makes good use of the three-dimensional structural information of lung nodules and can more effectively improve the accuracy of lung nodule segmentation.

METHODS

Our approach is based on a typical encoding-decoding network structure for improvement. The improved model captures the features of multiple nodules in both 3-D and 2-D CT images, complements the information of the segmentation target's features and enhances the texture features at the edges of the pulmonary nodules through the dual-branch feature fusion module (DFFM) and the reverse attention context module (RACM), and employs central pooling instead of the maximal pooling operation, which is used to preserve the features around the target and to eliminate the edge-irrelevant features, to further improve the performance of the segmentation of the pulmonary nodules.

RESULTS

We evaluated this method on a wide range of 1186 nodules from the LUNA16 dataset, and averaging the results of ten cross-validated, the proposed method achieved the mean dice similarity coefficient (mDSC) of 84.57%, the mean overlapping error (mOE) of 18.73% and average processing of a case is about 2.07 s. Moreover, our results were compared with inter-radiologist agreement on the LUNA16 dataset, and the average difference was 0.74%.

CONCLUSION

The experimental results show that our method improves the accuracy of pulmonary nodules segmentation and also takes less time than more 3-D segmentation methods in terms of time.

Design of sports achievement prediction system based on U-net convolutional neural network in the context of machine learning

Sports plays a pivotal role in national development. To accurately predict college students' sports performance and motivate them to improve their physical fitness, this study constructs a sports achievement prediction system by using a U-Net Convolutional Neural Network (CNN) in machine learning. Firstly, the current state of physical education teachers' instructional proficiency is investigated and analyzed to identify existing problems. Secondly, an improved U-Net-based sports achievement prediction system is proposed. This method enhances the utilization and propagation of network features by incorporating dense connections, thus addressing gradient disappearance issues. Simultaneously, an improved mixed loss function is introduced to alleviate class imbalance. Finally, the effectiveness of the proposed system is validated through testing, demonstrating that the improved U-Net CNN algorithm yields superior results. Specifically, the prediction accuracy of the improved network for sports performance surpasses that of the original U-Net by 4.22 % and exceeds that of DUNet by 5.22 %. Compared with other existing prediction networks, the improved U-Net CNN model exhibits a superior achievement prediction ability. Consequently, the proposed system enhances teaching and learning efficiency and offers insights into applying artificial intelligence technology to smart classroom development.

An improved V-Net lung nodule segmentation model based on pixel threshold separation and attention mechanism

Accurate labeling of lung nodules in computed tomography (CT) images is crucial in early lung cancer diagnosis and before nodule resection surgery. However, the irregular shape of lung nodules in CT images and the complex lung environment make it much more challenging to segment lung nodules accurately. On this basis, we propose an improved V-Net segmentation method based on pixel threshold separation and attention mechanism for lung nodules. This method first offers a data augment strategy to solve the problem of insufficient samples in 3D medical datasets. In addition, we integrate the feature extraction module based on pixel threshold separation into the model to enhance the feature extraction ability under different thresholds on the one hand. On the other hand, the model introduces channel and spatial attention modules to make the model pay more attention to important semantic information and improve its generalization ability and accuracy. Experiments show that the Dice similarity coefficients of the improved model on the public datasets LUNA16 and LNDb are 94.9% and 81.1% respectively, and the sensitivities reach 92.7% and 76.9% respectively. which is superior to most existing UNet architecture models and comparable to the manual level segmentation results by medical technologists.

The differential computed tomography features between small benign and malignant solid solitary pulmonary nodules with different sizes

Background

Computed tomography (CT) has been widely known to be the first choice for the diagnosis of solid solitary pulmonary nodules (SSPNs). However, the smaller the SSPN is, the less the differential CT signs between benign and malignant SSPNs there are, which brings great challenges to their diagnosis. Therefore, this study aimed to investigate the differential CT features between small (≤15 mm) benign and malignant SSPNs with different sizes.

Methods

From May 2018 to November 2021, CT data of 794 patients with small SSPNs (≤15 mm) were retrospectively analyzed. SSPNs were divided into benign and malignant groups, and each group was further classified into three cohorts: cohort I (diameter ≤6 mm), cohort II (6 mm < diameter ≤8 mm), and cohort III (8 mm < diameter ≤15 mm). The differential CT features of benign and malignant SSPNs in three cohorts were identified. Multivariable logistic regression analyses were conducted to identify independent factors of benign SSPNs.

Results

In cohort I, polygonal shape and upper-lobe distribution differed significantly between groups (all P<0.05) and multiparametric analysis showed polygonal shape [adjusted odds ratio (OR): 12.165; 95% confidence interval (CI): 1.512-97.872; P=0.019] was the most effective variation for predicting benign SSPNs, with an area under the receiver operating characteristic curve (AUC) of 0.747 (95% CI: 0.640-0.855; P=0.001). In cohort II, polygonal shape, lobulation, pleural retraction, and air bronchogram differed significantly between groups (all P<0.05), and polygonal shape (OR: 8.870; 95% CI: 1.096-71.772; P=0.041) and the absence of pleural retraction (OR: 0.306; 95% CI: 0.106-0.883; P=0.028) were independent predictors of benign SSPNs, with an AUC of 0.778 (95% CI: 0.694-0.863; P<0.001). In cohort III, 12 CT features showed significant differences between groups (all P<0.05) and polygonal shape (OR: 3.953; 95% CI: 1.508-10.361; P=0.005); calcification (OR: 3.710; 95% CI: 1.305-10.551; P=0.014); halo sign (OR: 6.237; 95% CI: 2.838-13.710; P<0.001); satellite lesions (OR: 6.554; 95% CI: 3.225-13.318; P<0.001); and the absence of lobulation (OR: 0.066; 95% CI: 0.026-0.167; P<0.001), air space (OR: 0.405; 95% CI: 0.215-0.764; P=0.005), pleural retraction (OR: 0.297; 95% CI: 0.179-0.493; P<0.001), bronchial truncation (OR: 0.165; 95% CI: 0.090-0.303; P<0.001), and air bronchogram (OR: 0.363; 95% CI: 0.208-0.633; P<0.001) were independent predictors of benign SSPNs, with an AUC of 0.869 (95% CI: 0.840-0.897; P<0.001).

Conclusions

CT features vary between SSPNs with different sizes. Clarifying the differential CT features based on different diameter ranges may help to minimize ambiguities and discriminate the benign SSPNs from malignant ones.

A combined encoder-transformer-decoder network for volumetric segmentation of adrenal tumors

BACKGROUND

The morphology of the adrenal tumor and the clinical statistics of the adrenal tumor area are two crucial diagnostic and differential diagnostic features, indicating precise tumor segmentation is essential. Therefore, we build a CT image segmentation method based on an encoder-decoder structure combined with a Transformer for volumetric segmentation of adrenal tumors.

METHODS

This study included a total of 182 patients with adrenal metastases, and an adrenal tumor volumetric segmentation method combining encoder-decoder structure and Transformer was constructed. The Dice Score coefficient (DSC), Hausdorff distance, Intersection over union (IOU), Average surface distance (ASD) and Mean average error (MAE) were calculated to evaluate the performance of the segmentation method.

RESULTS

Analyses were made among our proposed method and other CNN-based and transformer-based methods. The results showed excellent segmentation performance, with a mean DSC of 0.858, a mean Hausdorff distance of 10.996, a mean IOU of 0.814, a mean MAE of 0.0005, and a mean ASD of 0.509. The boxplot of all test samples' segmentation performance implies that the proposed method has the lowest skewness and the highest average prediction performance.

CONCLUSIONS

Our proposed method can directly generate 3D lesion maps and showed excellent segmentation performance. The comparison of segmentation metrics and visualization results showed that our proposed method performed very well in the segmentation.

3D PET/CT tumor segmentation based on nnU-Net with GCN refinement

Objective. Whole-body positron emission tomography/computed tomography (PET/CT) scans are an important tool for diagnosing various malignancies (e.g. malignant melanoma, lymphoma, or lung cancer), and accurate segmentation of tumors is a key part of subsequent treatment. In recent years, convolutional neural network based segmentation methods have been extensively investigated. However, these methods often give inaccurate segmentation results, such as oversegmentation and undersegmentation. To address these issues, we propose a postprocessing method based on a graph convolutional network (GCN) to refine inaccurate segmentation results and improve the overall segmentation accuracy.Approach. First, nnU-Net is used as an initial segmentation framework, and the uncertainty in the segmentation results is analyzed. Certain and uncertain pixels are used to establish the nodes of a graph. Each node and its 6 neighbors form an edge, and 32 nodes are randomly selected as uncertain nodes to form edges. The highly uncertain nodes are used as the subsequent refinement targets. Second, the nnU-Net results of the certain nodes are used as labels to form a semisupervised graph network problem, and the uncertain part is optimized by training the GCN to improve the segmentation performance. This describes our proposed nnU-Net + GCN segmentation framework.Main results.We perform tumor segmentation experiments with the PET/CT dataset from the MICCIA2022 autoPET challenge. Among these data, 30 cases are randomly selected for testing, and the experimental results show that the false-positive rate is effectively reduced with nnU-Net + GCN refinement. In quantitative analysis, there is an improvement of 2.1% for the average Dice score, 6.4 for the 95% Hausdorff distance (HD95), and 1.7 for the average symmetric surface distance.Significance. The quantitative and qualitative evaluation results show that GCN postprocessing methods can effectively improve the tumor segmentation performance.

Deep neural network pulmonary nodule segmentation methods for CT images: Literature review and experimental comparisons

Automatic and accurate segmentation of pulmonary nodules in CT images can help physicians perform more accurate quantitative analysis, diagnose diseases, and improve patient survival. In recent years, with the development of deep learning technology, pulmonary nodule segmentation methods based on deep neural networks have gradually replaced traditional segmentation methods. This paper reviews the recent pulmonary nodule segmentation algorithms based on deep neural networks. First, the heterogeneity of pulmonary nodules, the interpretability of segmentation results, and external environmental factors are discussed, and then the open-source 2D and 3D models in medical segmentation tasks in recent years are applied to the Lung Image Database Consortium and Image Database Resource Initiative (LIDC) and Lung Nodule Analysis 16 (Luna16) datasets for comparison, and the visual diagnostic features marked by radiologists are evaluated one by one. According to the analysis of the experimental data, the following conclusions are drawn: (1) In the pulmonary nodule segmentation task, the performance of the 2D segmentation models DSC is generally better than that of the 3D segmentation models. (2) 'Subtlety', 'Sphericity', 'Margin', 'Texture', and 'Size' have more influence on pulmonary nodule segmentation, while 'Lobulation', 'Spiculation', and 'Benign and Malignant' features have less influence on pulmonary nodule segmentation. (3) Higher accuracy in pulmonary nodule segmentation can be achieved based on better-quality CT images. (4) Good contextual information acquisition and attention mechanism design positively affect pulmonary nodule segmentation.

An efficient deep neural network to segment lung nodule using optimized HDCCARUNet model

Lung cancer is a severe disease that may lead to death if left undiagnosed and untreated. Lung cancer recognition and segmentation is a difficult task in medical image processing. The study of Computed Tomography (CT) is an important phase for detecting abnormal tissues in the lung. The size of a nodule as well as the fine details of nodule can be varied for various images. Radiologists face a difficult task in diagnosing nodules from multiple images. Deep learning approaches outperform traditional learning algorithms when the data amount is large. One of the most common deep learning architectures is convolutional neural networks. Convolutional Neural Networks use pre-trained models like LeNet, AlexNet, GoogleNet, VGG16, VGG19, Resnet50, and others for learning features. This study proposes an optimized HDCCARUNet (Hybrid Dilated Convolutional Channel Attention Res-UNet) architecture, which combines an improved U-Net with a modified channel attention (MCA) block, and a HDAC (hybrid dilated attention convolutional) layer to accurately and effectively do medical image segmentation for various tasks. The attention mechanism aids in focusing on the desired outcome. The ability to dynamically allot input weights to neurons allows it to focus only on the most important information. In order to gather key details about different object features and infer a finer channel-wise attention, the proposed system uses a modified channel attention (MCA) block. The experiment is conducted on LIDC-IDRI dataset. The noises present in the dataset images are denoised by enhanced DWT filter and the performance is analysed at various noise levels. The proposed method achieves an accuracy rate of 99.58 % . Performance measures like accuracy, sensitivity, specificity, and ROC curves are evaluated and the system significantly outperforms other state-of-the-art systems.

Improved Complementary Pulmonary Nodule Segmentation Model Based on Multi-Feature Fusion

Accurate segmentation of lung nodules from pulmonary computed tomography (CT) slices plays a vital role in the analysis and diagnosis of lung cancer. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in the automatic segmentation of lung nodules. However, they are still challenged by the large diversity of segmentation targets, and the small inter-class variances between the nodule and its surrounding tissues. To tackle this issue, we propose a features complementary network according to the process of clinical diagnosis, which made full use of the complementarity and facilitation among lung nodule location information, global coarse area, and edge information. Specifically, we first consider the importance of global features of nodules in segmentation and propose a cross-scale weighted high-level feature decoder module. Then, we develop a low-level feature decoder module for edge feature refinement. Finally, we construct a complementary module to make information complement and promote each other. Furthermore, we weight pixels located at the nodule edge on the loss function and add an edge supervision to the deep supervision, both of which emphasize the importance of edges in segmentation. The experimental results demonstrate that our model achieves robust pulmonary nodule segmentation and more accurate edge segmentation.

Differentiation and prediction of pneumoconiosis stage by computed tomography texture analysis based on U-Net neural network

BACKGROUND AND OBJECTIVE

The progressive worsening of pneumoconiosis will ensue a hazardous physical condition in patients. This study details the differential diagnosis of the pneumoconiosis stage, by employing computed tomography (CT) texture analysis, based on U-Net neural network.

METHODS

The pneumoconiosis location from 92 patients at various stages was extracted by U-Net neural network. Mazda software was employed to analyze the texture features. Three dimensionality reduction methods set the best texture parameters. We applied four methods of the B11 module to analyze the selected texture parameters and calculate the misclassified rate (MCR). Finally, the receiver operating characteristic curve (ROC) of the texture parameters was analyzed, and the texture parameters with diagnostic efficiency were evaluated by calculating the area under curve (AUC).

RESULTS

The original film was processed by Gaussian and Laplace filters for a better display of the segmented area of pneumoconiosis in all stages. The MCR value obtained by the NDA analysis method under the MI dimension reduction method was the lowest, at 10.87%. In the filtered texture feature parameters, the best AUC was 0.821.

CONCLUSIONS

CT texture analysis based on the U-Net neural network can be used to identify the staging of pneumoconiosis.

A deep learning-based post-processing method for automated pulmonary lobe and airway trees segmentation using chest CT images in PET/CT

Background

The proposed algorithm could support accurate localization of lung disease. To develop and validate an automated deep learning model combined with a post-processing algorithm to segment six pulmonary anatomical regions in chest computed tomography (CT) images acquired during positron emission tomography/computed tomography (PET/CT) scans. The pulmonary regions have five pulmonary lobes and airway trees.

Methods

Patients who underwent both PET/CT imaging with an extra chest CT scan were retrospectively enrolled. The pulmonary segmentation of six regions in CT was performed via a convolutional neural network (CNN) of DenseVNet architecture with some post-processing algorithms. Three evaluation metrics were used to assess the performance of this method, which combined deep learning and the post-processing method. The agreement between the combined model and ground truth segmentations in the test set was analyzed.

Results

A total of 640 cases were enrolled. The combined model, which involved deep learning and post-processing methods, had a higher performance than the single deep learning model. In the test set, the all-lobes overall Dice coefficient, Hausdorff distance, and Jaccard coefficient were 0.972, 12.025 mm, and 0.948, respectively. The airway-tree Dice coefficient, Hausdorff distance, and Jaccard coefficient were 0.849, 32.076 mm, and 0.815, respectively. A good agreement was observed between our segmentation in every plot.

Conclusions

The proposed model combining two methods can automatically segment five pulmonary lobes and airway trees on chest CT imaging in PET/CT. The performance of the combined model was higher than the single deep learning model in each region in the test set.

Multi-scale segmentation squeeze-and-excitation UNet with conditional random field for segmenting lung tumor from CT images

BACKGROUND AND OBJECTIVE

Lung cancer counts among diseases with the highest global morbidity and mortality rates. The automatic segmentation of lung tumors from CT images is of vast significance. However, the segmentation faces several challenges, including variable shapes and different sizes, as well as complicated surrounding tissues.

METHODS

We propose a multi-scale segmentation squeeze-and-excitation UNet with a conditional random field (M-SegSEUNet-CRF) to automatically segment lung tumors from CT images. M-SegSEUNet-CRF employs a multi-scale strategy to solve the problem of variable tumor size. Through the spatially adaptive attention mechanism, the segmentation SE blocks embedded in 3D UNet are utilized to highlight tumor regions. The dense connected CRF framework is further added to delineate tumor boundaries at a detailed level. In total, 759 CT scans of patients with lung cancer were used to train and evaluate the M-SegSEUNet-CRF model (456 for training, 152 for validation, and 151 for test). Meanwhile, the public NSCLC-Radiomics and LIDC datasets have been utilized to validate the generalization of the proposed method. The role of different modules in the M-SegSEUNet-CRF model is analyzed by the ablation experiments, and the performance is compared with that of UNet, its variants and other state-of-the-art models.

RESULTS

M-SegSEUNet-CRF can achieve a Dice coefficient of 0.851 ± 0.071, intersection over union (IoU) of 0.747 ± 0.102, sensitivity of 0.827 ± 0.108, and positive predictive value (PPV) of 0.900 ± 0.107. Without a multi-scale strategy, the Dice coefficient drops to 0.820 ± 0.115; without CRF, it drops to 0.842 ± 0.082, and without both, it drops to 0.806 ± 0.120. M-SegSEUNet-CRF presented a higher Dice coefficient than 3D UNet (0.782 ± 0.115) and its variants (ResUNet, 0.797 ± 0.132; DenseUNet, 0.792 ± 0.111, and UNETR, 0.794 ± 0.130). Although the performance slightly declines with the decrease in tumor volume, M-SegSEUNet-CRF exhibits more obvious advantages than the other comparative models.

CONCLUSIONS

Our M-SegSEUNet-CRF model improves the segmentation ability of UNet through the multi-scale strategy and spatially adaptive attention mechanism. The CRF enables a more precise delineation of tumor boundaries. The M-SegSEUNet-CRF model integrates these characteristics and demonstrates outstanding performance in the task of lung tumor segmentation. It can furthermore be extended to deal with other segmentation problems in the medical imaging field.

MSAC-Net: 3D Multi-Scale Attention Convolutional Network for Multi-Spectral Imagery Pansharpening

Pansharpening fuses spectral information from the multi-spectral image and spatial information from the panchromatic image, generating super-resolution multi-spectral images with high spatial resolution. In this paper, we proposed a novel 3D multi-scale attention convolutional network (MSAC-Net) based on the typical U-Net framework for multi-spectral imagery pansharpening. MSAC-Net is designed via 3D convolution, and the attention mechanism replaces the skip connection between the contraction and expansion pathways. Multiple pansharpening layers at the expansion pathway are designed to calculate the reconstruction results for preserving multi-scale spatial information. The MSAC-Net performance is verified on the IKONOS and QuickBird satellites’ datasets, proving that MSAC-Net achieves comparable or superior performance to the state-of-the-art methods. Additionally, 2D and 3D convolution are compared, and the influences of the number of convolutions in the convolution block, the weight of multi-scale information, and the network’s depth on the network performance are analyzed.

Attention-VGG16-UNet: a novel deep learning approach for automatic segmentation of the median nerve in ultrasound images

BACKGROUND

Ultrasonography-an imaging technique that can show the anatomical section of nerves and surrounding tissues-is one of the most effective imaging methods to diagnose nerve diseases. However, segmenting the median nerve in two-dimensional (2D) ultrasound images is challenging due to the tiny and inconspicuous size of the nerve, the low contrast of images, and imaging noise. This study aimed to apply deep learning approaches to improve the accuracy of automatic segmentation of the median nerve in ultrasound images.

METHODS

In this study, we proposed an improved network called VGG16-UNet, which incorporates a contracting path and an expanding path. The contracting path is the VGG16 model with the 3 fully connected layers removed. The architecture of the expanding path resembles the upsampling path of U-Net. Moreover, attention mechanisms or/and residual modules were added to the U-Net and VGG16-UNet, which sequentially obtained Attention-UNet (A-UNet), Summation-UNet (S-UNet), Attention-Summation-UNet (AS-UNet), Attention-VGG16-UNet (A-VGG16-UNet), Summation-VGG16-UNet (S-VGG16-UNet), and Attention-Summation-VGG16-UNet (AS-VGG16-UNet). Each model was trained on the dataset of 910 median nerve images from 19 participants and tested on 207 frames from a new image sequence. The performance of the models was evaluated by metrics including Dice similarity coefficient (Dice), Jaccard similarity coefficient (Jaccard), Precision, and Recall. Based on the best segmentation results, we reconstructed a 3D median nerve image using the volume rendering method in the Visualization Toolkit (VTK) to assist in clinical nerve diagnosis.

RESULTS

The results of paired t-tests showed significant differences (P<0.01) in the metrics' values of different models. It showed that AS-UNet ranked first in U-Net models. The VGG16-UNet and its variants performed better than the corresponding U-Net models. Furthermore, the model's performance with the attention mechanism was superior to that with the residual module either based on U-Net or VGG16-UNet. The A-VGG16-UNet achieved the best performance (Dice =0.904±0.035, Jaccard =0.826±0.057, Precision =0.905±0.061, and Recall =0.909±0.061). Finally, we applied the trained A-VGG16-UNet to segment the median nerve in the image sequence, then reconstructed and visualized the 3D image of the median nerve.

CONCLUSIONS

This study demonstrates that the attention mechanism and residual module improve deep learning models for segmenting ultrasound images. The proposed VGG16-UNet-based models performed better than U-Net-based models. With segmentation, a 3D median nerve image can be reconstructed and can provide a visual reference for nerve diagnosis.