Anatomical Attention Guided Deep Networks for ROI Segmentation of Brain MR Images

Prior Knowledge-Guided U-Net for Automatic Clinical Target Volume Segmentation in Postmastectomy Radiation Therapy of Breast Cancer

PURPOSE

This study aimed to design and evaluate a prior-knowledge-guided U-Net (PK-UNet) for automatic clinical target volume (CTV) segmentation in postmastectomy radiation therapy for breast cancer.

METHODS AND MATERIALS

A total of 102 computed tomography (CT) scans from breast cancer patients who underwent postmastectomy were retrospectively collected. Of these, 80 scans were used for training with 5-fold cross-validation, and 22 scans for independent testing. The CTV included the chest wall, supraclavicular region, and axillary group III. The proposed PK-UNet method employs a 2-stage auto-segmentation process. Initially, the localization network categorizes CT slices based on the anatomic information of the CTV and generates prior knowledge labels. These outputs, along with the CT images, were fed into the final segmentation network. Quantitative evaluation was conducted using the mean Dice similarity coefficient (DSC), 95% Hausdorff distance, average surface distance, and surface DSC. A four-level objective scale evaluation was performed by 2 experienced radiation oncologists in a randomized double-blind manner.

RESULTS

Quantitative evaluations revealed that PK-UNet significantly outperformed state-of-the-art segmentation methods (P < .01), with a mean DSC of 0.90 ± 0.02 and a 95% Hausdorff distance of 2.82 ± 1.29 mm. The mean average surface distance of PK-UNet was 0.91 ± 0.22 mm and the surface DSC was 0.84 ± 0.07, significantly surpassing the performance of AdwU-Net (P < .01) and showing comparable results to other models. Clinical evaluation confirmed the efficacy of PK-UNet, with 81.8% of the predicted contours being acceptable for clinical application. The advantages of the auto-segmentation capability of PK-UNet were most evident in the superior and inferior slices and slices with discontinuities at the junctions of different subregions. The average manual correction time was reduced to 1.02 min, compared with 18.20 min for manual contouring leading to a 94.4% reduction in working time.

CONCLUSIONS

This study introduced the pioneering integration of prior medical knowledge into a deep learning framework for postmastectomy radiation therapy. This strategy addresses the challenges of CTV segmentation in postmastectomy radiation therapy and improves clinical workflow efficiency.

Automatic Segmentation of Bone Graft in Maxillary Sinus via Distance Constrained Network Guided by Prior Anatomical Knowledge

Maxillary Sinus Lifting is a crucial surgical procedure for addressing insufficient alveolar bone mass andsevere resorption in dental implant therapy. To accurately analyze the geometry changesof the bone graft (BG) in the maxillary sinus (MS), it is essential to perform quantitative analysis. However, automated BG segmentation remains a major challenge due to the complex local appearance, including blurred boundaries, lesion interference, implant and artifact interference, and BG exceeding the MS. Currently, there are few tools available that can efficiently and accurately segment BG from cone beam computed tomography (CBCT) image. In this paper, we propose a distance-constrained attention network guided by prior anatomical knowledge for the automatic segmentation of BG. First, a guidance strategy of preoperative prior anatomical knowledge is added to a deep neural network (DNN), which improves its ability to identify the dividing line between the MS and BG. Next, a coordinate attention gate is proposed, which utilizes the synergy of channel and position attention to highlight salient features from the skip connections. Additionally, the geodesic distance constraint is introduced into the DNN to form multi-task predictions, which reduces the deviation of the segmentation result. In the test experiment, the proposed DNN achieved a Dice similarity coefficient of 85.48 6.38%, an average surface distance error is 0.57 0.34mm, and a 95% Hausdorff distance of 2.64 2.09mm, which is superior to the comparison networks. It markedly improves the segmentation accuracy and efficiency of BG and has potential applications in analyzing its volume change and absorption rate in the future.

(DA-U)2Net: double attention U2Net for retinal vessel segmentation

BACKGROUND

Morphological changes in the retina are crucial and serve as valuable references in the clinical diagnosis of ophthalmic and cardiovascular diseases. However, the retinal vascular structure is complex, making manual segmentation time-consuming and labor-intensive.

METHODS

This paper proposes a retinal segmentation network that integrates feature channel attention and the Convolutional Block Attention Module (CBAM) attention within the U2Net model. First, a feature channel attention module is introduced into the RSU (Residual Spatial Unit) block of U2Net, forming an Attention-RSU block, which focuses more on significant areas during feature extraction and suppresses the influence of noise; Second, a Spatial Attention Module (SAM) is introduced into the high-resolution module of Attention-RSU to enrich feature extraction from both spatial and channel dimensions, and a Channel Attention Module (CAM) is integrated into the lowresolution module of Attention-RSU, which uses dual channel attention to reduce detail loss.Finally, dilated convolution is applied during the upscaling and downscaling processes to expand the receptive field in low-resolution states, allowing the model to better integrate contextual information.

RESULTS

The evaluation across multiple clinical datasets demonstrated excellent performance on various metrics, with an accuracy (ACC) of 98.71%.

CONCLUSION

The proposed Network is general enough and we believe it can be easily extended to other medical image segmentation tasks where large scale variation and complicated features are the main challenges.

Transferring Adult-Like Phase Images for Robust Multi-View Isointense Infant Brain Segmentation

Accurate tissue segmentation of infant brain in magnetic resonance (MR) images is crucial for charting early brain development and identifying biomarkers. Due to ongoing myelination and maturation, in the isointense phase (6-9 months of age), the gray and white matters of infant brain exhibit similar intensity levels in MR images, posing significant challenges for tissue segmentation. Meanwhile, in the adult-like phase around 12 months of age, the MR images show high tissue contrast and can be easily segmented. In this paper, we propose to effectively exploit adult-like phase images to achieve robust multi-view isointense infant brain segmentation. Specifically, in one way, we transfer adult-like phase images to the isointense view, which have similar tissue contrast as the isointense phase images, and use the transferred images to train an isointense-view segmentation network. On the other way, we transfer isointense phase images to the adult-like view, which have enhanced tissue contrast, for training a segmentation network in the adult-like view. The segmentation networks of different views form a multi-path architecture that performs multi-view learning to further boost the segmentation performance. Since anatomy-preserving style transfer is key to the downstream segmentation task, we develop a Disentangled Cycle-consistent Adversarial Network (DCAN) with strong regularization terms to accurately transfer realistic tissue contrast between isointense and adult-like phase images while still maintaining their structural consistency. Experiments on both NDAR and iSeg-2019 datasets demonstrate a significant superior performance of our method over the state-of-the-art methods.

Early cancer detection using deep learning and medical imaging: A survey

Cancer, characterized by the uncontrolled division of abnormal cells that harm body tissues, necessitates early detection for effective treatment. Medical imaging is crucial for identifying various cancers, yet its manual interpretation by radiologists is often subjective, labour-intensive, and time-consuming. Consequently, there is a critical need for an automated decision-making process to enhance cancer detection and diagnosis. Previously, a lot of work was done on surveys of different cancer detection methods, and most of them were focused on specific cancers and limited techniques. This study presents a comprehensive survey of cancer detection methods. It entails a review of 99 research articles collected from the Web of Science, IEEE, and Scopus databases, published between 2020 and 2024. The scope of the study encompasses 12 types of cancer, including breast, cervical, ovarian, prostate, esophageal, liver, pancreatic, colon, lung, oral, brain, and skin cancers. This study discusses different cancer detection techniques, including medical imaging data, image preprocessing, segmentation, feature extraction, deep learning and transfer learning methods, and evaluation metrics. Eventually, we summarised the datasets and techniques with research challenges and limitations. Finally, we provide future directions for enhancing cancer detection techniques.

Fully Automated Identification of Lymph Node Metastases and Lymphovascular Invasion in Endometrial Cancer From Multi-Parametric MRI by Deep Learning

BACKGROUND

Early and accurate identification of lymphatic node metastasis (LNM) and lymphatic vascular space invasion (LVSI) for endometrial cancer (EC) patients is important for treatment design, but difficult on multi-parametric MRI (mpMRI) images.

PURPOSE

To develop a deep learning (DL) model to simultaneously identify of LNM and LVSI of EC from mpMRI images.

STUDY TYPE

Retrospective.

POPULATION

Six hundred twenty-one patients with histologically proven EC from two institutions, including 111 LNM-positive and 168 LVSI-positive, divided into training, internal, and external test cohorts of 398, 169, and 54 patients, respectively.

FIELD STRENGTH/SEQUENCE

T2-weighted imaging (T2WI), contrast-enhanced T1WI (CE-T1WI), and diffusion-weighted imaging (DWI) were scanned with turbo spin-echo, gradient-echo, and two-dimensional echo-planar sequences, using either a 1.5 T or 3 T system.

ASSESSMENT

EC lesions were manually delineated on T2WI by two radiologists and used to train an nnU-Net model for automatic segmentation. A multi-task DL model was developed to simultaneously identify LNM and LVSI positive status using the segmented EC lesion regions and T2WI, CE-T1WI, and DWI images as inputs. The performance of the model for LNM-positive diagnosis was compared with those of three radiologists in the external test cohort.

STATISTICAL TESTS

Dice similarity coefficient (DSC) was used to evaluate segmentation results. Receiver Operating Characteristic (ROC) analysis was used to assess the performance of LNM and LVSI status identification. P value <0.05 was considered significant.

RESULTS

EC lesion segmentation model achieved mean DSC values of 0.700 ± 0.25 and 0.693 ± 0.21 in the internal and external test cohorts, respectively. For LNM positive/LVSI positive identification, the proposed model achieved AUC values of 0.895/0.848, 0.806/0.795, and 0.804/0.728 in the training, internal, and external test cohorts, respectively, and better than those of three radiologists (AUC = 0.770/0.648/0.674).

DATA CONCLUSION

The proposed model has potential to help clinicians to identify LNM and LVSI status of EC patients and improve treatment planning.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

Exploring Inherent Consistency for Semi-Supervised Anatomical Structure Segmentation in Medical Imaging

Due to the exorbitant expense of obtaining labeled data in the field of medical image analysis, semi-supervised learning has emerged as a favorable method for the segmentation of anatomical structures. Although semi-supervised learning techniques have shown great potential in this field, existing methods only utilize image-level spatial consistency to impose unsupervised regularization on data in label space. Considering that anatomical structures often possess inherent anatomical properties that have not been focused on in previous works, this study introduces the inherent consistency into semi-supervised anatomical structure segmentation. First, the prediction and the ground-truth are projected into an embedding space to obtain latent representations that encapsulate the inherent anatomical properties of the structures. Then, two inherent consistency constraints are designed to leverage these inherent properties by aligning these latent representations. The proposed method is plug-and-play and can be seamlessly integrated with existing methods, thereby collaborating to improve segmentation performance and enhance the anatomical plausibility of the results. To evaluate the effectiveness of the proposed method, experiments are conducted on three public datasets (ACDC, LA, and Pancreas). Extensive experimental results demonstrate that the proposed method exhibits good generalizability and outperforms several state-of-the-art methods.

MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion

Ultrasound image segmentation is a challenging task due to the complexity of lesion types, fuzzy boundaries, and low-contrast images along with the presence of noises and artifacts. To address these issues, we propose an end-to-end multi-scale feature extraction and fusion network (MEF-UNet) for the automatic segmentation of ultrasound images. Specifically, we first design a selective feature extraction encoder, including detail extraction stage and structure extraction stage, to precisely capture the edge details and overall shape features of the lesions. In order to enhance the representation capacity of contextual information, we develop a context information storage module in the skip-connection section, responsible for integrating information from adjacent two-layer feature maps. In addition, we design a multi-scale feature fusion module in the decoder section to merge feature maps with different scales. Experimental results indicate that our MEF-UNet can significantly improve the segmentation results in both quantitative analysis and visual effects.

A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping

Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e.g., brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.

A survey on cancer detection via convolutional neural networks: Current challenges and future directions

Cancer is a condition in which abnormal cells uncontrollably split and damage the body tissues. Hence, detecting cancer at an early stage is highly essential. Currently, medical images play an indispensable role in detecting various cancers; however, manual interpretation of these images by radiologists is observer-dependent, time-consuming, and tedious. An automatic decision-making process is thus an essential need for cancer detection and diagnosis. This paper presents a comprehensive survey on automated cancer detection in various human body organs, namely, the breast, lung, liver, prostate, brain, skin, and colon, using convolutional neural networks (CNN) and medical imaging techniques. It also includes a brief discussion about deep learning based on state-of-the-art cancer detection methods, their outcomes, and the possible medical imaging data used. Eventually, the description of the dataset used for cancer detection, the limitations of the existing solutions, future trends, and challenges in this domain are discussed. The utmost goal of this paper is to provide a piece of comprehensive and insightful information to researchers who have a keen interest in developing CNN-based models for cancer detection.

Dimensionality Reduction Hybrid U-Net for Brain Extraction in Magnetic Resonance Imaging

In various applications, such as disease diagnosis, surgical navigation, human brain atlas analysis, and other neuroimage processing scenarios, brain extraction is typically regarded as the initial stage in MRI image processing. Whole-brain semantic segmentation algorithms, such as U-Net, have demonstrated the ability to achieve relatively satisfactory results even with a limited number of training samples. In order to enhance the precision of brain semantic segmentation, various frameworks have been developed, including 3D U-Net, slice U-Net, and auto-context U-Net. However, the processing methods employed in these models are relatively complex when applied to 3D data models. In this article, we aim to reduce the complexity of the model while maintaining appropriate performance. As an initial step to enhance segmentation accuracy, the preprocessing extraction of full-scale information from magnetic resonance images is performed with a cluster tool. Subsequently, three multi-input hybrid U-Net model frameworks are tested and compared. Finally, we propose utilizing a fusion of two-dimensional segmentation outcomes from different planes to attain improved results. The performance of the proposed framework was tested using publicly accessible benchmark datasets, namely LPBA40, in which we obtained Dice overlap coefficients of 98.05%. Improvement was achieved via our algorithm against several previous studies.

Automated detection of regions of interest in cartridge case images using deep learning

This paper explores a deep-learning approach to evaluate the position of circular delimiters in cartridge case images. These delimiters define two regions of interest (ROI), corresponding to the breech face and the firing pin impressions, and are placed manually or by an image-processing algorithm. This positioning bears a significant impact on the performance of the image-matching algorithms for firearm identification, and an automated evaluation method would be beneficial to any computerized system. Our contribution consists in optimizing and training U-Net segmentation models from digital images of cartridge cases, intending to locate ROIs automatically. For the experiments, we used high-resolution 2D images from 1195 samples of cartridge cases fired by different 9MM firearms. Our results show that the segmentation models, trained on augmented data sets, exhibit a performance of 95.6% IoU (Intersection over Union) and 99.3% DC (Dice Coefficient) with a loss of 0.014 for the breech face images; and a performance of 95.9% IoU and 99.5% DC with a loss of 0.011 for the firing pin images. We observed that the natural shapes of predicted circles reduce the performance of segmentation models compared with perfect circles on ground truth masks suggesting that our method provide a more accurate segmentation of the real ROI shape. In practice, we believe that these results could be useful for firearms identification. In future work, the predictions may be used to evaluate the quality of delimiters on specimens in a database, or they could determine the region of interest on a cartridge case image.

An automatic interpretable deep learning pipeline for accurate Parkinson's disease diagnosis using quantitative susceptibility mapping and T1-weighted images

Parkinson's disease (PD) diagnosis based on magnetic resonance imaging (MRI) is still challenging clinically. Quantitative susceptibility maps (QSM) can potentially provide underlying pathophysiological information by detecting the iron distribution in deep gray matter (DGM) nuclei. We hypothesized that deep learning (DL) could be used to automatically segment all DGM nuclei and use relevant features for a better differentiation between PD and healthy controls (HC). In this study, we proposed a DL-based pipeline for automatic PD diagnosis based on QSM and T1-weighted (T1W) images. This consists of (1) a convolutional neural network model integrated with multiple attention mechanisms which simultaneously segments caudate nucleus, globus pallidus, putamen, red nucleus, and substantia nigra from QSM and T1W images, and (2) an SE-ResNeXt50 model with an anatomical attention mechanism, which uses QSM data and the segmented nuclei to distinguish PD from HC. The mean dice values for segmentation of the five DGM nuclei are all >0.83 in the internal testing cohort, suggesting that the model could segment brain nuclei accurately. The proposed PD diagnosis model achieved area under the the receiver operating characteristic curve (AUCs) of 0.901 and 0.845 on independent internal and external testing cohorts, respectively. Gradient-weighted class activation mapping (Grad-CAM) heatmaps were used to identify contributing nuclei for PD diagnosis on patient level. In conclusion, the proposed approach can potentially be used as an automatic, explainable pipeline for PD diagnosis in a clinical setting.

FMTM-feature-map-based transform model for brain image segmentation in tumor detection

The segmentation of brain images is a leading quantitative measure for detecting physiological changes and for analysing structural functions. Based on trends and dimensions of brain, the images indicate heterogeneity. Accurate brain tumour segmentation remains a critical challenge despite the persistent efforts of researchers were owing to a variety of obstacles. This impacts the outcome of tumour detection, causing errors. For addressing this issue, a Feature-Map based Transform Model (FMTM) is introduced to focus on heterogeneous features of input picture to map differences and intensity based on transition Fourier. Unchecked machine learning is used for reliable characteristic map recognition in this mapping process. For the determination of severity and variability, the method of identification depends on symmetry and texture. Learning instances are taught to improve precision using predefined data sets, regardless of loss of labels. The process is recurring until the maximum precision of tumour detection is achieved in low convergence. In this research, FMTM has been applied to brain tumour segmentation to automatically extract feature representations and produce accurate and steady performance because of promising performance made by powerful transition Fourier methods. The suggested model's performance is shown by the metrics processing time, precision, accuracy, and F1-Score.

Review of Semantic Segmentation of Medical Images Using Modified Architectures of UNET

In biomedical image analysis, information about the location and appearance of tumors and lesions is indispensable to aid doctors in treating and identifying the severity of diseases. Therefore, it is essential to segment the tumors and lesions. MRI, CT, PET, ultrasound, and X-ray are the different imaging systems to obtain this information. The well-known semantic segmentation technique is used in medical image analysis to identify and label regions of images. The semantic segmentation aims to divide the images into regions with comparable characteristics, including intensity, homogeneity, and texture. UNET is the deep learning network that segments the critical features. However, UNETs basic architecture cannot accurately segment complex MRI images. This review introduces the modified and improved models of UNET suitable for increasing segmentation accuracy.

FlowRAU-Net: Accelerated 4D Flow MRI of Aortic Valvular Flows With a Deep 2D Residual Attention Network

In this work, we propose a novel deep learning reconstruction framework for rapid and accurate reconstruction of 4D flow MRI data. Reconstruction is performed on a slice-by-slice basis by reducing artifacts in zero-filled reconstructed complex images obtained from undersampled k-space. A deep residual attention network FlowRAU-Net is proposed, trained separately for each encoding direction with 2D complex image slices extracted from complex 4D images at each temporal frame and slice position. The network was trained and tested on 4D flow MRI data of aortic valvular flow in 18 human subjects. Performance of the reconstructions was measured in terms of image quality, 3-D velocity vector accuracy, and accuracy in hemodynamic parameters. Reconstruction performance was measured for three different k-space undersamplings and compared with one state of the art compressed sensing reconstruction method and three deep learning-based reconstruction methods. The proposed method outperforms state of the art methods in all performance measures for all three different k-space undersamplings. Hemodynamic parameters such as blood flow rate and peak velocity from the proposed technique show good agreement with reference flow parameters. Visualization of the reconstructed image and velocity magnitude also shows excellent agreement with the fully sampled reference dataset. Moreover, the proposed method is computationally fast. Total 4D flow data (including all slices in space and time) for a subject can be reconstructed in 69 seconds on a single GPU. Although the proposed method has been applied to 4D flow MRI of aortic valvular flows, given a sufficient number of training samples, it should be applicable to other arterial flows.

A symmetric prior knowledge based deep learning model for intracerebral hemorrhage lesion segmentation

Background: Accurate localization and classification of intracerebral hemorrhage (ICH) lesions are of great significance for the treatment and prognosis of patients with ICH. The purpose of this study is to develop a symmetric prior knowledge based deep learning model to segment ICH lesions in computed tomography (CT). Methods: A novel symmetric Transformer network (Sym-TransNet) is designed to segment ICH lesions in CT images. A cohort of 1,157 patients diagnosed with ICH is established to train (n = 857), validate (n = 100), and test (n = 200) the Sym-TransNet. A healthy cohort of 200 subjects is added, establishing a test set with balanced positive and negative cases (n = 400), to further evaluate the accuracy, sensitivity, and specificity of the diagnosis of ICH. The segmentation results are obtained after data pre-processing and Sym-TransNet. The DICE coefficient is used to evaluate the similarity between the segmentation results and the segmentation gold standard. Furthermore, some recent deep learning methods are reproduced to compare with Sym-TransNet, and statistical analysis is performed to prove the statistical significance of the proposed method. Ablation experiments are conducted to prove that each component in Sym-TransNet could effectively improve the DICE coefficient of ICH lesions. Results: For the segmentation of ICH lesions, the DICE coefficient of Sym-TransNet is 0.716 ± 0.031 in the test set which contains 200 CT images of ICH. The DICE coefficients of five subtypes of ICH, including intraparenchymal hemorrhage (IPH), intraventricular hemorrhage (IVH), extradural hemorrhage (EDH), subdural hemorrhage (SDH), and subarachnoid hemorrhage (SAH), are 0.784 ± 0.039, 0.680 ± 0.049, 0.359 ± 0.186, 0.534 ± 0.455, and 0.337 ± 0.044, respectively. Statistical results show that the proposed Sym-TransNet can significantly improve the DICE coefficient of ICH lesions in most cases. In addition, the accuracy, sensitivity, and specificity of Sym-TransNet in the diagnosis of ICH in 400 CT images are 91.25%, 98.50%, and 84.00%, respectively. Conclusion: Compared with recent mainstream deep learning methods, the proposed Sym-TransNet can segment and identify different types of lesions from CT images of ICH patients more effectively. Moreover, the Sym-TransNet can diagnose ICH more stably and efficiently, which has clinical application prospects.

Brain-Computer Interface using neural network and temporal-spectral features

Brain-Computer Interfaces (BCIs) are increasingly useful for control. Such BCIs can be used to assist individuals who lost mobility or control over their limbs, for recreational purposes such as gaming or semi-autonomous driving, or as an interface toward man-machine integration. Thus far, the performance of algorithms used for thought decoding has been limited. We show that by extracting temporal and spectral features from electroencephalography (EEG) signals and, following, using deep learning neural network to classify those features, one can significantly improve the performance of BCIs in predicting which motor action was imagined by a subject. Our movement prediction algorithm uses Sequential Backward Selection technique to jointly choose temporal and spectral features and a radial basis function neural network for the classification. The method shows an average performance increase of 3.50% compared to state-of-the-art benchmark algorithms. Using two popular public datasets our algorithm reaches 90.08% accuracy (compared to an average benchmark of 79.99%) on the first dataset and 88.74% (average benchmark: 82.01%) on the second dataset. Given the high variability within- and across-subjects in EEG-based action decoding, we suggest that using features from multiple modalities along with neural network classification protocol is likely to increase the performance of BCIs across various tasks.

A Prior Knowledge-Guided, Deep Learning-Based Semiautomatic Segmentation for Complex Anatomy on Magnetic Resonance Imaging

PURPOSE

Despite recent substantial improvement in autosegmentation using deep learning (DL) methods, labor-intensive and time-consuming slice-by-slice manual editing is often needed, particularly for complex anatomy (eg, abdominal organs). This work aimed to develop a fast, prior knowledge-guided DL semiautomatic segmentation (DL-SAS) method for complex structures on abdominal magnetic resonance imaging (MRI) scans.

METHODS AND MATERIALS

A novel application using contours on an adjacent slice as a prior knowledge informant in a 2-dimensional UNet DL model to guide autosegmentation for a subsequent slice was implemented for DL-SAS. A generalized, instead of organ-specific, DL-SAS model was trained and tested for abdominal organs on T2-weighted MRI scans collected from 75 patients (65 for training and 10 for testing). The DL-SAS model performance was compared with 3 common autocontouring methods (linear interpolation, rigid propagation, and a full 3-dimensional DL autosegmentation model trained with the same training data set) based on various quantitative metrics including the Dice similarity coefficient (DSC) and ratio of acceptable slices (ROA) using paired t tests.

RESULTS

For the 10 testing cases, the DL-SAS model performed best with the slice interval (SI) of 1, resulting in an average DSC of 0.93 ± 0.02, 0.92 ± 0.02, 0.91 ± 0.02, 0.88 ± 0.03, and 0.87 ± 0.02 for the large bowel, stomach, small bowel, duodenum, and pancreas, respectively. The performance decreased with increased SIs from the guidance slice. The DL-SAS method performed significantly better (P < .05) than the other 3 methods. The ROA values were in the range of 48% to 66% for all the organs with an SI of 1 for DL-SAS, higher than those for linear interpolation (31%-57% for an SI of 1) and DL auto-segmentation (16%-51%).

CONCLUSIONS

The developed DL-SAS model segmented complex abdominal structures on MRI with high accuracy and efficiency and may be implemented as an interactive manual contouring tool or a contour editing tool in conjunction with a full autosegmentation process, facilitating fast and accurate segmentation for MRI-guided online adaptive radiation therapy.

Dual Encoder-Based Dynamic-Channel Graph Convolutional Network With Edge Enhancement for Retinal Vessel Segmentation

Retinal vessel segmentation with deep learning technology is a crucial auxiliary method for clinicians to diagnose fundus diseases. However, the deep learning approaches inevitably lose the edge information, which contains spatial features of vessels while performing down-sampling, leading to the limited segmentation performance of fine blood vessels. Furthermore, the existing methods ignore the dynamic topological correlations among feature maps in the deep learning framework, resulting in the inefficient capture of the channel characterization. To address these limitations, we propose a novel dual encoder-based dynamic-channel graph convolutional network with edge enhancement (DE-DCGCN-EE) for retinal vessel segmentation. Specifically, we first design an edge detection-based dual encoder to preserve the edge of vessels in down-sampling. Secondly, we investigate a dynamic-channel graph convolutional network to map the image channels to the topological space and synthesize the features of each channel on the topological map, which solves the limitation of insufficient channel information utilization. Finally, we study an edge enhancement block, aiming to fuse the edge and spatial features in the dual encoder, which is beneficial to improve the accuracy of fine blood vessel segmentation. Competitive experimental results on five retinal image datasets validate the efficacy of the proposed DE-DCGCN-EE, which achieves more remarkable segmentation results against the other state-of-the-art methods, indicating its potential clinical application.

BV-GAN: 3D time-of-flight magnetic resonance angiography cerebrovascular vessel segmentation using adversarial CNNs

Purpose: Cerebrovascular vessel segmentation is a key step in the detection of vessel pathology. Brain time-of-flight magnetic resonance angiography (TOF-MRA) is a main method used clinically for imaging of blood vessels using magnetic resonance imaging. This method is primarily used to detect narrowing, blockage of the arteries, and aneurysms. Despite its importance, TOF-MRA interpretation relies mostly on visual, subjective assessment performed by a neuroradiologist and is mostly based on maximum intensity projections reconstruction of the three-dimensional (3D) scan, thus reducing the acquired spatial resolution. Works tackling the central problem of automatically segmenting brain blood vessels typically suffer from memory and imbalance related issues. To address these issues, the spatial context of the segmentation consider by neural networks is typically restricted (e.g., by resolution reduction or analysis of environments of lower dimensions). Although efficient, such solutions hinder the ability of the neural networks to understand the complex 3D structures typical of the cerebrovascular system and to leverage this understanding for decision making. Approach: We propose a brain-vessels generative-adversarial-network (BV-GAN) segmentation model, that better considers connectivity and structural integrity, using prior based attention and adversarial learning techniques. Results: For evaluations, fivefold cross-validation experiments were performed on two datasets. BV-GAN demonstrates consistent improvement of up to 10% in vessel Dice score with each additive designed component to the baseline state-of-the-art models. Conclusions: Potentially, this automated 3D-approach could shorten analysis time, allow for quantitative characterization of vascular structures, and reduce the need to decrease resolution, overall improving diagnosis cerebrovascular vessel disorders.

NPCNet: Jointly Segment Primary Nasopharyngeal Carcinoma Tumors and Metastatic Lymph Nodes in MR Images

Nasopharyngeal carcinoma (NPC) is a malignant tumor whose survivability is greatly improved if early diagnosis and timely treatment are provided. Accurate segmentation of both the primary NPC tumors and metastatic lymph nodes (MLNs) is crucial for patient staging and radiotherapy scheduling. However, existing studies mainly focus on the segmentation of primary tumors, eliding the recognition of MLNs, and thus fail to comprehensively provide a landscape for tumor identification. There are three main challenges in segmenting primary NPC tumors and MLNs: variable location, variable size, and irregular boundary. To address these challenges, we propose an automatic segmentation network, named by NPCNet, to achieve segmentation of primary NPC tumors and MLNs simultaneously. Specifically, we design three modules, including position enhancement module (PEM), scale enhancement module (SEM), and boundary enhancement module (BEM), to address the above challenges. First, the PEM enhances the feature representations of the most suspicious regions. Subsequently, the SEM captures multiscale context information and target context information. Finally, the BEM rectifies the unreliable predictions in the segmentation mask. To that end, extensive experiments are conducted on our dataset of 9124 samples collected from 754 patients. Empirical results demonstrate that each module realizes its designed functionalities and is complementary to the others. By incorporating the three proposed modules together, our model achieves state-of-the-art performance compared with nine popular models.

Automatic segmentation model of intercondylar fossa based on deep learning: a novel and effective assessment method for the notch volume

BACKGROUND

Notch volume is associated with anterior cruciate ligament (ACL) injury. Manual tracking of intercondylar notch on MR images is time-consuming and laborious. Deep learning has become a powerful tool for processing medical images. This study aims to develop an MRI segmentation model of intercondylar fossa based on deep learning to automatically measure notch volume, and explore its correlation with ACL injury.

METHODS

The MRI data of 363 subjects (311 males and 52 females) with ACL injuries incurred during non-contact sports and 232 subjects (147 males and 85 females) with intact ACL were retrospectively analyzed. Each layer of intercondylar fossa was manually traced by radiologists on axial MR images. Notch volume was then calculated. We constructed an automatic segmentation system based on the architecture of Res-UNet for intercondylar fossa and used dice similarity coefficient (DSC) to compare the performance of segmentation systems by different networks. Unpaired t-test was performed to determine differences in notch volume between ACL-injured and intact groups, and between males and females.

RESULTS

The DSCs of intercondylar fossa based on different networks were all more than 0.90, and Res-UNet showed the best performance. The notch volume was significantly lower in the ACL-injured group than in the control group (6.12 ± 1.34 cm3 vs. 6.95 ± 1.75 cm3, P < 0.001). Females had lower notch volume than males (5.41 ± 1.30 cm3 vs. 6.76 ± 1.51 cm3, P < 0.001). Males and females who had ACL injuries had smaller notch than those with intact ACL (p < 0.001 and p < 0.005). Men had larger notches than women, regardless of the ACL injuries (p < 0.001).

CONCLUSION

Using a deep neural network to segment intercondylar fossa automatically provides a technical support for the clinical prediction and prevention of ACL injury and re-injury after surgery.

Attention Gate based dual-pathway Network for Vertebra Segmentation of X-ray Spine images

Automatic spine and vertebra segmentation from X-ray spine images is a critical and challenging problem in many computer-aid spinal image analysis and disease diagnosis applications. In this paper, a two-stage automatic segmentation framework for spine X-ray images is proposed, which can firstly locate the spine regions (including backbone, sacrum and illum) in the coarse stage and then identify eighteen vertebrae (i.e., cervical vertebra 1, thoracic vertebra 1-12 and lumbar vertebra 1-5) with isolate and clear boundary in the fine stage. A novel Attention Gate based dual-pathway Network (AGNet) composed of context and edge pathways is designed to extract semantic and boundary information for segmentation of both spine and vertebra regions. Multi-scale supervision mechanism is applied to explore comprehensive features and an Edge aware Fusion Mechanism (EFM) is proposed to fuse features extracted from the two pathways. Some other image processing skills, such as centralized backbone clipping, patch cropping and convex hull detection are introduced to further refine the vertebra segmentation results. Experimental validations on spine X-ray images dataset and vertebrae dataset suggest that the proposed AGNet achieves superior performance compared with state-of-the-art segmentation methods, and the coarse-to-fine framework can be implemented in real spinal diagnosis systems.

Multi parametric magnetic resonance imaging for radiation treatment planning

In recent years, multi-parametric magnetic resonance imaging (MpMRI) has played a major role in radiation therapy treatment planning. The superior soft tissue contrast, functional or physiological imaging capabilities and the flexibility of site-specific image sequence development has placed MpMRI at the forefront. In this article, the present status of MpMRI for external beam radiation therapy planning is reviewed. Common MpMRI sequences, preprocessing and QA strategies are briefly discussed, and various image registration techniques and strategies are addressed. Image segmentation methods including automatic segmentation and deep learning techniques for organs at risk and target delineation are reviewed. Due to the advancement in MRI guided online adaptive radiotherapy, treatment planning considerations addressing MRI only planning are also discussed. This article is protected by copyright. All rights reserved.

Analyzing fibrous tissue pattern in fibrous dysplasia bone images using deep R-CNN networks for segmentation

Predictive health monitoring systems help to detect human health threats in the early stage. Evolving deep learning techniques in medical image analysis results in efficient feedback in quick time. Fibrous dysplasia (FD) is a genetic disorder, triggered by the mutation in Guanine Nucleotide binding protein with alpha stimulatory activities in the human bone genesis. It slowly occupies the bone marrow and converts the bone cell into fibrous tissues. It weakens the bone structure and leads to permanent disability. This paper proposes the study of FD bone image analyzing techniques with deep networks. Also, the linear regression model is annotated for predicting the bone abnormality levels with observed coefficients. Modern image processing begins with various image filters. It describes the edges, shades, texture values of the receptive field. Different types of segmentation and edge detection mechanisms are applied to locate the tumor, lesion, and fibrous tissues in the bone image. Extract the fibrous region in the bone image using the region-based convolutional neural network algorithm. The segmented results are compared with their accuracy metrics. The segmentation loss is reduced by each iteration. The overall loss is 0.24% and the accuracy is 99%, segmenting the masked region produces 98% of accuracy, and building the bounding boxes is 99% of accuracy.

Automatic radiotherapy delineation quality assurance on prostate MRI with deep learning in a multicentre clinical trial

Volume delineation quality assurance (QA) is particularly important in clinical trial settings where consistent protocol implementation is required, as outcomes will affect future as well current patients. Currently, where feasible, this is conducted manually, which is time consuming and resource intensive. Although previous studies mostly focused on automating delineation QA on CT, magnetic resonance imaging (MRI) is being increasingly used in radiotherapy treatment. In this work, we propose to perform automatic delineation QA on prostate MRI for both the clinical target volume (CTV) and organs-at-risk (OARs) by using delineations generated by 3D Unet variants as benchmarks for QA. These networks were trained on a small gold standard atlas set and applied on a multicentre radiotherapy clinical trial dataset to generate benchmark delineations. Then, a QA stage was designed to recommend 'pass', 'minor correction' and 'major correction' for each manual delineation in the trial set by thresholding its Dice similarity coefficient to the network generated delineation. Among all 3D Unet variants explored, the Unet with anatomical gates in an AtlasNet architecture performed the best in delineation QA, achieving an area under the receiver operating characteristics curve of 0.97, 0.92, 0.89 and 0.97 for identifying unacceptable (major correction) delineations with a sensitivity of 0.93, 0.73, 0.74 and 0.90 at a specificity of 0.93, 0.86, 0.86 and 0.95 for bladder, prostate CTV, rectum and gel spacer respectively. To the best of our knowledge, this is the first study to propose automated delineation QA for a multicentre radiotherapy clinical trial with treatment planning MRI. The methods proposed in this work can potentially improve the accuracy and consistency of CTV and OAR delineation in radiotherapy treatment planning.

MDAN: Mirror Difference Aware Network for Brain Stroke Lesion Segmentation

Brain stroke lesion segmentation is of great importance for stroke rehabilitation neuroimaging analysis. Due to the large variance of stroke lesion shapes and similarities of tissue intensity distribution, it remains a challenging task. To help detect abnormalities, the anatomical symmetries of brain magnetic resonance (MR) images have been widely used as visual cues for clinical practices. However, most methods do not fully utilize structural symmetry information in brain images segmentation. This paper presents a novel mirror difference aware network (MDAN) for stroke lesion segmentation in an encoder-decoder architecture, aiming at holistically exploiting the symmetries of image features. Specifically, a differential feature augmentation (DFA) module is developed in the encoding path to highlight the semantically pathological asymmetries of the features in abnormalities. In the DFA module, a Siamese contrastive supervised loss is designed to enhance discriminative features, and a mirror position-based difference augmentation (MDA) module is used to further magnify the discrepancy information. Moreover, mirror feature fusion (MFF) modules are applied to fuse and transfer the information both of the original input and the horizontally flipped features to the decoding path. Extensive experiments on the Anatomical Tracings of Lesions After Stroke (ATLAS) dataset show the proposed MDAN outperforms the state-of-the-art methods.

Overall Survival Prediction for Gliomas Using a Novel Compound Approach

As a highly malignant tumor, the incidence and mortality of glioma are not optimistic. Predicting the survival time of patients with glioma by extracting the feature information from gliomas is beneficial for doctors to develop more targeted treatments. Magnetic resonance imaging (MRI) is a way to quickly and clearly capture the details of brain tissue. However, manually segmenting brain tumors from MRI will cost doctors a lot of energy, and doctors can only vaguely estimate the survival time of glioma patients, which are not conducive to the formulation of treatment plans. Therefore, automatically segmenting brain tumors and accurately predicting survival time has important significance. In this article, we first propose the NLSE-VNet model, which integrates the Non-Local module and the Squeeze-and-Excitation module into V-Net to segment three brain tumor sub-regions in multimodal MRI. Then extract the intensity, texture, wavelet, shape and other radiological features from the tumor area, and use the CNN network to extract the deep features. The factor analysis method is used to reduce the dimensionality of features, and finally the dimensionality-reduced features and clinical features such as age and tumor grade are combined into the random forest regression model to predict survival. We evaluate the effect on the BraTS 2019 and BraTS 2020 datasets. The average Dice of brain tumor segmentation tasks up to 79% and the average RMSE of the survival predictive task is as low as 311.5. The results indicate that the method in this paper has great advantages in segmentation and survival prediction of gliomas.

Incorporating the hybrid deformable model for improving the performance of abdominal CT segmentation via multi-scale feature fusion network

Automated multi-organ abdominal Computed Tomography (CT) image segmentation can assist the treatment planning, diagnosis, and improve many clinical workflows' efficiency. The 3-D Convolutional Neural Network (CNN) recently attained state-of-the-art accuracy, which typically relies on supervised training with many manual annotated data. Many methods used the data augmentation strategy with a rigid or affine spatial transformation to alleviate the over-fitting problem and improve the network's robustness. However, the rigid or affine spatial transformation fails to capture the complex voxel-based deformation in the abdomen, filled with many soft organs. We developed a novel Hybrid Deformable Model (HDM), which consists of the inter-and intra-patient deformation for more effective data augmentation to tackle this issue. The inter-patient deformations were extracted from the learning-based deformable registration between different patients, while the intra-patient deformations were formed using the random 3-D Thin-Plate-Spline (TPS) transformation. Incorporating the HDM enabled the network to capture many of the subtle deformations of abdominal organs. To find a better solution and achieve faster convergence for network training, we fused the pre-trained multi-scale features into the a 3-D attention U-Net. We directly compared the segmentation accuracy of the proposed method to the previous techniques on several centers' datasets via cross-validation. The proposed method achieves the average Dice Similarity Coefficient (DSC) 0.852, which outperformed the other state-of-the-art on multi-organ abdominal CT segmentation results.

APRNet: A 3D Anisotropic Pyramidal Reversible Network with Multi-modal Cross-Dimension Attention for Brain Tissue Segmentation in MR Images

Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.

A Survey on Deep Learning for Neuroimaging-Based Brain Disorder Analysis

Deep learning has recently been used for the analysis of neuroimages, such as structural magnetic resonance imaging (MRI), functional MRI, and positron emission tomography (PET), and it has achieved significant performance improvements over traditional machine learning in computer-aided diagnosis of brain disorders. This paper reviews the applications of deep learning methods for neuroimaging-based brain disorder analysis. We first provide a comprehensive overview of deep learning techniques and popular network architectures by introducing various types of deep neural networks and recent developments. We then review deep learning methods for computer-aided analysis of four typical brain disorders, including Alzheimer's disease, Parkinson's disease, Autism spectrum disorder, and Schizophrenia, where the first two diseases are neurodegenerative disorders and the last two are neurodevelopmental and psychiatric disorders, respectively. More importantly, we discuss the limitations of existing studies and present possible future directions.