HIFUNet: Multi-Class Segmentation of Uterine Regions From MR Images Using Global Convolutional Networks for HIFU Surgery Planning

SDRD-Net: A Symmetric Dual-branch Residual Dense Network for OCT and US Image Fusion

Ultrasound (US) images have the advantages of no radiation, high penetration, and real-time imaging, and optical coherence tomography (OCT) has the advantage of high resolution. The purpose of fusing endometrial images from optical coherence tomography (OCT) and ultrasound (US) is to combine the advantages of different modalities to ultimately obtain more complete information on endometrial thickness. To better integrate multimodal images, we first proposed a Symmetric Dual-branch Residual Dense (SDRD-Net) network for OCT and US endometrial image fusion. Firstly, using Multi-scale Residual Dense Blocks (MRDB) to extract shallow features of different modalities. Then, the Base Transformer Module (BTM) and Detail Extraction Module (DEM) are used to extract primary and advanced features. Finally, the primary and advanced features are decomposed and recombined through the Feature Fusion Module (FMM), and the fused image is output. We have conducted experiments across both private and public datasets, encompassing IVF and MIF tasks, achieving commendable results.

Deep learning assisted detection and segmentation of uterine fibroids using multi-orientation magnetic resonance imaging

PURPOSE

To develop deep learning models for automated detection and segmentation of uterine fibroids using multi-orientation MRI.

METHODS

Pre-treatment sagittal and axial T2-weighted MRI scans acquired from patients diagnosed with uterine fibroids were collected. The proposed segmentation models were constructed based on the three-dimensional nnU-Net framework. Fibroid detection efficacy was assessed, with subgroup analyses by size and location. The segmentation performance was evaluated using Dice similarity coefficients (DSCs), 95% Hausdorff distance (HD95), and average surface distance (ASD).

RESULTS

The internal dataset comprised 299 patients who were divided into the training set (n = 239) and the internal test set (n = 60). The external dataset comprised 45 patients. The sagittal T2WI model and the axial T2WI model demonstrated recalls of 74.4%/76.4% and precision of 98.9%/97.9% for fibroid detection in the internal test set. The models achieved recalls of 93.7%/95.3% for fibroids ≥ 4 cm. The recalls for International Federation of Gynecology and Obstetrics (FIGO) type 2-5, FIGO types 0\1\2(submucous), fibroids FIGO types 5\6\7(subserous) were 100%/100%, 73.3%/78.6%, and 80.3%/81.9%, respectively. The proposed models demonstrated good performance in segmentation of the uterine fibroids with mean DSCs of 0.789 and 0.804, HD95s of 9.996 and 10.855 mm, and ASDs of 2.035 and 2.115 mm in the internal test set, and with mean DSCs of 0.834 and 0.818, HD95s of 9.971 and 11.874 mm, and ASDs of 2.031 and 2.273 mm in the external test set.

CONCLUSION

The proposed deep learning models showed promise as reliable methods for automating the detection and segmentation of the uterine fibroids, particularly those of clinical relevance.

Automatic segmentation of white matter lesions on multi-parametric MRI: convolutional neural network versus vision transformer

BACKGROUND AND PURPOSE

White matter hyperintensities in brain MRI are key indicators of various neurological conditions, and their accurate segmentation is essential for assessing disease progression. This study aims to evaluate the performance of a 3D convolutional neural network and a 3D Transformer-based model for white matter hyperintensities segmentation, focusing on their efficacy with limited datasets and similar computational resources.

MATERIALS AND METHODS

We implemented a convolution-based model (3D ResNet-50 U-Net with spatial and channel squeeze & excitation) and a Transformer-based model (3D Swin Transformer with a convolutional stem). The models were evaluated on two clinical datasets from Kaohsiung Chang Gung Memorial Hospital and National Center for High-Performance Computing. Four metrics were used for evaluation: Dice similarity coefficient, lesion segmentation, lesion F1-Score, and lesion sensitivity.

RESULTS

The Transformer-based model, with appropriate adjustments, outperformed the well-established convolution-based model in foreground Dice similarity coefficient, lesion F1-Score, and sensitivity, demonstrating robust segmentation accuracy. DRLoc enhanced the Transformer's performance, achieving comparable results on internal and benchmark datasets despite limited data availability.

CONCLUSION

With comparable computational overhead, a Transformer-based model can surpass a well-established convolution-based model in white matter hyperintensities segmentation on small datasets by capturing global context effectively, making them suitable for clinical applications where computational resources are constrained.

nnU-Net based segmentation and 3D reconstruction of uterine fibroids with MRI images for HIFU surgery planning

High-Intensity Focused Ultrasound (HIFU) ablation represents a rapidly advancing non-invasive treatment modality that has achieved considerable success in addressing uterine fibroids, which constitute over 50% of benign gynecological tumors. Preoperative Magnetic Resonance Imaging (MRI) plays a pivotal role in the planning and guidance of HIFU surgery for uterine fibroids, wherein the segmentation of tumors holds critical significance. The segmentation process was previously manually executed by medical experts, entailing a time-consuming and labor-intensive procedure heavily reliant on clinical expertise. This study introduced deep learning-based nnU-Net models, offering a cost-effective approach for their application in the segmentation of uterine fibroids utilizing preoperative MRI images. Furthermore, 3D reconstruction of the segmented targets was implemented to guide HIFU surgery. The evaluation of segmentation and 3D reconstruction performance was conducted with a focus on enhancing the safety and effectiveness of HIFU surgery. Results demonstrated the nnU-Net's commendable performance in the segmentation of uterine fibroids and their surrounding organs. Specifically, 3D nnU-Net achieved Dice Similarity Coefficients (DSC) of 92.55% for the uterus, 95.63% for fibroids, 92.69% for the spine, 89.63% for the endometrium, 97.75% for the bladder, and 90.45% for the urethral orifice. Compared to other state-of-the-art methods such as HIFUNet, U-Net, R2U-Net, ConvUNeXt and 2D nnU-Net, 3D nnU-Net demonstrated significantly higher DSC values, highlighting its superior accuracy and robustness. In conclusion, the efficacy of the 3D nnU-Net model for automated segmentation of the uterus and its surrounding organs was robustly validated. When integrated with intra-operative ultrasound imaging, this segmentation method and 3D reconstruction hold substantial potential to enhance the safety and efficiency of HIFU surgery in the clinical treatment of uterine fibroids.

Development and validation of a deep learning-based method for automatic measurement of uterus, fibroid, and ablated volume in MRI after MR-HIFU treatment of uterine fibroids

INTRODUCTION

The non-perfused volume divided by total fibroid load (NPV/TFL) is a predictive outcome parameter for MRI-guided high-intensity focused ultrasound (MR-HIFU) treatments of uterine fibroids, which is related to long-term symptom relief. In current clinical practice, the MR-HIFU outcome parameters are typically determined by visual inspection, so an automated computer-aided method could facilitate objective outcome quantification. The objective of this study was to develop and evaluate a deep learning-based segmentation algorithm for volume measurements of the uterus, uterine fibroids, and NPVs in MRI in order to automatically quantify the NPV/TFL.

MATERIALS AND METHODS

A segmentation pipeline was developed and evaluated using expert manual segmentations of MRI scans of 115 uterine fibroid patients, screened for and/or undergoing MR-HIFU treatment. The pipeline contained three separate neural networks, one per target structure. The first step in the pipeline was uterus segmentation from contrast-enhanced (CE)-T1w scans. This segmentation was subsequently used to remove non-uterus background tissue for NPV and fibroid segmentation. In the following step, NPVs were segmented from uterus-only CE-T1w scans. Finally, fibroids were segmented from uterus-only T2w scans. The segmentations were used to calculate the volume for each structure. Reliability and agreement between manual and automatic segmentations, volumes, and NPV/TFLs were assessed.

RESULTS

For treatment scans, the Dice similarity coefficients (DSC) between the manually and automatically obtained segmentations were 0.90 (uterus), 0.84 (NPV) and 0.74 (fibroid). Intraclass correlation coefficients (ICC) were 1.00 [0.99, 1.00] (uterus), 0.99 [0.98, 1.00] (NPV) and 0.98 [0.95, 0.99] (fibroid) between manually and automatically derived volumes. For manually and automatically derived NPV/TFLs, the mean difference was 5% [-41%, 51%] (ICC: 0.66 [0.32, 0.85]).

CONCLUSION

The algorithm presented in this study automatically calculates uterus volume, fibroid load, and NPVs, which could lead to more objective outcome quantification after MR-HIFU treatments of uterine fibroids in comparison to visual inspection. When robustness has been ascertained in a future study, this tool may eventually be employed in clinical practice to automatically measure the NPV/TFL after MR-HIFU procedures of uterine fibroids.

Large-scale uterine myoma MRI dataset covering all FIGO types with pixel-level annotations

Uterine myomas are the most common pelvic tumors in women, which can lead to abnormal uterine bleeding, abdominal pain, pelvic compression symptoms, infertility, or adverse pregnancy. In this article, we provide a dataset named uterine myoma MRI dataset (UMD), which can be used for clinical research on uterine myoma imaging. The UMD is the largest publicly available uterine MRI dataset to date including 300 cases of uterine myoma T2-weighted imaging (T2WI) sagittal patient images and their corresponding annotation files. The UMD covers 9 types of uterine myomas classified by the International Federation of Obstetrics and Gynecology (FIGO), which were annotated and reviewed by 11 experienced doctors to ensure the authority of the annotated data. The UMD is helpful for uterine myomas classification and uterine 3D reconstruction tasks, which has important implications for clinical research on uterine myomas.

Weakly supervised segmentation of uterus by scribble labeling on endometrial cancer MR images

Uterine segmentation of endometrial cancer MR images can be a valuable diagnostic tool for gynecologists. However, uterine segmentation based on deep learning relies on artificial pixel-level annotation, which is time-consuming, laborious and subjective. To reduce the dependence on pixel-level annotation, a method of weakly supervised uterine segmentation on endometrial cancer MRI slices is proposed, which only requires scribble label and is enhanced by pseudo-label technology, exponential geodesic distance loss and input disturbance strategy. Specifically, the limitations caused by the shortage of supervision are addressed by dynamically mixing the two outputs of the dual branch network to generate pseudo-labels, expanding supervision information and promoting mutual supervision training. On the other hand, considering the large difference of grayscale intensity between the uterus and surrounding tissues, the exponential geodesic distance loss is introduced to enhance the ability of the network to capture the edge of the uterus. Input disturbance strategies are incorporated to adapt to the flexible and variable characteristics of the uterus and further improve the segmentation performance of the network. The proposed method is evaluated on MRI images from 135 cases of endometrial cancer. Compared with other four weakly supervised segmentation methods, the performance of the proposed method is the best, whose mean DI, HD95, Recall, Precision, ADP are 92.8%, 11.632, 92.7%, 93.6%, 6.5% and increasing by 2.1%, 9.144, 0.6%, 2.4%, 2.9% respectively. The experimental results demonstrate that the proposed method is more effective than other weakly supervised methods and achieves similar performance as those fully supervised.

An Instance Segmentation Model Based on Deep Learning for Intelligent Diagnosis of Uterine Myomas in MRI

Uterine myomas affect 70% of women of reproductive age, potentially impacting their fertility and health. Manual film reading is commonly used to identify uterine myomas, but it is time-consuming, laborious, and subjective. Clinical treatment requires the consideration of the positional relationship among the uterine wall, uterine cavity, and uterine myomas. However, due to their complex and variable shapes, the low contrast of adjacent tissues or organs, and indistinguishable edges, accurately identifying them in MRI is difficult. Our work addresses these challenges by proposing an instance segmentation network capable of automatically outputting the location, category, and masks of each organ and lesion. Specifically, we designed a new backbone that facilitates learning the shape features of object diversity, and filters out background noise interference. We optimized the anchor box generation strategy to provide better priors in order to enhance the process of bounding box prediction and regression. An adaptive iterative subdivision strategy ensures that the mask boundary details of objects are more realistic and accurate. We conducted extensive experiments to validate our network, which achieved better average precision (AP) results than those of state-of-the-art instance segmentation models. Compared to the baseline network, our model improved AP on the uterine wall, uterine cavity, and myomas by 8.8%, 8.4%, and 3.2%, respectively. Our work is the first to realize multiclass instance segmentation in uterine MRI, providing a convenient and objective reference for the clinical development of appropriate surgical plans, and has significant value in improving diagnostic efficiency and realizing the automatic auxiliary diagnosis of uterine myomas.

Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

Transformer, one of the latest technological advances of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer's key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.

DARU‐Net: A dual attention residual U‐Net for uterine fibroids segmentation on MRI

PURPOSE

Uterine fibroid is the most common benign tumor in female reproductive organs. In order to guide the treatment, it is crucial to detect the location, shape, and size of the tumor. This study proposed a deep learning approach based on attention mechanisms to segment uterine fibroids automatically on preoperative Magnetic Resonance (MR) images.

METHODS

The proposed method is based on U-Net architecture and integrates two attention mechanisms: channel attention of squeeze-and-excitation (SE) blocks with residual connections, spatial attention of pyramid pooling module (PPM). We did the ablation study to verify the performance of these two attention mechanisms module and compared DARU-Net with other deep learning methods. All experiments were performed on a clinical dataset consisting of 150 cases collected from our hospital. Among them, 120 cases were used as the training set, and 30 cases are used as the test set. After preprocessing and data augmentation, we trained the network and tested it on the test dataset. We evaluated segmentation performance through the Dice similarity coefficient (DSC), precision, recall, and Jaccard index (JI).

RESULTS

The average DSC, precision, recall, and JI of DARU-Net reached 0.8066 ± 0.0956, 0.8233 ± 0.1255, 0.7913 ± 0.1304, and 0.6743 ± 0.1317. Compared with U-Net and other deep learning methods, DARU-Net was more accurate and stable.

CONCLUSION

This work proposed an optimized U-Net with channel and spatial attention mechanisms to segment uterine fibroids on preoperative MR images. Results showed that DARU-Net was able to accurately segment uterine fibroids from MR images.

Multi-scale feature pyramid fusion network for medical image segmentation

PURPOSE

Medical image segmentation is the most widely used technique in diagnostic and clinical research. However, accurate segmentation of target organs from blurred border regions and low-contrast adjacent organs in Computed tomography (CT) imaging is crucial for clinical diagnosis and treatment.

METHODS

In this article, we propose a Multi-Scale Feature Pyramid Fusion Network (MS-Net) based on the codec structure formed by the combination of Multi-Scale Attention Module (MSAM) and Stacked Feature Pyramid Module (SFPM). Among them, MSAM is used to skip connections, which aims to extract different levels of context details by dynamically adjusting the receptive fields under different network depths; the SFPM including multi-scale strategies and multi-layer Feature Perception Module (FPM) is nested in the network at the deepest point, which aims to better focus the network's attention on the target organ by adaptively increasing the weight of the features of interest.

RESULTS

Experiments demonstrate that the proposed MS-Net significantly improved the Dice score from 91.74% to 94.54% on CHAOS, from 97.59% to 98.59% on Lung, and from 82.55% to 86.06% on ISIC 2018, compared with U-Net. Additionally, comparisons with other six state-of-the-art codec structures also show the presented network has great advantages on evaluation indicators such as Miou, Dice, ACC and AUC.

CONCLUSION

The experimental results show that both the MSAM and SFPM techniques proposed in this paper can assist the network to improve the segmentation effect, so that the proposed MS-Net method achieves better results in the CHAOS, Lung and ISIC 2018 segmentation tasks.

Deep learning enables automated MRI-based estimation of uterine volume also in patients with uterine fibroids undergoing high-intensity focused ultrasound therapy

BACKGROUND

High-intensity focused ultrasound (HIFU) is used for the treatment of symptomatic leiomyomas. We aim to automate uterine volumetry for tracking changes after therapy with a 3D deep learning approach.

METHODS

A 3D nnU-Net model in the default setting and in a modified version including convolutional block attention modules (CBAMs) was developed on 3D T2-weighted MRI scans. Uterine segmentation was performed in 44 patients with routine pelvic MRI (standard group) and 56 patients with uterine fibroids undergoing ultrasound-guided HIFU therapy (HIFU group). Here, preHIFU scans (n = 56), postHIFU imaging maximum one day after HIFU (n = 54), and the last available follow-up examination (n = 53, days after HIFU: 420 ± 377) were included. The training was performed on 80% of the data with fivefold cross-validation. The remaining data were used as a hold-out test set. Ground truth was generated by a board-certified radiologist and a radiology resident. For the assessment of inter-reader agreement, all preHIFU examinations were segmented independently by both.

RESULTS

High segmentation performance was already observed for the default 3D nnU-Net (mean Dice score = 0.95 ± 0.05) on the validation sets. Since the CBAM nnU-Net showed no significant benefit, the less complex default model was applied to the hold-out test set, which resulted in accurate uterus segmentation (Dice scores: standard group 0.92 ± 0.07; HIFU group 0.96 ± 0.02), which was comparable to the agreement between the two readers.

CONCLUSIONS

This study presents a method for automatic uterus segmentation which allows a fast and consistent assessment of uterine volume. Therefore, this method could be used in the clinical setting for objective assessment of therapeutic response to HIFU therapy.

Clinical Artificial Intelligence Applications in Radiology: Chest and Abdomen

Organ segmentation, chest radiograph classification, and lung and liver nodule detections are some of the popular artificial intelligence (AI) tasks in chest and abdominal radiology due to the wide availability of public datasets. AI algorithms have achieved performance comparable to humans in less time for several organ segmentation tasks, and some lesion detection and classification tasks. This article introduces the current published articles of AI applied to chest and abdominal radiology, including organ segmentation, lesion detection, classification, and predicting prognosis.