1
|
Wen M, Shcherbakov P, Xu Y, Li J, Hu Y, Zhou Q, Liang H, Yuan L, Zhang X. A temporal enhanced semi-supervised training framework for needle segmentation in 3D ultrasound images. Phys Med Biol 2024; 69:115023. [PMID: 38684166 DOI: 10.1088/1361-6560/ad450b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 04/29/2024] [Indexed: 05/02/2024]
Abstract
Objective.Automated biopsy needle segmentation in 3D ultrasound images can be used for biopsy navigation, but it is quite challenging due to the low ultrasound image resolution and interference similar to the needle appearance. For 3D medical image segmentation, such deep learning networks as convolutional neural network and transformer have been investigated. However, these segmentation methods require numerous labeled data for training, have difficulty in meeting the real-time segmentation requirement and involve high memory consumption.Approach.In this paper, we have proposed the temporal information-based semi-supervised training framework for fast and accurate needle segmentation. Firstly, a novel circle transformer module based on the static and dynamic features has been designed after the encoders for extracting and fusing the temporal information. Then, the consistency constraints of the outputs before and after combining temporal information are proposed to provide the semi-supervision for the unlabeled volume. Finally, the model is trained using the loss function which combines the cross-entropy and Dice similarity coefficient (DSC) based segmentation loss with mean square error based consistency loss. The trained model with the single ultrasound volume input is applied to realize the needle segmentation in ultrasound volume.Main results.Experimental results on three needle ultrasound datasets acquired during the beagle biopsy show that our approach is superior to the most competitive mainstream temporal segmentation model and semi-supervised method by providing higher DSC (77.1% versus 76.5%), smaller needle tip position (1.28 mm versus 1.87 mm) and length (1.78 mm versus 2.19 mm) errors on the kidney dataset as well as DSC (78.5% versus 76.9%), needle tip position (0.86 mm versus 1.12 mm) and length (1.01 mm versus 1.26 mm) errors on the prostate dataset.Significance.The proposed method can significantly enhance needle segmentation accuracy by training with sequential images at no additional cost. This enhancement may further improve the effectiveness of biopsy navigation systems.
Collapse
Affiliation(s)
- Mingwei Wen
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| | - Pavel Shcherbakov
- Institute for Control Science, Russian Academy of Sciences, 65, Profsoyuznaya str., Moscow 117997, Russia
| | - Yang Xu
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Jing Li
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Yi Hu
- Hubei Medical Devices Quality Supervision and Test Institute, Wuhan, 430075, People's Republic of China
| | - Quan Zhou
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| | - Huageng Liang
- Department of Urology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, No 13, Hangkong Road, Wuhan 430022, People's Republic of China
| | - Li Yuan
- Department of Ultrasound imaging, Wuhan Children's Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, People's Republic of China
| | - Xuming Zhang
- Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, No 1037, Luoyu Road, Wuhan 430074, People's Republic of China
| |
Collapse
|
2
|
Xie L, Udupa JK, Tong Y, Torigian DA, Huang Z, Kogan RM, Wootton D, Choy KR, Sin S, Wagshul ME, Arens R. Automatic upper airway segmentation in static and dynamic MRI via anatomy-guided convolutional neural networks. Med Phys 2021; 49:324-342. [PMID: 34773260 DOI: 10.1002/mp.15345] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Revised: 09/08/2021] [Accepted: 10/29/2021] [Indexed: 11/10/2022] Open
Abstract
PURPOSE Upper airway segmentation on MR images is a prerequisite step for quantitatively studying the anatomical structure and function of the upper airway and surrounding tissues. However, the complex variability of intensity and shape of anatomical structures and different modes of image acquisition commonly used in this application makes automatic upper airway segmentation challenging. In this paper, we develop and test a comprehensive deep learning-based segmentation system for use on MR images to address this problem. MATERIALS AND METHODS In our study, both static and dynamic MRI data sets are utilized, including 58 axial static 3D MRI studies, 22 mid-retropalatal dynamic 2D MRI studies, 21 mid-retroglossal dynamic 2D MRI studies, 36 mid-sagittal dynamic 2D MRI studies, and 23 isotropic dynamic 3D MRI studies, involving a total of 160 subjects and over 20 000 MRI slices. Samples of static and 2D dynamic MRI data sets were randomly divided into training, validation, and test sets by an approximate ratio of 5:2:3. Considering that the variability of annotation data among 3D dynamic MRIs was greater than for other MRI data sets, we increased the ratio of training data for these data to improve the robustness of the model. We designed a unified framework consisting of the following procedures. For static MRI, a generalized region-of-interest (GROI) strategy is applied to localize the partitions of nasal cavity and other portions of upper airway in axial data sets as two separate subobjects. Subsequently, the two subobjects are segmented by two separate 2D U-Nets. The two segmentation results are combined as the whole upper airway structure. The GROI strategy is also applied to other MRI modes. To minimize false-positive and false-negative rates in the segmentation results, we employed a novel loss function based explicitly on these rates to train the segmentation networks. An inter-reader study is conducted to test the performance of our system in comparison to human variability in ground truth (GT) segmentation of these challenging structures. RESULTS The proposed approach yielded mean Dice coefficients of 0.84±0.03, 0.89±0.13, 0.84±0.07, and 0.86±0.05 for static 3D MRI, mid-retropalatal/mid-retroglossal 2D dynamic MRI, mid-sagittal 2D dynamic MRI, and isotropic dynamic 3D MRI, respectively. The quantitative results show excellent agreement with manual delineation results. The inter-reader study results demonstrate that the segmentation performance of our approach is statistically indistinguishable from manual segmentations considering the inter-reader variability in GT. CONCLUSIONS The proposed method can be utilized for routine upper airway segmentation from static and dynamic MR images with high accuracy and efficiency. The proposed approach has the potential to be employed in other dynamic MRI-related applications, such as lung or heart segmentation.
Collapse
Affiliation(s)
- Lipeng Xie
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China.,Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Jayaram K Udupa
- Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Yubing Tong
- Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Drew A Torigian
- Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Zihan Huang
- Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Rachel M Kogan
- Medical Image Processing Group, Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - David Wootton
- The Cooper Union for the Advancement of Science and Art, New York, New York, USA
| | - Kok R Choy
- The Cooper Union for the Advancement of Science and Art, New York, New York, USA
| | - Sanghun Sin
- Albert Einstein College of Medicine, Bronx, New York, USA
| | - Mark E Wagshul
- Albert Einstein College of Medicine, Bronx, New York, USA
| | - Raanan Arens
- Albert Einstein College of Medicine, Bronx, New York, USA
| |
Collapse
|