1
|
Osman YBM, Li C, Huang W, Wang S. Collaborative Learning for Annotation-Efficient Volumetric MR Image Segmentation. J Magn Reson Imaging 2024; 60:1604-1614. [PMID: 38156427 DOI: 10.1002/jmri.29194] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 12/05/2023] [Accepted: 12/05/2023] [Indexed: 12/30/2023] Open
Abstract
BACKGROUND Deep learning has presented great potential in accurate MR image segmentation when enough labeled data are provided for network optimization. However, manually annotating three-dimensional (3D) MR images is tedious and time-consuming, requiring experts with rich domain knowledge and experience. PURPOSE To build a deep learning method exploring sparse annotations, namely only a single two-dimensional slice label for each 3D training MR image. STUDY TYPE Retrospective. POPULATION Three-dimensional MR images of 150 subjects from two publicly available datasets were included. Among them, 50 (1377 image slices) are for prostate segmentation. The other 100 (8800 image slices) are for left atrium segmentation. Five-fold cross-validation experiments were carried out utilizing the first dataset. For the second dataset, 80 subjects were used for training and 20 were used for testing. FIELD STRENGTH/SEQUENCE 1.5 T and 3.0 T; axial T2-weighted and late gadolinium-enhanced, 3D respiratory navigated, inversion recovery prepared gradient echo pulse sequence. ASSESSMENT A collaborative learning method by integrating the strengths of semi-supervised and self-supervised learning schemes was developed. The method was trained using labeled central slices and unlabeled noncentral slices. Segmentation performance on testing set was reported quantitatively and qualitatively. STATISTICAL TESTS Quantitative evaluation metrics including boundary intersection-over-union (B-IoU), Dice similarity coefficient, average symmetric surface distance, and relative absolute volume difference were calculated. Paired t test was performed, and P < 0.05 was considered statistically significant. RESULTS Compared to fully supervised training with only the labeled central slice, mean teacher, uncertainty-aware mean teacher, deep co-training, interpolation consistency training (ICT), and ambiguity-consensus mean teacher, the proposed method achieved a substantial improvement in segmentation accuracy, increasing the mean B-IoU significantly by more than 10.0% for prostate segmentation (proposed method B-IoU: 70.3% ± 7.6% vs. ICT B-IoU: 60.3% ± 11.2%) and by more than 6.0% for left atrium segmentation (proposed method B-IoU: 66.1% ± 6.8% vs. ICT B-IoU: 60.1% ± 7.1%). DATA CONCLUSIONS A collaborative learning method trained using sparse annotations can segment prostate and left atrium with high accuracy. LEVEL OF EVIDENCE 0 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Yousuf Babiker M Osman
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Cheng Li
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Weijian Huang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
- Peng Cheng Laboratory, Shenzhen, China
| | - Shanshan Wang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Peng Cheng Laboratory, Shenzhen, China
| |
Collapse
|
2
|
Lee SA, Kim HS, Yang E, Yoon YC, Lee JH, Choi BO, Kim JH. Efficient data labeling strategies for automated muscle segmentation in lower leg MRIs of Charcot-Marie-Tooth disease patients. PLoS One 2024; 19:e0310203. [PMID: 39241036 PMCID: PMC11379393 DOI: 10.1371/journal.pone.0310203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2024] [Accepted: 08/24/2024] [Indexed: 09/08/2024] Open
Abstract
We aimed to develop efficient data labeling strategies for ground truth segmentation in lower-leg magnetic resonance imaging (MRI) of patients with Charcot-Marie-Tooth disease (CMT) and to develop an automated muscle segmentation model using different labeling approaches. The impact of using unlabeled data on model performance was further examined. Using axial T1-weighted MRIs of 120 patients with CMT (60 each with mild and severe intramuscular fat infiltration), we compared the performance of segmentation models obtained using several different labeling strategies. The effect of leveraging unlabeled data on segmentation performance was evaluated by comparing the performances of few-supervised, semi-supervised (mean teacher model), and fully-supervised learning models. We employed a 2D U-Net architecture and assessed its performance by comparing the average Dice coefficients (ADC) using paired t-tests with Bonferroni correction. Among few-supervised models utilizing 10% labeled data, labeling three slices (the uppermost, central, and lowermost slices) per subject exhibited a significantly higher ADC (90.84±3.46%) compared with other strategies using a single image slice per subject (uppermost, 87.79±4.41%; central, 89.42±4.07%; lowermost, 89.29±4.71%, p < 0.0001) or all slices per subject (85.97±9.82%, p < 0.0001). Moreover, semi-supervised learning significantly enhanced the segmentation performance. The semi-supervised model using the three-slices strategy showed the highest segmentation performance (91.03±3.67%) among 10% labeled set models. Fully-supervised model showed an ADC of 91.39±3.76. A three-slice-based labeling strategy for ground truth segmentation is the most efficient method for developing automated muscle segmentation models of CMT lower leg MRI. Additionally, semi-supervised learning with unlabeled data significantly enhances segmentation performance.
Collapse
Affiliation(s)
- Seung-Ah Lee
- Research Institute for Future Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Hyun Su Kim
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Ehwa Yang
- Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Young Cheol Yoon
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Ji Hyun Lee
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Byung-Ok Choi
- Department of Neurology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - Jae-Hun Kim
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea
| |
Collapse
|
3
|
Liu Z, Kainth K, Zhou A, Deyer TW, Fayad ZA, Greenspan H, Mei X. A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation. NMR IN BIOMEDICINE 2024; 37:e5143. [PMID: 38523402 DOI: 10.1002/nbm.5143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 02/15/2024] [Accepted: 02/16/2024] [Indexed: 03/26/2024]
Abstract
Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.
Collapse
Affiliation(s)
- Zelong Liu
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Komal Kainth
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Alexander Zhou
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Timothy W Deyer
- East River Medical Imaging, New York, New York, USA
- Department of Radiology, Cornell Medicine, New York, New York, USA
| | - Zahi A Fayad
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Hayit Greenspan
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Xueyan Mei
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
4
|
Shoieb DA, Fathalla KM, Youssef SM, Younes A. CAT-Seg: cascaded medical assistive tool integrating residual attention mechanisms and Squeeze-Net for 3D MRI biventricular segmentation. Phys Eng Sci Med 2024; 47:153-168. [PMID: 37999903 DOI: 10.1007/s13246-023-01352-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 10/31/2023] [Indexed: 11/25/2023]
Abstract
Cardiac image segmentation is a critical step in the early detection of cardiovascular disease. The segmentation of the biventricular is a prerequisite for evaluating cardiac function in cardiac magnetic resonance imaging (CMRI). In this paper, a cascaded model CAT-Seg is proposed for segmentation of 3D-CMRI volumes. CAT-Seg addresses the problem of biventricular confusion with other regions and localized the region of interest (ROI) to reduce the scope of processing. A modified DeepLabv3+ variant integrating SqueezeNet (SqueezeDeepLabv3+) is proposed as a part of CAT-Seg. SqueezeDeepLabv3+ handles the different shapes of the biventricular through the different cardiac phases, as the biventricular only accounts for small portion of the volume slices. Also, CAT-Seg presents a segmentation approach that integrates attention mechanisms into 3D Residual UNet architecture (3D-ResUNet) called 3D-ARU to improve the segmentation results of the three major structures (left ventricle (LV), Myocardium (Myo), and right ventricle (RV)). The integration of the spatial attention mechanism into ResUNet handles the fuzzy edges of the three structures. The proposed model achieves promising results in training and testing with the Automatic Cardiac Diagnosis Challenge (ACDC 2017) dataset and the external validation using MyoPs. CAT-Seg demonstrates competitive performance with state-of-the-art models. On ACDC 2017, CAT-Seg is able to segment LV, Myo, and RV with an average minimum dice symmetry coefficient (DSC) performance gap of 1.165%, 4.36%, and 3.115% respectively. The average maximum improvement in terms of DSC in segmenting LV, Myo and RV is 4.395%, 6.84% and 7.315% respectively. On MyoPs external validation, CAT-Seg outperformed the state-of-the-art in segmenting LV, Myo, and RV with an average minimum performance gap of 6.13%, 5.44%, and 2.912% respectively.
Collapse
Affiliation(s)
- Doaa A Shoieb
- Computer Engineering Department, Arab Academy for Science, Technology and Maritime Transport (AASTMT), Alexandria, 1029, Egypt.
| | - Karma M Fathalla
- Computer Engineering Department, Arab Academy for Science, Technology and Maritime Transport (AASTMT), Alexandria, 1029, Egypt
| | - Sherin M Youssef
- Computer Engineering Department, Arab Academy for Science, Technology and Maritime Transport (AASTMT), Alexandria, 1029, Egypt
| | - Ahmed Younes
- Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt
| |
Collapse
|
5
|
Lecesne E, Simon A, Garreau M, Barone-Rochette G, Fouard C. Segmentation of cardiac infarction in delayed-enhancement MRI using probability map and transformers-based neural networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107841. [PMID: 37865006 DOI: 10.1016/j.cmpb.2023.107841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 09/15/2023] [Accepted: 10/01/2023] [Indexed: 10/23/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic segmentation of myocardial infarction is of great clinical interest for the quantitative evaluation of myocardial infarction (MI). Late Gadolinium Enhancement cardiac MRI (LGE-MRI) is commonly used in clinical practice to quantify MI, which is crucial for clinical diagnosis and treatment of cardiac diseases. However, the segmentation of infarcted tissue in LGE-MRI is highly challenging due to its high anisotropy and inhomogeneities. METHODS The innovative aspect of our work lies in the utilization of a probability map of the healthy myocardium to guide the localization of infarction, as well as the combination of 2D U-Net and U-Net transformers to achieve the final segmentation. Instead of employing a binary segmentation map, we propose using a probability map of the normal myocardium, obtained through a dedicated 2D U-Net. To leverage spatial information, we employ a U-Net transformers network where we incorporate the probability map into the original image as an additional input. Then, To address the limitations of U-Net in segmenting accurately the contours, we introduce an adapted loss function. RESULTS Our method has been evaluated on the 2020 MICCAI EMIDEC challenge dataset, yielding competitive results. Specifically, we achieved a Dice score of 92.94% for the myocardium and 92.36% for the infarction. These outcomes highlight the competitiveness of our approach. CONCLUSION In the case of the infarction class, our proposed method outperforms state-of-the-art techniques across all metrics evaluated in the challenge, establishing its superior performance in infarction segmentation. This study further reinforces the importance of integrating a contour loss into the segmentation process.
Collapse
Affiliation(s)
- Erwan Lecesne
- Univ Rennes, Inserm, LTSI - UMR 1099, Rennes, 35000, France.
| | - Antoine Simon
- Univ Rennes, Inserm, LTSI - UMR 1099, Rennes, 35000, France
| | | | - Gilles Barone-Rochette
- Clinic of Cardiology, Cardiovascular and Thoracic Department, University Hospital of Grenoble, Grenoble, 38000, France
| | - Céline Fouard
- Univ. Grenoble Alpes, CNRS, UMR 5525, VetAgro Sup, Grenoble INP, TIMC, Grenoble, 38000, France
| |
Collapse
|
6
|
Ng CKC. Generative Adversarial Network (Generative Artificial Intelligence) in Pediatric Radiology: A Systematic Review. CHILDREN (BASEL, SWITZERLAND) 2023; 10:1372. [PMID: 37628371 PMCID: PMC10453402 DOI: 10.3390/children10081372] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 08/07/2023] [Accepted: 08/09/2023] [Indexed: 08/27/2023]
Abstract
Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1-158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.
Collapse
Affiliation(s)
- Curtise K. C. Ng
- Curtin Medical School, Curtin University, GPO Box U1987, Perth, WA 6845, Australia; or ; Tel.: +61-8-9266-7314; Fax: +61-8-9266-2377
- Curtin Health Innovation Research Institute (CHIRI), Faculty of Health Sciences, Curtin University, GPO Box U1987, Perth, WA 6845, Australia
| |
Collapse
|
7
|
Sistaninejhad B, Rasi H, Nayeri P. A Review Paper about Deep Learning for Medical Image Analysis. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2023; 2023:7091301. [PMID: 37284172 PMCID: PMC10241570 DOI: 10.1155/2023/7091301] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Revised: 02/12/2023] [Accepted: 04/21/2023] [Indexed: 06/08/2023]
Abstract
Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep learning has revamped medical image analysis, yielding excellent results in image processing tasks such as registration, segmentation, feature extraction, and classification. The prime motivations for this are the availability of computational resources and the resurgence of deep convolutional neural networks. Deep learning techniques are good at observing hidden patterns in images and supporting clinicians in achieving diagnostic perfection. It has proven to be the most effective method for organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Many deep learning approaches have been published to analyze medical images for various diagnostic purposes. In this paper, we review the work exploiting current state-of-the-art deep learning approaches in medical image processing. We begin the survey by providing a synopsis of research works in medical imaging based on convolutional neural networks. Second, we discuss popular pretrained models and general adversarial networks that aid in improving convolutional networks' performance. Finally, to ease direct evaluation, we compile the performance metrics of deep learning models focusing on COVID-19 detection and child bone age prediction.
Collapse
Affiliation(s)
| | - Habib Rasi
- Sahand University of Technology, East Azerbaijan, New City of Sahand, Iran
| | - Parisa Nayeri
- Khoy University of Medical Sciences, West Azerbaijan, Khoy, Iran
| |
Collapse
|
8
|
Jafari M, Shoeibi A, Khodatars M, Ghassemi N, Moridian P, Alizadehsani R, Khosravi A, Ling SH, Delfan N, Zhang YD, Wang SH, Gorriz JM, Alinejad-Rokny H, Acharya UR. Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review. Comput Biol Med 2023; 160:106998. [PMID: 37182422 DOI: 10.1016/j.compbiomed.2023.106998] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 03/01/2023] [Accepted: 04/28/2023] [Indexed: 05/16/2023]
Abstract
In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.
Collapse
Affiliation(s)
- Mahboobeh Jafari
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Afshin Shoeibi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; Data Science and Computational Intelligence Institute, University of Granada, Spain.
| | - Marjane Khodatars
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Navid Ghassemi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Parisa Moridian
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Sai Ho Ling
- Faculty of Engineering and IT, University of Technology Sydney (UTS), Australia
| | - Niloufar Delfan
- Faculty of Computer Engineering, Dept. of Artificial Intelligence Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Yu-Dong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Shui-Hua Wang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Juan M Gorriz
- Data Science and Computational Intelligence Institute, University of Granada, Spain; Department of Psychiatry, University of Cambridge, UK
| | - Hamid Alinejad-Rokny
- BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; UNSW Data Science Hub, The University of New South Wales, Sydney, NSW, 2052, Australia; Health Data Analytics Program, Centre for Applied Artificial Intelligence, Macquarie University, Sydney, 2109, Australia
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia; Dept. of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
9
|
Sethi Y, Patel N, Kaka N, Desai A, Kaiwan O, Sheth M, Sharma R, Huang H, Chopra H, Khandaker MU, Lashin MMA, Hamd ZY, Emran TB. Artificial Intelligence in Pediatric Cardiology: A Scoping Review. J Clin Med 2022; 11:7072. [PMID: 36498651 PMCID: PMC9738645 DOI: 10.3390/jcm11237072] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Revised: 11/22/2022] [Accepted: 11/26/2022] [Indexed: 12/05/2022] Open
Abstract
The evolution of AI and data science has aided in mechanizing several aspects of medical care requiring critical thinking: diagnosis, risk stratification, and management, thus mitigating the burden of physicians and reducing the likelihood of human error. AI modalities have expanded feet to the specialty of pediatric cardiology as well. We conducted a scoping review searching the Scopus, Embase, and PubMed databases covering the recent literature between 2002-2022. We found that the use of neural networks and machine learning has significantly improved the diagnostic value of cardiac magnetic resonance imaging, echocardiograms, computer tomography scans, and electrocardiographs, thus augmenting the clinicians' diagnostic accuracy of pediatric heart diseases. The use of AI-based prediction algorithms in pediatric cardiac surgeries improves postoperative outcomes and prognosis to a great extent. Risk stratification and the prediction of treatment outcomes are feasible using the key clinical findings of each CHD with appropriate computational algorithms. Notably, AI can revolutionize prenatal prediction as well as the diagnosis of CHD using the EMR (electronic medical records) data on maternal risk factors. The use of AI in the diagnostics, risk stratification, and management of CHD in the near future is a promising possibility with current advancements in machine learning and neural networks. However, the challenges posed by the dearth of appropriate algorithms and their nascent nature, limited physician training, fear of over-mechanization, and apprehension of missing the 'human touch' limit the acceptability. Still, AI proposes to aid the clinician tomorrow with precision cardiology, paving a way for extremely efficient human-error-free health care.
Collapse
Affiliation(s)
- Yashendra Sethi
- PearResearch, Dehradun 248001, India
- Department of Medicine, Government Doon Medical College, Dehradun 248001, India
| | - Neil Patel
- PearResearch, Dehradun 248001, India
- Department of Medicine, GMERS Medical College, Himmatnagar 383001, India
| | - Nirja Kaka
- PearResearch, Dehradun 248001, India
- Department of Medicine, GMERS Medical College, Himmatnagar 383001, India
| | - Ami Desai
- Department of Medicine, SMIMER Medical College, Surat 395010, India
| | - Oroshay Kaiwan
- PearResearch, Dehradun 248001, India
- Department of Medicine, Northeast Ohio Medical University, Rootstown, OH 44272, USA
| | - Mili Sheth
- Department of Medicine, GMERS Gandhinagar, Gandhinagar 382012, India
| | - Rupal Sharma
- Department of Medicine, Government Medical College, Nagpur 440003, India
| | - Helen Huang
- Faculty of Medicine and Health Science, Royal College of Surgeons in Ireland, D02 YN77 Dublin, Ireland
| | - Hitesh Chopra
- Chitkara College of Pharmacy, Chitkara University, Rajpura 140401, India
| | - Mayeen Uddin Khandaker
- Centre for Applied Physics and Radiation Technologies, School of Engineering and Technology, Sunway University, Bandar Sunway 47500, Malaysia
| | - Maha M. A. Lashin
- Department of Biomedical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, P.O. 84428, Riyadh 11671, Saudi Arabia
| | - Zuhal Y. Hamd
- Department of Radiological Sciences, College of Health and Rehabilitation Sciences, Princess Nourah bint Abdulrahman University, P.O. 84428, Riyadh 11671, Saudi Arabia
| | - Talha Bin Emran
- Department of Pharmacy, BGC Trust University Bangladesh, Chittagong 4381, Bangladesh
- Department of Pharmacy, Faculty of Allied Health Sciences, Daffodil International University, Dhaka 1207, Bangladesh
| |
Collapse
|
10
|
Liu S, Wang H, Li Y, Li X, Cao G, Cao W. AHU-MultiNet: Adaptive loss balancing based on homoscedastic uncertainty in multi-task medical image segmentation network. Comput Biol Med 2022; 150:106157. [PMID: 37859277 DOI: 10.1016/j.compbiomed.2022.106157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 09/05/2022] [Accepted: 09/24/2022] [Indexed: 11/19/2022]
Abstract
Medical image segmentation is an important field in medical image analysis and a vital part of computer-aided diagnosis. Due to the challenges in acquiring image annotations, semi-supervised learning has attracted high attention in medical image segmentation. Despite their impressive performance, most existing semi-supervised approaches lack attention to ambiguous regions (e.g., some edges or corners around the organs). To achieve better performance, we propose a novel semi-supervised method called Adaptive Loss Balancing based on Homoscedastic Uncertainty in Multi-task Medical Image Segmentation Network (AHU-MultiNet). This model contains the main task for segmentation, one auxiliary task for signed distance, and another auxiliary task for contour detection. Our multi-task approach can effectively and sufficiently extract the semantic information of medical images by auxiliary tasks. Simultaneously, we introduce an inter-task consistency to explore the underlying information of the images and regularize the predictions in the right direction. More importantly, we notice and analyze that searching an optimal weighting manually to balance each task is a difficult and time-consuming process. Therefore, we introduce an adaptive loss balancing strategy based on homoscedastic uncertainty. Experimental results show that the two auxiliary tasks explicitly enforce shape-priors on the segmentation output to further generate more accurate masks under the adaptive loss balancing strategy. On several standard benchmarks, the 2018 Atrial Segmentation Challenge and the 2017 Liver Tumor Segmentation Challenge, our proposed method achieves improvements and outperforms the new state-of-the-art in semi-supervised learning.
Collapse
Affiliation(s)
- Shasha Liu
- The MOE Research Center for Software/Hardware Co-Design Engineering, East China Normal University, Shanghai, China.
| | - Hailing Wang
- The MOE Research Center for Software/Hardware Co-Design Engineering, East China Normal University, Shanghai, China.
| | - Yan Li
- The MOE Research Center for Software/Hardware Co-Design Engineering, East China Normal University, Shanghai, China.
| | - Xiaohu Li
- The MOE Research Center for Software/Hardware Co-Design Engineering, East China Normal University, Shanghai, China.
| | - Guitao Cao
- The MOE Research Center for Software/Hardware Co-Design Engineering, East China Normal University, Shanghai, China.
| | - Wenming Cao
- The College of Information Engineering, Shenzhen University, Shenzhen, China.
| |
Collapse
|
11
|
Yue C, Ye M, Wang P, Huang D, Lu X. Generative Adversarial Network Combined with SE-ResNet and Dilated Inception Block for Segmenting Retinal Vessels. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:3585506. [PMID: 36072751 PMCID: PMC9441346 DOI: 10.1155/2022/3585506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/15/2022] [Revised: 07/12/2022] [Accepted: 07/29/2022] [Indexed: 11/22/2022]
Abstract
This study develops an accurate method based on the generative adversarial network (GAN) that targets the issue of the current discontinuity of micro vessel segmentation in the retinal segmentation images. The processing of images has become increasingly efficient since the advent of deep learning method. We have proposed an improved GAN combined with SE-ResNet and dilated inception block for the segmenting retinal vessels (SAD-GAN). The GAN model has been improved with respect to the following points. (1) In the generator, the original convolution block is replaced with SE-ResNet module. Furthermore, SE-Net can extract the global channel information, while concomitantly strengthening and weakening the key features and invalid features, respectively. The residual structure can alleviate the issue of gradient disappearance. (2) The inception block and dilated convolution are introduced into the discriminator, which enhance the transmission of features and expand the acceptance domain for improved extraction of the deep network features. (3) We have included the attention mechanism in the discriminator for combining the local features with the corresponding global dependencies, and for highlighting the interdependent channel mapping. SAD-GAN performs satisfactorily on public retina datasets. On DRIVE dataset, ROC_AUC and PR_AUC reach 0.9813 and 0.8928, respectively. On CHASE_DB1 dataset, ROC_AUC and PR_AUC reach 0.9839 and 0.9002, respectively. Experimental results demonstrate that the generative adversarial model, combined with deep convolutional neural network, enhances the segmentation accuracy of the retinal vessels far above that of certain state-of-the-art methods.
Collapse
Affiliation(s)
- Chen Yue
- School of Medical Information, Wannan Medical College, Wuhu 241002, China
- Research Center of Health Big Data Mining and Applications, Wannan Medical College, Wuhu 241002, China
| | - Mingquan Ye
- School of Medical Information, Wannan Medical College, Wuhu 241002, China
- Research Center of Health Big Data Mining and Applications, Wannan Medical College, Wuhu 241002, China
| | - Peipei Wang
- School of Medical Information, Wannan Medical College, Wuhu 241002, China
- Research Center of Health Big Data Mining and Applications, Wannan Medical College, Wuhu 241002, China
| | - Daobin Huang
- School of Medical Information, Wannan Medical College, Wuhu 241002, China
- Research Center of Health Big Data Mining and Applications, Wannan Medical College, Wuhu 241002, China
| | - Xiaojie Lu
- School of Medical Information, Wannan Medical College, Wuhu 241002, China
- Research Center of Health Big Data Mining and Applications, Wannan Medical College, Wuhu 241002, China
| |
Collapse
|
12
|
Wang H, Gu H, Qin P, Wang J. U-shaped GAN for Semi-Supervised Learning and Unsupervised Domain Adaptation in High Resolution Chest Radiograph Segmentation. Front Med (Lausanne) 2022; 8:782664. [PMID: 35096877 PMCID: PMC8792862 DOI: 10.3389/fmed.2021.782664] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Accepted: 12/14/2021] [Indexed: 01/03/2023] Open
Abstract
Deep learning has achieved considerable success in medical image segmentation. However, applying deep learning in clinical environments often involves two problems: (1) scarcity of annotated data as data annotation is time-consuming and (2) varying attributes of different datasets due to domain shift. To address these problems, we propose an improved generative adversarial network (GAN) segmentation model, called U-shaped GAN, for limited-annotated chest radiograph datasets. The semi-supervised learning approach and unsupervised domain adaptation (UDA) approach are modeled into a unified framework for effective segmentation. We improve GAN by replacing the traditional discriminator with a U-shaped net, which predicts each pixel a label. The proposed U-shaped net is designed with high resolution radiographs (1,024 × 1,024) for effective segmentation while taking computational burden into account. The pointwise convolution is applied to U-shaped GAN for dimensionality reduction, which decreases the number of feature maps while retaining their salient features. Moreover, we design the U-shaped net with a pretrained ResNet-50 as an encoder to reduce the computational burden of training the encoder from scratch. A semi-supervised learning approach is proposed learning from limited annotated data while exploiting additional unannotated data with a pixel-level loss. U-shaped GAN is extended to UDA by taking the source and target domain data as the annotated data and the unannotated data in the semi-supervised learning approach, respectively. Compared to the previous models dealing with the aforementioned problems separately, U-shaped GAN is compatible with varying data distributions of multiple medical centers, with efficient training and optimizing performance. U-shaped GAN can be generalized to chest radiograph segmentation for clinical deployment. We evaluate U-shaped GAN with two chest radiograph datasets. U-shaped GAN is shown to significantly outperform the state-of-the-art models.
Collapse
Affiliation(s)
- Hongyu Wang
- Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, China
| | - Hong Gu
- Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, China
| | - Pan Qin
- Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, China
| | - Jia Wang
- Department of Surgery, The Second Hospital of Dalian Medical University, Dalian, China
| |
Collapse
|
13
|
URO-GAN: An untrustworthy region optimization approach for adipose tissue segmentation based on adversarial learning. APPL INTELL 2022. [DOI: 10.1007/s10489-021-02976-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
14
|
Jeong JJ, Tariq A, Adejumo T, Trivedi H, Gichoya JW, Banerjee I. Systematic Review of Generative Adversarial Networks (GANs) for Medical Image Classification and Segmentation. J Digit Imaging 2022; 35:137-152. [PMID: 35022924 PMCID: PMC8921387 DOI: 10.1007/s10278-021-00556-w] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 11/23/2021] [Accepted: 11/26/2021] [Indexed: 11/28/2022] Open
Abstract
In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.
Collapse
Affiliation(s)
- Jiwoong J Jeong
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA.
| | - Amara Tariq
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA
| | | | - Hari Trivedi
- Department of Radiology, Emory School of Medicine, Atlanta, USA
| | - Judy W Gichoya
- Department of Radiology, Emory School of Medicine, Atlanta, USA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, USA.,Department of Radiology, Emory School of Medicine, Atlanta, USA
| |
Collapse
|
15
|
Xun S, Li D, Zhu H, Chen M, Wang J, Li J, Chen M, Wu B, Zhang H, Chai X, Jiang Z, Zhang Y, Huang P. Generative adversarial networks in medical image segmentation: A review. Comput Biol Med 2022; 140:105063. [PMID: 34864584 DOI: 10.1016/j.compbiomed.2021.105063] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Revised: 11/14/2021] [Accepted: 11/20/2021] [Indexed: 12/13/2022]
Abstract
PURPOSE Since Generative Adversarial Network (GAN) was introduced into the field of deep learning in 2014, it has received extensive attention from academia and industry, and a lot of high-quality papers have been published. GAN effectively improves the accuracy of medical image segmentation because of its good generating ability and capability to capture data distribution. This paper introduces the origin, working principle, and extended variant of GAN, and it reviews the latest development of GAN-based medical image segmentation methods. METHOD To find the papers, we searched on Google Scholar and PubMed with the keywords like "segmentation", "medical image", and "GAN (or generative adversarial network)". Also, additional searches were performed on Semantic Scholar, Springer, arXiv, and the top conferences in computer science with the above keywords related to GAN. RESULTS We reviewed more than 120 GAN-based architectures for medical image segmentation that were published before September 2021. We categorized and summarized these papers according to the segmentation regions, imaging modality, and classification methods. Besides, we discussed the advantages, challenges, and future research directions of GAN in medical image segmentation. CONCLUSIONS We discussed in detail the recent papers on medical image segmentation using GAN. The application of GAN and its extended variants has effectively improved the accuracy of medical image segmentation. Obtaining the recognition of clinicians and patients and overcoming the instability, low repeatability, and uninterpretability of GAN will be an important research direction in the future.
Collapse
Affiliation(s)
- Siyi Xun
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong, 250358, China
| | - Dengwang Li
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong, 250358, China.
| | - Hui Zhu
- Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, Shandong, China
| | - Min Chen
- The Second Hospital of Shandong University, Shandong University, The Department of Medicine, The Second Hospital of Shandong University, Jinan, China
| | - Jianbo Wang
- Department of Radiation Oncology, Qilu Hospital, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, 250012, China
| | - Jie Li
- Department of Infectious Disease, Shandong Provincial Hospital Affiliated to Shandong University, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, China
| | - Meirong Chen
- Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, Shandong, China
| | - Bing Wu
- Laibo Biotechnology Co., Ltd., Jinan, Shandong, China
| | - Hua Zhang
- LinkingMed Technology Co., Ltd., Beijing, China
| | - Xiangfei Chai
- Huiying Medical Technology (Beijing) Co., Ltd., Beijing, China
| | - Zekun Jiang
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong, 250358, China
| | - Yan Zhang
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong, 250358, China
| | - Pu Huang
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong, 250358, China.
| |
Collapse
|
16
|
Chen X, Zhang C, Zhao J, Xiong Z, Zha ZJ, Wu F. Weakly Supervised Neuron Reconstruction From Optical Microscopy Images With Morphological Priors. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:3205-3216. [PMID: 33999814 DOI: 10.1109/tmi.2021.3080695] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Manually labeling neurons from high-resolution but noisy and low-contrast optical microscopy (OM) images is tedious. As a result, the lack of annotated data poses a key challenge when applying deep learning techniques for reconstructing neurons from noisy and low-contrast OM images. While traditional tracing methods provide a possible way to efficiently generate labels for supervised network training, the generated pseudo-labels contain many noisy and incorrect labels, which lead to severe performance degradation. On the other hand, the publicly available dataset, BigNeuron, provides a large number of single 3D neurons that are reconstructed using various imaging paradigms and tracing methods. Though the raw OM images are not fully available for these neurons, they convey essential morphological priors for complex 3D neuron structures. In this paper, we propose a new approach to exploit morphological priors from neurons that have been reconstructed for training a deep neural network to extract neuron signals from OM images. We integrate a deep segmentation network in a generative adversarial network (GAN), expecting the segmentation network to be weakly supervised by pseudo-labels at the pixel level while utilizing the supervision of previously reconstructed neurons at the morphology level. In our morphological-prior-guided neuron reconstruction GAN, named MP-NRGAN, the segmentation network extracts neuron signals from raw images, and the discriminator network encourages the extracted neurons to follow the morphology distribution of reconstructed neurons. Comprehensive experiments on the public VISoR-40 dataset and BigNeuron dataset demonstrate that our proposed MP-NRGAN outperforms state-of-the-art approaches with less training effort.
Collapse
|
17
|
Naderi AM, Bu H, Su J, Huang MH, Vo K, Trigo Torres RS, Chiao JC, Lee J, Lau MPH, Xu X, Cao H. Deep learning-based framework for cardiac function assessment in embryonic zebrafish from heart beating videos. Comput Biol Med 2021; 135:104565. [PMID: 34157469 DOI: 10.1016/j.compbiomed.2021.104565] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Revised: 06/07/2021] [Accepted: 06/08/2021] [Indexed: 11/26/2022]
Abstract
Zebrafish is a powerful and widely-used model system for a host of biological investigations, including cardiovascular studies and genetic screening. Zebrafish are readily assessable during developmental stages; however, the current methods for quantifying and monitoring cardiac functions mainly involve tedious manual work and inconsistent estimations. In this paper, we developed and validated a Zebrafish Automatic Cardiovascular Assessment Framework (ZACAF) based on a U-net deep learning model for automated assessment of cardiovascular indices, such as ejection fraction (EF) and fractional shortening (FS) from microscopic videos of wildtype and cardiomyopathy mutant zebrafish embryos. Our approach yielded favorable performance with accuracy above 90% compared with manual processing. We used only black and white regular microscopic recordings with frame rates of 5-20 frames per second (fps); thus, the framework could be widely applicable with any laboratory resources and infrastructure. Most importantly, the automatic feature holds promise to enable efficient, consistent, and reliable processing and analysis capacity for large amounts of videos, which can be generated by diverse collaborating teams.
Collapse
Affiliation(s)
- Amir Mohammad Naderi
- Department of Electrical Engineering and Computer Science, University of California, Irvine, CA, USA
| | - Haisong Bu
- Department of Biochemistry and Molecular Biology/Department of Cardiovascular Medicine, Mayo Clinic Rochester, MN, USA
| | - Jingcheng Su
- Department of Electrical Engineering and Computer Science, University of California, Irvine, CA, USA
| | - Mao-Hsiang Huang
- Department of Biomechatronics Engineering, National Taiwan University, Taipei, Taiwan
| | - Khuong Vo
- Department of Computer Science, University of California, Irvine, CA, USA
| | | | - J-C Chiao
- Department of Electrical and Computer Engineering, Southern Methodist University, Dallas, TX, USA
| | - Juhyun Lee
- Department of Bioengineering, University of Texas, Arlington, TX, USA
| | | | - Xiaolei Xu
- Department of Biochemistry and Molecular Biology/Department of Cardiovascular Medicine, Mayo Clinic Rochester, MN, USA
| | - Hung Cao
- Department of Electrical Engineering and Computer Science, University of California, Irvine, CA, USA; Department of Biomedical Engineering, University of California, Irvine, CA, USA; Sensoriis, Inc, Edmonds, WA, USA.
| |
Collapse
|
18
|
Wang Y, Zhang J. CMMCSegNet: Cross-Modality Multicascade Indirect LGE Segmentation on Multimodal Cardiac MR. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2021:9942149. [PMID: 34194539 PMCID: PMC8203380 DOI: 10.1155/2021/9942149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/13/2021] [Accepted: 05/24/2021] [Indexed: 11/18/2022]
Abstract
Since Late-Gadolinium Enhancement (LGE) of cardiac magnetic resonance (CMR) visualizes myocardial infarction, and the balanced-Steady State Free Precession (bSSFP) cine sequence can capture cardiac motions and present clear boundaries; multimodal CMR segmentation has played an important role in the assessment of myocardial viability and clinical diagnosis, while automatic and accurate CMR segmentation still remains challenging due to a very small amount of labeled LGE data and the relatively low contrasts of LGE. The main purpose of our work is to learn the real/fake bSSFP modality with ground truths to indirectly segment the LGE modality of cardiac MR by using a proposed cross-modality multicascade framework: cross-modality translation network and automatic segmentation network, respectively. In the segmentation stage, a novel multicascade pix2pix network is designed to segment the fake bSSFP sequence obtained from a cross-modality translation network. Moreover, we propose perceptual loss measuring features between ground truth and prediction, which are extracted from the pretrained vgg network in the segmentation stage. We evaluate the performance of the proposed method on the multimodal CMR dataset and verify its superiority over other state-of-the-art approaches under different network structures and different types of adversarial losses in terms of dice accuracy in testing. Therefore, the proposed network is promising for Indirect Cardiac LGE Segmentation in clinical applications.
Collapse
Affiliation(s)
- Yu Wang
- School of Mathematics and Computational Science, Xiangtan University, Xiangtan, Hunan 411105, China
| | - Jianping Zhang
- School of Mathematics and Computational Science, Xiangtan University, Xiangtan, Hunan 411105, China
| |
Collapse
|
19
|
Tibamoso-Pedraza G, Navarro I, Dion P, Raboisson MJ, Lapierre C, Miró J, Ratté S, Duong L. Design of heart phantoms for ultrasound imaging of ventricular septal defects. Int J Comput Assist Radiol Surg 2021; 17:177-184. [PMID: 34021458 DOI: 10.1007/s11548-021-02406-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Accepted: 05/11/2021] [Indexed: 12/24/2022]
Abstract
PURPOSE Ventricular septal defects (VSDs) are common congenital heart malformations. Echocardiography used during VSD hybrid cardiac procedures requires extensive training for image acquisition and interpretation. Cardiac surgery simulators with heart phantoms have shown usefulness for such training, but they are limited in visualization and characterization of complex VSD. This study explores a new method to build patient-specific heart phantoms with VSD, with proper tissue echogenicity for ultrasound imaging. METHODS Heart phantoms were designed from preoperative imaging of three patients with complex VSDs. Each whole heart phantom, including atrial and ventricular septums, was obtained by manual segmentation and by surface reconstruction, then by molding and by casting in different materials. Heart phantoms in silicone and polyvinyl alcohol cryogel (PVA-C) were considered, and they were reconstructed in 3-D using 2-D freehand ultrasound imaging. RESULTS An electromagnetic measurement system was used to measure the mean VSD diameters from the heart phantoms. Errors were evaluated below 1.0 mm for mean VSD diameters between 6.2 and 7.5 mm. CONCLUSION Patient-specific heart phantoms promise for representing complex heart malformations such as VSDs. PVA-C showed better tissue echogenicity than silicone for VSDs visualization and characterization.
Collapse
Affiliation(s)
- Gerardo Tibamoso-Pedraza
- Interventional Imaging Lab, Department of Software and IT Engineering, École de technologie supérieure, 1100 Notre-Dame West, Montreal, H3C 1K3, Canada.
| | - Iñaki Navarro
- Cardiology, Department of Pediatrics, CHU Sainte-Justine, Montreal, H3T 1C5, Canada
| | - Patrice Dion
- Interventional Imaging Lab, Department of Software and IT Engineering, École de technologie supérieure, 1100 Notre-Dame West, Montreal, H3C 1K3, Canada
| | | | - Chantale Lapierre
- Cardiology, Department of Pediatrics, CHU Sainte-Justine, Montreal, H3T 1C5, Canada
| | - Joaquim Miró
- Cardiology, Department of Pediatrics, CHU Sainte-Justine, Montreal, H3T 1C5, Canada
| | - Sylvie Ratté
- Interventional Imaging Lab, Department of Software and IT Engineering, École de technologie supérieure, 1100 Notre-Dame West, Montreal, H3C 1K3, Canada
| | - Luc Duong
- Interventional Imaging Lab, Department of Software and IT Engineering, École de technologie supérieure, 1100 Notre-Dame West, Montreal, H3C 1K3, Canada
| |
Collapse
|