1
|
Xu J, Xu HL, Cao YN, Huang Y, Gao S, Wu QJ, Gong TT. The performance of deep learning on thyroid nodule imaging predicts thyroid cancer: A systematic review and meta-analysis of epidemiological studies with independent external test sets. Diabetes Metab Syndr 2023; 17:102891. [PMID: 37907027 DOI: 10.1016/j.dsx.2023.102891] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 10/06/2023] [Accepted: 10/15/2023] [Indexed: 11/02/2023]
Abstract
BACKGROUND AND AIMS It is still controversial whether deep learning (DL) systems add accuracy to thyroid nodule imaging classification based on the recent available evidence. We conducted this study to analyze the current evidence of DL in thyroid nodule imaging diagnosis in both internal and external test sets. METHODS Until the end of December 2022, PubMed, IEEE, Embase, Web of Science, and the Cochrane Library were searched. We included primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. This systematic review was registered on PROSPERO (CRD42022362892). RESULTS We evaluated evidence from 17 primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. Fourteen studies were deemed eligible for meta-analysis. The pooled sensitivity, specificity, and area under the curve (AUC) of these DL algorithms were 0.89 (95% confidence interval 0.87-0.90), 0.84 (0.82-0.86), and 0.93 (0.91-0.95), respectively. For the internal validation set, the pooled sensitivity, specificity, and AUC were 0.91 (0.89-0.93), 0.88 (0.85-0.91), and 0.96 (0.93-0.97), respectively. In the external validation set, the pooled sensitivity, specificity, and AUC were 0.87 (0.85-0.89), 0.81 (0.77-0.83), and 0.91 (0.88-0.93), respectively. Notably, in subgroup analyses, DL algorithms still demonstrated exceptional diagnostic validity. CONCLUSIONS Current evidence suggests DL-based imaging shows diagnostic performances comparable to clinicians for differentiating thyroid nodules in both the internal and external test sets.
Collapse
Affiliation(s)
- Jin Xu
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China
| | - He-Li Xu
- Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Yi-Ning Cao
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China; Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Ying Huang
- Department of Ultrasound, Shengjing Hospital of China Medical University, Shenyang, China
| | - Song Gao
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Qi-Jun Wu
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China; Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China; Key Laboratory of Reproductive and Genetic Medicine (China Medical University), National Health Commission, Shenyang, China.
| | - Ting-Ting Gong
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China.
| |
Collapse
|
2
|
Zhao G, Kong D, Xu X, Hu S, Li Z, Tian J. Deep learning-based classification of breast lesions using dynamic ultrasound video. Eur J Radiol 2023; 165:110885. [PMID: 37290361 DOI: 10.1016/j.ejrad.2023.110885] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 03/27/2023] [Accepted: 05/17/2023] [Indexed: 06/10/2023]
Abstract
PURPOSE We intended to develop a deep-learning-based classification model based on breast ultrasound dynamic video, then evaluate its diagnostic performance in comparison with the classic model based on ultrasound static image and that of different radiologists. METHOD We collected 1000 breast lesions from 888 patients from May 2020 to December 2021. Each lesion contained two static images and two dynamic videos. We divided these lesions randomly into training, validation, and test sets by the ratio of 7:2:1. Two deep learning (DL) models, namely DL-video and DL-image, were developed based on 3D Resnet-50 and 2D Resnet-50 using 2000 dynamic videos and 2000 static images, respectively. Lesions in the test set were evaluated to compare the diagnostic performance of two models and six radiologists with different seniority. RESULTS The area under the curve of the DL-video model was significantly higher than those of the DL-image model (0.969 vs. 0.925, P = 0.0172) and six radiologists (0.969 vs. 0.779-0.912, P < 0.05). All radiologists performed better when evaluating the dynamic videos compared to the static images. Furthermore, radiologists performed better with increased seniority both in reading images and videos. CONCLUSIONS The DL-video model can discern more detailed spatial and temporal information for accurate classification of breast lesions than the conventional DL-image model and radiologists, and its clinical application can further improve the diagnosis of breast cancer.
Collapse
Affiliation(s)
- Guojia Zhao
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China; Department of Ultrasound, Lin Yi People's Hospital, Linyi, Shandong, China
| | | | - Xiangli Xu
- The Second Hospital of Harbin, Harbin, Heilongjiang, China
| | - Shunbo Hu
- Lin Yi University, Linyi, Shandong, China.
| | - Ziyao Li
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China.
| | - Jiawei Tian
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China.
| |
Collapse
|
3
|
Witowski J, Heacock L, Reig B, Kang SK, Lewin A, Pysarenko K, Patel S, Samreen N, Rudnicki W, Łuczyńska E, Popiela T, Moy L, Geras KJ. Improving breast cancer diagnostics with deep learning for MRI. Sci Transl Med 2022; 14:eabo4802. [PMID: 36170446 DOI: 10.1126/scitranslmed.abo4802] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Abstract
Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has a high sensitivity in detecting breast cancer but often leads to unnecessary biopsies and patient workup. We used a deep learning (DL) system to improve the overall accuracy of breast cancer diagnosis and personalize management of patients undergoing DCE-MRI. On the internal test set (n = 3936 exams), our system achieved an area under the receiver operating characteristic curve (AUROC) of 0.92 (95% CI: 0.92 to 0.93). In a retrospective reader study, there was no statistically significant difference (P = 0.19) between five board-certified breast radiologists and the DL system (mean ΔAUROC, +0.04 in favor of the DL system). Radiologists' performance improved when their predictions were averaged with DL's predictions [mean ΔAUPRC (area under the precision-recall curve), +0.07]. We demonstrated the generalizability of the DL system using multiple datasets from Poland and the United States. An additional reader study on a Polish dataset showed that the DL system was as robust to distribution shift as radiologists. In subgroup analysis, we observed consistent results across different cancer subtypes and patient demographics. Using decision curve analysis, we showed that the DL system can reduce unnecessary biopsies in the range of clinically relevant risk thresholds. This would lead to avoiding biopsies yielding benign results in up to 20% of all patients with BI-RADS category 4 lesions. Last, we performed an error analysis, investigating situations where DL predictions were mostly incorrect. This exploratory work creates a foundation for deployment and prospective analysis of DL-based models for breast MRI.
Collapse
Affiliation(s)
- Jan Witowski
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA.,Center for Advanced Imaging Innovation and Research, New York University, New York, NY 10016, USA
| | - Laura Heacock
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Beatriu Reig
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Stella K Kang
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA.,Department of Population Health, New York University Grossman School of Medicine, New York NY 10016, USA
| | - Alana Lewin
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Kristine Pysarenko
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Shalin Patel
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Naziya Samreen
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Wojciech Rudnicki
- Electroradiology Department, Jagiellonian University Medical College, 31-126 Kraków, Poland
| | - Elżbieta Łuczyńska
- Electroradiology Department, Jagiellonian University Medical College, 31-126 Kraków, Poland
| | - Tadeusz Popiela
- Chair of Radiology, Jagiellonian University Medical College, 31-501 Kraków, Poland
| | - Linda Moy
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA.,Center for Advanced Imaging Innovation and Research, New York University, New York, NY 10016, USA.,Vilcek Institute of Graduate Biomedical Sciences, New York University Grossman School of Medicine, New York, NY 10016, USA.,Perlmutter Cancer Center, New York University Langone Health, New York, NY 10016, USA
| | - Krzysztof J Geras
- Department of Radiology, New York University Grossman School of Medicine, New York, NY 10016, USA.,Center for Advanced Imaging Innovation and Research, New York University, New York, NY 10016, USA.,Vilcek Institute of Graduate Biomedical Sciences, New York University Grossman School of Medicine, New York, NY 10016, USA.,Center for Data Science, New York University, New York NY 10011, USA.,Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, New York NY 10012, USA
| |
Collapse
|
4
|
Zhao C, Xiao M, Ma L, Ye X, Deng J, Cui L, Guo F, Wu M, Luo B, Chen Q, Chen W, Guo J, Li Q, Zhang Q, Li J, Jiang Y, Zhu Q. Enhancing Performance of Breast Ultrasound in Opportunistic Screening Women by a Deep Learning-Based System: A Multicenter Prospective Study. Front Oncol 2022; 12:804632. [PMID: 35223484 PMCID: PMC8867611 DOI: 10.3389/fonc.2022.804632] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 01/07/2022] [Indexed: 12/21/2022] Open
Abstract
PURPOSE To validate the feasibility of S-Detect, an ultrasound computer-aided diagnosis (CAD) system using deep learning, in enhancing the diagnostic performance of breast ultrasound (US) for patients with opportunistic screening-detected breast lesions. METHODS Nine medical centers throughout China participated in this prospective study. Asymptomatic patients with US-detected breast masses were enrolled and received conventional US, S-Detect, and strain elastography subsequently. The final pathological results are referred to as the gold standard for classifying breast mass. The diagnostic performances of the three methods and the combination of S-Detect and elastography were evaluated and compared, including sensitivity, specificity, and area under the receiver operating characteristics (AUC) curve. We also compared the diagnostic performances of S-Detect among different study sites. RESULTS A total of 757 patients were enrolled, including 460 benign and 297 malignant cases. S-Detect exhibited significantly higher AUC and specificity than conventional US (AUC, S-Detect 0.83 [0.80-0.85] vs. US 0.74 [0.70-0.77], p < 0.0001; specificity, S-Detect 74.35% [70.10%-78.28%] vs. US 54.13% [51.42%-60.29%], p < 0.0001), with no decrease in sensitivity. In comparison to that of S-Detect alone, the AUC value significantly was enhanced after combining elastography and S-Detect (0.87 [0.84-0.90]), without compromising specificity (73.93% [68.60%-78.78%]). Significant differences in the S-Detect's performance were also observed across different study sites (AUC of S-Detect in Groups 1-4: 0.89 [0.84-0.93], 0.84 [0.77-0.89], 0.85 [0.76-0.92], 0.75 [0.69-0.80]; p [1 vs. 4] < 0.0001, p [2 vs. 4] = 0.0165, p [3 vs. 4] = 0.0157). CONCLUSIONS Compared with the conventional US, S-Detect presented higher overall accuracy and specificity. After S-Detect and strain elastography were combined, the performance could be further enhanced. The performances of S-Detect also varied among different centers.
Collapse
Affiliation(s)
- Chenyang Zhao
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mengsu Xiao
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Li Ma
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Xinhua Ye
- Department of Ultrasound, First Affiliated Hospital, Nanjing Medical University, Nanjing, China
| | - Jing Deng
- Department of Ultrasound, First Affiliated Hospital, Nanjing Medical University, Nanjing, China
| | - Ligang Cui
- Department of Ultrasound, Peking University Third Hospital, Beijing, China
| | - Fajin Guo
- Department of Ultrasound, Beijing Hospital, Beijing, China
| | - Min Wu
- Department of Ultrasound, Nanjing Drum Tower Hospital, Nanjing, China
| | - Baoming Luo
- Department of Ultrasound, Sun Yat-sen Memorial Hospital, Guangzhou, China
| | - Qin Chen
- Department of Ultrasound, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Wu Chen
- Department of Ultrasound, First Hospital of Shanxi Medical University, Taiyuan, China
| | - Jun Guo
- Department of Ultrasound, Aero Space Central Hospital, Beijing, China
| | - Qian Li
- Department of Ultrasound, Henan Provincial Cancer Hospital, Zhengzhou, China
| | - Qing Zhang
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jianchu Li
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuxin Jiang
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Qingli Zhu
- Department of Ultrasound, Chinese Academy of Medical Sciences and Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
5
|
Deep learning in image-based breast and cervical cancer detection: a systematic review and meta-analysis. NPJ Digit Med 2022; 5:19. [PMID: 35169217 PMCID: PMC8847584 DOI: 10.1038/s41746-022-00559-z] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Accepted: 12/22/2021] [Indexed: 12/15/2022] Open
Abstract
Accurate early detection of breast and cervical cancer is vital for treatment success. Here, we conduct a meta-analysis to assess the diagnostic performance of deep learning (DL) algorithms for early breast and cervical cancer identification. Four subgroups are also investigated: cancer type (breast or cervical), validation type (internal or external), imaging modalities (mammography, ultrasound, cytology, or colposcopy), and DL algorithms versus clinicians. Thirty-five studies are deemed eligible for systematic review, 20 of which are meta-analyzed, with a pooled sensitivity of 88% (95% CI 85–90%), specificity of 84% (79–87%), and AUC of 0.92 (0.90–0.94). Acceptable diagnostic performance with analogous DL algorithms was highlighted across all subgroups. Therefore, DL algorithms could be useful for detecting breast and cervical cancer using medical imaging, having equivalent performance to human clinicians. However, this tentative assertion is based on studies with relatively poor designs and reporting, which likely caused bias and overestimated algorithm performance. Evidence-based, standardized guidelines around study methods and reporting are required to improve the quality of DL research.
Collapse
|
6
|
The diagnostic performance of ultrasound computer-aided diagnosis system for distinguishing breast masses: a prospective multicenter study. Eur Radiol 2022; 32:4046-4055. [PMID: 35066633 DOI: 10.1007/s00330-021-08452-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2021] [Revised: 10/11/2021] [Accepted: 10/31/2021] [Indexed: 12/31/2022]
|
7
|
Wang XY, Cui LG, Feng J, Chen W. Artificial intelligence for breast ultrasound: An adjunct tool to reduce excessive lesion biopsy. Eur J Radiol 2021; 138:109624. [PMID: 33706046 DOI: 10.1016/j.ejrad.2021.109624] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Revised: 02/24/2021] [Accepted: 03/01/2021] [Indexed: 01/30/2023]
Abstract
PURPOSE To determine whether adding an artificial intelligence (AI) system to breast ultrasound (US) can reduce unnecessary biopsies. METHODS Conventional US and AI analyses were prospectively performed on 173 suspicious breast lesions before US-guided core needle biopsy or vacuum-assisted excision. Conventional US images were retrospectively reviewed according to the BI-RADS 2013 lexicon and categories. Two downgrading stratifications based on AI assessments were manually used to downgrade the BI-RADS category 4A lesions to category 3. Stratification A was used to downgrade if the assessments of both orthogonal sections of a lesion from AI were possibly benign. Stratification B was used to downgrade if the assessment of any of the orthogonal sections was possibly benign. The effects of AI-based diagnosis on lesions to reduce unnecessary biopsy were analyzed using histopathological results as reference standards. RESULTS Forty-three lesions diagnosed as BI-RADS category 4A by conventional US received AI-based hypothetical downgrading. While downgrading with stratification A, 14 biopsies were correctly avoided. The biopsy rate for BI-RADS category 4A lesions decreased from 100 % to 67.4 % (P < 0.001). While downgrading with stratification B, 27 biopsies could be avoided with two malignancies missed, and the biopsy rate would decrease to 37.2 % (P < 0.05, compared with conventional US and stratification A). CONCLUSION Adding an AI system to breast US could reduce unnecessary lesion biopsies. Downgrading stratification A was recommended for its lower misdiagnosis rate.
Collapse
Affiliation(s)
- Xin-Yi Wang
- Department of Ultrasound, Peking University Third Hospital, Beijing 10091, China; Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education/ Beijing), Breast Center, Peking University Cancer Hospital & Institute, Beijing 100142, China
| | - Li-Gang Cui
- Department of Ultrasound, Peking University Third Hospital, Beijing 10091, China.
| | - Jie Feng
- Department of Ultrasound, Peking University Third Hospital, Beijing 10091, China; Department of Ultrasound, Haicang Hospital, Xiamen 361000, China
| | - Wen Chen
- Department of Ultrasound, Peking University Third Hospital, Beijing 10091, China
| |
Collapse
|