1
|
Yao X, Tang M, Lu M, Zhou J, Yang D. Interpretable machine learning models for predicting skip metastasis in cN0 papillary thyroid cancer based on clinicopathological and elastography radiomics features. Front Oncol 2025; 14:1457660. [PMID: 39868368 PMCID: PMC11758178 DOI: 10.3389/fonc.2024.1457660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2024] [Accepted: 12/10/2024] [Indexed: 01/28/2025] Open
Abstract
Background Skip lymph node metastasis (SLNM) in papillary thyroid cancer (PTC) involves cancer cells bypassing central nodes to directly metastasize to lateral nodes, often undetected by standard preoperative ultrasonography. Although multiple models exist to identify SLNM, they are inadequate for clinically node-negative (cN0) patients, resulting in underestimated metastatic risks and compromised treatment effectiveness. Our study aims to develop and validate a machine learning (ML) model that combines elastography radiomics with clinicopathological data to predict pre-surgical SLNM risk in cN0 PTC patients with increased risk of lymph node metastasis (LNM), improving their treatment strategies. Methods Our study conducted a retrospective analysis of 485 newly diagnosed primary PTC patients, divided into training and external validation cohorts. Patients were categorized into SLNM and non-SLNM groups based on follow-up outcomes and postoperative pathology. We collected preoperative clinicopathological data and extracted, standardized radiomics features from elastography imaging to develop various ML models. These models were internally validated using radiomics and clinicopathological data, with the optimal model's feature importance analyzed through the Shapley Additive Explanations (SHAP) approach and subsequently externally validated. Results In our study of 485 patients, 67 (13.8%) exhibited SLNM. The extreme gradient boosting (XGBoost) model, integrating elastography radiomics with clinicopathological data, demonstrated superior performance in both internal and external validations. SHAP analysis identified five key determinants of SLNM: three radiomics features from elastography images, one clinical variable, and one pathological variable. Conclusion Our evaluation highlights the XGBoost model, which integrates elastography radiomics and clinicopathological data, as the most effective ML approach for the prediction of SLNM in cN0 PTC patients with increased risk of LNM. This innovative model significantly enhances the accuracy of risk assessments for SLNM, enabling personalized treatments that could reduce postoperative metastases in these patients.
Collapse
Affiliation(s)
- Xiaohua Yao
- Departments of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine &Health Sciences, Shanghai, China
| | - Mingming Tang
- Department of Endocrinology, Jiading District Central Hospital Affiliated Shanghai University of Medicine & Health Sciences, Shanghai, China
| | - Min Lu
- Departments of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine &Health Sciences, Shanghai, China
| | - Jie Zhou
- Departments of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine &Health Sciences, Shanghai, China
| | - Debin Yang
- Departments of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine &Health Sciences, Shanghai, China
| |
Collapse
|
2
|
Cui Y, Fu C, Si C, Li J, Kang Y, Huang Y, Cui K. Analysis and Comparison of the Malignant Thyroid Nodules Not Recommended for Biopsy in ACR TIRADS and AI TIRADS With a Large Sample of Surgical Series. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2023; 42:1225-1233. [PMID: 36394431 DOI: 10.1002/jum.16132] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 10/31/2022] [Accepted: 11/06/2022] [Indexed: 05/18/2023]
Abstract
OBJECTIVE To determine if the artificial intelligence-based Thyroid Imaging, Reporting and Data System (AI TIRADS) would perform better than the American College of Radiology (ACR) TIRADS in monitoring malignant thyroid nodules not recommended for biopsy using follow-up thresholds. METHODS A total of 3499 thyroid nodules with surgical histopathology and ultrasound features were retrospectively reviewed and categorized using ACR TIRADS and AI TIRADS. The recommendations for biopsy and follow-up divided nodules into three groups 1) fine needle aspiration (FNA), 2) follow-up ultrasound, and 3) no further evaluation. RESULTS Of the total 1608 malignant nodules in this study, 974 malignant nodules would not be biopsied in ACR TIRADS compared with 967 in AI TIRADS. While 60.0% (584/974) of these non-biopsied malignancies could be followed-up by ultrasound in ACR TIRADS and 62.8% (607/967) in AI TIRADS. For the malignancies of no further evaluation, 97.4% (380/390) were sized <10 mm in ACR TIRADS and 93.3% (336/360) in AI TIRADS. Compared with ACR TIRADS, AI TIRADS had lower unnecessary FNA rate and missing cancer rate (41.0% vs 47.8% and 22.8% vs 27.5%, P < .05, respectively) while having higher specificity and AUC as well as lower sensitivity (65.0% vs 57.9%, 0.895 vs 0.881, and 96.1% vs 97.8%, all P < .05). CONCLUSIONS Using the follow-up thresholds, more than half of the malignancies not being biopsied were monitored by ultrasound in both ACR TIRADS and AI TIRADS, and AI TIRADS had lower missing cancer rate. More than 90% of malignancies recommended for no further evaluation were <10 mm in diameter.
Collapse
Affiliation(s)
- Yiyang Cui
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Chao Fu
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Caifeng Si
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Jing Li
- Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Yaning Kang
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Yuanjing Huang
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Kefei Cui
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| |
Collapse
|
3
|
Mohan U, Sunny SP, Mendonca P, Kuriakose MA, Kannan S, Suresh A. Systematic Review and Meta-Analysis to Identify the Immunocytochemical Markers Effective in Delineating Benign from Malignant Thyroid Lesions in FNAC Samples. Endocr Pathol 2022; 33:243-256. [PMID: 35596875 DOI: 10.1007/s12022-022-09721-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/04/2022] [Indexed: 11/03/2022]
Abstract
Conventional cytology-based diagnosis for thyroid cancer is limited with more than 30-45% of nodules categorized as indeterminate, necessitating surgery for confirming or refuting the diagnosis. This systematic review and meta-analysis were aimed at identifying immunocytochemical markers effective in delineating benign from malignant thyroid lesions in fine needle aspiration cytology (FNAC) samples, thereby improving the accuracy of cytology diagnosis. A systematic review of relevant articles (2000-2021) from online databases was carried out and the search protocol registered in PROSPERO database (CRD42021229121). The quality of studies was assessed using QUADAS-2. Review Manager 5.4.1 from Cochrane collaboration and MetaDisc Version 1.4 was used to conduct the meta-analysis. Bias in the studies were visually analyzed using funnel plots, and statistical significance was evaluated by Egger's test. Systematic review identified 64 original articles, while meta-analysis in eligible articles (n = 41) identified a panel of 5 markers, Galectin-3, HBME-1, CK-19, CD-56, and TPO. Assessment of the diagnostic performance revealed that Gal-3 (sensitivity: 0.81; CI: 0.79-0.83; specificity: 0.84; CI: 0.82-0.85) and HBME-1 (sensitivity: 0.83; Cl: 0.81-0.86; specificity: 0.85; CI: 0.83-0.86) showed high accuracy in delineating benign from malignant thyroid nodules. Efficacy analysis in indeterminate nodules showed Gal-3 and HBME-1 have high specificity of 0.86 (CI 0.84-0.89) and 0.82 (CI 0.78-0.86), respectively, and low sensitivity of 0.76 (CI 0.72-0.80) and 0.75 (CI 0.70-0.80), respectively. Diagnostic odds ratio (DOR) of Galectin-3 and HBME-1 were 39.18 (CI 23.38-65.65) and 24.44 (CI 11.16-53.54), respectively. Significant publication bias was observed for the markers Galectin-3 and CK-19 (p < 0.05). The panel of 5 markers identified from this meta-analysis are high-confidence candidates that need to be validated in thyroid cytology to establish their efficacy and enable clinical applicability.
Collapse
Affiliation(s)
- Uma Mohan
- Department of Head and Neck Oncology, Mazumdar Shaw Medical Center, NH Health City, Bangalore, India
- Integrated Head and Neck Oncology Program, DSRG-5, Mazumdar Shaw Center for Translational Research, Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Sumsum P Sunny
- Department of Head and Neck Oncology, Mazumdar Shaw Medical Center, NH Health City, Bangalore, India
- Integrated Head and Neck Oncology Program, DSRG-5, Mazumdar Shaw Center for Translational Research, Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Pramila Mendonca
- Department of Head and Neck Oncology, Mazumdar Shaw Medical Center, NH Health City, Bangalore, India
- Integrated Head and Neck Oncology Program, DSRG-5, Mazumdar Shaw Center for Translational Research, Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Moni A Kuriakose
- Department of Head and Neck Oncology, Mazumdar Shaw Medical Center, NH Health City, Bangalore, India
- Integrated Head and Neck Oncology Program, DSRG-5, Mazumdar Shaw Center for Translational Research, Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Subramanian Kannan
- Department of Endocrinology, Mazumdar Shaw Medical Center, NH Health City, Bangalore, India
| | - Amritha Suresh
- Integrated Head and Neck Oncology Program, DSRG-5, Mazumdar Shaw Center for Translational Research, Mazumdar Shaw Medical Foundation, Bangalore, India.
| |
Collapse
|
4
|
Xu L, Zhou Y, Li Y, Lu B, Liu T. Reducing Unnecessary Biopsy of American College of Radiology Thyroid Imaging Reporting and Data System Category 4 Nodules. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2021; 40:227-236. [PMID: 32870530 DOI: 10.1002/jum.15391] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Revised: 05/20/2020] [Accepted: 06/01/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVES To assess the effectiveness of strain elastography (SE), Virtual Touch tissue imaging and quantification (VTIQ; Siemens Medical Solutions, Mountain View, CA) and their combination (SE + VTIQ) in distinguishing benign from malignant American College of Radiology Thyroid Imaging Reporting and Data System category 4 nodules (TR4) to reduce unnecessary biopsy. METHODS In this retrospective study, 985 thyroid nodules from 920 patients were initially enrolled and examined with conventional ultrasound, SE, and VTIQ. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of SE, VTIQ, and SE + VTIQ were calculated respectively. An area under the receiver operating characteristic curve analysis was applied to evaluate the diagnostic efficiency of SE, VTIQ, and SE + VTIQ in the differentiation of benign and malignant TR4 nodules. RESULTS A total of 864 patients with 922 TR4 nodules were enrolled ultimately, as 63 thyroid nodules from 56 patients were excluded because they did not meet the inclusion criteria of this study. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of SE, VTIQ, and SE + VTIQ were 72.4% versus 86.3% versus 82.1%, 77.7% versus 80.5% versus 83.5%, 70.6% versus 76.1% versus 80.2%, 73.4% versus 76.3% versus 83.5%, and 75.5% versus 79.7% versus 82.8%, respectively. The areas under the receiver operating characteristic curve for the diagnosis of TR4 nodules by SE, VTIQ, and SE + VTIQ were 0.751, 0.817, and 0.844. CONCLUSIONS In spite of a slight decrease in the sensitivity, the application of combining SE and VTIQ techniques can improve the specificity and accuracy of TR4 nodule diagnosis, which can help reduce unnecessary biopsies of TR4 nodules.
Collapse
Affiliation(s)
- Lei Xu
- Department of Ultrasound, Jinhua Hospital of Zhejiang University, Jinhua, China
| | - Yibo Zhou
- Department of Ultrasound, Jinhua Hospital of Zhejiang University, Jinhua, China
| | - Yingru Li
- Department of Ultrasound, Jinhua Hospital of Zhejiang University, Jinhua, China
| | - Bin Lu
- Department of Ultrasound, Jinhua Hospital of Zhejiang University, Jinhua, China
| | - Tian Liu
- Department of Ultrasound, Jinhua Hospital of Zhejiang University, Jinhua, China
| |
Collapse
|
5
|
Chen L, Shi YX, Liu YC, Zhan J, Diao XH, Chen Y, Zhan WW. The values of shear wave elastography in avoiding repeat fine-needle aspiration for thyroid nodules with nondiagnostic and undetermined cytology. Clin Endocrinol (Oxf) 2019; 91:201-208. [PMID: 31004514 DOI: 10.1111/cen.13992] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/24/2019] [Revised: 04/15/2019] [Accepted: 04/16/2019] [Indexed: 11/30/2022]
Abstract
OBJECTIVE To evaluate the value of shear wave elastography (SWE) in avoiding repeat fine-needle aspiration of thyroid nodules with nondiagnostic and undetermined cytology. METHODS A total of 232 thyroid nodules with nondiagnostic (n = 132) and undetermined (n = 100) cytology underwent ultrasound (US) and SWE, followed by repeat ultrasound (US)-guided fine-needle aspiration cytology (FNAC). The final diagnosis was based on cytological or pathological findings. The US and SWE characteristics of benign and malignant nodules were compared using the χ2 -test. The receiver operating characteristic (ROC) curves of the thyroid imaging reporting and data system (TI-RADS) categories from the US and the EMean and ESD from the SWE were graphed, and the areas under the curves (AUCs) were compared using a Z test. RESULTS There were significant differences between the benign and malignant nodules in terms of the echogenicity, shape, margin, calcification and TI-RADS categories (all P < 0.05). The differences were significant between the malignant and benign nodules for EMean [(34.57 ± 14.81) kPa vs. (19.18 ± 7.09) kPa] and ESD [(13.68 ± 13.01) kPa vs. (3.97 ± 2.58) kPa] (both P < 0.001). Though the difference in the AUCs of EMean (0.864) and ESD (0.876) was not significant (P = 0.745), they both had higher diagnostic performances in comparison with TI-RADS categories (0.762) (all P < 0.05). Moreover, ESD attained a sensitivity of 100% with a relatively higher specificity of 49.75% when its cut-off value was 3.3 kPa. CONCLUSIONS Shear wave elastography is a promising imaging method for reducing repeat FNAC for benign thyroid nodules with nondiagnostic and undetermined cytology when using ESD as an index.
Collapse
Affiliation(s)
- Lin Chen
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University Medical School, Shanghai, China
| | - Yi-Xin Shi
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
| | - Ying-Chun Liu
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
| | - Jia Zhan
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
| | - Xue-Hong Diao
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
| | - Yue Chen
- Department of Ultrasound, Huadong Hospital, Fudan University, Shanghai, China
| | - Wei-Wei Zhan
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University Medical School, Shanghai, China
| |
Collapse
|
6
|
Cunha GB, Marino LCI, Yamaya A, Kochi C, Monte O, Longui CA, Cury AN, Fleury EDFC. Elastography for the evaluation of thyroid nodules in pediatric patients. Radiol Bras 2019; 52:141-147. [PMID: 31210685 PMCID: PMC6561376 DOI: 10.1590/0100-3984.2018.0034] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Objective To evaluate the usefulness of elastography (using manual compression) as an
additional diagnostic tool for children and adolescents with thyroid
nodules. Materials and Methods This was a prospective study conducted between September 2012 and August 2013
at a hospital in Brazil. We performed elastography, ultrasound, and
fine-needle aspiration biopsy in 32 patients between 6 and 18 years of age
who had, in total, 38 thyroid nodules. Results The elastography findings correlated with the histopathological diagnosis in
78.5% of cases. In three patients, an unnecessary thyroidectomy could have
been avoided if the elastography results had been prioritized. Only one
malignant thyroid nodule was found to show high elasticity. Conclusion Our results suggest that high elasticity of a nodule on elastography is
associated with a low risk of thyroid cancer. If further confirmed in other
studies, elastography may prove useful as a complementary test for screening
thyroid nodules in children.
Collapse
Affiliation(s)
| | | | - André Yamaya
- Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, SP, Brazil
| | - Cristiane Kochi
- Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, SP, Brazil
| | - Osmar Monte
- Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, SP, Brazil
| | | | - Adriano Namo Cury
- Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, SP, Brazil
| | | |
Collapse
|
7
|
Sengul D, Sengul I, Van Slycke S. Risk stratification of the thyroid nodule with Bethesda indeterminate cytology, category III, IV, V on the one surgeon-performed US-guided fine-needle aspiration with 27-gauge needle, verified by histopathology of thyroidectomy: the additional value of one surgeon-performed elastography. Acta Chir Belg 2019; 119:38-46. [PMID: 30606092 DOI: 10.1080/00015458.2018.1551769] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
BACKGROUND The aim is to assess the value of strain elastography (SE) in differentiating likelihood of malignancy for the thyroid nodules, possessing the Bethesda Category III, IV, and V indeterminate cytology. METHODS The data was obtained by ultrasonography (US)-guided fine-needle aspiration (US-g-FNA) via 27-gauge needle, with the verification of indicated thyroidectomies in a retrospective analysis, from April 2010 to April 2014, by enrolling the documents of 262 consecutive patients, with 327 thyroid nodules, subjected to one-surgeon performed neck US, SE, and US-g-FNA with 27-G needle to rule out the malignancy. RESULTS 122 of 327 cases were Bethesda Category III, IV, and V with histopathologically benign, 110 (90.2%); PTC, 7 (5.7%); FTC, 4 (3.3%); HCC, 1 (0.8%). Tsukuba Elasticity Score (TES) 1, 2, 3, 4, and 5 were detected as 38 (31.1%), 8 (6.6%), 59 (48.4%), 4 (3.3%), and 13 (10.7%), respectively for the cases with the indeterminate cytology. No significant difference was detected between TES 4 and 5 and malign histopathology by McNemar test (p = .727) with a good level of concordance, the kappa coefficient, 0.737. CONCLUSION SE may be a useful tool in differentiating malign from benign thyroid nodules by selecting surgery adaptation even for Bethesda indeterminate cytology on FNAC.
Collapse
Affiliation(s)
- Demet Sengul
- Department of Pathology, Giresun University Faculty of Medicine, Giresun, Turkey
| | - Ilker Sengul
- Division of Endocrine Surgery, Giresun University Faculty of Medicine, Giresun, Turkey
- Department of General Surgery, Giresun University Faculty of Medicine, Giresun, Turkey
| | - Sam Van Slycke
- Department of General and Endocrine Surgery, OLV Ziekenhuis Aalst, Aalst, Belgium
| |
Collapse
|
8
|
Demet S, Ilker S. Effect of anatomo-topographic and sonographic Polarity of the thyroid nodules on the thyroid malignancy by evaluating its impression on the relationship between the Bethesda System, TBSRTC, Strain Elastography Score and the thyroid histopathology. SANAMED 2019. [DOI: 10.24125/sanamed.v14i1.325] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
Background: The goal is to evaluate the association between the topo-sonographic polarity of the thyroid nodules and the thyroid malignancy by analyzing its effect on The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC), score of Strain Elastography (SE) for thyroid as Tsukuba Elasticity Score (TES), and histopathologic assessment. Material and Methods: A preliminary single-center retrospective study was carried out by including the documents of 641 consecutive eligible patients, possessing 770 thyroid nodules which undergone neck ultrasonography (US), Doppler US, SE, and US-guided-fine needle aspiration (FNA) during April 2011 to April 2017. The stiffness had been measured by TES of SE. The ability of the prediction of the malignancy by the polarity of 770 thyroid nodules considering the association between; i) TBSRTC and histopathology and ii) TES and histopathology had been evaluated. Results: Of the 770 thyroid nodules evaluated, 408 (53.0%) were located at the superior pole (Pol 1) while 362 (47.0%) were at the inferior pole (Pol 0) with 0.9046 AUC and 0.8171 AUC for the association between TBSRTC and histopathology and 0.9280 AUC and 0.7888 AUC for the association between TES and histopathology, respectively. However, those difference were not significant for Pol 1, topographically. Conclusion: The topograghic and sonographic polarity of the thyroid nodules may not be useful for estimating the thyroid malignancy by using the association between TBSRTC and histopathology with TES and histopathology. However, the association with Pol 1, the superior thyroid pole, was stronger though the difference was not significant.
Collapse
|
9
|
Xu JM, Chen YJ, Dang YY, Chen M. Association Between Preoperative US, Elastography Features and Prognostic Factors of Papillary Thyroid Cancer With BRAF V600E Mutation. Front Endocrinol (Lausanne) 2019; 10:902. [PMID: 32038479 PMCID: PMC6987316 DOI: 10.3389/fendo.2019.00902] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/06/2019] [Accepted: 12/10/2019] [Indexed: 11/13/2022] Open
Abstract
Purpose: To investigate the value of US and elastography for predicting prognostic factors of papillary thyroid cancer (PTC) in the positive BRAFV600E Mutation group. Materials and Methods: A total of 116 BRAFV600E Mutation patients with PTCs were enrolled in this prospective study, who were preoperatively evaluated by US, US elasticity imaging (EI), and Virtual Touch tissue imaging (VTI) and Virtual Touch tissue quantification (VTQ) of acoustic radiation force impulse (ARFI) imaging. Multivariate logistic regression analysis was performed to assess 23 independent variables for predicting prognostic factors. Diagnostic performance was evaluated with receiver operating characteristic (ROC) curve analysis. Results: Forty-two (36.2%) of 116 PTC patients with BRAFV600E Mutation had central lymph node metastasis (LNM). Nine (7.8%) and fifty-six (48.3%) had lateral LNM and extra-thyroidal extension (ETE), respectively. In multivariate logistic regression analyses, rich internal flow [odds ratio [OR]: 6.66] was the best predictor for central LNM, followed by male sex (OR: 4.22), halo sign absence (OR: 2.78) (all P < 0.05). VTQ ratio (OR: 1.57) was the only predictor for lateral LNM (P = 0.02). Rich internal flow (OR: 6.33) was the strongest predictor for ETE, followed by male sex (OR: 3.29), halo sign absence (OR: 2.90), and VTQ ratio (OR: 1.63) (all P < 0.05). Conclusion: VTQ ratio on ARFI imaging, rich internal flow and halo sign absence on US are the predicting prognostic factors in PTC patients with BRAFV600E Mutation. The specificities were significantly increased by combining ARFI imaging and US features, which has a potential to avoid unnecessary therapeutic neck dissection in the high-risk PTC patients.
Collapse
Affiliation(s)
- Jun-Mei Xu
- Department of Medical Ultrasound, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Tong Ji University, Shanghai, China
- Jun-Mei Xu
| | - Yong-Jun Chen
- Department of Medical Ultrasound, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Tong Ji University, Shanghai, China
| | - Yuan-Yuan Dang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Tong Ji University, Shanghai, China
| | - Man Chen
- Department of Medical Ultrasound, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
- *Correspondence: Man Chen
| |
Collapse
|
10
|
Choi YJ, Baek JH, Shin JH, Shim WH, Kim SO, Lee WH, Song DE, Kim TY, Chung KW, Lee JH. Web-based thyroid imaging reporting and data system: Malignancy risk of atypia of undetermined significance or follicular lesion of undetermined significance thyroid nodules calculated by a combination of ultrasonography features and biopsy results. Head Neck 2018; 40:1917-1925. [PMID: 29756265 DOI: 10.1002/hed.25173] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2017] [Revised: 12/03/2017] [Accepted: 02/15/2018] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND The purpose of this study was to construct a web-based predictive model using ultrasound characteristics and subcategorized biopsy results for thyroid nodules of atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS) to stratify the risk of malignancy. METHODS Data included 672 thyroid nodules from 656 patients from a historical cohort. We analyzed ultrasound images of thyroid nodules and biopsy results according to nuclear atypia and architectural atypia. Multivariate logistic regression analysis was performed to predict whether nodules were diagnosed as malignant or benign. RESULTS The ultrasound features, including spiculated margin, marked hypoechogenicity, calcifications, biopsy results, and cytologic atypia, showed significant differences between groups. A 13-point risk scoring system was developed, and the area under the curve (AUC) of the receiver operating characteristic (ROC) curve of the development and validation sets were 0.837 and 0.830, respectively (http://www.gap.kr/thyroidnodule_b3.php). CONCLUSION We devised a web-based predictive model using the combined information of ultrasound characteristics and biopsy results for AUS/FLUS thyroid nodules to stratify the malignant risk.
Collapse
Affiliation(s)
- Young Jun Choi
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| | - Jung Hwan Baek
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| | - Jung Hee Shin
- Department of Radiology and Center for Imaging Science, Thyroid Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea
| | - Woo Hyun Shim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| | - Seon-Ok Kim
- Department of Clinical Epidemiology and Biostatistics, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| | - Won-Hong Lee
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| | - Dong Eun Song
- Department of Pathology, University of Ulsan College of Medicine, Asan Medical Center, Songpa-Gu, Seoul, Korea
| | - Tae Yong Kim
- Department of Endocrinology and Metabolism, University of Ulsan College of Medicine, Asan Medical Center, Songpa-Gu, Seoul, Korea
| | - Ki-Wook Chung
- Department of Surgery, University of Ulsan College of Medicine, Asan Medical Center, Songpa-Gu, Seoul, Korea
| | - Jeong Hyun Lee
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Songpa-Gu, Seoul, Korea
| |
Collapse
|
11
|
Deng XH, Tang LN, Liu SQ, Li XL, He YP, Xu HX. A Proposal to Stratify the Intermediate-Risk Thyroid Nodules According to the AACE/ACE/AME Guidelines with Ultrasound Features. Sci Rep 2017; 7:17901. [PMID: 29263433 PMCID: PMC5738367 DOI: 10.1038/s41598-017-18207-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Accepted: 12/05/2017] [Indexed: 11/09/2022] Open
Abstract
To propose a risk stratification system for intermediate-risk thyroid nodules (TNs) according to American Association of Clinical Endocrinologists, American College of Endocrinology and Associazione Medici Endocrinologi Medical (AACE/ACE/AME) Guideline with ultrasound (US) features. 1000 patients with 1000 nodules (902 benign nodules and 98 malignant nodules) were included. All the nodules were confirmed with either fine needle aspiration (FNA) cytology and follow-up or histology results after surgery. Univariate analysis and binary multivariate logic regression analysis were applied to analyze the possible risk US features associated with malignancy. Receiver operating characteristic curves (ROC) were drew and compared. Univariate analysis and binary multivariate logistic regression analysis showed that indeterminate hyper-echoic spot (OR = 4.544), slightly ill-defined margin (OR = 2.559), slight hyper-echogenicity (OR = 1.992) and no macro-calcification (OR = 1.921) were risk factors for the intermediate-risk thyroid nodules (TNs). A predicting model was established based on the 4 risk factors. The risk rates of malignancy were 5.7% (26/455) in Stage I, 11.0% (49/445) in Stage II, 23.1% (21/91) in Stage III, 33.3% (3/9) in Stage IV. In conclusion, for the intermediate-risk TNs, special attention should be paid to the TNs with indeterminate hyper-echoic spot, slightly ill-defined margin, slight hyper-echogenicity, or no macro-calcification. The probability of malignancy increased with the number of risk factors increasing.
Collapse
Affiliation(s)
- Xiao-Hong Deng
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Department of Medical Ultrasound, Fujian Cancer Hospital & Fujian Medical University Cancer Hospital, Fuzhou, 350014, China
| | - Li-Na Tang
- Department of Medical Ultrasound, Fujian Cancer Hospital & Fujian Medical University Cancer Hospital, Fuzhou, 350014, China
| | - Shui-Qing Liu
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Department of Medical Ultrasound, Changzhou First People's Hospital & The Third Affiliated Hospital of Suzhou University, Changzhou, 213003, China
| | - Xiao-Long Li
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Ya-Ping He
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Hui-Xiong Xu
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China. .,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China. .,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China.
| |
Collapse
|
12
|
Ha SM, Kim JK, Baek JH. Detection of Malignancy Among Suspicious Thyroid Nodules <1 cm on Ultrasound with Various Thyroid Image Reporting and Data Systems. Thyroid 2017; 27:1307-1315. [PMID: 28791920 DOI: 10.1089/thy.2017.0034] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
BACKGROUND In patients undergoing active surveillance of papillary thyroid microcarcinoma, definitive therapy-usually preceded by a definitive diagnostic procedure-is not recommended until evidence of disease progression is obtained, as stated in the American Thyroid Association guidelines. This is because the deferring of definitive diagnosis and therapy until disease progression has no impact on the disease-specific survival. This study evaluated the malignancy rate and probability of thyroid nodules, which was further stratified based on the size cutoff value of 1 cm, with suspicious findings on ultrasonography (US), by using various malignant stratification systems. METHODS The data were retrospectively collected between January 2003 and June 2003 from nine university hospitals that had previously participated in the Korean Society of Thyroid Radiology multicenter study on the ultrasonographic differentiation between benign and malignant thyroid nodules. In total, 829 thyroid nodules from 711 patients (620 women, 91 men; Mage = 48.7 years; range 6-98 years; 351 malignant and 478 benign nodules) were included. The probability for malignancy of thyroid nodules was calculated, which was further stratified by size, by using four different types of malignant risk-stratification systems. The factors that could differentiate benign from malignant nodules were assessed using the chi-square test. RESULTS In the suspicious thyroid nodules <1 cm on US, the malignancy probability ranged from 77.4% to 82.8%; the lowest rate was found in the Korean Society of Thyroid Radiology multicenter study, whereas the highest rate was noted in the Web-based system. Thus, the probability of benign nodules among suspicious thyroid nodules <1 cm on US was 17.2-22.6%. CONCLUSION A biopsy should be considered before active surveillance to exclude benign nodules with suspicious US features, and could thus prevent unnecessary active surveillance and patient anxiety.
Collapse
Affiliation(s)
- Su Min Ha
- 1 Department of Radiology and Thyroid Center, Chung-Ang University Hospital, Chung-Ang University College of Medicine , Seoul, Korea
| | - Jae Kyun Kim
- 1 Department of Radiology and Thyroid Center, Chung-Ang University Hospital, Chung-Ang University College of Medicine , Seoul, Korea
| | - Jung Hwan Baek
- 2 Department of Radiology and the Research Institute of Radiology, University of Ulsan College of Medicine , Asan Medical Center, Seoul, Korea
| |
Collapse
|
13
|
Wang Y, Lei KR, He YP, Li XL, Ren WW, Zhao CK, Bo XW, Wang D, Sun CY, Xu HX. Malignancy risk stratification of thyroid nodules: comparisons of four ultrasound Thyroid Imaging Reporting and Data Systems in surgically resected nodules. Sci Rep 2017; 7:11560. [PMID: 28912438 PMCID: PMC5599531 DOI: 10.1038/s41598-017-11863-0] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2017] [Accepted: 08/30/2017] [Indexed: 01/23/2023] Open
Abstract
To compare the efficiency of four different ultrasound (US) Thyroid Imaging Reporting and Data Systems (TI-RADS) in malignancy risk stratification in surgically resected thyroid nodules (TNs). The study included 547 benign TNs and 464 malignant TNs. US images of the TNs were retrospectively reviewed and categorized according to the TI-RADSs published by Horvath E et al. (TI-RADS H), Park et al. (TI-RADS P), Kwak et al. (TI-RADS K) and Russ et al. (TI-RADS R). The diagnostic performances for the four TI-RADSs were then compared. At multivariate analysis, among the suspicious US features, marked hypoechogenicity was the most significant independent predictor for malignancy (OR: 15.344, 95% CI: 5.313-44.313) (P < 0.05). Higher sensitivity was seen in TI-RADS H, TI-RADS K, TI-RADS R comparing with TI-RADS P (P < 0.05 for all), whereas the specificity, accuracy and area under the ROC curve (Az) of TI-RADS P were the highest (all P < 0.05). Higher specificity, accuracy and Az were seen in TI-RADS K compared with TI-RADS R (P = 0.003). With its higher sensitivity, TI-RADS K, a simple predictive model, is practical and convenient for the management of TNs in clinical practice. The study indicates that there is a good concordance between TI-RADS categories and histopathology.
Collapse
Affiliation(s)
- Ying Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Department of Medical Ultrasound, Yangpu Hospital, Tongji University School of Medicine, Shanghai, 200090, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Kai-Rong Lei
- Department of Medical Ultrasound, Yangpu Hospital, Tongji University School of Medicine, Shanghai, 200090, China
| | - Ya-Ping He
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Xiao-Long Li
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Wei-Wei Ren
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Chong-Ke Zhao
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Xiao-Wan Bo
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Dan Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Cheng-Yu Sun
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Department of Medical Ultrasound, Yangpu Hospital, Tongji University School of Medicine, Shanghai, 200090, China
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Hui-Xiong Xu
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.
- Shanghai Center for Thyroid Diseases, Shanghai, 200072, China.
| |
Collapse
|
14
|
He YP, Xu HX, Zhao CK, Sun LP, Li XL, Yue WW, Guo LH, Wang D, Ren WW, Wang Q, Qu S. Cytologically indeterminate thyroid nodules: increased diagnostic performance with combination of US TI-RADS and a new scoring system. Sci Rep 2017; 7:6906. [PMID: 28761147 PMCID: PMC5537223 DOI: 10.1038/s41598-017-07353-y] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2017] [Accepted: 06/27/2017] [Indexed: 01/10/2023] Open
Abstract
To investigate the diagnostic performance of combination of ultrasound (US) thyroid imaging reporting and data system (TI-RADS) and a new US scoring system for diagnosing thyroid nodules (TNs) with indeterminate results (Bethesda categories III, IV and V) on fine-needle aspiration (FNA) cytology. 453 patients with 453 cytologically indeterminate TNs were included in this study. Multivariate analyses were performed to construct the scoring system. The diagnostic performances of TI-RADS and the combined method were evaluated and compared. Multivariate analyses revealed that marked hypoechogenicity, taller than wide shape and absence of halo sign were independent predictors for malignancy in cytologically indeterminate TNs. Scoring system was thereafter defined as follows: risk score (RS) = 3.2 x (if marked hypoechogenicity) + 2.8 x (if taller than wide shape) + 1.3 x (if absence of halo sign). Compared with TI-RADS alone, the areas under the receiver operating characteristic curves (AUC), specificity, accuracy and positive predictive value (PPV) of the combined method increased significantly with 0.731 versus 0.569, 48.5% versus 14.1%, 76.2% versus 62.3%, and 70.9% versus 59.9%, respectively (all P < 0.05). The combination of TI-RADS and new US scoring system showed superior diagnostic performances in predicting malignant TNs with indeterminate FNA cytology results in comparison with TI-RADS alone.
Collapse
Affiliation(s)
- Ya-Ping He
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Hui-Xiong Xu
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China. .,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China. .,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China.
| | - Chong-Ke Zhao
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Li-Ping Sun
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Xiao-Long Li
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Wen-Wen Yue
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Le-Hang Guo
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Dan Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Wei-Wei Ren
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Qiao Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China
| | - Shen Qu
- Thyroid Institute, Tongji University School of Medicine, Shanghai, 200072, China.,Shanghai Center for Thyroid Diseases, Shanghai, 200072, China.,Department of Endocrinology & Metabolism, Shanghai Tenth People's Hospital, Tongji University School of Medicine, Shanghai, 200072, China
| |
Collapse
|