Validation of Three Scoring Risk-Stratification Models for Thyroid Nodules

Comparison of diagnostic performance of two ultrasound risk stratification systems for thyroid nodules: a systematic review and meta-analysis

OBJECTIVES

To assume the ideal cut-off values and diagnostic performance of two thyroid imaging reporting and data systems published by the Korean Thyroid Association/Korean Society of Thyroid Radiology (Korean TI-RADS) and the American Thyroid Association (ATA TI-RADS).

METHODS

Eighteen studies with 25,422 patients from PubMed, SCOPUS, Embase, Web of Science, and Cochrane Library databases up to August 2022. True and false positive and negative values with characteristics were extracted.

RESULTS

The highest area under the receiver operating characteristic curve (AUC) was 0.893 and 0.887 for Korean and ATA TI-RADS. High suspicion was judged as the best cut-off value with the highest AUC based on optimal sensitivity and specificity. In determining the risk of malignant thyroid nodules, high suspicion in Korean and ATA TI-RADS showed sensitivity as 71.3% and 73.5%, specificity as 7.9% and 86.4%, diagnostic odds ratios as 20.0289 and 20.9076, AUC as 0.893 and 0.887. There was no significant difference when directly comparing the diagnostic accuracy of both TI-RADS.

CONCLUSION

The two risk stratification systems had good diagnostic performance with high AUC and no significant differences. The ideal cut-off can depend on the medical condition or thyroid nodules, because the changes of cut-off point may reciprocally alter sensitivity and specificity.

Diagnostic performance of the EU TI-RADS and ACR TI-RADS scoring systems in predicting thyroid malignancy

INTRODUCTION

Several ultrasound scoring systems have been developed to stratify the risk of malignancy of thyroid nodules, including ACR (American College of Radiology) and EU (European) TI-RADS. This study aimed to assess the diagnostic performance of these two classifications using histology as a reference standard.

METHODS

It was a single-centre, retrospective study including 156 patients who underwent thyroidectomy. Ultrasound data of 198 nodules (99 malignant nodules and 99 benign nodules) were analysed. Both classifications were applied for all nodules.

RESULTS

Ultrasound criteria associated with malignancy were solid composition (OR=7.81; p < 10^-3 ), hypoechoic character (OR=16.42; p < 10^-3 ), irregular contours (OR=7.47; p < 10^-3 ), taller-than-wide shape (OR=3.58; p = 0.02), microcalcifications (OR=3.02; p = .006) and the presence of cervical adenopathy (OR=3.89; p = .006). The prevalence of malignancy was 15.5%, 69% and 76.9% for EU TI-RADS categories 3, 4 and 5, respectively. It was 33.3%, 57% and 91.1% for ACR TI-RADS categories 3, 4 and 5, respectively. For category 5, EU TI-RADS and ACR TI-RADS had sensitivities of 60% and 41%, specificities of 82% and 96%, respectively. For categories 4 and 5 combined, the diagnostic performance of these two classification systems became comparable with a sensitivity of 89% and 86% for EU-TIRADS and ACR-TIRADS, respectively. The area under the ROC curve was 0.81 for the EU TI-RADS classification and 0.82 for the ACR TI-RADS classification.

CONCLUSIONS

EU TI-RADS and ACR TI-RADS scoring systems seem to be comparable in predicting malignancy in thyroid nodules.

Diagnostic Performance of ACR and Kwak TI-RADS for Benign and Malignant Thyroid Nodules: An Update Systematic Review and Meta-Analysis

(1) Background: To determine the optimal cut-off values of two risk stratification systems to discriminate malignant thyroid nodules and to compare the diagnostic performance; (2) Methods: True and false positive and negative data were collected, and methodological quality was assessed for forty-six studies involving 39,085 patients; (3) Results: The highest area under the receiver operating characteristic (ROC) curve (AUC) of ACR and Kwak TI-RADS were 0.875 and 0.884. Based on the optimal sensitivity and specificity, the highest accuracy values of ROC curves or diagnostic odds ratios (DOR) were taken as the cut-off values for TR4 (moderate suspicious) and 4B. The sensitivity, specificity, DOR, and AUC by ACR (TR4) and Kwak TI-RADS (4B) for malignancy risk stratification of thyroid nodules were 94.3% and 96.4%; 52.2% and 53.7%; 17.5185 and 31.8051; 0.786 and 0.884, respectively. There were no significant differences in diagnostic accuracy in any of the direction comparisons of the two systems; (4) Conclusions: ACR and Kwak TI-RADS had good diagnostic performances (AUCs > 85%). Although we determined the best cut-off values in individual risk stratification systems based on statistical assessment, clinicians can adjust the optimal cut-off value according to the clinical purpose of the ultrasonography because raising or lowering cut-points leads to reciprocal changes in sensitivity and specificity.

Comparative Analysis of Computer-Aided Diagnosis and Computer-Assisted Subjective Assessment in Thyroid Ultrasound

The value of computer-aided diagnosis (CAD) and computer-assisted techniques equipped with different TIRADS remains ambiguous. Parallel diagnosis performances of computer-assisted subjective assessments and CAD were compared based on AACE, ATA, EU, and KSThR TIRADS. CAD software computed the diagnosis of 162 thyroid nodule sonograms. Two raters (R₁ and R₂) independently rated the sonographic features of the nodules using an online risk calculator while blinded to pathology results. Diagnostic efficiency measures were calculated based on the final pathology results. R₁ had higher diagnostic performance outcomes than CAD with similarities between KSThR (SEN: 90.3% vs. 83.9%, p = 0.57; SPEC: 46% vs. 51%, p = 0.21; AUROC: 0.76 vs. 0.67, p = 0.02), and EU (SEN: 85.5% vs. 79%, p = 0.82; SPEC: 62% vs. 55%, p = 0.27; AUROC: 0.74 vs. 0.67, p = 0.06). Similarly, R₂ had higher AUROC and specificity but lower sensitivity than CAD (KSThR-AUROC: 0.74 vs. 0.67, p = 0.13; SPEC: 61% vs. 46%, p = 0.02 and SEN: 75.8% vs. 83.9%, p = 0.31, and EU-AUROC: 0.69 vs. 0.67, p = 0.57, SPEC: 64% vs. 55%, p = 0.19, and SEN: 71% vs. 79%, p = 0.51, respectively). CAD had higher sensitivity but lower specificity than both R₁ and R₂ with AACE for 114 specified nodules (SEN: 92.5% vs. 88.7%, p = 0.50; 92.5% vs. 79.3%, p = 0.02, and SPEC: 26.2% vs. 54.1%, p = 0.001; 26.2% vs. 62.3%, p < 0.001, respectively). All diagnostic performance outcomes were comparable for ATA with 96 specified nodules. Computer-assisted subjective interpretation using KSThR is more ideal for ruling out papillary thyroid carcinomas than CAD. Future larger multi-center and multi-rater prospective studies with a diverse representation of thyroid cancers are necessary to validate these findings.

Comparison of British Thyroid Association, American College of Radiology TIRADS and Artificial Intelligence TIRADS with histological correlation: diagnostic performance for predicting thyroid malignancy and unnecessary fine needle aspiration rate

Objectives:

To compare diagnostic performance of British Thyroid Association (BTA), American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS) and Artificial Intelligence TIRADS (AI-TIRADS) for thyroid nodule malignancy. To determine comparative unnecessary fine needle aspiration (FNA) rates.

Methods:

218 thyroid nodules with definitive histology obtained during 2017 were included. Ultrasound images were reviewed retrospectively in consensus by two subspecialist radiologists, blinded to histopathology, and nodules assigned a BTA, ACR-TIRADS and AI-TIRADS grade. Nodule laterality and size were recorded to allow accurate histopathological correlation and determine which nodules met criteria for FNA.

Results:

77 (35.3%) nodules were malignant. Deeming ultrasound Grade 4–5 as test-positive and 1–2 as test-negative, sensitivity and specificity for BTA was 98.28 and 42.55%, for ACR-TIRADS: 95.24 and 40.57% and for AI-TIRADS: 93.44 and 45.71%. FNA was indicated in 101 (71.6%), 67 (47.5%) and 65 (46.1%) benign nodules utilising BTA, ACR-TIRADS and AI-TIRADS respectively. The unnecessary FNA rate was significantly higher with BTA (46.3%) compared to ACR-TIRADS (30.7%) and AI-TIRADS (29.8%) p < 0.001.

Conclusion:

BTA, ACR-TIRADS and AI-TIRADS had similar diagnostic performance for predicting thyroid nodule malignancy with sensitivity >93% for all systems when considering ultrasound Grade 4–5 as malignant and Grade 1–2 as benign. ACR-TIRADS and AI-TIRADS both had a significantly lower rate of recommended FNA in benign nodules compared to BTA.

Advances in knowledge:

BTA, ACR-TIRADS and AI-TIRADS have comparable diagnostic performance with high sensitivity but relatively low specificity for predicting thyroid nodule malignancy in this cohort using histology as gold-standard. Using Grade 1–2 as benign and 4–5 as malignant there were more false negatives with TIRADS but this improved when taking other features into account while BTA had a significantly higher rate of unnecessary FNA.

Thyroid imaging reporting and data system (TIRADS) for ultrasound features of nodules: multicentric retrospective study in China

PURPOSE

To establish a practical and simplified Chinese thyroid imaging reporting and data system (C-TIRADS) based on the Chinese patient database.

METHODS

A total of 2141 thyroid nodules that were neither cystic nor spongy were used in the current study. These specimens were derived from 2141 patients in 131 alliance hospitals of the Chinese Artificial Intelligence Alliance for Thyroid and Breast Ultrasound. The ultrasound features, including location, orientation, margin, halo, composition, echogenicity, echotexture, echogenic foci and posterior features were assessed. Univariate and multivariate analyses were performed to investigate the association between ultrasound features and malignancy. The regression equation, the weighting, and the counting methods were used to determine the malignant risk of the thyroid nodules. The areas under the receiver operating characteristic curve (Az values) were calculated.

RESULTS

Of the 2141 thyroid nodules, 1572 were benign, 565 were malignant, and 4 were borderline. Vertical orientation, ill-defined, or irregular margin (including extrathyroidal extension), microcalcifications, solid, and markedly hypoechoic were positively associated with malignancy, while comet-tail artifacts were negatively associated with malignancy. The logistic regression equation yielded the highest Az value of 0.913, which was significantly higher than that obtained using the weighting method (0.893) and the counting method (0.890); however, no significant difference was found between the latter two. The C-TIRADS, based on the counting method, was designed following the principle of balancing the diagnostic performance and sensitivity of the risk stratification with the ease of use.

CONCLUSIONS

A relatively simple C-TIRADS was established using the counting value of positive and negative ultrasound features.

Diagnostic Performance of Ultrasound Computer-Aided Diagnosis Software Compared with That of Radiologists with Different Levels of Expertise for Thyroid Malignancy: A Multicenter Prospective Study

The aim of the work described here was to evaluate the diagnostic performance of ultrasound thyroid computer-aided diagnosis (CAD) software. This multicenter prospective study included 494 patients (565 thyroid nodules) who underwent surgery or biopsy after ultrasonography at four hospitals from January 2019 to September 2019. The diagnostic performance metrics of different readers were calculated and compared with the pathologic results. The sensitivity of CAD was outstanding and was equivalent to that of a senior radiologist (90.51% vs. 88.47%, p > 0.05). The area under the curve of CAD was equivalent to that of a junior radiologist (0.748 vs. 0.739, p > 0.05). However, the specificity was only 49.63%, which was lower than those of the three radiologists (75.56%, 85.93% and 90.37% for the junior, intermediate and senior radiologists, respectively). The diagnostic performance of the junior radiologist was significantly improved with the aid of CAD (junior + CAD). The sensitivity and area under the curve of junior + CAD were improved from 72.20% to 89.93% and from 0.739 to 0.816, respectively (both p values <0.05), and the positive predictive value, negative predictive value and κ coefficient improved from 76.3% to 78.6%, 82.0% to 86.8% and 0.394 to 0.511, respectively. Though specificity slightly decreased from 75.56% to 73.33%, the difference was not statistically significant (p > 0.05). In general, the clinical application value of CAD is promising, and its instrumental value for junior radiologists is significant.

Explore the Diagnostic Efficiency of Chinese Thyroid Imaging Reporting and Data Systems by Comparing With the Other Four Systems (ACR TI-RADS, Kwak-TIRADS, KSThR-TIRADS, and EU-TIRADS): A Single-Center Study

PURPOSE

To explore the characteristics of C-TIRADS by comparing it with ACR-TIRADS, Kwak-TIRADS, KSThR-TIRADS and EU-TIRADS.

METHODS

A total of 1096 nodules were collected from 884 patients undergoing thyroidectomy in our center between May 2018 and December 2020. Divided the nodules into two groups: ">10mm" and "≤10mm". Ultrasound characteristics of each nodule were observed and recorded by 2 doctors, then classified based on ACR-TIRADS, Kwak-TIRADS, KSThR-TIRADS, EU-TIRADS, and C-TIRADS.

RESULTS

A total of 682 benign nodules cases (62.23%) and 414 malignant nodules cases (37.77%) were identified. The ICC value of each guideline was:0.937(ACR-TIRADS), 0.858(EU-IRADS), 0.811(Kwak-TIRADS), 0.835(KTA/KSThR-TIRADS) and 0.854(C-TIRADS). The nodule malignancy rates in the groups(Kwak-TIRADS 4B, C-TIRADS 4B、4C) of two sizes were significantly different (all p<0.05). There was no statistical difference in the other grades of two sizes (all p>0.05). Unnecessary biopsy rates were the lowest in C-TIRADS (49.02% p<0.001). Furthermore, Kwak-TIRADS had the highest sensitivity and NPV (89.9%, 91.0%, all p<0.05), while C-TIRADS had the highest specificity and PPV (82.3%, 69.2%, all p<0.05). C-TIRADS and Kwak-TIRADS had the highest accuracy (76.0%, 72.5%, P=0.071). The AUCs of the 5 guidelines were C-TIRADS(0.816, P<0.05), Kwak-TIRADS(0.789, P<0.05) KTA/KSThR-TIRADS and ACR-TIRADS(0.773, 0.763, P=0.305), EU-TIRADS(0.734, P<0.05). The AUCs of the five guidelines were not statistically different between "nodules>10mm" and "nodules ≤ 10mm" (all P>0.05).

CONCLUSIONS

All five guides showed excellent interobserver agreement. C-TIRADS was slightly efficient than Kwak-IRADS, KTA/KSThR-TIRADS and ACR-TIRADS, and had greater advantages than EU-TIRADS. The diagnostic abilities of the five guidelines for "nodules ≤ 10mm" were not inferior to that of "nodules> 10mm". C-TIRADS is simple and easy to implement and can provide effective thyroid tumor risk stratification for thyroid nodule diagnosis, especially in China.

Development and validation of a Web-based malignancy risk-stratification system of thyroid nodules

OBJECTIVES

Previous publications on risk-stratification systems for malignant thyroid nodules were based on conventional ultrasound only. We aimed to develop a practical and simplified prediction model for categorizing the malignancy risk of thyroid nodules based on clinical data, biochemical data, conventional ultrasound and real-time elastography.

DESIGN

Retrospective cohort study.

PATIENTS

A total of 2818 patients (1890 female, mean age, 45.5 ± 13.2 years) with 2850 thyroid nodules were retrospectively evaluated between April 2011 and October 2016. 26.8% nodules were malignant.

MEASUREMENTS

We used a randomly divided sample of 80% of the nodules to perform a multivariate logistic regression analysis. Cut-points were determined to create a risk-stratification scoring system. Patients were classified as having low, moderate and high probability of malignancy according to their scores. We validated the models to the remaining 20% of the nodules. The area under the curve (AUC) was used to evaluate the discrimination ability of the systems.

RESULTS

Ten variables were selected as predictors of malignancy. The point-based scoring systems with and without elasticity score achieved similar AUCs of 0.916 (95% confidence interval [CI]: 0.885-0.948) and 0.906 (95% CI: 0.872-0.941) when validated. Malignancy risk was segmented from 0% to 100.0% and was positively associated with an increase in risk scores. We then developed a Web-based risk-stratification system of thyroid nodules (http: thynodscore.com).

CONCLUSION

A simple and reliable Web-based risk-stratification system could be practically used in stratifying the risk of malignancy in thyroid nodules.

Ultrasound grading of thyroid nodules using the BTA U-scoring guidelines - Is there evidence of intra-and interobserver variability?

Introduction

U-score ultrasound classification (graded U1-U5) is widely used to grade thyroid nodules based on benign and malignant sonographic features. It is well established that ultrasound is an operator-dependent imaging modality and thus more susceptible to subjective variances between operators when using imaging-based scoring systems. We aimed to assess whether there is any intra- or interobserver variability when U-scoring thyroid nodules and whether previous thyroid ultrasound experience has an effect on this variability.

Methods

A total of 14 ultrasound operators were identified (five experienced thyroid operators, five with intermediate experience and four with no experience) and were asked to U-score images from 20 thyroid cases shown as a single projection, with and without Doppler flow. The cases were subsequently rescored by the 14 operators after six weeks. The first and second round U-scores for the three operator groups were then analysed using Fleiss' kappa to assess interobserver variability and Cochran's Q test to determine any intraobserver variability.

Results

We found no significant interobserver variability on combined assessment of all operators with fair agreement in round 1 (Fleiss' kappa = 0.30, p <0.0001) and slight agreement in round 2 (Fleiss' kappa = 0.19, p < 0.0001). Cochran's Q test revealed no significant intraobserver variability in all 14 operators between round 1 and round 2 (all p>0.05).

Conclusions

We found no statistically significant inter- or intraobserver variability in the U-scoring of thyroid nodules between all participants reinforcing the validity of this scoring method in clinical practice, allaying concerns regarding potential subjective biases in reporting.

Comparison of Diagnostic Performance of Five Different Ultrasound TI-RADS Classification Guidelines for Thyroid Nodules

Objectives

We aimed to evaluate and compare the diagnostic performance of five ultrasound thyroid imaging reporting and data system (TI-RADS) classification guidelines for thyroid nodules through a review and meta-analysis.

Methods

We searched for relevant studies before February 2020 in PubMed. Then we pooled the sensitivity, specificity, likelihood ratios, diagnostic odds ratios, and area under the summary receiver operating characteristic curves. And the diagnostic odds ratios were used to compare the performance.

Results

We totally included 19 studies with 4,696 lesions in this research. The pooled sensitivity of American College of Radiology (ACR) guidelines, American Thyroid Association (ATA) guidelines, TI-RADS proposed by Kwak (Kwak TI-RADS), Korean Thyroid Association/Korean Society of Thyroid Radiology (KTA/KSThR) guidelines for malignancy risk and European Thyroid Association (ETA) guidelines is between 0.84 and 0.94. The pooled specificity is 0.68, 0.44, 0.62, 0.47, and 0.61, respectively. And the RDOR is 1.57 (ACR vs ATA), 1.37 (ACR vs ETA), 1.80 (ACR vs Kawk), 1.74 (ARC vs KTA).

Conclusions

The results suggest that five classification guidelines are all effective methods for differential diagnosis of benign and malignant thyroid nodules and ACR guideline is a better choice.

A correlation study between thyroid imaging report and data systems and the Bethesda system for reporting thyroid cytology with surgical follow-up - an ultrasound-trained cytopathologist's experience

INTRODUCTION

Thyroid Imaging Reporting and Data Systems (TI-RADS) including a modified system (M TI-RADS) and American College Radiology (ACR) TI-RADS are introduced. Correlation studies between both TI-RADSs and The Bethesda System for Reporting Thyroid Cytology (TBSRTC) for Ultrasound-guided fine-needle aspiration (US-FNA) cases with surgical follow-up (SFU) are presented.

MATERIALS AND METHODS

Three hundred and forty six thyroid US-FNA cases performed at the US-FNA clinic of Stony Brook University Hospital by an Ultrasound-trained interventional Cytopathologist between December 1, 2014 to February 28, 2018 were reviewed. All ultrasound images from 346 cases were evaluated and assigned M TI-RADS scores based on ultrasound features present in Table 1. One hundred and fifty four cases from Dcember 1, 2014 to February 28, 2016 were assigned scores using ACR TI-RADS.TBSRTC category along with SFU data was collected and correlated with both TI-RADS scores by using Pearson's correlation coefficient (r). Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy for M TI-RADS and TBSRTC were also calculated using SFU diagnosis as a gold standard. McNemar test was applied to evaluate the significance of the difference between M TI-RADS and TBSRTC.

RESULTS

Based on 346 thyroid nodules studied, the overall r-value between M TI-RADS scores and TBSRTC categories is 0.465. The r-values between Composition, Echogenicity, Shape, Margin, Echogenic foci, the summary points, and the converted TR level of ACR TI-RADS and TBSRTC are 0.145, 0.042, 0.259, 0.310, 0.405, and 0.379, respectively. Fifty (14.5%) cases have SFUs. Considering TI-RADS 5&6 and TBSRTC 5&6 as positives, and TI-RADS 2 and TBSRTC 2 as negatives, the Sensitivity, Specificity, PPV, NPV, and accuracy are 96%, 53%, 76%, 89%, 79% for TI-RADS vs 100%, 93%, 96%, 100%, 97% for TBSRTC, respectively (P = .038). The data for indeterminate cases are summarized.

CONCLUSION

Both TI-RADSs could be helpful to a moderate degree. M TI-RADS appeared to correlate to TBSRTC slightly better than that of ACR TI-RADS. TBSRTC is significantly more accurate than M TI-RADS for the majority of determined cases.

2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: the C-TIRADS

Thyroid nodules are very common all over the world, and China is no exception. Ultrasound plays an important role in determining the risk stratification of thyroid nodules, which is critical for clinical management of thyroid nodules. For the past few years, many versions of TIRADS (Thyroid Imaging Reporting and Data System) have been put forward by several institutions with the aim to identify whether nodules require fine-needle biopsy or ultrasound follow-up. However, no version of TIRADS has been widely adopted worldwide till date. In China, as many as ten versions of TIRADS have been used in different hospitals nationwide, causing a lot of confusion. With the support of the Superficial Organ and Vascular Ultrasound Group of the Society of Ultrasound in Medicine of the Chinese Medical Association, the Chinese-TIRADS that is in line with China's national conditions and medical status was established based on literature review, expert consensus, and multicenter data provided by the Chinese Artificial Intelligence Alliance for Thyroid and Breast Ultrasound.

Diagnostic Performance of Four Ultrasound Risk Stratification Systems: A Systematic Review and Meta-Analysis

Background: Several ultrasound (US)-based risk stratification systems have been increasingly used for the optimal management of thyroid nodules. However, there are considerable discrepancies across these systems. This study aimed to summarize and compare the category-based diagnostic performance in the detection of thyroid cancer of different US-based risk stratification systems from four societies: the American College of Radiology-Thyroid Imaging Reporting and Data System (ACR-TIRADS), the American Thyroid Association (ATA), the Korean Thyroid Association/Korean Society of Thyroid Radiology (KTA/KSThR; K-TIRADS), and the European Thyroid Association (EU-TIRADS). Methods: MEDLINE/PubMed and EMBASE databases were searched to identify original articles investigating the category-based diagnostic performance according to at least one of the following guidelines: ACR-TIRADS, ATA, K-TIRADS, and EU-TIRADS. Pooled sensitivity and specificity were calculated using a bivariate random-effects model. A subgroup analysis on nodules of 1 cm or larger and a meta-regression analysis to identify factors associated with the diagnostic performance were performed. Results: A total of 29 articles including 33,748 thyroid nodules met the eligibility criteria and were included in the analysis. For ACR-TIRADS, the pooled sensitivity and specificity were, respectively, 66% and 91% for category 5 and 95% and 55% for category 4 or 5. For ATA, the pooled sensitivity and specificity were, respectively, 74% and 88% for category 5 and 91% and 64% for category 4 or 5. For K-TIRADS, the pooled sensitivity and specificity were, respectively, 55% and 95% for category 5 and 89% and 64% for category 4 or 5. For EU-TIRADS, the pooled sensitivity and specificity were, respectively, 82% and 90% for category 5 and 96% and 52% for category 4 or 5. Study location, proportion of female patients and malignant nodules, and study design were associated with study heterogeneity. Conclusions: The overall diagnostic performance of the four US-based risk stratification systems was comparable.

Accuracy of thyroid imaging reporting and data system category 4 or 5 for diagnosing malignancy: a systematic review and meta-analysis

OBJECTIVES

To determine the accuracies of the American College of Radiology (ACR)-thyroid imaging reporting and data systems (TIRADS), Korean (K)-TIRADS, and European (EU)-TIRADS for diagnosing malignancy in thyroid nodules.

METHODS

Original studies reporting the diagnostic accuracy of TIRADS for determining malignancy on ultrasound were identified in MEDLINE and EMBASE up to June 23, 2019. The meta-analytic summary sensitivity and specificity were obtained for TIRADS category 5 (TR-5) and category 4 or 5 (TR-4/5), using a bivariate random effects model. To explore study heterogeneity, meta-regression analyses were performed.

RESULTS

Of the 34 eligible articles (37,585 nodules), 25 used ACR-TIRADS, 12 used K-TIRADS, and seven used EU-TIRADS. For TR-5, the meta-analytic sensitivity was highest for EU-TIRADS (78% [95% confidence interval, 64-88%]), followed by ACR-TIRADS (70% [61-79%]) and K-TIRADS (64% [58-70%]), although the differences were not significant. K-TIRADS showed the highest meta-analytic specificity (93% [91-95%]), which was similar to ACR-TIRADS (89% [85-92%]) and EU-TIRADS (89% [77-95%]). For TR-4/5, all three TIRADS systems had sensitivities higher than 90%. K-TIRADS had the highest specificity (61% [50-72%]), followed by ACR-TIRADS (49% [43-56%]) and EU-TIRADS (48% [35-62%]), although the differences were not significant. Considerable threshold effects were noted with ACR- and K-TIRADS (p ≤ 0.01), with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity (p ≤ 0.05).

CONCLUSIONS

There was no significant difference among these three international TIRADS, but the trend toward higher sensitivity with EU-TIRADS and higher specificity with K-TIRADS.

KEY POINTS

• For TIRADS category 5, the meta-analytic sensitivity was highest for the EU-TIRADS, followed by the ACR-TIRADS and the K-TIRADS, although the differences were not significant. • For TIRADS category 5, K-TIRADS showed the highest meta-analytic specificity, which was similar to ACR-TIRADS and EU-TIRADS. • Considerable threshold effects were noted with ACR- and K-TIRADS, with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity.

Interobserver agreement and efficacy of consensus reading in Kwak-, EU-, and ACR-thyroid imaging recording and data systems and ATA guidelines for the ultrasound risk stratification of thyroid nodules

PURPOSE

To investigate the interobserver agreement (IA) and the impact of consensus reading using four risk stratification systems for thyroid nodules (TN).

METHODS

Four experienced specialists independently rated US images of 80 TN according to the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines. The cases were randomly extracted from a prospectively acquired database (n > 1500 TN). The observers were blinded to clinical data. This study was divided into two sessions (S1 and S2) with 40 image sets each. After every session, a consensus reading was carried out (C1, C2). Subsequently, the effect of C1 was tested in S2 with 40 new cases followed by C2. Fleiss' kappa (κ) was calculated for S1 and S2 to estimate the IA and learning curves. The results of C1 and C2 were used as reference for diagnostic accuracy calculations.

RESULTS

IA significantly increased (p < 0.01) after C1 with κ values of 0.375 (0.615), 0.411 (0.596), 0.321 (0.569), and 0.410 (0.583) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines in S1 (S2), respectively. ROC analysis (C1 + C2) revealed similar areas under the curve (AUC) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines (0.635, 0.675, 0.694, and 0.654, respectively, n.s.). AUC did not increase from C1 (0.677 ± 0.010) to C2 (0.632 ± 0.052, n.s.). ATA Guidelines were not applicable in five cases.

CONCLUSIONS

IA and diagnostic accuracy were very similar for the four investigated risk stratification systems. Consensus reading sessions significantly improved the IA but did not affect the diagnostic accuracy.

Nodules with nonspecific ultrasound pattern according to the 2015 American Thyroid Association malignancy risk stratification system: A comparison to the Thyroid Imaging Reporting and Data System (TIRADS-Na)

New sonographic patterns have been recommended by the 2015 American Thyroid Association (ATA) to stratify nodules in terms of malignancy risk and help guide biopsy decision. This study aimed to compare the ultrasound part of the ATA guidelines and the Thyroid Imaging Reporting and Data System (TIRADS-Na).In 2013 to 2016, 708 thyroid nodules in 505 patients were confirmed by postoperative histopathology. Hypoechogenicity, solidity, microcalcification, irregular margin, and a taller-than-wide shape were considered features suggesting malignancy. Sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) were obtained for the TIRADS and ATA guidelines.Of the 708 nodules, 341(48.2%) and 367(51.8%) were benign and malignant, respectively. Based on the ultrasound 2015 ATA guidelines, 62 nodules had nonspecific pattern (both malignant and benign features); malignancy rates of nodules with very low, low, intermediate, and high suspicion, and nonspecific pattern were 0, 17.7%, 57.9%, 90.0%, and 69.4%, respectively (P < .001). Malignancy rates of categories 2/3/4/5 nodules by TIRADS were 0, 8.1%, 67.0%, and 90.1%, respectively (P < .001). Based on pathological results, the AUC, sensitivity, specificity, NPV, and PPV were 0.926, 96.7%, 81.5%, 84.9%, and 95.9% for TIRADS, and 0.920, 93.5%, 82.4%, 85.1%, and 92.1% for ATA patterns, respectively. The TIRADS was generally more efficient than the 2015 ATA guidelines, especially for nodules >2 cm in diameter or those with nonspecific pattern.The TIRADS show a relative superiority over the ultrasound 2015 ATA guidelines, especially for nodules with >2 cm diameter or nonspecific pattern.

Ultrasound imaging classifications of thyroid nodules for malignancy risk stratification and clinical management: state of the art

Assessing the risk of malignancy in the thyroid with ultrasound (US) is crucial in patients with nodules, as it can aid in selecting those who should have a fine-needle aspiration (FNA) biopsy performed. Many studies have examined whether the US characteristics of thyroid nodules are useful indicators of histological malignancy. Overall, these investigations have identified a few US features that are significantly more frequent in malignant thyroid nodules which can be coalesced into a defining set to be used as an indicator of a higher risk of malignancy. Despite these efforts, none of these classifications have been widely adopted worldwide, and there are still conflicting recommendations from different institutions. Understanding the role and appropriate utilization of these systems could facilitate the effective interpretation and communication of thyroid US findings among referring physicians and radiologists. In this comprehensive review, we outline the major US classification systems of thyroid nodules published in the last few years.

Diagnostic accuracy of different computer-aided diagnostic systems for malignant and benign thyroid nodules classification in ultrasound images: A systematic review and meta-analysis protocol

OBJECTIVE

The aim of this study was to determine the diagnostic accuracy of different computer-aided diagnostic (CAD) systems for thyroid nodules classification.

METHODS

A systematic search of the literature was conducted from inception until March, 2019 using the PubMed, EMBASE, Web of science, and Cochrane library. Literature selection and data extraction were conducted by 2 independent reviewers. Numerical values for sensitivity and specificity were obtained from false negative (FN), false positive (FP), true negative (TN), and true positive (TP) rates, presented alongside graphical representations with boxes marking the values and horizontal lines showing the confidence intervals (CIs). Summary receiver operating characteristic (SROC) curves were applied to assess the performance of diagnostic tests. Data were processed using Review Manager 5.3 and Stata 15. The methodological quality of included studies was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.

TRIAL REGISTRATION NUMBER

PROSPERO CRD42019132540.

Validation and comparison of three newly-released Thyroid Imaging Reporting and Data Systems for cancer risk determination

PURPOSE

To validate and compare diagnostic value of three newly-released Thyroid Imaging Reporting and Data Systems (TIRADS) for cancer risk determination.

METHODS

Total 2031 patients with 2465 thyroid nodules were recruited for this study. Ultrasound (US) images were categorized based on three TIRADS editions established by Korean Society of Thyroid Radiology (KSThR), European Thyroid Association (ETA) and American College of Radiology (ACR). ROC curves were established to compare diagnostic value.

RESULTS

Total 1460 benign and 1005 malignant nodules were enrolled. The malignancy rates of each category in KSThR-TIRADS were 2.8%, 5.1%, 33.7% and 79.6%, respectively. For European-TIRADS, 0, 3.1, 22.8, and 73.5% of nodules categorized as 2 to 5 were malignant. Distribution of carcinomas among ACR-TIRADS categories was 0%, 2.3%, 7.5%, 40.1% and 81.4%, respectively. In terms of diagnostic value, KSThR-TIRADS had highest AUC (0.855) and specificity (87.4%), while lowest (71.4%) sensitivity. ACR-TIRADS showed best sensitivity (96.6%) with lowest specificity (52.9%) and the AUC (0.846) was slightly lower than KSThR-TIRADS. Total 56.1, 45.4, and 37.4% fine-needle aspiration biopsy (FNAB) were recommended by KSThR, ETA and ACR, revealing 42.8%, 44.5% and 53.6% malignant lesions, respectively. The rate of unnecessary FNAB was lowest with the ACR (17.3%), followed by ETA (25.2%) and KSThR (32.1%).

CONCLUSION

All these US models showed great value in predicting thyroid malignancy. Among them, KSThR-TIRADS showed the most effective diagnostic performance in specificity, while ACR-TIRADS yielded best sensitivity. As for FNAB criteria, ACR-TIRADS showed the lowest rate of unnecessary FNAB and highest rate of malignancy in FNAB.

Fine needle aspiration biopsy indications for thyroid nodules: compare a point-based risk stratification system with a pattern-based risk stratification system

OBJECTIVES

We aim to compare the diagnostic performance to assess thyroid nodules and reliability for recommending fine needle aspiration biopsy (FNAB) between American College of Radiology thyroid image reporting and data system (ACR TI-RADS) and American Thyroid Association (ATA) guidelines.

METHODS

In total, this retrospective study included 1001 consecutive thyroid nodules in 918 patients from May 2016 to December 2017. US features of the thyroid nodules, including composition, echogenicity, shape, margins, echogenic foci, and size, were reviewed and were classified according to ACR TI-RADS and ATA guidelines, respectively. The diagnostic performance to assess thyroid nodules and reliability for recommending fine needle aspiration biopsy were compared between ACR TI-RADS and ATA guidelines.

RESULTS

Of the 1001 thyroid nodules, 609 (60.8%) were benign and 392 (39.2%) were malignant. The sensitivity, specificity, PPV, NPV, and accuracy were 96.7%, 77.3%, 73.3%, 97.3%, and 84.9%, respectively, for ACR TI-RADS and 99.2%, 16.1%, 43.2%, 97.0%, and 48.7%, respectively, for ATA guidelines. AUC of ACR TI-RADS was significantly greater than ATA guidelines (0.935 (0.918, 0.949) vs 0.884 (0.862, 0.903), p < 0.001). Biopsy yield of malignancy, biopsy rate of malignancy, and unnecessary FNAB rate were 59.5%, 91.3%, and 40.5%, respectively, for ACR TI-RDS and 38.5%, 97.4%, and 61.5%, respectively, for ATA guidelines.

CONCLUSIONS

ACR TI-RADS was more accurate than ATA guidelines for differentiating malignant thyroid nodules from benign nodules and more reliable than ATA guidelines for recommending thyroid nodules for FNAB.

KEY POINTS

• Malignant risk of thyroid nodules can be stratified by ultrasound. • American College of Radiology guidelines were more accurate for differentiating malignant thyroid nodules from benign nodules. • American College of Radiology guidelines were more reliable for recommending thyroid nodules for biopsy.

Assessment of the American College of Radiology Thyroid Imaging Reporting and Data System for Thyroid Nodule Malignancy Risk Stratification in a Pediatric Population

OBJECTIVE

The purpose of this study is to assess the diagnostic performance of the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TIRADS) for malignancy risk in pediatric thyroid nodules.

MATERIALS AND METHODS

Two radiologists reviewed ultrasound images of 74 tissue-proven thyroid nodules in 62 children. Points were given for individual features and then added to determine the ACR TI-RADS category, ranging from 1 (benign) to 5 (high suspicion). Kappa coefficients were generated to assess intra- and interobserver agreement. Generalized linear mixed-effects models were used to estimate the odds of malignancy with construction of a supplementary ROC curve.

RESULTS

Fifty-four nodules were benign and 20 were malignant, with a median ACR TI-RADS category of 4 (interquartile range, 4-5). Nineteen of 20 (95.0%) malignant nodules were rated as TI-RADS category 4 or 5. There was substantial intraobserver agreement (κ = 0.69-0.77; p < 0.001) and moderate interobserver agreement (κ = 0.37; p = 0.002) for TIRADS category. Univariable analysis showed that, with every 1-unit increase of TI-RADS category, the likelihood of malignancy increased 2.63 times (95% CI, 1.08-6.41; p = 0.03). After adjusting for nodule size, TI-RADS category remained marginally associated with malignancy (adjusted odds ratio, 2.27; 95% CI, 0.93-5.54; p = 0.07). The AUC was 0.75 (95% CI, 0.64-0.86). An optimal cut point of TI-RADS category 5 was selected, with TI-RADS category 5 nodules 10.44 times (95% CI, 2.71-40.21; p < 0.0001) more likely than categories 1-4 nodules to be malignant.

CONCLUSION

ACR TI-RADS discriminates well between malignant and benign nodules in a pediatric population, particularly at TI-RADS category 5.

Malignancy risk of initially benign thyroid nodules: validation with various Thyroid Imaging Reporting and Data System guidelines

OBJECTIVE

Some authors have found little or no diagnostic benefit from repeated biopsy of benign thyroid nodules. However, to our knowledge, integration of Thyroid Imaging Reporting and Data System (TIRADS) guidelines with one biopsy for sufficient benign thyroid nodule diagnosis has not been previously described. We investigated malignancy rate and probability by using various malignancy stratification systems in initially biopsy-proven benign nodules and sought to determine their clinical relevance in management of benign thyroid nodules.

METHODS

This retrospective study collected 6762 thyroid nodules from 6493 consecutive patients who underwent biopsy between January 2013 and December 2013. The initial biopsy with ≥1 year of follow-up was used as the gold standard for benign diagnosis. For our study purpose, we analyzed 2747 (57.0%, 2747 of 4822, 532 women, 2111 men; 229 malignant and 2518 benign) thyroid nodules diagnosed by initial biopsy with 28.2 ± 9.1 (range, 12-41) months of follow-up. We calculated the malignancy probability of thyroid nodules by using various malignancy risk stratification systems.

RESULTS

The overall calculated thyroid malignancy rate was 8.3% (229 of 2747). Initially biopsy-proven benign nodules exhibited a ≤3.0% malignancy probability when assessed as "low suspicion" by Korean-TIRADS (K-TIRADS), "low suspicion" by the ATA guideline, and "very probably benign" by the French TIRADS guideline and gave a score of ≤3 by the web-based TIRADS.

CONCLUSION

When initially biopsy-proven benign nodules exhibit a "low suspicion" US pattern and low malignancy probability, as stratified by various TIRADS guidelines, imaging surveillance instead of second biopsy is warranted.

KEY POINTS

• One biopsy is sufficient for initially biopsy-proven benign nodules. • Repetitive biopsy is necessary for imaging-pathology mismatched nodules. • Scoring risk stratification permits personalized management.

Combination of Maximum Shear Wave Elasticity Modulus and TIRADS Improves the Diagnostic Specificity in Characterizing Thyroid Nodules: A Retrospective Study

OBJECTIVES

The present study is aimed at evaluating the diagnostic value of combining shear wave elastography (SWE) parameters and the thyroid imaging reporting and data system (TIRADS) for differentiating between benign and malignant thyroid nodules.

METHODS

Patients who underwent conventional ultrasonography (US) and SWE before surgery were enrolled in the current study. Each nodule was given a TIRADS risk score. The effectiveness of the SWE parameters was assessed by odds ratios (ORs). The SWE scoring risk stratification was proposed beyond 95% probability, and the desired values were obtained according to the log-normal distribution. The area under the receiver-operating characteristic (AUC) was used to compare the diagnostic performance between TIRADS-alone and TIRADS + SWE.

RESULTS

A total of 262 patients with 298 thyroid nodules were enrolled in our study. The pathological analyses were conducted on 121 benign and 177 malignant nodules. The AUC values for TIRADS-alone and TIRADS + SWE were 0.896 (accuracy 83.2%) and 0.917 (accuracy 84.2%), respectively. However, the TIRADS + SWE scores showed a higher specificity (88.4%) and positive predictive value (91.2%) as compared with the TIRADS-alone of 73.6% and 83.2%, respectively.

CONCLUSIONS

Combining SWE and TIRADS improves the specificity of TIRADS-alone in differentiating between benign and malignant thyroid nodules.