Comparison of Thyroid Risk Categorization Systems and Fine-Needle Aspiration Recommendations in a Multi-Institutional Thyroid Ultrasound Registry

ChatGPT-4's Accuracy in Estimating Thyroid Nodule Features and Cancer Risk From Ultrasound Images

OBJECTIVE

To evaluate the performance of GPT-4 and GPT-4o in accurately identifying features and categories from thyroid nodule ultrasound images following the American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS).

METHODS

This comparative validation study, conducted between October 2023 and May 2024, utilized 202 thyroid ultrasound images sourced from 3 open-access databases. Both complete and cropped versions of each image were independently evaluated by expert radiologists to establish a reference standard for TI-RADS features and categories. GPT-4 and GPT-4o were prompted to analyze each image, and their generated TI-RADS outputs were compared to the reference standard.

RESULTS

GPT-4 demonstrated high specificity but low sensitivity when assessing complete thyroid ultrasound images across most TI-RADS categories, resulting in mixed overall accuracy. For low-risk nodules (benign), GPT-4 achieved 25.0% sensitivity, 99.5% specificity, and 93.6% accuracy. In contrast, in the higher risk moderately suspicious category GPT-4 showed 75% sensitivity, 22.2% specificity, and 42.1% accuracy. While GPT-4 effectively identified features like smooth margins (73% vs 65% the reference standard), it struggled to identify other features like isoechoic echogenicity (5% vs 46%), and echogenic foci (3% vs 27%). The assessment of cropped images using both GPT-4 and GPT-4o followed similar patterns, though with slight deviations indicating a decrease in performance compared to GPT-4's assessment of complete images.

CONCLUSION

While GPT-4 and GPT-4o models show potential for improving the efficiency of thyroid nodule triage, their performance remains suboptimal, particularly in higher-risk categories. Further refinement and validation of these models are necessary before clinical implementation.

Cost-Effectiveness of the ACR TIRADS Compared to the ATA 2015 Risk Stratification Systems in the Evaluation of Incidental Thyroid Nodules

RATIONALE AND OBJECTIVES

Thyroid nodules are a common incidental imaging finding and prone to overdiagnosis. Several risk stratification systems have been developed to reduce unnecessary work-up, with two of the most utilized including the American Thyroid Association 2015 (ATA2015) and the newer American College of Radiology Thyroid Imaging, Reporting and Data System (TIRADS) guidelines. The purpose of this study is to evaluate the cost-effectiveness of the ATA2015 versus the TIRADS guidelines in the management of incidental thyroid nodules.

METHODS

A cost-utility analysis was conducted using decision tree modeling, evaluating adult patients with incidental thyroid nodules < 4 cm. Model inputs were populated using published literature, observational data, and expert opinion. Single-payer perspective, Canadian dollar currency, five-year time horizon, willingness to pay (WTP) threshold of $50,000, and discount rate of 1.5% per annum were utilized. Scenario, deterministic and probabilistic sensitivity analyses were performed. The primary outcome was the incremental cost-effectiveness ratio (ICER) expressed as incremental cost per quality-adjusted life year (QALY) gained.

RESULTS

For the base case scenario, TIRADS dominated the ATA2015 strategy by a slim margin, producing 0.005 more QALYs at $25 less cost. Results were sensitive to the malignancy rate of biopsy and the utilities of a patient with a benign nodule/subclinical malignancy or under surveillance. Probabilistic sensitivity analysis showed that TIRADS was the more cost-effective option 79.7% of the time.

CONCLUSION

The TIRADS guidelines may be the more cost-effective strategy by a small margin compared to ATA2015 in most scenarios when used to risk stratify incidental thyroid nodules.

The Horizon of Thyroid Imaging Reporting and Data System in the Diagnostic Performance of Thyroid Nodules: Clinical Application and Future Perspectives

The widespread occurrence of thyroid nodules and the typically slow progression of thyroid cancer have led to the development of the thyroid imaging reporting and data system (TI-RADS). The primary objectives behind the development of TI-RADS were to minimize unnecessary biopsies of non-cancerous nodules, enhance the overall precision of diagnosis and establish a uniform risk-stratification framework based on the lexicon to notify healthcare professionals of nodules that require a biopsy. The identification and precise diagnosis of thyroid nodules have led to improved clinical practice examination reports within the general population. TI-RADS is a risk-stratification system related to thyroid lesions and based on ultrasound characteristics and is similar to the structure of the breast imaging reporting and data system. There are various versions of TI-RADS, with some being widely used and adequately validated, while others lacking thorough evaluation. TI-RADS uses a numerical scoring system for characteristics, and its categories are determined by the cumulative score of a thyroid nodule, indicating the likelihood of it being benign or malignant. In this article, the various TI-RADS systems were examined as a successful method for producing precise and comprehensive documentation, with a particular emphasis on their functionality, similarities, distinctions and potential future developments.

Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study

OBJECTIVES

This study aimed to propose a deep learning (DL)-based framework for identifying the composition of thyroid nodules and assessing their malignancy risk.

METHODS

We conducted a retrospective multicenter study using ultrasound images from four hospitals. Convolutional neural network (CNN) models were constructed to classify ultrasound images of thyroid nodules into solid and non-solid, as well as benign and malignant. A total of 11,201 images of 6784 nodules were used for training, validation, and testing. The area under the receiver-operating characteristic curve (AUC) was employed as the primary evaluation index.

RESULTS

The models had AUCs higher than 0.91 in the benign and malignant grading of solid thyroid nodules, with the Inception-ResNet AUC being the highest at 0.94. In the test set, the best algorithm for identifying benign and malignant thyroid nodules had a sensitivity of 0.88, and a specificity of 0.86. In the human vs. DL test set, the best algorithm had a sensitivity of 0.93, and a specificity of 0.86. The Inception-ResNet model performed better than the senior physicians (p < 0.001). The sensitivity and specificity of the optimal model based on the external test set were 0.90 and 0.75, respectively.

CONCLUSIONS

This research demonstrates that CNNs can assist thyroid nodule diagnosis and reduce the rate of unnecessary fine-needle aspiration (FNA).

CLINICAL RELEVANCE STATEMENT

High-resolution ultrasound has led to increased detection of thyroid nodules. This results in unnecessary fine-needle aspiration and anxiety for patients whose nodules are benign. Deep learning can solve these problems to some extent.

KEY POINTS

• Thyroid solid nodules have a high probability of malignancy. • Our models can improve the differentiation between benign and malignant solid thyroid nodules. • The differential performance of one model was superior to that of senior radiologists. Applying this could reduce the rate of unnecessary fine-needle aspiration of solid thyroid nodules.

Improving the Efficacy of ACR TI-RADS Through Deep Learning-Based Descriptor Augmentation

Thyroid nodules occur in up to 68% of people, 95% of which are benign. Of the 5% of malignant nodules, many would not result in symptoms or death, yet 600,000 FNAs are still performed annually, with a PPV of 5-7% (up to 30%). Artificial intelligence (AI) systems have the capacity to improve diagnostic accuracy and workflow efficiency when integrated into clinical decision pathways. Previous studies have evaluated AI systems against physicians, whereas we aim to compare the benefits of incorporating AI into their final diagnostic decision. This work analyzed the potential for artificial intelligence (AI)-based decision support systems to improve physician accuracy, variability, and efficiency. The decision support system (DSS) assessed was Koios DS, which provides automated sonographic nodule descriptor predictions and a direct cancer risk assessment aligned to ACR TI-RADS. The study was conducted retrospectively between (08/2020) and (10/2020). The set of cases used included 650 patients (21% male, 79% female) of age 53 ± 15. Fifteen physicians assessed each of the cases in the set, both unassisted and aided by the DSS. The order of the reading condition was randomized, and reading blocks were separated by a period of 4 weeks. The system's impact on reader accuracy was measured by comparing the area under the ROC curve (AUC), sensitivity, and specificity of readers with and without the DSS with FNA as ground truth. The impact on reader variability was evaluated using Pearson's correlation coefficient. The impact on efficiency was determined by comparing the average time per read. There was a statistically significant increase in average AUC of 0.083 [0.066, 0.099] and an increase in sensitivity and specificity of 8.4% [5.4%, 11.3%] and 14% [12.5%, 15.5%], respectively, when aided by Koios DS. The average time per case decreased by 23.6% (p = 0.00017), and the observed Pearson's correlation coefficient increased from r = 0.622 to r = 0.876 when aided by Koios DS. These results indicate that providing physicians with automated clinical decision support significantly improved diagnostic accuracy, as measured by AUC, sensitivity, and specificity, and reduced inter-reader variability and interpretation times.

Multimodality US versus Thyroid Imaging Reporting and Data System Criteria in Recommending Fine-Needle Aspiration of Thyroid Nodules

Background Current guidelines recommend the use of conventional US for risk stratification and management of thyroid nodules. However, fine-needle aspiration (FNA) is often recommended in benign nodules. Purpose To compare the diagnostic performance of multimodality US (including conventional US, strain elastography, and contrast-enhanced US [CEUS]) with the American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS) in the recommendation of FNA for thyroid nodules to reduce unnecessary biopsies. Materials and Methods In this prospective study, 445 consecutive participants with thyroid nodules from nine tertiary referral hospitals were recruited between October 2020 and May 2021. With univariable and multivariable logistic regression, the prediction models incorporating sonographic features, evaluated with interobserver agreement, were constructed and internally validated with bootstrap resampling technique. In addition, discrimination, calibration, and decision curve analysis were performed. Results A total of 434 thyroid nodules confirmed at pathologic analysis (259 malignant thyroid nodules) in 434 participants (mean age, 45 years ± 12 [SD]; 307 female participants) were included. Four multivariable models incorporated participant age, nodule features at US (proportion of cystic components, echogenicity, margin, shape, punctate echogenic foci), elastography features (stiffness), and CEUS features (blood volume). In recommending FNA in thyroid nodules, the highest area under the receiver operating characteristic curve (AUC) was 0.85 (95% CI: 0.81, 0.89) for the multimodality US model, and the lowest AUC was 0.63 (95% CI: 0.59, 0.68) for TI-RADS (P < .001). At the 50% risk threshold, 31% (95% CI: 26, 38) of FNA procedures could be avoided with multimodality US compared with 15% (95% CI: 12, 19) with TI-RADS (P < .001). Conclusion Multimodality US had better performance in recommending FNA to avoid unnecessary biopsies than the TI-RADS. Clinical trial registration no. NCT04574258 © RSNA, 2023 Supplemental material is available for this article.

Differences in the management of thyroid nodules in children and adolescents as compared to adults

PURPOSE OF REVIEW

To describe recent advances in the diagnosis and management of thyroid nodules in both children and adults, highlighting differences between the two groups.

RECENT FINDINGS

Thyroid nodules are less common in children than adults but the risk of malignancy is higher. Screening recommendations for patients with certain genetic syndromes or specific risk factors for thyroid nodules and cancer continue to evolve ultrasound is the most appropriate imaging modality for evaluating thyroid nodules in all patients, but radiographic risk stratification systems validated in adults have not been extensively studied in children. Criteria for proceeding to fine needle aspiration (FNA) biopsy differ some between children and adults, with nodule size being less of a consideration in young patients. Molecular testing continues to improve the diagnostic capabilities of FNA for all age groups, but options for pediatric thyroid nodules are currently limited. In children, only oncogene panels are validated and their exact utility, along with miRNA classifiers, is an evolving area of study. For adults, data support selection of a specific molecular test based on ultrasound characteristics and pretest probability of malignancy. Multiple series have shown that fusion mutations are more common in pediatric thyroid nodules, while point mutations are seen more often in adult thyroid nodules. Molecular data help guide management recommendations regarding the need for surgery when FNA is indeterminate, but clinical use is still being refined. Radiofrequency ablation is a nonoperative approach that has gained significant traction for the treatment of symptomatic benign nodules in adults, but data are currently extremely limited in children.

SUMMARY

There are difference in the diagnosis and treatment of thyroid nodules in adults and children, the nuances of which are important for clinical management.

Clinician Agreement on the Classification of Thyroid Nodules Ultrasound Features: A Survey of 2 Endocrine Societies

CONTEXT

Thyroid nodule risk stratification allows clinicians to standardize the evaluation of thyroid cancer risk according to ultrasound features.

OBJECTIVE

To evaluate interrater agreement among clinicians assessing thyroid nodules ultrasound features and thyroid cancer risk categories.

DESIGN, SETTING, AND PARTICIPANTS

We surveyed Endocrine Society and Latin American Thyroid Society members to assess their interpretation of composition, echogenicity, shape, margins, and presence of echogenic foci of 10 thyroid nodule cases. The risk category for thyroid cancer was calculated following the American College of Radiology-Thyroid Imaging Reporting & Data System (ACR-TIRADS) framework from individual responses.

MAIN OUTCOMES AND MEASURES

We used descriptive statistics and Gwet's agreement coefficient (AC1) to assess the primary outcome of interrater agreement for ACR-TIRADS risk category. As secondary outcomes, the interrater agreement for individual features and a subgroup analysis of interrater agreement for the ACR-TIRADS category were performed (ultrasound reporting system, type of practice, and number of monthly appraisals).

RESULTS

A total of 144 participants were included, mostly endocrinologists. There was moderate level of agreement for the absence of echogenic foci (AC1 0.53, 95% CI 0.24-0.81) and composition (AC1 0.54, 95% CI 0.36-0.71). The agreement for margins (AC1 0.24, 95% CI 0.15-0.33), echogenicity (AC1 0.34, 95% CI 0.22-0.46), and shape assessment (AC1 0.42, 95% CI 0.13-0.70) was lower. The overall agreement for ACR-TIRADS assessment was AC1 0.29, (95% CI 0.13-0.45). The AC1 of ACR-TIRADS among subgroups was similar.

CONCLUSIONS

This study found high variation of judgments about ACR-TIRADS risk category and individual features, which poses a potential challenge for the widescale implementation of thyroid nodule risk stratification.

Evaluation of the efficacy of EU-TIRADS and ACR-TIRADS in risk stratification of pediatric patients with thyroid nodules

BACKGROUND

Pediatric thyroid nodules have a lower prevalence but a higher rate of malignancy (ROM) than those in adults. Ultrasound features suspected of malignancy lead to fine needle aspiration biopsy (FNAB) and subsequent cytological determination, upon which management is decided. Based on the characteristics of ultrasound, to standardize clinician decisions and avoid unnecessary FNAB, the European Thyroid Association and the American Radiology College have established guidelines for Thyroid Imaging, Reporting and Data System (EU-TIRADS and ACR-TIRADS) for ROM stratification of thyroid nodules. The aim of this study is to evaluate the diagnostic performance of ACR-TIRADS and EU-TIRADS in pediatric age.

MATERIALS AND METHODS

Subjects younger than 18 years of age with thyroid nodules greater than 0.5 cm observed in the 2000-2020 period were included.

RESULTS

Data from 200 subjects were collected. The overall ROM was 13%, rising to 26% if nodules with a diameter >1 cm were considered. Patients with a malignant nodule were more likely to have a higher EU-TIRADS score (p=0.03). Missed cancer diagnoses were 26.9%. Using the EU-TIRADS system, 40% of FNABs could have been avoided, while this scoring system would have resulted in FNAB being performed in 12% of cases where the assessment of ultrasound features would not recommend FNAB. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were 73.1%, 57.1%, 73.1%, and 50%, respectively. Even considering the ACR-TIRADS, a higher score correlated with a higher ROM (p<0.001). This system missed 6 diagnoses of cancer (23.1%). Using the ACR-TIRADS system, 45.3% of FNABs could have been avoided, while FNAB should have been performed in 12% of cases where it was not recommended by ultrasound characteristics. Sensitivity, specificity, PPV and NPV were 76.9%, 50%, 76.9%, and 42.9%, respectively.

CONCLUSION

The present study confirms the correspondence of the EU-TIRADS and ACR-TIRADS categories with respect to malignancy but indicates not entirely satisfactory performance compared to FNAB alone. However, the use of the two TIRADS systems should be encouraged in multicentre studies to increase their performance and establish paediatric-specific points in the scoring criteria.