Applications of machine learning and deep learning to thyroid imaging: where do we stand?

Artificial Intelligence Model Assisting Thyroid Nodule Diagnosis and Management: A Multicenter Diagnostic Study

CONTEXT

It is not clear how to integrate artificial intelligence (AI)-based models into diagnostic workflows.

OBJECTIVE

To develop and validate a deep-learning-based AI model (AI-Thyroid) for thyroid cancer diagnosis, and to explore how this improves diagnostic performance.

METHODS

The system was trained using 19 711 images of 6163 patients in a tertiary hospital (Ajou University Medical Center; AUMC). It was validated using 11 185 images of 4820 patients in 24 hospitals (test set 1) and 4490 images of 2367 patients in AUMC (test set 2). The clinical implications were determined by comparing the findings of six physicians with different levels of experience (group 1: 4 trainees, and group 2: 2 faculty radiologists) before and after AI-Thyroid assistance.

RESULTS

The area under the receiver operating characteristic (AUROC) curve of AI-Thyroid was 0.939. The AUROC, sensitivity, and specificity were 0.922, 87.0%, and 81.5% for test set 1 and 0.938, 89.9%, and 81.6% for test set 2. The AUROCs of AI-Thyroid did not differ significantly according to the prevalence of malignancies (>15.0% vs ≤15.0%, P = .226). In the simulated scenario, AI-Thyroid assistance changed the AUROC, sensitivity, and specificity from 0.854 to 0.945, from 84.2% to 92.7%, and from 72.9% to 86.6% (all P < .001) in group 1, and from 0.914 to 0.939 (P = .022), from 78.6% to 85.5% (P = .053) and from 91.9% to 92.5% (P = .683) in group 2. The interobserver agreement improved from moderate to substantial in both groups.

CONCLUSION

AI-Thyroid can improve diagnostic performance and interobserver agreement in thyroid cancer diagnosis, especially in less-experienced physicians.

Diagnostics of Thyroid Cancer Using Machine Learning and Metabolomics

The objective of this research is, with the analysis of existing data of thyroid cancer (TC) metabolites, to develop a machine-learning model that can diagnose TC using metabolite biomarkers. Through data mining, pathway analysis, and machine learning (ML), the model was developed. We identified seven metabolic pathways related to TC: Pyrimidine metabolism, Tyrosine metabolism, Glycine, serine, and threonine metabolism, Pantothenate and CoA biosynthesis, Arginine biosynthesis, Phenylalanine metabolism, and Phenylalanine, tyrosine, and tryptophan biosynthesis. The ML classifications' accuracies were confirmed through 10-fold cross validation, and the most accurate classification was 87.30%. The metabolic pathways identified in relation to TC and the changes within such pathways can contribute to more pattern recognition for diagnostics of TC patients and assistance with TC screening. With independent testing, the model's accuracy for other unique TC metabolites was 92.31%. The results also point to a possibility for the development of using ML methods for TC diagnostics and further applications of ML in general cancer-related metabolite analysis.

Machine Learning Model as a Useful Tool for Prediction of Thyroid Nodules Histology, Aggressiveness and Treatment-Related Complications

Thyroid nodules are very common, 5-15% of which are malignant. Despite the low mortality rate of well-differentiated thyroid cancer, some variants may behave aggressively, making nodule differentiation mandatory. Ultrasound and fine-needle aspiration biopsy are simple, safe, cost-effective and accurate diagnostic tools, but have some potential limits. Recently, machine learning (ML) approaches have been successfully applied to healthcare datasets to predict the outcomes of surgical procedures. The aim of this work is the application of ML to predict tumor histology (HIS), aggressiveness and post-surgical complications in thyroid patients. This retrospective study was conducted at the ENT Division of Eastern Piedmont University, Novara (Italy), and reported data about 1218 patients who underwent surgery between January 2006 and December 2018. For each patient, general information, HIS and outcomes are reported. For each prediction task, we trained ML models on pre-surgery features alone as well as on both pre- and post-surgery data. The ML pipeline included data cleaning, oversampling to deal with unbalanced datasets and exploration of hyper-parameter space for random forest models, testing their stability and ranking feature importance. The main results are (i) the construction of a rich, hand-curated, open dataset including pre- and post-surgery features (ii) the development of accurate yet explainable ML models. Results highlight pre-screening as the most important feature to predict HIS and aggressiveness, and that, in our population, having an out-of-range (Low) fT3 dosage at pre-operative examination is strongly associated with a higher aggressiveness of the disease. Our work shows how ML models can find patterns in thyroid patient data and could support clinicians to refine diagnostic tools and improve their accuracy.

Quantitative Diagnosis Progress of Ultrasound Imaging Technology in Thyroid Diffuse Diseases

High-frequency ultrasound (HFUS), the imaging modality of choice for thyroid screening, is most commonly used in the study of diffuse thyroid disease (DTD) with Hashimoto's thyroiditis (HT) and Graves' disease (GD). DTD can involve thyroid function and severely affect life quality, so early diagnosis is important for the development of timely clinical intervention strategies. Previously, the diagnosis of DTD relied on qualitative ultrasound imaging and related laboratory tests. In recent years, with the development of multimodal imaging and intelligent medicine, ultrasound and other diagnostic imaging techniques have gradually become more widely used for quantitative assessment of the structure and function of DTD. In this paper, we review the current status and progress of quantitative diagnostic ultrasound imaging techniques for DTD.

Investigating the diagnostic efficiency of a computer-aided diagnosis system for thyroid nodules in the context of Hashimoto's thyroiditis

Objectives

This study aims to investigate the efficacy of a computer-aided diagnosis (CAD) system in distinguishing between benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis (HT) and to evaluate the role of the CAD system in reducing unnecessary biopsies of benign lesions.

Methods

We included a total of 137 nodules from 137 consecutive patients (mean age, 43.5 ± 11.8 years) who were histopathologically diagnosed with HT. The two-dimensional ultrasound images and videos of all thyroid nodules were analyzed by the CAD system and two radiologists with different experiences according to ACR TI-RADS. The diagnostic cutoff values of ACR TI-RADS were divided into two categories (TR4 and TR5), and then the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of the CAD system and the junior and senior radiologists were compared in both cases. Moreover, ACR TI-RADS classification was revised according to the results of the CAD system, and the efficacy of recommended fine-needle aspiration (FNA) was evaluated by comparing the unnecessary biopsy rate and the malignant rate of punctured nodules.

Results

The accuracy, sensitivity, specificity, PPV, and NPV of the CAD system were 0.876, 0.905, 0.830, 0.894, and 0.846, respectively. With TR4 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.628, and 0.722, respectively, and the CAD system had the highest AUC (P < 0.0001). With TR5 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.654, and 0.812, respectively, and the CAD system had a higher AUC than the junior radiologist (P < 0.0001) but comparable to the senior radiologist (P = 0.0709). With the assistance of the CAD system, the number of TR4 nodules was decreased by both junior and senior radiologists, the malignant rate of punctured nodules increased by 30% and 22%, and the unnecessary biopsies of benign lesions were both reduced by nearly half.

Conclusions

The CAD system based on deep learning can improve the diagnostic performance of radiologists in identifying benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis and can play a role in FNA recommendations to reduce unnecessary biopsy rates.

Development of a machine learning-based fine-grained risk stratification system for thyroid nodules using predefined clinicoradiological features

OBJECTIVE

We constructed and validated a machine learning-based malignancy risk estimation model using predefined clinicoradiological features, and evaluated its clinical utility for the management of thyroid nodules.

METHODS

In total, 5708 benign (n = 4597) and malignant (n = 1111) thyroid nodules were collected from 5081 consecutive patients treated in 26 institutions. Seventeen experienced radiologists evaluated nodule characteristics on ultrasonographic images. Eight predictive models were used to stratify the thyroid nodules according to malignancy risk; model performance was assessed via nested 10-fold cross-validation. The best-performing algorithm was externally validated using data for 454 thyroid nodules from a tertiary hospital, then compared to the Thyroid Imaging Reporting and Data System (TIRADS)-based interpretations of radiologists (American College of Radiology, European and Korean TIRADS, and AACE/ACE/AME guidelines).

RESULTS

The area under the receiver operating characteristic (AUROC) curves of the algorithms ranged from 0.773 to 0.862. The sensitivities, specificities, positive predictive values, and negative predictive values of the best-performing models were 74.1-76.6%, 80.9-83.4%, 49.2-51.9%, and 93.0-93.5%, respectively. For the external validation set, the ElasticNet values were 83.2%, 89.2%, 81.8%, and 90.1%, respectively. The corresponding TIRADS values were 66.5-85.0%, 61.3-80.8%, 45.9-72.1%, and 81.5-90.3%, respectively. The new model exhibited a significantly higher AUROC and specificity than did the TIRADS risk stratification, although its sensitivity was similar.

CONCLUSION

We developed a reliable machine learning-based predictive model that demonstrated enhanced specificity when stratifying thyroid nodules according to malignancy risk. This system will contribute to improved personalized management of thyroid nodules.

KEY POINTS

• The area under the receiver operating characteristic (AUROC) curve, sensitivity, and specificity of our model were 0.914, 83.2%, and 89.2%, respectively (derived using the validation dataset). • Compared to the TIRADS values, the AUROC and specificity are significantly higher, while the sensitivity is similar. • An interactive version of our AI algorithm is at http://tirads.cdss.co.kr .

Application of Deep Learning to Reduce the Rate of Malignancy Among BI-RADS 4A Breast Lesions Based on Ultrasonography

The aim of the work described here was to develop an ultrasound (US) image-based deep learning model to reduce the rate of malignancy among breast lesions diagnosed as category 4A of the Breast Imaging-Reporting and Data System (BI-RADS) during the pre-operative US examination. A total of 479 breast lesions diagnosed as BI-RADS 4A in pre-operative US examination were enrolled. There were 362 benign lesions and 117 malignant lesions confirmed by postoperative pathology with a malignancy rate of 24.4%. US images were collected from the database server. They were then randomly divided into training and testing cohorts at a ratio of 4:1. To correctly classify malignant and benign tumors diagnosed as BI-RADS 4A in US, four deep learning models, including MobileNet, DenseNet121, Xception and Inception V3, were developed. The performance of deep learning models was compared using the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Meanwhile, the robustness of the models was evaluated by five-fold cross-validation. Among the four models, the MobileNet model turned to be the optimal model with the best performance in classifying benign and malignant lesions among BI-RADS 4A breast lesions. The AUROC, accuracy, sensitivity, specificity, PPV and NPV of the optimal model in the testing cohort were 0.897, 0.913, 0.926, 0.899, 0.958 and 0.784, respectively. About 14.4% of patients were expected to be upgraded to BI-RADS 4B in US with the assistance of the MobileNet model. The deep learning model MobileNet can help to reduce the rate of malignancy among BI-RADS 4A breast lesions in pre-operative US examinations, which is valuable to clinicians in tailoring treatment for suspicious breast lesions identified on US.

Development of a Machine Learning-Based Screening Method for Thyroid Nodules Classification by Solving the Imbalance Challenge in Thyroid Nodules Data

BACKGROUND

This study aims to show the impact of imbalanced data and the typical evaluation methods in developing and misleading assessments of machine learning-based models for preoperative thyroid nodules screening.

STUDY DESIGN

A retrospective study.

METHODS

The ultrasonography features for 431 thyroid nodules cases were extracted from medical records of 313 patients in Babol, Iran. Since thyroid nodules are commonly benign, the relevant data are usually unbalanced in classes. It can lead to the bias of learning models toward the majority class. To solve it, a hybrid resampling method called the Smote-was used to creating balance data. Following that, the support vector classification (SVC) algorithm was trained by balance and unbalanced datasets as Models 2 and 3, respectively, in Python language programming. Their performance was then compared with the logistic regression model as Model 1 that fitted traditionally.

RESULTS

The prevalence of malignant nodules was obtained at 14% (n = 61). In addition, 87% of the patients in this study were women. However, there was no difference in the prevalence of malignancy for gender. Furthermore, the accuracy, area under the curve, and geometric mean values were estimated at 92.1%, 93.2%, and 76.8% for Model 1, 91.3%, 93%, and 77.6% for Model 2, and finally, 91%, 92.6% and 84.2% for Model 3, respectively. Similarly, the results identified Micro calcification, Taller than wide shape, as well as lack of ISO and hyperechogenicity features as the most effective malignant variables.

CONCLUSION

Paying attention to data challenges, such as data imbalances, and using proper criteria measures can improve the performance of machine learning models for preoperative thyroid nodules screening.

Convolutional Neural Network for Predicting Thyroid Cancer Based on Ultrasound Elastography Image of Perinodular Region

We aimed to develop deep learning models based on perinodular regions' shear-wave elastography (SWE) images and ultrasound (US) images of thyroid nodules (TNs) and determine their performances in predicting thyroid cancer. A total of 1747 American College of Radiology Thyroid Imaging Reporting & Data System 4 (TR4) thyroid nodules (TNs) in 1582 patients were included in this retrospective study. US images, SWE images, and 2 quantitative SWE parameters (maximum elasticity of TNs; 5-point average maximum elasticity of TNs) were obtained. Based on US and SWE images of TNs and perinodular tissue, respectively, 7 single-image convolutional neural networks (CNN) models [US, internal SWE, 0.5 mm SWE, 1.0 mm SWE, 1.5 mm SWE, 2.0 mm SWE of perinodular tissue, and whole SWE region of interest (ROI) image] and another 6 fusional-image CNN models (US + internal SWE, US + 0.5 mm SWE, US + 1.0 mm SWE, US + 1.5 mm SWE, US + 2.0 mm SWE, US + ROI SWE) were established using RestNet18. All of the CNN models and quantitative SWE parameters were built on a training cohort (1247 TNs) and evaluated on a validation cohort (500 TNs). In predicting thyroid cancer, US + 2.0 mm SWE image CNN model obtained the highest area under the curve in 10 mm < TNs ≤ 20 mm (0.95 for training; 0.92 for validation) and TNs > 20 mm (0.95 for training; 0.92 for validation), while US + 1.0 mm SWE image CNN model obtained the highest area under the curve in TNs ≤ 10 mm (0.95 for training; 0.92 for validation). The CNN models based on the fusion of SWE segmentation images and US images improve the radiological diagnostic accuracy of thyroid cancer.

Artificial Intelligence (AI) Tools for Thyroid Nodules on Ultrasound, From the AJR Special Series on AI Applications

Artificial intelligence (AI) methods for evaluating thyroid nodules on ultrasound have been widely described in the literature, with reported performance of AI tools matching or in some instances surpassing radiologists. As these data have accumulated, products for classification and risk stratification of thyroid nodules on ultrasound have become commercially available. This article reviews FDA-approved products currently on the market, with a focus on product features, reported performance, and considerations for implementation. The products perform risk stratification primarily using the Thyroid Imaging Reporting and Data System (TI-RADS), though may provide additional prediction tools independent of TI-RADS. Key issues in implementation include integration with radiologist interpretation, impact on workflow and efficiency, and performance monitoring. AI applications beyond nodule classification, including report construction and incidental findings follow-up, are also described. Anticipated future directions of research and development in AI tools for thyroid nodules are highlighted.

Diagnosis of Thyroid Nodules Based on Image Enhancement and Deep Neural Networks

The diagnosis of thyroid nodules at an early stage is a challenging task. Manual diagnosis of thyroid nodules is labor-intensive and time-consuming. Meanwhile, due to the difference of instruments and technical personnel, the original thyroid nodule ultrasound images collected are very different. In order to make better use of ultrasound image information of thyroid nodules, some image processing methods are indispensable. In this paper, we developed a method for automatic thyroid nodule classification based on image enhancement and deep neural networks. The selected image enhancement method is histogram equalization, and the neural networks have four-layer network nodes in our experiments. The dataset in this paper consists of thyroid nodule images of 508 patients. The data are divided into 80% training and 20% validation sets. A comparison result demonstrates that our method can achieve a better performance than other normal machine learning methods. The experimental results show that our method has achieved 0.901961 accuracy, 0.894737 precision, 1 recall, and 0.944444 F1-score. At the same time, we also considered the influence of network structure, activation function of network nodes, number of training iterations, and other factors on the classification results. The experimental results show that the optimal network structure is 2500-40-2-1, the optimal activation function is logistic function, and the best number of training iterations is 500.

Thyroid cancer under the scope of emerging technologies

The vast majority of thyroid cancers originate from follicular cells. We outline outstanding issues at each step along the path of cancer patient care, from prevention to post-treatment follow-up and highlight how emerging technologies will help address them in the coming years. Three directions will dominate the coming technological landscape. Genomics will reveal tumoral evolutionary history and shed light on how these cancers arise from the normal epithelium and the genomics alteration driving their progression. Transcriptomics will gain cellular and spatial resolution providing a full account of intra-tumor heterogeneity and opening a window on the microenvironment supporting thyroid tumor growth. Artificial intelligence will set morphological analysis on an objective quantitative ground laying the foundations of a systematic thyroid tumor classification system. It will also integrate into unified representations the molecular and morphological perspectives on thyroid cancer.

Comparative Analysis of Computer-Aided Diagnosis and Computer-Assisted Subjective Assessment in Thyroid Ultrasound

The value of computer-aided diagnosis (CAD) and computer-assisted techniques equipped with different TIRADS remains ambiguous. Parallel diagnosis performances of computer-assisted subjective assessments and CAD were compared based on AACE, ATA, EU, and KSThR TIRADS. CAD software computed the diagnosis of 162 thyroid nodule sonograms. Two raters (R₁ and R₂) independently rated the sonographic features of the nodules using an online risk calculator while blinded to pathology results. Diagnostic efficiency measures were calculated based on the final pathology results. R₁ had higher diagnostic performance outcomes than CAD with similarities between KSThR (SEN: 90.3% vs. 83.9%, p = 0.57; SPEC: 46% vs. 51%, p = 0.21; AUROC: 0.76 vs. 0.67, p = 0.02), and EU (SEN: 85.5% vs. 79%, p = 0.82; SPEC: 62% vs. 55%, p = 0.27; AUROC: 0.74 vs. 0.67, p = 0.06). Similarly, R₂ had higher AUROC and specificity but lower sensitivity than CAD (KSThR-AUROC: 0.74 vs. 0.67, p = 0.13; SPEC: 61% vs. 46%, p = 0.02 and SEN: 75.8% vs. 83.9%, p = 0.31, and EU-AUROC: 0.69 vs. 0.67, p = 0.57, SPEC: 64% vs. 55%, p = 0.19, and SEN: 71% vs. 79%, p = 0.51, respectively). CAD had higher sensitivity but lower specificity than both R₁ and R₂ with AACE for 114 specified nodules (SEN: 92.5% vs. 88.7%, p = 0.50; 92.5% vs. 79.3%, p = 0.02, and SPEC: 26.2% vs. 54.1%, p = 0.001; 26.2% vs. 62.3%, p < 0.001, respectively). All diagnostic performance outcomes were comparable for ATA with 96 specified nodules. Computer-assisted subjective interpretation using KSThR is more ideal for ruling out papillary thyroid carcinomas than CAD. Future larger multi-center and multi-rater prospective studies with a diverse representation of thyroid cancers are necessary to validate these findings.

A Computer-Aided Diagnosis System and Thyroid Imaging Reporting and Data System for Dual Validation of Ultrasound-Guided Fine-Needle Aspiration of Indeterminate Thyroid Nodules

Purpose

The fully automatic AI-Sonic computer-aided design (CAD) system was employed for the detection and diagnosis of benign and malignant thyroid nodules. The aim of this study was to investigate the efficiency of the AI-Sonic CAD system with the use of a deep learning algorithm to improve the diagnostic accuracy of ultrasound-guided fine-needle aspiration (FNA).

Methods

A total of 138 thyroid nodules were collected from 124 patients and diagnosed by an expert, a novice, and the Thyroid Imaging Reporting and Data System (TI-RADS). Diagnostic efficiency and feasibility were compared among the expert, novice, and CAD system. The application of the CAD system to enhance the diagnostic efficiency of novices was assessed. Moreover, with the experience of the expert as the gold standard, the values of features detected by the CAD system were also analyzed. The efficiency of FNA was compared among the expert, novice, and CAD system to determine whether the CAD system is helpful for the management of FNA.

Result

In total, 56 malignant and 82 benign thyroid nodules were collected from the 124 patients (mean age, 46.4 ± 12.1 years; range, 12–70 years). The diagnostic area under the curve of the CAD system, expert, and novice were 0.919, 0.891, and 0.877, respectively (p < 0.05). In regard to feature detection, there was no significant differences in the margin and composition between the benign and malignant nodules (p > 0.05), while echogenicity and the existence of echogenic foci were of great significance (p < 0.05). For the recommendation of FNA, the results showed that the CAD system had better performance than the expert and novice (p < 0.05).

Conclusions

Precise diagnosis and recommendation of FNA are continuing hot topics for thyroid nodules. The CAD system based on deep learning had better accuracy and feasibility for the diagnosis of thyroid nodules, and was useful to avoid unnecessary FNA. The CAD system is potentially an effective auxiliary approach for diagnosis and asymptomatic screening, especially in developing areas.

Ultrasound Image Classification of Thyroid Nodules Using Machine Learning Techniques

Background and Objectives: Thyroid nodules are lumps of solid or liquid-filled tumors that form inside the thyroid gland, which can be malignant or benign. Our aim was to test whether the described features of the Thyroid Imaging Reporting and Data System (TI-RADS) could improve radiologists' decision making when integrated into a computer system. In this study, we developed a computer-aided diagnosis system integrated into multiple-instance learning (MIL) that would focus on benign-malignant classification. Data were available from the Universidad Nacional de Colombia. Materials and Methods: There were 99 cases (33 Benign and 66 malignant). In this study, the median filter and image binarization were used for image pre-processing and segmentation. The grey level co-occurrence matrix (GLCM) was used to extract seven ultrasound image features. These data were divided into 87% training and 13% validation sets. We compared the support vector machine (SVM) and artificial neural network (ANN) classification algorithms based on their accuracy score, sensitivity, and specificity. The outcome measure was whether the thyroid nodule was benign or malignant. We also developed a graphic user interface (GUI) to display the image features that would help radiologists with decision making. Results: ANN and SVM achieved an accuracy of 75% and 96% respectively. SVM outperformed all the other models on all performance metrics, achieving higher accuracy, sensitivity, and specificity score. Conclusions: Our study suggests promising results from MIL in thyroid cancer detection. Further testing with external data is required before our classification model can be employed in practice.

Application of Artificial Intelligence Technology in Oncology: Towards the Establishment of Precision Medicine

In recent years, advances in artificial intelligence (AI) technology have led to the rapid clinical implementation of devices with AI technology in the medical field. More than 60 AI-equipped medical devices have already been approved by the Food and Drug Administration (FDA) in the United States, and the active introduction of AI technology is considered to be an inevitable trend in the future of medicine. In the field of oncology, clinical applications of medical devices using AI technology are already underway, mainly in radiology, and AI technology is expected to be positioned as an important core technology. In particular, "precision medicine," a medical treatment that selects the most appropriate treatment for each patient based on a vast amount of medical data such as genome information, has become a worldwide trend; AI technology is expected to be utilized in the process of extracting truly useful information from a large amount of medical data and applying it to diagnosis and treatment. In this review, we would like to introduce the history of AI technology and the current state of medical AI, especially in the oncology field, as well as discuss the possibilities and challenges of AI technology in the medical field.