Intraobserver and Interobserver Variability in Ultrasound Measurements of Thyroid Nodules

Deep learning-assisted ultrasonic diagnosis of cervical lymph node metastasis of thyroid cancer: a retrospective study of 3059 patients

Objective

This study aimed to develop a deep learning system to identify and differentiate the metastatic cervical lymph nodes (CLNs) of thyroid cancer.

Methods

From January 2014 to December 2020, 3059 consecutive patients with suspected with metastatic CLNs of thyroid cancer were retrospectively enrolled in this study. All CLNs were confirmed by fine needle aspiration. The patients were randomly divided into the training (1228 benign and 1284 metastatic CLNs) and test (307 benign and 240 metastatic CLNs) groups. Grayscale ultrasonic images were used to develop and test the performance of the Y-Net deep learning model. We used the Y-Net network model to segment and differentiate the lymph nodes. The Dice coefficient was used to evaluate the segmentation efficiency. Sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) were used to evaluate the classification efficiency.

Results

In the test set, the median Dice coefficient was 0.832. The sensitivity, specificity, accuracy, PPV, and NPV were 57.25%, 87.08%, 72.03%, 81.87%, and 66.67%, respectively. We also used the Y-Net classified branch to evaluate the classification efficiency of the LNs ultrasonic images. The classification branch model had sensitivity, specificity, accuracy, PPV, and NPV of 84.78%, 80.23%, 82.45%, 79.35%, and 85.61%, respectively. For the original ultrasonic reports, the sensitivity, specificity, accuracy, PPV, and NPV were 95.14%, 34.3%, 64.66%, 59.02%, 87.71%, respectively. The Y-Net model yielded better accuracy than the original ultrasonic reports.

Conclusion

The Y-Net model can be useful in assisting sonographers to improve the accuracy of the classification of ultrasound images of metastatic CLNs.

Inter-observer variation in two-dimensional and three-dimensional ultrasound measurement of papillary thyroid microcarcinoma

BACKGROUNDS

The reliable ultrasound (US) measurements of papillary thyroid microcarcinoma (PTMC) are very important during active surveillance. This prospective study was design to investigate the inter-observer reliability and agreement of two- dimensional ultrasound(2DUS) and three-dimensional ultrasound(3DUS) in the measurement of maximum diameter and volume for PTMC.

METHODS

This prospective study included 51 consecutive patients with solitary PTMC confirmed by biopsy. Two independent observers performed measurements of each tumor using a standardized measurement protocol. The maximum diameter was the largest one of the three diameters measured on the largest transverse and longitudinal 2DUS images. 2DUS volume was calculated using ellipsoid formula method. The virtual organ computer aided analysis(VOCAL) was used to determine 3DUS volume. The inter-observer reliability was assessed using intraclass correlation coefficient(ICC) with 95% confidence intervals(CIs). Bland-Altman analysis was used to evaluate agreement, and expressed as a bias with 95% limits of agreement(LOA).

RESULTS

The maximum diameter was 0.78 ± 0.14 cm. Volume measured by 3DUS was significantly smaller than that by 2DUS(0.163 ± 0.074 cm3 vs. 0.175 ± 0.078 cm3, P = 0.005). The ICCs of inter-observer reliability of maximum diameter, 2DUS volume and 3DUS volume was 0.922(0.864-0.955), 0.928(0.874-0.959), and 0.974(0.955-0.985), respectively. The ICCs of 2DUS and 3DUS volume was 0.955(0.909-0.976). The inter-observer agreement of maximum diameter, 2DUS volume and 3DUS volume was 1.096(0.7322 to 1.459), 1.008(0.5802-1.435), and 1.011(0.7576-1.265), respectively. The inter-observer agreement of 2DUS and 3DUS volume was 1.096(0.7322 to 1.459).

CONCLUSION

Maximum diameter had the lowest degree of observer variation among all the measurements. Volume measured by 3DUS had lower variability and higher repeatability than that by 2DUS, which might be helpful to provide more reliable estimates of tumor size for PTMC.

Artificial intelligence in the diagnosis of thyroid cancer: Recent advances and future directions

The diagnosis and management of thyroid cancer is fraught with challenges despite the advent of innovative diagnostic, surgical, and chemotherapeutic modalities. Challenges like inaccuracy in prognostication, uncertainty in cytopathological diagnosis, trouble in differentiating follicular neoplasms, intra-observer and inter-observer variability on ultrasound imaging preclude personalised treatment in thyroid cancer. Artificial intelligence (AI) is bringing a paradigm shift to the healthcare, powered by quick advancement of the analytic techniques. Several recent studies have shown remarkable progress in thyroid cancer diagnostics based on AI-assisted algorithms. Application of AI techniques in thyroid ultrasonography and cytopathology have shown remarkable impro-vement in sensitivity and specificity over the traditional diagnostic modalities. AI has also been explored in the development of treatment algorithms for indeterminate nodules and for prognostication in the patients with thyroid cancer. The benefits of high repeatability and straightforward implementation of AI in the management of thyroid cancer suggest that it holds promise for clinical application. Limited clinical experience and lack of prospective validation studies remain the biggest drawbacks. Developing verified and trustworthy algorithms after extensive testing and validation using prospective, multi-centre trials is crucial for the future use of AI in the pipeline of precision medicine in the management of thyroid cancer.

Thyroid Nodule Detection and Region Estimation in Ultrasound Images: A Comparison between Physicians and an Automated Decision Support System Approach

BACKGROUND

Thyroid nodules are very common. In most cases, they are benign, but they can be malignant in a low percentage of cases. The accurate assessment of these nodules is critical to choosing the next diagnostic steps and potential treatment. Ultrasound (US) imaging, the primary modality for assessing these nodules, can lack objectivity due to varying expertise among physicians. This leads to observer variability, potentially affecting patient outcomes.

PURPOSE

This study aims to assess the potential of a Decision Support System (DSS) in reducing these variabilities for thyroid nodule detection and region estimation using US images, particularly in lesser experienced physicians.

METHODS

Three physicians with varying levels of experience evaluated thyroid nodules on US images, focusing on nodule detection and estimating cystic and solid regions. The outcomes were compared to those obtained from a DSS for comparison. Metrics such as classification match percentage and variance percentage were used to quantify differences.

RESULTS

Notable disparities exist between physician evaluations and the DSS assessments: the overall classification match percentage was just 19.2%. Individually, Physicians 1, 2, and 3 had match percentages of 57.6%, 42.3%, and 46.1% with the DSS, respectively. Variances in assessments highlight the subjectivity and observer variability based on physician experience levels.

CONCLUSIONS

The evident variability among physician evaluations underscores the need for supplementary decision-making tools. Given its consistency, the CAD offers potential as a reliable "second opinion" tool, minimizing human-induced variabilities in the critical diagnostic process of thyroid nodules using US images. Future integration of such systems could bolster diagnostic precision and improve patient outcomes.

Use of 3D ultrasound to characterise temporal changes in thyroid nodules: an in vitro study

OBJECTIVE

Thyroid nodules are extremely common, with prevalence rate up to 68%, yet only 7-15% of these are malignant. Many nodules require surveillance and 2-dimensional ultrasound (2D US) is used. Issues include the huge workload of obtaining and labeling images and difficulty comparing sizes of nodules over time due to large inter-operator variability. Inaccuracies may result in unnecessary FNAC or missed diagnosis of malignant nodules.

METHODS

We compared two techniques: freehand plain 2D US against freehand 2D US with gyroscopic guidance, both followed by 3D reconstruction using software. We measured the volume of nodules and a normal thyroid gland.

RESULTS

We found 2D US with gyroscopic guidance to be superior to plain 2D US as 3D reconstructions of greater accuracy are produced. The volume of the thyroid lobe measured 8.42 cm3 ± 0.94 was reasonably close to the normal average volume. However, the measured volume of the ellipsoidal nodule by the software is 8.69 cm3 ± 0.97 while the measured volume of the spherical nodule is 7.09 cm3 ± 0.79. As the expected volume of the nodules were 4.24cm3 and 4.19 cm3 respectively, the measured volume of the nodule was not accurate. The time taken to characterise nodules was reduced greatly from over 30 min in usual procedure to less than 10 min.

CONCLUSION

We find 3D US promising for evaluating size of thyroid nodules, with potential to study other TIRAD characteristics. Freehand 2D US with gyroscopic guidance shows the most promise for producing reliable, accurate and faster 3D reconstructions of thyroid nodules.

A Reappraisal of Suspicious Sonographic Features of Thyroid Nodules: Shape Is Not an Independent Predictor of Malignancy

CONTEXT

For the correct clinical application of the sonographic risk-stratification systems, the definition of independent risk features that are foundational to each system is crucial.

OBJECTIVE

The aim of this study was to identify the gray-scale sonographic features independently associated with malignancy, and to compare different definitions.

METHODS

This prospective, diagnostic accuracy study took place in a single thyroid nodule referral center. All patients consecutively referred to our center for fine-needle aspiration cytology of a thyroid nodule between November 1, 2015 and March 30, 2020, were enrolled before cytology. Each nodule was examined by 2 experienced clinicians to record the sonographic features on a rating form. Histologic (when available) or cytologic diagnosis was used as the reference standard. For each single sonographic feature and definition, the sensitivity, specificity, positive and negative predictive values, and diagnostic odds ratios (DOR) were calculated. The significant predictors were then included in a multivariable regression model.

RESULTS

The final study cohort consisted of 903 nodules in 852 patients. A total of 76 nodules (8.4%) were malignant. Six features were independent predictors of malignancy: suspicious lymph node (DOR 16.23), extrathyroidal extension (DOR 6.60), irregular or infiltrative margins (DOR 7.13), marked hypoechogenicity (DOR 3.16), solid composition (DOR 3.61), and punctate hyperechoic foci (including microcalcifications and indeterminate foci; DOI 2.69). Taller-than-wide shape was not confirmed as an independent predictor.

CONCLUSION

We identified the key suspicious features of thyroid nodules and provided a simplified definition of some debated ones. Malignancy rate increases with number of features.

Identifying and Predicting Diverse Patterns of Benign Nodule Growth

CONTEXT

The natural history of benign thyroid nodules is typically characterized by slow growth and minimal risk of malignant transformation. Available data have, to date, been unable to elucidate the diversity of benign nodule growth patterns over time nor predictive of which patients follow which pattern.

OBJECTIVE

We aimed to better define the diverse patterns of benign nodule behavior and their predictors.

METHODS

We prospectively studied 389 consecutive patients with solitary, solid, cytologically benign thyroid nodules ≥1 cm and follow-up ultrasound for at least 4 years. Demographic, sonographic, biochemical data were collected at initial evaluation, and subsequent growth patterns were identified over the follow-up. Predictors of growth at initial evaluation and 3 years of follow-up were defined.

RESULTS

The mean (±SD) follow-up was 7.7 (±2.7) years. Three distinct growth patterns were identified: A) stagnant nodules with average growth rate < 0.2 mm/year; B) slow-growing nodules with a rate 0.2 to 1.0 mm/year; and C) fast-growing nodules increasing > 1.0 mm/year. Fast-growing nodules represented 17.2% of the cohort, and were more frequent in patients younger than 50 years (OR 2.2 [1.2-4.1], P = 0.016), and in larger nodules (2.0-2.9 cm, OR 3.5 [1.7-7.1], P = 0.001; >3.0 cm, OR 4.4 [1.8-10.4], P = 0.001 vs reference 1-1.9 cm). In a multiple regression model, nodule growth at 3 years at an average growth rate over 0.2 mm/year over 3 years since initial evaluation was an independent predictor of longer-term fast nodule growth, even after adjusting for age, biological sex, TSH level, and nodule size (P < 0.001).

CONCLUSION

The natural history of benign nodule growth is diverse, with over 80% of nodules demonstrating minimal to no growth long-term. Nearly 20% of cytologically benign nodules may exhibit a fast, continued growth pattern, which can be predicted by the 3-year growth rate pattern. These findings can help inform decision making for tailored benign nodule follow-up and monitoring.

Artificial intelligence in thyroid ultrasound

Artificial intelligence (AI), particularly deep learning (DL) algorithms, has demonstrated remarkable progress in image-recognition tasks, enabling the automatic quantitative assessment of complex medical images with increased accuracy and efficiency. AI is widely used and is becoming increasingly popular in the field of ultrasound. The rising incidence of thyroid cancer and the workload of physicians have driven the need to utilize AI to efficiently process thyroid ultrasound images. Therefore, leveraging AI in thyroid cancer ultrasound screening and diagnosis cannot only help radiologists achieve more accurate and efficient imaging diagnosis but also reduce their workload. In this paper, we aim to present a comprehensive overview of the technical knowledge of AI with a focus on traditional machine learning (ML) algorithms and DL algorithms. We will also discuss their clinical applications in the ultrasound imaging of thyroid diseases, particularly in differentiating between benign and malignant nodules and predicting cervical lymph node metastasis in thyroid cancer. Finally, we will conclude that AI technology holds great promise for improving the accuracy of thyroid disease ultrasound diagnosis and discuss the potential prospects of AI in this field.

Inter-Rater Reliability of Thyroid Ultrasound Risk Criteria: A Systematic Review and Meta-Analysis

OBJECTIVE

The most commonly employed diagnostic criteria for identifying thyroid nodules include Thyroid Imaging and Reporting Data System (TI-RADS) and American Thyroid Association (ATA) guidelines. The purpose of this systematic review and meta-analysis is to determine the inter-rater reliability of thyroid ultrasound criteria.

METHODS

We performed a library search of MEDLINE (Ovid), EMBASE (Ovid), and Web of Science for full-text articles published from January 2005 to June 2022. We included full-text primary research articles that used TI-RADS and/or ATA guidelines to evaluate thyroid nodules in adults. These included studies must have calculated inter-rater reliability using any validated metric. The Quality Appraisal for Reliability Studies (QAREL) was used to assess study quality. We planned for a random-effects meta-analysis, in addition to covariate and publication bias analyses. This study was performed in accordance with Preferred Reporting Items for a Systematic Review and Meta-analysis guidelines and registered prior to conduction (International prospective register of systematic reviews-PROSPERO: CRD42021275072).

RESULTS

Of the 951 articles identified via the database search, 35 met eligibility criteria. All studies were observational. The most commonly utilized criteria were ACR Thyroid Imaging and Reporting Data System (TI-RADS) and/or ATA criteria, while the majority of studies employed Κ statistics. For ACR TI-RADS, the pooled Κ was 0.51 (95% confidence interval [CI]: 0.42, 0.57; n = 7) while for ATA, the pooled Κ was 0.52 (95% CI: 0.37, 0.67; n = 3). Due to the small number of studies, covariate or publication bias analyses were not performed.

CONCLUSION

Ultrasound criteria demonstrate moderate inter-rater reliability, but these findings are impacted by poor study quality and a lack of standardization. Laryngoscope, 133:485-493, 2023.

Interobserver and Intraobserver Reliability in Sonoelastographic Assessment of Thyroid Nodules

BACKGROUND

Thyroid ultrasonography is the first and perhaps most fundamental step for the radiological distinction of benign and malignant nodules. In this study, 2 radiologists reviewed the sonoelastographic and Doppler images of thyroid nodules and evaluated for the intraobserver and interobserver reliability.

PURPOSE

We aimed to determine confusing nodule identifiers and sonographic features differently defined by observers.

METHODS

A total of 157 nodules in 91 patients (male/female, 72:19) with ages ranging from 18 to 72 years old were included in the study. Ultrasonographic images and video clips of the nodules were obtained and presented to 2 reviewers unaware of the cytopathology results. Two observers defined the characteristics of the nodules based on previously determined criteria. Then, intraobserver and interobserver correlation coefficients were calculated for each subcategory.

RESULTS

In the grayscale ultrasonographic examination, varying degrees from low to high interobserver correlation coefficients were obtained for different subcategories (between κ = 0.359 and κ = 0.821). In color Doppler examination, we obtained medium correlation coefficients ( κ = 0.493 and κ = 0.553). On the other hand, there was a high correlation coefficient in tissue compression elastography ( κ = 0.617 and κ = 0.638).According to our study results, elastographic pattern, shape of the nodule, presence of echogenic foci, and pathological lymph nodes are better predictors to determine the malignant potential of thyroid nodule with higher interobserver correlation. Therefore, these criteria may be used primarily for the evaluation of thyroid nodules. The intraobserver correlation coefficient was higher in the practitioner with longer experience, suggesting the importance of professional practice period on the decision-making process.

Clinical knowledge embedded method based on multi-task learning for thyroid nodule classification with ultrasound images

Objective. Thyroid nodules are common glandular abnormality that need to be diagnosed as benign or malignant to determine further treatments. Clinically, ultrasonography is the main diagnostic method, but it is highly subjective with severe variability. Recently, many deep-learning-based methods have been proposed to alleviate subjectivity and achieve good results yet, these methods often neglect important guidance from clinical knowledge. Our objective is to utilize such guidance for accurate and reliable thyroid nodule classification.Approach. In this study, a multi-task learning model embedded with clinical knowledge of ACR Thyroid Imaging, Reporting and Data System guideline is proposed. The clinical features defined in the guideline have strong correlations with malignancy and they were modeled as tasks alongside the pathological type. Multi-task learning was utilized to exploit the correlations to improve diagnostic performance. To alleviate the impact of noisy labels on clinical features, a loss-weighting strategy was proposed. Five-fold cross-validation was applied to an internal training set of size 4989, and an external test set of size 243 was used for evaluation.Main results. The proposed multi-task learning model achieved an average AUC of 0.901 and an ensemble AUC of 0.917 on the test set, which significantly outperformed the single-task baseline models.Significance. The results indicated that multi-task learning of clinical features can effectively classify thyroid nodules and reveal the possibility of using clinical indicators as auxiliary tasks to improve performance when diagnosing other diseases.

Diagnosis of cervical lymph node metastasis with thyroid carcinoma by deep learning application to CT images

Introduction

The incidence of thyroid diseases has increased in recent years, and cervical lymph node metastasis (LNM) is considered an important risk factor for locoregional recurrence. This study aims to develop a deep learning-based computer-aided diagnosis (CAD) method to diagnose cervical LNM with thyroid carcinoma on computed tomography (CT) images.

Methods

A new deep learning framework guided by the analysis of CT data for automated detection and classification of LNs on CT images is proposed. The presented CAD system consists of two stages. First, an improved region-based detection network is designed to learn pyramidal features for detecting small nodes at different feature scales. The region proposals are constrained by the prior knowledge of the size and shape distributions of real nodes. Then, a residual network with an attention module is proposed to perform the classification of LNs. The attention module helps to classify LNs in the fine-grained domain, improving the whole classification network performance.

Results

A total of 574 axial CT images (including 676 lymph nodes: 103 benign and 573 malignant lymph nodes) were retrieved from 196 patients who underwent CT for surgical planning. For detection, the data set was randomly subdivided into a training set (70%) and a testing set (30%), where each CT image was expanded to 20 images by rotation, mirror image, changing brightness, and Gaussian noise. The extended data set included 11,480 CT images. The proposed detection method outperformed three other detection architectures (average precision of 80.3%). For classification, ROI of lymph node metastasis labeled by radiologists were used to train the classification network. The 676 lymph nodes were randomly divided into 70% of the training set (73 benign and 401 malignant lymph nodes) and 30% of the test set (30 benign and 172 malignant lymph nodes). The classification method showed superior performance over other state-of-the-art methods with an accuracy of 96%, true positive and negative rates of 98.8 and 80%, respectively. It outperformed radiologists with an area under the curve of 0.894.

Discussion

The extensive experiments verify the high efficiency of the proposed method. It is considered instrumental in a clinical setting to diagnose cervical LNM with thyroid carcinoma using preoperative CT images. The future research can consider adding radiologists' experience and domain knowledge into the deep-learning based CAD method to make it more clinically significant.

Conclusion

The extensive experiments verify the high efficiency of the proposed method. It is considered instrumental in a clinical setting to diagnose cervical LNM with thyroid carcinoma using preoperative CT images.

Clinical value of artificial intelligence in thyroid ultrasound: a prospective study from the real world

OBJECTIVE

To evaluate the diagnostic performance of a commercial artificial intelligence (AI)-assisted ultrasonography (US) for thyroid nodules and to validate its value in real-world medical practice.

MATERIALS AND METHODS

From March 2021 to July 2021, 236 consecutive patients with 312 suspicious thyroid nodules were prospectively enrolled in this study. One experienced radiologist performed US examinations with a real-time AI system (S-Detect). US images and AI reports of the nodules were recorded. Nine residents and three senior radiologists were invited to make a "benign" or "malignant" diagnosis based on recorded US images without knowing the AI reports. After referring to AI reports, the diagnosis was made again. The diagnostic performance of AI, residents, and senior radiologists with and without AI reports were analyzed.

RESULTS

The sensitivity, accuracy, and AUC of the AI system were 0.95, 0.84, and 0.753, respectively, and were not statistically different from those of the experienced radiologists, but were superior to those of the residents (all p < 0.01). The AI-assisted resident strategy significantly improved the accuracy and sensitivity for nodules ≤ 1.5 cm (all p < 0.01), while reducing the unnecessary biopsy rate by up to 27.7% for nodules > 1.5 cm (p = 0.034).

CONCLUSIONS

The AI system achieved performance, for cancer diagnosis, comparable to that of an average senior thyroid radiologist. The AI-assisted strategy can significantly improve the overall diagnostic performance for less-experienced radiologists, while increasing the discovery of thyroid cancer ≤ 1.5 cm and reducing unnecessary biopsies for nodules > 1.5 cm in real-world medical practice.

KEY POINTS

• The AI system reached a senior radiologist-like level in the evaluation of thyroid cancer and could significantly improve the overall diagnostic performance of residents. • The AI-assisted strategy significantly improved ≤ 1.5 cm thyroid cancer screening AUC, accuracy, and sensitivity of the residents, leading to an increased detection of thyroid cancer while maintaining a comparable specificity to that of radiologists alone. • The AI-assisted strategy significantly reduced the unnecessary biopsy rate for thyroid nodules > 1.5 cm by the residents, while maintaining a comparable sensitivity to that of radiologists alone.

Orientation of the ultrasound probe to identify the taller-than-wide sign of thyroid malignancy: a registry-based study with the Thyroid Imaging Network of Korea

PURPOSE

Although the taller-than-wide (TTW) sign has been regarded as one of the most specific ultrasound (US) features of thyroid malignancy, uncertainty still exists regarding the US probe's orientation when evaluating it. This study investigated which US plane would be optimal to identify the TTW sign based on malignancy risk stratification using a registry-based imaging dataset.

METHODS

A previous study by 17 academic radiologists retrospectively analyzed the US images of 5,601 thyroid nodules (≥1 cm, 1,089 malignant and 4,512 benign) collected in the webbased registry of Thyroid Imaging Network of Korea through the collaboration of 26 centers. The present study assessed the diagnostic performance of the TTW sign itself and fine needle aspiration (FNA) indications via a comparison of four international guidelines, depending on the orientation of the US probe (criterion 1, transverse plane; criterion 2, either transverse or longitudinal plane).

RESULTS

Overall, the TTW sign was more frequent in malignant than in benign thyroid nodules (25.3% vs. 4.6%). However, the statistical differences between criteria 1 and 2 were negligible for sensitivity, specificity, and area under the curve (AUC) based on the size effect (all P<0.05, Cohen's d=0.19, 0.10, and 0.07, respectively). Moreover, the sensitivity, specificity, and AUC of the four FNA guidelines were similar between criteria 1 and 2 (all P>0.05, respectively).

CONCLUSION

A longitudinal US probe orientation provided little additional diagnostic value over the transverse orientation in detecting the TTW sign of thyroid nodules.

Deep learning for the diagnosis of suspicious thyroid nodules based on multimodal ultrasound images

OBJECTIVES

This study aimed to differentially diagnose thyroid nodules (TNs) of Thyroid Imaging Reporting and Data System (TI-RADS) 3-5 categories using a deep learning (DL) model based on multimodal ultrasound (US) images and explore its auxiliary role for radiologists with varying degrees of experience.

METHODS

Preoperative multimodal US images of 1,138 TNs of TI-RADS 3-5 categories were randomly divided into a training set (n = 728), a validation set (n = 182), and a test set (n = 228) in a 4:1:1.25 ratio. Grayscale US (GSU), color Doppler flow imaging (CDFI), strain elastography (SE), and region of interest mask (Mask) images were acquired in both transverse and longitudinal sections, all of which were confirmed by pathology. In this study, fivefold cross-validation was used to evaluate the performance of the proposed DL model. The diagnostic performance of the mature DL model and radiologists in the test set was compared, and whether DL could assist radiologists in improving diagnostic performance was verified. Specificity, sensitivity, accuracy, positive predictive value, negative predictive value, and area under the receiver operating characteristics curves (AUC) were obtained.

RESULTS

The AUCs of DL in the differentiation of TNs were 0.858 based on (GSU + SE), 0.909 based on (GSU + CDFI), 0.906 based on (GSU + CDFI + SE), and 0.881 based (GSU + Mask), which were superior to that of 0.825-based single GSU (p = 0.014, p< 0.001, p< 0.001, and p = 0.002, respectively). The highest AUC of 0.928 was achieved by DL based on (G + C + E + M)US, the highest specificity of 89.5% was achieved by (G + C + E)US, and the highest accuracy of 86.2% and sensitivity of 86.9% were achieved by DL based on (G + C + M)US. With DL assistance, the AUC of junior radiologists increased from 0.720 to 0.796 (p< 0.001), which was slightly higher than that of senior radiologists without DL assistance (0.796 vs. 0.794, p > 0.05). Senior radiologists with DL assistance exhibited higher accuracy and comparable AUC than that of DL based on GSU (83.4% vs. 78.9%, p = 0.041; 0.822 vs. 0.825, p = 0.512). However, the AUC of DL based on multimodal US images was significantly higher than that based on visual diagnosis by radiologists (p< 0.05).

CONCLUSION

The DL models based on multimodal US images showed exceptional performance in the differential diagnosis of suspicious TNs, effectively increased the diagnostic efficacy of TN evaluations by junior radiologists, and provided an objective assessment for the clinical and surgical management phases that follow.

Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry

Thyroid volumetry is crucial in the diagnosis, treatment, and monitoring of thyroid diseases. However, conventional thyroid volumetry with 2D ultrasound is highly operator-dependent. This study compares 2D and tracked 3D ultrasound with an automatic thyroid segmentation based on a deep neural network regarding inter- and intraobserver variability, time, and accuracy. Volume reference was MRI. 28 healthy volunteers (24—50 a) were scanned with 2D and 3D ultrasound (and by MRI) by three physicians (MD 1, 2, 3) with different experience levels (6, 4, and 1 a). In the 2D scans, the thyroid lobe volumes were calculated with the ellipsoid formula. A convolutional deep neural network (CNN) automatically segmented the 3D thyroid lobes. 26, 6, and 6 random lobe scans were used for training, validation, and testing, respectively. On MRI (T1 VIBE sequence) the thyroid was manually segmented by an experienced MD. MRI thyroid volumes ranged from 2.8 to 16.7ml (mean 7.4, SD 3.05). The CNN was trained to obtain an average Dice score of 0.94. The interobserver variability comparing two MDs showed mean differences for 2D and 3D respectively of 0.58 to 0.52ml (MD1 vs. 2), −1.33 to −0.17ml (MD1 vs. 3) and −1.89 to −0.70ml (MD2 vs. 3). Paired samples t-tests showed significant differences for 2D (p = .140, p = .002 and p = .002) and none for 3D (p = .176, p = .722 and p = .057). Intraobsever variability was similar for 2D and 3D ultrasound. Comparison of ultrasound volumes and MRI volumes showed a significant difference for the 2D volumetry of all MDs (p = .002, p = .009, p <.001), and no significant difference for 3D ultrasound (p = .292, p = .686, p = 0.091). Acquisition time was significantly shorter for 3D ultrasound. Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.

Optimal localization strategies for non-palpable breast cancers –A network meta-analysis of randomized controlled trials

Purpose

Mammographic screening programmes have increased detection rates of non-palpable breast cancers. In these cases, wire-guided localization (WGL) is the most common approach used to guide breast conserving surgery (BCS). Several RCTs have compared WGL to a range of novel localization techniques. We aimed to perform a network meta-analysis (NMA) of randomized controlled trials (RCTs) comparing methods of non-palpable breast cancer localization.

Methods

A NMA was performed according to PRISMA-NMA guidelines. Analysis was performed using R packages and Shiny.

Results

24 RCTs assessing 9 tumour localization methods in 4236 breasts were included. Margin positivity and reoperation rates were 16.9% (714/4236) and 14.3% (409/2870) respectively. Cryo-assisted localization had the highest margin positivity (28.2%, 58/206) and reoperation (18.9%, 39/206) rates. Compared to WGL (n = 2045 from 24 RCTs) only ultrasound guided localization (USGL) (n = 316 from 3 RCTs) significantly lowered margin positivity (odds ratio (OR): 0.192, 95% confidence interval (CI): 0.079–0.450) and reoperation rates (OR: 0.182, 95%CI: 0.069–0.434). Anchor-guided localization (n = 52, 1 RCT) significantly lowered margin positivity (OR: 0.229, 95%CI: 0.050–0.938) and magnetic-marker localization improved patient satisfaction (OR: 0.021, 95%CI: 0.001–0.548). There was no difference in operation duration, overall complications, haematoma, seroma, surgical site infection rates, or specimen size/vol/wt between methods.

Conclusion

USGL and AGL are non-inferior to WGL for the localization of non-palpable breast cancers. The reported data suggests that these techniques confer reduced margin positivity rates and requirement for re-operation. However, caution when interpreting results relating to RCTs with small sample sizes and further validation is required in larger prospective, randomized studies.

•

Ultrasound-guided (USGL) and anchor-guided (AGL) localization had optimal outcomes.

•

These methods significantly lowered margin positivity (odds ratio: 0.192 & 0.229).

•

However, small sample sizes in trials evaluating USGL and AGL limit these results.

•

Operation duration, complications, or specimen data were comparable for all methods.

An Artificial Intelligence Model Based on ACR TI-RADS Characteristics for US Diagnosis of Thyroid Nodules

Background US-based diagnosis of thyroid nodules is subjective and influenced by radiologists' experience levels. Purpose To develop an artificial intelligence model based on American College of Radiology Thyroid Imaging Reporting and Data System characteristics for diagnosing thyroid nodules and identifying nodule characteristics (hereafter, M_TI-RADS) and to compare the performance of M_TI-RADS, radiologists, and a model trained on benign and malignant status based on surgical histopathologic analysis (hereafter, M_Diag). Materials and Methods In this retrospective study, 1588 surgically proven nodules from 636 consecutive patients (mean age, 49 years ± 14 [SD]; 485 women) were included. M_TI-RADS and M_Diag were trained on US images of 1345 nodules (January 2018 to December 2019). The performance of M_TI-RADS was compared with that of M_Diag and radiologists with different experience levels on the test data set (243 nodules, January 2019 to December 2019) with the DeLong method and McNemar test. Results The area under the receiver operating characteristic curve (AUC) and sensitivity of M_TI-RADS were 0.91 and 83% (55 of 66 nodules), respectively, which were not significantly different from those of experienced radiologists (0.93 [P = .45] and 92% [61 of 66 nodules; P = .07]) and exceeded those of junior radiologists (0.78 [P < .001] and 70% [46 of 66 nodules; P = .04]). The specificity of M_TI-RADS (87% [154 of 177 nodules]) was higher than that of both experienced and junior radiologists (80% [141 of 177 nodules; P = .02] and 75% [133 of 177 nodules; P = .001], respectively). The AUC of M_TI-RADS was higher than that of M_Diag (0.91 vs 0.84, respectively; P = .001). In the test set of 243 nodules, the consistency rates between M_TI-RADS and the experienced group were higher than those between M_TI-RADS and the junior group for composition (79% [n = 193] vs 73% [n = 178], respectively; P = .02), echogenicity (75% [n = 183] vs 68% [n = 166]; P = .04), shape (93% [n = 227] vs 88% [n = 215]; P = .04), and smooth or ill-defined margin (72% [n = 174] vs 63% [n = 152]; P = .002). Conclusion The area under the receiver operating characteristic curve (AUC) of an artificial intelligence model based on the American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS) was higher than that of a model trained on benign and malignant status based on surgical histopathologic analysis. The AUC and sensitivity of the model based on TI-RADS exceeded those of junior radiologists; the specificity of the model was higher than that of both experienced and junior radiologists. © RSNA, 2022.

Contrast-enhanced ultrasound is a reliable and reproducible assessment of necrotic ablated volume after radiofrequency ablation for benign thyroid nodules: a retrospective study

PURPOSE

To investigate the intra- and inter-observer reliability and agreement of contrast-enhanced ultrasound (CEUS) in measuring ablated volume (Va) after radiofrequency ablation (RFA) for benign thyroid nodules.

MATERIALS

This retrospective study evaluated 65 patients with 74 benign thyroid nodules who underwent RFA. Patients were followed up at 1, 3, 6, and 12 months and every 12 months thereafter. Two independent observers measured the Va using CEUS during the same follow-up visit. The intra- and inter-observer reliability was assessed using intraclass correlation coefficient (ICC) with 95% confidence interval. The Bland-Altman analysis was used to evaluate the inter-observer agreement, which was expressed as a mean difference with 95% limit of agreement (LOA).

RESULTS

No significant difference was found in Va measurements by the two observers with a mean follow-up time of 41.17 ± 16.80 months (all p > 0.05). The intra- and inter-observer reliability were both excellent (ICC >0.90) at each follow-up period. The 95% LOA became wider over the follow-up period. The smallest 95% LOA was found at 1 month with a LOA from 0.8117 to 1.122, and the largest 95% LOA was from 0.5694 to 1.343 at 36 months.

CONCLUSIONS

CEUS could provide a reliable and reproducible assessment of Va after RFA for benign thyroid nodules. In clinical post-ablation follow-up, the irregular morphology of ablated area and the variation by different observers could not affect the assessment of Va by CEUS.

A deep learning-based method for detecting and classifying the ultrasound images of suspicious thyroid nodules

PURPOSE

The incidence of thyroid cancer has significantly increased in the last few decades. However, diagnosis of the thyroid nodules is labor and time intensive for radiologists and strongly depends on the personal experience of the radiologists. In this pursuit, the present study envisaged to develop a deep learning-based computer-aided diagnosis (CAD) method that enabled the automatic detection and classification of suspicious thyroid nodules in order to reduce the unnecessary fine-needle aspiration biopsy.

METHODS

The CAD method consisted of two main parts: detecting the location of thyroid nodules using a multiscale detection network and classifying the detected thyroid nodules by an attention-based classification network.

RESULTS

The performance of the proposed method was evaluated and compared with that of other state-of-the-art deep learning methods and experienced radiologists. The proposed detection method outperformed three other detection architectures (average precision, 82.1% vs. 78.3%, 77.2%, and 74.8%). Moreover, the classification method showed a superior performance compared with four other state-of-the-art classification networks (accuracy, 94.8% vs. 91.2%, 85.0%, 80.8%, and 72.1%) and that by experienced radiologists (mean value of area under the curve, 0.941 vs. 0.833).

CONCLUSIONS

Our study verified the high efficiency of the proposed detection method. The findings can help improve the diagnostic performance of radiologists. However, the developed CAD system requires more training and evaluation in a large-population study.

Ultrasound in active surveillance for low-risk papillary thyroid cancer: imaging considerations in case selection and disease surveillance

Active surveillance (AS) of small, low-risk papillary thyroid cancers (PTCs) is increasingly studied in prospective observational studies. Ultrasound is the primary imaging modality for case selection. While researchers have put forward selection criteria for PTCs based on size, absence of suspicious lymph nodes and tumor location, there are limited reported data highlighting inherent ultrasound limitations and guidelines for case selection and follow-up. We report our experience including imaging limitations encountered in the ongoing AS prospective observational study for PTCs measuring < 2 cm at our institute. We define disease progression as an increase in size of > 3 mm in the largest dimension of nodule or evidence of metastatic disease or extrathyroidal extension. Accurate, consistent and reproducible measurements of PTCs are essential in risk stratifying patients for the option of AS or disease progression. Interobserver discrepancy, shadowing from coarse calcification and background parenchyma heterogeneity or thyroiditis can limit accurate PTC size assessment and therefore hinder patient eligibility evaluation or AS follow-up. Following the ACR Thyroid Imaging, Reporting and Data System (TI-RADS) protocol of three-axes technique to measure a thyroid nodule enables reproducibility of measurements. In patients with multi-nodular goiter, accurate identification and labeling of the PTC is important to avoid mistaking with adjacent benign nodules at follow-up. Ultrasound assessment for extrathyroid extension of PTC, and relationship of PTC to trachea and the anatomic course of the recurrent laryngeal nerve are important considerations in evaluation for AS eligibility.

Standardization of thyroid ultrasound reporting in the community setting decreases biopsy rates

OBJECTIVE

With the rising incidence of thyroid cancer, a standardized approach to the evaluation of thyroid nodules is essential. Despite the presence of multiple national guidelines detailing evaluation and management of these nodules, significant variability exists in the information that is collected and reported to clinicians from diagnostic imaging. The aim of this study was to evaluate the impact of thyroid ultrasound standardization on thyroid cancer detection in a community practice setting.

DESIGN

As part of a physician-driven quality improvement project, a multidisciplinary team created an electronic worksheet to be utilized by sonographers to capture suspicious findings based on societal guidelines and agreed on institutional criteria for recommending fine needle aspiration (FNA) of thyroid nodules.

PATIENTS

For a one-year period prior to and after the intervention, all ultrasounds performed for suspected thyroid pathology, excluding patients undergoing follow-up imaging, were reviewed at two affiliated community hospitals served by a single radiology and pathology group.

MEASUREMENTS

The number of fine needle biopsies recommended and performed, as well as the percentage of FNAs positive for malignancy were evaluated.

RESULTS

A total of 608 and 675 ultrasounds were reviewed in pre- and post-standardization periods, respectively. Following standardization, there was a similar percentage of FNAs recommended (35% vs. 37%, p = .68), fewer FNAs per total ultrasounds performed (36% vs. 31%, p = .03), fewer FNAs performed when FNA was not explicitly recommended (9.9% vs. 2.8%, p = .000046) and an increased detection of cytology consistent with, or suspicious for, malignancy (5% vs. 11.5%, p = .0028).

CONCLUSIONS

Standardization of thyroid imaging protocol and management recommendations can reduce the number of FNAs performed and increase the percentage of positive tests in a community setting.

Anterior neck soft tissue measurements on computed tomography to predict difficult laryngoscopy: a retrospective study

Predicting difficult laryngoscopy is an essential component of the airway management. We aimed to evaluate the use of anterior neck soft tissue measurements on computed tomography for predicting difficult laryngoscopy and to present a clear measurement protocol. In this retrospective study, 281 adult patients whose tracheas were intubated using a direct laryngoscope for thyroidectomy were enrolled. On computed tomography, the distances from the midpoint of the thyrohyoid membrane to the closest concave point of the vallecular (membrane-to-vallecula distance; dMV), and to the most distant point of the epiglottis (membrane-to-epiglottis distance; dME) were measured, respectively. The extended distances straight to the skin anterior from the dMV and dME were called the skin-to-vallecula distance (dSV) and skin-to-epiglottis distance (dSE), respectively. Difficult laryngoscopy was defined by a Cormack-Lehane grade of > 2. Difficult laryngoscopy occurred in 40 (14%) cases. Among four indices, the dMV showed the highest prediction ability for difficult laryngoscopy with an area under the receiver operating characteristic curve of 0.884 (95% confidence interval 0.841–0.919, P < 0.001). The optimal dMV cut-off value for predicting difficult laryngoscopy was 2.33 cm (sensitivity 75.0%; specificity 93.8%). The current study provides novel evidence that increased dMV is a potential predictive indicator of difficult laryngoscopy.

An efficient deep convolutional neural network model for visual localization and automatic diagnosis of thyroid nodules on ultrasound images

Background

The aim of this study was to construct a deep convolutional neural network (CNN) model for localization and diagnosis of thyroid nodules on ultrasound and evaluate its diagnostic performance.

Methods

We developed and trained a deep CNN model called the Brief Efficient Thyroid Network (BETNET) using 16,401 ultrasound images. According to the parameters of the model, we developed a computer-aided diagnosis (CAD) system to localize and differentiate thyroid nodules. The validation dataset (1,000 images) was used to compare the diagnostic performance of the model using three state-of-the-art algorithms. We used an internal test set (300 images) to evaluate the BETNET model by comparing it with diagnoses from five radiologists with varying degrees of experience in thyroid nodule diagnosis. Lastly, we demonstrated the general applicability of our artificial intelligence (AI) system for diagnosing thyroid cancer in an external test set (1,032 images).

Results

The BETNET model accurately detected thyroid nodules in visualization experiments. The model demonstrated higher values for area under the receiver operating characteristic (AUC-ROC) curve [0.983, 95% confidence interval (CI): 0.973-0.990], sensitivity (99.19%), accuracy (98.30%), and Youden index (0.9663) than the three state-of-the-art algorithms (P<0.05). In the internal test dataset, the diagnostic accuracy of the BETNET model was 91.33%, which was markedly higher than the accuracy of one experienced (85.67%) and two less experienced radiologists (77.67% and 69.33%). The area under the ROC curve of the BETNET model (0.951) was similar to that of the two highly skilled radiologists (0.940 and 0.953) and significantly higher than that of one experienced and two less experienced radiologists (P<0.01). The kappa coefficient of the BETNET model and the pathology results showed good agreement (0.769). In addition, the BETNET model achieved an excellent diagnostic performance (AUC =0.970, 95% CI: 0.958-0.980) when applied to ultrasound images from another independent hospital.

Conclusions

We developed a deep learning model which could accurately locate and automatically diagnose thyroid nodules on ultrasound images. The BETNET model exhibited better diagnostic performance than three state-of-the-art algorithms, which in turn performed similarly in diagnosis as the experienced radiologists. The BETNET model has the potential to be applied to ultrasound images from other hospitals.

Interobserver Reproducibility in Sonographic Measurement of Diameter and Volume of Papillary Thyroid Microcarcinoma

Background: Active surveillance is recommended as an alternative to immediate surgery for low-risk papillary thyroid microcarcinoma (PTMC), and determining meaningful changes in diameter and volume on ultrasonography (US) is critical. However, interobserver reproducibility of the sonographic measurement of maximum diameter and volume of PTMC has not been well established. We aimed to determine the reproducibility in the measurement of maximum diameter and volume of PTMC on US. Methods: Consecutive patients who underwent US for pathologically proven PTMC between December 2018 and December 2019 were retrospectively reviewed. Two observers independently performed sonographic measurement of each nodule using standardized measurement methods. Each observer measured maximum transverse, anteroposterior, and longitudinal nodule diameters, and using these, nodule volume was calculated using the ellipsoid formula. Interobserver reproducibility in the measurement of the maximum diameter and volume was assessed using percentage reproducibility coefficient (RC). Z-tests of the intraclass correlation coefficients (ICCs) were used to compare the interobserver reproducibility in subgroups defined according to sonographic characteristics, such as the presence of microcalcification, nodule size, and parenchymal heterogeneity. Results: A total of 197 thyroid nodules from 188 patients were included in the study series. The percentage RCs were 71.8% [95% confidence interval, CI 65.4-79.7%] and 23.7% [CI 21.6-26.3%] for volume and maximum diameter measurements, respectively. There were no significant differences noted in the ICC values according to nodule orientation, presence of calcifications, size, or parenchymal heterogeneity. Conclusion: For PTMC, a difference of up to 24% in the maximum diameter and 72% in the volume may be considered to be within measurement error on US. This value may be used to determine the cutoff for defining meaningful change in the maximum diameter and volume for PTMC during active surveillance.

Ultrasound Cine Loop Standard Operating Procedure for Benign Thyroid Diseases-Evaluation of Non-Physician Application

Conventional ultrasound (US) is time-consuming, and results are subjected to high interobserver variability. In this study, the reliability of a novel thyroid US cine loop standard operating procedure (SOP) applied by non-physicians (Medical Technical Assistant, MTA) is investigated. Thirty-three consecutive patients (22 females, 11 males) were enrolled. Patients underwent conventional thyroid US performed by a nuclear medicine physician and additional MTA US cine loop according to a local SOP that includes transversal and sagittal cine loops covering the entire thyroid. The video sequences were transferred to the Picture Archiving and Communication System (PACS) for second reading purposes. MTA US data were not considered for medical reports but for blinded second reading review of the PACS images. The results of conventional physician US reports and reviewed MTA US cine loops were compared regarding size determinations of the thyroid and its nodules, as well as Thyroid Imaging Reporting and Data Systems (TIRADS) classification of all identified lesions. The results revealed very high concordance between conventional physician US and MTA US cine loop review for both size measurements and TIRADS classifications (r_(s) = 0.84-0.99, p < 0.0001 each). Minor technical impairments were identified. The evaluated thyroid US cine loop SOP enables reliable second reading results and can be applied by non-physicians.

A Preliminary Study of Quantitative Ultrasound for Cancer-Risk Assessment of Thyroid Nodules

BACKGROUND

Gray-scale, B-mode ultrasound (US) imaging is part of the standard clinical procedure for evaluating thyroid nodules (TNs). It is limited by its instrument- and operator-dependence and inter-observer variability. In addition, the accepted high-risk B-mode US TN features are more specific for detecting classic papillary thyroid cancer rather than the follicular variant of papillary thyroid cancer or follicular thyroid cancer. Quantitative ultrasound (QUS) is a technique that can non-invasively assess properties of tissue microarchitecture by exploiting information contained in raw ultrasonic radiofrequency (RF) echo signals that is discarded in conventional B-mode imaging. QUS provides quantitative parameter-value estimates that are a function of the properties of US scatterers and microarchitecture of the tissue. The purpose of this preliminary study was to assess the performance of QUS parameters in evaluating benign and malignant thyroid nodules.

METHODS

Patients from the Thyroid Health Center at the Boston Medical Center were recruited to participate. B-mode and RF data were acquired and analyzed in 225 TNs (24 malignant and 201 benign) from 208 patients. These data were acquired either before (167 nodules) or after (58 nodules) subjects underwent fine-needle biopsy (FNB). The performance of a combination of QUS parameters (CQP) was assessed and compared with the performance of B-mode risk-stratification systems.

RESULTS

CQP produced an ROC AUC value of 0.857 ± 0.033 compared to a value of 0.887 ± 0.033 (p=0.327) for the American College of Radiology Thyroid Imaging, Reporting and Data System (ACR TI-RADS) and 0.880 ± 0.041 (p=0.367) for the American Thyroid Association (ATA) risk-stratification system. Furthermore, using a CQP threshold of 0.263 would further reduce the number of unnecessary FNBs in 44% of TNs without missing any malignant TNs. When CQP used in combination with ACR TI-RADS, a potential additional reduction of 49 to 66% in unnecessary FNBs was demonstrated.

CONCLUSION

This preliminary study suggests that QUS may provide a method to classify TNs when used by itself or when combined with a conventional gray-scale US risk-stratification system and can potentially reduce the need to biopsy TNs.

Inter-observer reliability in ultrasound measurement of benign thyroid nodules in the follow-up of radiofrequency ablation: a retrospective study

PURPOSE

To investigate the inter-observer reliability of ultrasound (US) measurement in volume and volume reduction rate (VRR) of benign thyroid nodules during the follow-up of radiofrequency ablation (RFA).

MATERIALS

This retrospective study evaluated 76 patients with 88 benign thyroid nodules who underwent RFA. Two independent observers performed the US measurements during the same follow-up visit. The Bland-Altman analysis was used to evaluate the inter-observer reliability of volume and VRR (logarithmic transformation). The mean difference and 95% limits-of-agreement (LOA) were obtained after antilogarithm.

RESULTS

No significant differences were found in the volume and VRR measurements during a mean follow-up time of 35.52 ± 12.82 months. After antilogarithms, the mean difference of volume was 0.9961, 0.9987, 1.0016, 0.9972, 0.9977 and 0.9969 at 1, 3, 6, 12, 24 and 36 months, respectively. The 95% LOA of volume became wider over the follow-up period, and the largest one was between 0.8471 and 1.1733 at 36 months. The 95% LOA of VRR became narrower over the follow-up period, and the largest one was between 0.9541 and 1.0469 at 1 month. The incidence of regrowth was 20.45% and the largest 95% LOA of regrowth nodules was between 0.9028 and 1.284 at 12 months. At the same follow-up period, VRR had a narrower 95% LOA than volume. Compared with volume ≥10 ml, nodules <10 ml revealed a larger 95% LOA in both the volume and VRR.

CONCLUSION

The inter-observer reliability of the US measurements of benign thyroid nodules during the follow-up period of RFA was acceptable. The variation by different observers could not affect the evaluation of efficacy.

2020 Chinese guidelines for ultrasound malignancy risk stratification of thyroid nodules: the C-TIRADS

Thyroid nodules are very common all over the world, and China is no exception. Ultrasound plays an important role in determining the risk stratification of thyroid nodules, which is critical for clinical management of thyroid nodules. For the past few years, many versions of TIRADS (Thyroid Imaging Reporting and Data System) have been put forward by several institutions with the aim to identify whether nodules require fine-needle biopsy or ultrasound follow-up. However, no version of TIRADS has been widely adopted worldwide till date. In China, as many as ten versions of TIRADS have been used in different hospitals nationwide, causing a lot of confusion. With the support of the Superficial Organ and Vascular Ultrasound Group of the Society of Ultrasound in Medicine of the Chinese Medical Association, the Chinese-TIRADS that is in line with China's national conditions and medical status was established based on literature review, expert consensus, and multicenter data provided by the Chinese Artificial Intelligence Alliance for Thyroid and Breast Ultrasound.

Non-enhanced ultrasound is not a satisfactory modality for measuring necrotic ablated volume after radiofrequency ablation of benign thyroid nodules: a comparison with contrast-enhanced ultrasound

OBJECTIVE

To investigate the intra- and inter-observer reliability and agreement between gray-scale and Doppler ultrasound (US) and contrast-enhanced ultrasound (CEUS) in measuring ablated volume (Va) after radiofrequency ablation (RFA) for benign thyroid nodules.

METHODS

A total of 173 patients with 190 benign thyroid nodules who underwent RFA were included in this study. After RFA, the total volume of ablated nodule was divided into Va and the incompletely treated vital volume. Patients were followed up at 1, 3, 6, 12 months, and every 12 months thereafter. Two independent observers measured Va using US and CEUS during the same follow-up visit. The intra- and inter-observer reliability of the two measurement modalities was assessed using intraclass correlation coefficient (ICC) with 95% confidence interval. The Bland-Altman analysis was used to evaluate agreement, which was expressed as a mean difference with 95% limits of agreement (LOA).

RESULTS

The mean follow-up time was 23.17 ± 12.70 months. Va measured by US was significantly larger than by CEUS (p < 0.001). The intra- and inter-observer reliability decreased over the follow-up period and became moderate in both subgroups at 12 months (all ICC < 0.75). The mean difference and LOA became larger and wider during the follow-up. The best agreement was found in nodules < 10 ml at 1 month with a mean difference of 1.166 and LOA between 0.413 and 3.294.

CONCLUSIONS

The intra- and inter-observer reliability and agreement of US and CEUS in measuring Va were unsatisfactory. CEUS should be considered when Va was needed for further evaluation or in the case of nodules with suspected regrowth.

KEY POINTS

• Va measured by gray-scale and Doppler US was significantly larger than that by CEUS. • Va measured by gray-scale and Doppler US lacked intra- and inter-observer reliability and agreement with CEUS. • CEUS should be preceded to gray-scale and Doppler US for the measurement of Va.

Ultrasonography for the Prediction of High-Volume Lymph Node Metastases in Papillary Thyroid Carcinoma: Should Surgeons Believe Ultrasound Results?

Background

Lymph node metastasis (LNM) often occurs in papillary thyroid carcinoma (PTC); the efficacy of ultrasound for predicting high-volume lymph node metastases (LNMs) in patients with PTC remains unexplored.

Methods

The medical records of 2073 consecutive PTC patients were reviewed. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated to evaluate the efficacy of ultrasound. Risk factors for LNM/high-volume LNMs and lymph node involvement on ultrasound (usLNM) were identified by univariate and multivariate analyses.

Results

Of all the patients, 936 (45.2%) patients had LNMs, and 254 (12.3%) patients had high-volume LNMs. The sensitivity of ultrasound for detecting LNM/high-volume LNMs was 27.9% and 63.8%, respectively; the specificity was 93.1% and 90.3%, respectively. The NPV for ultrasound in detecting high-volume LNMs was 94.7%. In multivariate analysis, male sex (OR = 2.108, p < 0.001), tumor diameter > 1.0 cm (OR = 2.304, p < 0.001) and usLNM (+) (OR = 12.553, p < 0.001) were independent clinical risk factors for high-volume LNMs. Tumor diameter > 1 cm (OR = 3.036, p < 0.001) and male sex (OR = 1.642, p < 0.001) were independent clinical risk factors for usLNM; a skilled sonographer (OR = 1.121, p = 0.358) was not significantly associated with usLNM.

Conclusions

Lymph node involvement found by ultrasound has great predictive value for high-volume LNMs; the NPV is very high for patients without lymph node involvement on ultrasound. The ultrasound results do not appear to be influenced by the experience of the sonographer.

Electronic supplementary material

The online version of this article (10.1007/s00268-020-05755-0) contains supplementary material, which is available to authorized users.

Evaluation of a deep learning-based computer-aided diagnosis system for distinguishing benign from malignant thyroid nodules in ultrasound images

PURPOSE

Computer-aided diagnosis (CAD) systems assist in solving subjective diagnosis problems that typically rely on personal experience. A CAD system has been developed to differentiate malignant thyroid nodules from benign thyroid nodules in ultrasound images based on deep learning methods. The diagnostic performance was compared between the CAD system and the experienced attending radiologists.

METHODS

The ultrasound image dataset for training the CAD system included 651 malignant nodules and 386 benign nodules while the database for testing included 422 malignant nodules and 128 benign nodules. All the nodules were confirmed by pathology results. In the proposed CAD system, a support vector machine (SVM) is used for classification and fused features which combined the deep features extracted by a convolutional neural network (CNN) with the hand-crafted features such as the histogram of oriented gradient (HOG), local binary patterns (LBP), and scale invariant feature transform (SIFT) were obtained. The optimal feature subset was formed by selecting these fused features based on the maximum class separation distance and used as the training sample for the SVM.

RESULTS

The accuracy, sensitivity, and specificity of the CAD system were 92.5%, 96.4%, and 83.1%, respectively, which were higher than those of the experienced attending radiologists. The areas under the ROC curves of the CAD system and the attending radiologists were 0.881 and 0.819, respectively.

CONCLUSIONS

The CAD system for thyroid nodules exhibited a better diagnostic performance than experienced attending radiologists. The CAD system could be a reliable supplementary tool to diagnose thyroid nodules using ultrasonography. Macroscopic features in ultrasound images, such as the margins and shape of thyroid nodules, could influence the diagnostic efficiency of the CAD system.

Accuracy of thyroid imaging reporting and data system category 4 or 5 for diagnosing malignancy: a systematic review and meta-analysis

OBJECTIVES

To determine the accuracies of the American College of Radiology (ACR)-thyroid imaging reporting and data systems (TIRADS), Korean (K)-TIRADS, and European (EU)-TIRADS for diagnosing malignancy in thyroid nodules.

METHODS

Original studies reporting the diagnostic accuracy of TIRADS for determining malignancy on ultrasound were identified in MEDLINE and EMBASE up to June 23, 2019. The meta-analytic summary sensitivity and specificity were obtained for TIRADS category 5 (TR-5) and category 4 or 5 (TR-4/5), using a bivariate random effects model. To explore study heterogeneity, meta-regression analyses were performed.

RESULTS

Of the 34 eligible articles (37,585 nodules), 25 used ACR-TIRADS, 12 used K-TIRADS, and seven used EU-TIRADS. For TR-5, the meta-analytic sensitivity was highest for EU-TIRADS (78% [95% confidence interval, 64-88%]), followed by ACR-TIRADS (70% [61-79%]) and K-TIRADS (64% [58-70%]), although the differences were not significant. K-TIRADS showed the highest meta-analytic specificity (93% [91-95%]), which was similar to ACR-TIRADS (89% [85-92%]) and EU-TIRADS (89% [77-95%]). For TR-4/5, all three TIRADS systems had sensitivities higher than 90%. K-TIRADS had the highest specificity (61% [50-72%]), followed by ACR-TIRADS (49% [43-56%]) and EU-TIRADS (48% [35-62%]), although the differences were not significant. Considerable threshold effects were noted with ACR- and K-TIRADS (p ≤ 0.01), with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity (p ≤ 0.05).

CONCLUSIONS

There was no significant difference among these three international TIRADS, but the trend toward higher sensitivity with EU-TIRADS and higher specificity with K-TIRADS.

KEY POINTS

• For TIRADS category 5, the meta-analytic sensitivity was highest for the EU-TIRADS, followed by the ACR-TIRADS and the K-TIRADS, although the differences were not significant. • For TIRADS category 5, K-TIRADS showed the highest meta-analytic specificity, which was similar to ACR-TIRADS and EU-TIRADS. • Considerable threshold effects were noted with ACR- and K-TIRADS, with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity.

Taller-Than-Wide Shape: A New Definition Improves the Specificity of TIRADS Systems

INTRODUCTION

A taller-than-wide (TTW) shape is a suspicious feature of thyroid nodules commonly defined as an anteroposterior/transverse diameter (AP/T) ratio >1. An intraobserver variability of up to 18% in AP diameter evaluations has been described, which may lead to overreporting of this feature. To potentially improve the reliability of the TTW definition, we propose an arbitrary ratio of ≥1.2.

OBJECTIVE

The aim of this study was to estimate the impact of this definition on diagnostic performance.

METHODS

We prospectively analyzed 553 thyroid nodules referred for cytology evaluation at an academic center. Before fine-needle aspiration, two examiners jointly defined all sonographic features considered in risk stratification systems developed by the American Thyroid Association (ATA), the American Association of Clinical Endocrinologists (AACE), the American College of Radiology (ACR TIRADS), the European Thyroid Association (EU-TIRADS), and the Korean Society of Thyroid Radiology (K-TIRADS). TTW was defined according to the current definition (AP/T diameter ratio >1) and an arbitrary alternative definition (AP/T ratio >1.2).

RESULTS

The alternative definition classified fewer nodules as TTW (28, 5.1% vs. 94, 17%). The current and proposed definitions have a sensitivity of 26.2 and 11.9% (p = 0.03) and a specificity of 83.8 and 95.5% (p < 0.001). Thus, as a single feature, the arbitrary definition has a lower sensitivity and a higher specificity. When applied to sonographic risk stratification systems, however, the proposed definition would increase the number of avoided biopsies (up to 58.2% for ACR TIRADS) and the specificity of all systems, without negative impact on sensitivity or diagnostic odds ratio.

CONCLUSIONS

Re-defining TTW nodules as those with an AP/T ratio ≥1.2 improves this marker's specificity for malignancy. Using this definition in risk stratification systems will increase their specificity, reducing the number of suggested biopsies without significantly diminishing their overall diagnostic performance.

Intraobserver and interobserver reliability in sonographic size measurements of gallbladder polyps

OBJECTIVES

To evaluate the intraobserver and interobserver reliability of gallbladder polyp measurements using transabdominal US and the factors that affect reliability.

METHODS

From November 2017 to February 2018, two radiologists measured the maximum diameter of 91 gallbladder polyps using transabdominal US. Intraobserver and interobserver agreement were determined using 95% Bland-Altman limits of agreement and intraclass correlation coefficients (ICCs). The effects of image settings, polyp location, and polyp size were evaluated by comparing ICCs using z tests.

RESULTS

The intraobserver agreement rates were 0.960 (95% confidence interval [CI], 0.939-0.973) for observer 1 and 0.962 (95% CI, 0.943-0.975) for observer 2. The ICCs between the two observers were 0.963 (95% CI, 0.926-0.979) for the first measurement and 0.973 (95% CI, 0.950-0.984) for the second measurement. The 95% limits of agreement on repeated measurements were 22.3-25.2% of the mean, and those between the two observers were 25.5-34.2% of the mean. ICCs for large polyps (≥ 5 mm) were significantly higher than those for small polyps (< 5 mm). There were no significant differences in the ICCs between image settings and polyp location.

CONCLUSIONS

Polyp size measurements using transabdominal US are highly repeatable and reproducible. Polyp size significantly affects the reliability of measurement. Diameter changes of approximately less than 25% may fall within the measurement error; this should be considered while interpreting the change in size during follow-up US, especially for small polyps.

KEY POINTS

• Gallbladder polyp size measurement using transabdominal US is highly repeatable and reproducible. • Diameter changes of approximately less than 25% should be interpreted carefully, especially in small polyps.

Effect of the location and size of thyroid nodules on the diagnostic performance of ultrasound elastography: A retrospective analysis

OBJECTIVES

Ultrasound-guided fine-needle aspiration biopsies are recommended for the detection of suspicious thyroid nodules. However, the best approach regarding suspicious ultrasound features for thyroid nodules is still unclear. This study aimed to evaluate the effect of location and size of thyroid nodules on the diagnostic performance of strain ultrasound elastography. In addition, this study evaluated whether ultrasound elastography predicts malignancy in thyroid nodules.

METHODS

Data regarding the size, depth, and distance from the carotid artery of nodules, the elasticity contrast index, and the nature of nodules were analyzed.

RESULTS

There was no significant difference in the depth (p=0.092) and the distance from the carotid artery (p=0.061) between benign and suspicious nodules. Suspicious nodules were smaller than benign nodules (p<0.0001, q=23.84) and had a higher elasticity contrast index (p<0.0001, q=21.05). The depth of nodules and the size of the nodule were not associated with the correct value of the elasticity contrast index (p>0.05 for both). The diagnostic performance of ultrasound elastography was not affected by the distance of the nodules from the carotid artery if they were located ≥15 mm from the carotid artery (p=0.5960). However, if the suspicious nodules were located <15 mm from the carotid artery, the diagnostic accuracy was hampered (p=0.006).

CONCLUSIONS

The strain ultrasound elastography should be carefully evaluated when small thyroid nodules are located near the carotid artery.

The role of ultrasound examination for early identification of lymph-node metastasis of cutaneous squamous cell carcinoma: results from a single institutional center

BACKGROUND

Metastasis from cutaneous squamous cell carcinoma (cSCC) mainly involve the regional nodal basin, with an incidence ranging from 2-4% until 15% in case of high-risk tumors. When dealing with high-risk cSCC, ultrasound examination is recommended every 3-4 months during follow-up. We aimed to determine the role of US examination in the early diagnosis of nodal metastasis from cSCC.

METHODS

We conducted a retrospective cohort study enrolling consecutive cases of histopathologically verified cSCCs from January 2007 to March 2018. All the enrolled cases were followed for at least one year and all cases of histopathologically verified metastasis were registered. We also reported if ultrasound of the regional basin was performed between the primary diagnosis and metastasis and how the latter was identified, through ultrasounds or clinically. A Kaplan-Meier survival analysis was conducted on patients undergoing ultrasounds during follow-up.

RESULTS

A total of 1881 cases, belonging to 1441 patients were included. Thirty-one cases of nodal metastasis diagnosed after the primary tumor, in as many patients, were identified. All of the selected metastasis derived from high-risk primary cSCCs. Only in 19 cases ultrasound examination was performed during follow-up; of these, 10 were diagnosed through ultrasounds and 9 clinically. Survival analysis demonstrated that the time interval between primary tumor and metastasis was significantly lower for patients with metastasis diagnosed by ultrasounds than clinically (P=0.036).

CONCLUSIONS

Our study highlighted the need to optimize the use of nodal ultrasound examination for high-risk cSCCs in order to early detect metastasis.

Value and Diagnostic Efficacy of Fetal Morphology Assessment Using Ultrasound in A Poor-Resource Setting

BACKGROUND

Ultrasound is operator-dependent, and its value and efficacy in fetal morphology assessment in a low-resource setting is poorly understood. We assessed the value and efficacy of fetal morphology ultrasound assessment in a Nigerian setting.

MATERIALS AND METHODS

We surveyed fetal morphology ultrasound performed across five facilities and followed-up each fetus to ascertain the outcome. Fetuses were surveyed in the second trimester (18th-22nd weeks) using the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) guideline. Clinical and surgical reports were used as references to assess the diagnostic efficacy of ultrasound in livebirths, and autopsy reports to confirm anomalies in terminated pregnancies, spontaneous abortions, intrauterine fetal deaths, and still births. We calculated sensitivity, specificity, positive and negative predictive values, Area under the curve (AUC), Youden index, likelihood ratios, and post-test probabilities.

RESULTS

In total, 6520 fetuses of women aged 15-46 years (mean = 31.7 years) were surveyed. The overall sensitivity, specificity, and AUC were 77.1 (95% CI: 68-84.6), 99.5 (95% CI: 99.3-99.7), and 88.3 (95% CI: 83.7-92.2), respectively. Other performance metrics were: positive predictive value, 72.4 (95% CI: 64.7-79.0), negative predictive value, 99.6 (95% CI: 99.5-99.7), and Youden index (77.1%). Abnormality prevalence was 1.67% (95% CI: 1.37-2.01), and the positive and negative likelihood ratios were 254 (95% CI: 107.7-221.4) and 0.23 (95% CI: 0.16-0.33), respectively. The post-test probability for positive test was 72% (95% CI: 65-79).

CONCLUSION

Fetal morphology assessment is valuable in a poor economics setting, however, the variation in the diagnostic efficacy across facilities and the limitations associated with the detection of circulatory system anomalies need to be addressed.

Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT

PURPOSE

To develop a deep learning-based computer-aided diagnosis (CAD) system for use in the CT diagnosis of cervical lymph node metastasis (LNM) in patients with thyroid cancer.

METHODS

A total of 995 axial CT images that included benign (n = 647) and malignant (n = 348) lymph nodes were collected from 202 patients with thyroid cancer who underwent CT for surgical planning between July 2017 and January 2018. The datasets were randomly split into training (79.0%), validation (10.5%), and test (10.5%) datasets. Eight deep convolutional neural network (CNN) models were used to classify the images into metastatic or benign lymph nodes. Pretrained networks were used on the ImageNet and the best-performing algorithm was selected. Class-specific discriminative regions were visualized with attention heatmap using a global average pooling method.

RESULTS

The area under the ROC curve (AUROC) for the tested algorithms ranged from 0.909 to 0.953. The sensitivity, specificity, and accuracy of the best-performing algorithm were all 90.4%, respectively. Attention heatmap highlighted important subregions for further clinical review.

CONCLUSION

A deep learning-based CAD system could accurately classify cervical LNM in patients with thyroid cancer on preoperative CT with an AUROC of 0.953. Whether this approach has clinical utility will require evaluation in a clinical setting.

KEY POINTS

• A deep learning-based CAD system could accurately classify cervical lymph node metastasis. The AUROC for the eight tested algorithms ranged from 0.909 to 0.953. • Of the eight models, the ResNet50 algorithm was the best-performing model for the validation dataset with 0.953 AUROC. The sensitivity, specificity, and accuracy of the ResNet50 model were all 90.4%, respectively, in the test dataset. • Based on its high accuracy of 90.4%, we consider that this model may be useful in a clinical setting to detect LNM on preoperative CT in patients with thyroid cancer.

Accuracy of the European Thyroid Imaging Reporting and Data System (EU-TIRADS) in the valuation of thyroid nodule malignancy in reference to the post-surgery histological results

Purpose

To assess the clinical usefulness of the European Thyroid Imaging and Reporting Data System (EU-TIRADS) in the valuation of thyroid nodules malignancy in reference to post-surgery histological results.

Material and methods

Pre-operative ultrasound was performed in consecutive patients admitted for thyroid surgery between June 2017 and January 2018. Thyroid nodules were classified according to EU-TIRADS to five groups: 1-5. At least one fine-needle aspiration biopsy (FNAB)/patient (dominant or suspected nodule) was performed in an outpatient clinic. The final diagnosis was based on the histological result. The percentage of cancers in each EU-TIRADS group was evaluated. Finally, sensitivity, specificity, accuracy, as well as positive and negative predictive values for malignancy were assessed.

Results

Fifty-two patients with a total of 140 thyroid nodules (median: 3 nodules/thyroid [minimum-maximum: 1-6]) were enrolled in the study. Thyroid cancer was diagnosed in 0% (0/6) in EU-TIRADS 2; 0% (0/92) in EU-TIRADS 3; 5.9% (2/34) in EU-TIRADS 4, and 75% (6/8) in EU-TIRADS 5. In nodules assessed as EU-TIRADS ≥ 4 sensitivity, specificity, positive and negative predictive values for malignancy were, respectively: 75% (CI 95%: 40.7-93.5), 94.1% (CI 95%: 86.0-98.5), 75% (CI 95%: 40.7-93.5), and 94.1% (CI 95%: 86.0-98.5).

Conclusions

EU-TIRADS is a valuable and simple tool for assessment of the risk of malignancy of thyroid nodules and demonstrates a high ultrasound correlation with histological post-surgery results. FNAB should be performed in all nodules assessed as EU-TIRADS ≥ 4, due to higher risk of malignancy.

Deep Learning-Based Computer-Aided Diagnosis System for Localization and Diagnosis of Metastatic Lymph Nodes on Ultrasound: A Pilot Study

BACKGROUND

The presence of metastatic lymph nodes is a prognostic indicator for patients with thyroid carcinomas and is an important determinant of clinical decision making. However, evaluating neck lymph nodes requires experience and is labor- and time-intensive. Therefore, the development of a computer-aided diagnosis (CAD) system to identify and differentiate metastatic lymph nodes may be useful.

METHODS

From January 2008 to December 2016, we retrieved clinical records for 804 consecutive patients with 812 lymph nodes. The status of all lymph nodes was confirmed by fine-needle aspiration. The datasets were split into training (263 benign and 286 metastatic lymph nodes), validation (30 benign and 33 metastatic lymph nodes), and test (100 benign and 100 metastatic lymph nodes). Using the VGG-Class Activation Map model, we developed a CAD system to localize and differentiate the metastatic lymph nodes. We then evaluated the diagnostic performance of this CAD system in our test set.

RESULTS

In the test set, the accuracy, sensitivity, and specificity of our model for predicting lymph node malignancy were 83.0%, 79.5%, and 87.5%, respectively. The CAD system clearly detected the locations of the lymph nodes, which not only provided identifying data, but also demonstrated the basis of decisions.

CONCLUSION

We developed a deep learning-based CAD system for the localization and differentiation of metastatic lymph nodes from thyroid cancer on ultrasound. This CAD system is highly sensitive and may be used as a screening tool; however, as it is relatively less specific, the screening results should be validated by experienced physicians.