A systematic review of machine learning based thyroid tumor characterisation using ultrasonographic images

A weakly-supervised follicle segmentation method in ultrasound images

Accurate follicle segmentation in ultrasound images is crucial for monitoring follicle development, a key factor in fertility treatments. However, obtaining pixel-level annotations for fully supervised instance segmentation is often impractical due to time and workload constraints. This paper presents a weakly supervised instance segmentation method that leverages bounding boxes as approximate annotations, aiming to assist clinicians with automated tools for follicle development monitoring. We propose the Weakly Supervised Follicle Segmentation (WSFS) method, a novel one-stage weakly supervised segmentation technique model designed to enhance the ultrasound images of follicles, which incorporates a Convolutional Neural Network (CNN) backbone augmented with a Feature Pyramid Network (FPN) module for multi-scale feature representation, critical for capturing the diverse sizes and shapes of follicles. By leveraging Multiple Instance Learning (MIL), we formulated the learning process in a weakly supervised manner and developed an end-to-end trainable model that efficiently addresses the issue of annotation scarcity. Furthermore, the WSFS can be used as a prompt proposal to enhance the performance of the Segmentation Anything Model (SAM), a well-known pre-trained segmentation model utilizing few-shot learning strategies. In addition, this study introduces the Follicle Ultrasound Image Dataset (FUID), addressing the scarcity in reproductive health data and aiding future research in computer-aided diagnosis. The experimental results on both the public dataset USOVA3D and private dataset FUID showed that our method performs competitively with fully supervised methods. Our approach achieves performance with mAP of 0.957, IOU of 0.714 and Dice Score of 0.83, competitive to fully supervised methods that rely on pixel-level labeled masks, despite operating with less detailed annotations.

Predicting the efficacy of microwave ablation of benign thyroid nodules from ultrasound images using deep convolutional neural networks

BACKGROUND

Thyroid nodules are frequent in clinical settings, and their diagnosis in adults is growing, with some persons experiencing symptoms. Ultrasound-guided thermal ablation can shrink nodules and alleviate discomfort. Because the degree and rate of lesion absorption vary greatly between individuals, there is no reliable model for predicting the therapeutic efficacy of thermal ablation.

METHODS

Five convolutional neural network models including VGG19, Resnet 50, EfficientNetB1, EfficientNetB0, and InceptionV3, pre-trained with ImageNet, were compared for predicting the efficacy of ultrasound-guided microwave ablation (MWA) for benign thyroid nodules using ultrasound data. The patients were randomly assigned to one of two data sets: training (70%) or validation (30%). Accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve (AUC) were all used to assess predictive performance.

RESULTS

In the validation set, fine-tuned EfficientNetB1 performed best, with an AUC of 0.85 and an ACC of 0.79.

CONCLUSIONS

The study found that our deep learning model accurately predicts nodules with VRR < 50% after a single MWA session. Indeed, when thermal therapies compete with surgery, anticipating which nodules will be poor responders provides useful information that may assist physicians and patients determine whether thermal ablation or surgery is the preferable option. This was a preliminary study of deep learning, with a gap in actual clinical applications. As a result, more in-depth study should be undertaken to develop deep-learning models that can better help clinics. Prospective studies are expected to generate high-quality evidence and improve clinical performance in subsequent research.

Computer-aided diagnosis of hepatic cystic echinococcosis based on deep transfer learning features from ultrasound images

Hepatic cystic echinococcosis (HCE), a life-threatening liver disease, has 5 subtypes, i.e., single-cystic, polycystic, internal capsule collapse, solid mass, and calcified subtypes. And each subtype has different treatment methods. An accurate diagnosis is the prerequisite for effective HCE treatment. However, clinicians with less diagnostic experience often make misdiagnoses of HCE and confuse its 5 subtypes in clinical practice. Computer-aided diagnosis (CAD) techniques can help clinicians to improve their diagnostic performance. This paper aims to propose an efficient CAD system that automatically differentiates 5 subtypes of HCE from the ultrasound images. The proposed CAD system adopts the concept of deep transfer learning and uses a pre-trained convolutional neural network (CNN) named VGG19 to extract deep CNN features from the ultrasound images. The proven classifier models, k - nearest neighbor (KNN) and support vecter machine (SVM) models, are integrated to classify the extracted deep CNN features. 3 distinct experiments with the same deep CNN features but different classifier models (softmax, KNN, SVM) are performed. The experiments followed 10 runs of the five-fold cross-validation process on a total of 1820 ultrasound images and the results were compared using Wilcoxon signed-rank test. The overall classification accuracy from low to high was 90.46 ± 1.59% for KNN classifier, 90.92 ± 2.49% for transfer learned VGG19, and 92.01 ± 1.48% for SVM, indicating SVM classifiers with deep CNN features achieved the best performance (P < 0.05). Other performance measures used in the study are specificity, sensitivity, precision, F1-score, and area under the curve (AUC). In addition, the paper addresses a practical aspect by evaluating the system with smaller training data to demonstrate the capability of the proposed classification system. The observations of the study imply that transfer learning is a useful technique when the availability of medical images is limited. The proposed classification system by using deep CNN features and SVM classifier is potentially helpful for clinicians to improve their HCE diagnostic performance in clinical practice.

Identification of lesion location and discrimination between benign and malignant findings in thyroid ultrasound imaging

Thyroid nodules are a common thyroid disorder, and ultrasound imaging, as the primary diagnostic tool, is susceptible to variations based on the physician's experience, leading to misdiagnosis. This paper constructs an end-to-end thyroid nodule detection framework based on YOLOv8, enabling automatic detection and classification of nodules by extracting grayscale and elastic features from ultrasound images. First, an attention-weighted DCN is introduced to enhance superficial feature extraction and capture local information. Next, the CPCA mechanism is employed to reduce the interference of redundant information. Finally, a feature fusion network based on an aggregation-distribution mechanism is utilized to improve the learning capability of fine-grained features, enhancing the performance of early nodule detection. Experimental results demonstrate that our method is accurate and effective for thyroid nodule detection, achieving diagnostic rates of 89.3% for benign and 90.4% for malignant nodules based on tests conducted on 611 clinical ultrasound images, with a mean Average Precision at IoU = 0.5 (mAP@50) of 95.5%, representing a 6.6% improvement over baseline models.

Performance of ACR-TIRADS in assessing thyroid nodules does not vary according to patient age

AIMS

A few studies have evaluated the performance of the American College of Radiology Thyroid Imaging Reporting And Data System (ACR-TIRADS) in pediatric and elderly patients and found differences between the latter two age groups and middle adulthood. Thus, the present study was undertaken to explore the possible variation of ACR-TIRADS performance across different ages of patients.

METHODS

A retrospective population undergoing thyroidectomy was selected to use histology as the reference standard. Ultrasound images were reviewed, and alignment of ACR-TIRADS with the corresponding histological diagnosis was made afterwards. Results of the age groups were compared. The ACR-TIRADS diagnostic performance was calculated considering the assessment of nodules across risk categories (i.e., from TR1 to TR5), rate of unnecessary FNAC (UN-FNAC), and rate of necessary but non-performed FNAC (NNP-FNAC).

RESULTS

Overall, 114 patients with a total of 220 nodules (46 carcinomas) were included. The rate of UN-FNAC was 66.3%, being 93.1% in TR3, 82.1% in TR4, and 31.4% in TR5. There were 15 NNP-FNACs. No significant difference was observed between age groups in terms of sample size, nodule, cancer, and FNAC. The nodule assessment according to ACR-TIRADS categories did not vary across ages. Sensitivity and specificity recorded in three age tertiles were not significantly different.

CONCLUSIONS

The present study shows that the performance of ACR-TIRADS is not significantly influenced by patient age.

Deep learning based binary classification of diabetic retinopathy images using transfer learning approach

Objective

Diabetic retinopathy (DR) is a common problem of diabetes, and it is the cause of blindness worldwide. Detection of diabetic radiology disease in the early detection stage is crucial for preventing vision loss. In this work, a deep learning-based binary classification of DR images has been proposed to classify DR images into healthy and unhealthy. Transfer learning-based 20 pre-trained networks have been fine-tuned using a robust dataset of diabetic radiology images. The combined dataset has been collected from three robust databases of diabetic patients annotated by experienced ophthalmologists indicating healthy or non-healthy diabetic retina images.

Method

This work has improved robust models by pre-processing the DR images by applying a denoising algorithm, normalization, and data augmentation. In this work, three rubout datasets of diabetic retinopathy images have been selected, named DRD- EyePACS, IDRiD, and APTOS-2019, for the extensive experiments, and a combined diabetic retinopathy image dataset has been generated for the exhaustive experiments. The datasets have been divided into training, testing, and validation sets, and the models use classification accuracy, sensitivity, specificity, precision, F1-score, and ROC-AUC to assess the model's efficiency for evaluating network performance. The present work has selected 20 different pre-trained networks based on three categories: Series, DAG, and lightweight.

Results

This study uses pre-processed data augmentation and normalization of data to solve overfitting problems. From the exhaustive experiments, the three best pre-trained have been selected based on the best classification accuracy from each category. It is concluded that the trained model ResNet101 based on the DAG category effectively identifies diabetic retinopathy disease accurately from radiological images from all cases. It is noted that 97.33% accuracy has been achieved using ResNet101 in the category of DAG network.

Conclusion

Based on the experiment results, the proposed model ResNet101 helps healthcare professionals detect retina diseases early and provides practical solutions to diabetes patients. It also gives patients and experts a second opinion for early detection of diabetic retinopathy.

CystNet: An AI driven model for PCOS detection using multilevel thresholding of ultrasound images

Polycystic Ovary Syndrome (PCOS) is a widespread endocrinological dysfunction impacting women of reproductive age, categorized by excess androgens and a variety of associated syndromes, consisting of acne, alopecia, and hirsutism. It involves the presence of multiple immature follicles in the ovaries, which can disrupt normal ovulation and lead to hormonal imbalances and associated health complications. Routine diagnostic methods rely on manual interpretation of ultrasound (US) images and clinical assessments, which are time-consuming and prone to errors. Therefore, implementing an automated system is essential for streamlining the diagnostic process and enhancing accuracy. By automatically analyzing follicle characteristics and other relevant features, this research aims to facilitate timely intervention and reduce the burden on healthcare professionals. The present study proposes an advanced automated system for detecting and classifying PCOS from ultrasound images. Leveraging Artificial Intelligence (AI) based techniques, the system examines affected and unaffected cases to enhance diagnostic accuracy. The pre-processing of input images incorporates techniques such as image resizing, normalization, augmentation, Watershed technique, multilevel thresholding, etc. approaches for precise image segmentation. Feature extraction is facilitated by the proposed CystNet technique, followed by PCOS classification utilizing both fully connected layers with 5-fold cross-validation and traditional machine learning classifiers. The performance of the model is rigorously evaluated using a comprehensive range of metrics, incorporating AUC score, accuracy, specificity, precision, F1-score, recall, and loss, along with a detailed confusion matrix analysis. The model demonstrated a commendable accuracy of [Formula: see text] when utilizing a fully connected classification layer, as determined by a thorough 5-fold cross-validation process. Additionally, it has achieved an accuracy of [Formula: see text] when employing an ensemble ML classifier. This proposed approach could be suggested for predicting PCOS or similar diseases using datasets that exhibit multimodal characteristics.

Role of inter- and extra-lesion tissue, transfer learning, and fine-tuning in the robust classification of breast lesions

Accurate and unbiased classification of breast lesions is pivotal for early diagnosis and treatment, and a deep learning approach can effectively represent and utilize the digital content of images for more precise medical image analysis. Breast ultrasound imaging is useful for detecting and distinguishing benign masses from malignant masses. Based on the different ways in which benign and malignant tumors affect neighboring tissues, i.e., the pattern of growth and border irregularities, the penetration degree of the adjacent tissue, and tissue-level changes, we investigated the relationship between breast cancer imaging features and the roles of inter- and extra-lesional tissues and their impact on refining the performance of deep learning classification. The novelty of the proposed approach lies in considering the features extracted from the tissue inside the tumor (by performing an erosion operation) and from the lesion and surrounding tissue (by performing a dilation operation) for classification. This study uses these new features and three pre-trained deep neuronal networks to address the challenge of breast lesion classification in ultrasound images. To improve the classification accuracy and interpretability of the model, the proposed model leverages transfer learning to accelerate the training process. Three modern pre-trained CNN architectures (MobileNetV2, VGG16, and EfficientNetB7) are used for transfer learning and fine-tuning for optimization. There are concerns related to the neuronal networks producing erroneous outputs in the presence of noisy images, variations in input data, or adversarial attacks; thus, the proposed system uses the BUS-BRA database (two classes/benign and malignant) for training and testing and the unseen BUSI database (two classes/benign and malignant) for testing. Extensive experiments have recorded accuracy and AUC as performance parameters. The results indicate that the proposed system outperforms the existing breast cancer detection algorithms reported in the literature. AUC values of 1.00 are calculated for VGG16 and EfficientNet-B7 in the dilation cases. The proposed approach will facilitate this challenging and time-consuming classification task.

Revolutionizing breast ultrasound diagnostics with EfficientNet-B7 and Explainable AI

Breast cancer is a leading cause of mortality among women globally, necessitating precise classification of breast ultrasound images for early diagnosis and treatment. Traditional methods using CNN architectures such as VGG, ResNet, and DenseNet, though somewhat effective, often struggle with class imbalances and subtle texture variations, leading to reduced accuracy for minority classes such as malignant tumors. To address these issues, we propose a methodology that leverages EfficientNet-B7, a scalable CNN architecture, combined with advanced data augmentation techniques to enhance minority class representation and improve model robustness. Our approach involves fine-tuning EfficientNet-B7 on the BUSI dataset, implementing RandomHorizontalFlip, RandomRotation, and ColorJitter to balance the dataset and improve model robustness. The training process includes early stopping to prevent overfitting and optimize performance metrics. Additionally, we integrate Explainable AI (XAI) techniques, such as Grad-CAM, to enhance the interpretability and transparency of the model's predictions, providing visual and quantitative insights into the features and regions of ultrasound images influencing classification outcomes. Our model achieves a classification accuracy of 99.14%, significantly outperforming existing CNN-based approaches in breast ultrasound image classification. The incorporation of XAI techniques enhances our understanding of the model's decision-making process, thereby increasing its reliability and facilitating clinical adoption. This comprehensive framework offers a robust and interpretable tool for the early detection and diagnosis of breast cancer, advancing the capabilities of automated diagnostic systems and supporting clinical decision-making processes.

Diagnostic value of an interpretable machine learning model based on clinical ultrasound features for follicular thyroid carcinoma

Background

Follicular thyroid carcinoma (FTC) and follicular thyroid adenoma (FTA) present diagnostic challenges due to overlapping clinical and ultrasound features. Improving the diagnosis of FTC can enhance patient prognosis and effectiveness in clinical management. This study seeks to develop a predictive model for FTC based on ultrasound features using machine learning (ML) algorithms and assess its diagnostic effectiveness.

Methods

Patients diagnosed with FTA or FTC based on surgical pathology between January 2009 and February 2023 at Zhejiang Provincial Cancer Hospital and Zhejiang Provincial People's Hospital were retrospectively included. A total of 562 patients from Zhejiang Provincial Cancer Hospital comprised the training set, and 218 patients from Zhejiang Provincial People's Hospital constituted the validation set. Subsequently, clinical parameters and ultrasound characteristics of the patients were collected. The diagnostic parameters were analyzed using the least absolute shrinkage and selection operator and multivariate logistic regression screening methods. Next, a comparative analysis was performed using seven ML models. The area under the receiver operating characteristic (ROC) curve (AUC), accuracy, sensitivity, specificity, positive predicted value (PPV), negative predicted value (NPV), precision, recall, and comprehensive evaluation index (F-score) were calculated to compare the diagnostic efficacy among the seven models and determine the optimal model. Further, the optimal model was validated, and the SHapley Additive ExPlanations (SHAP) approach was applied to explain the significance of the model variables. Finally, an individualized risk assessment was conducted.

Results

Age, echogenicity, thyroglobulin antibody (TGAb), echotexture, composition, triiodothyronine (T3), thyroglobulin (TG), margin, thyroid-stimulating hormone (TSH), calcification, and halo thickness >2 mm were influential factors for diagnosing FTC. The XGBoost model was identified as the optimal model after a comprehensive evaluation. The AUC of this model in the validation set was 0.969 [95% confidence interval (CI), 0.946-0.992], while its precision sensitivity, specificity, and accuracy were 0.791, 0.930, 0.913 and 0.917, respectively.

Conclusions

XGBoost model based on ultrasound features was constructed and interpreted using the SHAP method, providing evidence for the diagnosis of FTC and guidance for the personalized treatment of patients.

Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach

BACKGROUND

Breast cancer is the most common cancer among women, and ultrasound is a usual tool for early screening. Nowadays, deep learning technique is applied as an auxiliary tool to provide the predictive results for doctors to decide whether to make further examinations or treatments. This study aimed to develop a hybrid learning approach for breast ultrasound classification by extracting more potential features from local and multi-center ultrasound data.

METHODS

We proposed a hybrid learning approach to classify the breast tumors into benign and malignant. Three multi-center datasets (BUSI, BUS, OASBUD) were used to pretrain a model by federated learning, then every dataset was fine-tuned at local. The proposed model consisted of a convolutional neural network (CNN) and a graph neural network (GNN), aiming to extract features from images at a spatial level and from graphs at a geometric level. The input images are small-sized and free from pixel-level labels, and the input graphs are generated automatically in an unsupervised manner, which saves the costs of labor and memory space.

RESULTS

The classification AUCROC of our proposed method is 0.911, 0.871 and 0.767 for BUSI, BUS and OASBUD. The balanced accuracy is 87.6%, 85.2% and 61.4% respectively. The results show that our method outperforms conventional methods.

CONCLUSIONS

Our hybrid approach can learn the inter-feature among multi-center data and the intra-feature of local data. It shows potential in aiding doctors for breast tumor classification in ultrasound at an early stage.

Automatic segmentation of 15 critical anatomical labels and measurements of cardiac axis and cardiothoracic ratio in fetal four chambers using nnU-NetV2

BACKGROUND

Accurate segmentation of critical anatomical structures in fetal four-chamber view images is essential for the early detection of congenital heart defects. Current prenatal screening methods rely on manual measurements, which are time-consuming and prone to inter-observer variability. This study develops an AI-based model using the state-of-the-art nnU-NetV2 architecture for automatic segmentation and measurement of key anatomical structures in fetal four-chamber view images.

METHODS

A dataset, consisting of 1,083 high-quality fetal four-chamber view images, was annotated with 15 critical anatomical labels and divided into training/validation (867 images) and test (216 images) sets. An AI-based model using the nnU-NetV2 architecture was trained on the annotated images and evaluated using the mean Dice coefficient (mDice) and mean intersection over union (mIoU) metrics. The model's performance in automatically computing the cardiac axis (CAx) and cardiothoracic ratio (CTR) was compared with measurements from sonographers with varying levels of experience.

RESULTS

The AI-based model achieved a mDice coefficient of 87.11% and an mIoU of 77.68% for the segmentation of critical anatomical structures. The model's automated CAx and CTR measurements showed strong agreement with those of experienced sonographers, with respective intraclass correlation coefficients (ICCs) of 0.83 and 0.81. Bland-Altman analysis further confirmed the high agreement between the model and experienced sonographers.

CONCLUSION

We developed an AI-based model using the nnU-NetV2 architecture for accurate segmentation and automated measurement of critical anatomical structures in fetal four-chamber view images. Our model demonstrated high segmentation accuracy and strong agreement with experienced sonographers in computing clinically relevant parameters. This approach has the potential to improve the efficiency and reliability of prenatal cardiac screening, ultimately contributing to the early detection of congenital heart defects.