Artificial Intelligence for Ultrasound Informative Image Selection of Metacarpal Head Cartilage. A Pilot Study

Pre-trained convolutional neural network with transfer learning by artificial illustrated images classify power Doppler ultrasound images of rheumatoid arthritis joints

OBJECTIVE

To study the classification performance of a pre-trained convolutional neural network (CNN) with transfer learning by artificial joint ultrasonography images in rheumatoid arthritis (RA).

METHODS

This retrospective study focused on abnormal synovial vascularity and created 870 artificial joint ultrasound images based on the European League Against Rheumatism/Outcome Measure in Rheumatology scoring system. One CNN, the Visual Geometry Group (VGG)-16, was trained with transfer learning using the 870 artificial images for initial training and the original plus five additional images for second training. The models were then tested for the ability to classify joints using real joint ultrasound images obtained from patients with RA. The study was registered in UMIN Clinical Trials Registry (UMIN000054321).

RESULTS

A total of 156 clinical joint ultrasound images from 74 patients with RA were included. The initial model showed moderate classification performance, but the area under curve (AUC) for grade 1 synovitis was particularly low (0.59). The second model showed improvement in classifying grade 1 synovitis (AUC 0.73).

CONCLUSIONS

Artificial images may be useful for training VGG-16. The present novel approach of using artificial images as an alternative to actual images for training a CNN has the potential to be applied in medical imaging fields that face difficulties in collecting real clinical images.

The usefulness of automated high frequency ultrasound image analysis in atopic dermatitis staging

The last decades have brought an interest in ultrasound applications in dermatology. Especially in the case of atopic dermatitis, where the formation of a subepidermal low echogenic band (SLEB) may serve as an independent indicator of the effects of treatment, the use of ultrasound is of particular interest. This study proposes and evaluates the computer-aided diagnosis method for assessing atopic dermatitis (AD). The fully automated image processing framework combines advanced machine learning techniques for fast, reliable, and repeatable HFUS image analysis, supporting clinical decisions. The proposed methodology comprises accurate SLEB segmentation followed by a classification step. The data set includes 20 MHz images of 80 patients diagnosed with AD according to Hanifin and Rajka criteria, which were evaluated before and after treatment. The ground true labels- clinical evaluation based on Investigator Global Assessment index (IGA score) together with ultrasound skin examination was performed. For reliable analysis, in further experiments, two experts annotated the HFUS images twice in two-week intervals. The analysis aimed to verify whether the fully automated method can classify the HFUS images at the expert level. The Dice coefficient values for segmentation reached 0.908 for SLEB and 0.936 for the entry echo layer. The accuracy of SLEB presence detection results (IGA0) is equal to 98% and slightly outperforms the experts' assessment, which reaches 96%. The overall accuracy of the AD assessment was equal to 69% (Cohen's kappa 0.78) and was comparable with the experts' assessment, ranging between 64% and 70% (Cohen's kappa 0.73-0.79). The results indicate that the automated method can be applied to AD assessment, and its combination with standard diagnosis may benefit repeatable analysis and a better understanding of the processes that take place within the skin and aid treatment monitoring.

Automatic Assessment of AK Stage Based on Dermatoscopic and HFUS Imaging-A Preliminary Study

Background: Actinic keratoses (AK) usually occur on sun-exposed areas in elderly patients with Fitzpatrick I-II skin types. Dermatoscopy and ultrasonography are two non-invasive tools helpful in examining clinically suspicious lesions. This study presents the usefulness of image-processing algorithms in AK staging based on dermatoscopic and ultrasonographic images. Methods: In 54 patients treated at the Department of Dermatology of Poznan University of Medical Sciences, clinical, dermatoscopic, and ultrasound examinations were performed. The clinico-dermoscopic AK classification was based on three-point Zalaudek scale. The ultrasound images were recorded with DermaScan C, Cortex Technology device, 20 MHz. The dataset consisted of 162 image pairs. The developed algorithm includes automated segmentation of ultrasound data utilizing a CFPNet-M model followed by handcrafted feature extraction. The dermatoscopic image analysis includes both handcrafted and convolutional neural network features, which, combined with ultrasound descriptors, are used in support vector machine-based classification. The network models were trained on public datasets. The influence of each modality on the final classification was evaluated. Results: The most promising results were obtained for the dermatoscopic analysis with the use of neural network model (accuracy 81%) and its combination with ultrasound scans (accuracy 79%). Conclusions: The application of machine learning-based algorithms in dermatoscopic and ultrasound image analysis machine learning in the staging of AKs may be beneficial in clinical practice in terms of predicting the risk of progression. Further experiments are warranted, as incorporating more images is likely to improve classification accuracy of the system.

Artificial intelligence model for segmentation and severity scoring of osteophytes in hand osteoarthritis on ultrasound images

Objective

To develop an artificial intelligence (AI) model able to perform both segmentation of hand joint ultrasound images for osteophytes, bone, and synovium and perform osteophyte severity scoring following the EULAR-OMERACT grading system (EOGS) for hand osteoarthritis (OA).

Methods

One hundred sixty patients with pain or reduced function of the hands were included. Ultrasound images of the metacarpophalangeal (MCP), proximal interphalangeal (PIP), distal interphalangeal (DIP), and first carpometacarpal (CMC1) joints were then manually segmented for bone, synovium and osteophytes and scored from 0 to 3 according to the EOGS for OA. Data was divided into a training, validation, and test set. The AI model was trained on the training data to perform bone, synovium, and osteophyte identification on the images. Based on the manually performed image segmentation, an AI was trained to classify the severity of osteophytes according to EOGS from 0 to 3. Percent Exact Agreement (PEA) and Percent Close Agreement (PCA) were assessed on individual joints and overall. PCA allows a difference of one EOGS grade between doctor assessment and AI.

Results

A total of 4615 ultrasound images were used for AI development and testing. The developed AI model scored on the test set for the MCP joints a PEA of 76% and PCA of 97%; for PIP, a PEA of 70% and PCA of 97%; for DIP, a PEA of 59% and PCA of 94%, and CMC a PEA of 50% and PCA of 82%. Combining all joints, we found a PEA between AI and doctor assessments of 68% and a PCA of 95%.

Conclusion

The developed AI model can perform joint ultrasound image segmentation and severity scoring of osteophytes, according to the EOGS. As proof of concept, this first version of the AI model is successful, as the agreement performance is slightly higher than previously found agreements between experts when assessing osteophytes on hand OA ultrasound images. The segmentation of the image makes the AI explainable to the doctor, who can immediately see why the AI applies a given score. Future validation in hand OA cohorts is necessary though.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Identification of tophi in ultrasound imaging based on transfer learning and clinical practice

Gout is a common metabolic disorder characterized by deposits of monosodium urate monohydrate crystals (tophi) in soft tissue, triggering intense and acute arthritis with intolerable pain as well as articular and periarticular inflammation. Tophi can also promote chronic inflammatory and erosive arthritis. 2015 ACR/EULAR Gout Classification criteria include clinical, laboratory, and imaging findings, where cases of gout are indicated by a threshold score of ≥ 8. Some imaging-related findings, such as a double contour sign in ultrasound, urate in dual-energy computed tomography, or radiographic gout-related erosion, generate a score of up to 4. Clearly, the diagnosis of gout is largely assisted by imaging findings; however, dual-energy computed tomography is expensive and exposes the patient to high levels of radiation. Although musculoskeletal ultrasound is non-invasive and inexpensive, the reliability of the results depends on expert experience. In the current study, we applied transfer learning to train a convolutional neural network for the identification of tophi in ultrasound images. The accuracy of predictions varied with the convolutional neural network model, as follows: InceptionV3 (0.871 ± 0.020), ResNet101 (0.913 ± 0.015), and VGG19 (0.918 ± 0.020). The sensitivity was as follows: InceptionV3 (0.507 ± 0.060), ResNet101 (0.680 ± 0.056), and VGG19 (0.747 ± 0.056). The precision was as follows: InceptionV3 (0.767 ± 0.091), ResNet101 (0.863 ± 0.098), and VGG19 (0.825 ± 0.062). Our results demonstrate that it is possible to retrain deep convolutional neural networks to identify the patterns of tophi in ultrasound images with a high degree of accuracy.

Computer-Aided Diagnosis Methods for High-Frequency Ultrasound Data Analysis: A Review

Over the last few decades, computer-aided diagnosis systems have become a part of clinical practice. They have the potential to assist clinicians in daily diagnostic tasks. The image processing techniques are fast, repeatable, and robust, which helps physicians to detect, classify, segment, and measure various structures. The recent rapid development of computer methods for high-frequency ultrasound image analysis opens up new diagnostic paths in dermatology, allergology, cosmetology, and aesthetic medicine. This paper, being the first in this area, presents a research overview of high-frequency ultrasound image processing techniques, which have the potential to be a part of computer-aided diagnosis systems. The reviewed methods are categorized concerning the application, utilized ultrasound device, and image data-processing type. We present the bridge between diagnostic needs and already developed solutions and discuss their limitations and future directions in high-frequency ultrasound image analysis. A search was conducted of the technical literature from 2005 to September 2022, and in total, 31 studies describing image processing methods were reviewed. The quantitative and qualitative analysis included 39 algorithms, which were selected as the most effective in this field. They were completed by 20 medical papers and define the needs and opportunities for high-frequency ultrasound application and CAD development.

Responding to and Driving Change in Rheumatology: Report from the 12th International Immunology Summit 2021

The coronavirus disease 2019 (COVID-19) pandemic has accelerated changes to rheumatology daily clinical practice. The main goal of the 12th International Immunology Summit, held 25-26 June, 2021 (virtual meeting), was to provide direction for these active changes rather than undergoing change reactively in order to improve patient outcomes. This review describes and explores the concept of change in rheumatology clinical practice based on presentations from the Immunology Summit. Many of the changes to rheumatology practice brought about by the COVID-19 pandemic may be considered as having a positive impact on disease management and may help with the long-term development of more patient-focused treatment. Rheumatologists can contribute key knowledge regarding the use of immunosuppressive agents in the context of the pandemic, and according to the European League Against Rheumatism, they should be involved in any multidisciplinary COVID-19 guideline committees. New technologies, including telemedicine and artificial intelligence, represent an opportunity for physicians to individualise patient treatment and improve disease management. Despite major advances in the treatment of rheumatic diseases, the efficacy of available disease-modifying anti-rheumatic drugs (DMARDs) remains suboptimal and data regarding serological biomarkers are limited. Synovial tissue biomarkers, such as CD68+ macrophages, have shown promise in elucidating pathogenesis and targeting treatment to the individual patient. In spondyloarthritis (SpA) or psoriatic arthritis (PsA), information regarding the effectiveness of the available agents with different mechanisms of action may be integrated to manage patients using a treat-to-target approach. Early diagnosis of SpA and PsA is important for optimisation of treatment response and long-term outcomes. Improving our understanding of disease pathogenesis and practice methods may help reduce diagnostic delays, thereby optimising disease outcomes in patients with rheumatic diseases.

High-Frequency Ultrasound Dataset for Deep Learning-Based Image Quality Assessment

This study aims at high-frequency ultrasound image quality assessment for computer-aided diagnosis of skin. In recent decades, high-frequency ultrasound imaging opened up new opportunities in dermatology, utilizing the most recent deep learning-based algorithms for automated image analysis. An individual dermatological examination contains either a single image, a couple of pictures, or an image series acquired during the probe movement. The estimated skin parameters might depend on the probe position, orientation, or acquisition setup. Consequently, the more images analyzed, the more precise the obtained measurements. Therefore, for the automated measurements, the best choice is to acquire the image series and then analyze its parameters statistically. However, besides the correctly received images, the resulting series contains plenty of non-informative data: Images with different artifacts, noise, or the images acquired for the time stamp when the ultrasound probe has no contact with the patient skin. All of them influence further analysis, leading to misclassification or incorrect image segmentation. Therefore, an automated image selection step is crucial. To meet this need, we collected and shared 17,425 high-frequency images of the facial skin from 516 measurements of 44 patients. Two experts annotated each image as correct or not. The proposed framework utilizes a deep convolutional neural network followed by a fuzzy reasoning system to assess the acquired data’s quality automatically. Different approaches to binary and multi-class image analysis, based on the VGG-16 model, were developed and compared. The best classification results reach 91.7% accuracy for the first, and 82.3% for the second analysis, respectively.

Development of a convolutional neural network for the identification and the measurement of the median nerve on ultrasound images acquired at carpal tunnel level

Background

Deep learning applied to ultrasound (US) can provide a feedback to the sonographer about the correct identification of scanned tissues and allows for faster and standardized measurements. The most frequently adopted parameter for US diagnosis of carpal tunnel syndrome is the increasing of the cross-sectional area (CSA) of the median nerve. Our aim was to develop a deep learning algorithm, relying on convolutional neural networks (CNNs), for the localization and segmentation of the median nerve and the automatic measurement of its CSA on US images acquired at the proximal inlet of the carpal tunnel.

Methods

Consecutive patients with rheumatic and musculoskeletal disorders were recruited. Transverse US images were acquired at the carpal tunnel inlet, and the CSA was manually measured. Anatomical variants were registered. The dataset consisted of 246 images (157 for training, 40 for validation, and 49 for testing) from 103 patients each associated with manual annotations of the nerve boundary. A Mask R-CNN, state-of-the-art CNN for image semantic segmentation, was trained on this dataset to accurately localize and segment the median nerve section. To evaluate the performances on the testing set, precision (Prec), recall (Rec), mean average precision (mAP), and Dice similarity coefficient (DSC) were computed. A sub-analysis excluding anatomical variants was performed. The CSA was automatically measured by the algorithm.

Results

The algorithm correctly identified the median nerve in 41/49 images (83.7%) and in 41/43 images (95.3%) excluding anatomical variants. The following metrics were obtained (with and without anatomical variants, respectively): Prec 0.86 ± 0.33 and 0.96 ± 0.18, Rec 0.88 ± 0.33 and 0.98 ± 0.15, mAP 0.88 ± 0.33 and 0.98 ± 0.15, and DSC 0.86 ± 0.19 and 0.88 ± 0.19. The agreement between the algorithm and the sonographer CSA measurements was excellent [ICC 0.97 (0.94–0.98)].

Conclusions

The developed algorithm has shown excellent performances, especially if excluding anatomical variants. Future research should aim at expanding the US image dataset including a wider spectrum of normal anatomy and pathology. This deep learning approach has shown very high potentiality for a fully automatic support for US assessment of carpal tunnel syndrome.