1
|
Cipolletta E, Fiorentino MC, Vreju FA, Moccia S, Filippucci E. Editorial: Artificial intelligence in rheumatology and musculoskeletal diseases. Front Med (Lausanne) 2024; 11:1402871. [PMID: 38646556 PMCID: PMC11026684 DOI: 10.3389/fmed.2024.1402871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Accepted: 03/25/2024] [Indexed: 04/23/2024] Open
Affiliation(s)
- Edoardo Cipolletta
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, Ancona, Italy
- Academic Rheumatology, University of Nottingham, Nottingham, United Kingdom
| | | | - Florentin Ananu Vreju
- Department of Rheumatology, University of Medicine and Pharmacy of Craiova, Craiova, Romania
| | - Sara Moccia
- Department of Excellence in Robotics and AI, The Biorobotics Institute, Scuola Superiore Sant'anna, Pisa, Italy
| | - Emilio Filippucci
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, Ancona, Italy
| |
Collapse
|
2
|
Overgaard BS, Christensen ABH, Terslev L, Savarimuthu TR, Just SA. Artificial intelligence model for segmentation and severity scoring of osteophytes in hand osteoarthritis on ultrasound images. Front Med (Lausanne) 2024; 11:1297088. [PMID: 38500949 PMCID: PMC10944993 DOI: 10.3389/fmed.2024.1297088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 02/12/2024] [Indexed: 03/20/2024] Open
Abstract
Objective To develop an artificial intelligence (AI) model able to perform both segmentation of hand joint ultrasound images for osteophytes, bone, and synovium and perform osteophyte severity scoring following the EULAR-OMERACT grading system (EOGS) for hand osteoarthritis (OA). Methods One hundred sixty patients with pain or reduced function of the hands were included. Ultrasound images of the metacarpophalangeal (MCP), proximal interphalangeal (PIP), distal interphalangeal (DIP), and first carpometacarpal (CMC1) joints were then manually segmented for bone, synovium and osteophytes and scored from 0 to 3 according to the EOGS for OA. Data was divided into a training, validation, and test set. The AI model was trained on the training data to perform bone, synovium, and osteophyte identification on the images. Based on the manually performed image segmentation, an AI was trained to classify the severity of osteophytes according to EOGS from 0 to 3. Percent Exact Agreement (PEA) and Percent Close Agreement (PCA) were assessed on individual joints and overall. PCA allows a difference of one EOGS grade between doctor assessment and AI. Results A total of 4615 ultrasound images were used for AI development and testing. The developed AI model scored on the test set for the MCP joints a PEA of 76% and PCA of 97%; for PIP, a PEA of 70% and PCA of 97%; for DIP, a PEA of 59% and PCA of 94%, and CMC a PEA of 50% and PCA of 82%. Combining all joints, we found a PEA between AI and doctor assessments of 68% and a PCA of 95%. Conclusion The developed AI model can perform joint ultrasound image segmentation and severity scoring of osteophytes, according to the EOGS. As proof of concept, this first version of the AI model is successful, as the agreement performance is slightly higher than previously found agreements between experts when assessing osteophytes on hand OA ultrasound images. The segmentation of the image makes the AI explainable to the doctor, who can immediately see why the AI applies a given score. Future validation in hand OA cohorts is necessary though.
Collapse
Affiliation(s)
- Benjamin Schultz Overgaard
- Section of Rheumatology, Department of Medicine, Svendborg Hospital – Odense University Hospital, Svendborg, Denmark
| | | | - Lene Terslev
- Center for Rheumatology and Spine Disease, Rigshospitalet, Glostrup, Denmark
| | | | - Søren Andreas Just
- Section of Rheumatology, Department of Medicine, Svendborg Hospital – Odense University Hospital, Svendborg, Denmark
| |
Collapse
|
3
|
Stoel BC, Staring M, Reijnierse M, van der Helm-van Mil AHM. Deep learning in rheumatological image interpretation. Nat Rev Rheumatol 2024; 20:182-195. [PMID: 38332242 DOI: 10.1038/s41584-023-01074-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/20/2023] [Indexed: 02/10/2024]
Abstract
Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.
Collapse
Affiliation(s)
- Berend C Stoel
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands.
| | - Marius Staring
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Monique Reijnierse
- Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | | |
Collapse
|
4
|
Lin TM, Lee HY, Chang CK, Lin KH, Chang CC, Wu BF, Peng SJ. Identification of tophi in ultrasound imaging based on transfer learning and clinical practice. Sci Rep 2023; 13:12507. [PMID: 37532752 PMCID: PMC10397312 DOI: 10.1038/s41598-023-39508-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 07/26/2023] [Indexed: 08/04/2023] Open
Abstract
Gout is a common metabolic disorder characterized by deposits of monosodium urate monohydrate crystals (tophi) in soft tissue, triggering intense and acute arthritis with intolerable pain as well as articular and periarticular inflammation. Tophi can also promote chronic inflammatory and erosive arthritis. 2015 ACR/EULAR Gout Classification criteria include clinical, laboratory, and imaging findings, where cases of gout are indicated by a threshold score of ≥ 8. Some imaging-related findings, such as a double contour sign in ultrasound, urate in dual-energy computed tomography, or radiographic gout-related erosion, generate a score of up to 4. Clearly, the diagnosis of gout is largely assisted by imaging findings; however, dual-energy computed tomography is expensive and exposes the patient to high levels of radiation. Although musculoskeletal ultrasound is non-invasive and inexpensive, the reliability of the results depends on expert experience. In the current study, we applied transfer learning to train a convolutional neural network for the identification of tophi in ultrasound images. The accuracy of predictions varied with the convolutional neural network model, as follows: InceptionV3 (0.871 ± 0.020), ResNet101 (0.913 ± 0.015), and VGG19 (0.918 ± 0.020). The sensitivity was as follows: InceptionV3 (0.507 ± 0.060), ResNet101 (0.680 ± 0.056), and VGG19 (0.747 ± 0.056). The precision was as follows: InceptionV3 (0.767 ± 0.091), ResNet101 (0.863 ± 0.098), and VGG19 (0.825 ± 0.062). Our results demonstrate that it is possible to retrain deep convolutional neural networks to identify the patterns of tophi in ultrasound images with a high degree of accuracy.
Collapse
Affiliation(s)
- Tzu-Min Lin
- Division of Allergy, Immunology and Rheumatology, Department of Internal Medicine, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Division of Rheumatology, Immunology and Allergy, Department of Internal Medicine, Taipei Medical University Hospital, Taipei, Taiwan
| | - Hsiang-Yen Lee
- Division of Rheumatology, Immunology and Allergy, Department of Internal Medicine, Taipei Medical University Hospital, Taipei, Taiwan
| | - Ching-Kuei Chang
- Division of Rheumatology, Immunology and Allergy, Department of Internal Medicine, Taipei Medical University Hospital, Taipei, Taiwan
| | - Ke-Hung Lin
- Division of Rheumatology, Immunology and Allergy, Department of Internal Medicine, Taipei Medical University Hospital, Taipei, Taiwan
| | - Chi-Ching Chang
- Division of Allergy, Immunology and Rheumatology, Department of Internal Medicine, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Division of Rheumatology, Immunology and Allergy, Department of Internal Medicine, Taipei Medical University Hospital, Taipei, Taiwan
| | - Bing-Fei Wu
- Institute of Electrical and Control Engineering, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Syu-Jyun Peng
- Professional Master Program in Artificial Intelligence in Medicine, College of Medicine, Taipei Medical University, No. 250, Wuxing St., Xinyi Dist., Taipei City, 110, Taiwan.
- Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei Medical University, Taipei, Taiwan.
| |
Collapse
|
5
|
Jacobs E, Wainman B, Bowness J. Applying artificial intelligence to the use of ultrasound as an educational tool: A focus on ultrasound-guided regional anesthesia. ANATOMICAL SCIENCES EDUCATION 2023. [PMID: 36880869 DOI: 10.1002/ase.2266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 02/10/2023] [Accepted: 02/25/2023] [Indexed: 06/18/2023]
Affiliation(s)
- Emma Jacobs
- Department of Anaesthesia, Royal Gwent Hospital, Aneurin Bevan University Health Board, Newport, UK
| | - Bruce Wainman
- Education Program in Anatomy, McMaster University, Hamilton, Ontario, Canada
- Department of Pathology and Molecular Science, McMaster University, Hamilton, Ontario, Canada
| | - James Bowness
- Department of Anaesthesia, Royal Gwent Hospital, Aneurin Bevan University Health Board, Newport, UK
- OxSTaR Center, Nuffield Division of Anaesthetics, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| |
Collapse
|
6
|
Czajkowska J, Borak M. Computer-Aided Diagnosis Methods for High-Frequency Ultrasound Data Analysis: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22218326. [PMID: 36366024 PMCID: PMC9653964 DOI: 10.3390/s22218326] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 10/21/2022] [Accepted: 10/25/2022] [Indexed: 05/31/2023]
Abstract
Over the last few decades, computer-aided diagnosis systems have become a part of clinical practice. They have the potential to assist clinicians in daily diagnostic tasks. The image processing techniques are fast, repeatable, and robust, which helps physicians to detect, classify, segment, and measure various structures. The recent rapid development of computer methods for high-frequency ultrasound image analysis opens up new diagnostic paths in dermatology, allergology, cosmetology, and aesthetic medicine. This paper, being the first in this area, presents a research overview of high-frequency ultrasound image processing techniques, which have the potential to be a part of computer-aided diagnosis systems. The reviewed methods are categorized concerning the application, utilized ultrasound device, and image data-processing type. We present the bridge between diagnostic needs and already developed solutions and discuss their limitations and future directions in high-frequency ultrasound image analysis. A search was conducted of the technical literature from 2005 to September 2022, and in total, 31 studies describing image processing methods were reviewed. The quantitative and qualitative analysis included 39 algorithms, which were selected as the most effective in this field. They were completed by 20 medical papers and define the needs and opportunities for high-frequency ultrasound application and CAD development.
Collapse
|
7
|
Felten R, Rosine N. Responding to and Driving Change in Rheumatology: Report from the 12th International Immunology Summit 2021. Rheumatol Ther 2022; 9:705-719. [PMID: 35279812 PMCID: PMC8917828 DOI: 10.1007/s40744-022-00437-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Accepted: 02/25/2022] [Indexed: 11/28/2022] Open
Abstract
The coronavirus disease 2019 (COVID-19) pandemic has accelerated changes to rheumatology daily clinical practice. The main goal of the 12th International Immunology Summit, held 25-26 June, 2021 (virtual meeting), was to provide direction for these active changes rather than undergoing change reactively in order to improve patient outcomes. This review describes and explores the concept of change in rheumatology clinical practice based on presentations from the Immunology Summit. Many of the changes to rheumatology practice brought about by the COVID-19 pandemic may be considered as having a positive impact on disease management and may help with the long-term development of more patient-focused treatment. Rheumatologists can contribute key knowledge regarding the use of immunosuppressive agents in the context of the pandemic, and according to the European League Against Rheumatism, they should be involved in any multidisciplinary COVID-19 guideline committees. New technologies, including telemedicine and artificial intelligence, represent an opportunity for physicians to individualise patient treatment and improve disease management. Despite major advances in the treatment of rheumatic diseases, the efficacy of available disease-modifying anti-rheumatic drugs (DMARDs) remains suboptimal and data regarding serological biomarkers are limited. Synovial tissue biomarkers, such as CD68+ macrophages, have shown promise in elucidating pathogenesis and targeting treatment to the individual patient. In spondyloarthritis (SpA) or psoriatic arthritis (PsA), information regarding the effectiveness of the available agents with different mechanisms of action may be integrated to manage patients using a treat-to-target approach. Early diagnosis of SpA and PsA is important for optimisation of treatment response and long-term outcomes. Improving our understanding of disease pathogenesis and practice methods may help reduce diagnostic delays, thereby optimising disease outcomes in patients with rheumatic diseases.
Collapse
Affiliation(s)
- Renaud Felten
- Service de Rhumatologie and CNR RESO, Hôpitaux Universitaires de Strasbourg, 1, Avenue Molière, BP 83049, 67098, Strasbourg, France.
| | - Nicolas Rosine
- Service de Rhumatologie, Hôpital Pitié-Salpêtrière, Sorbonne Université, Paris, France
| |
Collapse
|
8
|
High-Frequency Ultrasound Dataset for Deep Learning-Based Image Quality Assessment. SENSORS 2022; 22:s22041478. [PMID: 35214381 PMCID: PMC8875486 DOI: 10.3390/s22041478] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 02/09/2022] [Accepted: 02/12/2022] [Indexed: 12/04/2022]
Abstract
This study aims at high-frequency ultrasound image quality assessment for computer-aided diagnosis of skin. In recent decades, high-frequency ultrasound imaging opened up new opportunities in dermatology, utilizing the most recent deep learning-based algorithms for automated image analysis. An individual dermatological examination contains either a single image, a couple of pictures, or an image series acquired during the probe movement. The estimated skin parameters might depend on the probe position, orientation, or acquisition setup. Consequently, the more images analyzed, the more precise the obtained measurements. Therefore, for the automated measurements, the best choice is to acquire the image series and then analyze its parameters statistically. However, besides the correctly received images, the resulting series contains plenty of non-informative data: Images with different artifacts, noise, or the images acquired for the time stamp when the ultrasound probe has no contact with the patient skin. All of them influence further analysis, leading to misclassification or incorrect image segmentation. Therefore, an automated image selection step is crucial. To meet this need, we collected and shared 17,425 high-frequency images of the facial skin from 516 measurements of 44 patients. Two experts annotated each image as correct or not. The proposed framework utilizes a deep convolutional neural network followed by a fuzzy reasoning system to assess the acquired data’s quality automatically. Different approaches to binary and multi-class image analysis, based on the VGG-16 model, were developed and compared. The best classification results reach 91.7% accuracy for the first, and 82.3% for the second analysis, respectively.
Collapse
|
9
|
Smerilli G, Cipolletta E, Sartini G, Moscioni E, Di Cosmo M, Fiorentino MC, Moccia S, Frontoni E, Grassi W, Filippucci E. Development of a convolutional neural network for the identification and the measurement of the median nerve on ultrasound images acquired at carpal tunnel level. Arthritis Res Ther 2022; 24:38. [PMID: 35135598 PMCID: PMC8822696 DOI: 10.1186/s13075-022-02729-6] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Accepted: 01/21/2022] [Indexed: 12/28/2022] Open
Abstract
Background Deep learning applied to ultrasound (US) can provide a feedback to the sonographer about the correct identification of scanned tissues and allows for faster and standardized measurements. The most frequently adopted parameter for US diagnosis of carpal tunnel syndrome is the increasing of the cross-sectional area (CSA) of the median nerve. Our aim was to develop a deep learning algorithm, relying on convolutional neural networks (CNNs), for the localization and segmentation of the median nerve and the automatic measurement of its CSA on US images acquired at the proximal inlet of the carpal tunnel. Methods Consecutive patients with rheumatic and musculoskeletal disorders were recruited. Transverse US images were acquired at the carpal tunnel inlet, and the CSA was manually measured. Anatomical variants were registered. The dataset consisted of 246 images (157 for training, 40 for validation, and 49 for testing) from 103 patients each associated with manual annotations of the nerve boundary. A Mask R-CNN, state-of-the-art CNN for image semantic segmentation, was trained on this dataset to accurately localize and segment the median nerve section. To evaluate the performances on the testing set, precision (Prec), recall (Rec), mean average precision (mAP), and Dice similarity coefficient (DSC) were computed. A sub-analysis excluding anatomical variants was performed. The CSA was automatically measured by the algorithm. Results The algorithm correctly identified the median nerve in 41/49 images (83.7%) and in 41/43 images (95.3%) excluding anatomical variants. The following metrics were obtained (with and without anatomical variants, respectively): Prec 0.86 ± 0.33 and 0.96 ± 0.18, Rec 0.88 ± 0.33 and 0.98 ± 0.15, mAP 0.88 ± 0.33 and 0.98 ± 0.15, and DSC 0.86 ± 0.19 and 0.88 ± 0.19. The agreement between the algorithm and the sonographer CSA measurements was excellent [ICC 0.97 (0.94–0.98)]. Conclusions The developed algorithm has shown excellent performances, especially if excluding anatomical variants. Future research should aim at expanding the US image dataset including a wider spectrum of normal anatomy and pathology. This deep learning approach has shown very high potentiality for a fully automatic support for US assessment of carpal tunnel syndrome.
Collapse
Affiliation(s)
- Gianluca Smerilli
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy.
| | - Edoardo Cipolletta
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy
| | - Gianmarco Sartini
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy
| | - Erica Moscioni
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy
| | - Mariachiara Di Cosmo
- Department of Information Engineering, Polytechnic University of Marche, Ancona, Italy
| | | | - Sara Moccia
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Emanuele Frontoni
- Department of Information Engineering, Polytechnic University of Marche, Ancona, Italy
| | - Walter Grassi
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy
| | - Emilio Filippucci
- Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, "Carlo Urbani" Hospital, Via Aldo Moro 25, 60035, Jesi, Ancona, Italy
| |
Collapse
|
10
|
Pane S, Faoro G, Sinibaldi E, Iacovacci V, Menciassi A. Ultrasound Acoustic Phase Analysis Enables Robotic Visual-Servoing of Magnetic Microrobots. IEEE T ROBOT 2022. [DOI: 10.1109/tro.2022.3143072] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
11
|
Current Status and Advancement of Ultrasound Imaging Technologies in Musculoskeletal Studies. CURRENT PHYSICAL MEDICINE AND REHABILITATION REPORTS 2021. [DOI: 10.1007/s40141-021-00337-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|