Detection of rheumatoid arthritis from hand radiographs using a convolutional neural network

The biomedical applications of artificial intelligence: an overview of decades of research

A significant area of computer science called artificial intelligence (AI) is successfully applied to the analysis of intricate biological data and the extraction of substantial associations from datasets for a variety of biomedical uses. AI has attracted significant interest in biomedical research due to its features: (i) better patient care through early diagnosis and detection; (ii) enhanced workflow; (iii) lowering medical errors; (v) lowering medical costs; (vi) reducing morbidity and mortality; (vii) enhancing performance; (viii) enhancing precision; and (ix) time efficiency. Quantitative metrics are crucial for evaluating AI implementations, providing insights, enabling informed decisions, and measuring the impact of AI-driven initiatives, thereby enhancing transparency, accountability, and overall impact. The implementation of AI in biomedical fields faces challenges such as ethical and privacy concerns, lack of awareness, technology unreliability, and professional liability. A brief discussion is given of the AI techniques, which include Virtual screening (VS), DL, ML, Hidden Markov models (HMMs), Neural networks (NNs), Generative models (GMs), Molecular dynamics (MD), and Structure-activity relationship (SAR) models. The study explores the application of AI in biomedical fields, highlighting its enhanced predictive accuracy, treatment efficacy, diagnostic efficiency, faster decision-making, personalised treatment strategies, and precise medical interventions.

Algorithmic approaches in hand imaging for rheumatic musculoskeletal diseases: A systematic literature review

OBJECTIVE

This systematic literature review provides a comprehensive overview of the use of machine learning (ML) in hand imaging of rheumatic musculoskeletal diseases (RMDs). The review evaluates ML algorithms, imaging modalities, patient populations, validation methods, and areas for improvement.

METHODS

The review was conducted following PRISMA guidelines and registered with PROSPERO. Articles were retrieved from PubMed, EMBASE, and Scopus using relevant MeSH terms and keywords. The search, executed in October 2024, was conducted manually and with BiBot, an AI-based tool for literature reviews. Studies focusing on ML applications in osteoarthritis (OA), rheumatoid arthritis (RA), and psoriatic arthritis (PsA) were included.

RESULTS

From 400 initially identified studies, 32 met the inclusion criteria. RA was the most studied disease (88 %), followed by OA (22 %) and PsA (9 %). Convolutional neural networks (CNNs) were the most frequently used algorithms (50 %). Standard radiographs (59 %) were the predominant imaging modality, followed by MRI (16 %). Despite recommendations for ML studies, external validation was conducted in only 15 % of studies, and just 6 % of datasets were publicly available. Interpretability tools were employed in 28 % of studies to enhance clinical relevance.

CONCLUSION

ML has significant potential to improve diagnostics and disease management in hand imaging of RMDs. However, key challenges remain, including the need for increased external validation, broader disease coverage (OA and PsA), and improved data-sharing practices to enhance reproducibility and clinical adoption.

Deep learning analysis for rheumatologic imaging: current trends, future directions, and the role of human

Rheumatic diseases, such as rheumatoid arthritis (RA), osteoarthritis (OA), and spondyloarthritis (SpA), present diagnostic and management challenges due to their impact on connective tissues and the musculoskeletal system. Traditional imaging techniques, including plain radiography, ultrasounds, computed tomography, and magnetic resonance imaging (MRI), play a critical role in diagnosing and monitoring these conditions, but face limitations like inter-observer variability and time-consuming assessments. Recently, deep learning (DL), a subset of artificial intelligence, has emerged as a promising tool for enhancing medical imaging analysis. Convolutional neural networks, a DL model type, have shown great potential in medical image classification, segmentation, and anomaly detection, often surpassing human performance in tasks like tumor identification and disease severity grading. In rheumatology, DL models have been applied to plain radiography, ultrasounds, and MRI for assessing joint damage, synovial inflammation, and disease progression in RA, OA, and SpA patients. Despite the promise of DL, challenges such as data bias, limited explainability, and the need for large annotated datasets remain significant barriers to its widespread adoption. Furthermore, human oversight and value judgment are essential for ensuring the ethical use and effective implementation of DL in clinical settings. This review provides a comprehensive overview of DL's applications in rheumatologic imaging and explores its future potential in enhancing diagnosis, treatment decisions, and personalized medicine.

Predicting autoimmune diseases: A comprehensive review of classic biomarkers and advances in artificial intelligence

Autoimmune diseases comprise a spectrum of disorders characterized by the dysregulation of immune tolerance, resulting in tissue or organ damage and inflammation. Their prevalence has been on the rise, significantly impacting patients' quality of life and escalating healthcare costs. Consequently, the prediction of autoimmune diseases has recently garnered substantial interest among researchers. Despite their wide heterogeneity, many autoimmune diseases exhibit a consistent pattern of paraclinical findings that hold predictive value. From serum biomarkers to various machine learning approaches, the array of available methods has been continuously expanding. The emergence of artificial intelligence (AI) presents an exciting new range of possibilities, with notable advancements already underway. The ultimate objective should revolve around disease prevention across all levels. This review provides a comprehensive summary of the most recent data pertaining to the prediction of diverse autoimmune diseases and encompasses both traditional biomarkers and the latest innovations in AI.

Radiograph-based rheumatoid arthritis diagnosis via convolutional neural network

OBJECTIVES

Rheumatoid arthritis (RA) is a severe and common autoimmune disease. Conventional diagnostic methods are often subjective, error-prone, and repetitive works. There is an urgent need for a method to detect RA accurately. Therefore, this study aims to develop an automatic diagnostic system based on deep learning for recognizing and staging RA from radiographs to assist physicians in diagnosing RA quickly and accurately.

METHODS

We develop a CNN-based fully automated RA diagnostic model, exploring five popular CNN architectures on two clinical applications. The model is trained on a radiograph dataset containing 240 hand radiographs, of which 39 are normal and 201 are RA with five stages. For evaluation, we use 104 hand radiographs, of which 13 are normal and 91 RA with five stages.

RESULTS

The CNN model achieves good performance in RA diagnosis based on hand radiographs. For the RA recognition, all models achieve an AUC above 90% with a sensitivity over 98%. In particular, the AUC of the GoogLeNet-based model is 97.80%, and the sensitivity is 100.0%. For the RA staging, all models achieve over 77% AUC with a sensitivity over 80%. Specifically, the VGG16-based model achieves 83.36% AUC with 92.67% sensitivity.

CONCLUSION

The presented GoogLeNet-based model and VGG16-based model have the best AUC and sensitivity for RA recognition and staging, respectively. The experimental results demonstrate the feasibility and applicability of CNN in radiograph-based RA diagnosis. Therefore, this model has important clinical significance, especially for resource-limited areas and inexperienced physicians.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Deep learning discrimination of rheumatoid arthritis from osteoarthritis on hand radiography

PURPOSE

To develop a deep learning model to distinguish rheumatoid arthritis (RA) from osteoarthritis (OA) using hand radiographs and to evaluate the effects of changing pretraining and training parameters on model performance.

MATERIALS AND METHODS

A convolutional neural network was retrospectively trained on 9714 hand radiograph exams from 8387 patients obtained from 2017 to 2021 at seven hospitals within an integrated healthcare network. Performance was assessed using an independent test set of 250 exams from 146 patients. Binary discriminatory capacity (no arthritis versus arthritis; RA versus not RA) and three-way classification (no arthritis versus OA versus RA) were evaluated. The effects of additional pretraining using musculoskeletal radiographs, using all views as opposed to only the posteroanterior view, and varying image resolution on model performance were also investigated. Area under the receiver operating characteristic curve (AUC) and Cohen's kappa coefficient were used to evaluate diagnostic performance.

RESULTS

For no arthritis versus arthritis, the model achieved an AUC of 0.975 (95% CI: 0.957, 0.989). For RA versus not RA, the model achieved an AUC of 0.955 (95% CI: 0.919, 0.983). For three-way classification, the model achieved a kappa of 0.806 (95% CI: 0.742, 0.866) and accuracy of 87.2% (95% CI: 83.2%, 91.2%) on the test set. Increasing image resolution increased performance up to 1024 × 1024 pixels. Additional pretraining on musculoskeletal radiographs and using all views did not significantly affect performance.

CONCLUSION

A deep learning model can be used to distinguish no arthritis, OA, and RA on hand radiographs with high performance.

Multi-pose-based convolutional neural network model for diagnosis of patients with central lumbar spinal stenosis

Although the role of plain radiographs in diagnosing lumbar spinal stenosis (LSS) has declined in importance since the advent of magnetic resonance imaging (MRI), diagnostic ability of plain radiographs has improved dramatically when combined with deep learning. Previously, we developed a convolutional neural network (CNN) model using a radiograph for diagnosing LSS. In this study, we aimed to improve and generalize the performance of CNN models and overcome the limitation of the single-pose-based CNN (SP-CNN) model using multi-pose radiographs. Individuals with severe or no LSS, confirmed using MRI, were enrolled. Lateral radiographs of patients in three postures were collected. We developed a multi-pose-based CNN (MP-CNN) model using the encoders of the three SP-CNN model (extension, flexion, and neutral postures). We compared the validation results of the MP-CNN model using four algorithms pretrained with ImageNet. The MP-CNN model underwent additional internal and external validations to measure generalization performance. The ResNet50-based MP-CNN model achieved the largest area under the receiver operating characteristic curve (AUROC) of 91.4% (95% confidence interval [CI] 90.9-91.8%) for internal validation. The AUROC of the MP-CNN model were 91.3% (95% CI 90.7-91.9%) and 79.5% (95% CI 78.2-80.8%) for the extra-internal and external validation, respectively. The MP-CNN based heatmap offered a logical decision-making direction through optimized visualization. This model holds potential as a screening tool for LSS diagnosis, offering an explainable rationale for its prediction.

Artificial intelligence in rheumatoid arthritis: potential applications and future implications

The widespread adoption of digital health records, coupled with the rise of advanced diagnostic testing, has resulted in an explosion of patient data, comparable in scope to genomic datasets. This vast information repository offers significant potential for improving patient outcomes and decision-making, provided one can extract meaningful insights from it. This is where artificial intelligence (AI) tools like machine learning (ML) and deep learning come into play, helping us leverage these enormous datasets to predict outcomes and make informed decisions. AI models can be trained to analyze and interpret patient data, including physician notes, laboratory testing, and imaging, to aid in the management of patients with rheumatic diseases. As one of the most common autoimmune diseases, rheumatoid arthritis (RA) has attracted considerable attention, particularly concerning the evolution of diagnostic techniques and therapeutic interventions. Our aim is to underscore those areas where AI, according to recent research, demonstrates promising potential to enhance the management of patients with RA.

Deep Learned Segmentations of Inflammation for Novel ⁹⁹mTc-maraciclatide Imaging of Rheumatoid Arthritis

Rheumatoid arthritis (RA) is an autoimmune disease that causes joint pain, stiffness, and erosion. Power Doppler ultrasound and MRI are imaging modalities used in detecting and monitoring the disease, but they have limitations. ⁹⁹mTc-maraciclatide gamma camera imaging is a novel technique that can detect joint inflammation at all sites in a single examination and has been shown to correlate with power Doppler ultrasound. In this work, we investigate if machine learning models can be used to automatically segment regions of normal, low, and highly inflamed tissue from 192 ⁹⁹mTc-maraciclatide scans of the hands and wrists from 48 patients. Two models were trained: a thresholding model that learns lower and upper threshold values and a neural-network-based nnU-Net model that uses a convolutional neural network (CNN). The nnU-Net model showed 0.94 ± 0.01, 0.51 ± 0.14, and 0.76 ± 0.16 modified Dice scores for segmenting the normal, low, and highly inflamed tissue, respectively, when compared to clinical segmented labels. This outperforms the thresholding model, which achieved modified Dice scores of 0.92 ± 0.01, 0.14 ± 0.07, and 0.35 ± 0.21, respectively. This is an important first step in developing artificial intelligence (AI) tools to assist clinicians' workflow in the use of this new radiopharmaceutical.

Artificial intelligence in patient-specific hand surgery: a scoping review of literature

PURPOSE

The implementation of artificial intelligence in hand surgery and rehabilitation is gaining popularity. The purpose of this scoping review was to give an overview of implementations of artificial intelligence in hand surgery and rehabilitation and their current significance in clinical practice.

METHODS

A systematic literature search of the MEDLINE/PubMed and Cochrane Collaboration libraries was conducted. The review was conducted according to the framework outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews. A narrative summary of the papers is presented to give an orienting overview of this rapidly evolving topic.

RESULTS

Primary search yielded 435 articles. After application of the inclusion/exclusion criteria and addition of supplementary search, 235 articles were included in the final review. In order to facilitate navigation through this heterogenous field, the articles were clustered into four groups of thematically related publications. The most common applications of artificial intelligence in hand surgery and rehabilitation target automated image analysis of anatomic structures, fracture detection and localization and automated screening for other hand and wrist pathologies such as carpal tunnel syndrome, rheumatoid arthritis or osteoporosis. Compared to other medical subspecialties the number of applications in hand surgery is still small.

CONCLUSION

Although various promising applications of artificial intelligence in hand surgery and rehabilitation show strong performances, their implementation mostly takes place within the context of experimental studies. Therefore, their use in daily clinical routine is still limited.

Medical imaging in rheumatoid arthritis: A review on deep learning approach

Arthritis is a musculoskeletal disorder. Millions of people have arthritis, making it one of the most common joint disorders. Osteoarthritis (OA) and rheumatoid arthritis (RA) are the most common types of arthritis among the many different types available. Pain, stiffness, and inflammation are among the early signs of arthritis, which can progress to severe immobility at a later stage if left untreated. Although arthritis cannot be cured at any point in time, it can be managed if diagnosed and treated correctly. Clinical diagnostic and medical imaging methods are currently used to evaluate OA and RA, both debilitating conditions. This review is focused on deep learning approaches used by taking medical imaging (X-rays and magnetic resonance imaging) as input for the detection of RA.

A survey of artificial intelligence in rheumatoid arthritis

The article offers a survey of currently notable artificial intelligence methods (released between 2019-2023), with a particular emphasis on the latest advancements in detecting rheumatoid arthritis (RA) at an early stage, providing early treatment, and managing the disease. We discussed challenges in these areas followed by specific artificial intelligence (AI) techniques and summarized advances, relevant strengths, and obstacles. Overall, the application of AI in the fields of RA has the potential to enable healthcare professionals to detect RA at an earlier stage, thereby facilitating timely intervention and better disease management. However, more research is required to confirm the precision and dependability of AI in RA, and several problems such as technological and ethical concerns related to these approaches must be resolved before their widespread adoption.

Artificial Intelligence in Rheumatoid Arthritis: Current Status and Future Perspectives: A State-of-the-Art Review

Investigation of the potential applications of artificial intelligence (AI), including machine learning (ML) and deep learning (DL) techniques, is an exponentially growing field in medicine and healthcare. These methods can be critical in providing high-quality care to patients with chronic rheumatological diseases lacking an optimal treatment, like rheumatoid arthritis (RA), which is the second most prevalent autoimmune disease. Herein, following reviewing the basic concepts of AI, we summarize the advances in its applications in RA clinical practice and research. We provide directions for future investigations in this field after reviewing the current knowledge gaps and technical and ethical challenges in applying AI. Automated models have been largely used to improve RA diagnosis since the early 2000s, and they have used a wide variety of techniques, e.g., support vector machine, random forest, and artificial neural networks. AI algorithms can facilitate screening and identification of susceptible groups, diagnosis using omics, imaging, clinical, and sensor data, patient detection within electronic health record (EHR), i.e., phenotyping, treatment response assessment, monitoring disease course, determining prognosis, novel drug discovery, and enhancing basic science research. They can also aid in risk assessment for incidence of comorbidities, e.g., cardiovascular diseases, in patients with RA. However, the proposed models may vary significantly in their performance and reliability. Despite the promising results achieved by AI models in enhancing early diagnosis and management of patients with RA, they are not fully ready to be incorporated into clinical practice. Future investigations are required to ensure development of reliable and generalizable algorithms while they carefully look for any potential source of bias or misconduct. We showed that a growing body of evidence supports the potential role of AI in revolutionizing screening, diagnosis, and management of patients with RA. However, multiple obstacles hinder clinical applications of AI models. Incorporating the machine and/or deep learning algorithms into real-world settings would be a key step in the progress of AI in medicine.

Automated evaluation of rheumatoid arthritis from hand radiographs using Machine Learning and deep learning techniques

The aim and objectives of the study are as follows: (i) to implement automated patch-based classification of hand X-ray images using modified pre-trained convolutional neural network (CNN) models; (ii) to develop a customized CNN model for automated feature extraction and classification of hand X-ray images and to compare the performance of customized CNN models with non-linear and linear kernels; (iii) to construct the hand crafted feature fusion (SIFT+ Customized CNN features) and categorize the normal and RA using Machine Learning classifiers. The model was trained on 75 images (10,000 patches) of hand radiographs and tested using 25 images (500 patches) that were not included in the training set. The accuracy of the modified pre-trained model GoogLeNet was 89% and the proposed custom model three achieved an accuracy of 95%. The sensitivity and specificity of GoogLeNet were 84% and 90% respectively. The custom model three attained the sensitivity and specificity as 95% and 94% respectively. Furthermore, when compared to the features extracted (SIFT + CNN) from the customized models, the custom3 model outperformed well for the classification of RA compared to ML classifiers. Thus a custom CNN-based computer-aided diagnostic tool can be used as an effective method for the detection of RA.

Diagnostic triage in patients with central lumbar spinal stenosis using a deep learning system of radiographs

OBJECTIVE

Magnetic resonance imaging (MRI) is the gold-standard tool for diagnosing lumbar spinal stenosis (LSS), but it is difficult to promptly examine all suspected cases with MRI considering the modality's high cost and limited accessibility. Although radiography is an efficient screening technique owing to its low cost, rapid operability, and wide availability, its diagnostic accuracy is relatively poor. In this study, the authors aimed to develop a deep learning model with a convolutional neural network (CNN) for diagnosing severe central LSS using radiography and to evaluate radiological diagnostic features using gradient-weighted class activation mapping (Grad-CAM).

METHODS

Patients who had undergone both spinal MRI and radiography in the period from May 1, 2005, to December 31, 2017, were screened. According to the formal MRI report, participants were consecutively included in the severe central LSS or healthy control group, and radiographs for both groups were collected. A CNN-based transfer learning algorithm was developed to classify radiographic findings as LSS or normal (binary classification). The proposed models were evaluated using six performance metrics: area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and positive and negative predictive values.

RESULTS

The VGG19 model achieved the highest accuracy with an AUROC of 90.0% (95% CI 89.8%-90.3%) by training 12,442 images. Accuracy was 82.8% (95% CI 82.5%-83.1%) by averaging 5-fold models. Feature points on Grad-CAM were reasonable, and the features could be categorized into reduced disc height, narrow foramina, short pedicle, and hyperdense facet joint. The AUROC in the extra validation was 89.3% (95% CI 88.7%-90.0%). Accuracy was 81.8% (95% CI 80.6%-83.0%) by averaging 5-fold models. Multivariate logistic regression analysis showed that a combination of demographic factors (age and sex) did not improve the model performance.

CONCLUSIONS

The algorithm trained by a CNN to identify central LSS on radiographs showed high diagnostic accuracy and is expected to be useful as a triage tool. The algorithm could accurately localize the stenotic lesion to assist physicians in the identification of LSS.

Increasing trend of radiographic features of knee osteoarthritis in rheumatoid arthritis patients before total knee arthroplasty

To investigate the trend and factors related to the occurrence of osteoarthritis (OA)-like features on knee radiographs of rheumatoid arthritis (RA) patients undergoing total knee arthroplasty (TKA) in the recent decades. To classify antero-posterior knee radiographs into 'RA' and 'OA-like RA' groups, a deep learning model was developed by training the network using knee radiographs of end-stage arthropathy in RA patients obtained during 2002-2005 and in primary OA patients obtained during 2007-2009. We used this model to categorize 796 knee radiographs, which were recorded in RA patients before TKA during 2006-2020, into 'OA-like RA' and 'RA' groups. The annual ratio of 'OA-like RA' was investigated. Moreover, univariate and multivariate analyses were performed to identify the factors associated with the classification as OA-like RA using clinical data from 240 patients. The percentage of 'OA-like RA' had significant increasing trend from 20.9% in 2006 to 67.7% in 2020. Higher body mass index, use of biologics, and lower level of C-reactive protein were identified as independent factors for 'OA-like RA'. An increasing trend of knee radiographs with OA-like features was observed in RA patients in the recent decades, which might be attributed to recent advances in pharmacotherapy.

Improved diagnosis of rheumatoid arthritis using an artificial neural network

Rheumatoid arthritis (RA) is chronic systemic disease that can cause joint damage, disability and destructive polyarthritis. Current diagnosis of RA is based on a combination of clinical and laboratory features. However, RA diagnosis can be difficult at its disease onset on account of overlapping symptoms with other arthritis, so early recognition and diagnosis of RA permit the better management of patients. In order to improve the medical diagnosis of RA and evaluate the effects of different clinical features on RA diagnosis, we applied an artificial neural network (ANN) as the training algorithm, and used fivefold cross-validation to evaluate its performance. From each sample, we obtained data on 6 features: age, sex, rheumatoid factor, anti-citrullinated peptide antibody (CCP), 14-3-3η, and anti-carbamylated protein (CarP) antibodies. After training, this ANN model assigned each sample a probability for being either an RA patient or a non-RA patient. On the validation dataset, the F1 for all samples by this ANN model was 0.916, which was higher than the 0.906 we previously reported using an optimal threshold algorithm. Therefore, this ANN algorithm not only improved the accuracy of RA diagnosis, but also revealed that anti-CCP had the greatest effect while age and anti-CarP had a weaker on RA diagnosis.

Deep Learning-Based Computer-Aided Diagnosis of Rheumatoid Arthritis with Hand X-ray Images Conforming to Modified Total Sharp/van der Heijde Score

Introduction: Rheumatoid arthritis (RA) is a systemic autoimmune disease; early diagnosis and treatment are crucial for its management. Currently, the modified total Sharp score (mTSS) is widely used as a scoring system for RA. The standard screening process for assessing mTSS is tedious and time-consuming. Therefore, developing an efficient mTSS automatic localization and classification system is of urgent need for RA diagnosis. Current research mostly focuses on the classification of finger joints. Due to the insufficient detection ability of the carpal part, these methods cannot cover all the diagnostic needs of mTSS. Method: We propose not only an automatic label system leveraging the You Only Look Once (YOLO) model to detect the regions of joints of the two hands in hand X-ray images for preprocessing of joint space narrowing in mTSS, but also a joint classification model depending on the severity of the mTSS-based disease. In the image processing of the data, the window level is used to simulate the processing method of the clinician, the training data of the different carpal and finger bones of human vision are separated and integrated, and the resolution is increased or decreased to observe the changes in the accuracy of the model. Results: Integrated data proved to be beneficial. The mean average precision of the proposed model in joint detection of joint space narrowing reached 0.92, and the precision, recall, and F1 score all reached 0.94 to 0.95. For the joint classification, the average accuracy was 0.88, and the accuracy of severe, mild, and healthy reached 0.91, 0.79, and 0.9, respectively. Conclusions: The proposed model is feasible and efficient. It could be helpful for subsequent research on computer-aided diagnosis in RA. We suggest that applying the one-hand X-ray imaging protocol can improve the accuracy of mTSS classification model in determining mild disease if it is used in clinical practice.

Deep Learning-Based Classification of Inflammatory Arthritis by Identification of Joint Shape Patterns—How Neural Networks Can Tell Us Where to “Deep Dive” Clinically

Objective:

We investigated whether a neural network based on the shape of joints can differentiate between rheumatoid arthritis (RA), psoriatic arthritis (PsA), and healthy controls (HC), which class patients with undifferentiated arthritis (UA) are assigned to, and whether this neural network is able to identify disease-specific regions in joints.

Methods

We trained a novel neural network on 3D articular bone shapes of hand joints of RA and PsA patients as well as HC. Bone shapes were created from high-resolution peripheral-computed-tomography (HR-pQCT) data of the second metacarpal bone head. Heat maps of critical spots were generated using GradCAM. After training, we fed shape patterns of UA into the neural network to classify them into RA, PsA, or HC.

Results

Hand bone shapes from 932 HR-pQCT scans of 617 patients were available. The network could differentiate the classes with an area-under-receiver-operator-curve of 82% for HC, 75% for RA, and 68% for PsA. Heat maps identified anatomical regions such as bare area or ligament attachments prone to erosions and bony spurs. When feeding UA data into the neural network, 86% were classified as “RA,” 11% as “PsA,” and 3% as “HC” based on the joint shape.

Conclusion

We investigated neural networks to differentiate the shape of joints of RA, PsA, and HC and extracted disease-specific characteristics as heat maps on 3D joint shapes that can be utilized in clinical routine examination using ultrasound. Finally, unspecific diseases such as UA could be grouped using the trained network based on joint shape.

A deep learning classification of metacarpophalangeal joints synovial proliferation in rheumatoid arthritis by ultrasound images

OBJECTIVE

To evaluate if an automatic classification of rheumatoid arthritis (RA) metacarpophalangeal joint conditions in ultrasound images is feasible by deep learning (DL) method, to provide a more objective, automated, and fast way of RA diagnosis in clinical setting.

MATERIALS AND METHODS

DenseNet-based DL model was used and both training and testing are implemented in TensorFlow 1.13.1 with Keras DL libraries. The area under curve (AUC), accuracy, sensitivity, and specificity values with 95% CIs were reported. The statistical analysis was performed by using scikit-learn libraries in Python 3.7.

RESULTS

A total of 1337 RA ultrasound images were acquired from 208 patients, the number of images is 313, 657, 178, and 189 in OESS Grade L0, L1, L2, and L3, respectively. In Classification Scenario 1 SP-no versus SP-yes, three experiments with region of interest of size 192 × 448 (Group 1), 96 × 224 (Group 2), and 96 × 224 stacked with pre-segmented annotated mask of SP area (Group 3) as input achieve an AUC of 0.863 (95% CI: 0.809, 0.917), 0.861 (95% CI: 0.805, 0.916), and 0.886 (95% CI: 0.836, 0.936), respectively. In Classification Scenario 2 Healthy versus Diseased, experiments in Group 1, Group 2 and Group 3 achieve an AUC of 0.848 (95% CI: 0.799, 0.896), 0.864 (95% CI: 0.819, 0.909), and 0.916 (95% CI: 0.883, 0.952), respectively.

CONCLUSION

We combined DenseNet model with ultrasound images for RA condition assessment. The feasibility of using DL to create an automatic RA condition classification system was also demonstrated. The proposed method can be an alternative to the initial screening of RA patients.

Automated Classification of Rheumatoid Arthritis, Osteoarthritis, and Normal Hand Radiographs with Deep Learning Methods

Rheumatoid arthritis and hand osteoarthritis are two different arthritis that causes pain, function limitation, and permanent joint damage in the hands. Plain hand radiographs are the most commonly used imaging methods for the diagnosis, differential diagnosis, and monitoring of rheumatoid arthritis and osteoarthritis. In this retrospective study, the You Only Look Once (YOLO) algorithm was used to obtain hand images from original radiographs without data loss, and classification was made by applying transfer learning with a pre-trained VGG-16 network. The data augmentation method was applied during training. The results of the study were evaluated with performance metrics such as accuracy, sensitivity, specificity, and precision calculated from the confusion matrix, and AUC (area under the ROC curve) calculated from ROC (receiver operating characteristic) curve. In the classification of rheumatoid arthritis and normal hand radiographs, 90.7%, 92.6%, 88.7%, 89.3%, and 0.97 accuracy, sensitivity, specificity, precision, and AUC results, respectively, and in the classification of osteoarthritis and normal hand radiographs, 90.8%, 91.4%, 90.2%, 91.4%, and 0.96 accuracy, sensitivity, specificity, precision, and AUC results were obtained, respectively. In the classification of rheumatoid arthritis, osteoarthritis, and normal hand radiographs, an 80.6% accuracy result was obtained. In this study, to develop an end-to-end computerized method, the YOLOv4 algorithm was used for object detection, and a pre-trained VGG-16 network was used for the classification of hand radiographs. This computer-aided diagnosis method can assist clinicians in interpreting hand radiographs, especially in rheumatoid arthritis and osteoarthritis.

Toward Overcoming Treatment Failure in Rheumatoid Arthritis

Rheumatoid arthritis (RA) is an autoimmune disorder characterized by inflammation and bone erosion. The exact mechanism of RA is still unknown, but various immune cytokines, signaling pathways and effector cells are involved. Disease-modifying antirheumatic drugs (DMARDs) are commonly used in RA treatment and classified into different categories. Nevertheless, RA treatment is based on a "trial-and-error" approach, and a substantial proportion of patients show failed therapy for each DMARD. Over the past decades, great efforts have been made to overcome treatment failure, including identification of biomarkers, exploration of the reasons for loss of efficacy, development of sequential or combinational DMARDs strategies and approval of new DMARDs. Here, we summarize these efforts, which would provide valuable insights for accurate RA clinical medication. While gratifying, researchers realize that these efforts are still far from enough to recommend specific DMARDs for individual patients. Precision medicine is an emerging medical model that proposes a highly individualized and tailored approach for disease management. In this review, we also discuss the potential of precision medicine for overcoming RA treatment failure, with the introduction of various cutting-edge technologies and big data.

Rheumatoid Arthritis Diagnosis: Deep Learning vs. Humane

Rheumatoid arthritis (RA) is a systemic autoimmune disease that preferably affects small joints. As the well-timed diagnosis of the disease is essential for the treatment of the patient, several works have been conducted in the field of deep learning to develop fast and accurate automatic methods for RA diagnosis. These works mainly focus on medical images as they use X-ray and ultrasound images as input for their models. In this study, we review the conducted works and compare the methods that use deep learning with the procedure that is commonly followed by a medical doctor for the RA diagnosis. The results show that 93% of the works use only image modalities as input for the models as distinct from the medical procedure where more patient medical data are taken into account. Moreover, only 15% of the works use direct explainability methods, meaning that the efforts for solving the trustworthiness issue of deep learning models were limited. In this context, this work reveals the gap between the deep learning approaches and the medical doctors’ practices traditionally applied and brings to light the weaknesses of the current deep learning technology to be integrated into a trustworthy context inside the existed medical infrastructures.

A generalized deep learning framework for automatic rheumatoid arthritis severity grading

Knee rheumatoid arthritis (RA) is the highly prevalent, chronic, progressive condition in the world. To diagnose this disease in the early stage in detail analysis with magnetic resonance (MR) image is possible. The imaging modality feature allows unbiased assessment of joint space narrowing (JSN), cartilage volume, and other vital features. This provides a fine-grained RA severity evaluation of the knee, contrasted to the benchmark, and generally used Kellgren Lawrence (KL) assessment. In this research, an intelligent system is developed to predict KL grade from the knee dataset. Our approach is based on hybrid deep learning of 50 layers (ResNet50) with skip connections. The proposed approach also uses Adam optimizer to provide learning linearity in the training stage. Our approach yields KL grade and JSN for femoral and tibial tissue with lateral and medial compartments. Furthermore, the approach also yields area under curve (AUC) of 0.98, accuracy 96.85%, mean absolute error (MAE) 0.015, precision 98.31%, and other commonly used parameters for the existence of radiographic RA progression which is improved than the existing state-of-the-art.

An introduction to machine learning and analysis of its use in rheumatic diseases

Machine learning (ML) is a computerized analytical technique that is being increasingly employed in biomedicine. ML often provides an advantage over explicitly programmed strategies in the analysis of multidimensional information by recognizing relationships in the data that were not previously appreciated. As such, the use of ML in rheumatology is increasing, and numerous studies have employed ML to classify patients with rheumatic autoimmune inflammatory diseases (RAIDs) from medical records and imaging, biometric or gene expression data. However, these studies are limited by sample size, the accuracy of sample labelling, and absence of datasets for external validation. In addition, there is potential for ML models to overfit or underfit the data and, thereby, these models might produce results that cannot be replicated in an unrelated dataset. In this Review, we introduce the basic principles of ML and discuss its current strengths and weaknesses in the classification of patients with RAIDs. Moreover, we highlight the successful analysis of the same type of input data (for example, medical records) with different algorithms, illustrating the potential plasticity of this analytical approach. Altogether, a better understanding of ML and the future application of advanced analytical techniques based on this approach, coupled with the increasing availability of biomedical data, may facilitate the development of meaningful precision medicine for patients with RAIDs.

Machine Learning in Rheumatic Diseases

With advances in information technology, the demand for using data science to enhance healthcare and disease management is rapidly increasing. Among these technologies, machine learning (ML) has become ubiquitous and indispensable for solving complex problems in many scientific fields, including medical science. ML allows the development of guidelines and framing of the evaluation system for complex diseases based on massive data. In the analysis of rheumatic diseases, which are chronic and remarkably heterogeneous, ML can be anticipated to be extremely helpful in deciphering and revealing the inherent interrelationships in disease development and progression, which can further enhance the overall understanding of the disease, optimize patients' stratification, calibrate therapeutic strategies, and predict prognosis and outcomes. In this review, the basics of ML, its potential clinical applications in rheumatology, together with its strengths and limitations are summarized.

An Efficient CNN for Hand X-Ray Classification of Rheumatoid Arthritis

Hand Radiography (RA) is one of the prime tests for checking the progress of rheumatoid joint inflammation in human bone joints. Recognizing the specific phase of RA is a difficult assignment, as human abilities regularly curb the techniques for it. Convolutional neural network (CNN) is the center for hand recognition for recognizing complex examples. The human cerebrum capacities work in a high-level way, so CNN has been planned depending on organic neural-related organizations in humans for imitating its unpredictable capacities. This article accordingly presents the convolutional neural network (CNN) which has the ability to naturally gain proficiency with the qualities and anticipate the class of hand radiographs from an expansive informational collection. The reproduction of the CNN halfway layers, which depict the elements of the organization, is likewise appeared. For arrangement of the model, a dataset of 290 radiography images is utilized. The result indicates that hand X-rays are rated with an accuracy of 94.46% by the proposed methodology. Our experiments show that the network sensitivity is observed to be 0.95 and the specificity is observed to be 0.82.

Practical applications of deep learning: classifying the most common categories of plain radiographs in a PACS using a neural network

OBJECTIVES

The goal of the present study was to classify the most common types of plain radiographs using a neural network and to validate the network's performance on internal and external data. Such a network could help improve various radiological workflows.

METHODS

All radiographs from the year 2017 (n = 71,274) acquired at our institution were retrieved from the PACS. The 30 largest categories (n = 58,219, 81.7% of all radiographs performed in 2017) were used to develop and validate a neural network (MobileNet v1.0) using transfer learning. Image categories were extracted from DICOM metadata (study and image description) and mapped to the WHO manual of diagnostic imaging. As an independent, external validation set, we used images from other institutions that had been stored in our PACS (n = 5324).

RESULTS

In the internal validation, the overall accuracy of the model was 90.3% (95%CI: 89.2-91.3%), whereas, for the external validation set, the overall accuracy was 94.0% (95%CI: 93.3-94.6%).

CONCLUSIONS

Using data from one single institution, we were able to classify the most common categories of radiographs with a neural network. The network showed good generalizability on the external validation set and could be used to automatically organize a PACS, preselect radiographs so that they can be routed to more specialized networks for abnormality detection or help with other parts of the radiological workflow (e.g., automated hanging protocols; check if ordered image and performed image are the same). The final AI algorithm is publicly available for evaluation and extension.

KEY POINTS

• Data from one single institution can be used to train a neural network for the correct detection of the 30 most common categories of plain radiographs. • The trained model achieved a high accuracy for the majority of categories and showed good generalizability to images from other institutions. • The neural network is made publicly available and can be used to automatically organize a PACS or to preselect radiographs so that they can be routed to more specialized neural networks for abnormality detection.

End-To-End Computerized Diagnosis of Spondylolisthesis Using Only Lumbar X-rays

Lumbar spondylolisthesis (LS) is the anterior shift of one of the lower vertebrae about the subjacent vertebrae. There are several symptoms to define LS, and these symptoms are not detected in the early stages of LS. This leads to disease progress further without being identified. Thus, advanced treatment mechanisms are required to implement for diagnosing LS, which is crucial in terms of early diagnosis, rehabilitation, and treatment planning. Herein, a transfer learning-based CNN model is developed that uses only lumbar X-rays. The model was trained with 1922 images, and 187 images were used for validation. Later, the model was tested with 598 images. During training, the model extracts the region of interests (ROIs) via Yolov3, and then the ROIs are split into training and validation sets. Later, the ROIs are fed into the fine-tuned MobileNet CNN to accomplish the training. However, during testing, the images enter the model, and then they are classified as spondylolisthesis or normal. The end-to-end transfer learning-based CNN model reached the test accuracy of 99%, whereas the test sensitivity was 98% and the test specificity 99%. The performance results are encouraging and state that the model can be used in outpatient clinics where any experts are not present.

Computer-aided diagnosis based on hand thermal, RGB images, and grip force using artificial intelligence as screening tool for rheumatoid arthritis in women

Rheumatoid arthritis (RA) is an autoimmune disorder that typically affects people between 23 and 60 years old causing chronic synovial inflammation, symmetrical polyarthritis, destruction of large and small joints, and chronic disability. Clinical diagnosis of RA is stablished by current ACR-EULAR criteria, and it is crucial for starting conventional therapy in order to minimize damage progression. The 2010 ACR-EULAR criteria include the presence of swollen joints, elevated levels of rheumatoid factor or anti-citrullinated protein antibodies (ACPA), elevated acute phase reactant, and duration of symptoms. In this paper, a computer-aided system for helping in the RA diagnosis, based on quantitative and easy-to-acquire variables, is presented. The participants in this study were all female, grouped into two classes: class I, patients diagnosed with RA (n = 100), and class II corresponding to controls without RA (n = 100). The novel approach is constituted by the acquisition of thermal and RGB images, recording their hand grip strength or gripping force. The weight, height, and age were also obtained from all participants. The color layout descriptors (CLD) were obtained from each image for having a compact representation. After, a wrapper forward selection method in a range of classification algorithms included in WEKA was performed. In the feature selection process, variables such as hand images, grip force, and age were found relevant, whereas weight and height did not provide important information to the classification. Our system obtains an AUC ROC curve greater than 0.94 for both thermal and RGB images using the RandomForest classifier. Thirty-eight subjects were considered for an external test in order to evaluate and validate the model implementation. In this test, an accuracy of 94.7% was obtained using RGB images; the confusion matrix revealed our system provides a correct diagnosis for all participants and failed in only two of them (5.3%). Graphical abstract.