Automated Classification of Rheumatoid Arthritis, Osteoarthritis, and Normal Hand Radiographs with Deep Learning Methods

Algorithmic approaches in hand imaging for rheumatic musculoskeletal diseases: A systematic literature review

OBJECTIVE

This systematic literature review provides a comprehensive overview of the use of machine learning (ML) in hand imaging of rheumatic musculoskeletal diseases (RMDs). The review evaluates ML algorithms, imaging modalities, patient populations, validation methods, and areas for improvement.

METHODS

The review was conducted following PRISMA guidelines and registered with PROSPERO. Articles were retrieved from PubMed, EMBASE, and Scopus using relevant MeSH terms and keywords. The search, executed in October 2024, was conducted manually and with BiBot, an AI-based tool for literature reviews. Studies focusing on ML applications in osteoarthritis (OA), rheumatoid arthritis (RA), and psoriatic arthritis (PsA) were included.

RESULTS

From 400 initially identified studies, 32 met the inclusion criteria. RA was the most studied disease (88 %), followed by OA (22 %) and PsA (9 %). Convolutional neural networks (CNNs) were the most frequently used algorithms (50 %). Standard radiographs (59 %) were the predominant imaging modality, followed by MRI (16 %). Despite recommendations for ML studies, external validation was conducted in only 15 % of studies, and just 6 % of datasets were publicly available. Interpretability tools were employed in 28 % of studies to enhance clinical relevance.

CONCLUSION

ML has significant potential to improve diagnostics and disease management in hand imaging of RMDs. However, key challenges remain, including the need for increased external validation, broader disease coverage (OA and PsA), and improved data-sharing practices to enhance reproducibility and clinical adoption.

Radiography in Inflammatory Arthritis: Current Roles and Updates in Automated Assessment

Plain radiography continues to be a crucial imaging modality in the field of rheumatology. It provides a comprehensive view of bone-related changes and highlights soft tissue abnormalities. The significance of radiography extends to early disease detection, aiding in differentiating various conditions and monitoring the effectiveness of treatment. It remains the preferred imaging technique for evaluating disease progression, offering insights into cumulative damage over time.In the early stages of arthritis, magnetic resonance imaging and ultrasound are the preferred methods because they can identify subtle disease activity, such as synovitis, tenosynovitis, and dactylitis, osteitis or bone edema, and enthesitis. But they have a lower specificity in distinguishing among various rheumatic conditions.We evaluate the use of radiography in inflammatory arthropathies, highlighting its role in differential diagnoses. Advances in automated radiographic assessment for arthritis are addressed. The discussion encompasses rheumatoid arthritis, juvenile idiopathic arthritis, other connective tissue diseases, and spondyloarthritis.

Multistage deep learning methods for automating radiographic sharp score prediction in rheumatoid arthritis

The Sharp-van der Heijde score (SvH) is crucial for assessing joint damage in rheumatoid arthritis (RA) through radiographic images. However, manual scoring is time-consuming and subject to variability. This study proposes a multistage deep learning model to predict the Overall Sharp Score (OSS) from hand X-ray images. The framework involves four stages: image preprocessing, hand segmentation with UNet, joint identification via YOLOv7, and OSS prediction utilizing a custom Vision Transformer (ViT). Evaluation metrics included Intersection over Union (IoU), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Huber loss, and Intraclass Correlation Coefficient (ICC). The model was trained using stratified group 3-fold cross-validation on a dataset of 679 patients and tested externally on 291 subjects. The joint identification model achieved 99% accuracy. The ViT model achieved the best OSS prediction for patients with Sharp scores < 50. It achieved a Huber loss of 4.9, an RMSE of 9.73, and an MAE of 5.35, demonstrating a strong correlation with expert scores (ICC = 0.702, P < 0.001). This study is the first to apply a ViT for OSS prediction in RA. It presents an efficient and automated alternative for overall damage assessment. This approach may reduce reliance on manual scoring.

Artificial intelligence in rheumatology research: what is it good for?

Artificial intelligence (AI) is transforming rheumatology research, with a myriad of studies aiming to improve diagnosis, prognosis and treatment prediction, while also showing potential capability to optimise the research workflow, improve drug discovery and clinical trials. Machine learning, a key element of discriminative AI, has demonstrated the ability of accurately classifying rheumatic diseases and predicting therapeutic outcomes by using diverse data types, including structured databases, imaging and text. In parallel, generative AI, driven by large language models, is becoming a powerful tool for optimising the research workflow by supporting with content generation, literature review automation and clinical decision support. This review explores the current applications and future potential of both discriminative and generative AI in rheumatology. It also highlights the challenges posed by these technologies, such as ethical concerns and the need for rigorous validation and regulatory oversight. The integration of AI in rheumatology promises substantial advancements but requires a balanced approach to optimise benefits and minimise potential possible downsides.

Optimizing knee osteoarthritis severity prediction on MRI images using deep stacking ensemble technique

Knee osteoarthritis (KOA) represents a well-documented degenerative arthropathy prevalent among the elderly population. KOA is a persistent condition, also referred to as progressive joint Disease, stemming from the continual deterioration of cartilage. Predominantly afflicting individuals aged 45 and above, this ailment is commonly labeled as a "wear and tear" joint disorder, targeting joints such as the knee, hand, hips, and spine. Osteoarthritis symptoms typically increase gradually, contributing to the deterioration of articular cartilage. Prominent indicators encompass pain, stiffness, tenderness, swelling, and the development of bone spurs. Diagnosis typically involves the utilization of Radiographic X-ray images, Magnetic Resonance Imaging (MRI), and Computed Tomography (CT) Scan by medical professionals and experts. However, this conventional approach is time-consuming, and also sometimes tedious for medical professionals. In order to address the limitation of time and expedite the diagnostic process, deep learning algorithms have been implemented in the medical field. In the present investigation, four pre-trained models, specifically CNN, AlexNet, ResNet34 and ResNet-50, were utilized to predict the severity of KOA. Further, a Deep stack ensemble technique was employed to achieve optimal performance resulting to the accuracy of 99.71%.

Can eye-tracking help to create a new method for X-ray analysis of rheumatoid arthritis patients, including joint segmentation and scoring methods?

Reading hand and foot X-rays in rheumatoid arthritis patients is difficult and time-consuming. In research, physicians use the modified Sharp van der Heijde Sharp (mvdH) score by reading of hand and foot radiographs. The aim of this study was to create a new method of determining the mvdH via eye tracking and to study its concordance with the mvdH score. We created a new method of quantifying the mvdH score based on reading time of a reader monitored via eye tracking (Tobii Pro Lab software) after training with the aid of a metronome. Radiographs were read twice by the trained eye-tracking reader and once by an experienced reference radiologist. A total of 440 joints were selected; 416 could be interpreted for erosion, and 396 could be interpreted for joint space narrowing (JSN) when read by eye tracking (eye tracking could not measure the time spent when two pathological joints were too close together). The agreement between eye tracking mvdH Sharp score and classical mvdH Sharp score yes (at least one erosion or JSN) versus no (no erosion or no JSN) was excellent for both erosions (kappa 0.97; 95% CI: 0.96-0.99) and JSN (kappa: 0.95; 95% CI: 0.93-0.097). This agreement by class (0 to 10) remained excellent for both erosions (kappa 0.82; 95% CI: 0.79-0.0.85) and JSN (kappa: 0.68; 95% CI: 0.65-0.0.71). To conclude, eye-tracking reading correlates strongly with classical mvdH-Sharp and is useful for assessing severity, segmenting joints and establishing a rapid score for lesions.

Predicting autoimmune diseases: A comprehensive review of classic biomarkers and advances in artificial intelligence

Autoimmune diseases comprise a spectrum of disorders characterized by the dysregulation of immune tolerance, resulting in tissue or organ damage and inflammation. Their prevalence has been on the rise, significantly impacting patients' quality of life and escalating healthcare costs. Consequently, the prediction of autoimmune diseases has recently garnered substantial interest among researchers. Despite their wide heterogeneity, many autoimmune diseases exhibit a consistent pattern of paraclinical findings that hold predictive value. From serum biomarkers to various machine learning approaches, the array of available methods has been continuously expanding. The emergence of artificial intelligence (AI) presents an exciting new range of possibilities, with notable advancements already underway. The ultimate objective should revolve around disease prevention across all levels. This review provides a comprehensive summary of the most recent data pertaining to the prediction of diverse autoimmune diseases and encompasses both traditional biomarkers and the latest innovations in AI.

RA-XTNet: A Novel CNN Model to Predict Rheumatoid Arthritis from Hand Radiographs and Thermal Images: A Comparison with CNN Transformer and Quantum Computing

The aim and objective of the research are to develop an automated diagnosis system for the prediction of rheumatoid arthritis (RA) based on artificial intelligence (AI) and quantum computing for hand radiographs and thermal images. The hand radiographs and thermal images were segmented using a UNet++ model and color-based k-means clustering technique, respectively. The attributes from the segmented regions were generated using the Speeded-Up Robust Features (SURF) feature extractor and classification was performed using k-star and Hoeffding classifiers. For the ground truth and the predicted test image, the study utilizing UNet++ segmentation achieved a pixel-wise accuracy of 98.75%, an intersection over union (IoU) of 0.87, and a dice coefficient of 0.86, indicating a high level of similarity. The custom RA-X-ray thermal imaging (XTNet) surpassed all the models for the detection of RA with a classification accuracy of 90% and 93% for X-ray and thermal imaging modalities, respectively. Furthermore, the study employed quantum support vector machine (QSVM) as a quantum computing approach which yielded an accuracy of 93.75% and 87.5% for the detection of RA from hand X-ray and thermal images. In addition, vision transformer (ViT) was employed to classify RA which obtained an accuracy of 80% for hand X-rays and 90% for thermal images. Thus, depending on the performance measures, the RA-XTNet model can be used as an effective automated diagnostic method to diagnose RA accurately and rapidly in hand radiographs and thermal images.

Identification and Removal of Pollen Spectral Interference in the Classification of Hazardous Substances Based on Excitation Emission Matrix Fluorescence Spectroscopy

Sensitively detecting hazardous and suspected bioaerosols is crucial for safeguarding public health. The potential impact of pollen on identifying bacterial species through fluorescence spectra should not be overlooked. Before the analysis, the spectrum underwent preprocessing steps, including normalization, multivariate scattering correction, and Savitzky-Golay smoothing. Additionally, the spectrum was transformed using difference, standard normal variable, and fast Fourier transform techniques. A random forest algorithm was employed for the classification and identification of 31 different types of samples. The fast Fourier transform improved the classification accuracy of the sample excitation-emission matrix fluorescence spectrum data by 9.2%, resulting in an accuracy of 89.24%. The harmful substances, including Staphylococcus aureus, ricin, beta-bungarotoxin, and Staphylococcal enterotoxin B, were clearly distinguished. The spectral data transformation and classification algorithm effectively eliminated the interference of pollen on other components. Furthermore, a classification and recognition model based on spectral feature transformation was established, demonstrating excellent application potential in detecting hazardous substances and protecting public health. This study provided a solid foundation for the application of rapid detection methods for harmful bioaerosols.

Advancing precision rheumatology: applications of machine learning for rheumatoid arthritis management

Rheumatoid arthritis (RA) is an autoimmune disease causing progressive joint damage. Early diagnosis and treatment is critical, but remains challenging due to RA complexity and heterogeneity. Machine learning (ML) techniques may enhance RA management by identifying patterns within multidimensional biomedical data to improve classification, diagnosis, and treatment predictions. In this review, we summarize the applications of ML for RA management. Emerging studies or applications have developed diagnostic and predictive models for RA that utilize a variety of data modalities, including electronic health records, imaging, and multi-omics data. High-performance supervised learning models have demonstrated an Area Under the Curve (AUC) exceeding 0.85, which is used for identifying RA patients and predicting treatment responses. Unsupervised learning has revealed potential RA subtypes. Ongoing research is integrating multimodal data with deep learning to further improve performance. However, key challenges remain regarding model overfitting, generalizability, validation in clinical settings, and interpretability. Small sample sizes and lack of diverse population testing risks overestimating model performance. Prospective studies evaluating real-world clinical utility are lacking. Enhancing model interpretability is critical for clinician acceptance. In summary, while ML shows promise for transforming RA management through earlier diagnosis and optimized treatment, larger scale multisite data, prospective clinical validation of interpretable models, and testing across diverse populations is still needed. As these gaps are addressed, ML may pave the way towards precision medicine in RA.

Detection and Grading of Radiographic Hand Osteoarthritis Using an Automated Machine Learning Platform

OBJECTIVE

Automated machine learning (autoML) platforms allow health care professionals to play an active role in the development of machine learning (ML) algorithms according to scientific or clinical needs. The aim of this study was to develop and evaluate such a model for automated detection and grading of distal hand osteoarthritis (OA).

METHODS

A total of 13,690 hand radiographs from 2,863 patients within the Swiss Cohort of Quality Management (SCQM) and an external control data set of 346 non-SCQM patients were collected and scored for distal interphalangeal OA (DIP-OA) using the modified Kellgren/Lawrence (K/L) score. Giotto (Learn to Forecast [L2F]) was used as an autoML platform for training two convolutional neural networks for DIP joint extraction and subsequent classification according to the K/L scores. A total of 48,892 DIP joints were extracted and then used to train the classification model. Heatmaps were generated independently of the platform. User experience of a web application as a provisional user interface was investigated by rheumatologists and radiologists.

RESULTS

The sensitivity and specificity of this model for detecting DIP-OA were 79% and 86%, respectively. The accuracy for grading the correct K/L score was 75%, with a κ score of 0.76. The accuracy per DIP-OA class differed, with 86% for no OA (defined as K/L scores 0 and 1), 71% for a K/L score of 2, 46% for a K/L score of 3, and 67% for a K/L score of 4. Similar values were obtained in an independent external test set. Qualitative and quantitative user experience testing of the web application revealed a moderate to high demand for automated DIP-OA scoring among rheumatologists. Conversely, radiologists expressed a low demand, except for the use of heatmaps.

CONCLUSION

AutoML platforms are an opportunity to develop clinical end-to-end ML algorithms. Here, automated radiographic DIP-OA detection is both feasible and usable, whereas grading among individual K/L scores (eg, for clinical trials) remains challenging.

A Stepwise Approach to Analyzing Musculoskeletal Imaging Data With Artificial Intelligence

The digitization of medical records and expanding electronic health records has created an era of "Big Data" with an abundance of available information ranging from clinical notes to imaging studies. In the field of rheumatology, medical imaging is used to guide both diagnosis and treatment of a wide variety of rheumatic conditions. Although there is an abundance of data to analyze, traditional methods of image analysis are human resource intensive. Fortunately, the growth of artificial intelligence (AI) may be a solution to handle large datasets. In particular, computer vision is a field within AI that analyzes images and extracts information. Computer vision has impressive capabilities and can be applied to rheumatologic conditions, necessitating a need to understand how computer vision works. In this article, we provide an overview of AI in rheumatology and conclude with a five step process to plan and conduct research in the field of computer vision. The five steps include (1) project definition, (2) data handling, (3) model development, (4) performance evaluation, and (5) deployment into clinical care.

Artificial intelligence in osteoarthritis detection: A systematic review and meta-analysis

OBJECTIVES

As an increasing number of studies apply artificial intelligence (AI) algorithms in osteoarthritis (OA) detection, we performed a systematic review and meta-analysis to pool the data on diagnostic performance metrics of AI, and to compare them with clinicians' performance.

MATERIALS AND METHODS

A search in PubMed and Scopus was performed to find studies published up to April 2022 that evaluated and/or validated an AI algorithm for the detection or classification of OA. We performed a meta-analysis to pool the data on the metrics of diagnostic performance. Subgroup analysis based on the involved joint and meta-regression based on multiple parameters were performed to find potential sources of heterogeneity. The risk of bias was assessed using Prediction Model Study Risk of Bias Assessment Tool reporting guidelines.

RESULTS

Of the 61 studies included, 27 studies with 91 contingency tables provided sufficient data to enter the meta-analysis. The pooled sensitivities for AI algorithms and clinicians on internal validation test sets were 88% (95% confidence interval [CI]: 86,91) and 80% (95% CI: 68,88) and pooled specificities were 81% (95% CI: 75,85) and 79% (95% CI: 80,85), respectively. At external validation, the pooled sensitivity and specificity for AI algorithms were 94% (95% CI: 90,97) and 91% (95% CI: 77,97), respectively.

CONCLUSION

Although the results of this meta-analysis should be interpreted with caution due to the potential pitfalls in the included studies, the promising role of AI as a diagnostic adjunct to radiologists is indisputable.

Deep learning discrimination of rheumatoid arthritis from osteoarthritis on hand radiography

PURPOSE

To develop a deep learning model to distinguish rheumatoid arthritis (RA) from osteoarthritis (OA) using hand radiographs and to evaluate the effects of changing pretraining and training parameters on model performance.

MATERIALS AND METHODS

A convolutional neural network was retrospectively trained on 9714 hand radiograph exams from 8387 patients obtained from 2017 to 2021 at seven hospitals within an integrated healthcare network. Performance was assessed using an independent test set of 250 exams from 146 patients. Binary discriminatory capacity (no arthritis versus arthritis; RA versus not RA) and three-way classification (no arthritis versus OA versus RA) were evaluated. The effects of additional pretraining using musculoskeletal radiographs, using all views as opposed to only the posteroanterior view, and varying image resolution on model performance were also investigated. Area under the receiver operating characteristic curve (AUC) and Cohen's kappa coefficient were used to evaluate diagnostic performance.

RESULTS

For no arthritis versus arthritis, the model achieved an AUC of 0.975 (95% CI: 0.957, 0.989). For RA versus not RA, the model achieved an AUC of 0.955 (95% CI: 0.919, 0.983). For three-way classification, the model achieved a kappa of 0.806 (95% CI: 0.742, 0.866) and accuracy of 87.2% (95% CI: 83.2%, 91.2%) on the test set. Increasing image resolution increased performance up to 1024 × 1024 pixels. Additional pretraining on musculoskeletal radiographs and using all views did not significantly affect performance.

CONCLUSION

A deep learning model can be used to distinguish no arthritis, OA, and RA on hand radiographs with high performance.

Machine learning application in autoimmune diseases: State of art and future prospectives

Autoimmune diseases are a group of disorders resulting from an alteration of immune tolerance, characterized by the formation of autoantibodies and the consequent development of heterogeneous clinical manifestations. Diagnosing autoimmune diseases is often complicated, and the available prognostic tools are limited. Machine learning allows us to analyze large amounts of data and carry out complex calculations quickly and with minimal effort. In this work, we examine the literature focusing on the use of machine learning in the field of the main systemic (systemic lupus erythematosus and rheumatoid arthritis) and organ-specific autoimmune diseases (type 1 diabetes mellitus, autoimmune thyroid, gastrointestinal, and skin diseases). From our analysis, interesting applications of machine learning emerged for developing algorithms useful in the early diagnosis of disease or prognostic models (risk of complications, therapeutic response). Subsequent studies and the creation of increasingly rich databases to be supplied to the algorithms will eventually guide the clinician in the diagnosis, allowing intervention when the pathology is still in an early stage and immediately directing towards a correct therapeutic approach.

Enchondroma Detection from Hand Radiographs with an Interactive Deep Learning Segmentation Tool-A Feasibility Study

Enchondromas are common benign bone tumors, usually presenting in the hand. They can cause symptoms such as swelling and pain but often go un-noticed. If the tumor expands, it can diminish the bone cortices and predispose the bone to fracture. Diagnosis is based on clinical investigation and radiographic imaging. Despite their typical appearance on radiographs, they can primarily be misdiagnosed or go totally unrecognized in the acute trauma setting. Earlier applications of deep learning models to image classification and pattern recognition suggest that this technique may also be utilized in detecting enchondroma in hand radiographs. We trained a deep learning model with 414 enchondroma radiographs to detect enchondroma from hand radiographs. A separate test set of 131 radiographs (47% with an enchondroma) was used to assess the performance of the trained deep learning model. Enchondroma annotation by three clinical experts served as our ground truth in assessing the deep learning model's performance. Our deep learning model detected 56 enchondromas from the 62 enchondroma radiographs. The area under receiver operator curve was 0.95. The F1 score for area statistical overlapping was 69.5%. Our deep learning model may be a useful tool for radiograph screening and raising suspicion of enchondroma.

Artificial intelligence in rheumatoid arthritis: potential applications and future implications

The widespread adoption of digital health records, coupled with the rise of advanced diagnostic testing, has resulted in an explosion of patient data, comparable in scope to genomic datasets. This vast information repository offers significant potential for improving patient outcomes and decision-making, provided one can extract meaningful insights from it. This is where artificial intelligence (AI) tools like machine learning (ML) and deep learning come into play, helping us leverage these enormous datasets to predict outcomes and make informed decisions. AI models can be trained to analyze and interpret patient data, including physician notes, laboratory testing, and imaging, to aid in the management of patients with rheumatic diseases. As one of the most common autoimmune diseases, rheumatoid arthritis (RA) has attracted considerable attention, particularly concerning the evolution of diagnostic techniques and therapeutic interventions. Our aim is to underscore those areas where AI, according to recent research, demonstrates promising potential to enhance the management of patients with RA.

A Review on the Recent Applications of Deep Learning in Predictive Drug Toxicological Studies

Drug toxicity prediction is an important step in ensuring patient safety during drug design studies. While traditional preclinical studies have historically relied on animal models to evaluate toxicity, recent advances in deep-learning approaches have shown great promise in advancing drug safety science and reducing animal use in preclinical studies. However, deep-learning-based approaches also face challenges in handling large biological data sets, model interpretability, and regulatory acceptance. In this review, we provide an overview of recent developments in deep-learning-based approaches for predicting drug toxicity, highlighting their potential advantages over traditional methods and the need to address their limitations. Deep-learning models have demonstrated excellent performance in predicting toxicity outcomes from various data sources such as chemical structures, genomic data, and high-throughput screening assays. The potential of deep learning for automated feature engineering is also discussed. This review emphasizes the need to address ethical concerns related to the use of deep learning in drug toxicity studies, including the reduction of animal use and ensuring regulatory acceptance. Furthermore, emerging applications of deep learning in drug toxicity prediction, such as predicting drug-drug interactions and toxicity in rare subpopulations, are highlighted. The integration of deep-learning-based approaches with traditional methods is discussed as a way to develop more reliable and efficient predictive models for drug safety assessment, paving the way for safer and more effective drug discovery and development. Overall, this review highlights the critical role of deep learning in predictive toxicology and drug safety evaluation, emphasizing the need for continued research and development in this rapidly evolving field. By addressing the limitations of traditional methods, leveraging the potential of deep learning for automated feature engineering, and addressing ethical concerns, deep-learning-based approaches have the potential to revolutionize drug toxicity prediction and improve patient safety in drug discovery and development.

A survey of artificial intelligence in rheumatoid arthritis

The article offers a survey of currently notable artificial intelligence methods (released between 2019-2023), with a particular emphasis on the latest advancements in detecting rheumatoid arthritis (RA) at an early stage, providing early treatment, and managing the disease. We discussed challenges in these areas followed by specific artificial intelligence (AI) techniques and summarized advances, relevant strengths, and obstacles. Overall, the application of AI in the fields of RA has the potential to enable healthcare professionals to detect RA at an earlier stage, thereby facilitating timely intervention and better disease management. However, more research is required to confirm the precision and dependability of AI in RA, and several problems such as technological and ethical concerns related to these approaches must be resolved before their widespread adoption.

An Adaptive Early Stopping Technique for DenseNet169-Based Knee Osteoarthritis Detection Model

Knee osteoarthritis (OA) detection is an important area of research in health informatics that aims to improve the accuracy of diagnosing this debilitating condition. In this paper, we investigate the ability of DenseNet169, a deep convolutional neural network architecture, for knee osteoarthritis detection using X-ray images. We focus on the use of the DenseNet169 architecture and propose an adaptive early stopping technique that utilizes gradual cross-entropy loss estimation. The proposed approach allows for the efficient selection of the optimal number of training epochs, thus preventing overfitting. To achieve the goal of this study, the adaptive early stopping mechanism that observes the validation accuracy as a threshold was designed. Then, the gradual cross-entropy (GCE) loss estimation technique was developed and integrated to the epoch training mechanism. Both adaptive early stopping and GCE were incorporated into the DenseNet169 for the OA detection model. The performance of the model was measured using several metrics including accuracy, precision, and recall. The obtained results were compared with those obtained from the existing works. The comparison shows that the proposed model outperformed the existing solutions in terms of accuracy, precision, recall, and loss performance, which indicates that the adaptive early stopping coupled with GCE improved the ability of DenseNet169 to accurately detect knee OA.

Vision-based estimation of MDS-UPDRS scores for quantifying Parkinson's disease tremor severity

Parkinson's disease (PD) is a common neurodegenerative movement disorder among older individuals. As one of the typical symptoms of PD, tremor is a critical reference in the PD assessment. A widely accepted clinical approach to assessing tremors in PD is based on part III of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS). However, expert assessment of tremor is a time-consuming and laborious process that poses considerable challenges to the medical evaluation of PD. In this paper, we proposed a novel model, Global Temporal-difference Shift Network (GTSN), to estimate the MDS-UPDRS score of PD tremors based on video. The PD tremor videos were scored according to the majority vote of multiple raters. We used Eulerian Video Magnification (EVM) pre-processing to enhance the representations of subtle PD tremors in the videos. To make the model better focus on the tremors in the video, we proposed a special temporal difference module, which stacks the current optical flow to the result of inter-frame difference. The prediction scores were obtained from the Residual Networks (ResNet) embedded with a novel module, the Global Shift Module (GSM), which allowed the features of the current segment to include the global segment features. We carried out independent experiments using PD tremor videos of different body parts based on the scoring content of the MDS-UPDRS. On a fairly large dataset, our method achieved an accuracy of 90.6% for hands with rest tremors, 85.9% for tremors in the leg, and 89.0% for the jaw. An accuracy of 84.9% was obtained for postural tremors. Our study demonstrated the effectiveness of computer-assisted assessment for PD tremors based on video analysis. The latest version of the code is available at https://github.com/199507284711/PD-GTSN.

Detecting hand joint ankylosis and subluxation in radiographic images using deep learning: A step in the development of an automatic radiographic scoring system for joint destruction

We propose a wrist joint subluxation/ankylosis classification model for an automatic radiographic scoring system for X-ray images. In managing rheumatoid arthritis, the evaluation of joint destruction is important. The modified total Sharp score (mTSS), which is conventionally used to evaluate joint destruction of the hands and feet, should ideally be automated because the required time depends on the skill of the evaluator, and there is variability between evaluators. Since joint subluxation and ankylosis are given a large score in mTSS, we aimed to estimate subluxation and ankylosis using a deep neural network as a first step in developing an automatic radiographic scoring system for joint destruction. We randomly extracted 216 hand X-ray images from an electronic medical record system for the learning experiments. These images were acquired from patients who visited the rheumatology department of Keio University Hospital in 2015. Using our newly developed annotation tool, well-trained rheumatologists and radiologists labeled the mTSS to the wrist, metacarpal phalangeal joints, and proximal interphalangeal joints included in the images. We identified 21 X-ray images containing one or more subluxation joints and 42 X-ray images with ankylosis. To predict subluxation/ankylosis, we conducted five-fold cross-validation with deep neural network models: AlexNet, ResNet, DenseNet, and Vision Transformer. The best performance on wrist subluxation/ankylosis classification was as follows: accuracy, precision, recall, F1 value, and AUC were 0.97±0.01/0.89±0.04, 0.92±0.12/0.77±0.15, 0.77±0.16/0.71±0.13, 0.82±0.11/0.72±0.09, and 0.92±0.08/0.85±0.07, respectively. The classification model based on a deep neural network was trained with a relatively small dataset; however, it showed good accuracy. In conclusion, we provided data collection and model training schemes for mTSS prediction and showed an important contribution to building an automated scoring system.

Automated evaluation of rheumatoid arthritis from hand radiographs using Machine Learning and deep learning techniques

The aim and objectives of the study are as follows: (i) to implement automated patch-based classification of hand X-ray images using modified pre-trained convolutional neural network (CNN) models; (ii) to develop a customized CNN model for automated feature extraction and classification of hand X-ray images and to compare the performance of customized CNN models with non-linear and linear kernels; (iii) to construct the hand crafted feature fusion (SIFT+ Customized CNN features) and categorize the normal and RA using Machine Learning classifiers. The model was trained on 75 images (10,000 patches) of hand radiographs and tested using 25 images (500 patches) that were not included in the training set. The accuracy of the modified pre-trained model GoogLeNet was 89% and the proposed custom model three achieved an accuracy of 95%. The sensitivity and specificity of GoogLeNet were 84% and 90% respectively. The custom model three attained the sensitivity and specificity as 95% and 94% respectively. Furthermore, when compared to the features extracted (SIFT + CNN) from the customized models, the custom3 model outperformed well for the classification of RA compared to ML classifiers. Thus a custom CNN-based computer-aided diagnostic tool can be used as an effective method for the detection of RA.