Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data

Deep learning analysis for rheumatologic imaging: current trends, future directions, and the role of human

Rheumatic diseases, such as rheumatoid arthritis (RA), osteoarthritis (OA), and spondyloarthritis (SpA), present diagnostic and management challenges due to their impact on connective tissues and the musculoskeletal system. Traditional imaging techniques, including plain radiography, ultrasounds, computed tomography, and magnetic resonance imaging (MRI), play a critical role in diagnosing and monitoring these conditions, but face limitations like inter-observer variability and time-consuming assessments. Recently, deep learning (DL), a subset of artificial intelligence, has emerged as a promising tool for enhancing medical imaging analysis. Convolutional neural networks, a DL model type, have shown great potential in medical image classification, segmentation, and anomaly detection, often surpassing human performance in tasks like tumor identification and disease severity grading. In rheumatology, DL models have been applied to plain radiography, ultrasounds, and MRI for assessing joint damage, synovial inflammation, and disease progression in RA, OA, and SpA patients. Despite the promise of DL, challenges such as data bias, limited explainability, and the need for large annotated datasets remain significant barriers to its widespread adoption. Furthermore, human oversight and value judgment are essential for ensuring the ethical use and effective implementation of DL in clinical settings. This review provides a comprehensive overview of DL's applications in rheumatologic imaging and explores its future potential in enhancing diagnosis, treatment decisions, and personalized medicine.

Enhancing clinical trial outcome prediction with artificial intelligence: a systematic review

Clinical trials are pivotal in drug development yet fraught with uncertainties and resource-intensive demands. The application of AI models to forecast trial outcomes could mitigate failures and expedite the drug discovery process. This review discusses AI methodologies that impact clinical trial outcomes, focusing on clinical text embedding, trial multimodal learning, and prediction techniques, while addressing practical challenges and opportunities.

A Progressive Risk Formulation for Enhanced Deep Learning based Total Knee Replacement Prediction in Knee Osteoarthritis

We developed deep learning models for predicting Total Knee Replacement (TKR) need within various time horizons in knee osteoarthritis patients, with a novel capability: the models can perform TKR prediction using a single scan, and furthermore when a previous scan is available, they leverage a progressive risk formulation to improve their predictions. Unlike conventional approaches that treat each scan of a patient independently, our method incorporates a constraint based on disease's progressive nature, ensuring that predicted TKR risk either increases or remains stable over time when multiple scans of a knee are available. This was achieved by enforcing a progressive risk formulation constraint during training with patients who have more than one available scan in the studies. Knee radiographs and MRIs from the Osteoarthritis Initiative (OAI) and Multicenter Osteoarthritis Study (MOST) were used in this work and deep learning models were trained to predict TKR within 1, 2, and 4-year time periods. The proposed approach, utilizing a dual-model risk constraint architecture, demonstrated superior performance compared to baseline - conventional models trained with standard binary cross entropy loss. It achieved an AUROC of 0.87 and AUPRC of 0.47 for 1-year TKR prediction on the OAI radiograph test set, considerably improving over the baseline AUROC of 0.79 and AUPRC of 0.34. For the MOST radiograph test set, the proposed approach achieved an AUROC of 0.77 and AUPRC of 0.25 for 1-year predictions, outperforming the baseline AUROC of 0.71 and AUPRC of 0.19. Similar trends were observed in the MRI testsets.

Beyond traditional orthopaedic data analysis: AI, multimodal models and continuous monitoring

Multimodal artificial intelligence (AI) has the potential to revolutionise healthcare by enabling the simultaneous processing and integration of various data types, including medical imaging, electronic health records, genomic information and real-time data. This review explores the current applications and future potential of multimodal AI across healthcare, with a particular focus on orthopaedic surgery. In presurgical planning, multimodal AI has demonstrated significant improvements in diagnostic accuracy and risk prediction, with studies reporting an Area under the receiving operator curve presenting good to excellent performance across various orthopaedic conditions. Intraoperative applications leverage advanced imaging and tracking technologies to enhance surgical precision, while postoperative care has been advanced through continuous patient monitoring and early detection of complications. Despite these advances, significant challenges remain in data integration, standardisation, and privacy protection. Technical solutions such as federated learning (allowing decentralisation of models) and edge computing (allowing data analysis to happen on site or closer to site instead of multipurpose datacenters) are being developed to address these concerns while maintaining compliance with regulatory frameworks. As this field continues to evolve, the integration of multimodal AI promises to advance personalised medicine, improve patient outcomes, and transform healthcare delivery through more comprehensive and nuanced analysis of patient data. Level of Evidence: Level V.

Multimodal model for knee osteoarthritis KL grading from plain radiograph

Knee osteoarthritis presents a significant health challenge for many adults globally. At present, there are no pharmacological treatments that can cure this medical condition. The primary method for managing the progress of knee osteoarthritis is through early identification. Currently, X-ray imaging serves as a key modality for predicting the onset of osteoarthritis. Nevertheless, the traditional manual interpretation of X-rays is susceptible to inaccuracies, largely due to the varying levels of expertise among radiologists. In this paper, we propose a multimodal model based on pre-trained vision and language models for the identification of the knee osteoarthritis severity Kellgren-Lawrence (KL) grading. Using Vision transformer and Pre-training of deep bidirectional transformers for language understanding (BERT) for images and texts embeddings extraction helps Transformer encoders extracts more distinctive hidden-states that facilitates the learning process of the neural network classifier. The multimodal model was trained and tested on the OAI dataset, and the results showed remarkable performance compared to the related works. Experimentally, the evaluation of the model on the test set comprising X-ray images demonstrated an overall accuracy of 82.85%, alongside a precision of 84.54% and a recall of 82.89%.

Machine learning-based prediction of contralateral knee osteoarthritis development using the Osteoarthritis Initiative and the Multicenter Osteoarthritis Study dataset

Having osteoarthritis in one knee is reported as an independent risk factor for developing contralateral knee osteoarthritis (KOA). However, no study has been designed to predict the development of contralateral KOA (cKOA). The authors hypothesized that specific risk factors for cKOA development exist and that it could be accurately predicted with the assistance of machine learning. KOA was defined using the Kellgren-Lawrence grade (KLG) of 2 or higher. Data from 1353 unilateral KOA patients (900 from the Osteoarthritis Initiative [OAI] and 453 from the Multicenter Osteoarthritis Study [MOST]) over 4-5 years of follow-up were examined. The risk factors for cKOA development were analyzed, and a machine learning model was developed to predict cKOA using OAI as the development data set and MOST as the test data set. cKOA developed in 172 (19.1%) and 178 (39.3%) of the patients (OAI and MOST, respectively) over a period of 4-5 years. A machine learning model was developed using the Tree-based Pipeline Optimization Tool algorithm. This model utilized nine variables, including baseline lateral joint space narrowing grade of the contralateral knee (odds ratio 4.475). The area under the curve of the receiver operating characteristics curve, along with accuracy, precision, and F1-score, were recorded as 0.69, 0.60, 0.50, and 0.58, respectively, in the test data set. The development of cKOA could be effectively predicted using a limited number of variables through machine learning. Surgeons should consider the development of cKOA in patients with identified risk factors when managing KOA patients.

Generating synthetic past and future states of Knee Osteoarthritis radiographs using Cycle-Consistent Generative Adversarial Neural Networks

Knee Osteoarthritis (KOA), a leading cause of disability worldwide, is challenging to detect early due to subtle radiographic indicators. Diverse, extensive datasets are needed but are challenging to compile because of privacy, data collection limitations, and the progressive nature of KOA. However, a model capable of projecting genuine radiographs into different OA stages could augment data pools, enhance algorithm training, and offer pre-emptive prognostic insights. In this study, we developed a Cycle-Consistent Adversarial Network (CycleGAN) to generate synthetic past and future stages of KOA on any genuine radiograph. The model's effectiveness was validated through its impact on a KOA specialized Convolutional Neural Network (CNN). Transformations towards synthetic future disease states resulted in 83.76% of none-to-doubtful stage images being classified as moderate-to-severe stages, while retroactive transformations led to 75.61% of severe-stage images being classified as none-to-doubtful stages. Similarly, transformations from mild stages achieved 76.00% correct classification towards future stages and 69.00% for past stages. The CycleGAN demonstrated an exceptional ability to expand the knee joint space and eliminate bone-outgrowths (osteophytes), key radiographic indicators of disease progression. These results signify a promising potential for enhancing diagnostic models, data augmentation, and educational and prognostic uses. Nevertheless, further refinement, validation, and a broader evaluation process encompassing both CNN-based assessments and expert medical feedback are emphasized for future research and development.

DIKOApp: An AI-Based Diagnostic System for Knee Osteoarthritis

The diagnosis of knee osteoarthritis is challenging due to its complex nature and various contributing factors. With the advancement of artificial intelligence (AI) technology, some computer vision-based methods have been developed to address this task. However, when applied in practice, these methods encounter numerous challenges. Training a powerful AI model to effectively analyze a wide range of medical images is crucial. On the other hand, collecting and accurately labeling a significant number of medical images in the real world is necessary. Specifically, when dealing with knee images from specific regions like Vietnam, certain unique biological characteristics make it difficult to utilize and trust previously published studies. To effectively address these challenges, we introduce DIKOApp, an automatic diagnostic application for knee osteoarthritis based on the DIKO framework, trained on a dataset specifically built for the Vietnamese population. This framework is designed with two stages that leverage medical knowledge and computer vision techniques. The DIKO framework leverages efficient data sampling and augmentation framework to handle medical images in the real world more effectively. When evaluated using a real-world knee image dataset from Vietnamese individuals, the DIKO model demonstrates impressive performance with an accuracy of 89.34% and an F1-score of 0.88. By utilizing the capabilities of the DIKO framework, DIKOApp shows practical and promising real-world potential, enabling doctors and healthcare service providers to diagnose pathological conditions more accurately while requiring less diagnostic time, thereby improving the lives of patients.

Machine learning for clustering and classification of early knee osteoarthritis using single-leg standing kinematics

Objective

Detection of early osteoarthritis (EOA) of the knee is crucial for effective management and improved outcomes. This study investigated the application of machine learning techniques to single-leg standing (SLS) kinematics to classify and predict EOA. (1) To identify distinct groups based on SLS kinematic patterns using unsupervised learning algorithms, (2) to develop supervised learning models to predict EOA status, and (3) to identify the most influential kinematic variables associated with EOA.

Methods

Total 43 manufacturing workers (86 legs) aged 40-70 years were evaluated. The participants were categorized using an Early Osteoarthritis Questionnaire. Single-leg standing kinematics was captured using 2D video analysis to assess the horizontal displacement of six key anatomical points (trunk, pelvis, femur, knee, lower leg, and ankle) in the frontal plane. K-means clustering was used for unsupervised learning, whereas six supervised machine learning algorithms were trained and validated for EOA classification.

Results

In our machine learning models, we used 258 data points derived from three repeated measurements per participant. K-means clustering revealed three distinct groups based on SLS kinematics and demographic characteristics. The random forest algorithm achieved the highest classification accuracy (area under the receiver operating characteristic curve = 1.000, accuracy = 1.000) in distinguishing between individuals with and without EOA. Pelvic and ankle horizontal displacements were identified as the most influential predictors of EOA classification.

Conclusions

Machine learning analysis of SLS kinematics shows significant potential for the early detection of knee osteoarthritis. Identification of key kinematic predictors, particularly pelvic and ankle movements, provides new insights into targeted interventions and screening protocols for rehabilitation.

Multimodality Fusion Aspects of Medical Diagnosis: A Comprehensive Review

Utilizing information from multiple sources is a preferred and more precise method for medical experts to confirm a diagnosis. Each source provides critical information about the disease that might otherwise be absent in other modalities. Combining information from various medical sources boosts confidence in the diagnosis process, enabling the creation of an effective treatment plan for the patient. The scarcity of medical experts to diagnose diseases motivates the development of automatic diagnoses relying on multimodal data. With the progress in artificial intelligence technology, automated diagnosis using multimodal fusion techniques is now possible. Nevertheless, the concept of multimodal medical diagnosis is still new and requires an understanding of the diverse aspects of multimodal data and its related challenges. This review article examines the various aspects of multimodal medical diagnosis to equip readers, academicians, and researchers with necessary knowledge to advance multimodal medical research. The chosen articles in the study underwent thorough screening from reputable journals and publishers to offer high-quality content to readers, who can then apply the knowledge to produce quality research. Besides, the need for multimodal information and the associated challenges are discussed with solutions. Additionally, ethical issues of using artificial intelligence in medical diagnosis is also discussed.

Two-Stage Classification of Future Knee Osteoarthritis Severity After 8 Years Using MRI: Data from the Osteoarthritis Initiative

Currently, there are no methods or tools available in clinical practice for classifying future knee osteoarthritis (KOA). In this study, we aimed to fill this gap by classifying future KOA into three severity grades: KL01 (healthy), KL2 (moderate), and KL34 (severe) based on the Kellgren-Lawrance scale. Due to the complex nature of multiclass classification, we used a two-stage method, which separates the classification task into two binary classifications (KL01 vs. KL234 in the first stage and KL2 vs. KL34 in the second stage). Our machine learning (ML) model used two Balanced Random Forest algorithms and was trained with gender, age, height, weight, and quantitative knee morphology obtained from magnetic resonance imaging. Our training dataset comprised longitudinal 8-year follow-up data of 1213 knees from the Osteoarthritis Initiative. Through extensive experimentation with various feature combinations, we identified KL baseline and weight as the most essential features, while gender surprisingly proved to be one of the least influential feature. Our best classification model generated a weighted F1 score of 79.0% and a balanced accuracy of 65.9%. The area under the receiver operating characteristic curve was 83.0% for healthy (KL01) versus moderate (KL2) or severe (KL34) KOA patients and 86.6% for moderate (KL2) versus severe (KL34) KOA patients. We found a statistically significant difference in performance between our two-stage classification model and the traditional single-stage classification model. These findings demonstrate the encouraging results of our two-stage classification model for multiclass KOA severity classification, suggesting its potential application in clinical settings in future.

Development and accuracy of an artificial intelligence model for predicting the progression of hip osteoarthritis using plain radiographs and clinical data: a retrospective study

BACKGROUND

Predicting the progression of hip osteoarthritis (OA) remains challenging, and no reliable predictive method has been established. This study aimed to develop an artificial intelligence (AI) model to predict hip OA progression via plain radiographs and patient data and to determine its accuracy.

METHODS

This retrospective study utilized anteroposterior pelvic radiographs of consecutive patients with hip OA who underwent primary unilateral total hip arthroplasty. Radiographs diagnosed with Kellgren-Lawrence (KL) grade 0-2 were extracted from 361 patients and 1697 images. This AI model was developed to predict whether OA would progress from KL grade 0-2 to KL grade ≥ 3 within n years (n = 3, 4, 5). A gradient-boosting decision tree approach was utilized according to feature extractions obtained by a convolutional neural network from radiographs and patient data (height, body weight, sex, age, and KL grade given by an orthopedic surgeon) with five-fold cross-validation. The model performance was assessed using accuracy, specificity, sensitivity, and the area under the receiver operating characteristic curve (AUC).

RESULTS

The mean accuracy, specificity, sensitivity, and AUC of our prediction model were, respectively, 81.8%, 88.0%, 66.7%, and 0.836 for 3 years; 79.8%, 85.0%, 71.6%, and 0.836 for 4 years; and 78.5%, 80.4%, 76.9%, and 0.846 for 5 years.

CONCLUSIONS

The proposed AI model performed adequately in predicting hip OA progression and may be clinically applicable with additional datasets and validation.

Microbiome-Augmented Model for Predicting Knee Osteoarthritis Progression Based on Gut Microbiota and Kellgren-Lawrence Classification

PURPOSE

 Knee osteoarthritis (OA) is a widespread chronic degenerative condition that may experience slow or rapid deterioration. The gut-joint axis represents a bidirectional relationship in OA onset and progression. This study aimed to establish and validate a prediction model of knee OA disease progression.

METHODS

This prospective cohort investigation involved 296 patients diagnosed with knee OA using X-ray and CT scans at Taizhou People's Hospital from January 2020 to January 2022. Fecal samples and general information were collected for gut microbiota analysis. Least absolute shrinkage and selection operator (LASSO) regression and various prediction models, including microbiome-augmented models, were employed for knee OA risk prediction. The models predicting Kellgren-Lawrence classification one year later were evaluated by accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC).

RESULTS

A total of 270 patients were involved in our study. After random assignment, 214 patients belonged to the training set and 56 patients belonged to the test set. The final intestinal flora included in the analysis included the following 12 species. Shannon index of patients with a Grade I Kellgren-Lawrence Classification after one year was lower than those with a Grade II/III after one year (P=0.018). The best model was the microbiome-augmented model built by Light GBM (LGBM). The AUC of this model in the training set was 0.812 (0.754-0.870), the sensitivity was 0.804 (0.725-0.883), the specificity was 0.744 (0.664-0.823), the PPV was 0.722 (0.638-0.807), the NPV was 0.821 (0.748-0.894), and the accuracy was 0.771 (0.715-0.827). The AUC of this model in the testing set was 0.876 (0.781-0.972), the sensitivity was 0.759 (0.603-0.914), the specificity was 0.917 (0.806-1.000), the PPV was 0.917 (0.806-1.000), the NPV was 0.759 (0.603-0.914), and the accuracy was 0.830 (0.729-0.931).  Conclusion: One year later, the microbiome-augmented model constructed by LGBM for knee OA patients based on general and gut microbiota data using the Kellgren-Lawrence classification demonstrated the highest performance. This approach could aid in identifying patients at risk of rapid disease progression, facilitating early intervention and personalized treatments. Furthermore, it offers a novel perspective on the gut-joint axis's role in OA.

Understanding the role of machine learning in predicting progression of osteoarthritis

Aims

Machine learning (ML), a branch of artificial intelligence that uses algorithms to learn from data and make predictions, offers a pathway towards more personalized and tailored surgical treatments. This approach is particularly relevant to prevalent joint diseases such as osteoarthritis (OA). In contrast to end-stage disease, where joint arthroplasty provides excellent results, early stages of OA currently lack effective therapies to halt or reverse progression. Accurate prediction of OA progression is crucial if timely interventions are to be developed, to enhance patient care and optimize the design of clinical trials.

Methods

A systematic review was conducted in accordance with PRISMA guidelines. We searched MEDLINE and Embase on 5 May 2024 for studies utilizing ML to predict OA progression. Titles and abstracts were independently screened, followed by full-text reviews for studies that met the eligibility criteria. Key information was extracted and synthesized for analysis, including types of data (such as clinical, radiological, or biochemical), definitions of OA progression, ML algorithms, validation methods, and outcome measures.

Results

Out of 1,160 studies initially identified, 39 were included. Most studies (85%) were published between 2020 and 2024, with 82% using publicly available datasets, primarily the Osteoarthritis Initiative. ML methods were predominantly supervised, with significant variability in the definitions of OA progression: most studies focused on structural changes (59%), while fewer addressed pain progression or both. Deep learning was used in 44% of studies, while automated ML was used in 5%. There was a lack of standardization in evaluation metrics and limited external validation. Interpretability was explored in 54% of studies, primarily using SHapley Additive exPlanations.

Conclusion

Our systematic review demonstrates the feasibility of ML models in predicting OA progression, but also uncovers critical limitations that currently restrict their clinical applicability. Future priorities should include diversifying data sources, standardizing outcome measures, enforcing rigorous validation, and integrating more sophisticated algorithms. This paradigm shift from predictive modelling to actionable clinical tools has the potential to transform patient care and disease management in orthopaedic practice.

Deep learning to combat knee osteoarthritis and severity assessment by using CNN-based classification

BACKGROUND

In today's digital age, various diseases drastically reduce people's quality of life. Arthritis is one amongst the most common and debilitating maladies. Osteoarthritis affects several joints, including the hands, knees, spine, and hips. This study focuses on the medical disorder underlying Knee Osteoarthritis (KOA) which severely impairs people's quality of life. KOA is characterised by restricted mobility, stiffness, and terrible pain and can be caused by a range of factors such as ageing, obesity, and traumas. This degenerative disorder leads to progressive wear and tear of the knee joint.

METHODS

To combat arthritis in the kneecap, this study employs a 12-layer Convolutional Neural Network (CNN) to reach deep learning capabilities. A collection of data from the Osteoarthritis Initiative (OAI) is used to classify KOA. Through the use of medical image processing; the study ascertains whether an individual has this ailment. A sophisticated CNN architecture created especially for binary classification and KOA severity utilising deep learning algorithms is the main component of this work.

RESULTS

The cross-entropy loss function is an important component of the model's laborious design that classifies data into two groups. The remaining section uses the Kellgren-Lawrence (KL) grade to classify the disease's severity. In the binary classification, the proposed algorithm outperforms previous methods with an accuracy rate of 92.3%, and in the multiclassification, its accuracy rate is 78.4% which is superior to the previous findings.

CONCLUSION

Looking ahead, the research broadens the scope of this work by gathering information from various sources and using these methods on a wider range of datasets and situations. The potential for major advancements in the field of osteoarthritis detection and classification is highlighted by this forward-looking approach. Furthermore, this method reduces the intervention of medical practitioners and ultimately results in accurate diagnosis.

CLINICAL TRIAL NUMBER

Not applicable.

Development of the machine learning model that is highly validated and easily applicable to predict radiographic knee osteoarthritis progression

Many models using the aid of artificial intelligence have been recently proposed to predict the progression of knee osteoarthritis. However, previous models have not been properly validated with an external data set or have reported poor predictive performances. Therefore, the purpose of this study was to design a machine learning model for knee osteoarthritis progression, focusing on high validation quality and clinical applicability. A retrospective analysis was conducted on prospectively collected data, using the Osteoarthritis Initiative data set (5966 knees) for model development and the Multicenter Osteoarthritis Study data set (3392 knees) for validation. The analysis aimed to predict Kellgren-Lawrence grade (KLG) progression over 4-5 years in knees with initial KLG of 0, 1, or 2. Possible predictors included demographics, comorbidities, history of meniscectomy, gait speed, Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scores, and radiological findings. The Random Forest algorithm was employed for the predictive model development. Baseline KLG, contralateral knee osteoarthritis, lateral joint space narrowing (JSN) grade, BMI, medial JSN grade, and total WOMAC score were six features selected for the model in descending order of importance. Odds ratios of baseline KLG, contralateral knee osteoarthritis, and lateral JSN grade were 1.76, 2.59, and 4.74, respectively (all p < 0.001). The area-under-the-curve of the ROC curve in the validation set was 0.76 with an accuracy of 0.68 and an F1-score of 0.56. The progression of knee osteoarthritis in 4 ~ 5 years could be well-predicted using easily available variables. This simple and validated model may aid surgeons in knee osteoarthritis patient management.

Deep Learning for Automatic Knee Osteoarthritis Severity Grading and Classification

Introduction

Knee osteoarthritis (OA) is a prevalent condition that significantly impacts the quality of life, often leading to the need for knee replacement surgery. Accurate and timely identification of knee degeneration is crucial for effective treatment and management. Traditional methods of diagnosing OA rely heavily on radiological assessments, which can be time-consuming and subjective. This study aims to address these challenges by developing a deep learning-based method to predict the likelihood of knee replacement and the Kellgren-Lawrence (KL) grade of knee OA from X-ray images.

Methodology

We employed the Osteoarthritis Initiative (OAI) dataset and utilized a transfer learning approach with the Inception V3 architecture to enhance the accuracy of OA detection. Our approach involved training 14 different models-Xception, VGG16, VGG19, ResNet50, ResNet101, ResNet152, ResNet50V2, ResNet101V2, ResNet152V2, Inception V3, Inception, ResNetV2, DenseNet121, DenseNet169, DenseNet201-and comparing their performance.

Results

The study incorporated pixel ratio computation and picture pre-processing, alongside a decision tree model for prediction. Our experiments revealed that the Inception V3 model achieved the highest training accuracy of 91% and testing accuracy of 67%, with notable performance in both training and validation phases. This model effectively identified the presence and severity of OA, correlating with the Kellgren-Lawrence scale and facilitating the assessment of knee replacement needs.

Conclusion

By integrating advanced deep learning techniques with radiological diagnostics, our methodology supports radiologists in making more accurate and prompt decisions regarding knee degeneration. The Inception V3 model stands out as the optimal choice for knee X-ray analysis, contributing to more efficient and timely healthcare delivery for patients with knee osteoarthritis.

Diagnosing the Severity of Knee Osteoarthritis Using Regression Scores From Artificial Intelligence Convolution Neural Networks

BACKGROUND

This study focused on using deep learning neural networks to classify the severity of osteoarthritis in the knee. A continuous regression score of osteoarthritis severity has yet to be explored using artificial intelligence machine learning, which could offer a more nuanced assessment of osteoarthritis.

MATERIALS AND METHODS

This study used 8260 radiographic images from The Osteoarthritis Initiative to develop and assess four neural network models (VGG16, EfficientNetV2 small, ResNet34, and DenseNet196). Each model generated a regressor score of the osteoarthritis severity based on Kellgren-Lawrence grading scale criteria. Primary performance outcomes assessed were area under the curve (AUC), accuracy, and mean absolute error (MAE) for each model. Secondary outcomes evaluated were precision, recall, and F-1 score.

RESULTS

The EfficientNet model architecture yielded the strongest AUC (0.83), accuracy (71%), and MAE (0.42) compared with VGG16 (AUC: 0.74; accuracy: 57%; MAE: 0.54), ResNet34 (AUC: 0.76; accuracy: 60%; MAE: 0.53), and DenseNet196 (AUC: 0.78; accuracy: 62%; MAE: 0.49).

CONCLUSION

Convolutional neural networks offer an automated and accurate way to quickly assess and diagnose knee radiographs for osteoarthritis. The regression score models evaluated in this study demonstrated superior AUC, accuracy, and MAE compared with standard convolutional neural network models. The EfficientNet model exhibited the best overall performance, including the highest AUC (0.83) noted in the literature. The artificial intelligence-generated regressor exhibits a finer progression of knee osteoarthritis by quantifying severity of various hallmark features. Potential applications for this technology include its use as a screening tool in determining patient suitability for orthopedic referral. [Orthopedics. 2024;47(5):e247-e254.].

Bimodal machine learning model for unstable hips in infants: integration of radiographic images with automatically-generated clinical measurements

Bimodal convolutional neural networks (CNNs) are frequently combined with patient information or several medical images to enhance the diagnostic performance. However, the technologies that integrate automatically generated clinical measurements within the images are scarce. Hence, we developed a bimodal model that produced automatic algorithm for clinical measurement (aaCM) from radiographic images and integrated the model with CNNs. In this multicenter research project, the diagnostic performance of the model was investigated with 813 radiographic hip images of infants at risk of developmental dysplasia of the hips (232 and 581 images of unstable and stable hips, respectively), with the ground truth defined by provocative examinations. The results indicated that the accuracy of aaCM was equal or higher than that of specialists, and the bimodal model showed better diagnostic performance than LightGBM, XGBoost, SVM, and single CNN models. aaCM can provide expert's knowledge in a high level, and our proposed bimodal model has better performance than the state-of-art models.

Predicting the onset of end-stage knee osteoarthritis over two- and five-years using machine learning

OBJECTIVE

Identifying participants who will progress to advanced stage in knee osteoarthritis (KOA) trials remains a significant challenge. Current tools, relying on total knee replacements (TKR), fall short in reliability due to the extraneous factors influencing TKR decisions. Acknowledging these limitations, our study identifies a critical need for a more robust metric to assess severe KOA. The end-stage KOA (esKOA) measure, which combines symptomatic and radiographic criteria, serves as a solid indicator. To enhance future trials that use esKOA as an endpoint, our study focuses on developing and validating a machine-learning tool to identify individuals likely to develop esKOA within 2 to 5 years.

DESIGN

Utilizing the Osteoarthritis Initiative (OAI) data, we trained models on 3,114 participants and validated them with 606 participants for the right knee, and similarly for the left knee, with external validation from the Multicentre Osteoarthritis Study (MOST) involving 1,602 participants. We aimed to predict esKOA onset at 2-to-2.5 years and 4-to-5 years, defining esKOA by severe radiographic KOA with moderate/severe symptoms or mild/moderate radiographic KOA with persistent/intense symptoms. Our analysis considered 51 candidate predictors, including demographics, clinical history, physical examination, and X-ray evaluations. An online tool predicting esKOA progression, based on models with ten and nine predictors for the right and left knees, respectively, was developed.

RESULTS

External validation (MOST) for the right knee at 2.5 years yielded an Area Under Curve (AUC) of 0.847 (95 % CI 0.811 to 0.882), and at 5 years, 0.853 (95 % CI 0.823 to 0.881); for the left knee at 2.5 years, AUC was 0.824 (95 % CI 0.782 to 0.857), and at 5 years, 0.807 (95 % CI 0.768 to 0.843). Models with fewer predictors demonstrated comparable performance. The online tool is available at: https://eskoa.shinyapps.io/webapp/.

CONCLUSION

Our study unveils a robust, externally validated machine learning tool proficient in predicting the onset of esKOA over the next 2 to 5 years. Our tool can lead to more efficient KOA trials.

An interpretable knee replacement risk assessment system for osteoarthritis patients

Objective

Knee osteoarthritis (OA) is a complex disease with heterogeneous representations. Although it is modifiable to prevention and early treatment, there still lacks a reliable and accurate prognostic tool. Hence, we aim to develop a quantitative and self-administrable knee replacement (KR) risk stratification system for knee osteoarthritis (KOA) patients with clinical features.

Method

A total of 14 baseline features were extracted from 9592 cases in the Osteoarthritis Initiative (OAI) cohort. A survival model was constructed using the Random Survival Forests algorithm. The prediction performance was evaluated with the concordance index (C-index) and average receiver operating characteristic curve (AUC). A three-class KR risk stratification system was built to differentiate three distinct KR-free survival groups. Thereafter, Shapley Additive Explanations (SHAP) was introduced for model explanation.

Results

KR incidence was accurately predicted by the model with a C-index of 0.770 (±0.0215) and an average AUC of 0.807 (±0.0181) with 14 clinical features. Three distinct survival groups were observed from the ten-point KR risk stratification system with a four-year KR rate of 0.79%, 5.78%, and 16.2% from the low, medium, and high-risk groups respectively. KR is mainly caused by pain medication use, age, surgery history, diabetes, and a high body mass index, as revealed by SHAP.

Conclusion

A self-administrable and interpretable KR survival model was developed, underscoring a KR risk scoring system to stratify KOA patients. It will encourage regular self-assessments within the community and facilitate personalised healthcare for both primary and secondary prevention of KOA.

Dual-energy X-ray absorptiometry derived knee shape may provide a useful imaging biomarker for predicting total knee replacement: Findings from a study of 37,843 people in UK Biobank

Objective

We aimed to create an imaging biomarker for knee shape using knee dual-energy x-ray absorptiometry (DXA) scans and investigate its potential association with subsequent total knee replacement (TKR), independently of radiographic features of knee osteoarthritis and established risk factors.

Methods

Using a 129-point statistical shape model, knee shape (expressed as a B-score) and minimum joint space width (mJSW) of the medial joint compartment (binarized as above or below the first quartile) were derived. Osteophytes were manually graded in a subset of images and an overall score was assigned. Cox proportional hazards models were used to examine the associations of B-score, mJSW and osteophyte score with TKR risk, adjusting for age, sex, height and weight.

Results

The analysis included 37,843 individuals (mean age 63.7 years). In adjusted models, B-score was associated with TKR: each unit increase in B-score, reflecting one standard deviation from the mean healthy shape, corresponded to a hazard ratio (HR) of 2.25 (2.08, 2.43), while a lower mJSW had a HR of 2.28 (1.88, 2.77). Among the 6719 images scored for osteophytes, mJSW was replaced by osteophyte score in the most strongly predictive model for TKR. In ROC analyses, a model combining B-score, osteophyte score, and demographics outperformed a model including demographics alone (AUC ​= ​0.87 vs 0.73).

Conclusions

Using statistical shape modelling, we derived a DXA-based imaging biomarker for knee shape that was associated with kOA progression. When combined with osteophytes and demographic data, this biomarker may help identify individuals at high risk of TKR, facilitating targeted interventions.

Highly Flexible, High-Performance, and Stretchable Piezoelectric Sensor Based on a Hierarchical Droplet-Shaped Ceramics with Enhanced Damage Tolerance

Stretchable self-powered sensors are of significant interest in next-generation wearable electronics. However, current strategies for creating stretchable piezoelectric sensors based on piezoelectric polymers or 0-3 piezoelectric composites face several challenges such as low piezoelectric activity, low sensitivity, and poor durability. In this paper, a biomimetic soft-rigid hybrid strategy is used to construct a new form of highly flexible, high-performance, and stretchable piezoelectric sensor. Inspired by the hinged bivalve Cristaria plicata, hierarchical droplet-shaped ceramics are manufactured and used as rigid components, where computational models indicate that the unique arched curved surface and rounded corners of this bionic structure can alleviate stress concentrations. To ensure electrical connectivity of the piezoelectric phase during stretching, a patterned liquid metal acts as a soft circuit and a silicone polymer with optimized wettability and stretchability serves as a soft component that forms a strong mechanical interlock with the hierarchical ceramics. The novel sensor design exhibits excellent sensitivity and durability, where the open circuit voltage remains stable after 5000 stretching cycles at 60% strain and 5000 twisting cycles at 180°. To demonstrate its potential in heathcare applications, this new stretchable sensor is successfully used for wireless gesture recognition and assessing the progression of knee osteoarthritis.

Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data, and Symptomatic Assessments

OBJECTIVE

In this study, we propose a novel framework that utilizes deep learning and attention mechanisms to predict the radiographic progression of patellofemoral osteoarthritis (PFOA) over a period of 7 years.

MATERIAL AND METHODS

This study included subjects (1,832 subjects, 3,276 knees) from the baseline of the Multicenter Osteoarthritis Study (MOST). Patellofemoral joint regions of interest were identified using an automated landmark detection tool (BoneFinder) on lateral knee X-rays. An end-to-end deep learning method was developed for predicting PFOA progression based on imaging data in a five-fold cross-validation setting. To evaluate the performance of the models, a set of baselines based on known risk factors were developed and analyzed using gradient boosting machine (GBM). Risk factors included age, sex, body mass index, and Western Ontario and McMaster Universities Arthritis Index score, and the radiographic osteoarthritis stage of the tibiofemoral joint (Kellgren and Lawrence [KL] score). Finally, to increase predictive power, we trained an ensemble model using both imaging and clinical data.

RESULTS

Among the individual models, the performance of our deep convolutional neural network attention model achieved the best performance with an area under the receiver operating characteristic curve (AUC) of 0.856 and average precision (AP) of 0.431, slightly outperforming the deep learning approach without attention (AUC = 0.832, AP = 0.4) and the best performing reference GBM model (AUC = 0.767, AP = 0.334). The inclusion of imaging data and clinical variables in an ensemble model allowed statistically more powerful prediction of PFOA progression (AUC = 0.865, AP = 0.447), although the clinical significance of this minor performance gain remains unknown. The spatial attention module improved the predictive performance of the backbone model, and the visual interpretation of attention maps focused on the joint space and the regions where osteophytes typically occur.

CONCLUSION

This study demonstrated the potential of machine learning models to predict the progression of PFOA using imaging and clinical variables. These models could be used to identify patients who are at high risk of progression and prioritize them for new treatments. However, even though the accuracy of the models were excellent in this study using the MOST dataset, they should be still validated using external patient cohorts in the future.

A Stepwise Approach to Analyzing Musculoskeletal Imaging Data With Artificial Intelligence

The digitization of medical records and expanding electronic health records has created an era of "Big Data" with an abundance of available information ranging from clinical notes to imaging studies. In the field of rheumatology, medical imaging is used to guide both diagnosis and treatment of a wide variety of rheumatic conditions. Although there is an abundance of data to analyze, traditional methods of image analysis are human resource intensive. Fortunately, the growth of artificial intelligence (AI) may be a solution to handle large datasets. In particular, computer vision is a field within AI that analyzes images and extracts information. Computer vision has impressive capabilities and can be applied to rheumatologic conditions, necessitating a need to understand how computer vision works. In this article, we provide an overview of AI in rheumatology and conclude with a five step process to plan and conduct research in the field of computer vision. The five steps include (1) project definition, (2) data handling, (3) model development, (4) performance evaluation, and (5) deployment into clinical care.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Expanding from unilateral to bilateral: A robust deep learning-based approach for predicting radiographic osteoarthritis progression

OBJECTIVE

To develop and validate a deep learning (DL) model for predicting osteoarthritis (OA) progression based on bilateral knee joint views.

METHODS

In this retrospective study, knee joints from bilateral posteroanterior knee radiographs of participants in the Osteoarthritis Initiative were analyzed. At baseline, participants were divided into testing set 1 and development set according to the different enrolled sites. The development set was further divided into a training set and a validation set in an 8:2 ratio for model development. At 48-month follow-up, eligible patients were formed testing set 2. The Bilateral Knee Neural Network (BikNet) was developed using bilateral views, with the knee to be predicted as the main view and the contralateral knee as the auxiliary view. DenseNet and ResNext were also trained and compared as the unilateral model. Two reader tests were conducted to evaluate the model's value in predicting incident OA.

RESULTS

Totally 3583 participants were evaluated. The BikNet we proposed outperformed ResNext and DenseNet (all area under the curve [AUC] < 0.71, P < 0.001) with AUC values of 0.761 and 0.745 in testing sets 1 and 2, respectively. With assistance of the BikNet increased clinicians' sensitivity (from 28.1-63.2% to 42.1-68.4%) and specificity (from 57.4-83.4% to 64.1-87.5%) of incident OA prediction and improved inter-observer reliability.

CONCLUSION

The DL model, constructed based on bilateral knee views, holds promise for enhancing the assessment of OA and demonstrates greater robustness during subsequent follow-up evaluations as compared with unilateral models. BikNet represents a potential tool or imaging biomarker for predicting OA progression.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

AI applications in musculoskeletal imaging: a narrative review

This narrative review focuses on clinical applications of artificial intelligence (AI) in musculoskeletal imaging. A range of musculoskeletal disorders are discussed using a clinical-based approach, including trauma, bone age estimation, osteoarthritis, bone and soft-tissue tumors, and orthopedic implant-related pathology. Several AI algorithms have been applied to fracture detection and classification, which are potentially helpful tools for radiologists and clinicians. In bone age assessment, AI methods have been applied to assist radiologists by automatizing workflow, thus reducing workload and inter-observer variability. AI may potentially aid radiologists in identifying and grading abnormal findings of osteoarthritis as well as predicting the onset or progression of this disease. Either alone or combined with radiomics, AI algorithms may potentially improve diagnosis and outcome prediction of bone and soft-tissue tumors. Finally, information regarding appropriate positioning of orthopedic implants and related complications may be obtained using AI algorithms. In conclusion, rather than replacing radiologists, the use of AI should instead help them to optimize workflow, augment diagnostic performance, and keep up with ever-increasing workload.Relevance statement This narrative review provides an overview of AI applications in musculoskeletal imaging. As the number of AI technologies continues to increase, it will be crucial for radiologists to play a role in their selection and application as well as to fully understand their potential value in clinical practice. Key points • AI may potentially assist musculoskeletal radiologists in several interpretative tasks.• AI applications to trauma, age estimation, osteoarthritis, tumors, and orthopedic implants are discussed.• AI should help radiologists to optimize workflow and augment diagnostic performance.

Machine Learning Models for Prediction of Progression of Knee Osteoarthritis: A Comprehensive Analysis

Prediction of the progression of knee osteoarthritis (KOA) is a very challenging task. Early identification of risk factors plays a vital role in diagnosing KOA. Thus, machine learning models are used to predict the progression of KOA. The purpose of the present study is to find out the efficacy of various machine learning models to identify the progression of KOA. A comprehensive literature search was conducted in international databases like Google Scholar, PubMed, Web of Science, and Scopus. Studies published from the year 2010 to May 2023 on the machine learning approach to diagnose KOA were included in the study. A total of 15 studies were selected and analyzed which included machine learning as an approach to diagnose KOA. The present study found that machine learning methods are the best methods to diagnose KOA early. Various methods like deep learning, machine learning, convolutional neural network (CNN), and multi-layer perceptron showed good accuracy in diagnosing its progression. The machine learning approach has attracted significant interest from scientists and researchers and has led to a new automated approach to diagnose KOA, which will help in designing treatment approaches.

How AI May Transform Musculoskeletal Imaging

While musculoskeletal imaging volumes are increasing, there is a relative shortage of subspecialized musculoskeletal radiologists to interpret the studies. Will artificial intelligence (AI) be the solution? For AI to be the solution, the wide implementation of AI-supported data acquisition methods in clinical practice requires establishing trusted and reliable results. This implementation will demand close collaboration between core AI researchers and clinical radiologists. Upon successful clinical implementation, a wide variety of AI-based tools can improve the musculoskeletal radiologist's workflow by triaging imaging examinations, helping with image interpretation, and decreasing the reporting time. Additional AI applications may also be helpful for business, education, and research purposes if successfully integrated into the daily practice of musculoskeletal radiology. The question is not whether AI will replace radiologists, but rather how musculoskeletal radiologists can take advantage of AI to enhance their expert capabilities.

Fusion of electronic health records and radiographic images for a multimodal deep learning prediction model of atypical femur fractures

Atypical femur fractures (AFF) represent a very rare type of fracture that can be difficult to discriminate radiologically from normal femur fractures (NFF). AFFs are associated with drugs that are administered to prevent osteoporosis-related fragility fractures, which are highly prevalent in the elderly population. Given that these fractures are rare and the radiologic changes are subtle currently only 7% of AFFs are correctly identified, which hinders adequate treatment for most patients with AFF. Deep learning models could be trained to classify automatically a fracture as AFF or NFF, thereby assisting radiologists in detecting these rare fractures. Historically, for this classification task, only imaging data have been used, using convolutional neural networks (CNN) or vision transformers applied to radiographs. However, to mimic situations in which all available data are used to arrive at a diagnosis, we adopted an approach of deep learning that is based on the integration of image data and tabular data (from electronic health records) for 159 patients with AFF and 914 patients with NFF. We hypothesized that the combinatorial data, compiled from all the radiology departments of 72 hospitals in Sweden and the Swedish National Patient Register, would improve classification accuracy, as compared to using only one modality. At the patient level, the area under the ROC curve (AUC) increased from 0.966 to 0.987 when using the integrated set of imaging data and seven pre-selected variables, as compared to only using imaging data. More importantly, the sensitivity increased from 0.796 to 0.903. We found a greater impact of data fusion when only a randomly selected subset of available images was used to make the image and tabular data more balanced for each patient. The AUC then increased from 0.949 to 0.984, and the sensitivity increased from 0.727 to 0.849. These AUC improvements are not large, mainly because of the already excellent performance of the CNN (AUC of 0.966) when only images are used. However, the improvement is clinically highly relevant considering the importance of accuracy in medical diagnostics. We expect an even greater effect when imaging data from a clinical workflow, comprising a more diverse set of diagnostic images, are used.

Convolutional-neural-network-based radiographs evaluation assisting in early diagnosis of the periodontal bone loss via periapical radiograph

Background/Purpose

The preciseness of detecting periodontal bone loss is examiners dependent, and this leads to low reliability. The need for automated assistance systems on dental radiographic images has been increased. To the best of our knowledge, no studies have quantitatively and automatically staged periodontitis using dental periapical radiographs. The purpose of this study was to evaluate periodontal bone loss and periodontitis stage on dental periapical radiographs using deep convolutional neural networks (CNNs).

Materials and methods

336 periapical radiographic images (teeth: 390) between January 2017 and December 2019 were collected and de-identified. All periapical radiographic image datasets were divided into training dataset (n = 82, teeth: 123) and test dataset (n = 336, teeth: 390). For creating an optimal deep CNN algorithm model, the training datasets were directly used for the segmentation and individual tooth detection. To evaluate the diagnostic power, we calculated the degree of alveolar bone loss deviation between our proposed method and ground truth, the Pearson correlation coefficients (PCC), and the diagnostic accuracy of the proposed method in the test datasets.

Results

The periodontal bone loss degree deviation between our proposed method and the ground truth drawn by the three periodontists was 6.5 %. In addition, the overall PCC value of our proposed system and the periodontists' diagnoses was 0.828 (P < 0.01). The total diagnostic accuracy of our proposed method was 72.8 %. The diagnostic accuracy was highest for stage III (97.0 %).

Conclusion

This tool helps with diagnosis and prevents omission, and this may be especially helpful for inexperienced younger doctors and doctors in underdeveloped countries. It could also dramatically reduce the workload of clinicians and timely access to periodontist care for people requiring advanced periodontal treatment.

Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data

Deep neural networks are often applied to medical images to automate the problem of medical diagnosis. However, a more clinically relevant question that practitioners usually face is how to predict the future trajectory of a disease. Current methods for prognosis or disease trajectory forecasting often require domain knowledge and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents-a radiologist and a general practitioner - we predict prognosis with two transformer-based components that share information with each other. The first transformer in this framework aims to analyze the imaging data, and the second one leverages its internal states as inputs, also fusing them with auxiliary clinical data. The temporal nature of the problem is modeled within the transformer states, allowing us to treat the forecasting problem as a multi-task classification, for which we propose a novel loss. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer's disease clinical status directly from raw multi-modal data. The proposed method outperforms multiple state-of-the-art baselines with respect to performance and calibration, both of which are needed for real-world applications. An open-source implementation of our method is made publicly available at https://github.com/Oulu-IMEDS/CLIMATv2.

A framework of biomarkers for skeletal aging: a consensus statement by the Aging Biomarker Consortium

The skeleton is an important structural and metabolic organ in human body, while aging is the physiological basis for degenerative skeletal diseases. China has the largest aging population in the world and faces great challenges in preventing and managing diseases related to skeletal aging. To address these challenges, the Aging China Biomarkers Consortium (ABC) has reached an expert consensus on biomarkers of skeletal aging by synthesizing the literature and insights from scientists and clinicians. The consensus provides a comprehensive assessment of biomarkers associated with skeletal aging and proposes a systematic framework that categorizes biomarkers into three dimensions, namely, functional, structural, and humoral dimensions. Within each dimension, the ABC recommended clinical and evidential research-based biomarkers for physiological aging and degenerative pathologies of the skeleton. This expert consensus aims to lay the foundation for future studies to assess the prediction, diagnosis, early warning, and treatment of diseases associated with skeletal aging, with the ultimate goal of improving the skeletal health of elderly populations in China and around the world.

Development of an automated optimal distance feature-based decision system for diagnosing knee osteoarthritis using segmented X-ray images

Knee Osteoarthritis (KOA) is a leading cause of disability and physical inactivity. It is a degenerative joint disease that affects the cartilage, cushions the bones, and protects them from rubbing against each other during motion. If not treated early, it may lead to knee replacement. In this regard, early diagnosis of KOA is necessary for better treatment. Nevertheless, manual KOA detection is time-consuming and error-prone for large data hubs. In contrast, an automated detection system aids the specialist in diagnosing KOA grades accurately and quickly. So, the main objective of this study is to create an automated decision system that can analyze KOA and classify the severity grades, utilizing the extracted features from segmented X-ray images. In this study, two different datasets were collected from the Mendeley and Kaggle database and combined to generate a large data hub containing five classes: Grade 0 (Healthy), Grade 1 (Doubtful), Grade 2 (Minimal), Grade 3 (Moderate), and Grade 4 (Severe). Several image processing techniques were employed to segment the region of interest (ROI). These included Gradient-weighted Class Activation Mapping (Grad-Cam) to detect the ROI, cropping the ROI portion, applying histogram equalization (HE) to improve contrast, brightness, and image quality, and noise reduction (using Otsu thresholding, inverting the image, and morphological closing). Besides, the focus filtering method was utilized to eliminate unwanted images. Then, six feature sets (morphological, GLCM, statistical, texture, LBP, and proposed features) were generated from segmented ROIs. After evaluating the statistical significance of the features and selection methods, the optimal feature set (prominent six distance features) was selected, and five machine learning (ML) models were employed. Additionally, a decision-making strategy based on the six optimal features is proposed. The XGB model outperformed other models with a 99.46 % accuracy, using six distance features, and the proposed decision-making strategy was validated by testing 30 images.

Deep learning applications in osteoarthritis imaging

Deep learning (DL) is one of the most exciting new areas in medical imaging. This article will provide a review of current applications of DL in osteoarthritis (OA) imaging, including methods used for cartilage lesion detection, OA diagnosis, cartilage segmentation, and OA risk assessment. DL techniques have been shown to have similar diagnostic performance as human readers for detecting and grading cartilage lesions within the knee on MRI. A variety of DL methods have been developed for detecting and grading the severity of knee OA and various features of knee OA on X-rays using standardized classification systems with diagnostic performance similar to human readers. Multiple DL approaches have been described for fully automated segmentation of cartilage and other knee tissues and have achieved higher segmentation accuracy than currently used methods with substantial reductions in segmentation times. Various DL models analyzing baseline X-rays and MRI have been developed for OA risk assessment. These models have shown high diagnostic performance for predicting a wide variety of OA outcomes, including the incidence and progression of radiographic knee OA, the presence and progression of knee pain, and future total knee replacement. The preliminary results of DL applications in OA imaging have been encouraging. However, many DL techniques require further technical refinement to maximize diagnostic performance. Furthermore, the generalizability of DL approaches needs to be further investigated in prospective studies using large image datasets acquired at different institutions with different imaging hardware before they can be implemented in clinical practice and research studies.

Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

Bimodal artificial intelligence using TabNet for differentiating spinal cord tumors-Integration of patient background information and images

We proposed a bimodal artificial intelligence that integrates patient information with images to diagnose spinal cord tumors. Our model combines TabNet, a state-of-the-art deep learning model for tabular data for patient information, and a convolutional neural network for images. As training data, we collected 259 spinal tumor patients (158 for schwannoma and 101 for meningioma). We compared the performance of the image-only unimodal model, table-only unimodal model, bimodal model using a gradient-boosting decision tree, and bimodal model using TabNet. Our proposed bimodal model using TabNet performed best (area under the receiver-operating characteristic curve [AUROC]: 0.91) in the training data and significantly outperformed the physicians' performance. In the external validation using 62 cases from the other two facilities, our bimodal model showed an AUROC of 0.92, proving the robustness of the model. The bimodal analysis using TabNet was effective for differentiating spinal tumors.

The Viability of an Artificial Intelligence/Machine Learning Prediction Model to Determine Candidates for Knee Arthroplasty

BACKGROUND

The purpose of this study is to assess the viability of a knee arthroplasty prediction model using 3-view X-rays that helps determine if patients with knee pain are candidates for total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA), or are not arthroplasty candidates.

METHODS

Analysis was performed using radiographic and surgical data from a high-volume joint replacement practice. The dataset included 3 different X-ray views (anterior-posterior, lateral, and sunrise) for 2,767 patients along with information of whether that patient underwent an arthroplasty surgery (UKA or TKA) or not. This resulted in a dataset including 8,301 images from 2,707 patients. This dataset was then split into a training set (70%) and holdout test set (30%). A computer vision model was trained using a transfer learning approach. The performance of the computer vision model was evaluated on the holdout test set. Accuracy and multiclass receiver operating characteristic area under curve was used to evaluate the performance of the model.

RESULTS

The artificial intelligence model achieved an accuracy of 87.8% on the holdout test set and a quadratic Cohen's kappa score of 0.811. The multiclass receiver operating characteristic area under curve score for TKA was calculated to be 0.97; for UKA a score of 0.96 and for No Surgery a score of 0.98 was achieved. An accuracy of 93.8% was achieved for predicting Surgery versus No Surgery and 88% for TKA versus not TKA was achieved.

CONCLUSION

The artificial intelligence/machine learning model demonstrated viability for predicting which patients are candidates for a UKA, TKA, or no surgical intervention.

Charting Aging Trajectories of Knee Cartilage Thickness for Early Osteoarthritis Risk Prediction: An MRI Study from the Osteoarthritis Initiative Cohort

Knee osteoarthritis (OA), a prevalent joint disease in the U.S., poses challenges in terms of predicting of its early progression. Although high-resolution knee magnetic resonance imaging (MRI) facilitates more precise OA diagnosis, the heterogeneous and multifactorial aspects of OA pathology remain significant obstacles for prognosis. MRI-based scoring systems, while standardizing OA assessment, are both time-consuming and labor-intensive. Current AI technologies facilitate knee OA risk scoring and progression prediction, but these often focus on the symptomatic phase of OA, bypassing initial-stage OA prediction. Moreover, their reliance on complex algorithms can hinder clinical interpretation. To this end, we make this effort to construct a computationally efficient, easily-interpretable, and state-of-the-art approach aiding in the radiographic OA (rOA) auto-classification and prediction of the incidence and progression, by contrasting an individual's cartilage thickness with a similar demographic in the rOA-free cohort. To better visualize, we have developed the toolset for both prediction and local visualization. A movie demonstrating different subtypes of dynamic changes in local centile scores during rOA progression is available at https://tli3.github.io/KneeOA/. Specifically, we constructed age-BMI-dependent reference charts for knee OA cartilage thickness, based on MRI scans from 957 radiographic OA (rOA)-free individuals from the Osteoarthritis Initiative cohort. Then we extracted local and global centiles by contrasting an individual's cartilage thickness to the rOA-free cohort with a similar age and BMI. Using traditional boosting approaches with our centile-based features, we obtain rOA classification of KLG ≤ 1 versus KLG = 2 (AUC = 0.95, F1 = 0.89), KLG ≤ 1 versus KLG ≥ 2 (AUC = 0.90, F1 = 0.82) and prediction of KLG2 progression (AUC = 0.98, F1 = 0.94), rOA incidence (KLG increasing from < 2 to ≥ 2; AUC = 0.81, F1 = 0.69) and rOA initial transition (KLG from 0 to 1; AUC = 0.64, F1 = 0.65) within a future 48-month period. Such performance in classifying KLG ≥ 2 matches that of deep learning methods in recent literature. Furthermore, its clinical interpretation suggests that cartilage changes, such as thickening in lateral femoral and anterior femoral regions and thinning in lateral tibial regions, may serve as indicators for prediction of rOA incidence and early progression. Meanwhile, cartilage thickening in the posterior medial and posterior lateral femoral regions, coupled with a reduction in the central medial femoral region, may signify initial phases of rOA transition.

Toward New Assessment of Knee Cartilage Degeneration

OBJECTIVE

Assessment of human joint cartilage is a crucial tool to detect and diagnose pathological conditions. This exploratory study developed a workflow for 3D modeling of cartilage and bone based on multimodal imaging. New evaluation metrics were created and, a unique set of data was gathered from healthy controls and patients with clinically evaluated degeneration or trauma.

DESIGN

We present a novel methodology to evaluate knee bone and cartilage based on features extracted from magnetic resonance imaging (MRI) and computed tomography (CT) data. We developed patient specific 3D models of the tibial, femoral, and patellar bones and cartilages. Forty-seven subjects with a history of degenerative disease, traumatic events, or no symptoms or trauma (control group) were recruited in this study. Ninety-six different measurements were extracted from each knee, 78 2D and 18 3D measurements. We compare the sensitivity of different metrics to classify the cartilage condition and evaluate degeneration.

RESULTS

Selected features extracted show significant difference between the 3 groups. We created a cumulative index of bone properties that demonstrated the importance of bone condition to assess cartilage quality, obtaining the greatest sensitivity on femur within medial and femoropatellar compartments. We were able to classify degeneration with a maximum recall value of 95.9 where feature importance analysis showed a significant contribution of the 3D parameters.

CONCLUSION

The present work demonstrates the potential for improving sensitivity in cartilage assessment. Indeed, current trends in cartilage research point toward improving treatments and therefore our contribution is a first step toward sensitive and personalized evaluation of cartilage condition.

Gait, physical activity and tibiofemoral cartilage damage: a longitudinal machine learning analysis in the Multicenter Osteoarthritis Study

OBJECTIVE

To (1) develop and evaluate a machine learning model incorporating gait and physical activity to predict medial tibiofemoral cartilage worsening over 2 years in individuals without advanced knee osteoarthritis and (2) identify influential predictors in the model and quantify their effect on cartilage worsening.

DESIGN

An ensemble machine learning model was developed to predict worsened cartilage MRI Osteoarthritis Knee Score at follow-up from gait, physical activity, clinical and demographic data from the Multicenter Osteoarthritis Study. Model performance was evaluated in repeated cross-validations. The top 10 predictors of the outcome across 100 held-out test sets were identified by a variable importance measure. Their effect on the outcome was quantified by g-computation.

RESULTS

Of 947 legs in the analysis, 14% experienced medial cartilage worsening at follow-up. The median (2.5-97.5th percentile) area under the receiver operating characteristic curve across the 100 held-out test sets was 0.73 (0.65-0.79). Baseline cartilage damage, higher Kellgren-Lawrence grade, greater pain during walking, higher lateral ground reaction force impulse, greater time spent lying and lower vertical ground reaction force unloading rate were associated with greater risk of cartilage worsening. Similar results were found for the subset of knees with baseline cartilage damage.

CONCLUSIONS

A machine learning approach incorporating gait, physical activity and clinical/demographic features showed good performance for predicting cartilage worsening over 2 years. While identifying potential intervention targets from the model is challenging, lateral ground reaction force impulse, time spent lying and vertical ground reaction force unloading rate should be investigated further as potential early intervention targets to reduce medial tibiofemoral cartilage worsening.

DeepKOA: a deep-learning model for predicting progression in knee osteoarthritis using multimodal magnetic resonance images from the osteoarthritis initiative

Background

No investigations have thoroughly explored the feasibility of combining magnetic resonance (MR) images and deep-learning methods for predicting the progression of knee osteoarthritis (KOA). We thus aimed to develop a potential deep-learning model for predicting OA progression based on MR images for the clinical setting.

Methods

A longitudinal case-control study was performed using data from the Foundation for the National Institutes of Health (FNIH), composed of progressive cases [182 osteoarthritis (OA) knees with both radiographic and pain progression for 24-48 months] and matched controls (182 OA knees not meeting the case definition). DeepKOA was developed through 3-dimensional (3D) DenseNet169 to predict KOA progression over 24-48 months based on sagittal intermediate-weighted turbo-spin echo sequences with fat-suppression (SAG-IW-TSE-FS), sagittal 3D dual-echo steady-state water excitation (SAG-3D-DESS-WE) and its axial and coronal multiplanar reformation, and their combined MR images with patient-level labels at baseline, 12, and 24 months to eventually determine the probability of progression. The classification performance of the DeepKOA was evaluated using 5-fold cross-validation. An X-ray-based model and traditional models that used clinical variables via multilayer perceptron were built. Combined models were also constructed, which integrated clinical variables with DeepKOA. The area under the curve (AUC) was used as the evaluation metric.

Results

The performance of SAG-IW-TSE-FS in predicting OA progression was similar or higher to that of other single and combined sequences. The DeepKOA based on SAG-IW-TSE-FS achieved an AUC of 0.664 (95% CI: 0.585-0.743) at baseline, 0.739 (95% CI: 0.703-0.775) at 12 months, and 0.775 (95% CI: 0.686-0.865) at 24 months. The X-ray-based model achieved an AUC ranging from 0.573 to 0.613 at 3 time points. However, adding clinical variables to DeepKOA did not improve performance (P>0.05). Initial visualizations from gradient-weighted class activation mapping (Grad-CAM) indicated that the frequency with which the patellofemoral joint was highlighted increased as time progressed, which contrasted the trend observed in the tibiofemoral joint. The meniscus, the infrapatellar fat pad, and muscles posterior to the knee were highlighted to varying degrees.

Conclusions

This study initially demonstrated the feasibility of DeepKOA in the prediction of KOA progression and identified the potential responsible structures which may enlighten the future development of more clinically practical methods.

A role for artificial intelligence applications inside and outside of the operating theatre: a review of contemporary use associated with total knee arthroplasty

BACKGROUND

Artificial intelligence (AI) has become involved in many aspects of everyday life, from voice-activated virtual assistants built into smartphones to global online search engines. Similarly, many areas of modern medicine have found ways to incorporate such technologies into mainstream practice. Despite the enthusiasm, robust evidence to support the utility of AI in contemporary total knee arthroplasty (TKA) remains limited. The purpose of this review was to provide an up-to-date summary of the use of AI in TKA and to explore its current and future value.

METHODS

Initially, a structured systematic review of the literature was carried out, following PRISMA search principles, with the aim of summarising the understanding of the field and identifying clinical and knowledge gaps.

RESULTS

A limited body of published work exists in this area. Much of the available literature is of poor methodological quality and many published studies could be best described as "demonstration of concepts" rather than "proof of concepts". There exists almost no independent validation of reported findings away from designer/host sites, and the extrapolation of key results to general orthopaedic sites is limited.

CONCLUSION

While AI has certainly shown value in a small number of specific TKA-associated applications, the majority to date have focused on risk, cost and outcome prediction, rather than surgical care, per se. Extensive future work is needed to demonstrate external validity and reliability in non-designer settings. Well-performed studies are warranted to ensure that the scientific evidence base supporting the use of AI in knee arthroplasty matches the global hype.

Automatic Measuring of Finger Joint Space Width on Hand Radiograph using Deep Learning and Conventional Computer Vision Methods

Hand osteoarthritis (OA) severity can be assessed visually through radiographs using semi-quantitative grading systems. However, these grading systems are subjective and cannot distinguish minor differences. Joint space width (JSW) compensates for these disadvantages, as it quantifies the severity of OA by accurately measuring the distances between joint bones. Current methods used to assess JSW require users' interaction to identify the joints and delineate initial joint boundary, which is time-consuming. To automate this process and offer a more efficient and robust measurement for JSW, we proposed two novel methods to measure JSW: 1) The segmentation-based (SEG) method, which uses traditional computer vision techniques to calculate JSW; 2) The regression-based (REG) method, which is a deep learning approach employing a modified VGG-19 network to predict JSW. On a dataset with 3,591 hand radiographs, 10,845 DIP joints were cut as regions of interest and served as input to the SEG and REG methods. The bone masks of the ROI images generated by a U-Net model were sent as input in addition to the ROIs. The ground truth of JSW was labeled by a trained research assistant using a semi-automatic tool. Compared with the ground truth, the REG method achieved a correlation coefficient of 0.88 and mean square error (MSE) of 0.02 mm on the testing set; the SEG method achieved a correlation coefficient of 0.42 and MSE of 0.15 mm. Results show the REG method has promising performance in automatic JSW measurement and in general, Deep Learning approaches can facilitate the automatic quantification of distance features in medical images.

Atlas-based finite element analyses with simpler constitutive models predict personalized progression of knee osteoarthritis: data from the osteoarthritis initiative

New technologies are required to support a radical shift towards preventive healthcare. Here we focus on evaluating the possibility of finite element (FE) analysis-aided prevention of knee osteoarthritis (OA), a disease that affects 100 million citizens in the US and EU and this number is estimated to increase drastically. Current clinical methods to diagnose or predict joint health status relies on symptoms and tissue failures obtained from clinical imaging. In a joint with no detectable injuries, the diagnosis of the future health of the knee can be assumed to be very subjective. Quantitative approaches are therefore needed to assess the personalized risk for the onset and development of knee OA. FE analysis utilizing an atlas-based modeling approach has shown a preliminary capability for simulating subject-specific cartilage mechanical responses. However, it has been verified with a very limited subject number. Thus, the aim of this study is to verify the real capability of the atlas-based approach to simulate cartilage degeneration utilizing different material descriptions for cartilage. A fibril reinforced poroviscoelastic (FRPVE) material formulation was considered as state-of-the-art material behavior, since it has been preliminary validated against real clinical follow-up data. Simulated mechanical tissue responses and predicted cartilage degenerations within knee joint with FRPVE material were compared against simpler constitutive models for cartilage. The capability of the atlas-based modeling to offer a feasible approach with quantitative evaluation for the risk for the OA development (healthy vs osteoarthritic knee, p < 0.01, AUC ~ 0.7) was verified with 214 knees. Furthermore, the results suggest that accuracy for simulation of cartilage degeneration with simpler material models is similar to models using FPRVE materials if the material parameters are chosen properly.

A voting-based machine learning approach for classifying biological and clinical datasets

BACKGROUND

Different machine learning techniques have been proposed to classify a wide range of biological/clinical data. Given the practicability of these approaches accordingly, various software packages have been also designed and developed. However, the existing methods suffer from several limitations such as overfitting on a specific dataset, ignoring the feature selection concept in the preprocessing step, and losing their performance on large-size datasets. To tackle the mentioned restrictions, in this study, we introduced a machine learning framework consisting of two main steps. First, our previously suggested optimization algorithm (Trader) was extended to select a near-optimal subset of features/genes. Second, a voting-based framework was proposed to classify the biological/clinical data with high accuracy. To evaluate the efficiency of the proposed method, it was applied to 13 biological/clinical datasets, and the outcomes were comprehensively compared with the prior methods.

RESULTS

The results demonstrated that the Trader algorithm could select a near-optimal subset of features with a significant level of p-value < 0.01 relative to the compared algorithms. Additionally, on the large-sie datasets, the proposed machine learning framework improved prior studies by ~ 10% in terms of the mean values associated with fivefold cross-validation of accuracy, precision, recall, specificity, and F-measure.

CONCLUSION

Based on the obtained results, it can be concluded that a proper configuration of efficient algorithms and methods can increase the prediction power of machine learning approaches and help researchers in designing practical diagnosis health care systems and offering effective treatment plans.