Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data

Predicting the onset of end-stage knee osteoarthritis over two- and five-years using machine learning

OBJECTIVE

Identifying participants who will progress to advanced stage in knee osteoarthritis (KOA) trials remains a significant challenge. Current tools, relying on total knee replacements (TKR), fall short in reliability due to the extraneous factors influencing TKR decisions. Acknowledging these limitations, our study identifies a critical need for a more robust metric to assess severe KOA. The end-stage KOA (esKOA) measure, which combines symptomatic and radiographic criteria, serves as a solid indicator. To enhance future trials that use esKOA as an endpoint, our study focuses on developing and validating a machine-learning tool to identify individuals likely to develop esKOA within 2 to 5 years.

DESIGN

Utilizing the Osteoarthritis Initiative (OAI) data, we trained models on 3,114 participants and validated them with 606 participants for the right knee, and similarly for the left knee, with external validation from the Multicentre Osteoarthritis Study (MOST) involving 1,602 participants. We aimed to predict esKOA onset at 2-to-2.5 years and 4-to-5 years, defining esKOA by severe radiographic KOA with moderate/severe symptoms or mild/moderate radiographic KOA with persistent/intense symptoms. Our analysis considered 51 candidate predictors, including demographics, clinical history, physical examination, and X-ray evaluations. An online tool predicting esKOA progression, based on models with ten and nine predictors for the right and left knees, respectively, was developed.

RESULTS

External validation (MOST) for the right knee at 2.5 years yielded an Area Under Curve (AUC) of 0.847 (95 % CI 0.811 to 0.882), and at 5 years, 0.853 (95 % CI 0.823 to 0.881); for the left knee at 2.5 years, AUC was 0.824 (95 % CI 0.782 to 0.857), and at 5 years, 0.807 (95 % CI 0.768 to 0.843). Models with fewer predictors demonstrated comparable performance. The online tool is available at: https://eskoa.shinyapps.io/webapp/.

CONCLUSION

Our study unveils a robust, externally validated machine learning tool proficient in predicting the onset of esKOA over the next 2 to 5 years. Our tool can lead to more efficient KOA trials.

An interpretable knee replacement risk assessment system for osteoarthritis patients

Objective

Knee osteoarthritis (OA) is a complex disease with heterogeneous representations. Although it is modifiable to prevention and early treatment, there still lacks a reliable and accurate prognostic tool. Hence, we aim to develop a quantitative and self-administrable knee replacement (KR) risk stratification system for knee osteoarthritis (KOA) patients with clinical features.

Method

A total of 14 baseline features were extracted from 9592 cases in the Osteoarthritis Initiative (OAI) cohort. A survival model was constructed using the Random Survival Forests algorithm. The prediction performance was evaluated with the concordance index (C-index) and average receiver operating characteristic curve (AUC). A three-class KR risk stratification system was built to differentiate three distinct KR-free survival groups. Thereafter, Shapley Additive Explanations (SHAP) was introduced for model explanation.

Results

KR incidence was accurately predicted by the model with a C-index of 0.770 (±0.0215) and an average AUC of 0.807 (±0.0181) with 14 clinical features. Three distinct survival groups were observed from the ten-point KR risk stratification system with a four-year KR rate of 0.79%, 5.78%, and 16.2% from the low, medium, and high-risk groups respectively. KR is mainly caused by pain medication use, age, surgery history, diabetes, and a high body mass index, as revealed by SHAP.

Conclusion

A self-administrable and interpretable KR survival model was developed, underscoring a KR risk scoring system to stratify KOA patients. It will encourage regular self-assessments within the community and facilitate personalised healthcare for both primary and secondary prevention of KOA.

Dual-energy X-ray absorptiometry derived knee shape may provide a useful imaging biomarker for predicting total knee replacement: Findings from a study of 37,843 people in UK Biobank

Objective

We aimed to create an imaging biomarker for knee shape using knee dual-energy x-ray absorptiometry (DXA) scans and investigate its potential association with subsequent total knee replacement (TKR), independently of radiographic features of knee osteoarthritis and established risk factors.

Methods

Using a 129-point statistical shape model, knee shape (expressed as a B-score) and minimum joint space width (mJSW) of the medial joint compartment (binarized as above or below the first quartile) were derived. Osteophytes were manually graded in a subset of images and an overall score was assigned. Cox proportional hazards models were used to examine the associations of B-score, mJSW and osteophyte score with TKR risk, adjusting for age, sex, height and weight.

Results

The analysis included 37,843 individuals (mean age 63.7 years). In adjusted models, B-score was associated with TKR: each unit increase in B-score, reflecting one standard deviation from the mean healthy shape, corresponded to a hazard ratio (HR) of 2.25 (2.08, 2.43), while a lower mJSW had a HR of 2.28 (1.88, 2.77). Among the 6719 images scored for osteophytes, mJSW was replaced by osteophyte score in the most strongly predictive model for TKR. In ROC analyses, a model combining B-score, osteophyte score, and demographics outperformed a model including demographics alone (AUC ​= ​0.87 vs 0.73).

Conclusions

Using statistical shape modelling, we derived a DXA-based imaging biomarker for knee shape that was associated with kOA progression. When combined with osteophytes and demographic data, this biomarker may help identify individuals at high risk of TKR, facilitating targeted interventions.

Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data, and Symptomatic Assessments

OBJECTIVE

 In this study, we propose a novel framework that utilizes deep learning and attention mechanisms to predict the radiographic progression of patellofemoral osteoarthritis (PFOA) over a period of 7 years.

MATERIAL AND METHODS

 This study included subjects (1,832 subjects, 3,276 knees) from the baseline of the Multicenter Osteoarthritis Study (MOST). Patellofemoral joint regions of interest were identified using an automated landmark detection tool (BoneFinder) on lateral knee X-rays. An end-to-end deep learning method was developed for predicting PFOA progression based on imaging data in a five-fold cross-validation setting. To evaluate the performance of the models, a set of baselines based on known risk factors were developed and analyzed using gradient boosting machine (GBM). Risk factors included age, sex, body mass index, and Western Ontario and McMaster Universities Arthritis Index score, and the radiographic osteoarthritis stage of the tibiofemoral joint (Kellgren and Lawrence [KL] score). Finally, to increase predictive power, we trained an ensemble model using both imaging and clinical data.

RESULTS

 Among the individual models, the performance of our deep convolutional neural network attention model achieved the best performance with an area under the receiver operating characteristic curve (AUC) of 0.856 and average precision (AP) of 0.431, slightly outperforming the deep learning approach without attention (AUC = 0.832, AP = 0.4) and the best performing reference GBM model (AUC = 0.767, AP = 0.334). The inclusion of imaging data and clinical variables in an ensemble model allowed statistically more powerful prediction of PFOA progression (AUC = 0.865, AP = 0.447), although the clinical significance of this minor performance gain remains unknown. The spatial attention module improved the predictive performance of the backbone model, and the visual interpretation of attention maps focused on the joint space and the regions where osteophytes typically occur.

CONCLUSION

 This study demonstrated the potential of machine learning models to predict the progression of PFOA using imaging and clinical variables. These models could be used to identify patients who are at high risk of progression and prioritize them for new treatments. However, even though the accuracy of the models were excellent in this study using the MOST dataset, they should be still validated using external patient cohorts in the future.

Highly Flexible, High-Performance, and Stretchable Piezoelectric Sensor Based on a Hierarchical Droplet-Shaped Ceramics with Enhanced Damage Tolerance

Stretchable self-powered sensors are of significant interest in next-generation wearable electronics. However, current strategies for creating stretchable piezoelectric sensors based on piezoelectric polymers or 0-3 piezoelectric composites face several challenges such as low piezoelectric activity, low sensitivity, and poor durability. In this paper, a biomimetic soft-rigid hybrid strategy is used to construct a new form of highly flexible, high-performance, and stretchable piezoelectric sensor. Inspired by the hinged bivalve Cristaria plicata, hierarchical droplet-shaped ceramics are manufactured and used as rigid components, where computational models indicate that the unique arched curved surface and rounded corners of this bionic structure can alleviate stress concentrations. To ensure electrical connectivity of the piezoelectric phase during stretching, a patterned liquid metal acts as a soft circuit and a silicone polymer with optimized wettability and stretchability serves as a soft component that forms a strong mechanical interlock with the hierarchical ceramics. The novel sensor design exhibits excellent sensitivity and durability, where the open circuit voltage remains stable after 5000 stretching cycles at 60% strain and 5000 twisting cycles at 180°. To demonstrate its potential in heathcare applications, this new stretchable sensor is successfully used for wireless gesture recognition and assessing the progression of knee osteoarthritis.

A Stepwise Approach to Analyzing Musculoskeletal Imaging Data With Artificial Intelligence

The digitization of medical records and expanding electronic health records has created an era of "Big Data" with an abundance of available information ranging from clinical notes to imaging studies. In the field of rheumatology, medical imaging is used to guide both diagnosis and treatment of a wide variety of rheumatic conditions. Although there is an abundance of data to analyze, traditional methods of image analysis are human resource intensive. Fortunately, the growth of artificial intelligence (AI) may be a solution to handle large datasets. In particular, computer vision is a field within AI that analyzes images and extracts information. Computer vision has impressive capabilities and can be applied to rheumatologic conditions, necessitating a need to understand how computer vision works. In this article, we provide an overview of AI in rheumatology and conclude with a five step process to plan and conduct research in the field of computer vision. The five steps include (1) project definition, (2) data handling, (3) model development, (4) performance evaluation, and (5) deployment into clinical care.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Expanding from unilateral to bilateral: A robust deep learning-based approach for predicting radiographic osteoarthritis progression

OBJECTIVE

To develop and validate a deep learning (DL) model for predicting osteoarthritis (OA) progression based on bilateral knee joint views.

METHODS

In this retrospective study, knee joints from bilateral posteroanterior knee radiographs of participants in the Osteoarthritis Initiative were analyzed. At baseline, participants were divided into testing set 1 and development set according to the different enrolled sites. The development set was further divided into a training set and a validation set in an 8:2 ratio for model development. At 48-month follow-up, eligible patients were formed testing set 2. The Bilateral Knee Neural Network (BikNet) was developed using bilateral views, with the knee to be predicted as the main view and the contralateral knee as the auxiliary view. DenseNet and ResNext were also trained and compared as the unilateral model. Two reader tests were conducted to evaluate the model's value in predicting incident OA.

RESULTS

Totally 3583 participants were evaluated. The BikNet we proposed outperformed ResNext and DenseNet (all area under the curve [AUC] < 0.71, P < 0.001) with AUC values of 0.761 and 0.745 in testing sets 1 and 2, respectively. With assistance of the BikNet increased clinicians' sensitivity (from 28.1-63.2% to 42.1-68.4%) and specificity (from 57.4-83.4% to 64.1-87.5%) of incident OA prediction and improved inter-observer reliability.

CONCLUSION

The DL model, constructed based on bilateral knee views, holds promise for enhancing the assessment of OA and demonstrates greater robustness during subsequent follow-up evaluations as compared with unilateral models. BikNet represents a potential tool or imaging biomarker for predicting OA progression.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

AI applications in musculoskeletal imaging: a narrative review

This narrative review focuses on clinical applications of artificial intelligence (AI) in musculoskeletal imaging. A range of musculoskeletal disorders are discussed using a clinical-based approach, including trauma, bone age estimation, osteoarthritis, bone and soft-tissue tumors, and orthopedic implant-related pathology. Several AI algorithms have been applied to fracture detection and classification, which are potentially helpful tools for radiologists and clinicians. In bone age assessment, AI methods have been applied to assist radiologists by automatizing workflow, thus reducing workload and inter-observer variability. AI may potentially aid radiologists in identifying and grading abnormal findings of osteoarthritis as well as predicting the onset or progression of this disease. Either alone or combined with radiomics, AI algorithms may potentially improve diagnosis and outcome prediction of bone and soft-tissue tumors. Finally, information regarding appropriate positioning of orthopedic implants and related complications may be obtained using AI algorithms. In conclusion, rather than replacing radiologists, the use of AI should instead help them to optimize workflow, augment diagnostic performance, and keep up with ever-increasing workload.Relevance statement This narrative review provides an overview of AI applications in musculoskeletal imaging. As the number of AI technologies continues to increase, it will be crucial for radiologists to play a role in their selection and application as well as to fully understand their potential value in clinical practice. Key points • AI may potentially assist musculoskeletal radiologists in several interpretative tasks.• AI applications to trauma, age estimation, osteoarthritis, tumors, and orthopedic implants are discussed.• AI should help radiologists to optimize workflow and augment diagnostic performance.

Machine Learning Models for Prediction of Progression of Knee Osteoarthritis: A Comprehensive Analysis

Prediction of the progression of knee osteoarthritis (KOA) is a very challenging task. Early identification of risk factors plays a vital role in diagnosing KOA. Thus, machine learning models are used to predict the progression of KOA. The purpose of the present study is to find out the efficacy of various machine learning models to identify the progression of KOA. A comprehensive literature search was conducted in international databases like Google Scholar, PubMed, Web of Science, and Scopus. Studies published from the year 2010 to May 2023 on the machine learning approach to diagnose KOA were included in the study. A total of 15 studies were selected and analyzed which included machine learning as an approach to diagnose KOA. The present study found that machine learning methods are the best methods to diagnose KOA early. Various methods like deep learning, machine learning, convolutional neural network (CNN), and multi-layer perceptron showed good accuracy in diagnosing its progression. The machine learning approach has attracted significant interest from scientists and researchers and has led to a new automated approach to diagnose KOA, which will help in designing treatment approaches.

How AI May Transform Musculoskeletal Imaging

While musculoskeletal imaging volumes are increasing, there is a relative shortage of subspecialized musculoskeletal radiologists to interpret the studies. Will artificial intelligence (AI) be the solution? For AI to be the solution, the wide implementation of AI-supported data acquisition methods in clinical practice requires establishing trusted and reliable results. This implementation will demand close collaboration between core AI researchers and clinical radiologists. Upon successful clinical implementation, a wide variety of AI-based tools can improve the musculoskeletal radiologist's workflow by triaging imaging examinations, helping with image interpretation, and decreasing the reporting time. Additional AI applications may also be helpful for business, education, and research purposes if successfully integrated into the daily practice of musculoskeletal radiology. The question is not whether AI will replace radiologists, but rather how musculoskeletal radiologists can take advantage of AI to enhance their expert capabilities.

Fusion of electronic health records and radiographic images for a multimodal deep learning prediction model of atypical femur fractures

Atypical femur fractures (AFF) represent a very rare type of fracture that can be difficult to discriminate radiologically from normal femur fractures (NFF). AFFs are associated with drugs that are administered to prevent osteoporosis-related fragility fractures, which are highly prevalent in the elderly population. Given that these fractures are rare and the radiologic changes are subtle currently only 7% of AFFs are correctly identified, which hinders adequate treatment for most patients with AFF. Deep learning models could be trained to classify automatically a fracture as AFF or NFF, thereby assisting radiologists in detecting these rare fractures. Historically, for this classification task, only imaging data have been used, using convolutional neural networks (CNN) or vision transformers applied to radiographs. However, to mimic situations in which all available data are used to arrive at a diagnosis, we adopted an approach of deep learning that is based on the integration of image data and tabular data (from electronic health records) for 159 patients with AFF and 914 patients with NFF. We hypothesized that the combinatorial data, compiled from all the radiology departments of 72 hospitals in Sweden and the Swedish National Patient Register, would improve classification accuracy, as compared to using only one modality. At the patient level, the area under the ROC curve (AUC) increased from 0.966 to 0.987 when using the integrated set of imaging data and seven pre-selected variables, as compared to only using imaging data. More importantly, the sensitivity increased from 0.796 to 0.903. We found a greater impact of data fusion when only a randomly selected subset of available images was used to make the image and tabular data more balanced for each patient. The AUC then increased from 0.949 to 0.984, and the sensitivity increased from 0.727 to 0.849. These AUC improvements are not large, mainly because of the already excellent performance of the CNN (AUC of 0.966) when only images are used. However, the improvement is clinically highly relevant considering the importance of accuracy in medical diagnostics. We expect an even greater effect when imaging data from a clinical workflow, comprising a more diverse set of diagnostic images, are used.

Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data

Deep neural networks are often applied to medical images to automate the problem of medical diagnosis. However, a more clinically relevant question that practitioners usually face is how to predict the future trajectory of a disease. Current methods for prognosis or disease trajectory forecasting often require domain knowledge and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents-a radiologist and a general practitioner - we predict prognosis with two transformer-based components that share information with each other. The first transformer in this framework aims to analyze the imaging data, and the second one leverages its internal states as inputs, also fusing them with auxiliary clinical data. The temporal nature of the problem is modeled within the transformer states, allowing us to treat the forecasting problem as a multi-task classification, for which we propose a novel loss. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer's disease clinical status directly from raw multi-modal data. The proposed method outperforms multiple state-of-the-art baselines with respect to performance and calibration, both of which are needed for real-world applications. An open-source implementation of our method is made publicly available at https://github.com/Oulu-IMEDS/CLIMATv2.

Convolutional-neural-network-based radiographs evaluation assisting in early diagnosis of the periodontal bone loss via periapical radiograph

Background/Purpose

The preciseness of detecting periodontal bone loss is examiners dependent, and this leads to low reliability. The need for automated assistance systems on dental radiographic images has been increased. To the best of our knowledge, no studies have quantitatively and automatically staged periodontitis using dental periapical radiographs. The purpose of this study was to evaluate periodontal bone loss and periodontitis stage on dental periapical radiographs using deep convolutional neural networks (CNNs).

Materials and methods

336 periapical radiographic images (teeth: 390) between January 2017 and December 2019 were collected and de-identified. All periapical radiographic image datasets were divided into training dataset (n = 82, teeth: 123) and test dataset (n = 336, teeth: 390). For creating an optimal deep CNN algorithm model, the training datasets were directly used for the segmentation and individual tooth detection. To evaluate the diagnostic power, we calculated the degree of alveolar bone loss deviation between our proposed method and ground truth, the Pearson correlation coefficients (PCC), and the diagnostic accuracy of the proposed method in the test datasets.

Results

The periodontal bone loss degree deviation between our proposed method and the ground truth drawn by the three periodontists was 6.5 %. In addition, the overall PCC value of our proposed system and the periodontists' diagnoses was 0.828 (P < 0.01). The total diagnostic accuracy of our proposed method was 72.8 %. The diagnostic accuracy was highest for stage III (97.0 %).

Conclusion

This tool helps with diagnosis and prevents omission, and this may be especially helpful for inexperienced younger doctors and doctors in underdeveloped countries. It could also dramatically reduce the workload of clinicians and timely access to periodontist care for people requiring advanced periodontal treatment.

Development of an automated optimal distance feature-based decision system for diagnosing knee osteoarthritis using segmented X-ray images

Knee Osteoarthritis (KOA) is a leading cause of disability and physical inactivity. It is a degenerative joint disease that affects the cartilage, cushions the bones, and protects them from rubbing against each other during motion. If not treated early, it may lead to knee replacement. In this regard, early diagnosis of KOA is necessary for better treatment. Nevertheless, manual KOA detection is time-consuming and error-prone for large data hubs. In contrast, an automated detection system aids the specialist in diagnosing KOA grades accurately and quickly. So, the main objective of this study is to create an automated decision system that can analyze KOA and classify the severity grades, utilizing the extracted features from segmented X-ray images. In this study, two different datasets were collected from the Mendeley and Kaggle database and combined to generate a large data hub containing five classes: Grade 0 (Healthy), Grade 1 (Doubtful), Grade 2 (Minimal), Grade 3 (Moderate), and Grade 4 (Severe). Several image processing techniques were employed to segment the region of interest (ROI). These included Gradient-weighted Class Activation Mapping (Grad-Cam) to detect the ROI, cropping the ROI portion, applying histogram equalization (HE) to improve contrast, brightness, and image quality, and noise reduction (using Otsu thresholding, inverting the image, and morphological closing). Besides, the focus filtering method was utilized to eliminate unwanted images. Then, six feature sets (morphological, GLCM, statistical, texture, LBP, and proposed features) were generated from segmented ROIs. After evaluating the statistical significance of the features and selection methods, the optimal feature set (prominent six distance features) was selected, and five machine learning (ML) models were employed. Additionally, a decision-making strategy based on the six optimal features is proposed. The XGB model outperformed other models with a 99.46 % accuracy, using six distance features, and the proposed decision-making strategy was validated by testing 30 images.

Deep learning applications in osteoarthritis imaging

Deep learning (DL) is one of the most exciting new areas in medical imaging. This article will provide a review of current applications of DL in osteoarthritis (OA) imaging, including methods used for cartilage lesion detection, OA diagnosis, cartilage segmentation, and OA risk assessment. DL techniques have been shown to have similar diagnostic performance as human readers for detecting and grading cartilage lesions within the knee on MRI. A variety of DL methods have been developed for detecting and grading the severity of knee OA and various features of knee OA on X-rays using standardized classification systems with diagnostic performance similar to human readers. Multiple DL approaches have been described for fully automated segmentation of cartilage and other knee tissues and have achieved higher segmentation accuracy than currently used methods with substantial reductions in segmentation times. Various DL models analyzing baseline X-rays and MRI have been developed for OA risk assessment. These models have shown high diagnostic performance for predicting a wide variety of OA outcomes, including the incidence and progression of radiographic knee OA, the presence and progression of knee pain, and future total knee replacement. The preliminary results of DL applications in OA imaging have been encouraging. However, many DL techniques require further technical refinement to maximize diagnostic performance. Furthermore, the generalizability of DL approaches needs to be further investigated in prospective studies using large image datasets acquired at different institutions with different imaging hardware before they can be implemented in clinical practice and research studies.

Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

Bimodal artificial intelligence using TabNet for differentiating spinal cord tumors-Integration of patient background information and images

We proposed a bimodal artificial intelligence that integrates patient information with images to diagnose spinal cord tumors. Our model combines TabNet, a state-of-the-art deep learning model for tabular data for patient information, and a convolutional neural network for images. As training data, we collected 259 spinal tumor patients (158 for schwannoma and 101 for meningioma). We compared the performance of the image-only unimodal model, table-only unimodal model, bimodal model using a gradient-boosting decision tree, and bimodal model using TabNet. Our proposed bimodal model using TabNet performed best (area under the receiver-operating characteristic curve [AUROC]: 0.91) in the training data and significantly outperformed the physicians' performance. In the external validation using 62 cases from the other two facilities, our bimodal model showed an AUROC of 0.92, proving the robustness of the model. The bimodal analysis using TabNet was effective for differentiating spinal tumors.

The Viability of an Artificial Intelligence/Machine Learning Prediction Model to Determine Candidates for Knee Arthroplasty

BACKGROUND

The purpose of this study is to assess the viability of a knee arthroplasty prediction model using 3-view X-rays that helps determine if patients with knee pain are candidates for total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA), or are not arthroplasty candidates.

METHODS

Analysis was performed using radiographic and surgical data from a high-volume joint replacement practice. The dataset included 3 different X-ray views (anterior-posterior, lateral, and sunrise) for 2,767 patients along with information of whether that patient underwent an arthroplasty surgery (UKA or TKA) or not. This resulted in a dataset including 8,301 images from 2,707 patients. This dataset was then split into a training set (70%) and holdout test set (30%). A computer vision model was trained using a transfer learning approach. The performance of the computer vision model was evaluated on the holdout test set. Accuracy and multiclass receiver operating characteristic area under curve was used to evaluate the performance of the model.

RESULTS

The artificial intelligence model achieved an accuracy of 87.8% on the holdout test set and a quadratic Cohen's kappa score of 0.811. The multiclass receiver operating characteristic area under curve score for TKA was calculated to be 0.97; for UKA a score of 0.96 and for No Surgery a score of 0.98 was achieved. An accuracy of 93.8% was achieved for predicting Surgery versus No Surgery and 88% for TKA versus not TKA was achieved.

CONCLUSION

The artificial intelligence/machine learning model demonstrated viability for predicting which patients are candidates for a UKA, TKA, or no surgical intervention.

Charting Aging Trajectories of Knee Cartilage Thickness for Early Osteoarthritis Risk Prediction: An MRI Study from the Osteoarthritis Initiative Cohort

Knee osteoarthritis (OA), a prevalent joint disease in the U.S., poses challenges in terms of predicting of its early progression. Although high-resolution knee magnetic resonance imaging (MRI) facilitates more precise OA diagnosis, the heterogeneous and multifactorial aspects of OA pathology remain significant obstacles for prognosis. MRI-based scoring systems, while standardizing OA assessment, are both time-consuming and labor-intensive. Current AI technologies facilitate knee OA risk scoring and progression prediction, but these often focus on the symptomatic phase of OA, bypassing initial-stage OA prediction. Moreover, their reliance on complex algorithms can hinder clinical interpretation. To this end, we make this effort to construct a computationally efficient, easily-interpretable, and state-of-the-art approach aiding in the radiographic OA (rOA) auto-classification and prediction of the incidence and progression, by contrasting an individual's cartilage thickness with a similar demographic in the rOA-free cohort. To better visualize, we have developed the toolset for both prediction and local visualization. A movie demonstrating different subtypes of dynamic changes in local centile scores during rOA progression is available at https://tli3.github.io/KneeOA/. Specifically, we constructed age-BMI-dependent reference charts for knee OA cartilage thickness, based on MRI scans from 957 radiographic OA (rOA)-free individuals from the Osteoarthritis Initiative cohort. Then we extracted local and global centiles by contrasting an individual's cartilage thickness to the rOA-free cohort with a similar age and BMI. Using traditional boosting approaches with our centile-based features, we obtain rOA classification of KLG ≤ 1 versus KLG = 2 (AUC = 0.95, F1 = 0.89), KLG ≤ 1 versus KLG ≥ 2 (AUC = 0.90, F1 = 0.82) and prediction of KLG2 progression (AUC = 0.98, F1 = 0.94), rOA incidence (KLG increasing from < 2 to ≥ 2; AUC = 0.81, F1 = 0.69) and rOA initial transition (KLG from 0 to 1; AUC = 0.64, F1 = 0.65) within a future 48-month period. Such performance in classifying KLG ≥ 2 matches that of deep learning methods in recent literature. Furthermore, its clinical interpretation suggests that cartilage changes, such as thickening in lateral femoral and anterior femoral regions and thinning in lateral tibial regions, may serve as indicators for prediction of rOA incidence and early progression. Meanwhile, cartilage thickening in the posterior medial and posterior lateral femoral regions, coupled with a reduction in the central medial femoral region, may signify initial phases of rOA transition.

Toward New Assessment of Knee Cartilage Degeneration

OBJECTIVE

Assessment of human joint cartilage is a crucial tool to detect and diagnose pathological conditions. This exploratory study developed a workflow for 3D modeling of cartilage and bone based on multimodal imaging. New evaluation metrics were created and, a unique set of data was gathered from healthy controls and patients with clinically evaluated degeneration or trauma.

DESIGN

We present a novel methodology to evaluate knee bone and cartilage based on features extracted from magnetic resonance imaging (MRI) and computed tomography (CT) data. We developed patient specific 3D models of the tibial, femoral, and patellar bones and cartilages. Forty-seven subjects with a history of degenerative disease, traumatic events, or no symptoms or trauma (control group) were recruited in this study. Ninety-six different measurements were extracted from each knee, 78 2D and 18 3D measurements. We compare the sensitivity of different metrics to classify the cartilage condition and evaluate degeneration.

RESULTS

Selected features extracted show significant difference between the 3 groups. We created a cumulative index of bone properties that demonstrated the importance of bone condition to assess cartilage quality, obtaining the greatest sensitivity on femur within medial and femoropatellar compartments. We were able to classify degeneration with a maximum recall value of 95.9 where feature importance analysis showed a significant contribution of the 3D parameters.

CONCLUSION

The present work demonstrates the potential for improving sensitivity in cartilage assessment. Indeed, current trends in cartilage research point toward improving treatments and therefore our contribution is a first step toward sensitive and personalized evaluation of cartilage condition.

Gait, physical activity and tibiofemoral cartilage damage: a longitudinal machine learning analysis in the Multicenter Osteoarthritis Study

OBJECTIVE

To (1) develop and evaluate a machine learning model incorporating gait and physical activity to predict medial tibiofemoral cartilage worsening over 2 years in individuals without advanced knee osteoarthritis and (2) identify influential predictors in the model and quantify their effect on cartilage worsening.

DESIGN

An ensemble machine learning model was developed to predict worsened cartilage MRI Osteoarthritis Knee Score at follow-up from gait, physical activity, clinical and demographic data from the Multicenter Osteoarthritis Study. Model performance was evaluated in repeated cross-validations. The top 10 predictors of the outcome across 100 held-out test sets were identified by a variable importance measure. Their effect on the outcome was quantified by g-computation.

RESULTS

Of 947 legs in the analysis, 14% experienced medial cartilage worsening at follow-up. The median (2.5-97.5th percentile) area under the receiver operating characteristic curve across the 100 held-out test sets was 0.73 (0.65-0.79). Baseline cartilage damage, higher Kellgren-Lawrence grade, greater pain during walking, higher lateral ground reaction force impulse, greater time spent lying and lower vertical ground reaction force unloading rate were associated with greater risk of cartilage worsening. Similar results were found for the subset of knees with baseline cartilage damage.

CONCLUSIONS

A machine learning approach incorporating gait, physical activity and clinical/demographic features showed good performance for predicting cartilage worsening over 2 years. While identifying potential intervention targets from the model is challenging, lateral ground reaction force impulse, time spent lying and vertical ground reaction force unloading rate should be investigated further as potential early intervention targets to reduce medial tibiofemoral cartilage worsening.

DeepKOA: a deep-learning model for predicting progression in knee osteoarthritis using multimodal magnetic resonance images from the osteoarthritis initiative

Background

No investigations have thoroughly explored the feasibility of combining magnetic resonance (MR) images and deep-learning methods for predicting the progression of knee osteoarthritis (KOA). We thus aimed to develop a potential deep-learning model for predicting OA progression based on MR images for the clinical setting.

Methods

A longitudinal case-control study was performed using data from the Foundation for the National Institutes of Health (FNIH), composed of progressive cases [182 osteoarthritis (OA) knees with both radiographic and pain progression for 24-48 months] and matched controls (182 OA knees not meeting the case definition). DeepKOA was developed through 3-dimensional (3D) DenseNet169 to predict KOA progression over 24-48 months based on sagittal intermediate-weighted turbo-spin echo sequences with fat-suppression (SAG-IW-TSE-FS), sagittal 3D dual-echo steady-state water excitation (SAG-3D-DESS-WE) and its axial and coronal multiplanar reformation, and their combined MR images with patient-level labels at baseline, 12, and 24 months to eventually determine the probability of progression. The classification performance of the DeepKOA was evaluated using 5-fold cross-validation. An X-ray-based model and traditional models that used clinical variables via multilayer perceptron were built. Combined models were also constructed, which integrated clinical variables with DeepKOA. The area under the curve (AUC) was used as the evaluation metric.

Results

The performance of SAG-IW-TSE-FS in predicting OA progression was similar or higher to that of other single and combined sequences. The DeepKOA based on SAG-IW-TSE-FS achieved an AUC of 0.664 (95% CI: 0.585-0.743) at baseline, 0.739 (95% CI: 0.703-0.775) at 12 months, and 0.775 (95% CI: 0.686-0.865) at 24 months. The X-ray-based model achieved an AUC ranging from 0.573 to 0.613 at 3 time points. However, adding clinical variables to DeepKOA did not improve performance (P>0.05). Initial visualizations from gradient-weighted class activation mapping (Grad-CAM) indicated that the frequency with which the patellofemoral joint was highlighted increased as time progressed, which contrasted the trend observed in the tibiofemoral joint. The meniscus, the infrapatellar fat pad, and muscles posterior to the knee were highlighted to varying degrees.

Conclusions

This study initially demonstrated the feasibility of DeepKOA in the prediction of KOA progression and identified the potential responsible structures which may enlighten the future development of more clinically practical methods.

A role for artificial intelligence applications inside and outside of the operating theatre: a review of contemporary use associated with total knee arthroplasty

BACKGROUND

Artificial intelligence (AI) has become involved in many aspects of everyday life, from voice-activated virtual assistants built into smartphones to global online search engines. Similarly, many areas of modern medicine have found ways to incorporate such technologies into mainstream practice. Despite the enthusiasm, robust evidence to support the utility of AI in contemporary total knee arthroplasty (TKA) remains limited. The purpose of this review was to provide an up-to-date summary of the use of AI in TKA and to explore its current and future value.

METHODS

Initially, a structured systematic review of the literature was carried out, following PRISMA search principles, with the aim of summarising the understanding of the field and identifying clinical and knowledge gaps.

RESULTS

A limited body of published work exists in this area. Much of the available literature is of poor methodological quality and many published studies could be best described as "demonstration of concepts" rather than "proof of concepts". There exists almost no independent validation of reported findings away from designer/host sites, and the extrapolation of key results to general orthopaedic sites is limited.

CONCLUSION

While AI has certainly shown value in a small number of specific TKA-associated applications, the majority to date have focused on risk, cost and outcome prediction, rather than surgical care, per se. Extensive future work is needed to demonstrate external validity and reliability in non-designer settings. Well-performed studies are warranted to ensure that the scientific evidence base supporting the use of AI in knee arthroplasty matches the global hype.

Atlas-based finite element analyses with simpler constitutive models predict personalized progression of knee osteoarthritis: data from the osteoarthritis initiative

New technologies are required to support a radical shift towards preventive healthcare. Here we focus on evaluating the possibility of finite element (FE) analysis-aided prevention of knee osteoarthritis (OA), a disease that affects 100 million citizens in the US and EU and this number is estimated to increase drastically. Current clinical methods to diagnose or predict joint health status relies on symptoms and tissue failures obtained from clinical imaging. In a joint with no detectable injuries, the diagnosis of the future health of the knee can be assumed to be very subjective. Quantitative approaches are therefore needed to assess the personalized risk for the onset and development of knee OA. FE analysis utilizing an atlas-based modeling approach has shown a preliminary capability for simulating subject-specific cartilage mechanical responses. However, it has been verified with a very limited subject number. Thus, the aim of this study is to verify the real capability of the atlas-based approach to simulate cartilage degeneration utilizing different material descriptions for cartilage. A fibril reinforced poroviscoelastic (FRPVE) material formulation was considered as state-of-the-art material behavior, since it has been preliminary validated against real clinical follow-up data. Simulated mechanical tissue responses and predicted cartilage degenerations within knee joint with FRPVE material were compared against simpler constitutive models for cartilage. The capability of the atlas-based modeling to offer a feasible approach with quantitative evaluation for the risk for the OA development (healthy vs osteoarthritic knee, p < 0.01, AUC ~ 0.7) was verified with 214 knees. Furthermore, the results suggest that accuracy for simulation of cartilage degeneration with simpler material models is similar to models using FPRVE materials if the material parameters are chosen properly.

A voting-based machine learning approach for classifying biological and clinical datasets

BACKGROUND

Different machine learning techniques have been proposed to classify a wide range of biological/clinical data. Given the practicability of these approaches accordingly, various software packages have been also designed and developed. However, the existing methods suffer from several limitations such as overfitting on a specific dataset, ignoring the feature selection concept in the preprocessing step, and losing their performance on large-size datasets. To tackle the mentioned restrictions, in this study, we introduced a machine learning framework consisting of two main steps. First, our previously suggested optimization algorithm (Trader) was extended to select a near-optimal subset of features/genes. Second, a voting-based framework was proposed to classify the biological/clinical data with high accuracy. To evaluate the efficiency of the proposed method, it was applied to 13 biological/clinical datasets, and the outcomes were comprehensively compared with the prior methods.

RESULTS

The results demonstrated that the Trader algorithm could select a near-optimal subset of features with a significant level of p-value < 0.01 relative to the compared algorithms. Additionally, on the large-sie datasets, the proposed machine learning framework improved prior studies by ~ 10% in terms of the mean values associated with fivefold cross-validation of accuracy, precision, recall, specificity, and F-measure.

CONCLUSION

Based on the obtained results, it can be concluded that a proper configuration of efficient algorithms and methods can increase the prediction power of machine learning approaches and help researchers in designing practical diagnosis health care systems and offering effective treatment plans.

An Interpretable Machine Learning Model for Predicting 10-Year Total Hip Arthroplasty Risk

BACKGROUND

As the demand for total hip arthroplasty (THA) rises, a predictive model for THA risk may aid patients and clinicians in augmenting shared decision-making. We aimed to develop and validate a model predicting THA within 10 years in patients using demographic, clinical, and deep learning (DL)-automated radiographic measurements.

METHODS

Patients enrolled in the Osteoarthritis Initiative were included. DL algorithms measuring osteoarthritis- and dysplasia-relevant parameters on baseline pelvis radiographs were developed. Demographic, clinical, and radiographic measurement variables were then used to train generalized additive models to predict THA within 10 years from baseline. A total of 4,796 patients were included (9,592 hips; 58% female; 230 THAs (2.4%)). Model performance using 1) baseline demographic and clinical variables 2) radiographic variables, and 3) all variables were compared.

RESULTS

Using 110 demographic and clinical variables, the model had a baseline area under the receiver operating curve (AUROC) of 0.68 and area under the precision recall curve (AUPRC) of 0.08. Using 26 DL-automated hip measurements, the AUROC was 0.77 and AUPRC was 0.22. Combining all variables, the model improved to an AUROC of 0.81 and AUPRC of 0.28. Three of the top five predictive features in the combined model were radiographic variables including minimum joint space along with hip pain and analgesic use. Partial dependency plots revealed predictive discontinuities for radiographic measurements consistent with literature thresholds of osteoarthritis progression and hip dysplasia.

CONCLUSION

A machine learning model predicting 10-year THA performed more accurately with DL radiographic measurements. The model weighted predictive variables in concordance with clinical THA-pathology assessments.

A joint multi-modal learning method for early-stage knee osteoarthritis disease classification

Osteoarthritis (OA) is a progressive and chronic disease. Identifying the early stages of OA disease is important for the treatment and care of patients. However, most state-of-the-art methods only use single-modal data to predict disease status, so that these methods usually ignore complementary information in multi-modal data. In this study, we develop an integrated multi-modal learning method (MMLM) that uses an interpretable strategy to select and fuse clinical, imaging, and demographic features to classify the grade of early-stage knee OA disease. MMLM applies XGboost and ResNet50 to extract two heterogeneous features from the clinical data and imaging data, respectively. And then we integrate these extracted features with demographic data. To avoid the negative effects of redundant features in a direct integration of multiple features, we propose a L1-norm-based optimization method (MMLM) to regularize the inter-correlations among the multiple features. MMLM was assessed using the Osteoarthritis Initiative (OAI) data set with machine learning classifiers. Extensive experiments demonstrate that MMLM improves the performance of the classifiers. Furthermore, a visual analysis of the important features in the multimodal data verified the relations among the modalities when classifying the grade of knee OA disease.

A review on multimodal machine learning in medical diagnostics

Nowadays, the increasing number of medical diagnostic data and clinical data provide more complementary references for doctors to make diagnosis to patients. For example, with medical data, such as electrocardiography (ECG), machine learning algorithms can be used to identify and diagnose heart disease to reduce the workload of doctors. However, ECG data is always exposed to various kinds of noise and interference in reality, and medical diagnostics only based on one-dimensional ECG data is not trustable enough. By extracting new features from other types of medical data, we can implement enhanced recognition methods, called multimodal learning. Multimodal learning helps models to process data from a range of different sources, eliminate the requirement for training each single learning modality, and improve the robustness of models with the diversity of data. Growing number of articles in recent years have been devoted to investigating how to extract data from different sources and build accurate multimodal machine learning models, or deep learning models for medical diagnostics. This paper reviews and summarizes several recent papers that dealing with multimodal machine learning in disease detection, and identify topics for future research.

IJES-OA Net: A residual neural network to classify knee osteoarthritis from radiographic images based on the edges of the intra-joint spaces

Among the musculoskeletal disorders in the world, osteoarthritis is the most common, affecting most of the body joints, especially the knee. Clinical radiographic imaging methods are commonly used to diagnose osteoarthritis thanks to their cheapness and availability. Due to the low quality and indiscernibility of these images, however, accurate osteoarthritis diagnosis has always faced inaccuracies, such as the wrong diagnosis. One of the osteoarthritis hallmarks is joint space narrowing. Thus, its degree and severity can be determined relatively by assessing the space between the bones in the joint. Therefore, in this research, a deep residual neural network, termed IJES-OA Net, is presented to automatically grade (classify) the severity of knee osteoarthritis via radiographs. This is achieved by tuning it in a way to have it focused on the distance of the edges of the bones inside the knee joint. Experimental results which are conducted on MOST (for training) and OAI (for validation and testing) datasets show that the IJES-OA Net achieves high average accuracy as well as average precision (80.23% and 0.802, respectively) while having less complexity compared to other methods. Additionally, the resulting attention maps from IJES-OA Net are accurate enough that increase experts' reliance on the provided results.

Prediction of progression rate and fate of osteoarthritis: Comparison of machine learning algorithms

Appropriate prediction models can assist healthcare systems in delaying or reversing osteoarthritis (OA) progression. We aimed to identify a reliable algorithm for predicting the progression rate and fate of OA based on patient-specific information. From May 2003 to 2019, 83,280 knees were collected. Age, sex, body mass index, bone mineral density, physical demands for occupation, comorbidities, and initial Kellgren-Lawrence (K-L) grade were used as variables for the prediction models. The prediction targets were divided into dichotomous groups for even distribution. We compared the performances of logistic regression (LR), random forest (RF), and extreme gradient boost (XGB) algorithms. Each algorithm had the best precision when the model used all variables. XGB showed the best results in accuracy, recall, F1 score, specificity, and error rates (progression rate/fate of OA: 0.710/0.877, 0.542/0.637, 0.637/0.758, 0.859/0.981, and 0.290/0.123, respectively). The feature importance of RF and XGB had the same order up to the top six for each prediction target. Age and initial K-L grade had the highest feature importance in RF and XGB for the progression rate and fate of OA, respectively. The XGB and RF machine learning algorithms showed better performance than conventional LR in predicting the progression rate and fate of OA. The best performance was obtained when all variables were combined using the XGB algorithm. For each algorithm, the initial K-L grade and physical demand for occupation were the greatest contributors with superior feature importance compared with the others.

Do Individualized Patient-Specific Situations Predict the Progression Rate and Fate of Knee Osteoarthritis? Prediction of Knee Osteoarthritis

Factors affecting the progression rate and fate of osteoarthritis need to be analyzed when considering patient-specific situation. This study aimed to identify the rate of remarkable progression and fate of primary knee osteoarthritis based on patient-specific situations. Between May 2003 and May 2019, 83,280 patients with knee pain were recruited for this study from the clinical data warehouse. Finally, 2492 knees with pain that were followed up for more than one year were analyzed. For analyzing affecting factors, patient-specific information was categorized and classified as demographic, radiologic, social, comorbidity disorders, and surgical intervention data. The degree of contribution of factors to the progression rate and the fate of osteoarthritis was analyzed. Bone mineral density (BMD), Kellgren-Lawrence (K-L) grade, and physical occupational demands were major contributors to the progression rate of osteoarthritis. Hypertension, initial K-L grade, and physical occupational demands were major contributors to the outcome of osteoarthritis. The progression rate and fate of osteoarthritis were mostly affected by the initial K-L grade and physical occupational demands. Patients who underwent surgical intervention for less than five years had the highest proportion of initial K-L grade 2 (49.0%) and occupations with high physical demand (41.3%). In identifying several contributing factors, the initial K-L grade and physical occupational demands were the most important factors. BMD and hypertension were also major contributors to the progression and fate of osteoarthritis, and the degree of contribution was lower compared to the two major factors.

Test-retest precision and longitudinal cartilage thickness loss in the IMI-APPROACH cohort

OBJECTIVE

To investigate the test-retest precision and to report the longitudinal change in cartilage thickness, the percentage of knees with progression and the predictive value of the machine-learning-estimated structural progression score (s-score) for cartilage thickness loss in the IMI-APPROACH cohort - an exploratory, 5-center, 2-year prospective follow-up cohort.

DESIGN

Quantitative cartilage morphology at baseline and at least one follow-up visit was available for 270 of the 297 IMI-APPROACH participants (78% females, age: 66.4 ± 7.1 years, body mass index (BMI): 28.1 ± 5.3 kg/m², 55% with radiographic knee osteoarthritis (OA)) from 1.5T or 3T MRI. Test-retest precision (root mean square coefficient of variation) was assessed from 34 participants. To define progressor knees, smallest detectable change (SDC) thresholds were computed from 11 participants with longitudinal test-retest scans. Binary logistic regression was used to evaluate the odds of progression in femorotibial cartilage thickness (threshold: -211 μm) for the quartile with the highest vs the quartile with the lowest s-scores.

RESULTS

The test-retest precision was 69 μm for the entire femorotibial joint. Over 24 months, mean cartilage thickness loss in the entire femorotibial joint reached -174 μm (95% CI: [-207, -141] μm, 32.7% with progression). The s-score was not associated with 24-month progression rates by MRI (OR: 1.30, 95% CI: [0.52, 3.28]).

CONCLUSION

IMI-APPROACH successfully enrolled participants with substantial cartilage thickness loss, although the machine-learning-estimated s-score was not observed to be predictive of cartilage thickness loss. IMI-APPROACH data will be used in subsequent analyses to evaluate the impact of clinical, imaging, biomechanical and biochemical biomarkers on cartilage thickness loss and to refine the machine-learning-based s-score.

CLINICALTRIALS

GOV IDENTIFICATION

NCT03883568.

Systematic review of artificial intelligence tack in preventive orthopaedics: is the land coming soon?

PURPOSE

This study aims to describe and assess the current stage of the artificial intelligence (AI) technology integration in preventive orthopaedics of the knee and hip joints.

MATERIALS AND METHODS

The study was conducted in strict compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. Literature databases were searched for articles describing the development and validation of AI models aimed at diagnosing knee or hip joint pathologies or predicting their development or course in patients. The quality of the included articles was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and QUADAS-AI tools.

RESULTS

56 articles were found that meet all the inclusion criteria. We identified two problems that block the full integration of AI into the routine of an orthopaedic physician. The first of them is related to the insufficient amount, variety and quality of data for training, and validation and testing of AI models. The second problem is the rarity of rational evaluation of models, which is why their real quality cannot always be evaluated.

CONCLUSION

The vastness and relevance of the studied topic are beyond doubt. Qualitative and optimally validated models exist in all four scopes considered. Additional optimization and confirmation of the models' quality on various datasets are the last technical stumbling blocks for creating usable software and integrating them into the routine of an orthopaedic physician.

Radiographic Biomarkers for Knee Osteoarthritis: A Narrative Review

Conventional radiography remains the most widely available imaging modality in clinical practice in knee osteoarthritis. Recent research has been carried out to develop novel radiographic biomarkers to establish the diagnosis and to monitor the progression of the disease. The growing number of publications on this topic over time highlights the necessity of a renewed review. Herein, we propose a narrative review of a selection of original full-text articles describing human studies on radiographic imaging biomarkers used for the prediction of knee osteoarthritis-related outcomes. To achieve this, a PubMed database search was used. A total of 24 studies were obtained and then classified based on three outcomes: (1) prediction of radiographic knee osteoarthritis incidence, (2) knee osteoarthritis progression and (3) knee arthroplasty risk. Results showed that numerous studies have reported the relevance of joint space narrowing score, Kellgren-Lawrence score and trabecular bone texture features as potential bioimaging markers in the prediction of the three outcomes. Performance results of reviewed prediction models were presented in terms of the area under the receiver operating characteristic curves. However, fair and valid comparisons of the models' performance were not possible due to the lack of a unique definition of each of the three outcomes.

Osteoarthritis today: Lost in translation?

Osteoarthritis (OA) is a highly prevalent joint disease that is associated with pain, loss of function, and high direct and indirect economic costs. The current therapeutic options are inadequate, providing only a moderate symptom relief without the possibility of disease modification. While treatment options and personalized medicines are increasing for many complex diseases, OA drug development has been impeded by the advanced state of disease at the time of diagnosis and intervention, heterogeneity in both symptoms and rates of progression, and a lack of validated biomarkers and relevant outcome measures. This review article summarizes the OA landscape, including therapies in development as potential OA treatments, potential biomarkers undergoing evaluation by the US Food and Drug Administration, and a summary of current OA treatment guidelines, with a particular focus on the knee OA.

Prediction Models for Knee Osteoarthritis: Review of Current Models and Future Directions

Background

Knee osteoarthritis (OA) is a prevalent joint disease. Clinical prediction models consider a wide range of risk factors for knee OA. This review aimed to evaluate published prediction models for knee OA and identify opportunities for future model development.

Methods

We searched Scopus, PubMed, and Google Scholar using the terms knee osteoarthritis, prediction model, deep learning, and machine learning. All the identified articles were reviewed by one of the researchers and we recorded information on methodological characteristics and findings. We only included articles that were published after 2000 and reported a knee OA incidence or progression prediction model.

Results

We identified 26 models of which 16 employed traditional regression-based models and 10 machine learning (ML) models. Four traditional and five ML models relied on data from the Osteoarthritis Initiative. There was significant variation in the number and type of risk factors. The median sample size for traditional and ML models was 780 and 295, respectively. The reported Area Under the Curve (AUC) ranged between 0.6 and 1.0. Regarding external validation, 6 of the 16 traditional models and only 1 of the 10 ML models validated their results in an external data set.

Conclusion

Diverse use of knee OA risk factors, small, non-representative cohorts, and use of magnetic resonance imaging which is not a routine evaluation tool of knee OA in daily clinical practice are some of the main limitations of current knee OA prediction models.

The KNee OsteoArthritis Prediction (KNOAP2020) challenge: An image analysis challenge to predict incident symptomatic radiographic knee osteoarthritis from MRI and X-ray images

OBJECTIVES

The KNee OsteoArthritis Prediction (KNOAP2020) challenge was organized to objectively compare methods for the prediction of incident symptomatic radiographic knee osteoarthritis within 78 months on a test set with blinded ground truth.

DESIGN

The challenge participants were free to use any available data sources to train their models. A test set of 423 knees from the Prevention of Knee Osteoarthritis in Overweight Females (PROOF) study consisting of magnetic resonance imaging (MRI) and X-ray image data along with clinical risk factors at baseline was made available to all challenge participants. The ground truth outcomes, i.e., which knees developed incident symptomatic radiographic knee osteoarthritis (according to the combined ACR criteria) within 78 months, were not provided to the participants. To assess the performance of the submitted models, we used the area under the receiver operating characteristic curve (ROCAUC) and balanced accuracy (BACC).

RESULTS

Seven teams submitted 23 entries in total. A majority of the algorithms were trained on data from the Osteoarthritis Initiative. The model with the highest ROCAUC (0.64 (95% confidence interval (CI): 0.57-0.70)) used deep learning to extract information from X-ray images combined with clinical variables. The model with the highest BACC (0.59 (95% CI: 0.52-0.65)) ensembled three different models that used automatically extracted X-ray and MRI features along with clinical variables.

CONCLUSION

The KNOAP2020 challenge established a benchmark for predicting incident symptomatic radiographic knee osteoarthritis. Accurate prediction of incident symptomatic radiographic knee osteoarthritis is a complex and still unsolved problem requiring additional investigation.

Multiscale In Silico Modeling of Cartilage Injuries

Injurious loading of the joint can be accompanied by articular cartilage damage and trigger inflammation. However, it is not well-known which mechanism controls further cartilage degradation, ultimately leading to post-traumatic osteoarthritis. For personalized prognostics, there should also be a method that can predict tissue alterations following joint and cartilage injury. This chapter gives an overview of experimental and computational methods to characterize and predict cartilage degradation following joint injury. Two mechanisms for cartilage degradation are proposed. In (1) biomechanically driven cartilage degradation, it is assumed that excessive levels of strain or stress of the fibrillar or non-fibrillar matrix lead to proteoglycan loss or collagen damage and degradation. In (2) biochemically driven cartilage degradation, it is assumed that diffusion of inflammatory cytokines leads to degradation of the extracellular matrix. When implementing these two mechanisms in a computational in silico modeling workflow, supplemented by in vitro and in vivo experiments, it is shown that biomechanically driven cartilage degradation is concentrated on the damage environment, while inflammation via synovial fluid affects all free cartilage surfaces. It is also proposed how the presented in silico modeling methodology may be used in the future for personalized prognostics and treatment planning of patients with a joint injury.

OA-Pain-Sense: Machine Learning Prediction of Hip and Knee Osteoarthritis Pain from IMU Data

Joint pain is a prominent symptom of Hip and Knee Osteoarthritis (OA), impairing patients’ movements and affecting the joint mechanics of walking. Self-report questionnaires are currently the gold standard for Hip OA and Knee OA pain assessment, presenting several problems, including the fact that older individuals often fail to provide accurate self-pain reports. Passive methods to assess pain are desirable. This study aims to explore the feasibility of OA-Pain-Sense, a passive, automatic Machine Learning-based approach that predicts patients’ self-reported pain levels using SpatioTemporal Gait features extracted from the accelerometer signal gathered from an anterior-posterior wearable sensor. To mitigate inter-subject variability, we investigated two types of data rescaling: subject-level and dataset-level. We explored six different binary machine learning classification models for discriminating pain in patients with Hip OA or Knee OA from healthy controls. In rigorous evaluation, OA-Pain-Sense achieved an average accuracy of 86.79% using the Decision Tree and 83.57% using Support Vector Machine classifiers for distinguishing Hip OA and Knee OA patients from healthy subjects, respectively. Our results demonstrate that OA-Pain-Sense is feasible, paving the way for the development of a pain assessment algorithm that can support clinical decision-making and be used on any wearable device, such as smartphones.

Defining and predicting radiographic knee osteoarthritis progression: a systematic review of findings from the osteoarthritis initiative

PURPOSE

The purposes of this systematic review were to (1) identify the commonly used definitions of radiographic KOA progression, (2) summarize the important associative risk factors for disease progression based on findings from the OAI study and (3) summarize findings from radiographic KOA progression prediction modeling studies regarding the characterization of progression and outcomes.

METHODS

A systematic review was performed by conducting a literature search of definitions, risk factors and predictive models for radiographic KOA progression that utilized data from the OAI database. Radiographic progression was further characterized into "accelerated KOA" and "typical progression," as defined by included studies.

RESULTS

Of 314 studies identified, 41 studies were included in the present review. Twenty-eight (28) studies analyzed risk factors associated with KOA progression, and 13 studies created or validated prediction models or risk calculators for progression. Kellgren-Lawrence (KL) grade based on radiographs was most commonly used to characterize KOA progression (50%), followed by joint space width (JSW) narrowing (32%) generally over 48 months. Risk factors with the highest odds ratios (OR) for progression included periarticular bone mineral density (OR 10.40), any knee injury within 1 year (OR 9.22) and baseline bone mineral lesions (OR 7.92). Nine prediction modeling studies utilized both clinical and structural risk factors to inform their models, and combined models outperformed purely clinical or structural models.

CONCLUSION

The cumulative evidence suggests that combinations of structural and clinical risk factors may be able to predict radiographic KOA progression, particularly in patients with accelerated progression. Clinically relevant and feasible prediction models and risk calculators may provide valuable decision-making support when caring for patients at risk of KOA progression, although standardization in modeling and variable identification does not yet exist.

Over-the-Counter Breast Cancer Classification Using Machine Learning and Patient Registration Records

This study aims to determine the feasibility of machine learning (ML) and patient registration record to be utilised to develop an over-the-counter (OTC) screening model for breast cancer risk estimation. Data were retrospectively collected from women who came to the Hospital Universiti Sains Malaysia, Malaysia for breast-related problems. Eight ML models were used: k-nearest neighbour (kNN), elastic-net logistic regression, multivariate adaptive regression splines, artificial neural network, partial least square, random forest, support vector machine (SVM), and extreme gradient boosting. Features utilised for the development of the screening models were limited to information in the patient registration form. The final model was evaluated in terms of performance across a mammographic density. Additionally, the feature importance of the final model was assessed using the model agnostic approach. kNN had the highest Youden J index, precision, and PR-AUC, while SVM had the highest F2 score. The kNN model was selected as the final model. The model had a balanced performance in terms of sensitivity, specificity, and PR-AUC across the mammographic density groups. The most important feature was the age at examination. In conclusion, this study showed that ML and patient registration information are feasible to be used as the OTC screening model for breast cancer.

Multimodal machine learning in precision health: A scoping review

Machine learning is frequently being leveraged to tackle problems in the health sector including utilization for clinical decision-support. Its use has historically been focused on single modal data. Attempts to improve prediction and mimic the multimodal nature of clinical expert decision-making has been met in the biomedical field of machine learning by fusing disparate data. This review was conducted to summarize the current studies in this field and identify topics ripe for future research. We conducted this review in accordance with the PRISMA extension for Scoping Reviews to characterize multi-modal data fusion in health. Search strings were established and used in databases: PubMed, Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 128 articles were included in the analysis. The most common health areas utilizing multi-modal methods were neurology and oncology. Early fusion was the most common data merging strategy. Notably, there was an improvement in predictive performance when using data fusion. Lacking from the papers were clear clinical deployment strategies, FDA-approval, and analysis of how using multimodal approaches from diverse sub-populations may improve biases and healthcare disparities. These findings provide a summary on multimodal data fusion as applied to health diagnosis/prognosis problems. Few papers compared the outputs of a multimodal approach with a unimodal prediction. However, those that did achieved an average increase of 6.4% in predictive accuracy. Multi-modal machine learning, while more robust in its estimations over unimodal methods, has drawbacks in its scalability and the time-consuming nature of information concatenation.