Assessment of knee pain from MR imaging using a convolutional Siamese network

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

This systematic literature review examines state-of-the-art Explainable Artificial Intelligence (XAI) methods applied to medical image analysis, discussing current challenges and future research directions, and exploring evaluation metrics used to assess XAI approaches. With the growing efficiency of Machine Learning (ML) and Deep Learning (DL) in medical applications, there's a critical need for adoption in healthcare. However, their "black-box" nature, where decisions are made without clear explanations, hinders acceptance in clinical settings where decisions have significant medicolegal consequences. Our review highlights the advanced XAI methods, identifying how they address the need for transparency and trust in ML/DL decisions. We also outline the challenges faced by these methods and propose future research directions to improve XAI in healthcare. This paper aims to bridge the gap between cutting-edge computational techniques and their practical application in healthcare, nurturing a more transparent, trustworthy, and effective use of AI in medical settings. The insights guide both research and industry, promoting innovation and standardisation in XAI implementation in healthcare.

Gaining Insight into Updated MR Imaging for Quantitative Assessment of Cartilage Injury in Knee Osteoarthritis

PURPOSE OF THE REVIEW

Knee Osteoarthritis (KOA) entails progressive cartilage degradation, reviewed via MRI for morphology, biochemical composition, and microtissue alterations, discussing clinical advantages, limitations, and research applicability.

RECENT FINDINGS

Compositional MRI, like T2/T2* mapping, T1rho mapping, gagCEST, dGEMRIC, sodium imaging, diffusion-weighted imaging, and diffusion-tensor imaging, provide insights into cartilage injury in KOA. These methods quantitatively measure collagen, glycosaminoglycans, and water content, revealing important information about biochemical compositional and microstructural alterations. Innovative techniques like hybrid multi-dimensional MRI and diffusion-relaxation correlation spectrum imaging show potential in depicting initial cartilage changes at a sub-voxel level. Integration of automated image analysis tools addressed limitations in manual cartilage segmentation, ensuring robust and reproducible assessments of KOA cartilage. Compositional MRI techniques reveal microstructural changes in cartilage. Multi-dimensional MR imaging assesses biochemical alterations in KOA-afflicted cartilage, aiding early degeneration identification. Integrating artificial intelligence enhances cartilage analysis, optimal diagnostic accuracy for early KOA detection and monitoring.

Assessing the Value of Imaging Data in Machine Learning Models to Predict Patient-Reported Outcome Measures in Knee Osteoarthritis Patients

Knee osteoarthritis (OA) affects over 650 million patients worldwide. Total knee replacement is aimed at end-stage OA to relieve symptoms of pain, stiffness and reduced mobility. However, the role of imaging modalities in monitoring symptomatic disease progression remains unclear. This study aimed to compare machine learning (ML) models, with and without imaging features, in predicting the two-year Western Ontario and McMaster Universities Arthritis Index (WOMAC) score for knee OA patients. We included 2408 patients from the Osteoarthritis Initiative (OAI) database, with 629 patients from the Multicenter Osteoarthritis Study (MOST) database. The clinical dataset included 18 clinical features, while the imaging dataset contained an additional 10 imaging features. Minimal Clinically Important Difference (MCID) was set to 24, reflecting meaningful physical impairment. Clinical and imaging dataset models produced similar area under curve (AUC) scores, highlighting low differences in performance AUC < 0.025). For both clinical and imaging datasets, Gradient Boosting Machine (GBM) models performed the best in the external validation, with a clinically acceptable AUC of 0.734 (95% CI 0.687-0.781) and 0.747 (95% CI 0.701-0.792), respectively. The five features identified included educational background, family history of osteoarthritis, co-morbidities, use of osteoporosis medications and previous knee procedures. This is the first study to demonstrate that ML models achieve comparable performance with and without imaging features.

Identifying significant structural factors associated with knee pain severity in patients with osteoarthritis using machine learning

Our main objective was to use machine learning methods to identify significant structural factors associated with pain severity in knee osteoarthritis patients. Additionally, we assessed the potential of various classes of imaging data using machine learning techniques to gauge knee pain severity. The data of semi-quantitative assessments of knee radiographs, semi-quantitative assessments of knee magnetic resonance imaging (MRI), and MRI images from 567 individuals in the Osteoarthritis Initiative (OAI) were utilized to train a series of machine learning models. Models were constructed using five machine learning methods: random forests (RF), support vector machines (SVM), logistic regression (LR), decision tree (DT), and Bayesian (Bayes). Employing tenfold cross-validation, we selected the best-performing models based on the area under the curve (AUC). The study results indicate no significant difference in performance among models using different imaging data. Subsequently, we employed a convolutional neural network (CNN) to extract features from magnetic resonance imaging (MRI), and class activation mapping (CAM) was utilized to generate saliency maps, highlighting regions associated with knee pain severity. A radiologist reviewed the images, identifying specific lesions colocalized with the CAM. The review of 421 knees revealed that effusion/synovitis (30.9%) and cartilage loss (30.6%) were the most frequent abnormalities associated with pain severity. Our study suggests cartilage loss and synovitis/effusion lesions as significant structural factors affecting pain severity in patients with knee osteoarthritis. Furthermore, our study highlights the potential of machine learning for assessing knee pain severity using radiographs.

A semi-supervised multiview-MRI network for the detection of Knee Osteoarthritis

Knee OsteoArthritis (OA) is a prevalent chronic condition, affecting a significant proportion of the global population. Detecting knee OA is crucial as the degeneration of the knee joint is irreversible. In this paper, we introduce a semi-supervised multi-view framework and a 3D CNN model for detecting knee OA using 3D Magnetic Resonance Imaging (MRI) scans. We introduce a semi-supervised learning approach combining labeled and unlabeled data to improve the performance and generalizability of the proposed model. Experimental results show the efficacy of our proposed approach in detecting knee OA from 3D MRI scans using a large cohort of 4297 subjects. An ablation study was conducted to investigate the contributions of various components of the proposed model, providing insights into the optimal design of the model. Our results indicate the potential of the proposed approach to improve the accuracy and efficiency of OA diagnosis. The proposed framework reported an AUC of 93.20% for the detection of knee OA.

Artificial intelligence to analyze magnetic resonance imaging in rheumatology

Rheumatic disorders present a global health challenge, marked by inflammation and damage to joints, bones, and connective tissues. Accurate, timely diagnosis and appropriate management are crucial for favorable patient outcomes. Magnetic resonance imaging (MRI) has become indispensable in rheumatology, but interpretation remains laborious and variable. Artificial intelligence (AI), including machine learning (ML) and deep learning (DL), offers a means to improve and advance MRI analysis. This review examines current AI applications in rheumatology MRI analysis, addressing diagnostic support, disease classification, activity assessment, and progression monitoring. AI demonstrates promise, with high sensitivity, specificity, and accuracy, achieving or surpassing expert performance. The review also discusses clinical implementation challenges and future research directions to enhance rheumatic disease diagnosis and management.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Using deep learning-derived image features in radiologic time series to make personalised predictions: proof of concept in colonic transit data

OBJECTIVES

Siamese neural networks (SNN) were used to classify the presence of radiopaque beads as part of a colonic transit time study (CTS). The SNN output was then used as a feature in a time series model to predict progression through a CTS.

METHODS

This retrospective study included all patients undergoing a CTS in a single institution from 2010 to 2020. Data were partitioned in an 80/20 Train/Test split. Deep learning models based on a SNN architecture were trained and tested to classify images according to the presence, absence, and number of radiopaque beads and to output the Euclidean distance between the feature representations of the input images. Time series models were used to predict the total duration of the study.

RESULTS

In total, 568 images of 229 patients (143, 62% female, mean age 57) patients were included. For the classification of the presence of beads, the best performing model (Siamese DenseNET trained with a contrastive loss with unfrozen weights) achieved an accuracy, precision, and recall of 0.988, 0.986, and 1. A Gaussian process regressor (GPR) trained on the outputs of the SNN outperformed both GPR using only the number of beads and basic statistical exponential curve fitting with MAE of 0.9 days compared to 2.3 and 6.3 days (p < 0.05) respectively.

CONCLUSIONS

SNNs perform well at the identification of radiopaque beads in CTS. For time series prediction our methods were superior at identifying progression through the time series compared to statistical models, enabling more accurate personalised predictions.

CLINICAL RELEVANCE STATEMENT

Our radiologic time series model has potential clinical application in use cases where change assessment is critical (e.g. nodule surveillance, cancer treatment response, and screening programmes) by quantifying change and using it to make more personalised predictions.

KEY POINTS

• Time series methods have improved but application to radiology lags behind computer vision. Colonic transit studies are a simple radiologic time series measuring function through serial radiographs. • We successfully employed a Siamese neural network (SNN) to compare between radiographs at different points in time and then used the output of SNN as a feature in a Gaussian process regression model to predict progression through the time series. • This novel use of features derived from a neural network on medical imaging data to predict progression has potential clinical application in more complex use cases where change assessment is critical such as in oncologic imaging, monitoring for treatment response, and screening programmes.

Deep learning applications in osteoarthritis imaging

Deep learning (DL) is one of the most exciting new areas in medical imaging. This article will provide a review of current applications of DL in osteoarthritis (OA) imaging, including methods used for cartilage lesion detection, OA diagnosis, cartilage segmentation, and OA risk assessment. DL techniques have been shown to have similar diagnostic performance as human readers for detecting and grading cartilage lesions within the knee on MRI. A variety of DL methods have been developed for detecting and grading the severity of knee OA and various features of knee OA on X-rays using standardized classification systems with diagnostic performance similar to human readers. Multiple DL approaches have been described for fully automated segmentation of cartilage and other knee tissues and have achieved higher segmentation accuracy than currently used methods with substantial reductions in segmentation times. Various DL models analyzing baseline X-rays and MRI have been developed for OA risk assessment. These models have shown high diagnostic performance for predicting a wide variety of OA outcomes, including the incidence and progression of radiographic knee OA, the presence and progression of knee pain, and future total knee replacement. The preliminary results of DL applications in OA imaging have been encouraging. However, many DL techniques require further technical refinement to maximize diagnostic performance. Furthermore, the generalizability of DL approaches needs to be further investigated in prospective studies using large image datasets acquired at different institutions with different imaging hardware before they can be implemented in clinical practice and research studies.

Automated detection of genetic relatedness from fundus photographs using Siamese Neural Networks

Heritability of common eye diseases and ocular traits are relatively high. Here, we develop an automated algorithm to detect genetic relatedness from color fundus photographs (FPs). We estimated the degree of shared ancestry amongst individuals in the UK Biobank using KING software. A convolutional Siamese neural network-based algorithm was trained to output a measure of genetic relatedness using 7224 pairs (3612 related and 3612 unrelated) of FPs. The model achieved high performance for prediction of genetic relatedness; when computed Euclidean distances were used to determine probability of relatedness, the area under the receiver operating characteristic curve (AUROC) for identifying related FPs reached 0.926. We performed external validation of our model using FPs from the LIFE-Adult study and achieved an AUROC of 0.69. An occlusion map indicates that the optic nerve and its surrounding area may be the most predictive of genetic relatedness. We demonstrate that genetic relatedness can be captured from FP features. This approach may be used to uncover novel biomarkers for common ocular diseases.

Medical imaging in rheumatoid arthritis: A review on deep learning approach

Arthritis is a musculoskeletal disorder. Millions of people have arthritis, making it one of the most common joint disorders. Osteoarthritis (OA) and rheumatoid arthritis (RA) are the most common types of arthritis among the many different types available. Pain, stiffness, and inflammation are among the early signs of arthritis, which can progress to severe immobility at a later stage if left untreated. Although arthritis cannot be cured at any point in time, it can be managed if diagnosed and treated correctly. Clinical diagnostic and medical imaging methods are currently used to evaluate OA and RA, both debilitating conditions. This review is focused on deep learning approaches used by taking medical imaging (X-rays and magnetic resonance imaging) as input for the detection of RA.

Artificial Intelligence for Automatic Pain Assessment: Research Methods and Perspectives

Although proper pain evaluation is mandatory for establishing the appropriate therapy, self-reported pain level assessment has several limitations. Data-driven artificial intelligence (AI) methods can be employed for research on automatic pain assessment (APA). The goal is the development of objective, standardized, and generalizable instruments useful for pain assessment in different clinical contexts. The purpose of this article is to discuss the state of the art of research and perspectives on APA applications in both research and clinical scenarios. Principles of AI functioning will be addressed. For narrative purposes, AI-based methods are grouped into behavioral-based approaches and neurophysiology-based pain detection methods. Since pain is generally accompanied by spontaneous facial behaviors, several approaches for APA are based on image classification and feature extraction. Language features through natural language strategies, body postures, and respiratory-derived elements are other investigated behavioral-based approaches. Neurophysiology-based pain detection is obtained through electroencephalography, electromyography, electrodermal activity, and other biosignals. Recent approaches involve multimode strategies by combining behaviors with neurophysiological findings. Concerning methods, early studies were conducted by machine learning algorithms such as support vector machine, decision tree, and random forest classifiers. More recently, artificial neural networks such as convolutional and recurrent neural network algorithms are implemented, even in combination. Collaboration programs involving clinicians and computer scientists must be aimed at structuring and processing robust datasets that can be used in various settings, from acute to different chronic pain conditions. Finally, it is crucial to apply the concepts of explainability and ethics when examining AI applications for pain research and management.

Prediction of total knee replacement using deep learning analysis of knee MRI

Current methods for assessing knee osteoarthritis (OA) do not provide comprehensive information to make robust and accurate outcome predictions. Deep learning (DL) risk assessment models were developed to predict the progression of knee OA to total knee replacement (TKR) over a 108-month follow-up period using baseline knee MRI. Participants of our retrospective study consisted of 353 case-control pairs of subjects from the Osteoarthritis Initiative with and without TKR over a 108-month follow-up period matched according to age, sex, ethnicity, and body mass index. A traditional risk assessment model was created to predict TKR using baseline clinical risk factors. DL models were created to predict TKR using baseline knee radiographs and MRI. All DL models had significantly higher (p < 0.001) AUCs than the traditional model. The MRI and radiograph ensemble model and MRI ensemble model (where TKR risk predicted by several contrast-specific DL models were averaged to get the ensemble TKR risk prediction) had the highest AUCs of 0.90 (80% sensitivity and 85% specificity) and 0.89 (79% sensitivity and 86% specificity), respectively, which were significantly higher (p < 0.05) than the AUCs of the radiograph and multiple MRI models (where the DL models were trained to predict TKR risk using single contrast or 2 contrasts together as input). DL models using baseline MRI had a higher diagnostic performance for predicting TKR than a traditional model using baseline clinical risk factors and a DL model using baseline knee radiographs.

Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

Advances in artificial intelligence (AI), especially deep learning (DL), have facilitated magnetic resonance imaging (MRI) data analysis, enabling AI-assisted medical image diagnoses and prognoses. However, most of the DL models are considered as "black boxes". There is an unmet need to demystify DL models so domain experts can trust these high-performance DL models. This has resulted in a sub-domain of AI research called explainable artificial intelligence (XAI). In the last decade, many experts have dedicated their efforts to developing novel XAI methods that are competent at visualizing and explaining the logic behind data-driven DL models. However, XAI techniques are still in their infancy for medical MRI image analysis. This study aims to outline the XAI applications that are able to interpret DL models for MRI data analysis. We first introduce several common MRI data modalities. Then, a brief history of DL models is discussed. Next, we highlight XAI frameworks and elaborate on the principles of multiple popular XAI methods. Moreover, studies on XAI applications in MRI image analysis are reviewed across the tissues/organs of the human body. A quantitative analysis is conducted to reveal the insights of MRI researchers on these XAI techniques. Finally, evaluations of XAI methods are discussed. This survey presents recent advances in the XAI domain for explaining the DL models that have been utilized in MRI applications.

Highly Performing Automatic Detection of Structural Chromosomal Abnormalities Using Siamese Architecture

The detection of structural chromosomal abnormalities (SCA) is crucial for diagnosis, prognosis and management of many genetic diseases and cancers. This detection, done by highly qualified medical experts, is tedious and time-consuming. We propose a highly performing and intelligent method to assist cytogeneticists to screen for SCA. Each chromosome is present in two copies that make up a pair of chromosomes. Usually, SCA are present in only one copy of the pair. Convolutional neural networks (CNN) with Siamese architecture are particularly relevant for evaluating similarities between two images, which is why we used this method to detect abnormalities between both chromosomes of a given pair. As a proof-of-concept, we first focused on a deletion occurring on chromosome 5 (del(5q)) observed in hematological malignancies. Using our dataset, we conducted several experiments without and with data augmentation on seven popular CNN models. Overall, performances obtained were very relevant for detecting deletions, particularly with Xception and InceptionResNetV2 models achieving 97.50% and 97.01% of F₁-score, respectively. We additionally demonstrated that these models successfully recognized another SCA, inversion inv(3), which is one of the most difficult SCA to detect. The performance improved when the training was applied on inversion inv(3) dataset, achieving 94.82% of F₁-score. The technique that we propose in this paper is the first highly performing method based on Siamese architecture that allows the detection of SCA. Our code is publicly available at: https://github.com/MEABECHAR/ChromosomeSiameseAD.

Prediction of knee pain improvement over two years for knee osteoarthritis using a dynamic nomogram based on MRI-derived radiomics: a proof-of-concept study

OBJECTIVES

To develop and validate a nomogram to detect improved knee pain in osteoarthritis (OA) by integrating magnetic resonance imaging (MRI) radiomics signature of subchondral bone and clinical characteristics.

METHODS

Participants were selected from the Vitamin D Effects on Osteoarthritis (VIDEO) study. The primary outcome was 20% improvement of knee pain score over 2 years in participants administrated either vitamin D or placebo. Radiomics features of subchondral bone and clinical characteristics from 216 participants were extracted and analyzed. The participants were randomly split into the training and validation cohorts at a ratio of 8:2. Least absolute shrinkage and selection operator (LASSO) regression was used to select features and generate radiomics signatures. The optimal radiomics signature and clinical indicators were fitted into a nomogram using multivariable logistic regression model.

RESULTS

The nomogram showed favorable discrimination performance [AUC_training, 0.79 (95% CI: 0.72-0.79), AUC_validation, 0.83 (95% CI: 0.70-0.96)] as well as a good calibration. Additional contributing value of fusion radiomics signature to the nomogram was statistically significant (NRI, 0.23; IDI, 0.14, P < 0.001 in training cohort and NRI, 0.29; IDI, 0.18, P < 0.05 in validating cohort). Decision curve analysis confirmed the clinical usefulness of nomogram.

CONCLUSION

The radiomics-based nomogram comprising the MR radiomics signature and clinical variables achieves a favorable predictive efficacy and accuracy in differentiating improvement in knee pain among OA patients. This proof-of-concept study provides a promising way to predict clinically meaningful outcomes.

Transfer learning-assisted 3D deep learning models for knee osteoarthritis detection: Data from the osteoarthritis initiative

Knee osteoarthritis is one of the most common musculoskeletal diseases and is usually diagnosed with medical imaging techniques. Conventionally, case identification using plain radiography is practiced. However, we acknowledge that knee osteoarthritis is a 3D complexity; hence, magnetic resonance imaging will be the ideal modality to reveal the hidden osteoarthritis features from a three-dimensional view. In this work, the feasibility of well-known convolutional neural network (CNN) structures (ResNet, DenseNet, VGG, and AlexNet) to distinguish knees with and without osteoarthritis (OA) is investigated. Using 3D convolutional layers, we demonstrated the potential of 3D convolutional neural networks of 13 different architectures in knee osteoarthritis diagnosis. We used transfer learning by transforming 2D pre-trained weights into 3D as initial weights for the training of the 3D models. The performance of the models was compared and evaluated based on the performance metrics [balanced accuracy, precision, F1 score, and area under receiver operating characteristic (AUC) curve]. This study suggested that transfer learning indeed enhanced the performance of the models, especially for ResNet and DenseNet models. Transfer learning-based models presented promising results, with ResNet34 achieving the best overall accuracy of 0.875 and an F1 score of 0.871. The results also showed that shallow networks yielded better performance than deeper neural networks, demonstrated by ResNet18, DenseNet121, and VGG11 with AUC values of 0.945, 0.914, and 0.928, respectively. This encourages the application of clinical diagnostic aid for knee osteoarthritis using 3DCNN even in limited hardware conditions.

RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning

Multimodal ultrasound has demonstrated its power in the clinical assessment of rheumatoid arthritis (RA). However, for radiologists, it requires strong experience. In this paper, we propose a rheumatoid arthritis knowledge guided (RATING) system that automatically scores the RA activity and generates interpretable features to assist radiologists' decision-making based on deep learning. RATING leverages the complementary advantages of multimodal ultrasound images and solves the limited training data problem with self-supervised pretraining. RATING outperforms all of the existing methods, achieving an accuracy of 86.1% on a prospective test dataset and 85.0% on an external test dataset. A reader study demonstrates that the RATING system improves the average accuracy of 10 radiologists from 41.4% to 64.0%. As an assistive tool, not only can RATING indicate the possible lesions and enhance the diagnostic performance with multimodal ultrasound but it can also enlighten the road to human-machine collaboration in healthcare.

•

RATING is a medical knowledge-guided deep learning system for RA assessment

•

It leverages diagnostic paradigm and experience to enhance the robustness

•

Self-supervised pretraining ensures reliability even with limited training data

•

A clinical reader study demonstrates its effectiveness in assisting RA assessment

Rheumatoid arthritis (RA) has detrimental outcomes, including increased disability and mortality. To enhance the clinical assessment of RA, we propose a rheumatoid arthritis knowledge guided (RATING) system for scoring RA activity from multimodal ultrasound images. It combines the knowledge of clinical diagnosis with deep learning, serving as an example of designing deep learning systems for handling real clinical problems. We further integrated the system into the clinical decision-making process via human-machine collaboration and demonstrated significant improvements in assessment performance. We expect that our research will illuminate the road to human-machine collaboration and help transform clinical diagnostics and precision medicine in a wider range of biomedical research.

A Comprehensive Survey on Bone Segmentation Techniques in Knee Osteoarthritis Research: From Conventional Methods to Deep Learning

Knee osteoarthritis (KOA) is a degenerative joint disease, which significantly affects middle-aged and elderly people. The majority of KOA is primarily based on hyaline cartilage change, according to medical images. However, technical bottlenecks such as noise, artifacts, and modality pose enormous challenges for an objective and efficient early diagnosis. Therefore, the correct prediction of arthritis is an essential step for effective diagnosis and the prevention of acute arthritis, where early diagnosis and treatment can assist to reduce the progression of KOA. However, predicting the development of KOA is a difficult and urgent problem that, if addressed, could accelerate the development of disease-modifying drugs, in turn helping to avoid millions of total joint replacement procedures each year. In knee joint research and clinical practice there are segmentation approaches that play a significant role in KOA diagnosis and categorization. In this paper, we seek to give an in-depth understanding of a wide range of the most recent methodologies for knee articular bone segmentation; segmentation methods allow the estimation of articular cartilage loss rate, which is utilized in clinical practice for assessing the disease progression and morphological change, ranging from traditional techniques to deep learning (DL)-based techniques. Moreover, the purpose of this work is to give researchers a general review of the currently available methodologies in the area. Therefore, it will help researchers who want to conduct research in the field of KOA, as well as highlight deficiencies and potential considerations in application in clinical practice. Finally, we highlight the diagnostic value of deep learning for future computer-aided diagnostic applications to complete this review.

Deep learning approach to predict pain progression in knee osteoarthritis

OBJECTIVE

To develop and evaluate deep learning (DL) risk assessment models for predicting pain progression in subjects with or at risk of knee osteoarthritis (OA).

MATERIALS AND METHODS

The incidence and progression cohorts of the Osteoarthritis Initiative, a multi-center longitudinal study involving 9348 knees in 4674 subjects with or at risk of knee OA that began in 2004 and is ongoing, were used to conduct this retrospective analysis. A subset of knees without and with pain progression (defined as a 9-point or greater increase in pain score between baseline and two or more follow-up time points over the first 48 months) was randomly stratified into training (4200 knees with a mean age of 61.0 years and 60% female) and hold-out testing (500 knees with a mean age of 60.8 years and 60% female) datasets. A DL model was developed to predict pain progression using baseline knee radiographs. An artificial neural network was used to develop a traditional risk assessment model to predict pain progression using demographic, clinical, and radiographic risk factors. A combined model was developed to combine demographic, clinical, and radiographic risk factors with DL analysis of baseline knee radiographs. Area under the curve (AUC) analysis was performed using the hold-out testing dataset to evaluate model performance.

RESULTS

The traditional model had an AUC of 0.692 (66.9% sensitivity and 64.1% specificity). The DL model had an AUC of 0.770 (76.7% sensitivity and 70.5% specificity), which was significantly higher (p < 0.001) than the traditional model. The combined model had an AUC of 0.807 (72.3% sensitivity and 80.9% specificity), which was significantly higher (p < 0.05) than the traditional and DL models.

CONCLUSIONS

DL models using baseline knee radiographs had higher diagnostic performance for predicting pain progression than traditional models using demographic, clinical, and radiographic risk factors.

Osteoarthritis year in review 2021: imaging

PURPOSE

To provide a narrative review of original articles on imaging of osteoarthritis (OA) published between January 1, 2020 and March 31, 2021, with a special focus on imaging of inflammation, imaging of bone, cartilage and bone-cartilage interactions, imaging of peri-articular tissues, imaging scoring methods for OA, and artificial intelligence (AI) applied to OA imaging.

METHODS

The Embase, Pubmed, Medline, Cochrane databases were searched for original research articles in the English language on human, in vivo, imaging of OA published between January 1, 2020 and March 31, 2021. Search terms related to osteoarthritis combined with all imaging modalities and artificial intelligence were applied. A selection of articles reporting on one of the focus topics was discussed further.

RESULTS

The search resulted in 651 articles, of which 214 were deemed relevant to human OA imaging. Among the articles included, the knee joint (69%) and magnetic resonance imaging (MRI) (52%) were the predominant anatomical area and imaging modality studied. There were also a substantial number of papers (n = 46) reporting on AI applications in the field of OA imaging.

CONCLUSION

Imaging continues to play an important role in the assessment of OA. Recent advances in OA imaging include quantitative, non-contrast, and hybrid imaging techniques for improved characterization of multiple tissue processes in OA. In addition, an increasing effort in AI techniques is undertaken to enhance OA imaging acquisition and analysis.

Imaging Manifestations and Evaluation of Postoperative Complications of Bone and Joint Infections under Deep Learning

To explore and evaluate the imaging manifestations of postoperative complications of bone and joint infections based on deep learning, a retrospective study was performed on 40 patients with bone and joint infections in the Department of Orthopedics of Orthopedics Hospital of Henan Province of Luoyang City. Sensitivity and Dice similarity coefficient (DSC) were used to evaluate the image results by convolutional neural network (CNN) algorithm. Imaging features of postoperative complications in 40 patients were analyzed. Then, three imaging methods were used to diagnose the features. Sensitivity and specificity were used to evaluate the diagnostic performance of three imaging methods for imaging features. Compared with professional doctors and biomarker algorithms, the sensitivity of CNN algorithm proposed was 90.6%, and DSC was 84.1%. Compared with traditional methods, the CNN algorithm has higher image resolution and wider and more accurate lesion area recognition and division. The three manifestations of soft tissue abscess, periosteum swelling, and bone damage were postoperative imaging features of bone and joint infections. In addition, compared with X-ray, CT examination and MRI examination were better for the examination of imaging characteristics. CT and MRI had higher sensitivity and specificity than X-ray. The experimental results show that CNN algorithm can effectively identify and divide pathological images and assist doctors to diagnose the images more efficiently in clinic.

Emergence of Deep Learning in Knee Osteoarthritis Diagnosis

Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.

Automatic Assessment Of Hip Effusion From MRI

Joint effusion is a hallmark of osteoarthritis (OA) associated with stiffness, and may relate to pain, disability, and long-term outcomes. However, it is difficult to quantify accurately. We propose a new Deep Learning (DL) approach for automatic effusion assessment from Magnetic Resonance Imaging (MRI) using volumetric quantification measures (VQM). We developed a new multiplane ensemble convolutional neural network (CNN) approach for 1) localizing bony anatomy and 2) detecting effusion regions. CNNs were trained on femoral head and effusion regions manually segmented from 3856 images (63 patients). Upon validation on a non-overlapping set of 2040 images (34 patients) DL showed high agreement with ground-truth in terms of Dice score (0.85), sensitivity (0.86) and precision (0.83). Agreement of VQM per-patient was high for DL vs experts in term of Intraclass correlation coefficient (ICC)= 0.88[0.80,0.93]. We expect this technique to reduce inter-observer variability in effusion assessment, reducing expert time and potentially improving the quality of OA care.Clinical Relevance- Our technique for automatic assessment of hip MRI can be used for volumetric measurement of effusion. We expect this to reduce variability in OA biomarker assessment and provide more reliable indicators for disease progression.

3D MRI in Osteoarthritis

Osteoarthritis (OA) is among the top 10 burdensome diseases, with the knee the most affected joint. Magnetic resonance imaging (MRI) allows whole-knee assessment, making it ideally suited for imaging OA, considered a multitissue disease. Three-dimensional (3D) MRI enables the comprehensive assessment of OA, including quantitative morphometry of various joint tissues. Manual tissue segmentation on 3D MRI is challenging but may be overcome by advanced automated image analysis methods including artificial intelligence (AI). This review presents examples of the utility of 3D MRI for knee OA, focusing on the articular cartilage, bone, meniscus, synovium, and infrapatellar fat pad, and it highlights several applications of AI that facilitate segmentation, lesion detection, and disease classification.

Deep-Learning-Driven Quantification of Interstitial Fibrosis in Digitized Kidney Biopsies

Interstitial fibrosis and tubular atrophy (IFTA) on a renal biopsy are strong indicators of disease chronicity and prognosis. Techniques that are typically used for IFTA grading remain manual, leading to variability among pathologists. Accurate IFTA estimation using computational techniques can reduce this variability and provide quantitative assessment. Using trichrome-stained whole-slide images (WSIs) processed from human renal biopsies, we developed a deep-learning framework that captured finer pathologic structures at high resolution and overall context at the WSI level to predict IFTA grade. WSIs (n = 67) were obtained from The Ohio State University Wexner Medical Center. Five nephropathologists independently reviewed them and provided fibrosis scores that were converted to IFTA grades: ≤10% (none or minimal), 11% to 25% (mild), 26% to 50% (moderate), and >50% (severe). The model was developed by associating the WSIs with the IFTA grade determined by majority voting (reference estimate). Model performance was evaluated on WSIs (n = 28) obtained from the Kidney Precision Medicine Project. There was good agreement on the IFTA grading between the pathologists and the reference estimate (κ = 0.622 ± 0.071). The accuracy of the deep-learning model was 71.8% ± 5.3% on The Ohio State University Wexner Medical Center and 65.0% ± 4.2% on Kidney Precision Medicine Project data sets. Our approach to analyzing microscopic- and WSI-level changes in renal biopsies attempts to mimic the pathologist and provides a regional and contextual estimation of IFTA. Such methods can assist clinicopathologic diagnosis.

Deep learning for semi-automated unidirectional measurement of lung tumor size in CT

BACKGROUND

Performing Response Evaluation Criteria in Solid Tumor (RECISTS) measurement is a non-trivial task requiring much expertise and time. A deep learning-based algorithm has the potential to assist with rapid and consistent lesion measurement.

PURPOSE

The aim of this study is to develop and evaluate deep learning (DL) algorithm for semi-automated unidirectional CT measurement of lung lesions.

METHODS

This retrospective study included 1617 lung CT images from 8 publicly open datasets. A convolutional neural network was trained using 1373 training and validation images annotated by two radiologists. Performance of the DL algorithm was evaluated 244 test images annotated by one radiologist. DL algorithm's measurement consistency with human radiologist was evaluated using Intraclass Correlation Coefficient (ICC) and Bland-Altman plotting. Bonferroni's method was used to analyze difference in their diagnostic behavior, attributed by tumor characteristics. Statistical significance was set at p < 0.05.

RESULTS

The DL algorithm yielded ICC score of 0.959 with human radiologist. Bland-Altman plotting suggested 240 (98.4 %) measurements realized within the upper and lower limits of agreement (LOA). Some measurements outside the LOA revealed difference in clinical reasoning between DL algorithm and human radiologist. Overall, the algorithm marginally overestimated the size of lesion by 2.97 % compared to human radiologists. Further investigation indicated tumor characteristics may be associated with the DL algorithm's diagnostic behavior of over or underestimating the lesion size compared to human radiologist.

CONCLUSIONS

The DL algorithm for unidirectional measurement of lung tumor size demonstrated excellent agreement with human radiologist.

Application of Magnetic Resonance-Ultra Time Echo (MR-UTE) Imaging in the Analysis of the Degree of Degeneration of the Intervertebral Disc Cartilage Endplate

Magnetic resonance imaging (MRI) is the most widely used imaging method in clinical lumbar spine examination. Because of its advantages of non-radiation and good tissue contrast, magnetic resonance imaging provides rich and effective diagnostic information for clinic. The most commonly used sequence is type 2 (T2) sequence, which has a longer time (usually longer than 2000 ms). It shows well in long T2 tissues such as nucleus pulposus, cerebrospinal fluid and adipose tissue, showing moderator high signal in images, while for short T2 tissues such as cartilage endplate and anterior and posterior longitudinal zone, it is often no signal and low signal because of its short attenuation time, thus forming obvious tissue contrast. But at the same time, because the time is too long, for short T2 tissue, the signal has been attenuated to zero before sequence acquisition, so the complete structure can not be displayed directly. In this paper, the normal human lumbar intervertebral disc was studied by conventional magnetic resonance type 1 (T1), T2 and double-echo-UTE imaging techniques. Each part of lumbar intervertebral disc and the semi-quantitative analysis of anatomical structure in images were compared, and the advantages and characteristics of each sequence for each anatomical structure of lumbar intervertebral disc and the advantage of MR-UTE in intervertebral disc display were discussed. It has been found that UTE, as a new sequence which can effectively image short T2 tissue, is gradually applied from experiment to clinic in bone and joint system because of its shorter time. In the gross specimens of lumbar intervertebral disc, sequence can directly display the cartilage endplate and the short T2 tissue of the anterior and posterior longitudinal ligament.

Decision Support Systems in Temporomandibular Joint Osteoarthritis: A review of Data Science and Artificial Intelligence Applications

With the exponential growth of computational systems and increased patient data acquisition, dental research faces new challenges to manage a large quantity of information. For this reason, data science approaches are needed for the integrative diagnosis of multifactorial diseases, such as Temporomandibular joint (TMJ) Osteoarthritis (OA). The Data science spectrum includes data capture/acquisition, data processing with optimized web-based storage and management, data analytics involving in-depth statistical analysis, machine learning (ML) approaches, and data communication. Artificial intelligence (AI) plays a crucial role in this process. It consists of developing computational systems that can perform human intelligence tasks, such as disease diagnosis, using many features to help in the decision-making support. Patient's clinical parameters, imaging exams, and molecular data are used as the input in cross-validation tasks, and human annotation/diagnosis is also used as the gold standard to train computational learning models and automatic disease classifiers. This paper aims to review and describe AI and ML techniques to diagnose TMJ OA and data science approaches for imaging processing. We used a web-based system for multi-center data communication, algorithms integration, statistics deployment, and process the computational machine learning models. We successfully show AI and data-science applications using patients' data to improve the TMJ OA diagnosis decision-making towards personalized medicine.

Subchondral bone length in knee osteoarthritis: A deep learning derived imaging measure and its association with radiographic and clinical outcomes

OBJECTIVE

Develop a bone shape measure that reflects the extent of cartilage loss and bone flattening in knee osteoarthritis (OA) and test it against estimates of disease severity.

METHODS

A fast region-based convolutional neural network was trained to crop the knee joints in sagittal dual-echo steady state MRI sequences obtained from the Osteoarthritis Initiative (OAI). Publicly available annotations of the cartilage and menisci were used as references to annotate the tibia and the femur in 61 knees. Another deep neural network (U-Net) was developed to learn these annotations. Model predictions were compared with radiologist-driven annotations on an independent test set (27 knees). The U-Net was applied to automatically extract the knee joint structures on the larger OAI dataset (9,434 knees). We defined subchondral bone length (SBL), a novel shape measure characterizing the extent of overlying cartilage and bone flattening, and examined its relationship with radiographic joint space narrowing (JSN), concurrent WOMAC pain and disability as well as subsequent partial or total knee replacement (KR). Odds ratios for each outcome were estimated using relative changes in SBL on the OAI dataset into quartiles.

RESULT

Mean SBL values for knees with JSN were consistently different from knees without JSN. Greater changes of SBL from baseline were associated with greater pain and disability. For knees with medial or lateral JSN, the odds ratios between lowest and highest quartiles corresponding to SBL changes for future KR were 5.68 (95% CI:[3.90,8.27]) and 7.19 (95% CI:[3.71,13.95]), respectively.

CONCLUSION

SBL quantified OA status based on JSN severity. It has promise as an imaging marker in predicting clinical and structural OA outcomes.

Osteoarthritis year in review 2020: imaging

This narrative "Year in Review" highlights a selection of articles published between January 2019 and April 2020, to be presented at the OARSI World Congress 2020 within the field of osteoarthritis (OA) imaging. Articles were obtained from a PubMed search covering the above period, utilizing a variety of relevant search terms. We then selected original and review studies on OA-related imaging in humans, particularly those with direct clinical relevance, with a focus on the knee. Topics selected encompassed clinically relevant models of early OA, particularly imaging applications on cruciate ligament rupture, as these are of direct clinical interest and provide potential opportunity to evaluate preventive therapy. Further, imaging applications on structural modification of articular tissues in patients with established OA, by non-pharmacological, pharmacological and surgical interventions are summarized. Finally, novel deep learning approaches to imaging are reviewed, as these facilitate implementation and scaling of quantitative imaging application in clinical trials and clinical practice. Methodological or observational studies outside these key focus areas were not included. Studies focused on biology, biomechanics, biomarkers, genetics and epigenetics, and clinical studies that did not contain an imaging component are covered in other articles within the OARSI "Year in Review" series. In conclusion, exciting progress has been made in clinically validating human models of early OA, and the field of automated articular tissue segmentation. Most importantly though, it has been shown that structure modification of articular cartilage is possible, and future research should focus on the translation of these structural findings to clinical benefit.