A convolutional neural network with transfer learning for automatic discrimination between low and high-grade synovitis: a pilot study

AI am a rheumatologist: a practical primer to large language models for rheumatologists

Natural language processing (NLP), a subclass of artificial intelligence, large language models (LLMs), and its latest applications, such as Generative Pre-trained Transformers (GPT), ChatGPT, or LLAMA, have recently become one of the most discussed topics. Up to now, artificial intelligence and NLP ultimately impacted several areas, such as finance, economics and diagnostic/scoring systems in healthcare. Another area that artificial intelligence has affected and will continue to affect increasingly is academic life. This narrative review will define NLP, LLMs and their applications, discuss the opportunities and challenges that components of academic society will experience in rheumatology, and discuss the impact of NLP and LLMs in rheumatology healthcare.

Understanding the role and adoption of artificial intelligence techniques in rheumatology research: An in-depth review of the literature

The major and upward trend in the number of published research related to rheumatic and musculoskeletal diseases, in which artificial intelligence plays a key role, has exhibited the interest of rheumatology researchers in using these techniques to answer their research questions. In this review, we analyse the original research articles that combine both worlds in a five- year period (2017-2021). In contrast to other published papers on the same topic, we first studied the review and recommendation articles that were published during that period, including up to October 2022, as well as the publication trends. Secondly, we review the published research articles and classify them into one of the following categories: disease identification and prediction, disease classification, patient stratification and disease subtype identification, disease progression and activity, treatment response, and predictors of outcomes. Thirdly, we provide a table with illustrative studies in which artificial intelligence techniques have played a central role in more than twenty rheumatic and musculoskeletal diseases. Finally, the findings of the research articles, in terms of disease and/or data science techniques employed, are highlighted in a discussion. Therefore, the present review aims to characterise how researchers are applying data science techniques in the rheumatology medical field. The most immediate conclusions that can be drawn from this work are: multiple and novel data science techniques have been used in a wide range of rheumatic and musculoskeletal diseases including rare diseases; the sample size and the data type used are heterogeneous, and new technical approaches are expected to arrive in the short-middle term.

Deep Learning Algorithm Detects Presence of Disorganization of Retinal Inner Layers (DRIL)-An Early Imaging Biomarker in Diabetic Retinopathy

Purpose

To develop and train a deep learning-based algorithm for detecting disorganization of retinal inner layers (DRIL) on optical coherence tomography (OCT) to screen a cohort of patients with diabetic retinopathy (DR).

Methods

In this cross-sectional study, subjects over age 18, with ICD-9/10 diagnoses of type 2 diabetes with and without retinopathy and Cirrus HD-OCT imaging performed between January 2009 to September 2019 were included in this study. After inclusion and exclusion criteria were applied, a final total of 664 patients (5992 B-scans from 1201 eyes) were included for analysis. Five-line horizontal raster scans from Cirrus HD-OCT were obtained from the shared electronic health record. Two trained graders evaluated scans for presence of DRIL. A third physician grader arbitrated any disagreements. Of 5992 B-scans analyzed, 1397 scans (∼30%) demonstrated presence of DRIL. Graded scans were used to label training data for the convolution neural network (CNN) development and training.

Results

On a single CPU system, the best performing CNN training took ∼35 mins. Labeled data were divided 90:10 for internal training/validation and external testing purpose. With this training, our deep learning network was able to predict the presence of DRIL in new OCT scans with a high accuracy of 88.3%, specificity of 90.0%, sensitivity of 82.9%, and Matthews correlation coefficient of 0.7.

Conclusions

The present study demonstrates that a deep learning-based OCT classification algorithm can be used for rapid automated identification of DRIL. This developed tool can assist in screening for DRIL in both research and clinical decision-making settings.

Translational Relevance

A deep learning algorithm can detect disorganization of retinal inner layers in OCT scans.

A machine learning approach applied to gynecological ultrasound to predict progression-free survival in ovarian cancer patients

In a growing number of social and clinical scenarios, machine learning (ML) is emerging as a promising tool for implementing complex multi-parametric decision-making algorithms. Regarding ovarian cancer (OC), despite the standardization of features that can support the discrimination of ovarian masses into benign and malignant, there is a lack of accurate predictive modeling based on ultrasound (US) examination for progression-free survival (PFS). This retrospective observational study analyzed patients with epithelial ovarian cancer (EOC) who were followed in a tertiary center from 2018 to 2019. Demographic features, clinical characteristics, information about the surgery and post-surgery histopathology were collected. Additionally, we recorded data about US examinations according to the International Ovarian Tumor Analysis (IOTA) classification. Our study aimed to realize a tool to predict 12 month PFS in patients with OC based on a ML algorithm applied to gynecological ultrasound assessment. Proper feature selection was used to determine an attribute core set. Three different machine learning algorithms, namely Logistic Regression (LR), Random Forest (RFF), and K-nearest neighbors (KNN), were then trained and validated with five-fold cross-validation to predict 12 month PFS. Our analysis included n. 64 patients and 12 month PFS was achieved by 46/64 patients (71.9%). The attribute core set used to train machine learning algorithms included age, menopause, CA-125 value, histotype, FIGO stage and US characteristics, such as major lesion diameter, side, echogenicity, color score, major solid component diameter, presence of carcinosis. RFF showed the best performance (accuracy 93.7%, precision 90%, recall 90%, area under receiver operating characteristic curve (AUROC) 0.92). We developed an accurate ML model to predict 12 month PFS.

Machine learning applied to MRI evaluation for the detection of lymph node metastasis in patients with locally advanced cervical cancer treated with neoadjuvant chemotherapy

PURPOSE

Concurrent cisplatin-based chemotherapy and radiotherapy (CCRT) plus brachytherapy is the standard treatment for locally advanced cervical cancer (LACC). Platinum-based neoadjuvant chemotherapy (NACT) followed by radical hysterectomy is an alternative for patients with stage IB2-IIB disease. Therefore, the correct pre-treatment staging is essential to the proper management of this disease. Pelvic magnetic resonance imaging (MRI) is the gold standard examination but studies about MRI accuracy in the detection of lymph node metastasis (LNM) in LACC patients show conflicting data. Machine learning (ML) is emerging as a promising tool for unraveling complex non-linear relationships between patient attributes that cannot be solved by traditional statistical methods. Here we investigated whether ML might improve the accuracy of MRI in the detection of LNM in LACC patients.

METHODS

We analyzed retrospectively LACC patients who underwent NACT and radical hysterectomy from 2015 to 2020. Demographic, clinical and MRI characteristics before and after NACT were collected, as well as information about post-surgery histopathology. Random features elimination wrapper was used to determine an attribute core set. A ML algorithm, namely Extreme Gradient Boosting (XGBoost) was trained and validated with tenfold cross-validation. The performances of the algorithm were assessed.

RESULTS

Our analysis included n.92 patients. FIGO stage was IB2 in n.4/92 (4.3%), IB3 in n.42/92 (45%), IIA1 in n.1/92 (1.1%), IIA2 in n.16/92 (17.4%) and IIB in n.29/92 (31.5%). Despite detected neither at pre-treatment and post-treatment MRI in any patients, LNM occurred in n.16/92 (17%) patients. The attribute core set used to train ML algorithms included grading, histotypes, age, parity, largest diameter of lesion at either pre- and post-treatment MRI, presence/absence of fornix infiltration at pre-treatment MRI and FIGO stage. XGBoost showed a good performance (accuracy 89%, precision 83%, recall 78%, AUROC 0.79).

CONCLUSIONS

We developed an accurate model to predict LNM in LACC patients in NACT, based on a ML algorithm requiring few easy-to-collect attributes.

Dermatopathology of Malignant Melanoma in the Era of Artificial Intelligence: A Single Institutional Experience

The application of artificial intelligence (AI) algorithms in medicine could support diagnostic and prognostic analyses and decision making. In the field of dermatopathology, there have been various papers that have trained algorithms for the recognition of different types of skin lesions, such as basal cell carcinoma (BCC), seborrheic keratosis (SK) and dermal nevus. Furthermore, the difficulty in diagnosing particular melanocytic lesions, such as Spitz nevi and melanoma, considering the grade of interobserver variability among dermatopathologists, has led to an objective difficulty in training machine learning (ML) algorithms to a totally reliable, reportable and repeatable level. In this work we tried to train a fast random forest (FRF) algorithm, typically used for the classification of clusters of pixels in images, to highlight anomalous areas classified as melanoma “defects” following the Allen–Spitz criteria. The adopted image vision diagnostic protocol was structured in the following steps: image acquisition by selecting the best zoom level of the microscope; preliminary selection of an image with a good resolution; preliminary identification of macro-areas of defect in each preselected image; identification of a class of a defect in the selected macro-area; training of the supervised machine learning FRF algorithm by selecting the micro-defect in the macro-area; execution of the FRF algorithm to find an image vision performance indicator; and analysis of the output images by enhancing lesion defects. The precision achieved by the FRF algorithm proved to be appropriate with a discordance of 17% with respect to the dermatopathologist, allowing this type of supervised algorithm to be nominated as a help to the dermatopathologist in the challenging diagnosis of malignant melanoma.

The Use and Utility of Machine Learning in Achieving Precision Medicine in Systemic Sclerosis: A Narrative Review

Background: Systemic sclerosis (SSc) is a rare connective tissue disease that can affect different organs and has extremely heterogenous presentations. This complexity makes it difficult to perform an early diagnosis and a subsequent subclassification of the disease. This hinders a personalized approach in clinical practice. In this context, machine learning (ML), a branch of artificial intelligence (AI), is able to recognize relationships in data and predict outcomes. Methods: Here, we performed a narrative review concerning the application of ML in SSc to define the state of art and evaluate its role in a precision medicine context. Results: Currently, ML has been used to stratify SSc patients and identify those at high risk of severe complications. Additionally, ML may be useful in the early detection of organ involvement. Furthermore, ML might have a role in target therapy approach and in predicting drug response. Conclusion: Available evidence about the utility of ML in SSc is sparse but promising. Future improvements in this field could result in a big step toward precision medicine. Further research is needed to define ML application in clinical practice.

A Machine Learning Approach to Predict Remission in Patients With Psoriatic Arthritis on Treatment With Secukinumab

Background

Psoriatic Arthritis (PsA) is a multifactorial disease, and predicting remission is challenging. Machine learning (ML) is a promising tool for building multi-parametric models to predict clinical outcomes. We aimed at developing a ML algorithm to predict the probability of remission in PsA patients on treatment with Secukinumab (SEC).

Methods

PsA patients undergoing SEC treatment between September 2017 and September 2020 were retrospectively analyzed. At baseline and 12-month follow-up, we retrieved demographic and clinical characteristics, including Body Mass Index (BMI), disease phenotypes, Disease Activity in PsA (DAPSA), Leeds Enthesitis Index (LEI) and presence/absence of comorbidities, including fibromyalgia and metabolic syndrome. Two random feature elimination wrappers, based on an eXtreme Gradient Boosting (XGBoost) and Logistic Regression (LR), were trained and validated with 10-fold cross-validation for predicting 12-month DAPSA remission with an attribute core set with the least number of predictors. The performance of each algorithm was assessed in terms of accuracy, precision, recall and area under receiver operating characteristic curve (AUROC).

Results

One-hundred-nineteen patients were selected. At 12 months, 20 out of 119 patients (25.21%) achieved DAPSA remission. Accuracy and AUROC of XGBoost was of 0.97 ± 0.06 and 0.97 ± 0.07, overtaking LR (accuracy 0.73 ± 0.09, AUROC 0.78 ± 0.14). Baseline DAPSA, fibromyalgia and axial disease were the most important attributes for the algorithm and were negatively associated with 12-month DAPSA remission.

Conclusions

A ML approach may identify SEC good responders. Patients with a high disease burden and axial disease with comorbid fibromyalgia seem challenging to treat.

Validity of Machine Learning in Predicting Giant Cell Arteritis Flare After Glucocorticoids Tapering

Background

Inferential statistical methods failed in identifying reliable biomarkers and risk factors for relapsing giant cell arteritis (GCA) after glucocorticoids (GCs) tapering. A ML approach allows to handle complex non-linear relationships between patient attributes that are hard to model with traditional statistical methods, merging them to output a forecast or a probability for a given outcome.

Objective

The objective of the study was to assess whether ML algorithms can predict GCA relapse after GCs tapering.

Methods

GCA patients who underwent GCs therapy and regular follow-up visits for at least 12 months, were retrospectively analyzed and used for implementing 3 ML algorithms, namely, Logistic Regression (LR), Decision Tree (DT), and Random Forest (RF). The outcome of interest was disease relapse within 3 months during GCs tapering. After a ML variable selection method, based on a XGBoost wrapper, an attribute core set was used to train and test each algorithm using 5-fold cross-validation. The performance of each algorithm in both phases was assessed in terms of accuracy and area under receiver operating characteristic curve (AUROC).

Results

The dataset consisted of 107 GCA patients (73 women, 68.2%) with mean age ( ± SD) 74.1 ( ± 8.5) years at presentation. GCA flare occurred in 40/107 patients (37.4%) within 3 months after GCs tapering. As a result of ML wrapper, the attribute core set with the least number of variables used for algorithm training included presence/absence of diabetes mellitus and concomitant polymyalgia rheumatica as well as erythrocyte sedimentation rate level at GCs baseline. RF showed the best performance, being significantly superior to other algorithms in accuracy (RF 71.4% vs LR 70.4% vs DT 62.9%). Consistently, RF precision (72.1%) was significantly greater than those of LR (62.6%) and DT (50.8%). Conversely, LR was superior to RF and DT in recall (RF 60% vs LR 62.5% vs DT 47.5%). Moreover, RF AUROC (0.76) was more significant compared to LR (0.73) and DT (0.65).

Conclusions

RF algorithm can predict GCA relapse after GCs tapering with sufficient accuracy. To date, this is one of the most accurate predictive modelings for such outcome. This ML method represents a reproducible tool, capable of supporting clinicians in GCA patient management.

Computational pathology for musculoskeletal conditions using machine learning: advances, trends, and challenges

Histopathology is widely used to analyze clinical biopsy specimens and tissues from pre-clinical models of a variety of musculoskeletal conditions. Histological assessment relies on scoring systems that require expertise, time, and resources, which can lead to an analysis bottleneck. Recent advancements in digital imaging and image processing provide an opportunity to automate histological analyses by implementing advanced statistical models such as machine learning and deep learning, which would greatly benefit the musculoskeletal field. This review provides a high-level overview of machine learning applications, a general pipeline of tissue collection to model selection, and highlights the development of image analysis methods, including some machine learning applications, to solve musculoskeletal problems. We discuss the optimization steps for tissue processing, sectioning, staining, and imaging that are critical for the successful generalizability of an automated image analysis model. We also commenting on the considerations that should be taken into account during model selection and the considerable advances in the field of computer vision outside of histopathology, which can be leveraged for image analysis. Finally, we provide a historic perspective of the previously used histopathological image analysis applications for musculoskeletal diseases, and we contrast it with the advantages of implementing state-of-the-art computational pathology approaches. While some deep learning approaches have been used, there is a significant opportunity to expand the use of such approaches to solve musculoskeletal problems.

Deep Learning-Based Classification of Inflammatory Arthritis by Identification of Joint Shape Patterns—How Neural Networks Can Tell Us Where to “Deep Dive” Clinically

Objective:

We investigated whether a neural network based on the shape of joints can differentiate between rheumatoid arthritis (RA), psoriatic arthritis (PsA), and healthy controls (HC), which class patients with undifferentiated arthritis (UA) are assigned to, and whether this neural network is able to identify disease-specific regions in joints.

Methods

We trained a novel neural network on 3D articular bone shapes of hand joints of RA and PsA patients as well as HC. Bone shapes were created from high-resolution peripheral-computed-tomography (HR-pQCT) data of the second metacarpal bone head. Heat maps of critical spots were generated using GradCAM. After training, we fed shape patterns of UA into the neural network to classify them into RA, PsA, or HC.

Results

Hand bone shapes from 932 HR-pQCT scans of 617 patients were available. The network could differentiate the classes with an area-under-receiver-operator-curve of 82% for HC, 75% for RA, and 68% for PsA. Heat maps identified anatomical regions such as bare area or ligament attachments prone to erosions and bony spurs. When feeding UA data into the neural network, 86% were classified as “RA,” 11% as “PsA,” and 3% as “HC” based on the joint shape.

Conclusion

We investigated neural networks to differentiate the shape of joints of RA, PsA, and HC and extracted disease-specific characteristics as heat maps on 3D joint shapes that can be utilized in clinical routine examination using ultrasound. Finally, unspecific diseases such as UA could be grouped using the trained network based on joint shape.

MIXTURE of human expertise and deep learning-developing an explainable model for predicting pathological diagnosis and survival in patients with interstitial lung disease

Interstitial pneumonia is a heterogeneous disease with a progressive course and poor prognosis, at times even worse than those in the main cancer types. Histopathological examination is crucial for its diagnosis and estimation of prognosis. However, the evaluation strongly depends on the experience of pathologists, and the reproducibility of diagnosis is low. Herein, we propose MIXTURE (huMan-In-the-loop eXplainable artificial intelligence Through the Use of REcurrent training), an original method to develop deep learning models for extracting pathologically significant findings based on an expert pathologist's perspective with a small annotation effort. The procedure of MIXTURE consists of three steps as follows. First, we created feature extractors for tiles from whole slide images using self-supervised learning. The similar looking tiles were clustered based on the output features and then pathologists integrated the pathologically synonymous clusters. Using the integrated clusters as labeled data, deep learning models to classify the tiles into pathological findings were created by transfer-learning the feature extractors. We developed three models for different magnifications. Using these extracted findings, our model was able to predict the diagnosis of usual interstitial pneumonia, a finding suggestive of progressive disease, with high accuracy (AUC 0.90 in validation set and AUC 0.86 in test set). This high accuracy could not be achieved without the integration of findings by pathologists. The patients predicted as UIP had poorer prognosis (5-year overall survival [OS]: 55.4%) than those predicted as non-UIP (OS: 95.2%). The Cox proportional hazards model for each microscopic finding and prognosis pointed out dense fibrosis, fibroblastic foci, elastosis, and lymphocyte aggregation as independent risk factors. We suggest that MIXTURE may serve as a model approach to different diseases evaluated by medical imaging, including pathology and radiology, and be the prototype for explainable artificial intelligence that can collaborate with humans.

Artificial Intelligence in Dermatopathology: New Insights and Perspectives

In recent years, an increasing enthusiasm has been observed towards artificial intelligence and machine learning, involving different areas of medicine. Among these, although still in the embryonic stage, the dermatopathological field has also been partially involved, with the attempt to develop and train algorithms that could assist the pathologist in the differential diagnosis of complex melanocytic lesions. In this article, we face this new challenge of the modern era, carry out a review of the literature regarding the state of the art and try to determine promising future perspectives.

Exploring the Oral Microbiome in Rheumatic Diseases, State of Art and Future Prospective in Personalized Medicine with an AI Approach

The oral microbiome is receiving growing interest from the scientific community, as the mouth is the gateway for numerous potential etiopathogenetic factors in different diseases. In addition, the progression of niches from the mouth to the gut, defined as "oral-gut microbiome axis", affects several pathologies, as rheumatic diseases. Notably, rheumatic disorders (RDs) are conditions causing chronic, often intermittent pain affecting the joints or connective tissue. In this review, we examine evidence which supports a role for the oral microbiome in the etiology and progression of various RDs, including rheumatoid arthritis (RA), Sjogren's syndrome (SS), and systemic lupus erythematosus (SLE). In addition, we address the most recent studies endorsing the oral microbiome as promising diagnostic biomarkers for RDs. Lastly, we introduce the concepts of artificial intelligence (AI), in particular, machine learning (ML) and their general application for understanding the link between oral microbiota and rheumatic diseases, speculating the application of a possible AI approach-based that can be applied to personalized medicine in the future.

A Machine Learning Tool to Predict the Response to Neoadjuvant Chemotherapy in Patients with Locally Advanced Cervical Cancer

Despite several studies having identified factors associated with successful treatment outcomes in locally advanced cervical cancer, there is the lack of accurate predictive modeling for progression-free survival (PFS) in patients who undergo radical hysterectomy after neoadjuvant chemotherapy (NACT). Here we investigated whether machine learning (ML) may have the potential to provide a tool to predict neoadjuvant treatment response as PFS. In this retrospective observational study, we analyzed patients with locally advanced cervical cancer (FIGO stages IB2, IB3, IIA1, IIA2, IIB, and IIIC1) who were followed in a tertiary center from 2010 to 2018. Demographic and clinical characteristics were collected at either treatment baseline or at 24-month follow-up. Furthermore, we recorded data about magnetic resonance imaging (MRI) examinations and post-surgery histopathology. Proper feature selection was used to determine an attribute core set. Three different machine learning algorithms, namely Logistic Regression (LR), Random Forest (RFF), and K-nearest neighbors (KNN), were then trained and validated with 10-fold cross-validation to predict 24-month PFS. Our analysis included n. 92 patients. The attribute core set used to train machine learning algorithms included the presence/absence of fornix infiltration at pre-treatment MRI as well as of either parametrium invasion and lymph nodes involvement at post-surgery histopathology. RFF showed the best performance (accuracy 82.4%, precision 83.4%, recall 96.2%, area under receiver operating characteristic curve (AUROC) 0.82). We developed an accurate ML model to predict 24-month PFS.