Validation of a machine learning approach to estimate Systemic Lupus Erythematosus Disease Activity Index score categories and application in a real-world dataset

Systemic lupus in the era of machine learning medicine

Artificial intelligence and machine learning applications are emerging as transformative technologies in medicine. With greater access to a diverse range of big datasets, researchers are turning to these powerful techniques for data analysis. Machine learning can reveal patterns and interactions between variables in large and complex datasets more accurately and efficiently than traditional statistical methods. Machine learning approaches open new possibilities for studying SLE, a multifactorial, highly heterogeneous and complex disease. Here, we discuss how machine learning methods are rapidly being integrated into the field of SLE research. Recent reports have focused on building prediction models and/or identifying novel biomarkers using both supervised and unsupervised techniques for understanding disease pathogenesis, early diagnosis and prognosis of disease. In this review, we will provide an overview of machine learning techniques to discuss current gaps, challenges and opportunities for SLE studies. External validation of most prediction models is still needed before clinical adoption. Utilisation of deep learning models, access to alternative sources of health data and increased awareness of the ethics, governance and regulations surrounding the use of artificial intelligence in medicine will help propel this exciting field forward.

Machine learning, a new tool for the detection of immunodeficiency patterns in systemic lupus erythematosus

Systemic lupus erythematosus (SLE) is a complex autoimmune disease that affects several organs and causes variable clinical symptoms. Early diagnosis is currently the most effective way to save the lives of patients with SLE. But it is very difficult to detect in the early stages of the disease. Because of this, this study proposes a machine learning system to help diagnose patients with SLE. To carry out the research, the extreme gradient boosting method has been implemented due to its performance characteristics, as it allows high performance, scalability, accuracy, and low computational load. From this method we try to recognize patterns in the data obtained from patients, which allow the classification of SLE patients with high accuracy and differentiate these patients from controls. Several machine learning methods have been analyzed in this study. The proposed method achieves a higher prediction value of patients who may suffer from SLE than the rest of the compared systems. The proposed algorithm achieved an improvement in accuracy of 4.49% over k-Nearest Neighbors. As for the Support Vector Machine and Gaussian Naive Bayes (GNB) methods, they achieved a lower performance than the proposed one, reaching values of 83% and 81%, respectively. It should be noted that the proposed system showed a higher area under the curve (90%) and a balanced accuracy (90%) than the other machine learning methods. This study shows the usefulness of ML techniques for identifying and predicting SLE patients. These results demonstrate the possibility of developing automatic diagnostic support systems for SLE patients based on machine learning techniques.

Lupus Nephritis Risk Factors and Biomarkers: An Update

Lupus nephritis (LN) represents the most severe organ manifestation of systemic lupus erythematosus (SLE) in terms of morbidity and mortality. To reduce these risks, tremendous efforts have been made in the last decade to characterize the different steps of the disease and to develop biomarkers in order to better (i) unravel the pre-SLE stage (e.g., anti-nuclear antibodies and interferon signature); (ii) more timely initiation of therapy by improving early and accurate LN diagnosis (e.g., pathologic classification was revised); (iii) monitor disease activity and therapeutic response (e.g., recommendation to re-biopsy, new urinary biomarkers); (iv) prevent disease flares (e.g., serologic and urinary biomarkers); (v) mitigate the deterioration in the renal function; and (vi) reduce side effects with new therapeutic guidelines and novel therapies. However, progress is poor in terms of improvement with early death attributed to active SLE or infections, while later deaths are related to the chronicity of the disease and the use of toxic therapies. Consequently, an individualized treat-to-target strategy is mandatory, and for that, there is an unmet need to develop a set of accurate biomarkers to be used as the standard of care and adapted to each stage of the disease.

Application of Machine Learning Models in Systemic Lupus Erythematosus

Systemic Lupus Erythematosus (SLE) is a systemic autoimmune disease and is extremely heterogeneous in terms of immunological features and clinical manifestations. This complexity could result in a delay in the diagnosis and treatment introduction, with impacts on long-term outcomes. In this view, the application of innovative tools, such as machine learning models (MLMs), could be useful. Thus, the purpose of the present review is to provide the reader with information about the possible application of artificial intelligence in SLE patients from a medical perspective. To summarize, several studies have applied MLMs in large cohorts in different disease-related fields. In particular, the majority of studies focused on diagnosis and pathogenesis, disease-related manifestations, in particular Lupus Nephritis, outcomes and treatment. Nonetheless, some studies focused on peculiar features, such as pregnancy and quality of life. The review of published data demonstrated the proposal of several models with good performance, suggesting the possible application of MLMs in the SLE scenario.

Current state and completeness of reporting clinical prediction models using machine learning in systemic lupus erythematosus: A systematic review

OBJECTIVE

We carried out a systematic review (SR) of adherence in diagnostic and prognostic applications of ML in SLE using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement.

METHODS

A SR employing five databases was conducted from its inception until December 2021. We identified articles that evaluated the utilization of ML for prognostic and/or diagnostic purposes. This SR was reported based on the PRISMA guidelines. The TRIPOD statement assessed adherence to reporting standards. Assessment for risk of bias was done using PROBAST tool.

RESULTS

We included 45 studies: 29 (64.4%) diagnostic and 16 (35.5%) prognostic prediction- model studies. Overall, articles adhered by between 17% and 67% (median 43%, IQR 37-49%) to TRIPOD items. Only few articles reported the model's predictive performance (2.3%, 95% CI 0.06-12.0), testing of interaction terms (2.3%, 95% CI 0.06-12.0), flow of participants (50%, 95% CI; 34.6-65.4), blinding of predictors (2.3%, 95% CI 0.06-12.0), handling of missing data (36.4%, 95% CI 22.4-52.2), and appropriate title (20.5%, 95% CI 9.8-35.3). Some items were almost completely reported: the source of data (88.6%, 95% CI 75.4-96.2), eligibility criteria (86.4%, 95% CI 76.2-96.5), and interpretation of findings (88.6%, 95% CI 75.4-96.2). In addition, most of model studies had high risk of bias.

CONCLUSIONS

The reporting adherence of ML-based model developed for SLE, is currently inadequate. Several items deemed crucial for transparent reporting were not fully reported in studies on ML-based prediction models.

REVIEW REGISTRATION

PROSPERO ID# CRD42021284881. (Amended to limit the scope).

Machine Learning for Diagnosis of Systemic Lupus Erythematosus: A Systematic Review and Meta-Analysis

Background

Application of machine learning (ML) for identification of systemic lupus erythematosus (SLE) has been recently drawing increasing attention, while there is still lack of evidence-based support.

Methods

Systematic review and meta-analysis are conducted to evaluate its diagnostic accuracy and application prospect. PubMed, Embase, Cochrane Library, and Web of Science libraries are searched, in combination with manual searching and literature retrospection, for studies regarding machine learning for identifying SLE and neuropsychiatric systemic lupus erythematosus (NPSLE). Quality Assessment of Diagnostic Accuracy Studies (QUADA-2) is applied to assess the quality of included studies. Diagnostic accuracy of the SLE model and NPSLE model is assessed using the bivariate fixed-effect model, and the data are pooled. Summary receiver operator characteristic curve (SROC) is plotted, and area under the curve (AUC) is calculated.

Results

Eighteen (18) studies are included, in which ten (10) focused on SLE and eight (8) on NPSLE. The AUC of SLE identification is 0.95, the sensitivity is 0.90, the specificity is 0.89, the PLR is 8.4, the NLR is 0.12, and the DOR is 73. AUC of NPSLE identification is 0.89, the sensitivity is 0.83, the specificity is 0.83, the PLR is 5.0, the NLR is 0.20, and the DOR is 25.

Conclusion

Machine learning presented remarkable performance in identification of SLE and NPSLE. Based on the convenience for inclusion factor collection and non-invasiveness of detection, machine learning is expected to be widely applied in clinical practice to assist medical decision making.

Narrative Review of Machine Learning in Rheumatic and Musculoskeletal Diseases for Clinicians and Researchers: Biases, Goals, and Future Directions

There has been rapid growth in the use of artificial intelligence (AI) analytics in medicine in recent years, including in rheumatic and musculoskeletal diseases (RMDs). Such methods represent a challenge to clinicians, patients, and researchers, given the "black box" nature of most algorithms, the unfamiliarity of the terms, and the lack of awareness of potential issues around these analyses. Therefore, this review aims to introduce this subject area in a way that is relevant and meaningful to clinicians and researchers. We hope to provide some insights into relevant strengths and limitations, reporting guidelines, as well as recent examples of such analyses in key areas, with a focus on lessons learned and future directions in diagnosis, phenotyping, prognosis, and precision medicine in RMDs.

Recent advances in the use of machine learning and artificial intelligence to improve diagnosis, predict flares, and enrich clinical trials in lupus

PURPOSE OF REVIEW

Machine learning is a computational tool that is increasingly used for the analysis of medical data and has provided the promise of more personalized care.

RECENT FINDINGS

The frequency with which machine learning analytics are reported in lupus research is comparable with that of rheumatoid arthritis and cancer, yet the clinical application of these computational tools has yet to be translated into better care. Considerable work has been applied to the development of machine learning models for lupus diagnosis, flare prediction, and classification of disease using histology or other medical images, yet few models have been tested in external datasets and independent centers. Application of machine learning has yet to be reported for lupus clinical trial enrichment and automated identification of eligible patients. Integration of machine learning into lupus clinical care and clinical trials would benefit from collaborative development between clinicians and data scientists.

SUMMARY

Although the application of machine learning to lupus data is at a nascent stage, initial results suggest a promising future.

Big data analyses and individual health profiling in the arena of rheumatic and musculoskeletal diseases (RMDs)

Health care processes are under constant development and will need to embrace advances in technology and health science aiming to provide optimal care. Considering the perspective of increasing treatment options for people with rheumatic and musculoskeletal diseases, but in many cases not reaching all treatment targets that matter to patients, care systems bare potential to improve on a holistic level. This review provides an overview of systems and technologies under evaluation over the past years that show potential to impact diagnosis and treatment of rheumatic diseases in about 10 years from now. We summarize initiatives and studies from the field of electronic health records, biobanking, remote monitoring, and artificial intelligence. The combination and implementation of these opportunities in daily clinical care will be key for a new era in care of our patients. This aims to inform rheumatologists and healthcare providers concerned with chronic inflammatory musculoskeletal conditions about current important and promising developments in science that might substantially impact the management processes of rheumatic diseases in the 2030s.

Tailored treatment strategies and future directions in systemic lupus erythematosus

Systemic lupus erythematosus (SLE) represents a diagnostic and therapeutic challenge for physicians due to its protean manifestations and unpredictable course. The disease may manifest as multisystemic or organ-dominant and severity at presentation may vary according to age at onset (childhood-, adult- or late-onset SLE). Different manifestations may respond variably to different immunosuppressive medications and, even within the same organ-system, the severity of inflammation may vary from mild to organ-threatening. Current "state-of-the-art" in SLE treatment aims at remission or low disease activity in all organ systems. Apart from hydroxychloroquine and glucocorticoids (which should be used with caution), the choice of the appropriate immunosuppressive agent should be individualized and depend on the prevailing manifestation, severity stratification and patient childbearing potential. In this review, we provide an overview of therapeutic options for the various organ manifestations and severity patterns of the disease, different phenotypes (such as multisystem versus organ-dominant disease), as well as specific considerations, including lupus with antiphospholipid antibodies, childhood and late-onset disease, as well as treatment options during pregnancy and lactation.

Reproductive concerns and contributing factors in women of childbearing age with systemic lupus erythematosus

OBJECTIVES

Reproductive concerns are common in women of childbearing age with systemic lupus erythematosus (SLE) with inadequate disclosure. This study aimed to investigate the contributing factors of reproductive concerns and to evaluate their impact on health-related quality of life.

METHODS

One hundred eighty women of childbearing age with SLE were enrolled in this cross-sectional study in Affiliated Hospital of Nantong University from March 2021 to December 2021. A series of questionnaires were conducted: Systemic Lupus Erythematosus Disease Activity Index (SLEDAI), the Hospital Anxiety and Depression Scale (HADS), the Multidimensional Fatigue Inventory (MFI-20), Female Sexual Distress Scale-Revised (FSDS-R), Family Assessment Device (APGAR), the Medical Coping Modes Questionnaire (MCMQ), the Short-Form 36 (SF-36), and the Chinese version of Reproductive Concerns After Cancer (RCAC). Independent t test, one-way ANOVA, Mann-Whitney U test, Pearson/Spearman, and multiple linear stepwise regression were used to analyze the data.

RESULTS

The results indicated that female SLE patients were more concerned about the child's health and personal health than becoming pregnant, fertility potential, partner disclose and acceptance; SLE patients with the characteristics of living in rural residence, having no reproductive history, fearing unexpected pregnancy, sexual distress, and depression showed more serious fertility concerns. Meanwhile, most female SLE patients adopted active confrontation when facing reproductive concerns, and these patients were significantly lower in the dimension score of mental related quality of life.

CONCLUSIONS

Our study demonstrated that female SLE patients should be paid more attention to their fertility concerns and effective intervention measures should be carried out to relieve their reproductive concerns, so as to improve their long-term quality of life if their disease condition permits.

Validation of a machine learning approach to estimate expanded disability status scale scores for multiple sclerosis

Background

Disability assessment using the Expanded Disability Status Scale (EDSS) is important to inform treatment decisions and monitor the progression of multiple sclerosis. Yet, EDSS scores are documented infrequently in electronic medical records.

Objective

To validate a machine learning model to estimate EDSS scores for multiple sclerosis patients using clinical notes from neurologists.

Methods

A machine learning model was developed to estimate EDSS scores on specific encounter dates using clinical notes from neurologist visits. The OM1 MS Registry data were used to create a training cohort of 2632 encounters and a separate validation cohort of 857 encounters, all with clinician-recorded EDSS scores. Model performance was assessed using the area under the receiver-operating-characteristic curve (AUC), positive predictive value (PPV), and negative predictive value (NPV), calculated using a binarized version of the outcome. The Spearman R and Pearson R values were calculated. The model was then applied to encounters without clinician-recorded EDSS scores in the MS Registry.

Results

The model had a PPV of 0.85, NPV of 0.85, and AUC of 0.91. The model had a Spearman R value of 0.75 and Pearson R value of 0.74 when evaluating performance using the continuous estimated EDSS and clinician-recorded EDSS scores. Application of the model to eligible encounters resulted in the generation of eEDSS scores for an additional 190,282 encounters from 13,249 patients.

Conclusion

EDSS scores can be estimated with very good performance using a machine learning model applied to clinical notes, thus increasing the utility of real-world data sources for research purposes.

Validation of a machine learning approach to estimate Clinical Disease Activity Index Scores for rheumatoid arthritis

OBJECTIVE

Disease activity measures, such as the Clinical Disease Activity Index (CDAI), are important tools for informing treatment decisions and monitoring patient outcomes in rheumatoid arthritis (RA). Yet, documentation of CDAI scores in electronic medical records and other real-world data sources is inconsistent, making it challenging to use these data for research. The purpose of this study was to validate a machine learning model to estimate CDAI scores for patients with RA using clinical notes.

METHODS

A machine learning model was developed to estimate CDAI score values using clinical notes from a specific rheumatology visit. Data from the OM1 RA Registry were used to create a training cohort of 56 177 encounters and a separate validation cohort of 18 726 encounters, 11 985 of which passed a model-derived confidence filter; all included encounters had both a clinician-recorded CDAI score and a clinical note. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), positive predictive value (PPV) and negative predictive value (NPV), calculated using a binarised version of the outcome. The Spearman's R and Pearson's R values were also calculated.

RESULTS

The model had a PPV of 0.80, NPV of 0.84 and AUC of 0.88 when evaluating performance using the binarised version of the outcome. The model had a Spearman's R value of 0.72 and a Pearson's R value of 0.69 when evaluating performance using the continuous CDAI numeric scores.

CONCLUSION

A machine learning model estimates CDAI scores from clinical notes with good performance. Application of the model to real-world data sets may allow estimated CDAI scores to be used for research purposes.