Psychosis Relapse Prediction Leveraging Electronic Health Records Data and Natural Language Processing Enrichment Methods

Accuracy of machine learning methods in predicting prognosis of patients with psychotic spectrum disorders: a systematic review

OBJECTIVES

We aimed to examine the predictive accuracy of functioning, relapse or remission among patients with psychotic disorders, using machine learning methods. We also identified specific features that were associated with these clinical outcomes.

DESIGN

The methodology of this review was guided by the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.

DATA SOURCES

CINAHL, EMBASE, PubMed, PsycINFO, Scopus and ScienceDirect were searched for relevant articles from database inception until 21 November 2024.

ELIGIBILITY CRITERIA

Studies were included if they involved the use of machine learning methods to predict functioning, relapse and/or remission among individuals with psychotic spectrum disorders.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers screened the records from the database search. Risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies tool from Cochrane. Synthesised findings were presented in tables.

RESULTS

23 studies were included in the review, which were mostly conducted in the west (91%). Predictive summary area under the curve values for functioning, relapse and remission were 0.63-0.92 (poor to outstanding), 0.45-0.95 (poor to outstanding), 0.70-0.79 (acceptable), respectively. Logistic regression and random forest were the best performing algorithms. Factors influencing outcomes included demographic (age, ethnicity), illness (duration of untreated illness, types of symptoms), functioning (baseline functioning, interpersonal relationships and activity engagement), treatment variables (use of higher doses of antipsychotics, electroconvulsive therapy), data from passive sensor (call log, distance travelled, time spent in certain locations) and online activities (time of use, use of certain words, changes in search frequencies and length of queries).

CONCLUSION

Machine learning methods show promise in the prediction of prognosis (specifically functioning, relapse and remission) of mental disorders based on relevant collected variables. Future machine learning studies may want to focus on the inclusion of a broader swathe of variables including ecological momentary assessments, with a greater amount of good quality big data covering longer longitudinal illness courses and coupled with external validation of study findings.

PROSPERO REGISTRATION NUMBER

The review was registered on PROSPERO, ID: CRD42023441108.

Text mining of outpatient narrative notes to predict the risk of psychiatric hospitalization

The primary purpose of this paper is to investigate whether text mining of the outpatient narrative notes for patients with severe and persistent mental illness (SPMI) can strengthen the predictions concerning the probability of an upcoming hospital readmission. A five-year study of all clinical notes for SPMI patients at the outpatient clinic of a tertiary hospital was conducted. The clinical notes were studied using ensemble classification i.e., entity recognition. Confounding variables pertaining to the patient's health status were extracted by text mining. A mixed effects logistic regression model was used for estimating the re-hospitalization risk during a clinic visit. The factors included frequency and continuity of outpatient visits, alterations in medication prescriptions, the usage of long-acting anti-psychotic injections (LAIs), the presence or absence of a legal compulsory treatment order (CTO) and the hospitalizations. The appearance of certain words in the outpatient clinical notes has a statistically significant impact on the risk of an upcoming hospitalization. This study also reconfirms that the risk of a re-hospitalization of an SPMI patient is reduced by the presence of a CTO and the utilization of LAIs, whereas it is increased by the patient dropping out of outpatient care. Our findings pertaining to the risk of re-hospitalization could facilitate preventive interventions for SPMI patients with higher risk.

Neuroimaging and natural language processing-based classification of suicidal thoughts in major depressive disorder

Suicide is a growing public health problem around the world. The most important risk factor for suicide is underlying psychiatric illness, especially depression. Detailed classification of suicide in patients with depression can greatly enhance personalized suicide control efforts. This study used unstructured psychiatric charts and brain magnetic resonance imaging (MRI) records from a psychiatric outpatient clinic to develop a machine learning-based suicidal thought classification model. The study included 152 patients with new depressive episodes for development and 58 patients from a geographically different hospital for validation. We developed an eXtreme Gradient Boosting (XGBoost)-based classification models according to the combined types of data: independent components-map weightings from brain T1-weighted MRI and topic probabilities from clinical notes. Specifically, we used 5 psychiatric symptom topics and 5 brain networks for models. Anxiety and somatic symptoms topics were significantly more common in the suicidal group, and there were group differences in the default mode and cortical midline networks. The clinical symptoms plus structural brain patterns model had the highest area under the receiver operating characteristic curve (0.794) versus the clinical notes only and brain MRI only models (0.748 and 0.738, respectively). The results were consistent across performance metrics and external validation. Our findings suggest that focusing on personalized neuroimaging and natural language processing variables improves evaluation of suicidal thoughts.

Identifying Psychosis Episodes in Psychiatric Admission Notes via Rule-based Methods, Machine Learning, and Pre-Trained Language Models

Early and accurate diagnosis is crucial for effective treatment and improved outcomes, yet identifying psychotic episodes presents significant challenges due to its complex nature and the varied presentation of symptoms among individuals. One of the primary difficulties lies in the underreporting and underdiagnosis of psychosis, compounded by the stigma surrounding mental health and the individuals' often diminished insight into their condition. Existing efforts leveraging Electronic Health Records (EHRs) to retrospectively identify psychosis typically rely on structured data, such as medical codes and patient demographics, which frequently lack essential information. Addressing these challenges, our study leverages Natural Language Processing (NLP) algorithms to analyze psychiatric admission notes for the diagnosis of psychosis, providing a detailed evaluation of rule-based algorithms, machine learning models, and pre-trained language models. Additionally, the study investigates the effectiveness of employing keywords to streamline extensive note data before training and evaluating the models. Analyzing 4,617 initial psychiatric admission notes (1,196 cases of psychosis versus 3,433 controls) from 2005 to 2019, we discovered that the XGBoost classifier employing Term Frequency-Inverse Document Frequency (TF-IDF) features derived from notes pre-selected by expert-curated keywords, attained the highest performance with an F1 score of 0.8881 (AUROC [95% CI]: 0.9725 [0.9717, 0.9733]). BlueBERT demonstrated comparable efficacy an F1 score of 0.8841 (AUROC [95% CI]: 0.97 [0.9580, 0.9820]) on the same set of notes. Both models markedly outperformed traditional International Classification of Diseases (ICD) code-based detection methods from discharge summaries, which had an F1 score of 0.7608, thus improving the margin by 0.12. Furthermore, our findings indicate that keyword pre-selection markedly enhances the performance of both machine learning and pre-trained language models. This study illustrates the potential of NLP techniques to improve psychosis detection within admission notes and aims to serve as a foundational reference for future research on applying NLP for psychosis identification in EHR notes.

Psychotic disorders as a framework for precision psychiatry

People with psychotic disorders can show marked interindividual variations in the onset of illness, responses to treatment and relapse, but they receive broadly similar clinical care. Precision psychiatry is an approach that aims to stratify people with a given disorder according to different clinical outcomes and tailor treatment to their individual needs. At present, interindividual differences in outcomes of psychotic disorders are difficult to predict on the basis of clinical assessment alone. Therefore, current research in psychosis seeks to build models that predict outcomes by integrating clinical information with a range of biological measures. Here, we review recent progress in the application of precision psychiatry to psychotic disorders and consider the challenges associated with implementing this approach in clinical practice.

Chronic Disease Prediction Using the Common Data Model: Development Study

BACKGROUND

Chronic disease management is a major health issue worldwide. With the paradigm shift to preventive medicine, disease prediction modeling using machine learning is gaining importance for precise and accurate medical judgement.

OBJECTIVE

This study aimed to develop high-performance prediction models for 4 chronic diseases using the common data model (CDM) and machine learning and to confirm the possibility for the extension of the proposed models.

METHODS

In this study, 4 major chronic diseases-namely, diabetes, hypertension, hyperlipidemia, and cardiovascular disease-were selected, and a model for predicting their occurrence within 10 years was developed. For model development, the Atlas analysis tool was used to define the chronic disease to be predicted, and data were extracted from the CDM according to the defined conditions. A model for predicting each disease was built with 4 algorithms verified in previous studies, and the performance was compared after applying a grid search.

RESULTS

For the prediction of each disease, we applied 4 algorithms (logistic regression, gradient boosting, random forest, and extreme gradient boosting), and all models show greater than 80% accuracy. As compared to the optimized model's performance, extreme gradient boosting presented the highest predictive performance for the 4 diseases (diabetes, hypertension, hyperlipidemia, and cardiovascular disease) with 80% or greater and from 0.84 to 0.93 in area under the curve standards.

CONCLUSIONS

This study demonstrates the possibility for the preemptive management of chronic diseases by predicting the occurrence of chronic diseases using the CDM and machine learning. With these models, the risk of developing major chronic diseases within 10 years can be demonstrated by identifying health risk factors using our chronic disease prediction machine learning model developed with the real-world data-based CDM and National Health Insurance Corporation examination data that individuals can easily obtain.