Models to predict relapse in psychosis: A systematic review

Speech markers to predict and prevent recurrent episodes of psychosis: A narrative overview and emerging opportunities

Preventing relapse in schizophrenia improves long-term health outcomes. Repeated episodes of psychotic symptoms shape the trajectory of this illness and can be a detriment to functional recovery. Despite early intervention programs, high relapse rates persist, calling for alternative approaches in relapse prevention. Predicting imminent relapse at an individual level is critical for effective intervention. While clinical profiles are often used to foresee relapse, they lack the specificity and sensitivity needed for timely prediction. Here, we review the use of speech through Natural Language Processing (NLP) to predict a recurrent psychotic episode. Recent advancements in NLP of speech have shown the ability to detect linguistic markers related to thought disorder and other language disruptions within 2-4 weeks preceding a relapse. This approach has shown to be able to capture individual speech patterns, showing promise in its use as a prediction tool. We outline current developments in remote monitoring for psychotic relapses, discuss the challenges and limitations and present the speech-NLP based approach as an alternative to detect relapses with sufficient accuracy, construct validity and lead time to generate clinical actions towards prevention.

Predicting psychotic relapse following randomised discontinuation of paliperidone in individuals with schizophrenia or schizoaffective disorder: an individual participant data analysis

BACKGROUND

Predicting relapse for individuals with psychotic disorders is not well established, especially after discontinuation of antipsychotic treatment. We aimed to identify general prognostic factors of relapse for all participants (irrespective of treatment continuation or discontinuation) and specific predictors of relapse for treatment discontinuation, using machine learning.

METHODS

For this individual participant data analysis, we searched the Yale University Open Data Access Project's database for placebo-controlled, randomised antipsychotic discontinuation trials with participants with schizophrenia or schizoaffective disorder (aged ≥18 years). We included studies in which participants were treated with any antipsychotic study drug and randomly assigned to continue the same antipsychotic drug or to discontinue it and receive placebo. We assessed 36 prespecified baseline variables at randomisation to predict time to relapse, using univariate and multivariate proportional hazard regression models (including multivariate treatment group by variable interactions) with machine learning to categorise the variables as general prognostic factors of relapse, specific predictors of relapse, or both.

FINDINGS

We identified 414 trials, of which five trials with 700 participants (304 [43%] women and 396 [57%] men) were eligible for the continuation group and 692 participants (292 [42%] women and 400 [58%] men) were eligible for the discontinuation group (median age 37 [IQR 28-47] years for continuation group and 38 [28-47] years for discontinuation group). Out of the 36 baseline variables, general prognostic factors of increased risk of relapse for all participants were drug-positive urine; paranoid, disorganised, and undifferentiated types of schizophrenia (lower risk for schizoaffective disorder); psychiatric and neurological adverse events; higher severity of akathisia (ie, difficulty or inability to sit still); antipsychotic discontinuation; lower social performance; younger age; lower glomerular filtration rate; benzodiazepine comedication (lower risk for anti-epileptic comedication). Out of the 36 baseline variables, predictors of increased risk specifically after antipsychotic discontinuation were increased prolactin concentration, higher number of hospitalisations, and smoking. Both prognostic factors and predictors with increased risk after discontinuation were oral antipsychotic treatment (lower risk for long-acting injectables), higher last dosage of the antipsychotic study drug, shorter duration of antipsychotic treatment, and higher score on the Clinical Global Impression (CGI) severity scale The predictive performance (concordance index) for participants who were not used to train the model was 0·707 (chance level is 0·5).

INTERPRETATION

Routinely available general prognostic factors of psychotic relapse and predictors specific for treatment discontinuation could be used to support personalised treatment. Abrupt discontinuation of higher dosages of oral antipsychotics, especially for individuals with recurring hospitalisations, higher scores on the CGI severity scale, and increased prolactin concentrations, should be avoided to reduce the risk of relapse.

FUNDING

German Research Foundation and Berlin Institute of Health.

Digital smartphone intervention to recognise and manage early warning signs in schizophrenia to prevent relapse: the EMPOWER feasibility cluster RCT

BACKGROUND

Relapse is a major determinant of outcome for people with a diagnosis of schizophrenia. Early warning signs frequently precede relapse. A recent Cochrane Review found low-quality evidence to suggest a positive effect of early warning signs interventions on hospitalisation and relapse.

OBJECTIVE

How feasible is a study to investigate the clinical effectiveness and cost-effectiveness of a digital intervention to recognise and promptly manage early warning signs of relapse in schizophrenia with the aim of preventing relapse?

DESIGN

A multicentre, two-arm, parallel-group cluster randomised controlled trial involving eight community mental health services, with 12-month follow-up.

SETTINGS

Glasgow, UK, and Melbourne, Australia.

PARTICIPANTS

Service users were aged > 16 years and had a schizophrenia spectrum disorder with evidence of a relapse within the previous 2 years. Carers were eligible for inclusion if they were nominated by an eligible service user.

INTERVENTIONS

The Early signs Monitoring to Prevent relapse in psychosis and prOmote Wellbeing, Engagement, and Recovery (EMPOWER) intervention was designed to enable participants to monitor changes in their well-being daily using a mobile phone, blended with peer support. Clinical triage of changes in well-being that were suggestive of early signs of relapse was enabled through an algorithm that triggered a check-in prompt that informed a relapse prevention pathway, if warranted.

MAIN OUTCOME MEASURES

The main outcomes were feasibility of the trial and feasibility, acceptability and usability of the intervention, as well as safety and performance. Candidate co-primary outcomes were relapse and fear of relapse.

RESULTS

We recruited 86 service users, of whom 73 were randomised (42 to EMPOWER and 31 to treatment as usual). Primary outcome data were collected for 84% of participants at 12 months. Feasibility data for people using the smartphone application (app) suggested that the app was easy to use and had a positive impact on motivations and intentions in relation to mental health. Actual app usage was high, with 91% of users who completed the baseline period meeting our a priori criterion of acceptable engagement (> 33%). The median time to discontinuation of > 33% app usage was 32 weeks (95% confidence interval 14 weeks to ∞). There were 8 out of 33 (24%) relapses in the EMPOWER arm and 13 out of 28 (46%) in the treatment-as-usual arm. Fewer participants in the EMPOWER arm had a relapse (relative risk 0.50, 95% confidence interval 0.26 to 0.98), and time to first relapse (hazard ratio 0.32, 95% confidence interval 0.14 to 0.74) was longer in the EMPOWER arm than in the treatment-as-usual group. At 12 months, EMPOWER participants were less fearful of having a relapse than those in the treatment-as-usual arm (mean difference -4.29, 95% confidence interval -7.29 to -1.28). EMPOWER was more costly and more effective, resulting in an incremental cost-effectiveness ratio of £3041. This incremental cost-effectiveness ratio would be considered cost-effective when using the National Institute for Health and Care Excellence threshold of £20,000 per quality-adjusted life-year gained.

LIMITATIONS

This was a feasibility study and the outcomes detected cannot be taken as evidence of efficacy or effectiveness.

CONCLUSIONS

A trial of digital technology to monitor early warning signs that blended with peer support and clinical triage to detect and prevent relapse is feasible.

FUTURE WORK

A main trial with a sample size of 500 (assuming 90% power and 20% dropout) would detect a clinically meaningful reduction in relapse (relative risk 0.7) and improvement in other variables (effect sizes 0.3-0.4).

TRIAL REGISTRATION

This trial is registered as ISRCTN99559262.

FUNDING

This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 27. See the NIHR Journals Library website for further project information. Funding in Australia was provided by the National Health and Medical Research Council (APP1095879).

Prediction models in first-episode psychosis: systematic review and critical appraisal

BACKGROUND

People presenting with first-episode psychosis (FEP) have heterogenous outcomes. More than 40% fail to achieve symptomatic remission. Accurate prediction of individual outcome in FEP could facilitate early intervention to change the clinical trajectory and improve prognosis.

AIMS

We aim to systematically review evidence for prediction models developed for predicting poor outcome in FEP.

METHOD

A protocol for this study was published on the International Prospective Register of Systematic Reviews, registration number CRD42019156897. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance, we systematically searched six databases from inception to 28 January 2021. We used the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies and the Prediction Model Risk of Bias Assessment Tool to extract and appraise the outcome prediction models. We considered study characteristics, methodology and model performance.

RESULTS

Thirteen studies reporting 31 prediction models across a range of clinical outcomes met criteria for inclusion. Eleven studies used logistic regression with clinical and sociodemographic predictor variables. Just two studies were found to be at low risk of bias. Methodological limitations identified included a lack of appropriate validation, small sample sizes, poor handling of missing data and inadequate reporting of calibration and discrimination measures. To date, no model has been applied to clinical practice.

CONCLUSIONS

Future prediction studies in psychosis should prioritise methodological rigour and external validation in larger samples. The potential for prediction modelling in FEP is yet to be realised.

Psychosis Relapse Prediction Leveraging Electronic Health Records Data and Natural Language Processing Enrichment Methods

BACKGROUND

Identifying patients at a high risk of psychosis relapse is crucial for early interventions. A relevant psychiatric clinical context is often recorded in clinical notes; however, the utilization of unstructured data remains limited. This study aimed to develop psychosis-relapse prediction models using various types of clinical notes and structured data.

METHODS

Clinical data were extracted from the electronic health records of the Ajou University Medical Center in South Korea. The study population included patients with psychotic disorders, and outcome was psychosis relapse within 1 year. Using only structured data, we developed an initial prediction model, then three natural language processing (NLP)-enriched models using three types of clinical notes (psychological tests, admission notes, and initial nursing assessment) and one complete model. Latent Dirichlet Allocation was used to cluster the clinical context into similar topics. All models applied the least absolute shrinkage and selection operator logistic regression algorithm. We also performed an external validation using another hospital database.

RESULTS

A total of 330 patients were included, and 62 (18.8%) experienced psychosis relapse. Six predictors were used in the initial model and 10 additional topics from Latent Dirichlet Allocation processing were added in the enriched models. The model derived from all notes showed the highest value of the area under the receiver operating characteristic (AUROC = 0.946) in the internal validation, followed by models based on the psychological test notes, admission notes, initial nursing assessments, and structured data only (0.902, 0.855, 0.798, and 0.784, respectively). The external validation was performed using only the initial nursing assessment note, and the AUROC was 0.616.

CONCLUSIONS

We developed prediction models for psychosis relapse using the NLP-enrichment method. Models using clinical notes were more effective than models using only structured data, suggesting the importance of unstructured data in psychosis prediction.

Individualized prediction of three- and six-year outcomes of psychosis in a longitudinal multicenter study: a machine learning approach

Schizophrenia and related disorders have heterogeneous outcomes. Individualized prediction of long-term outcomes may be helpful in improving treatment decisions. Utilizing extensive baseline data of 523 patients with a psychotic disorder and variable illness duration, we predicted symptomatic and global outcomes at 3-year and 6-year follow-ups. We classified outcomes as (1) symptomatic: in remission or not in remission, and (2) global outcome, using the Global Assessment of Functioning (GAF) scale, divided into good (GAF ≥ 65) and poor (GAF < 65). Aiming for a robust and interpretable prediction model, we employed a linear support vector machine and recursive feature elimination within a nested cross-validation design to obtain a lean set of predictors. Generalization to out-of-study samples was estimated using leave-one-site-out cross-validation. Prediction accuracies were above chance and ranged from 62.2% to 64.7% (symptomatic outcome), and 63.5-67.6% (global outcome). Leave-one-site-out cross-validation demonstrated the robustness of our models, with a minor drop in predictive accuracies of 2.3% on average. Important predictors included GAF scores, psychotic symptoms, quality of life, antipsychotics use, psychosocial needs, and depressive symptoms. These robust, albeit modestly accurate, long-term prognostic predictions based on lean predictor sets indicate the potential of machine learning models complementing clinical judgment and decision-making. Future model development may benefit from studies scoping patient's and clinicians' needs in prognostication.

Utility of the READMIT Index to Identify Community Hospital 30-Day Psychiatric Readmissions

This case-controlled study determined the utility of the READMIT index to identify the risk for 30-day readmission of patients discharged from an urban community hospital psychiatric inpatient unit. Data was collected from 118 matched patient pairs from 2017 to 2018. Findings demonstrated the READMIT index did not effectively discriminate those patients who were likely to readmit within 30 days. However, the following factors were associated with likelihood of 30-day readmission: the inability to care for self, number of lifetime readmissions, the comorbidity of liver disease, as well as a history of substance abuse.

Identification of Risk Factors to Predict the Occurrences of Relapses in Individuals with Schizophrenia Spectrum Disorder in Iran

Schizophrenia Spectrum Disorder (SSD) is a chronic psychiatric disorder with a modest treatment outcome. In addition, relapses are commonplace. Here, we sought to identify factors that predict relapse latency and frequency. To this end, we retrospectively analyzed data for individuals with SSD. Medical records of 401 individuals with SSD were analyzed (mean age: 25.51 years; 63.6% males) covering a five-year period. Univariate and multivariate Penalized Likelihood Models with Shared Log-Normal Frailty were used to determine the correlation between discharge time and relapse and to identify risk factors. A total of 683 relapses were observed in males, and 422 relapses in females. The Relapse Hazard Ratio (RHR) decreased with age (RHR = 0.99, CI: (0.98–0.998)) and with participants’ adherence to pharmacological treatment (HR = 0.71, CI: 0.58–0.86). In contrast, RHR increased with a history of suicide attempts (HR = 1.32, CI: 1.09–1.60), and a gradual compared to a sudden onset of disease (HR = 1.45, CI: 1.02–2.05). Gender was not predictive. Data indicate that preventive and therapeutic interventions may be particularly important for individuals who are younger at disease onset, have a history of suicide attempts, have experienced a gradual onset of disease, and have difficulties adhering to medication.

Development and external validation of an admission risk prediction model after treatment from early intervention in psychosis services

Early Intervention in psychosis (EIP) teams are the gold standard treatment for first-episode psychosis (FEP). EIP is time-limited and clinicians are required to make difficult aftercare decisions that require weighing up individuals' wishes for treatment, risk of relapse, and health service capacity. Reliable decision-making tools could assist with appropriate resource allocation and better care. We aimed to develop and externally validate a readmission risk tool for application at the point of EIP discharge. All persons from EIP caseloads in two NHS Trusts were eligible for the study. We excluded those who moved out of the area or were only seen for assessment. We developed a model to predict the risk of hospital admission within a year of ending EIP treatment in one Trust and externally validated it in another. There were n = 831 participants in the development dataset and n = 1393 in the external validation dataset, with 79 (9.5%) and 162 (11.6%) admissions to inpatient hospital, respectively. Discrimination was AUC = 0.76 (95% CI 0.75; 0.77) in the development dataset and AUC = 0.70 (95% CI 0.66; 0.75) in the external dataset. Calibration plots in external validation suggested an underestimation of risk in the lower predicted probabilities and slight overestimation at predicted probabilities in the 0.1-0.2 range (calibration slope = 0.86, 95% CI 0.68; 1.05). Recalibration improved performance at lower predicted probabilities but underestimated risk at the highest range of predicted probabilities (calibration slope = 1.00, 95% CI 0.79; 1.21). We showed that a tool for predicting admission risk using routine data has good performance and could assist clinical decision-making. Refinement of the model, testing its implementation and further external validation are needed.

A critical appraisal of the clinical applicability and risk of bias of the predictive models for mortality and recurrence in patients with oropharyngeal cancer: Systematic review

The use of predictive models is becoming widespread. However, these models should be developed appropriately (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies [CHARMS] and Prediction model Risk Of Bias ASsessment Tool [PROBAST] statements). Concerning mortality/recurrence in oropharyngeal cancer, we are not aware of any systematic reviews of the predictive models. We carried out a systematic review of the MEDLINE/EMBASE databases of those predictive models. In these models, we analyzed the 11 domains of the CHARMS statement and the risk of bias and applicability, using the PROBAST tool. Six papers were finally included in the systematic review and all of them presented high risk of bias and several limitations in the statistical analysis. The applicability was satisfactory in five out of six studies. None of the models could be considered ready for use in clinical practice.

Are we getting any better at staying better? The long view on relapse and recovery in first episode nonaffective psychosis and schizophrenia

Relapse in, and recovery from, schizophrenia has been acknowledged since the disease was first described. In this review the authors summarize the long-term (>100 years) data on relapse and recovery in schizophrenia by reviewing the extant older and modern relevant literature. The authors systematically question the utility of pharmacological and nonpharmacological interventions, with an emphasis on first episode nonaffective psychosis. The method used is a narrative review of earlier meta-analytic and systematic reviews. Antipsychotic medication discontinuation studies suggest a role for prophylactic maintenance treatment in the majority of people with schizophrenia, despite recent debate on this subject. The authors conclude that long-term outcomes, including relapse and recovery rates, have improved in the last 100 years, though prospectively identifying those people who do not require long-term antipsychotic treatment has not yet been possible. Data also suggests that interventions and outcomes during the first 5 years of the disease can influence the long-term schizophrenia trajectory.

Prognostic models for predicting overall survival in metastatic castration-resistant prostate cancer: a systematic review

PURPOSE

Prognostic models are developed to estimate the probability of the occurrence of future outcomes incorporating multiple variables. We aimed to identify and summarize existing multivariable prognostic models developed for predicting overall survival in patients with metastatic castration-resistant prostate cancer (mCRPC).

METHODS

The protocol was prospectively registered (CRD42017064448). We systematically searched Medline and reference lists up to May 2018 and included experimental and observational studies, which developed and/or internally validated prognostic models for mCRPC patients and were further externally validated or updated. The outcome of interest was overall survival. Two authors independently performed literature screening and quality assessment.

RESULTS

We included 12 studies that developed models including 8750 patients aged 42-95 years. Models included 4-11 predictor variables, mostly hemoglobin, baseline PSA, alkaline phosphatase, performance status, and lactate dehydrogenase. Very few incorporated Gleason score. Two models included predictors related to docetaxel and mitoxantrone treatments. Model performance after internal validation showed similar discrimination power ranging from 0.62 to 0.73. Overall survival models were mainly constructed as nomograms or risk groups/score. Two models obtained an overall judgment of low risk of bias.

CONCLUSIONS

Most models were not suitable for clinical use due to methodological shortcomings and lack of external validation. Further external validation and/or model updating is required to increase prognostic accuracy and clinical applicability prior to their incorporation in clinical practice as a useful tool in patient management.