The importance of external validation to advance precision psychiatry

Using Electronic Health Records to Facilitate Precision Psychiatry

The use of clinical prediction models to produce individualized risk estimates can facilitate the implementation of precision psychiatry. As a source of data from large, clinically representative patient samples, electronic health records (EHRs) provide a platform to develop and validate clinical prediction models, as well as potentially implement them in routine clinical care. The current review describes promising use cases for the application of precision psychiatry to EHR data and considers their performance in terms of discrimination (ability to separate individuals with and without the outcome) and calibration (extent to which predicted risk estimates correspond to observed outcomes), as well as their potential clinical utility (weighing benefits and costs associated with the model compared to different approaches across different assumptions of the number needed to test). We review 4 externally validated clinical prediction models designed to predict psychosis onset, psychotic relapse, cardiometabolic morbidity, and suicide risk. We then discuss the prospects for clinically implementing these models and the potential added value of integrating data from evidence syntheses, standardized psychometric assessments, and biological data into EHRs. Clinical prediction models can utilize routinely collected EHR data in an innovative way, representing a unique opportunity to inform real-world clinical decision making. Combining data from other sources (e.g., meta-analyses) or enhancing EHR data with information from research studies (clinical and biomarker data) may enhance our abilities to improve the performance of clinical prediction models.

Psychosis metabolic risk calculator (PsyMetRiC) in early psychosis: External validation study in Finland

INTRODUCTION

Accurate detection of cardiometabolic risk in early psychosis is crucial to reducing somatic morbidity and mortality in people with psychotic disorders. We conducted an external validation of the psychosis metabolic risk calculator (PsyMetRiC), a cardiometabolic risk prediction tool developed in the UK and tailored for young people with psychosis. We compared the predictive accuracy and clinical usefulness of PsyMetRiC and a general population-based risk prediction tool for type 2 diabetes, the Finnish Diabetes Risk Score (FINDRISC).

METHODS

We included first-episode psychosis and ultra-high-risk for psychosis patients without metabolic syndrome aged 18-35 years from the Helsinki Early Psychosis and Turku Early Psychosis Study cohorts. We tested two versions of PsyMetRiC: the full model including age, sex, ethnicity, body-mass index, smoking status, prescription of metabolically-active antipsychotic medication, high-density lipoprotein, and triglyceride concentrations, and the partial-model excluding biochemical predictors, and the simplified FINDRISC including BMI, sex, systolic blood pressure, and fasting glucose. Discrimination, calibration, and decision curve analyses were used to assess the predictive performance and clinical usefulness of both PsyMetRiC and FINDRISC. We performed a site-specific re-calibration of PsyMetRiC (PsyMetRiC-Fi).

RESULTS

The study sample consisted of 278 individuals (all White European ethnicity, 58.6% male, mean age 24.8 years, 37.8% smoking, mean BMI 23.5). Discrimination was marginally better in the PsyMetRiC full model (C = 0.72, 95% CI, 0.59-0.82) compared with partial model (C = 0.70, 95% CI 0.59-0.80) or FINDRISC (C = 0.63, 95% CI 0.54-0.71). Calibration plots displayed evidence of minor miscalibration for PsyMetRiC, which corrected following recalibration. Miscalibration was more pronounced for FINDRISC. Decision curve analysis showed that PsyMetRiC offers likely clinical usefulness in improving cardiometabolic risk management in early psychosis compared with giving everyone or no one an intervention.

CONCLUSION

PsyMetRiC has utility in predicting cardiometabolic risk in Finnish patients with early psychosis. It has better discriminatory accuracy and offers more accurate risk prediction compared to other available strategies.

Biomarkers for Psychosis: Are We There Yet? Umbrella Review of 1478 Biomarkers

Background and Hypothesis

This umbrella review aims to comprehensively synthesize the evidence of association between peripheral, electrophysiological, neuroimaging, neuropathological, and other biomarkers and diagnosis of psychotic disorders.

Study Design

We selected systematic reviews and meta-analyses of observational studies on diagnostic biomarkers for psychotic disorders, published until February 1, 2018. Data extraction was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. Evidence of association between biomarkers and psychotic disorders was classified as convincing, highly suggestive, suggestive, weak, or non-significant, using a standardized classification. Quality analyses used the Assessment of Multiple Systematic Reviews (AMSTAR) tool.

Study Results

The umbrella review included 110 meta-analyses or systematic reviews corresponding to 3892 individual studies, 1478 biomarkers, and 392 210 participants. No factor showed a convincing level of evidence. Highly suggestive evidence was observed for transglutaminase autoantibodies levels (odds ratio [OR] = 7.32; 95% CI: 3.36, 15.94), mismatch negativity in auditory event-related potentials (standardized mean difference [SMD] = 0.73; 95% CI: 0.5, 0.96), P300 component latency (SMD = -0.6; 95% CI: -0.83, -0.38), ventricle-brain ratio (SMD = 0.61; 95% CI: 0.5, 0.71), and minor physical anomalies (SMD = 0.99; 95% CI: 0.64, 1.34). Suggestive evidence was observed for folate, malondialdehyde, brain-derived neurotrophic factor, homocysteine, P50 sensory gating (P50 S2/S1 ratio), frontal N-acetyl-aspartate, and high-frequency heart rate variability. Among the remaining biomarkers, weak evidence was found for 626 and a non-significant association for 833 factors.

Conclusions

While several biomarkers present highly suggestive or suggestive evidence of association with psychotic disorders, methodological biases, and underpowered studies call for future higher-quality research.

The Translational Future of Stress Neurobiology and Psychosis Vulnerability: A Review of the Evidence

Psychosocial stress is a well-established risk factor for psychosis, yet the neurobiological mechanisms underlying this relationship have yet to be fully elucidated. Much of the research in this field has investigated hypothalamic-pituitary-adrenal (HPA) axis function and immuno-inflammatory processes among individuals with established psychotic disorders. However, as such studies are limited in their ability to provide knowledge that can be used to develop preventative interventions, it is important to shift the focus to individuals with increased vulnerability for psychosis (i.e., high-risk groups). In the present article, we provide an overview of the current methods for identifying individuals at high-risk for psychosis and review the psychosocial stressors that have been most consistently associated with psychosis risk. We then describe a network of interacting physiological systems that are hypothesised to mediate the relationship between psychosocial stress and the manifestation of psychotic illness and critically review evidence that abnormalities within these systems characterise highrisk populations. We found that studies of high-risk groups have yielded highly variable findings, likely due to (i) the heterogeneity both within and across high-risk samples, (ii) the diversity of psychosocial stressors implicated in psychosis, and (iii) that most studies examine single markers of isolated neurobiological systems. We propose that to move the field forward, we require well-designed, largescale translational studies that integrate multi-domain, putative stress-related biomarkers to determine their prognostic value in high-risk samples. We advocate that such investigations are highly warranted, given that psychosocial stress is undoubtedly a relevant risk factor for psychotic disorders.