Bigger Data, Harder Questions-Opportunities Throughout Mental Health Care

Illusory generalizability of clinical prediction models

It is widely hoped that statistical models can improve decision-making related to medical treatments. Because of the cost and scarcity of medical outcomes data, this hope is typically based on investigators observing a model's success in one or two datasets or clinical contexts. We scrutinized this optimism by examining how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia. Models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. Pooling data across trials to predict outcomes in the trial left out did not improve predictions. These results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.

Machine learning prediction will be part of future treatment of depression

Machine learning (ML) is changing the way that medicine is practiced. While already clinically utilised in diagnostic radiology and outcome prediction in intensive care unit, ML approaches in psychiatry remain nascent. Implementing ML algorithms in psychiatry, particularly in the treatment of depression, is significantly more challenging than other areas of medicine in part because of the less demarcated disease nosology and greater variability in practice. Given the current exiguous capacity of clinicians to predict patient and treatment outcomes in depression, there is a significantly greater need for better predictive capability. Early studies have shown promising results. ML predictions were significantly better than chance within the sequenced treatment alternatives to relieve depression (STAR*D) trial (accuracy 64.6%, p < 0.0001) and combining medications to enhance depression outcomes (COMED) randomised Controlled Trial (RCT) (accuracy 59.6%, p = 0.043), with similar results found in larger scale, retrospective studies. The greater flexibility and dimensionality of ML approaches has been demonstrated in studies incorporating diverse input variables including electroencephalography scans, achieving 88% accuracy for treatment response, and cognitive test scores, achieving up to 72% accuracy for treatment response. The predicting response to depression treatment (PReDicT) trial tested ML informed prescribing of antidepressants against standard therapy and found there was both better outcomes for anxiety and functional endpoints despite the algorithm only having a balanced accuracy of 57.5%. Impeding the progress of ML algorithms in psychiatry are pragmatic hurdles, including accuracy, expense, acceptability and comprehensibility, and ethical hurdles, including medicolegal liability, clinical autonomy and data privacy. Notwithstanding impediments, it is clear that ML prediction algorithms will be part of depression treatment in the future and clinicians should be prepared for their arrival.

A generalizable connectome-based marker of in-scan sustained attention in neurodiverse youth

Difficulty with attention is an important symptom in many conditions in psychiatry, including neurodiverse conditions such as autism. There is a need to better understand the neurobiological correlates of attention and leverage these findings in healthcare settings. Nevertheless, it remains unclear if it is possible to build dimensional predictive models of attentional state in a sample that includes participants with neurodiverse conditions. Here, we use 5 datasets to identify and validate functional connectome-based markers of attention. In dataset 1, we use connectome-based predictive modeling and observe successful prediction of performance on an in-scan sustained attention task in a sample of youth, including participants with a neurodiverse condition. The predictions are not driven by confounds, such as head motion. In dataset 2, we find that the attention network model defined in dataset 1 generalizes to predict in-scan attention in a separate sample of neurotypical participants performing the same attention task. In datasets 3-5, we use connectome-based identification and longitudinal scans to probe the stability of the attention network across months to years in individual participants. Our results help elucidate the brain correlates of attentional state in youth and support the further development of predictive dimensional models of other clinically relevant phenotypes.

Functional Connectome-Based Predictive Modeling in Autism

Autism is a heterogeneous neurodevelopmental condition, and functional magnetic resonance imaging-based studies have helped advance our understanding of its effects on brain network activity. We review how predictive modeling, using measures of functional connectivity and symptoms, has helped reveal key insights into this condition. We discuss how different prediction frameworks can further our understanding of the brain-based features that underlie complex autism symptomatology and consider how predictive models may be used in clinical settings. Throughout, we highlight aspects of study interpretation, such as data decay and sampling biases, that require consideration within the context of this condition. We close by suggesting exciting future directions for predictive modeling in autism.

Influence of Short Video Application on College Students' Mental Health under Big Data Monitoring Environment

Short videos are increasingly being consumed by college students as crucial content in the age of big data since they are a perfect fit for this medium. Therefore, college students should place a high value on the utilization of short movies. In this study, a neural network is utilized to create a mental health prediction model for college students. The neural network is trained using its self-learning capability to map out the relationships between different elements and mental health. The enhanced algorithm minimizes the production of candidate item sets to some amount, as well as the algorithm's time and space requirements, significantly decreasing the initialization time of the transaction set. According to the research, the test sample's pattern recognition accuracy was 81.29%, whereas the training sample's accuracy for pattern recognition was 83.37%. The analysis's finding is that the enhanced mining algorithm offers a fresh approach to educating college students about their health.

Cross-trial prediction of depression remission using problem-solving therapy: A machine learning approach

BACKGROUND

Psychotherapy is a standard depression treatment; however, determining a patient's prognosis with therapy relies on clinical judgment that is subject to trial-and-error and provider variability.

PURPOSE

To develop machine learning (ML) algorithms to predict depression remission for patients undergoing 6 months of problem-solving therapy (PST).

METHOD

Using data from the treatment arm of 2 randomized trials, ML models were trained and validated on ENGAGE-2 (ClinicalTrials.gov, #NCT03841682) and tested on RAINBOW (ClinicalTrials.gov, #NCT02246413) for predictions at baseline and at 2-months. Primary outcome was depression remission using the Depression Symptom Checklist (SCL-20) score < 0.5 at 6 months. Predictor variables included baseline characteristics (sociodemographic, behavioral, clinical, psychosocial) and intervention engagement through 2-months.

RESULTS

Of the 26 candidate variables, 8 for baseline and 11 for 2-months were predictive of depression remission, and used to train the models. The best-performing model predicted remission with an accuracy significantly greater than chance in internal validation using the ENGAGE-2 cohort, at baseline [72.6% (SD = 3.6%), p < 0.0001] and at 2-months [72.3% (5.1%), p < 0.0001], and in external validation with the RAINBOW cohort at baseline [58.3% (0%), p < 0.0001] and at 2-months [62.3% (0%), p < 0.0001]. Model-agnostic explanations highlighted key predictors of depression remission at the cohort and patient levels, including female sex, lower self-reported sleep disturbance, lower sleep-related impairment, and lower negative problem orientation.

CONCLUSIONS

ML models using clinical and patient-reported data can predict depression remission for patients undergoing PST, affording opportunities for prospective identification of likely responders, and for developing personalized early treatment optimization along the patient care trajectory.

"Apples and pears are similar, but still different things." Bipolar disorder and schizophrenia- discrete disorders or just dimensions ?

Starting with the dichotomous view of Kraepelin, schizophrenia and bipolar disorder have traditionally been considered as separate entities. More recent, this taxonomic view of illnesses has been challenged and a continuum psychosis has been postulated based on genetic and neurobiological findings suggestive of a large overlap between disorders. In this paper we will review clinical and experimental data from genetics, morphology, phenomenology and illness progression demonstrating what makes schizophrenia and bipolar disorder different conditions, challenging the idea of the obsolescence of the categorical approach. However, perhaps it is also time to move beyond DSM and search for more refined clinical descriptions that could uncover clinical invariants matching better with molecular data. In the future, computational psychiatry employing artificial intelligence and machine learning might provide us a tool to overcome the gap between clinical descriptions (phenomenology) and neurobiology.

The promise of machine learning in predicting treatment outcomes in psychiatry

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

[Big Data, AI and Machine Learning for Precision Psychiatry: How are they changing the clinical practice?]

Currently, we are witnessing an increasing interest in predictive models and personalized diagnosis and treatment choice in psychiatric research. Against this background, the emerging field of Precision Psychiatry is trying to establish precise diagnostics and personalized therapy through Big Data. Electronic Health Records (EHR), smartphone-based data collection and advances in genotyping and imaging allow for a detailed clinical and neurobiological characterization of numerous patients. In order to revolutionize the treatment of psychiatric disorders, a personalization of psychiatry through machine learning (ML) and artificial intelligence (AI) is needed. We must therefore establish an AI ecosystem to develop and strictly validate custom-tailored AI and ML solutions. Furthermore, personalized predictions and detailed patient information must be integrated in AI-based Clinical Decision Support systems. Only in this way can Big Data, ML and AI support the clinician most effectively and help personalize treatment in psychiatry.

Machine Learning for Differential Diagnosis Between Clinical Conditions With Social Difficulty: Autism Spectrum Disorder, Early Psychosis, and Social Anxiety Disorder

Differential diagnosis in adult cohorts with social difficulty is confounded by comorbid mental health conditions, common etiologies, and shared phenotypes. Identifying shared and discriminating profiles can facilitate intervention and remediation strategies. The objective of the study was to identify salient features of a composite test battery of cognitive and mood measures using a machine learning paradigm in clinical cohorts with social interaction difficulties. We recruited clinical participants who met standardized diagnostic criteria for autism spectrum disorder (ASD: n = 62), early psychosis (EP: n = 48), or social anxiety disorder (SAD: N = 83) and compared them with a neurotypical comparison group (TYP: N = 43). Using five machine-learning algorithms and repeated cross-validation, we trained and tested classification models using measures of cognitive and executive function, lower- and higher-order social cognition and mood severity. Performance metrics were the area under the curve (AUC) and Brier Scores. Sixteen features successfully differentiated between the groups. The control versus social impairment cohorts (ASD, EP, SAD) were differentiated by social cognition, visuospatial memory and mood measures. Importantly, a distinct profile cluster drawn from social cognition, visual learning, executive function and mood, distinguished the neurodevelopmental cohort (EP and ASD) from the SAD group. The mean AUC range was between 0.891 and 0.916 for social impairment versus control cohorts and, 0.729 to 0.781 for SAD vs neurodevelopmental cohorts. This is the first study that compares an extensive battery of neuropsychological and self-report measures using a machine learning protocol in clinical and neurodevelopmental cohorts characterized by social impairment. Findings are relevant for diagnostic, intervention and remediation strategies for these groups.

Machine learning and big data analytics in bipolar disorder: A position paper from the International Society for Bipolar Disorders Big Data Task Force

OBJECTIVES

The International Society for Bipolar Disorders Big Data Task Force assembled leading researchers in the field of bipolar disorder (BD), machine learning, and big data with extensive experience to evaluate the rationale of machine learning and big data analytics strategies for BD.

METHOD

A task force was convened to examine and integrate findings from the scientific literature related to machine learning and big data based studies to clarify terminology and to describe challenges and potential applications in the field of BD. We also systematically searched PubMed, Embase, and Web of Science for articles published up to January 2019 that used machine learning in BD.

RESULTS

The results suggested that big data analytics has the potential to provide risk calculators to aid in treatment decisions and predict clinical prognosis, including suicidality, for individual patients. This approach can advance diagnosis by enabling discovery of more relevant data-driven phenotypes, as well as by predicting transition to the disorder in high-risk unaffected subjects. We also discuss the most frequent challenges that big data analytics applications can face, such as heterogeneity, lack of external validation and replication of some studies, cost and non-stationary distribution of the data, and lack of appropriate funding.

CONCLUSION

Machine learning-based studies, including atheoretical data-driven big data approaches, provide an opportunity to more accurately detect those who are at risk, parse-relevant phenotypes as well as inform treatment selection and prognosis. However, several methodological challenges need to be addressed in order to translate research findings to clinical settings.

Machine learning and big data in psychiatry: toward clinical applications

Psychiatry is a medical field concerned with the treatment of mental illness. Psychiatric disorders broadly relate to higher functions of the brain, and as such are richly intertwined with social, cultural, and experiential factors. This makes them exquisitely complex phenomena that depend on and interact with a large number of variables. Computational psychiatry provides two ways of approaching this complexity. Theory-driven computational approaches employ mechanistic models to make explicit hypotheses at multiple levels of analysis. Data-driven machine-learning approaches can make predictions from high-dimensional data and are generally agnostic as to the underlying mechanisms. Here, we review recent advances in the use of big data and machine-learning approaches toward the aim of alleviating the suffering that arises from psychiatric disorders.

Reproducible grey matter patterns index a multivariate, global alteration of brain structure in schizophrenia and bipolar disorder

Schizophrenia is a severe mental disorder characterized by numerous subtle changes in brain structure and function. Machine learning allows exploring the utility of combining structural and functional brain magnetic resonance imaging (MRI) measures for diagnostic application, but this approach has been hampered by sample size limitations and lack of differential diagnostic data. Here, we performed a multi-site machine learning analysis to explore brain structural patterns of T1 MRI data in 2668 individuals with schizophrenia, bipolar disorder or attention-deficit/ hyperactivity disorder, and healthy controls. We found reproducible changes of structural parameters in schizophrenia that yielded a classification accuracy of up to 76% and provided discrimination from ADHD, through it lacked specificity against bipolar disorder. The observed changes largely indexed distributed grey matter alterations that could be represented through a combination of several global brain-structural parameters. This multi-site machine learning study identified a brain-structural signature that could reproducibly differentiate schizophrenia patients from controls, but lacked specificity against bipolar disorder. While this currently limits the clinical utility of the identified signature, the present study highlights that the underlying alterations index substantial global grey matter changes in psychotic disorders, reflecting the biological similarity of these conditions, and provide a roadmap for future exploration of brain structural alterations in psychiatric patients.

Risk Assessment Tools and Data-Driven Approaches for Predicting and Preventing Suicidal Behavior

Risk assessment of suicidal behavior is a time-consuming but notoriously inaccurate activity for mental health services globally. In the last 50 years a large number of tools have been designed for suicide risk assessment, and tested in a wide variety of populations, but studies show that these tools suffer from low positive predictive values. More recently, advances in research fields such as machine learning and natural language processing applied on large datasets have shown promising results for health care, and may enable an important shift in advancing precision medicine. In this conceptual review, we discuss established risk assessment tools and examples of novel data-driven approaches that have been used for identification of suicidal behavior and risk. We provide a perspective on the strengths and weaknesses of these applications to mental health-related data, and suggest research directions to enable improvement in clinical practice.

The Science of Prognosis in Psychiatry: A Review

IMPORTANCE

Prognosis is a venerable component of medical knowledge introduced by Hippocrates (460-377 BC). This educational review presents a contemporary evidence-based approach for how to incorporate clinical risk prediction models in modern psychiatry. The article is organized around key methodological themes most relevant for the science of prognosis in psychiatry. Within each theme, the article highlights key challenges and makes pragmatic recommendations to improve scientific understanding of prognosis in psychiatry.

OBSERVATIONS

The initial step to building clinical risk prediction models that can affect psychiatric care involves designing the model: preparation of the protocol and definition of the outcomes and of the statistical methods (theme 1). Further initial steps involve carefully selecting the predictors, preparing the data, and developing the model in these data. A subsequent step is the validation of the model to accurately test its generalizability (theme 2). The next consideration is that the accuracy of the clinical prediction model is affected by the incidence of the psychiatric condition under investigation (theme 3). Eventually, clinical prediction models need to be implemented in real-world clinical routine, and this is usually the most challenging step (theme 4). Advanced methods such as machine learning approaches can overcome some problems that undermine the previous steps (theme 5). The relevance of each of these themes to current clinical risk prediction modeling in psychiatry is discussed and recommendations are given.

CONCLUSIONS AND RELEVANCE

Together, these perspectives intend to contribute to an integrative, evidence-based science of prognosis in psychiatry. By focusing on the outcome of the individuals, rather than on the disease, clinical risk prediction modeling can become the cornerstone for a scientific and personalized psychiatry.

Association between physical exercise and mental health in 1·2 million individuals in the USA between 2011 and 2015: a cross-sectional study

BACKGROUND

Exercise is known to be associated with reduced risk of all-cause mortality, cardiovascular disease, stroke, and diabetes, but its association with mental health remains unclear. We aimed to examine the association between exercise and mental health burden in a large sample, and to better understand the influence of exercise type, frequency, duration, and intensity.

METHODS

In this cross-sectional study, we analysed data from 1 237 194 people aged 18 years or older in the USA from the 2011, 2013, and 2015 Centers for Disease Control and Prevention Behavioral Risk Factors Surveillance System survey. We compared the number of days of bad self-reported mental health between individuals who exercised and those who did not, using an exact non-parametric matching procedure to balance the two groups in terms of age, race, gender, marital status, income, education level, body-mass index category, self-reported physical health, and previous diagnosis of depression. We examined the effects of exercise type, duration, frequency, and intensity using regression methods adjusted for potential confounders, and did multiple sensitivity analyses.

FINDINGS

Individuals who exercised had 1·49 (43·2%) fewer days of poor mental health in the past month than individuals who did not exercise but were otherwise matched for several physical and sociodemographic characteristics (W=7·42 × 10¹⁰, p<2·2 × 10^-16). All exercise types were associated with a lower mental health burden (minimum reduction of 11·8% and maximum reduction of 22·3%) than not exercising (p<2·2 × 10^-16 for all exercise types). The largest associations were seen for popular team sports (22·3% lower), cycling (21·6% lower), and aerobic and gym activities (20·1% lower), as well as durations of 45 min and frequencies of three to five times per week.

INTERPRETATION

In a large US sample, physical exercise was significantly and meaningfully associated with self-reported mental health burden in the past month. More exercise was not always better. Differences as a function of exercise were large relative to other demographic variables such as education and income. Specific types, durations, and frequencies of exercise might be more effective clinical targets than others for reducing mental health burden, and merit interventional study.

FUNDING

Cloud computing resources were provided by Microsoft.

Predicting Barriers to Treatment for Depression in a U.S. National Sample: A Cross-Sectional, Proof-of-Concept Study

OBJECTIVE

Even though safe and effective treatments for depression are available, many individuals with a diagnosis of depression do not obtain treatment. This study aimed to develop a tool to identify persons who might not initiate treatment among those who acknowledge a need.

METHODS

Data were aggregated from the 2008-2014 U.S. National Survey on Drug Use and Health (N=391,753), including 20,785 adults given a diagnosis of depression by a health care provider in the 12 months before the survey. Machine learning was applied to self-report survey items to develop strategies for identifying individuals who might not get needed treatment.

RESULTS

A derivation cohort aggregated between 2008 and 2013 was used to develop a model that identified the 30.6% of individuals with depression who reported needing but not getting treatment. When applied to independent responses from the 2014 cohort, the model identified 72% of those who did not initiate treatment (p<.01), with a balanced accuracy that was also significantly above chance (71%, p<.01). For individuals who did not get treatment, the model predicted 10 (out of 15) reasons that they endorsed as barriers to treatment, with balanced accuracies between 53% and 65% (p<.05 for all).

CONCLUSIONS

Considerable work is needed to improve follow-up and retention rates after the critical initial meeting in which a patient is given a diagnosis of depression. Routinely collected information about patients with depression could identify those at risk of not obtaining needed treatment, which may inform the development and implementation of interventions to reduce the prevalence of untreated depression.

Digital apothecaries: a vision for making health care interventions accessible worldwide

Evidence-based psychological interventions are growing in number but are not within reach of many individuals who could benefit from them. The recent revolution in digital technologies now makes it possible to reach people around the globe with digital interventions in the form of web sites, mobile applications, wearable devices, and so on. Although a plethora of digital interventions are available online few are evidence-based and individuals have little guidance to decide among the multitude of options. We propose the development of "digital apothecaries," that is, online repositories of evidence-based digital interventions. As portals to effective interventions, digital apothecaries would be useful to individuals who could access evidence-based interventions directly, to health care providers, who could identify specific digital tools to suggest to or use with their patients, and to researchers, who could study a range of tools with large samples, enabling comparative tests and evaluation of moderators of effects. We present a taxonomy of types of in-person and digital interventions ranging from traditional therapy without the use of digital tools to totally automated self-help interventions. This taxonomy highlights the potential of blending digital tools into health care systems to expand their reach. Digital apothecaries would provide access to evidence-based digital interventions (both free and paid versions), provide data on effectiveness (including effectiveness for diverse populations), and encourage the development and testing of more such tools. Other issues discussed include: criteria for inclusion of interventions into digital apothecaries; how digital tools could enhance health care for diverse populations; and cautionary notes regarding potential negative unintended consequences of the adoption of digital interventions into the health care system. In particular, we warn about the potential misuse of evidence-based digital interventions to justify reducing access to live providers. Digital apothecaries bring with them the promise of reducing health disparities by reaching large numbers of individuals across the world who need health interventions but are not currently receiving them. The health care field is encouraged to mindfully develop this promise, while being alert not to cause inadvertent harm.

Conceptual endophenotypes: A strategy to advance the impact of psychoneuroendocrinology in precision medicine

Psychobiological research has generated a tremendous amount of findings on the psychological, neuroendocrine, molecular and environmental processes that are directly relevant for mental and physical health, but have overwhelmed our capacity to meaningfully absorb, integrate, and utilize this knowledge base. Here, we reflect about suitable strategies to improve the translational success of psychoneuroendocrinological research in the era of precision medicine. Following a strategy advocated by the National Research Council and the tradition of endophenotype-based research, we advance here a new approach, termed "conceptual endophenotypes". We define the contextual and formal criteria of conceptual endophenotypes, outline criteria for filtering and selecting information, and describe how conceptual endophenotypes can be validated and implemented at the bedside. As proof-of-concept, we describe some of our findings from research that has adopted this approach in the context of stress-related disorders. We argue that conceptual endophenotypes engineer a bridge between the bench and the bedside. This approach readily lends itself to being continuously developed and implemented. Recent methodological advances, including digital phenotyping, machine learning, grassroots collaboration, and a learning healthcare system, may accelerate the development and implementation of this conceptual endophenotype approach.

Prediction complements explanation in understanding the developing brain

A central aim of human neuroscience is understanding the neurobiology of cognition and behavior. Although we have made significant progress towards this goal, reliance on group-level studies of the developed adult brain has limited our ability to explain population variability and developmental changes in neural circuitry and behavior. In this review, we suggest that predictive modeling, a method for predicting individual differences in behavior from brain features, can complement descriptive approaches and provide new ways to account for this variability. Highlighting the outsized scientific and clinical benefits of prediction in developmental populations including adolescence, we show that predictive brain-based models are already providing new insights on adolescent-specific risk-related behaviors. Together with large-scale developmental neuroimaging datasets and complementary analytic approaches, predictive modeling affords us the opportunity and obligation to identify novel treatment targets and individually tailor the course of interventions for developmental psychopathologies that impact so many young people today.