Toward dynamic phenotypes and the scalable measurement of human behavior

Cognitive Fluctuations in a Rare Disease Population: Leveraging Cognitive and Speech Ecological Momentary Assessment in Individuals with Phenylketonuria

BACKGROUND

Phenylketonuria (PKU) is a rare, hereditary disease that causes disruption in phenylalanine (Phe) metabolism. Despite early intervention, individuals with PKU may have difficulty in several different cognitive domains, including verbal fluency, processing speed, and executive functioning.

OBJECTIVE

The overarching goal of the Evaluating Fluctuations in Cognitive and Speech Characteristics in Phenylketonuria study (CSP Study) is to characterize the relationships among cognition, speech, mood, and blood-based biomarkers (Phe, Tyr) in individuals with early treated PKU. We describe our initial optimization pilot results that are guiding the ongoing CSP Study, while establishing feasibility and reliability of using ecological momentary assessment (EMA) in this clinical population.

METHODS

Twenty adults with PKU were enrolled in this study between December 2022 and March 2023 through the National PKU Alliance. Eighteen participants completed an extended baseline assessment followed by six EMAs over one month. The EMAs included digital cognitive tests measuring processing speed, sustained attention, executive functioning, as well as speech (semantic fluency) and mood measures. Participants had 60 minutes to complete the assessment; completion rates were around 70% (on average 4.78 out of 6 EMAs).

RESULTS

Completion rates of EMAs were above 70%, with stable performances across baseline measures and EMAs. Between person reliability (BPR) of the EMAs, representing the variance due to differences between individuals versus within individuals, is satisfactory with values close to (semantic fluency BPR: 0.7, sustained attention BPR: 0.72) or exceeding (processing speed: 0.93, executive functioning: 0.88) those data collected from a large normative database (N= 5039-10703), as well as slightly below or matching a prior study using a clinical group (N=18). As applicable, within person reliability was also computed; we demonstrated strong reliability for processing speed (0.87). A control analyses ensured that time of day (i.e., morning, afternoon, evening) did not impact performance; performance on tasks did not decrease if tested earlier versus later in the day (all ps>0.09). Similarly, to assess variability in task performance over the course of all EMAs, the coefficient of variability was computed: 28% for the task measuring sustained attention, 37% percent for semantic fluency, 15.8 % for the task measuring executive functioning, and 17.6% for processing speed. Performance appears more stable in tasks measuring processing speed and executive functioning than on tasks of sustained attention and semantic fluency.

CONCLUSIONS

Preliminary results of this study demonstrate strong reliability of cognitive EMA, indicating that EMA is a promising tool for evaluating fluctuations in cognitive status in this population. Future work should refine and expand the utility of these digital tools, determine how variable EMA frequencies might better characterize changes in functioning as they relate to blood-based biomarkers, and validate a singular battery that could be rapidly administered at scale and in clinical trials to determine progression of disease.

No Evidence of Reliability Across 36 Variations of the Emotional Dot-Probe Task in 9,600 Participants

The emotional dot-probe task is a widely used measure of attentional bias to threat. Recent work suggests, however, that subtraction-based behavioral measures of emotional dot-probe performance may not be appropriate for measuring such attentional biases due to poor reliability. In the two current studies, we systematically tested thirty-six versions of the emotional dot-probe that varied in stimuli (faces, scenes, snakes/spiders), timings (stimulus onset asynchrony of 100, 500, 900 milliseconds), stimulus orientations (horizontal, vertical), and trial types (e.g., threat congruent and threat incongruent). Across 9,600 participants, none of the 36 versions demonstrated internal reliability greater than zero. Reliability was similarly poor in anxious participants (based on Generalized Anxiety Disorder 7 Items or Brief Hypervigilance Scale). We conclude that the standard behavioral scores (reaction time- or accuracy-based difference scores) derived from the emotional dot-probe are not adequately reliable measures of attentional biases to threat in anxious or nonanxious populations.

Recent advances in understanding how compulsivity is related to behavioural addictions over their timecourse

Purpose of Review

Behavioural addictions involve loss of control over initially rewarding behaviours, which continue despite adverse consequences. Theoretical models suggest that these patterns of behaviour evolve over time, with compulsive and habitual behaviours held to reflect a loss of behavioural control. Compulsivity can be broadly described as a propensity for (or engagement in) repetitive behaviours that are not aligned with overall goals. Here, we consider whether compulsivity is associated with behavioural addictions at different stages of their development, based on self-report and neurocognitive measures.

Recent Findings

This review found that there is initial evidence that compulsive traits might predispose individuals to engage in problematic behaviours, and that self-report and neurocognitive measures of compulsivity are associated with severity of problematic behaviours even in the early stages of behavioural addictions. In the later stages of behavioural addiction, there is strong evidence for an association of gambling disorder with cognitive inflexibility, but less evidence for an association between compulsivity and other types of behavioural addiction.

Summary

Moving forwards, well-powered longitudinal studies, including studies using ecological momentary assessment (EMA), will be important in robustly developing our understanding of how compulsivity is related to behavioural addictions over their timecourse.

Increasing the Construct Validity of Computational Phenotypes of Mental Illness Through Active Inference and Brain Imaging

After more than 30 years since its inception, the utility of brain imaging for understanding and diagnosing mental illnesses is in doubt, receiving well-grounded criticisms from clinical practitioners. Symptom-based correlational approaches have struggled to provide psychiatry with reliable brain-imaging metrics. However, the emergence of computational psychiatry has paved a new path not only for understanding the psychopathology of mental illness but also to provide practical tools for clinical practice in terms of computational metrics, specifically computational phenotypes. However, these phenotypes still lack sufficient test-retest reliability. In this review, we describe recent works revealing that mind and brain-related computational phenotypes show structural (not random) variation over time, longitudinal changes. Furthermore, we show that these findings suggest that understanding the causes of these changes will improve the construct validity of the phenotypes with an ensuing increase in test-retest reliability. We propose that the active inference framework offers a general-purpose approach for causally understanding these longitudinal changes by incorporating brain imaging as observations within partially observable Markov decision processes.

Dynamic associations between glucose and ecological momentary cognition in Type 1 Diabetes

Type 1 diabetes (T1D) is a chronic condition characterized by glucose fluctuations. Laboratory studies suggest that cognition is reduced when glucose is very low (hypoglycemia) and very high (hyperglycemia). Until recently, technological limitations prevented researchers from understanding how naturally-occurring glucose fluctuations impact cognitive fluctuations. This study leveraged advances in continuous glucose monitoring (CGM) and cognitive ecological momentary assessment (EMA) to characterize dynamic, within-person associations between glucose and cognition in naturalistic environments. Using CGM and EMA, we obtained intensive longitudinal measurements of glucose and cognition (processing speed, sustained attention) in 200 adults with T1D. First, we used hierarchical Bayesian modeling to estimate dynamic, within-person associations between glucose and cognition. Consistent with laboratory studies, we hypothesized that cognitive performance would be reduced at low and high glucose, reflecting cognitive vulnerability to glucose fluctuations. Second, we used data-driven lasso regression to identify clinical characteristics that predicted individual differences in cognitive vulnerability to glucose fluctuations. Large glucose fluctuations were associated with slower and less accurate processing speed, although slight glucose elevations (relative to person-level means) were associated with faster processing speed. Glucose fluctuations were not related to sustained attention. Seven clinical characteristics predicted individual differences in cognitive vulnerability to glucose fluctuations: age, time in hypoglycemia, lifetime severe hypoglycemic events, microvascular complications, glucose variability, fatigue, and neck circumference. Results establish the impact of glucose on processing speed in naturalistic environments, suggest that minimizing glucose fluctuations is important for optimizing processing speed, and identify several clinical characteristics that may exacerbate cognitive vulnerability to glucose fluctuations.

Untapped Potential of Unobtrusive Observation for Studying Health Behaviors

Improving the environment is an important upstream intervention to promote population health by influencing health behaviors such as physical activity, smoking, and social distancing. Examples of promising environmental interventions include creating high-quality green spaces, building active transport infrastructure, and implementing urban planning regulations. However, there is little robust evidence to inform policy and decision makers about what kinds of environmental interventions are effective and for which populations. In this viewpoint, we make the case that this evidence gap exists partly because health behavior research is dominated by obtrusive methods that focus on studying individual behavior and that are less suitable for understanding environmental influences. In contrast, unobtrusive observation can assess how behavior varies in different environmental contexts. It thereby provides valuable data relating to how environments affect the behavior of populations, which is often useful knowledge for effectively and equitably tackling population health challenges such as obesity and noncommunicable diseases. Yet despite a long history, unobtrusive observation methods are currently underused in health behavior research. We discuss how developing the use of video technology and automated computer vision techniques can offer a scalable solution for assessing health behaviors, facilitating a more thorough investigation of how environments influence health behaviors. We also reflect on the important ethical challenges associated with unobtrusive observation and the use of these emerging video technologies. By increasing the use of unobtrusive observation alongside other methods, we strongly believe this will improve our understanding of the influences of the environment on health behaviors.

Returning Individual Research Results from Digital Phenotyping in Psychiatry

Psychiatry is rapidly adopting digital phenotyping and artificial intelligence/machine learning tools to study mental illness based on tracking participants' locations, online activity, phone and text message usage, heart rate, sleep, physical activity, and more. Existing ethical frameworks for return of individual research results (IRRs) are inadequate to guide researchers for when, if, and how to return this unprecedented number of potentially sensitive results about each participant's real-world behavior. To address this gap, we convened an interdisciplinary expert working group, supported by a National Institute of Mental Health grant. Building on established guidelines and the emerging norm of returning results in participant-centered research, we present a novel framework specific to the ethical, legal, and social implications of returning IRRs in digital phenotyping research. Our framework offers researchers, clinicians, and Institutional Review Boards (IRBs) urgently needed guidance, and the principles developed here in the context of psychiatry will be readily adaptable to other therapeutic areas.

A canonical trajectory of executive function maturation from adolescence to adulthood

Theories of human neurobehavioral development suggest executive functions mature from childhood through adolescence, underlying adolescent risk-taking and the emergence of psychopathology. Investigations with relatively small datasets or narrow subsets of measures have identified general executive function development, but the specific maturational timing and independence of potential executive function subcomponents remain unknown. Integrating four independent datasets (N = 10,766; 8-35 years old) with twenty-three measures from seventeen tasks, we provide a precise charting, multi-assessment investigation, and replication of executive function development from adolescence to adulthood. Across assessments and datasets, executive functions follow a canonical non-linear trajectory, with rapid and statistically significant development in late childhood to mid-adolescence (10-15 years old), before stabilizing to adult-levels in late adolescence (18-20 years old). Age effects are well captured by domain-general processes that generate reproducible developmental templates across assessments and datasets. Results provide a canonical trajectory of executive function maturation that demarcates the boundaries of adolescence and can be integrated into future studies.

Accurate Prediction of Momentary Cognition From Intensive Longitudinal Data

BACKGROUND

Deficits in cognitive performance are implicated in the development and maintenance of psychopathology. Emerging evidence further suggests that within-person fluctuations in cognitive performance may represent sensitive early markers of neuropsychiatric decline. Incorporating routine cognitive assessments into standard clinical care-to identify between-person differences and monitor within-person fluctuations-has the potential to improve diagnostic screening and treatment planning. In support of these goals, it is critical to understand to what extent cognitive performance varies under routine, remote assessment conditions (i.e., momentary cognition) in relation to a wide range of possible predictors.

METHODS

Using data-driven, high-dimensional methods, we ranked strong predictors of momentary cognition and evaluated out-of-sample predictive accuracy. Our approach leveraged innovations in digital technology, including ambulatory assessment of cognition and behavior 1) at scale (n = 122 participants, n = 94 females), 2) in naturalistic environments, and 3) within an intensive longitudinal study design (mean = 25.5 assessments/participant).

RESULTS

Reaction time (R2 > 0.70) and accuracy (0.56 >R2 > 0.35) were strongly predicted by age, between-person differences in mean performance, and time of day. Effects of self-reported, intraindividual fluctuations in environmental (e.g., noise) and internal (e.g., stress) states were also observed.

CONCLUSIONS

Our results provide robust estimates of effect size to characterize sources of cognitive variability, to support the identification of optimal windows for psychosocial interventions, and to possibly inform clinical evaluation under remote neuropsychological assessment conditions.

Improving the Reliability of Cognitive Task Measures: A Narrative Review

Cognitive tasks are capable of providing researchers with crucial insights into the relationship between cognitive processing and psychiatric phenomena. However, many recent studies have found that task measures exhibit poor reliability, which hampers their usefulness for individual differences research. Here, we provide a narrative review of approaches to improve the reliability of cognitive task measures. Specifically, we introduce a taxonomy of experiment design and analysis strategies for improving task reliability. Where appropriate, we highlight studies that are exemplary for improving the reliability of specific task measures. We hope that this article can serve as a helpful guide for experimenters who wish to design a new task, or improve an existing one, to achieve sufficient reliability for use in individual differences research.

Ecological Momentary Assessment of Cognition in Clinical and Community Samples: Reliability and Validity Study

BACKGROUND

The current methods of evaluating cognitive functioning typically rely on a single time point to assess and characterize an individual's performance. However, cognitive functioning fluctuates within individuals over time in relation to environmental, psychological, and physiological contexts. This limits the generalizability and diagnostic utility of single time point assessments, particularly among individuals who may exhibit large variations in cognition depending on physiological or psychological context (eg, those with type 1 diabetes [T1D], who may have fluctuating glucose concentrations throughout the day).

OBJECTIVE

We aimed to report the reliability and validity of cognitive ecological momentary assessment (EMA) as a method for understanding between-person differences and capturing within-person variation in cognition over time in a community sample and sample of adults with T1D.

METHODS

Cognitive performance was measured 3 times a day for 15 days in the sample of adults with T1D (n=198, recruited through endocrinology clinics) and for 10 days in the community sample (n=128, recruited from TestMyBrain, a web-based citizen science platform) using ultrabrief cognitive tests developed for cognitive EMA. Our cognitive EMA platform allowed for remote, automated assessment in participants' natural environments, enabling the measurement of within-person cognitive variation without the burden of repeated laboratory or clinic visits. This allowed us to evaluate reliability and validity in samples that differed in their expected degree of cognitive variability as well as the method of recruitment.

RESULTS

The results demonstrate excellent between-person reliability (ranging from 0.95 to 0.99) and construct validity of cognitive EMA in both the sample of adults with T1D and community sample. Within-person reliability in both samples (ranging from 0.20 to 0.80) was comparable with that observed in previous studies in healthy older adults. As expected, the full-length baseline and EMA versions of TestMyBrain tests correlated highly with one another and loaded together on the expected cognitive domains when using exploratory factor analysis. Interruptions had higher negative impacts on accuracy-based outcomes (β=-.34 to -.26; all P values <.001) than on reaction time-based outcomes (β=-.07 to -.02; P<.001 to P=.40).

CONCLUSIONS

We demonstrated that ultrabrief mobile assessments are both reliable and valid across 2 very different clinic versus community samples, despite the conditions in which cognitive EMAs are administered, which are often associated with more noise and variability. The psychometric characteristics described here should be leveraged appropriately depending on the goals of the cognitive assessment (eg, diagnostic vs everyday functioning) and the population being studied.

Individual differences in computational psychiatry: a review of current challenges

Bringing precision to the understanding and treatment of mental disorders requires instruments for studying clinically relevant individual differences. One promising approach is the development of computational assays: integrating computational models with cognitive tasks to infer latent patient-specific disease processes in brain computations. While recent years have seen many methodological advancements in computational modelling and many cross-sectional patient studies, much less attention has been paid to basic psychometric properties (reliability and construct validity) of the computational measures provided by the assays. In this review, we assess the extent of this issue by examining emerging empirical evidence. We find that many computational measures suffer from poor psychometric properties, which poses a risk of invalidating previous findings and undermining ongoing research efforts using computational assays to study individual (and even group) differences. We provide recommendations for how to address these problems and, crucially, embed them within a broader perspective on key developments that are needed for translating computational assays to clinical practice.

Developments in scalable strategies for detecting early markers of cognitive decline

Effective strategies for early detection of cognitive decline, if deployed on a large scale, would have individual and societal benefits. However, current detection methods are invasive or time-consuming and therefore not suitable for longitudinal monitoring of asymptomatic individuals. For example, biological markers of neuropathology associated with cognitive decline are typically collected via cerebral spinal fluid, cognitive functioning is evaluated from face-to-face assessments by experts and brain measures are obtained using expensive, non-portable equipment. Here, we describe scalable, repeatable, relatively non-invasive and comparatively inexpensive strategies for detecting the earliest markers of cognitive decline. These approaches are characterized by simple data collection protocols conducted in locations outside the laboratory: measurements are collected passively, by the participants themselves or by non-experts. The analysis of these data is, in contrast, often performed in a centralized location using sophisticated techniques. Recent developments allow neuropathology associated with potential cognitive decline to be accurately detected from peripheral blood samples. Advances in smartphone technology facilitate unobtrusive passive measurements of speech, fine motor movement and gait, that can be used to predict cognitive decline. Specific cognitive processes can be assayed using 'gamified' versions of standard laboratory cognitive tasks, which keep users engaged across multiple test sessions. High quality brain data can be regularly obtained, collected at-home by users themselves, using portable electroencephalography. Although these methods have great potential for addressing an important health challenge, there are barriers to be overcome. Technical obstacles include the need for standardization and interoperability across hardware and software. Societal challenges involve ensuring equity in access to new technologies, the cost of implementation and of any follow-up care, plus ethical issues.

Neuroscience from the comfort of your home: Repeated, self-administered wireless dry EEG measures brain function with high fidelity

Recent advances have enabled the creation of wireless, “dry” electroencephalography (EEG) recording systems, and easy-to-use engaging tasks, that can be operated repeatedly by naïve users, unsupervised in the home. Here, we evaluated the validity of dry-EEG, cognitive task gamification, and unsupervised home-based recordings used in combination. Two separate cohorts of participants—older and younger adults—collected data at home over several weeks using a wireless dry EEG system interfaced with a tablet for task presentation. Older adults (n = 50; 25 females; mean age = 67.8 years) collected data over a 6-week period. Younger male adults (n = 30; mean age = 25.6 years) collected data over a 4-week period. All participants were asked to complete gamified versions of a visual Oddball task and Flanker task 5–7 days per week. Usability of the EEG system was evaluated via participant adherence, percentage of sessions successfully completed, and quantitative feedback using the System Usability Scale. In total, 1,449 EEG sessions from older adults (mean = 28.9; SD = 6.64) and 684 sessions from younger adults (mean = 22.87; SD = 1.92) were collected. Older adults successfully completed 93% of sessions requested and reported a mean usability score of 84.5. Younger adults successfully completed 96% of sessions and reported a mean usability score of 88.3. Characteristic event-related potential (ERP) components—the P300 and error-related negativity—were observed in the Oddball and Flanker tasks, respectively. Using a conservative threshold for inclusion of artifact-free data, 50% of trials were rejected per at-home session. Aggregation of ERPs across sessions (2–4, depending on task) resulted in grand average signal quality with similar Standard Measurement Error values to those of single-session wet EEG data collected by experts in a laboratory setting from a young adult sample. Our results indicate that easy-to-use task-driven EEG can enable large-scale investigations in cognitive neuroscience. In future, this approach may be useful in clinical applications such as screening and tracking of treatment response.

Digital tools for the assessment of pharmacological treatment for depressive disorder: State of the art

Depression is an invalidating disorder, marked by phenotypic heterogeneity. Clinical assessments for treatment adjustments and data-collection for pharmacological research often rely on subjective representations of functioning. Better phenotyping through digital applications may add unseen information and facilitate disentangling the clinical characteristics and impact of depression and its pharmacological treatment in everyday life. Researchers, physicians, and patients benefit from well-understood digital phenotyping approaches to assess the treatment efficacy and side-effects. This review discusses the current possibilities and pitfalls of wearables and technology for the assessment of the pharmacological treatment of depression. Their applications in the whole spectrum of treatment for depression, including diagnosis, treatment of an episode, and monitoring of relapse risk and prevention are discussed. Multiple aspects are to be considered, including concerns that come with collecting sensitive data and health recordings. Also, privacy and trust are addressed. Available applications range from questionnaire-like apps to objective assessment of behavioural patterns and promises in handling suicidality. Nonetheless, interpretation and integration of this high-resolution information with other phenotyping levels, remains challenging. This review provides a state-of-the-art description of wearables and technology in digital phenotyping for monitoring pharmacological treatment in depression, focusing on the challenges and opportunities of its application in clinical trials and research.

Incorporating ecological momentary assessment into multimethod investigations of cognitive aging: Promise and practical considerations

Ecological momentary assessment (EMA) represents a promising approach to study cognitive aging. In contrast to laboratory-based studies, EMA involves the repeated sampling of experiences in daily life contexts, enabling investigators to gain access to dynamic processes (e.g., situational contexts, intraindividual variability) that are likely to strongly contribute to aging and age-related change across the adult life-span. As such, EMA approaches complement the prevailing research methods in the field of cognitive aging (e.g., laboratory-based paradigms, neuroimaging), while also providing the opportunity to replicate and extend findings from the laboratory in more naturalistic contexts. Following an overview of the methodological and conceptual strengths of EMA approaches in cognitive aging research, we discuss best practices for researchers interested in implementing EMA studies. A key goal is to highlight the tremendous potential for combining EMA methods with other laboratory-based approaches, in order to increase the robustness, replicability, and real-world implications of research findings in the field of cognitive aging. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Schizophrenia: Antipsychotics and drug development

The introduction of chlorpromazine and the work that ensued provided the foundation to reposition schizophrenia as a biological illness. The present paper follows the evolution of antipsychotics and their shift from 'typical' to 'atypical'. Atypicality is reviewed in reference to its original definition, clozapine's role, and developments that now leave the concept's utility in question. In a similar fashion, drug development is reviewed in the context of the illness' multiple symptom domains, as well as differences captured by clinical staging and phenotyping. Collectively, the evidence argues for a more nuanced approach to drug development that aligns with the illness' heterogeneity and complexity. Just as 'atypical' as a descriptor for antipsychotics may be outdated, it may be time to set aside the notion of developing drugs that treat 'schizophrenia'.

The promise of machine learning in predicting treatment outcomes in psychiatry

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

Technology meets tradition: a hybrid model for implementing digital tools in neuropsychology

The COVID-19 pandemic has significantly impacted the provision of mental health care services and the ability to provide neuropsychological evaluations. The inability to conduct traditional evaluations has left neuropsychologists with the unprecedented task of determining how to modify existing paradigms while balancing the need to provide services and adhere to safety parameters. The COVID-19 literature suggests clinicians are modifying their evaluations based on the following models: (1) continuing to administer in-person evaluations; (2) discontinuing all evaluations due to issues related to standardization, test security, and patient-specific characteristics; (3) conducting virtual evaluations; and/or (4) adopting a hybrid model incorporating both traditional and technology-based modalities. Given the challenges with models 1-3, along with the modifications in telehealth guidelines and insurance reimbursement rates, neuropsychologists are more poised than ever to solidify the implementation of a hybrid model that lasts beyond COVID-19. We introduce the term Hybrid Neuropsychology, a model for the future of neuropsychological evaluations that includes three Action Items: (1) building a technology-based practice; (2) integrating data science; and (3) engaging with innovators in other fields. Hybrid Neuropsychology will enable clinicians to effectively modernize their practice, improve health care equity, and ensure neuropsychology secures its place in a technology-based world.