The Role of Psychometrics in Individual Differences Research in Cognition: A Case Study of the AX-CPT

Impact of ERP Reliability Cutoffs on Sample Characteristics and Effect Sizes: Performance-Monitoring ERPs in Psychosis and Healthy Controls

In studies of event-related brain potentials (ERPs), it is common practice to exclude participants for having too few trials for analysis to ensure adequate score reliability (i.e., internal consistency). However, in research involving clinical samples, the impact of increasingly rigorous reliability standards on factors such as sample generalizability, patient versus control effect sizes, and effect sizes for within-group correlations with external variables is unclear. This study systematically evaluated whether different ERP reliability cutoffs impacted these factors in psychosis. Error-related negativity (ERN) and error positivity (Pe) were assessed during a modified flanker task in 97 patients with psychosis and 104 healthy comparison participants, who also completed measures of cognition and psychiatric symptoms. ERP reliability cutoffs had notably different effects on the factors considered. A recommended reliability cutoff of 0.80 resulted in sample bias due to systematic exclusion of patients with relatively few task errors, lower reported psychiatric symptoms, and higher levels of cognitive functioning. ERP score reliability lower than 0.80 resulted in generally smaller between- and within-group effect sizes, likely misrepresenting effect sizes. Imposing rigorous ERP reliability standards in studies of psychotic disorders might exclude high-functioning patients, which raises important considerations for the generalizability of clinical ERP research. Moving forward, we recommend examining characteristics of excluded participants, optimizing paradigms and processing pipelines for use in clinical samples, justifying reliability thresholds, and routinely reporting score reliability of all measurements, ERP or otherwise, used to examine individual differences, especially in clinical research.

No evidence for cross-paradigm transfer of abstract task knowledge in adults and school-aged children

Cognitive control is a hallmark of human cognition. A large number of studies have focused on the plasticity of cognitive control and examined how repeated task experience leads to the improvement of cognitive control in novel task environments. However, it has been demonstrated that training-induced changes are very selective and that transfer occurs within one task paradigm but not across different task paradigms. The current study tested the possibility that cross-paradigm transfer would occur if a common cognitive control strategy is employed across different task paradigms. Specifically, we examined whether prior experience of using reactive control in one task paradigm (i.e., either the cued task-switching paradigm or the AX-CPT) makes adults (N = 137) and 9- to 10-year-olds (N = 126) respond in a reactive way in a subsequent condition of another task paradigm in which proactive control could have been engaged. Bayesian generalized mixed-effects models revealed clear evidence of an absence of cross-paradigm transfer of reactive control in both adults and school-aged children. Based on these findings, we discuss to what extent learned control could be transferred across different task contexts and the task-specificity of proactive/reactive control strategies.

The impact of induced stress on reactive and proactive control in depression

Background

Depression, a widespread mental health issue, is often marked by impaired cognitive control, particularly in managing proactive and reactive processes. The Dual Mechanisms of Control (DMC) framework differentiates between these two modes of cognitive control: proactive control involves sustained goal maintenance, while reactive control is more stimulus-driven and transient. Stress, known to exacerbate cognitive dysfunction in depression, may influence the balance between these control processes, though the specific effects remain poorly understood. This study aimed to investigate how acute stress influences proactive and reactive control in individuals with depressive symptoms.

Methods

A total of 142 participants were divided into high-stress and control conditions and further categorized based on their depression levels, measured using the Beck Depression Inventory-II (BDI-II). Cognitive control was assessed using the AX-Continuous Performance Task (AX-CPT), both before and after exposure to a stress-inducing anagram task, which was designed to differentiate between high-stress and low-stress conditions.

Results

Participants exposed to the high-stress condition reported significantly greater stress and fatigue levels compared to the control group, validating the stress manipulation. Although the balance between reactive and proactive control, as measured by the Proactive Behavioral Index (PBI), did not show significant changes, depressive individuals in the high-stress condition exhibited a significant decline in their ability to retain contextual information, as indicated by a reduction in the d'-context index. This suggests that depressive individuals may be more prone to stress-induced difficulties in proactive control.

Discussion

These findings highlight the selective impact of stress on proactive cognitive control in individuals with depressive symptoms, shedding light on a potential cognitive vulnerability in depression. While the balance between reactive and proactive control remained stable, the impaired retention of contextual information post-stress points to a specific deficit in proactive control. This could have implications for targeted cognitive interventions, such as cognitive control training, aimed at enhancing resilience against stress in depressive populations. Future research should explore the long-term effects of stress on cognitive control, particularly in clinically diagnosed individuals.

Neurocognitive dynamics of preparatory and adaptive cognitive control: Insights from mass-univariate and multivariate pattern analysis of EEG data

Cognitive control refers to humans' ability to willingly align thoughts and actions with internally represented goals. Research indicates that cognitive control is not one-dimensional but rather integrates multiple sub-processes to cope with task demands successfully. In particular, the dynamic interplay between preparatory (i.e., prior to goal-relevant events) and adaptive (i.e., in response to unexpected demands) recruitment of neural resources is believed to facilitate successful behavioural performance. However, whether preparatory and adaptive processes draw from independent or shared neural resources, and how these align in the information processing stream, remains unclear. To address these issues, we recorded electroencephalographic data from 52 subjects while they performed a computerised task. Using a combination of mass-univariate and multivariate pattern analysis procedures, we found that different types of control triggered distinct sequences of brain activation patterns, and that the order and temporal extent of these patterns were dictated by the type of control used by the participants. Stimuli that fostered preparatory recruitment of control evoked a sequence of transient occipital-parietal, sustained central-parietal, and sustained fronto-central responses. In contrast, stimuli that indicated the need for quick behavioural adjustments triggered a sequence of transient occipital-parietal, fronto-central, and central parietal responses. There was also a considerable degree of overlap in the temporal evolution of these brain activation patterns, with behavioural performance being mainly related to the magnitude of the central-parietal and fronto-central responses. Our results demonstrate how different neurocognitive mechanisms, such as early attentional allocation and subsequent behavioural selection processes, are likely to contribute to cognitive control. Moreover, our findings extend prior work by showing that these mechanisms are engaged (at least partly) in parallel, rather than independently of each other.

Statistical learning ability at 17 months relates to early reading skills via oral language

Statistical learning ability has been found to relate to children's reading skills. Yet, statistical learning is also known to be vital for developing oral language skills, and oral language and reading skills relate strongly. These connections raise the question of whether statistical learning ability affects reading via oral language or directly. Statistical learning is multifaceted, and so different aspects of statistical learning might influence oral language and reading skills distinctly. In a longitudinal study, we determined how two aspects of statistical learning from an artificial language tested on 70 17-month-old infants-segmenting sequences from speech and generalizing the sequence structure-related to oral language skills measured at 54 months and reading skills measured at approximately 75 months. Statistical learning segmentation did not relate significantly to oral language or reading, whereas statistical learning generalization related to oral language, but only indirectly related to reading. Our results showed that children's early statistical learning ability was associated with learning to read via the children's oral language skills.

On the reliability of value-modulated attentional capture: An online replication and multiverse analysis

Stimuli predicting rewards are more likely to capture attention, even when they are not relevant to our current goals. Individual differences in value-modulated attentional capture (VMAC) have been associated with various psychopathological conditions in the scientific literature. However, the claim that this attentional bias can predict individual differences requires further exploration of the psychometric properties of the most common experimental paradigms. The current study replicated the VMAC effect in a large online sample (N = 182) and investigated the internal consistency, with a design that allowed us to measure the effect during learning (rewarded phase) and after acquisition, once feedback was omitted (unrewarded phase). Through the rewarded phase there was gradual increase of the VMAC effect, which did not decline significantly throughout the unrewarded phase. Furthermore, we conducted a reliability multiverse analysis for 288 different data preprocessing specifications across both phases. Specifications including more blocks in the analysis led to better reliability estimates in both phases, while specifications that removed more outliers also improved reliability, suggesting that specifications with more, but less noisy, trials led to better reliability estimates. Nevertheless, in most instances, especially those considering fewer blocks of trials, reliability estimates fell below the minimum recommended thresholds for research on individual differences. Given the present results, we encourage researchers working on VMAC to take into account reliability when designing studies aimed at capturing individual differences and provide recommendations to improve methodological practices.

Effects of stimulus onset asynchrony on cognitive control in healthy adults

The efficiency of cognitive control in healthy adults can be influenced by various factors, including the stimulus onset asynchrony (SOA) effect and strategy training. To address these issues, our study aims to investigate the impact of SOA on single-mode cognitive control using the Go/No-Go task, as well as the manifestation of proactive control within dual mechanisms of cognitive control through the AX-CPT task. In single-mode cognitive control, extending SOA led to significantly enhanced reaction times (RTs) during Go trials, suggesting improved task performance with increased preparation time. Moreover, the analysis revealed consistently higher accuracy rates in No-Go trials than to Go trials across all SOA levels, indicating robust inhibition processes unaffected by SOA variations. In the dual mechanisms of cognitive control, significant variations in RT and accuracy were observed among different trial types. Notably, participants exhibited superior performance in detecting targets during BY trials and shorter RTs in BX trials, indicative of efficient processing of target stimuli. Conversely, prolonged RTs in AY trials suggest proactive control strategies aimed at maintaining task-relevant information and inhibiting irrelevant responses. Overall, these findings highlight the effect of SOA on single-mode cognitive control and the emergence of proactive control within dual mechanisms of cognitive control in healthy adults.

A measure of reliability convergence to select and optimize cognitive tasks for individual differences research

Surging interest in individual differences has faced setbacks in light of recent replication crises in psychology, for example in brain-wide association studies exploring brain-behavior correlations. A crucial component of replicability for individual differences studies, which is often assumed but not directly tested, is the reliability of the measures we use. Here, we evaluate the reliability of different cognitive tasks on a dataset with over 250 participants, who each completed a multi-day task battery. We show how reliability improves as a function of number of trials, and describe the convergence of the reliability curves for the different tasks, allowing us to score tasks according to their suitability for studies of individual differences. We further show the effect on reliability of measuring over multiple time points, with tasks assessing different cognitive domains being differentially affected. Data collected over more than one session may be required to achieve trait-like stability.

Threat Appraisal and Pediatric Anxiety: Proof of Concept of a Latent Variable Approach

Elevated threat appraisal is a postulated neurodevelopmental mechanism of anxiety disorders. However, laboratory-assessed threat appraisals are task-specific and subject to measurement error. We utilized latent variable analysis to integrate youth's self-reported threat appraisals across different experimental tasks; we next examined associations with pediatric anxiety as well as behavioral and psychophysiological task indices. Ninety-two youth ages 8-17 years (M age=13.07, 65% female), including 51 with a primary anxiety disorder and 41 with no Axis I diagnosis, completed up to eight threat-exposure tasks. Anxiety symptoms were assessed using questionnaires and ecological momentary assessment. Appraisals both prior to and following threat exposures evidenced shared variance across tasks. Derived factor scores for threat appraisal were associated significantly with anxiety symptoms and variably with task indices; findings were comparable to task-specific measures and had several advantages. Results support an overarching construct of threat appraisal linked with pediatric anxiety, providing groundwork for more robust laboratory-based measurement.

Validation of the Amharic version of perceived access to healthcare services for patients with cervical cancer in Ethiopia: A second-order confirmatory factor analysis

BACKGROUND

Accessing healthcare services is a multifaceted phenomenon involving various elements, encompassing the demand, identification, reach, and utilization of healthcare needs. The literature offers methods for capturing patients' perceptions of healthcare access. However, to accurately measure patient perceptions, it is imperative to ensure the validity and reliability of such instruments by designing and implementing localized language versions.

AIM

The primary aim of this study was to validate the Amharic version of the perceived access to health-care services among patients diagnosed with cervical cancer in Ethiopia.

METHOD

A cross-sectional study was conducted among cervical cancer patients at oncology centers in Addis Ababa, Ethiopia. A consecutive sampling approach was used and data collection took place from January 1 to March 30, 2023. Following initial validation and pretesting, a KoboCollect mobile phone application was employed for data collection. Subsequently, the collected data underwent cleaning in Microsoft Excel and analysis through Amos software v.26 and R programming. Various validity and reliability tests, such as content validity, convergent validity, face validity, divergent validity, known-group validity, and reliability tests, were executed. A second-order confirmatory factor analysis was developed to calculate incremental model fit indices, including CFI and TLI, along with absolute measures, namely SRMR and RMSEA.

RESULTS

A total of 308 participants were involved in the study, with 202 (65.6%) being patients referred from outside Addis Ababa. The initial evaluation of content validity by expert panels indicated that all criteria were met, with a CVR range of 0.5 to 1, I-CVI values ranging from 0.75 to 1, an S-CVI value of 0.91, and face validity values ranging from 2.4 to 4.8. The internal consistency of items within the final constructs varied from 0.76 to 0.93. Convergent, known-group, and most divergent validity tests fell within acceptable fit ranges. Common incremental fit measures for CFI and TLI were achieved with corresponding values of 0.95 and 0.94, respectively. The absolute fit measures of SRMR and RMSEA were 0.04 and 0.07, indicating good and moderate fit, respectively.

CONCLUSION

The study indicated a high internal consistency and validity of items with good fit to the data, suggesting potential accuracy of the domains. A five-domain structure was developed which enables adequate assessment of perceived access to health-care services of patients with cervical cancer in Ethiopia. We suggest that the tool can be utilized in other patient populations with a consideration of additional constructs, such as geographic accessibility.

The psychometric upgrade psychophysiology needs

Although biological measurements are constrained by the same fundamental psychometric principles as self-report measurements, these essential principles are often neglected in most fields of neuroscience, including psychophysiology. Potential reasons for this neglect could include a lack of understanding of appropriate measurement theory or a lack of accessible software for psychometric analysis. Generalizability theory is a flexible and multifaceted measurement theory that is well suited to handling the nuances of psychophysiological data, such as the often unbalanced number of trials and intraindividual variability of scores of event-related brain potential (ERP) data. The ERP Reliability Analysis Toolbox (ERA Toolbox) was designed for psychophysiologists and is tractable software that can support the routine evaluation of psychometrics using generalizability theory. Psychometrics can guide task refinement, data-processing decisions, and selection of candidate biomarkers for clinical trials. The present review provides an extensive treatment of additional psychometric characteristics relevant to studies of psychophysiology, including validity and validation, standardization, dimensionality, and measurement invariance. Although the review focuses on ERPs, the discussion applies broadly to psychophysiological measures and beyond. The tools needed to rigorously assess psychometric reliability and validate psychophysiological measures are now readily available. With the profound implications that psychophysiological research can have on understanding brain-behavior relationships and the identification of biomarkers, there is simply too much at stake to ignore the crucial processes of evaluating psychometric reliability and validity.

On the psychometric evaluation of cognitive control tasks: An Investigation with the Dual Mechanisms of Cognitive Control (DMCC) battery

The domain of cognitive control has been a major focus of experimental, neuroscience, and individual differences research. Currently, however, no theory of cognitive control successfully unifies both experimental and individual differences findings. Some perspectives deny that there even exists a unified psychometric cognitive control construct to be measured at all. These shortcomings of the current literature may reflect the fact that current cognitive control paradigms are optimized for the detection of within-subject experimental effects rather than individual differences. In the current study, we examine the psychometric properties of the Dual Mechanisms of Cognitive Control (DMCC) task battery, which was designed in accordance with a theoretical framework that postulates common sources of within-subject and individual differences variation. We evaluated both internal consistency and test-retest reliability, and for the latter, utilized both classical test theory measures (i.e., split-half methods, intraclass correlation) and newer hierarchical Bayesian estimation of generative models. Although traditional psychometric measures suggested poor reliability, the hierarchical Bayesian models indicated a different pattern, with good to excellent test-retest reliability in almost all tasks and conditions examined. Moreover, within-task, between-condition correlations were generally increased when using the Bayesian model-derived estimates, and these higher correlations appeared to be directly linked to the higher reliability of the measures. In contrast, between-task correlations remained low regardless of theoretical manipulations or estimation approach. Together, these findings highlight the advantages of Bayesian estimation methods, while also pointing to the important role of reliability in the search for a unified theory of cognitive control.

Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of a study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.

Dynamics of cognitive predictors during reading acquisition in a sample of children overrepresented for dyslexia risk

Literacy acquisition is a complex process with genetic and environmental factors influencing cognitive and neural processes associated with reading. Previous research identified factors that predict word reading fluency (WRF), including phonological awareness (PA), rapid automatized naming (RAN), and speech-in-noise perception (SPIN). Recent theoretical accounts suggest dynamic interactions between these factors and reading, but direct investigations of such dynamics are lacking. Here, we investigated the dynamic effect of phonological processing and speech perception on WRF. More specifically, we evaluated the dynamic influence of PA, RAN, and SPIN measured in kindergarten (the year prior to formal reading instruction), first grade (the first year of formal reading instruction) and second grade on WRF in second and third grade. We also assessed the effect of an indirect proxy of family risk for reading difficulties using a parental questionnaire (Adult Reading History Questionnaire, ARHQ). We applied path modeling in a longitudinal sample of 162 Dutch-speaking children of whom the majority was selected to have an increased family and/or cognitive risk for dyslexia. We showed that parental ARHQ had a significant effect on WRF, RAN and SPIN, but unexpectedly not on PA. We also found effects of RAN and PA directly on WRF that were limited to first and second grade respectively, in contrast to previous research reporting pre-reading PA effects and prolonged RAN effects throughout reading acquisition. Our study provides important new insights into early prediction of later word reading abilities and into the optimal time window to target a specific reading-related subskill during intervention.

Learning to generalise but not segment an artificial language at 17 months predicts children's language skills 3 years later

We investigated whether learning an artificial language at 17 months was predictive of children's natural language vocabulary and grammar skills at 54 months. Children at 17 months listened to an artificial language containing non-adjacent dependencies, and were then tested on their learning to segment and to generalise the structure of the language. At 54 months, children were then tested on a range of standardised natural language tasks that assessed receptive and expressive vocabulary and grammar. A structural equation model demonstrated that learning the artificial language generalisation at 17 months predicted language abilities - a composite of vocabulary and grammar skills - at 54 months, whereas artificial language segmentation at 17 months did not predict language abilities at this age. Artificial language learning tasks - especially those that probe grammar learning - provide a valuable tool for uncovering the mechanisms driving children's early language development.

Using smartphones to optimise and scale-up the assessment of model-based planning

Model-based planning is thought to protect against over-reliance on habits. It is reduced in individuals high in compulsivity, but effect sizes are small and may depend on subtle features of the tasks used to assess it. We developed a diamond-shooting smartphone game that measures model-based planning in an at-home setting, and varied the game's structure within and across participants to assess how it affects measurement reliability and validity with respect to previously established correlates of model-based planning, with a focus on compulsivity. Increasing the number of trials used to estimate model-based planning did remarkably little to affect the association with compulsivity, because the greatest signal was in earlier trials. Associations with compulsivity were higher when transition ratios were less deterministic and depending on the reward drift utilised. These findings suggest that model-based planning can be measured at home via an app, can be estimated in relatively few trials using certain design features, and can be optimised for sensitivity to compulsive symptoms in the general population.

Validation of the Romanian Version of the Halitosis Associated Life-Quality Test (HALT) in a Cross-Sectional Study among Young Adults

Halitosis is a common condition with a significant impact on individuals' quality of life. The Halitosis Associated Life-Quality Test (HALT) is a reliable instrument for measuring this impact. This study aimed to introduce and validate the Romanian translation of the HALT questionnaire (R-HALT). We hypothesized that the R-HALT would demonstrate good reliability and validity in measuring the impact of halitosis on quality of life among Romanian teenagers and young adults. Our objectives were to translate and adapt the HALT, validate it among a cross-sectional group, and evaluate the extent of halitosis in this population. A multicentric cross-sectional design followed, which was approved by the Ethics Research Committee in Romania. The translation process involved independent translations, retro-translations, expert review, and pre-testing. The psychometric properties were evaluated among 150 patients (mean age 23.6 ± 1.8; 51% males) at dental clinics, including reliability, convergent, and discriminant validity, using accepted statistical measures such as Cronbach's alpha and Intraclass Correlation Coefficient (ICC). The R-HALT revealed strong internal consistency with Cronbach's alpha values ranging from 0.93 to 0.96, and an ICC value of 0.87 (95% CI = 0.70-0.99), demonstrating excellent test-retest reliability. Mean scores of individual items ranged from 0.82 ± 0.94 (Q3) to 3.23 ± 1.15 (Q11). The corrected item-total correlation ranged from 0.30 (Q2) to 0.90 (Q19). Organoleptic test scores diagnosed 41% (62 patients) with a score of 2, with increasing severity in 9% (13 patients) with a score of 5. The R-HALT exhibited robust reliability and validity in assessing the impact of halitosis among Romanian teenagers and young adults. The questionnaire is a strong tool for understanding, diagnosing, and managing halitosis in Romania, ultimately aiming to improve the quality of life of affected individuals. Further studies with diverse populations could enhance the applicability of the R-HALT.

Changes in adaptation to time horizons across development

When making decisions, the amount of time remaining matters. When time horizons are long, exploring unknown options can inform later decisions, but when time horizons are short, exploiting known options should be prioritized. While adults and adolescents adapt their exploration in this way, it is unclear when such adaptation emerges and how individuals behave when time horizons are ambiguous, as in many real-life situations. We examined these questions by having 5- to 6-year-olds (N = 43), 11- to 12-year-olds (N = 40), and adult college students (N = 49) in the United States complete a Simplified Horizons Task under short, long, and ambiguous time horizons. Adaptation to time horizons increased with age: older children and adults explored more when horizons were long than when short, and while some younger children adapted to time horizons, younger children overall did not show strong evidence of adapting. Under ambiguous horizons, older children and adults preferred to exploit over explore, while younger children did not show this preference. Thus, adaptation to time horizons is evident by ages 11-12 and may begin to emerge around 5-6 years, and children decrease their tendencies to explore under short and ambiguous time horizons with development. This developmental shift may lead to less learning but more adaptive decision making. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Improving the Reliability of Cognitive Task Measures: A Narrative Review

Cognitive tasks are capable of providing researchers with crucial insights into the relationship between cognitive processing and psychiatric phenomena. However, many recent studies have found that task measures exhibit poor reliability, which hampers their usefulness for individual differences research. Here, we provide a narrative review of approaches to improve the reliability of cognitive task measures. Specifically, we introduce a taxonomy of experiment design and analysis strategies for improving task reliability. Where appropriate, we highlight studies that are exemplary for improving the reliability of specific task measures. We hope that this article can serve as a helpful guide for experimenters who wish to design a new task, or improve an existing one, to achieve sufficient reliability for use in individual differences research.

The relationship between self, value-based reward, and emotion prioritisation effects

People show systematic biases in perception, memory, attention, and decision-making to prioritise information related to self, reward, and positive emotion. A long-standing set of experimental findings points towards putative common properties of these effects. However, the relationship between them remains largely unknown. Here, we addressed this question by assessing and linking these prioritisation effects generated by a common associative matching procedure in three experiments. Self, reward, and positive emotion prioritisation effects were assessed using cluster and shift function analyses to explore and test associations between these effects across individuals. Cluster analysis revealed two distinct patterns of the relationship between the biases. Individuals with faster responses showed a smaller reward and linear positive association between reward and emotion biases. Individuals with slower responses demonstrated a large reward and no association between reward and emotion biases. No evidence of the relationship between self and value-based reward or positive emotion prioritisation effects was found among the clusters. A shift function indicated a partial dominance of high-reward over low-reward distributions at later processing stages in participants with slower but not faster responses. Full stochastic dominance of self-relevance over others and positive over neutral emotion was pertinent to each subgroup of participants. Our findings suggest the independent origin of the self-prioritisation effect. In contrast, commonalities in cognitive mechanisms supporting value-based reward and positive emotion processing are subject to individual differences. These findings add important evidence to a steadily growing research base about the relationship between basic behavioural drivers.

Developing and Validating a Facial Emotion Recognition Task With Graded Intensity

Facial emotion recognition (FER) tasks are often digitally altered to vary expression intensity; however, such tasks have unknown psychometric properties. In these studies, an FER task was developed and validated-the Graded Emotional Face Task (GEFT)-which provided an opportunity to examine the psychometric properties of such tasks. Facial expressions were altered to produce five intensity levels for six emotions (e.g., 40% anger). In Study 1, 224 undergraduates viewed subsets of these faces and labeled the expressions. An item selection algorithm was used to maximize internal consistency and balance gender and ethnicity. In Study 2, 219 undergraduates completed the final GEFT and a multimethod battery of validity measures. Finally, in Study 3, 407 undergraduates oversampled for borderline personality disorder (BPD) completed the GEFT and a self-report BPD measure. Broad FER scales (e.g., overall anger) demonstrated evidence of reliability and validity; however, more specific subscales (e.g., 40% anger) had more variable psychometric properties. Notably, ceiling/floor effects appeared to decrease both internal consistency and limit external validity correlations. The findings are discussed from the perspective of measurement issues in the social cognition literature.

Individual differences in computational psychiatry: a review of current challenges

Bringing precision to the understanding and treatment of mental disorders requires instruments for studying clinically relevant individual differences. One promising approach is the development of computational assays: integrating computational models with cognitive tasks to infer latent patient-specific disease processes in brain computations. While recent years have seen many methodological advancements in computational modelling and many cross-sectional patient studies, much less attention has been paid to basic psychometric properties (reliability and construct validity) of the computational measures provided by the assays. In this review, we assess the extent of this issue by examining emerging empirical evidence. We find that many computational measures suffer from poor psychometric properties, which poses a risk of invalidating previous findings and undermining ongoing research efforts using computational assays to study individual (and even group) differences. We provide recommendations for how to address these problems and, crucially, embed them within a broader perspective on key developments that are needed for translating computational assays to clinical practice.

Self-caught methodologies for measuring mind wandering with meta-awareness: A systematic review

Mind wandering, also known as task-unrelated thought, refers to the drift of attention from a focal task or train of thought. Because self-caught measures of mind wandering require participants to spontaneously indicate when they notice their attention drift, self-caught methodologies provide a way to measure mind wandering with meta-awareness. Given the proposed role of meta-awareness in mental health and psychological interventions, an overview of existing self-caught methodologies would help clinicians and researchers make informed decisions when choosing or adapting a mind wandering or meta-awareness measure. This systematic review included 39 studies after 790 studies were assessed for eligibility. All studies operationalised mind wandering as instances of attention drift from a primary task. Three types of primary task were identified: (1) tasks adapted from computerised continuous performance tests (CPT) of sustained attention, (2) tasks involving focusing on the breath or a stream of aural beats, akin to in-vivo mindfulness meditation, (3) tasks involving an everyday life activity such as reading. Although data on mind wandering without meta-awareness (e.g., measured with probe-caught measures) was also obtained in many studies, such data was not always used in conjunction with self-caught mind wandering data to determine level of mind wandering meta-awareness. Few studies reported both reliability and validity of the measures used. This review shows that considerable methodological heterogeneity exists in the literature. Methodological variants of self-caught mind wandering methodologies are documented and examined, and suggestions for future research and clinical work are suggested.

Social contacts and loneliness affect the own age bias for emotional faces

Individuals are better at recognizing faces of their own age group (Own Age Bias) but it is unclear whether this bias occurs also for emotional faces and to what extent is affected by loneliness. Young individuals (N = 235) completed an age categorization task on faces of young and old individuals showing neutral, happy, and angry expressions. After a filler task, they categorized as seen or novel the original set of faces intermixed with a new set. Findings showed an Own Age Bias for novel young faces but no evidence that emotion eliminates it. Recognition accuracy was better for emotional faces, but the two factors did not interact. Importantly, low loneliness was linked to an Own Age Bias for novel happy faces. These findings are discussed in the context of current theoretical accounts of the Own Age Bias and of the effects of loneliness on attention and memory.

Evidence for modulation of planning and working memory capacities by transcranial direct current stimulation in a sample of adults with attention deficit hyperactivity disorder

BACKGROUND AND OBJECTIVE

Attention deficit hyperactivity disorder (ADHD) is a common neurodevelopmental disorder that affects up to 2.8% of the adult population. Albeit pharmacological and behavioral therapies alleviate some core symptoms of ADHD, they do not avail cognitive dysfunction adequately. Executive dysfunction has been considered to have a principal role in ADHD and has previously been linked to activity alterations in the prefrontal cortex. Transcranial Direct Current Stimulation (tDCS) is a noninvasive brain stimulation technique that may modulate prefrontal cortex activity and induce neuroplasticity, with preliminary results in ADHD. The aim of the present study is to assess the effect of repeated tDCS on measures of executive functions in adults with ADHD.

METHOD

In this randomized double-blind sham-controlled study, 22 adults with ADHD were allocated into two groups and were administered five consecutive sessions of 2 mA active/sham tDCS over the dorsolateral prefrontal cortex (right anodal/left cathodal). A neuropsychological test battery was administered before the first session and immediately after the last session.

RESULTS

The maximum number of digits and the total number of correct trials in the Digit Span Backward test increased in the active group (p = 0.017). The total move score in the Tower of London test decreased (p = 0.033), suggesting better planning ability. However, no significant differences were observed on Stroop Test and Trail Making Test after tDCS.

DISCUSSION

The present study corroborates the modulating effects of tDCS on planning and working memory in a small group of adults with ADHD. Our results highlighted that cognitive functions are modulable using tDCS in adults with ADHD.

The Effect of Mindfulness-based Programs on Cognitive Function in Adults: A Systematic Review and Meta-analysis

Mindfulness-based programs (MBPs) are increasingly utilized to improve mental health. Interest in the putative effects of MBPs on cognitive function is also growing. This is the first meta-analysis of objective cognitive outcomes across multiple domains from randomized MBP studies of adults. Seven databases were systematically searched to January 2020. Fifty-six unique studies (n = 2,931) were included, of which 45 (n = 2,238) were synthesized using robust variance estimation meta-analysis. Meta-regression and subgroup analyses evaluated moderators. Pooling data across cognitive domains, the summary effect size for all studies favored MBPs over comparators and was small in magnitude (g = 0.15; [0.05, 0.24]). Across subgroup analyses of individual cognitive domains/subdomains, MBPs outperformed comparators for executive function (g = 0.15; [0.02, 0.27]) and working memory outcomes (g = 0.23; [0.11, 0.36]) only. Subgroup analyses identified significant effects for studies of non-clinical samples, as well as for adults aged over 60. Across all studies, MBPs outperformed inactive, but not active comparators. Limitations include the primarily unclear within-study risk of bias (only a minority of studies were considered low risk), and that statistical constraints rendered some p-values unreliable. Together, results partially corroborate the hypothesized link between mindfulness practices and cognitive performance. This review was registered with PROSPERO [CRD42018100904].

Enhancing the Psychometric Properties of the Iowa Gambling Task Using Full Generative Modeling

Poor psychometrics, particularly low test-retest reliability, pose a major challenge for using behavioral tasks in individual differences research. Here, we demonstrate that full generative modeling of the Iowa Gambling Task (IGT) substantially improves test-retest reliability and may also enhance the IGT's validity for use in characterizing internalizing pathology, compared to the traditional analytic approach. IGT data ( n = 50 ) was collected across two sessions, one month apart. Our full generative model incorporated (1) the Outcome Representation Learning (ORL) computational model at the person-level and (2) a group-level model that explicitly modeled test-retest reliability, along with other group-level effects. Compared to the traditional 'summary score' (proportion good decks selected), the ORL model provides a theoretically rich set of performance metrics (Reward Learning Rate ( A + ) , Punishment Learning Rate ( A - ) , Win Frequency Sensitivity ( β f ) , Perseveration Tendency ( β p ) , Memory Decay ( K ) ), capturing distinct psychological processes. While test-retest reliability for the traditional summary score was only moderate (r = . 37 , BCa 95% CI [.04, .63]), test-retest reliabilities for ORL performance metrics produced by the full generative model were substantially improved, with test-retest correlations ranging between r = . 64 - . 82 for the five ORL parameters. Further, while summary scores showed no substantial associations with internalizing symptoms, ORL parameters were significantly associated with internalizing symptoms. Specifically, Punishment Learning Rate was associated with higher self-reported depression and Perseveration Tendency was associated with lower self-reported anhedonia. Generative modeling offers promise for advancing individual differences research using the IGT, and behavioral tasks more generally, through enhancing task psychometrics.

Experimental Manipulation of Guided Attention to the Shoulder Movement Task in Clinical Dohsa-hou Induces Shifts in the Reactive Mode and Indicates Flexible Cognitive Control Performance

The empirical basis for self-control in Dohsa-hou as it relates to effects on cognitive processes has been explored in a few studies of the Japanese psychotherapy, but not under standardized conditions with a strong predictive theory of control. This study reports on a series of experiments with the Dual Mechanisms of Control framework to clarify the possible regulatory mechanism of Dohsa-hou by focusing on shoulder movement, a key body movement task used by practitioners across applied settings. Cognitive control was operationalized with the AX version of Continuous Performance Test (AX-CPT) paradigm for proactive control and a modified Stroop task paradigm for reactive control in a 3-arm parallel group trial study design. Healthy Japanese university students were assigned to a Dohsa-hou group that performed a shoulder movement task for few minutes, an active control group that performed a similar task, or a passive control group comprised of a resting condition. A total of 55 participants performed the AX-CPT and 57 participants performed the modified Stroop task before and after the group manipulation. In the AX-CPT, an increase in the error rate of AY (true cue-false probe) trial from pre- to post-test was observed in the passive control group only, and found to be marginally higher in the passive control group relative to Dohsa-hou group at post-test. This indicated that Dohsa-hou moderated the activation of proactive control by repeated AX-CPT performance. The error rate of the Proactive Behavioral Index did not differ from zero at post-test only in the Dohsa-hou group, indicating flexible cognitive control. In the modified Stroop task, there was no difference between congruent and incongruent trials at post-test for the Dohsa-hou group only, indicating the facilitation of reactive control. The evidence for a balancing effect for the Dohsa-hou-based shoulder movement task indicates that clients experience a form of continuous self-monitoring, which might reduce mind-wandering from their focus on movement execution combined with iterative verbal feedback from the therapist. Overall, the results of the present study suggest that the self-regulatory mechanism promoted in clinical Dohsa-hou emphasizes guided shifts in attention to the reactive mode toward a balance of cognitive control.

Understanding the role of subpopulations and reliability in between-group studies

The replication crisis has led to a renewed discussion about the impacts of measurement quality on the precision of psychology research. High measurement quality is associated with low measurement error, yet the role of reliability in the quality of experimental research is not always well understood. In this study, we attempt to understand the role of reliability through its relationship with power while focusing on between-group designs for experimental studies. We outline a latent variable framework to investigate this nuanced relationship through equations. An under-evaluated aspect of the relationship is the variance and homogeneity of the subpopulation from which the study sample is drawn. Higher homogeneity implies a lower reliability, but yields higher power. We proceed to demonstrate the impact of this relationship between reliability and power by imitating different scenarios of large-scale replications with between-group designs. We find negative correlations between reliability and power when there are sizable differences in the latent variable variance and negligible differences in the other parameters across studies. Finally, we analyze the data from the replications of the ego depletion effect (Hagger et al., 2016) and the replications of the grammatical aspect effect (Eerland et al., 2016), each time with between-group designs, and the results align with previous findings. The applications show that a negative relationship between reliability and power is a realistic possibility with consequences for applied work. We suggest that more attention be given to the homogeneity of the subpopulation when study-specific reliability coefficients are reported in between-group studies.

Decreased preparatory activation and inattention to cues suggest lower activation of proactive cognitive control among high procrastinating students

Procrastination is a voluntary delay in completing an important task while being aware that this behavior may lead to negative outcomes. It has been shown that an increased tendency to procrastinate is associated with deficits in some aspects of cognitive control. However, none of the previous studies investigated these dysfunctions through the lenses of the Dual Mechanisms Framework, which differentiates proactive and reactive modes of control. The present study was designed to fill this gap, using behavioral and neurophysiological assessment during the completion of the AX-Continuous Performance Task (AX-CPT) by high (HP) and low (LP) procrastinating students (N = 139). Behavioral results indicated that HP (vs. LP) were characterized by increased attentional fluctuations (higher reaction time variability) and reduction in some indices of proactive cognitive control (lower d'-context and A-cue bias, but similar PBIs). Furthermore, the neurophysiological data showed that HP, compared with LP, allocated less attentional resources (lower P3b) to cues that help to predict the correct responses to upcoming probes. They also responded with reduced preparatory activity (smaller CNV) after cues presentation. The two groups did not differ in neural responses linked to conflict detection and inhibition (similar N2 and P3a). Obtained findings indicate that HP might present deficits in some cognitive functions that are essential for effective proactive control engagement, along with preserved levels of reactive cognitive control. In the present paper, we discuss the potential neural and cognitive mechanisms responsible for the observed effects.

Explanatory personality science in the neuroimaging era: The map is not the territory

The recent rise of magnetic resonance imaging (MRI) has led to a proliferation of studies that seek to link individual differences in personality directly to their neural correlates. These studies function to describe personality at a lower level of analysis, but they do little to advance the field's understanding of the causal mechanisms that give rise to personality traits. To transition to a more explanatory personality neuroscience, researchers should strive to conduct theory-driven empirical studies that bridge multiple levels of analysis. Effectively doing so will require a continued reliance on rich description, strong theories, large samples, and careful behavioral experimentation. Integrating these components will lead to more robust and informative studies of personality neuroscience that help to move the field closer to explaining the causal sources of individual differences.

Tools for Evaluating the Content, Efficacy, and Usability of Mobile Health Apps According to the Consensus-Based Standards for the Selection of Health Measurement Instruments: Systematic Review

Background

There are several mobile health (mHealth) apps in mobile app stores. These apps enter the business-to-customer market with limited controls. Both, apps that users use autonomously and those designed to be recommended by practitioners require an end-user validation to minimize the risk of using apps that are ineffective or harmful. Prior studies have reviewed the most relevant aspects in a tool designed for assessing mHealth app quality, and different options have been developed for this purpose. However, the psychometric properties of the mHealth quality measurement tools, that is, the validity and reliability of the tools for their purpose, also need to be studied. The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative has developed tools for selecting the most suitable measurement instrument for health outcomes, and one of the main fields of study was their psychometric properties.

Objective

This study aims to address and psychometrically analyze, following the COSMIN guideline, the quality of the tools that are used to measure the quality of mHealth apps.

Methods

From February 1, 2019, to December 31, 2019, 2 reviewers searched PubMed and Embase databases, identifying mHealth app quality measurement tools and all the validation studies associated with each of them. For inclusion, the studies had to be meant to validate a tool designed to assess mHealth apps. Studies that used these tools for the assessment of mHealth apps but did not include any psychometric validation were excluded. The measurement tools were analyzed according to the 10 psychometric properties described in the COSMIN guideline. The dimensions and items analyzed in each tool were also analyzed.

Results

The initial search showed 3372 articles. Only 10 finally met the inclusion criteria and were chosen for analysis in this review, analyzing 8 measurement tools. Of these tools, 4 validated ≥5 psychometric properties defined in the COSMIN guideline. Although some of the tools only measure the usability dimension, other tools provide information such as engagement, esthetics, or functionality. Furthermore, 2 measurement tools, Mobile App Rating Scale and mHealth Apps Usability Questionnaire, have a user version, as well as a professional version.

Conclusions

The Health Information Technology Usability Evaluation Scale and the Measurement Scales for Perceived Usefulness and Perceived Ease of Use were the most validated tools, but they were very focused on usability. The Mobile App Rating Scale showed a moderate number of validated psychometric properties, measures a significant number of quality dimensions, and has been validated in a large number of mHealth apps, and its use is widespread. It is suggested that the continuation of the validation of this tool in other psychometric properties could provide an appropriate option for evaluating the quality of mHealth apps.

EXPRESS: Working Memory Capacity and Dual Mechanisms of Cognitive Control: An Experimental-Correlational Approach

Working memory is thought to be strongly related to cognitive control. Recent studies have sought to understand this relationship under the prism of the dual mechanisms of control (DMC) framework, in which cognitive control is thought to operate in two distinct modes: proactive and reactive. Several authors have concluded that a high working memory capacity is associated with a tendency to engage the more effective mechanism of proactive control. However, the predicted pattern of proactive control use has never been observed; correlational evidence is made difficult to interpret by the overall superiority of participants with a high working memory capacity: they tend to perform better even when proactive control should be detrimental. In two experiments, we used an experimental-correlational approach to experimentally induce the use of reactive or proactive control in the AX-CPT. The relation between working memory capacity and performance was unaffected, incompatible with the hypothesis that the better performance of participants with a high working memory capacity in the task is due to their use of proactive control. It remains unclear how individual differences in working memory capacity relate to cognitive control under the DMC framework.

Raising the bar: improving methodological rigour in cognitive alcohol research

BACKGROUND AND AIMS

A range of experimental paradigms claim to measure the cognitive processes underpinning alcohol use, suggesting that heightened attentional bias, greater approach tendencies and reduced cue-specific inhibitory control are important drivers of consumption. This paper identifies methodological shortcomings within this broad domain of research and exemplifies them in studies focused specifically on alcohol-related attentional bias.

ARGUMENT AND ANALYSIS

We highlight five main methodological issues: (i) the use of inappropriately matched control stimuli; (ii) opacity of stimulus selection and validation procedures; (iii) a credence in noisy measures; (iv) a reliance on unreliable tasks; and (v) variability in design and analysis. This is evidenced through a review of alcohol-related attentional bias (64 empirical articles, 68 tasks), which reveals the following: only 53% of tasks use appropriately matched control stimuli; as few as 38% report their stimulus selection and 19% their validation procedures; less than 28% used indices capable of disambiguating attentional processes; 22% assess reliability; and under 2% of studies were pre-registered.

CONCLUSIONS

Well-matched and validated experimental stimuli, the development of reliable cognitive tasks and explicit assessment of their psychometric properties, and careful consideration of behavioural indices and their analysis will improve the methodological rigour of cognitive alcohol research. Open science principles can facilitate replication and reproducibility in alcohol research.

Psychometric Evaluation of the Comfort from Companion Animals Scale in a Sexual and Gender Minority Sample

Human-animal interaction research is growing in popularity and methodological rigor; however, there remains a need for psychometrically validated measures and inclusion of broader populations. This study addressed these gaps by reporting on the psychometric properties of the Comfort from Companion Animals Scale (CCAS) in a sample of sexual and gender minority emerging adults. Participants included 138 emerging adults between the ages of 18-21 years (M = 19.33 years, SD = 1.11; 38.4% racial/ethnic minority) who identified as a gender (48.6%) and/or sexual minority (98.6%) and who reported living with a companion animal in the past year. We utilized the following analytic methods: (a) confirmatory factor analyses to compare the unidimensional structure of the CCAS with the two alternative models, (b) multiple group analyses to test measurement invariance across demographic groups, and (c) structural equation models to evaluate construct validity. Preliminary analysis found that the majority of participants did not endorse the two lowest response options. To conduct invariance testing, we eliminated items 3, 5, and 8 from the CCAS and collapsed the lowest response options. The results of the confirmatory factor analysis supported the use of this revised unidimensional model. We found evidence of measurement invariance across gender identity, sexual orientation, and race/ethnicity groups. Construct validity was supported by comparing the CCAS with factors on the Pet Attachment and Life Impact Scale; the positive association between the CCAS and anxiety are discussed in the context of prior research. Overall, our findings highlight the importance of validating human-animal interaction measures across samples from diverse backgrounds. We recommend that future studies continue to test the CCAS and other measures of human-animal attachment among diverse samples to delineate which aspects of human-animal interaction may be most beneficial in promoting mental health in vulnerable populations.

Different measures of holistic face processing tap into distinct but partially overlapping mechanisms

Holistic processing, which includes the integration of facial features and analysis of their relations to one another, is a hallmark of what makes faces 'special'. Various experimental paradigms purport to measure holistic processing but these have often produced inconsistent results. This has led researchers to question the nature and structure of the mechanism(s) underlying holistic processing. Using an individual differences approach, researchers have examined relations between various measures of holistic processing in an attempt to resolve these questions. In keeping with this, we examined relationships between four commonly used measures of holistic face processing in a large group of participants (N = 223): (1) The Face Inversion Effect, (2) the Part Whole Effect (PWE), (3) the Composite Face Effect, and (4) the Configural Featural Detection Task (CFDT). Several novel methodological and analytical elements were introduced, including the use of factor analysis and the inclusion of control conditions to confirm the face specificity of all of the effects measured. The four indexes of holistic processing derived from each measure loaded onto two factors, one encompassing the PWE and the CFDT, and one encompassing the CE. The 16 conditions tested across the four tasks loaded onto four factors, each factor corresponding to a different measure. These results, together with those of other studies, suggest that holistic processing is a multifaceted construct and that different measures tap into distinct but partially overlapping elements of it.

Methods to split cognitive task data for estimating split-half reliability: A comprehensive review and systematic assessment

Estimating the reliability of cognitive task datasets is commonly done via split-half methods. We review four methods that differ in how the trials are split into parts: a first-second half split, an odd-even trial split, a permutated split, and a Monte Carlo-based split. Additionally, each splitting method could be combined with stratification by task design. These methods are reviewed in terms of the degree to which they are confounded with four effects that may occur in cognitive tasks: effects of time, task design, trial sampling, and non-linear scoring. Based on the theoretical review, we recommend Monte Carlo splitting (possibly in combination with stratification by task design) as being the most robust method with respect to the four confounds considered. Next, we estimated the reliabilities of the main outcome variables from four cognitive task datasets, each (typically) scored with a different non-linear algorithm, by systematically applying each splitting method. Differences between methods were interpreted in terms of confounding effects inflating or attenuating reliability estimates. For three task datasets, our findings were consistent with our model of confounding effects. Evidence for confounding effects was strong for time and task design and weak for non-linear scoring. When confounding effects occurred, they attenuated reliability estimates. For one task dataset, findings were inconsistent with our model but they may offer indicators for assessing whether a split-half reliability estimate is appropriate. Additionally, we make suggestions on further research of reliability estimation, supported by a compendium R package that implements each of the splitting methods reviewed here.

The Wisconsin Card Sorting Test: Split-Half Reliability Estimates for a Self-Administered Computerized Variant

Self-administered computerized assessment has the potential to increase the reach of neuropsychological assessment. The present study reports the first split-half reliability estimates for a self-administered computerized variant of the Wisconsin Card Sorting Test (WCST), which is considered as a gold standard for the neuropsychological assessment of executive functions. We analyzed data from a large sample of young volunteers (N = 375). Split-half reliability estimates for perseveration errors, set-loss errors, and inference errors were all above 0.90. Split-half reliability estimates for response time measures on switch and repeat trials exceeded 0.95. Our results indicated sufficient split-half reliability for a self-administered computerized WCST, paving the way for an advanced digital assessment of executive functions. We discuss potential effects of test formats, administration variants, and sample characteristics on split-half reliability.

Can a Theater Acting Intervention Enhance Inhibitory Control in Older Adults? A Brain-Behavior Investigation

Purpose: Studies of reactive and proactive modes of inhibitory control tend to show age-related declines and are accompanied by abnormalities in the prefrontal cortex. We explored which mode of inhibitory control would be more amenable to change and accrue greater benefits following engagement in a 4-week theater acting intervention in older adults. These gains were evaluated by performance on the AX-CPT task. We hypothesized that an increase in proactive control would relate to an increase in AY errors and a decrease in BX errors. In contrast, an increase in reactive control would be associated with a decrease in AY errors, no change in AY reaction time, and an increase in BX response time. Further, we posited that an increase in behavioral proactive control would accompany greater cue versus probe activity for previously identified regions in the prefrontal cortex. In contrast, an increase in behavioral reactive control would be accompanied by greater probe activation in these identified brain areas. Materials and Methods: The participants were 179 community-dwelling adults aged 60-89 years who were on average, college-educated. Participants were pseudo-randomly assigned to either an active-experiencing acting intervention condition (n = 93) or the active control condition (n = 86); participant assignment was subject to time of enrollment. Participants in both groups were trained by theater-actor researchers with expertise in acting interventions. In contrast to the active control participants who attended a course on theater acting, the acting-intervention group was required to consistently deploy proactive and reactive control mechanisms. Both groups met two times/week for 75-min for 4 weeks. Participant brain-behavioral performance on the AX-CPT task was evaluated prior to and after this four-week period. Results: No intervention effects were found in favor of proactive control. Behavioral evidence in favor of reactive control was weak. Brain-related benefits to reactive control was illustrated by greater probe-activation in Brodmann areas 6 and 8, relative to controls and pre-intervention. Conclusion: We found some evidence for improvements in reactive control via brain measures, attributed to engagement in the acting intervention.

Testing the attention-distractibility trait

Forster and Lavie (Journal of Experimental Psychology: Learning, Memory, and Cognition, 40[1], 251-260, 2014; Psychological Science, 27[2], 203-212, 2016) found that task-irrelevant distraction correlated positively with a measure of mind-wandering and a report of attention-deficit/hyperactivity disorder (ADHD) symptomology. Based primarily on these results, Forster and Lavie claimed to establish an attention-distractibility trait. Here, I tested whether these associations could be distinguished from associations with working memory capacity and task-relevant distraction (measured with an antisaccade task). With data collected from 226 subjects (ns differ among analyses), the results from the current study suggest that the measures of task-irrelevant distraction and working memory capacity were not (or only very weakly) associated with measures of mind wandering (measured both with a stand-alone questionnaire and in-task thought probes) and ADHD symptomology. Task-relevant interference (i.e., antisaccade accuracy) was associated with mind-wandering reports from in-task thought probes (presented in a separate task), but not the stand-alone mind wandering questionnaire or ADHD symptomology. Additionally, the measure of irrelevant-distraction exhibited low internal consistency suggesting that (as measured) it may not be a suitable individual difference measure. [Preregistration, data, analysis scripts and output are available via the Open Science Framework: https://osf.io/bhs24/ ].

Assessing Neuropsychological Functions in Middle Childhood: a Narrative Review of Measures and Their Psychometric Properties Across Context

Background

There is a significant number of neuropsychological measures for use among children aged 6–12 years. However, most of these tests have been developed in high-income contexts (HICs). To avoid or to at least to minimize bias in assessment, most researchers carry out cultural adaptations of these tools. In selecting sub-tests to adapt before using the entire test battery, researchers would benefit from having a reference source summarizing available tools and how easily they can be used in different context. This is where the paper makes a contribution. This narrative review has a twofold aim: first, to identify tools commonly used among 6–12-year-olds; second, to summarize the psychometric properties of these tools especially emphasizing their usage across different cultural contexts.

Methods

We searched the literature from 1 January 1987 to 31 December 2017 for tools used among children aged 6 to 12 years. An extensive search of PubMed, Psych Info and Web of Science using the keywords (i) neuropsychological or neurocognitive with (ii) assessment or test was done.

Results

A hundred and forty-five papers out of 306 reported on psychometric properties of different tools including Behavior Rating Inventory of Executive Functioning—BRIEF (count = 6), Visual-Motor Integration—VMI (count = 6), the Test of Memory Malingering—TOMM (count = 6), MSVT (count = 6) and Continuous Performance Tests—CPT (count = 6). Forty-six percent of the papers reported studies conducted in the USA. Most of these studies were based in high-income countries, which further highlights the need to validate these measures for use in lower- and middle-income countries. Psychometric check was adequate in most tests for measuring executive functioning such as BRIEF, although tests such as CPT that measure complex attention had mixed findings. Moreover, we found that these studies addressed certain aspects of validity and or reliability while leaving out others; thus, a comprehensive picture is lacking.

Conclusion

We propose further studies to thoroughly investigate and report the psychometric properties of these measures, especially in lower- and middle-income countries.

The Patient Perspective on Patient-Reported Outcome Measures Following Elective Hand Surgery: A Convergent Mixed-Methods Analysis

PURPOSE

Patient-reported outcome measures (PROMs) have traditionally been used for research purposes, but are now being used to evaluate outcomes from the patient's perspective and inform ongoing management and quality of care. We used quantitative and qualitative approaches to evaluate the short-version Disabilities of the Arm, Shoulder, and Hand (QuickDASH) and the Patient-Specific Functional Scale (PSFS) with regard to patient preference and measurement of patient goals and their responsiveness after treatment.

METHODS

Patients 18 years or older undergoing elective hand surgery received the QuickDASH and PSFS questionnaires before and at 6 weeks after surgery. Two additional questions intended to elicit patients' preferences regarding the QuickDASH and PSFS were included. Responsiveness was measured by change in pre- to postoperative score. We analyzed patients' responses to the 2 additional questions to identify themes in PROM preferences. Results from the quantitative and qualitative analyses were combined into a convergent mixed-methods (eg, quantitative and qualitative) analysis.

RESULTS

Thirty-eight patients completed preoperative questionnaires; 25 (66%) completed postoperative questionnaires. Seventeen patients (77%) preferred the PSFS, 3 (14%) had no preference, 2 (9%) preferred the QuickDASH. The average change from pre- to postoperative QuickDASH was -10 (SD, 20), and that of the PSFS was -27 (SD, 26). Ten patients (40%) reported QuickDASH score changes above the minimal clinically importance difference (MCID), 17 patients (68%) reported PSFS score changes above the MCID. Content analysis revealed 4 themes in preference for a PROM: instrument simplicity (ease of instrument understanding and completion), personalized assessment (individualization and relevance), goal directed (having measurable aims or objectives), distinct items (concrete or specific instrument items or functions).

CONCLUSIONS

Most patients felt the PSFS better measured their goals because it is a simple, personalized instrument with distinct domains.

CLINICAL RELEVANCE

Whereas standardized PROMs may better compare across populations, physicians, or conditions, employing PROMs that address patient-specific goals may better assess aspects of care most important to patients. A combination of these 2 types of PROMs can be used to assess outcomes and inform quality of care.

Opportunities for increased reproducibility and replicability of developmental neuroimaging

Many workflows and tools that aim to increase the reproducibility and replicability of research findings have been suggested. In this review, we discuss the opportunities that these efforts offer for the field of developmental cognitive neuroscience, in particular developmental neuroimaging. We focus on issues broadly related to statistical power and to flexibility and transparency in data analyses. Critical considerations relating to statistical power include challenges in recruitment and testing of young populations, how to increase the value of studies with small samples, and the opportunities and challenges related to working with large-scale datasets. Developmental studies involve challenges such as choices about age groupings, lifespan modelling, analyses of longitudinal changes, and data that can be processed and analyzed in a multitude of ways. Flexibility in data acquisition, analyses and description may thereby greatly impact results. We discuss methods for improving transparency in developmental neuroimaging, and how preregistration can improve methodological rigor. While outlining challenges and issues that may arise before, during, and after data collection, solutions and resources are highlighted aiding to overcome some of these. Since the number of useful tools and techniques is ever-growing, we highlight the fact that many practices can be implemented stepwise.

The Reliability of the Wisconsin Card Sorting Test in Clinical Practice

The Wisconsin Card Sorting Test (WCST) represents the gold standard for the neuropsychological assessment of executive function. However, very little is known about its reliability. In the current study, 146 neurological inpatients received the Modified WCST (M-WCST). Four basic measures (number of correct sorts, categories, perseverative errors, set-loss errors) and their composites were evaluated for split-half reliability. The reliability estimates of the number of correct sorts, categories, and perseverative errors fell into the desirable range (rel ≥ .90). The study therefore disclosed sufficiently reliable M-WCST measures, fostering the application of this eminent psychological test to neuropsychological assessment. Our data also revealed that the M-WCST possesses substantially better psychometric properties than would be expected from previous studies of WCST test-retest reliabilities obtained from non-patient samples. Our study of split-half reliabilities from discretionary construed and from randomly built M-WCST splits exemplifies a novel approach to the psychometric foundation of neuropsychology.

Interference control in working memory: Evidence for discriminant validity between removal and inhibition tasks

Working memory (WM) is a system for maintenance of and access to a limited number of goal-relevant representations in the service of higher cognition. Because of its limited capacity, WM requires interference-control processes, allowing us to avoid being distracted by irrelevant information. Recent research has proposed two interference-control processes, which are conceptually similar: (1) an active, item-wise removal process assumed to remove no-longer relevant information from WM, and (2) an inhibitory process assumed to suppress the activation of distractors against competing, goal-relevant representations. The purpose of this study was to determine the extent to which the tasks used to assess removal and inhibition measure the same interference-control construct. Results showed acceptable to good reliabilities for nearly all measures. Similar to previous studies, a structural equation modeling approach identified a reliable latent variable of removal. However, also similar to some previous studies, no latent variable of inhibition could be established. This was the case even when the correlation matrix used to compute the latent variable of inhibition was disattenuated for imperfect reliability. Critically, the individual measures of inhibition were unrelated to the latent variable of removal. These results provide tentative support for the notion that removal is not related to the interference-control processes assessed in inhibition tasks. This suggests that the removal process should be conceptualized as a process independent of the concept of inhibition, as proposed in computational WM models that implement removal as the "unbinding" of a WM item from the context in which it occurred.

Joint analysis of frontal theta synchrony and white matter following mild traumatic brain injury

Some of the most disabling aspects of mild traumatic brain injury (mTBI) include lingering deficits in executive functioning. It is known that mTBI can damage white matter tracts, but it remains unknown how this structural brain damage translates into cognitive deficits. This experiment utilized theta band phase synchrony to identify the dysfunctional neural operations that contribute to cognitive problems following mTBI. Sub-acute stage (< 2 weeks) mTBI patients (N = 52) and healthy matched controls (N = 32) completed a control-demanding task with concurrent EEG. Structural MRI was also collected. While there were no performance-specific behavioral differences between groups in the dot probe expectancy task, the degree of theta band phase synchrony immediately following injury predicted the degree of symptom recovery two months later. Although there were no differences in fractional anisotropy (FA) between groups, joint independent components analysis revealed that a smaller network of lower FA-valued voxels contributed to a diminished frontal theta phase synchrony network in the mTBI group. This finding suggests that frontal theta band markers of cognitive control are sensitive to sub-threshold structural aberrations following mTBI.

Psychometric properties of Meaningful Activity Participation Assessment (MAPA) in chronic stroke survivors

BACKGROUND

Meaningful Activity Participation Assessment (MAPA) is an appropriate tool for assessing both objective and subjective aspects of participation.

OBJECTIVES

This study aimed to investigate the psychometric properties of MAPA in chronic stroke survivors.

METHODS

Translation of MAPA was done according to the standard protocol of forward-backward translation. One hundred and seven chronic stroke survivors participated in this study. In addition to the MAPA, they were assessed by Satisfaction With Life Scale (SWLS), Center for Epidemiologic Studies of Depression Scale (CES-D), Life Satisfaction Index-Z (LSI-Z), Purpose in Life Test-Short Form (PIL-SF), and 36-Item Short-Form Survey (SF-36). To investigate the test-retest reliability, 37 participants were reassessed by MAPA after two weeks. Reliability, construct and known-groups validity were evaluated for MAPA.

RESULTS

The results showed an acceptable internal consistency (Cronbach's α = 0.79) and good test-retest reliability (ICC = 0.92) of MAPA. A significant moderate to high correlation was found between the MAPA and PIL-SF, CES-D, LSI-Z, SWLS, and different subscales of SF-36 (r = 0.32-0.65). MAPA showed good ability to differentiate between young adults (age≤ 65 years) and older adults (age> 65 years) with chronic stroke (P = .005) as well as between chronic stroke survivors with different levels of disability (P < .001).

CONCLUSIONS

The MAPA has appropriate reliability and validity in chronic stroke survivors and is suggested to be used in research and clinical settings.

Effects of HF-rTMS over the left and right DLPFC on proactive and reactive cognitive control

Previous research supports the distinction between proactive and reactive control. Although the dorsolateral prefrontal cortex (DLPFC) has been consistently related to these processes, lateralization of proactive and reactive control is still under debate. We manipulated brain activity to investigate the role of the left and right DLPFC in proactive and reactive cognitive control. Using a single-blind, sham-controlled crossover within-subjects design, 25 young healthy females performed the 'AX' Continuous Performance Task after receiving sham versus active High-Frequency repetitive Transcranial Magnetic Stimulation (HF-rTMS) to increase left and right DLPFC activity. RTs and pupillometry were used to assess patterns of proactive and reactive cognitive control and task-related resource allocation respectively. We observed that, compared to sham, HF-rTMS over the left DLPFC increased proactive control. After right DLPFC HF-rTMS, participants showed slower RTs on AX trials, suggesting more reactive control. However, this latter result was not supported by RTs on BX trials (i.e. the trial that specifically assess reactive control). Pupil measures showed a sustained increase in resource allocation after both active left and right HF-rTMS. Our results with RT data provide evidence on the role of the left DLPFC in proactive control and suggest that the right DLPFC is implicated in reactive control.