Measuring mentalizing: A comparison of scoring methods for the hinting task

Evaluation of a brief online multi-index assessment for predicting increased psychotic-like experiences in the community: A perceptual, cognitive and affective approach

Research has shown that impairments in perception, reasoning, and social cognition are evident across the psychosis continuum and are implicated in the transition from subclinical symptoms to clinical psychosis. In this pilot feasibility study, a brief computerised assessment of visual perception, reasoning, social cognition and emotion dysregulation was administered to 157 adults in the community alongside self-report measures of psychotic-like experiences. The feasibility, reliability, and the predictive validity of the assessment tool were examined. The assessment procedure was feasible, evidenced through high completion rates. However, reliability estimates were suboptimal for online assessment measures. Self-reported visual perception and state emotion dysregulation predicted psychotic-like experiences explaining 53% of the variance when controlling for age. This study provides preliminary evidence that state difficulties with emotion regulation and self-reported visual perception abnormalities can predict increased psychotic-like experiences in the community. Future adaptations could address technological issues encountered with assessment tasks and ensure measures are psychometrically robust when administered online. Brief online assessments hold potential for research of both cognition and affect along the psychosis continuum although caution must be exercised with the chosen methodology.

Mentalizing impairments and hypermentalizing bias in individuals with first-episode schizophrenia-spectrum disorder and at-risk mental state: the differential roles of neurocognition and social anxiety

Mentalizing, or theory of mind (ToM), impairments and self-referential hypermentalizing bias are well-evident in schizophrenia. However, findings compared to individuals with at-risk mental states (ARMS) are inconsistent, and investigations into the relationship between social cognitive impairments and social anxiety in the two populations are scarce. This study aimed to examine and compare these deficits in first-episode schizophrenia-spectrum disorder (FES) and ARMS, and to explore potential specific associations with neurocognition and symptomatology. Forty patients with FES, 40 individuals with ARMS, and 40 healthy controls (HC) completed clinical assessments, a battery of neurocognitive tasks, and three social cognitive tasks. The comic strip and hinting tasks were used to measure non-verbal and verbal mentalizing abilities, and the gaze perception task was employed to assess self-referential hypermentalizing bias. FES and ARMS showed comparable mentalizing impairments and self-referential hypermentalizing bias compared to HC. However, only ambiguous self-referential gaze perception (SRGP) bias remained significantly different between three groups after controlling for covariates. Findings suggested that self-referential hypermentalizing bias could be a specific deficit and may be considered a potential behavioral indicator in early-stage and prodromal psychosis. Moreover, working memory and social anxiety were related to the social cognitive impairments in ARMS, whereas higher-order executive functions and positive symptoms were associated with the impairments in FES. The current study indicates the presence of stage-specific mechanisms of mentalizing impairments and self-referential hypermentalizing bias, providing insights into the importance of personalized interventions to improve specific neurocognitive domains, social cognition, and clinical outcomes for FES and ARMS.

Harmonizing cross-cultural and transdiagnostic assessment of social cognition by expert panel consensus

Social cognition, the perception and processing of social information, is adversely affected in multiple psychiatric, neurological, and neurodevelopmental disorders, and these impairments negatively impact quality of life for individuals across the globe. Despite the clear importance of social cognition, efforts to advance research via harmonization of data across cultures and diagnoses has been stymied by the lack of uniformly used and suitable assessments. To address this issue, the current study conducted an expert survey and consensus process to identify social cognitive assessments that are best suited for cross-cultural and transdiagnostic use among adults. A large group of experts in social cognition were surveyed to gather nominations for cross-culturally and transdiagnostically appropriate measures. These measures were then critically evaluated by a smaller group of experts using a Delphi consensus process to identify the best existing tasks for each use. Ninety-eight experts, representing 25 countries, responded to the initial survey and nominated a total of 81 tasks. Initial rounds of the Delphi process identified 50 tasks with adequate psychometric properties that were then subdivided into social cognition domains. For each domain, members ranked the five best tasks, once for cross-cultural use and once for transdiagnostic use, and rated the suitability of those tasks for the intended use. No tasks were identified as ideally suited for either use; however, within each domain, 4-5 tasks emerged as the most consistently selected, and all were ranked as having "good" or better suitability for use. While there is still a critical need for social cognitive assessments that are specifically designed for cross-cultural and transdiagnostic use, there does appear to be a handful of existing tasks that are currently available and likely informative. Caution is warranted however, as these still require comprehensive evaluation in cross-cultural and transdiagnostic studies.

Understanding the Psychosis Spectrum Using a Hierarchical Model of Social Cognition

BACKGROUND AND HYPOTHESIS

Social cognitive impairments are central to psychosis, including lower severity psychosis-like experiences (PLEs). Nonetheless, progress has been hindered by social cognition's poorly defined factor structure, as well as limited work examining the specificity of social cognitive impairment to psychosis. The present study examined how PLEs relate to social cognition in the context of other psychopathology dimensions, using a hierarchical factors approach to social cognition.

STUDY DESIGN

Online community participants (N = 1026) completed psychosis, autism, and personality disorder questionnaires, as well as 3 social cognitive tasks that varied in methodology (vignette vs video) and construct (higher- vs lower-level social cognition). Exploratory (EFA) and confirmatory factor analyses (CFA) were used to model social cognition, with the best models being examined in association with PLEs and psychopathology dimensions.

STUDY RESULTS

EFA and CFA supported a hierarchical model of social cognition, with 2 higher-order factors emerging: verbal/vignette task methodology and a multimethod general social cognition factor. These higher-order factors accounted for task-level associations to psychopathology, with relations to positive symptoms (r = .23) and antagonism (r = .28). After controlling for other psychopathology, positive symptoms were most clearly related to tasks with verbal methodology (β = -0.34).

CONCLUSIONS

These results suggest that broad social cognitive processes and method effects may account for many previous findings in psychosis and psychopathology research. Additionally, accounting for broad social cognitive impairment may yield insights into more specific social cognitive processes as well.

Training individuals with schizophrenia to gain volitional control of the theory of mind network with real-time fMRI: A pilot study

Individuals diagnosed with schizophrenia spectrum disorders (SSDs) often demonstrate alterations in the Theory of Mind Network (ToM-N). Here, in this proof-of-concept, single-arm pilot study, we investigate whether participants with an SSD (N = 7) were able to learn to volitionally control regions of the ToM-N (dorso/middle/ventromedial prefrontal cortex [D/M/VMPFC], left temporoparietal junction [LTPJ], precuneus [PC], right superior temporal sulcus [RSTS], and right temporoparietal junction [RTPJ]) using real-time fMRI neurofeedback (rtfMRI-NF). Region-of-interest analyses demonstrate that after neurofeedback training, participants were able to gain volitional control in the following ToM-N brain regions during the transfer task, where no active feedback was given: right temporoparietal junction, precuneus, and dorso/ventromedial prefrontal cortex (neurofeedback effect Fs > 6.17, ps < .05). These findings suggest that trained volitional control over the ToM-N is tentatively feasible with rtfMRI neurofeedback in SSD, although findings need to be replicated with more robust designs that include a control group and larger samples.

Reliability of Theory of Mind Tasks in Schizophrenia, ASD, and Nonclinical Populations: A Systematic Review and Reliability Generalization Meta-analysis

Though theory of mind (ToM) is an important area of study for different disciplines, however, the psychometric evaluations of ToM tasks have yielded inconsistent results across studies and populations, raising the concerns about the accuracy, consistency, and generalizability of these tasks. This systematic review and meta-analysis examined the psychometric reliability of 27 distinct ToM tasks across 90 studies involving 2771 schizophrenia (SZ), 690 autism spectrum disorder (ASD), and 15,599 nonclinical populations (NC). Findings revealed that while all ToM tasks exhibited satisfactory internal consistency in ASD and SZ, about half of them were not satisfactory in NC, including the commonly used Reading the Mind in the Eye Test and Hinting Task. Other than that, Reading the Mind in the Eye Test showed acceptable reliability across populations, whereas Hinting Task had poor test-retest reliability. Notably, only Faux Pas Test and Movie for the Assessment of Social Cognition had satisfactory reliability across populations albeit limited numbers of studies. However, only ten studies examined the psychometric properties of ToM tasks in ASD adults, warranting additional evaluations. The study offered practical implications for selecting ToM tasks in research and clinical settings, and underscored the importance of having a robust psychometric reliability in ToM tasks across populations.

Measuring social cognition within the university: The Social Cognition Psychometric Evaluation (SCOPE) battery in an undergraduate sample

Many social cognitive assessments that were developed specifically for use in clinical populations are now being widely used in undergraduate populations, either to provide a comparison for clinical groups or to explore performance across the continuum from healthy to subclinical populations. However, the appropriateness of using these assessments in the general population is unclear. The current study, therefore, seeks to determine whether the Social Cognition Psychometric Evaluation (SCOPE) battery retains its psychometric properties when used in an undergraduate sample.The initial SCOPE battery was administered to 265 undergraduate participants, and the utility of the tasks was measured by assessing floor and ceiling effects. Internal consistency, discriminant and convergent validity are reported for each task, and the relationship between task performance and neurocognition and social functioning were assessed.Several of the tasks demonstrated limited utility, with the Hinting Task specifically demonstrating a pronounced ceiling effect. Only two measures of social cognitive biases achieved satisfactory internal consistency. Select tasks demonstrated small, but significant relationships with social functioning outcomes and significantly predicted 2-3% of variance above neurocognition alone.Despite the association with social functioning, results indicate that the psychometric properties demonstrated in the initial SCOPE study with a clinical sample do not fully extend to a healthy undergraduate population and highlight the need for social cognitive tasks that validly assess ability across the continuum.

The evaluation study for social cognition measures in Japan: Psychometric properties, relationships with social function, and recommendations

AIM

Patients with schizophrenia can have significant subjective difficulties in social cognition, but few undergo testing or treatment for social cognition. The Social Cognition Psychometric Evaluation (SCOPE) study recommends six social cognitive measures; however, the reliability and validity of these measures in different cultural and linguistic areas has not been adequately examined. We examined the psychometric properties of nine social cognitive measures and the relationship to social function, with the aim of determining the level of recommendation for social cognitive measures in clinical practice in Japan.

METHODS

For our test battery, an expert panel previously selected nine measures: the Bell Lysaker Emotion Recognition Task (BLERT); Facial Emotion Selection Test (FEST); Hinting Task (Hinting); Metaphor and Sarcasm Scenario Test (MSST); Intentionality Bias Task (IBT); Ambiguous Intentions and Hostility Questionnaire (AIHQ); Social Attribution Task-Multiple Choice (SAT-MC); SAT-MCII; and Biological Motion (BM) task. In total, 121 outpatients with schizophrenia and 70 healthy controls were included in the analysis, and the results were provided to an expert panel to determine the recommendations for each measure. The quantitative psychological indices of each measure were evaluated for practicality, tolerability, test-retest reliability, correlation with social function, and the incremental validity of social function.

RESULTS

Hinting and FEST received the highest recommendations for use in screening, severity assessment, and longitudinal assessment, followed by BLERT, MSST AIHQ, SAT-MC, and SAT-MCII, with IBT and BM receiving the lowest recommendations.

CONCLUSION

This study provides a uniform assessment tool that can be used in future international clinical trials for social cognitive impairment.

Age-Related Social Cognitive Performance in Individuals With Psychotic Disorders and Their First-Degree Relatives

BACKGROUND

Social cognitive impairment is a recognized feature of psychotic disorders. However, potential age-related differences in social cognitive impairment have rarely been studied.

STUDY DESIGN

Data came from 905 individuals with a psychotic disorder, 966 unaffected siblings, and 544 never-psychotic controls aged 18-55 who participated in the Genetic Risk and Outcome of Psychosis (GROUP) study. Multilevel linear models were fitted to study group main effects and the interaction between group and age on emotion perception and processing (EPP; degraded facial affect recognition) and theory of mind (ToM; hinting task) performance. Age-related differences in the association between socio-demographic and clinical factors, and EPP and ToM were also explored.

STUDY RESULTS

Across groups, EPP performance was associated with age (β = -0.02, z = -7.60, 95% CI: -0.02, -0.01, P < .001), with older participants performing worse than younger ones. A significant group-by-age interaction on ToM (X2(2) = 13.15, P = .001) indicated that older patients performed better than younger ones, while no age-related difference in performance was apparent among siblings and controls. In patients, the association between negative symptoms and ToM was stronger for younger than older patients (z = 2.16, P = .03).

CONCLUSIONS

The findings point to different age-related performance patterns on tests of 2 key social cognitive domains. ToM performance was better in older individuals, although this effect was only observed for patients. EPP was less accurate in older compared with younger individuals. These findings have implications with respect to when social cognitive training should be offered to patients.

Training volitional control of the theory of mind network with real-time fMRI neurofeedback

Is there a way improve our ability to understand the minds of others? Towards addressing this question, here, we conducted a single-arm, proof-of-concept study to evaluate whether real-time fMRI neurofeedback (rtfMRI-NF) from the temporo-parietal junction (TPJ) leads to volitional control of the neural network subserving theory of mind (ToM; the process by which we attribute and reason about the mental states of others). As additional aims, we evaluated the strategies used to self-regulate the network and whether volitional control of the ToM network was moderated by participant characteristics and associated with improved performance on behavioral measures. Sixteen participants underwent fMRI while completing a task designed to individually-localize the TPJ, and then three separate rtfMRI-NF scans during which they completed multiple runs of a training task while receiving intermittent, activation-based feedback from the TPJ, and one run of a transfer task in which no neurofeedback was provided. Region-of-interest analyses demonstrated volitional control in most regions during the training tasks and during the transfer task, although the effects were smaller in magnitude and not observed in one of the neurofeedback targets for the transfer task. Text analysis demonstrated that volitional control was most strongly associated with thinking about prior social experiences when up-regulating the neural signal. Analysis of behavioral performance and brain-behavior associations largely did not reveal behavior changes except for a positive association between volitional control in RTPJ and changes in performance on one ToM task. Exploratory analysis suggested neurofeedback-related learning occurred, although some degree of volitional control appeared to be conferred with the initial self-regulation strategy provided to participants (i.e., without the neurofeedback signal). Critical study limitations include the lack of a control group and pre-rtfMRI transfer scan, which prevents a more direct assessment of neurofeedback-induced volitional control, and a small sample size, which may have led to an overestimate and/or unreliable estimate of study effects. Nonetheless, together, this study demonstrates the feasibility of training volitional control of a social cognitive brain network, which may have important clinical applications. Given the study's limitations, findings from this study should be replicated with more robust experimental designs.

Comparison of Social-Evaluative Anxiety and Theory of Mind Functions in Social Anxiety Disorder, Schizophrenia, and Healthy Controls

INTRODUCTION

Despite the similarities in poor social competence and clinical manifestations of poor social behavior, no study has compared the theory of mind performance between social anxiety disorder (SAD) and schizophrenia, considering the effect of social-evaluative anxiety and neurocognitive functions. In our study, we aimed to compare the theory of mind functions and social-evaluative anxiety between patients with SAD and schizophrenia and healthy controls and to examine the relationship between the theory of mind, neurocognitive skills, and social-evaluative anxiety.

METHODS

Thirty-four consecutive patients with schizophrenia, 29 patients with SAD, and 30 controls matched by age, education level, and sex were enrolled in the study. Structured Clinical Interview for DSM, Beck Depression Inventory, Liebowitz Social Anxiety Scale, Theory of Mind measures (Reading the Mind in the Eyes Test, Hinting Task, Faux Pas Test), Social Appearance Anxiety Scale, Fear of Positive Evaluation Scale, Fear of Negative Evaluation Scale-Short Form, and neuropsychological tests were administered to all participants.

RESULTS

A greater significant deterioration in theory of mind and neurocognitive functions was found in patients with schizophrenia compared to those with SAD and healthy controls. Social evaluation anxiety was highest in patients with SAD. Although social-evaluative anxiety was associated with the theory of mind function in schizophrenia, only fear of positive evaluation was associated with SAD. In all groups, neither theory of mind nor neurocognitive ability measures were correlated with social anxiety levels and related symptoms.

CONCLUSIONS

The impaired theory of mind functioning detected in our study is more prominent in the schizophrenia group and largely independent of anxiety in schizophrenia and SAD. Although social evaluation anxiety, as a transdiagnostic concept, seems to be independent of theory of mind function in general, fear of positive evaluation seems to be associated with hinting in both disorders.

Brief battery of the Social Cognition Psychometric Evaluation study (BB-SCOPE): Development and validation in schizophrenia spectrum disorders

The aim of the present study was to develop an abbreviated social cognition (SC) battery for individuals with schizophrenia spectrum disorders (SSD) to reduce the heterogeneity of and increase the frequency of assessment of SC impairment. To this end, the present study utilized Item Response Theory to develop brief versions of SC tasks administered to individuals with SSD (n = 386) and individuals without a psychiatric diagnosis (n = 292) during the Social Cognition Psychometric Evaluation (SCOPE) Study. Seven brief measures of SC were evaluated (i.e., Ambiguous Intentions and Hostility Questionnaire [AIHQ], Bell Lysaker Emotion Recognition Task [BLERT], Penn Emotion Recognition Task, Reading the Mind in the Eyes Task, Hinting Task, Intentionality Bias Task, Relationships Across Domains Task), and the existing brief version of The Awareness of Social Inference Test was reviewed. Psychometric properties for each brief SC measure were evaluated and compared to the original measures. Based on psychometric properties and relationships with other measures of SC, neurocognition, and functioning, two brief tasks (AIHQ, BLERT) and the full-length Hinting task were recommended for inclusion in a brief battery of SC tasks from the SCOPE Study (BB-SCOPE). The resulting BB-SCOPE is efficient, with an estimated administration time of 15 min, and comprehensively assesses three domains of SC (i.e., attributional bias, emotion processing, theory of mind) to identify severe SC impairment. Scoring of BB-SCOPE is also straightforward and includes a recommended cut-point of 60 for identifying SC impairment.

Measuring mentalizing: A comparison of scoring methods for the hinting task

OBJECTIVE

The Social Cognition Psychometric Evaluation (SCOPE) study supported the utility and practicality of the Hinting task as a measure of social cognition/mentalizing in clinical trials, specifically with the SCOPE authors' stringent scoring system. However, it remains unclear whether the SCOPE scoring system is necessary for the task to be judged as psychometrically sound.

METHOD

Independent raters rescored data from the three phases of SCOPE using the Hinting task's original scoring criteria. Psychometric properties of the task when scored with the original criteria versus more stringent SCOPE criteria were compared in a large sample of individuals with chronic schizophrenia (n = 397) and matched controls (n = 300) as well as a smaller sample of individuals with early psychosis (n = 38) and controls (n = 39).

RESULTS

In both samples, SCOPE criteria resulted in lowered average scores and reduced ceiling effects. Further, revised scoring resulted in strengthened relationships between the hinting task and outcome measures in the chronic sample, and better differentiated early psychosis patients from controls. Conversely, test-retest reliability and internal consistency estimates were not improved using revised scoring and remained suboptimal, particularly for healthy controls.

CONCLUSION

Overall, SCOPE scoring criteria improved some psychometric properties and clinical utility, suggesting that these criteria should be considered for implementation.