The reliability and validity of the MATRICS functional assessment battery

A systematic review of social functioning outcome measures in schizophrenia with a focus on suitability for intervention research

Social functioning is an important part of recovery and a key treatment target in clinical research in schizophrenia. Evaluating and comparing interventions is challenged by the choice of many measures which focus on different aspects of functioning, with little to guide selection. This results in difficulties comparing outcomes of treatment where studies have used different measures. To improve the measurement of social functioning in intervention research, we aimed to provide practical information on suitability of measures. We conducted a systematic review of measures developed or psychometrically evaluated since 2007, and assessed and discussed the structure, content, quality, and the use of the measures in intervention research. Thirty-two measures of social functioning and 22 validation papers were identified. Measures included structured questionnaires, semi-structured interviews, and assessment of performance on specific tasks. The content of measures was organised into eight categories, which are in order of frequency with which they were covered by measures: activities of daily living, productive activity, relationships, leisure activities, cognition, anti-social behaviour, psychosis symptoms and self-esteem and empowerment. In terms of quality, most measures were rated as moderate, with the Personal and Social Performance Scale gaining the highest rating. However, there was little data on responsiveness of measures, or how they compare to objective or 'real-world' indicators of functioning. The Social Functioning Scale and Personal and Social Performance Scale have been most frequently used in intervention studies to date. Future research should aim to provide further data on psychometric properties relevant to intervention research.

Brain structural correlates of functional capacity in first-episode psychosis

Impaired functional capacity is a core feature of schizophrenia and presents even in first-episode psychosis (FEP) patients. Impairments in daily functioning tend to persist despite antipsychotic therapy but their neural basis is less clear. Previous studies suggest that volume loss in frontal cortex might be an important contributor, but findings are inconsistent. We aimed to comprehensively investigate the brain structural correlates of functional capacity in FEP using MRI and a reliable objective measure of functioning [University of California, San Diego Performance-Based Skills Assessment (UPSA)]. In a sample of FEP (n = 39) and a well-matched control group (n = 21), we measured cortical thickness, gray matter volume, and white matter tract integrity (fractional anisotropy, FA) within brain regions implicated by previous work. The FEP group had thinner cortex in various frontal regions and fusiform, and reduced FA in inferior longitudinal fasciculus (ILF). In FEP, poorer functional capacity correlated with reduced superior frontal volume and lower FA in left ILF. Importantly, frontal brain volumes and integrity of the ILF were identified as the structural correlates of functional capacity in FEP, controlling for other relevant factors. These findings enhance mechanistic understanding of functional capacity deficits in schizophrenia by specifying the underlying neural correlates. In future, this could help inform intervention strategies.

A systematic review of a functional assessment Tool: UCSD Performance-based skill assessment (UPSA)

Performance based assessment instruments have been employed in functional capacity measurement of mental disorders. The aim of this systematic review was to identify the psychometric properties of the UCSD Performance-based Skill Assessment (UPSA). A search was conducted using the PRISMA protocol and 'UPSA' as key word term on electronic databases, with a date range for articles published from 2001-2017. Published studies involving community-dwelling adults were included. Pharmacological and/or clinical interventions involving clinical outcomes and/or institutionalized samples were excluded. Data related to construct validity, test-retest reliability and sensitivity/specificity were extracted, summarized and analyzed according to UPSA versions and psychiatric disorders. Fifty-eight studies including 8782 Community-dwelling adults met selection criteria. Data supporting the construct and known-groups validity were extracted from 41 studies involving Schizophrenia and schizoaffective disorders and 17 studies involving other metal illness. The UPSA was culturally adapted to 8 different languages and employed in 17 countries. Few studies reported sensitivity and specificity and the cut-off points could not be generalized. Moderate to strong evidence of construct validity and test-retest reliability was found. Few studies proposed cut-off points. The UPSA showed good psychometric properties in different versions including those culturally adapted.

The Relationship Between Clinical and Personal Recovery in Patients With Schizophrenia Spectrum Disorders: A Systematic Review and Meta-analysis

Patients describe experiencing personal recovery despite ongoing symptoms of psychosis. The aim of the current research was to perform a meta-analysis investigating the relationship between clinical and personal recovery in patients with schizophrenia spectrum disorders. A comprehensive OvidSP database search was performed to identify relevant studies. Correlation coefficients of the relationship between clinical and personal recovery were retrieved from primary studies. Meta-analyses were performed, calculating mean weighted effect sizes for the association between clinical and personal recovery, hope, and empowerment. Additionally, associations between positive, negative, affective symptoms, general functioning, and personal recovery were investigated. The results show that heterogeneity across studies was substantial. Random effect meta-analysis of the relationship between symptom severity and personal recovery revealed a mean weighted correlation coefficient of r = -.21 (95% CI = -0.27 to -0.14, P < .001). We found the following mean weighted effect size for positive symptoms r = -.20 (95% CI = -0.27 to -0.12, P < .001), negative symptoms r = -.24 (95% CI = -0.33 to -0.15, P < .001), affective symptoms r = -.34 (95% CI = -0.44 to -0.24, P < .001) and functioning r = .21 (95% CI = -0.09 to 0.32, P < .001). The results indicate a significant small to medium association between clinical and personal recovery. Psychotic symptoms show a smaller correlation than affective symptoms with personal recovery. These findings suggest that clinical and personal recovery should both be considered in treatment and outcome monitoring of patients with schizophrenia spectrum disorders.

Neurocognitive Impairments Are More Severe in the Binge-Eating/Purging Anorexia Nervosa Subtype Than in the Restricting Subtype

Objective: To evaluate cognitive function impairment in patients with anorexia nervosa (AN) of either the restricting (ANR) or binge-eating/purging (ANBP) subtype. Method: We administered the Japanese version of the MATRICS Consensus Cognitive Battery to 22 patients with ANR, 18 patients with ANBP, and 69 healthy control subjects. Our participants were selected from among the patients at the Kobe University Hospital and community residents. Results: Compared to the healthy controls, the ANR group had significantly lower visual learning and social cognition scores, and the ANBP group had significantly lower processing speed, attention/vigilance, visual learning, reasoning/problem-solving, and social cognition scores. Compared to the ANR group, the ANBP group had significantly lower attention/vigilance scores. Discussion: The AN subtypes differed in cognitive function impairments. Participants with ANBP, which is associated with higher mortality rates than ANR, exhibited greater impairment severities, especially in the attention/vigilance domain, confirming the presence of impairments in continuous concentration. This may relate to the impulsivity, an ANBP characteristic reported in the personality research. Future studies can further clarify the cognitive impairments of each subtype by addressing the subtype cognitive functions and personality characteristics.

Issues and developments related to assessing function in serious mental illness

Serious mental illness (SMI) results in functional disability that imposes a significant burden on individuals, caregivers, and society. Development of novel treatments is under way in an effort to improve the illness domains of cognitive impairment and negative symptoms and subsequently to improve functional outcomes. The assessment of functional outcomes in SMI faces a number of challenges, including the proliferation of assessment instruments and the differential prioritization of functional goals among stakeholder groups. Functional assessments relying on self- and informant report present a number of limitations. Identifying alternative strategies to assess functioning that are reliable, valid, and sensitive to change is necessary for use in clinical trials. Measures of functional capacity have been proposed for clinical trials investigating compounds to treat cognitive impairment in schizophrenia. Alternative approaches employing effort-based decision making or daily activity recording using instruments such as the Daily Activity Report may be more appropriate for studies focused on improving negative symptoms.

Construct validity of the UCSD performance-based skills assessment-brief version (UPSA-B) in HIV disease

Among individuals living with HIV disease, approximately 60% experience problems with everyday functioning. The present study investigated the utility of the UCSD Performance-based Skills Assessment-Brief Version (UPSA-B) as a measure of functional capacity in HIV. We utilized a cross-sectional three-group design comparing individuals with HIV- associated neurocognitive disorder (HAND) (HIV + HAND+; n = 27), HIV+ neurocognitively normal individuals (HIV + HAND-; n = 51), and an HIV- comparison group (HIV-; n = 28) with broadly comparable demographics and non-HIV comorbidities. Participants were administered the UPSA-B, the Medication Management Test-Revised (MMT-R), and were assessed for manifest everyday functioning and quality of life, as part of a standardized clinical neurocognitive research battery. Results indicated that the HIV + HAND+ group had significantly lower UPSA-B scores than the HIV + HAND-group, but did not differ from the HIV- group. Among HIV+ individuals, UPSA-B scores were significantly related to MMT-R scores, all neurocognitive domains assessed, and education, but the UPSA-B was not related to manifest everyday functioning (e.g., unemployment), health-related quality of life, or HIV disease variables. Findings provide mixed support for the construct validity of the UPSA-B in HIV. Individuals impaired on the UPSA-B may be at increased risk for HAND, but the extent to which it detects general manifest everyday functioning problems is uncertain.

Metacognition as a Mediating Variable Between Neurocognition and Functional Outcome in First Episode Psychosis

BACKGROUND

Neurocognitive and functional outcome deficits have long been acknowledged in schizophrenia and neurocognition has been found to account for functional disability to a greater extent than psychopathology. Much of the variance in functional outcome however still remains unexplained and metacognition may mediate the relationship between neurocognition, functional capacity, and self-reported social and occupational function.

METHOD

Eighty first episode psychosis participants were recruited and completed measures of neurocognition (memory, executive function, and intelligence quotient), metacognition (Beck Cognitive Insight Scale, Metacognitive Awareness Interview), psychopathology (PANSS), and both functional capacity (UPSA) and real-life social and occupational function (The Time Use Survey). Path analyses investigated the relationships between variables through structural equation modeling.

RESULTS

A series of path models demonstrated that metacognition partially mediates the relationship between neurocognition and functional capacity, and fully mediates the relationship between functional capacity and social and occupational function.

CONCLUSION

The present study findings identify that metacognition may be critical to translating cognitive and functional skills into real-world contexts, and this relationship is found at early stages of illness. Understanding how individuals translate cognitive and functional skills into the real-world (the competence-performance gap) may offer valuable guidance to intervention programs. This finding is important to models of recovery as it suggests that intervention programs that focus on enhancing metacognition abilities may have a greater impact than traditional rehabilitation programs focusing on cognitive abilities, on social and occupational outcomes.

Initial development of a patient-reported outcome measure of experience with cognitive impairment associated with schizophrenia

Cognitive impairment is a serious, often distressing aspect of schizophrenia that affects patients' day-to-day lives. Although several interview-based instruments exist to assess cognitive functioning, a reliable measure developed based on the experiences of patients facing cognitive difficulties is needed to complement the objective performance-based assessments. The present article describes the initial development of a patient-reported outcome (PRO) measure to assess the subjective experience of cognitive impairment among patients with schizophrenia, the Patient-Reported Experience of Cognitive Impairment in Schizophrenia (PRECIS). The phases of development included the construction of a conceptual model based on the existing knowledge and two sets of qualitative interviews with patients: 1) concept elicitation interviews to ensure face and content validity from the perspective of people with schizophrenia and 2) cognitive debriefing of the initial item pool. Input from experts was elicited throughout the process. The initial conceptual model included seven domains. The results from concept elicitation interviews (n=80) supported these domains but yielded substantive changes to concepts within domains and to terminology. Based on these results, an initial pool of 53 items was developed to reflect the most common descriptions and languages used by the study participants. Cognitive debriefing interviews (n=22) resulted in the removal of 18 items and modification of 22 other items. The remaining 35 items represented 23 concepts within six domains plus two items assessing bother. The draft PRO measure is currently undergoing psychometric testing as a precursor to broad-based clinical and research use.

Language disturbance and functioning in first episode psychosis

Language disturbance has a central role in the presentation of psychotic disorders however its relationship with functioning requires further clarification, particularly in first episode psychosis (FEP). Both language disturbance and functioning can be evaluated with clinician-rated and performance-based measures. We aimed to investigate the concurrent association between clinician-rated and performance-based measures of language disturbance and functioning in FEP. We assessed 108 individuals presenting to an Early Intervention in Psychosis Service in Ireland. Formal thought disorder (FTD) dimensions and bizarre idiosyncratic thinking (BIT) were rated with structured assessment tools. Functioning was evaluated with a performance-based instrument, a clinician-rated measure and indicators of real-world functioning. The disorganisation dimension of FTD was significantly associated with clinician-rated measures of occupational and social functioning (Beta=-0.19, P<0.05 and Beta=-0.31, P<0.01, respectively). BIT was significantly associated with the performance-based measure of functioning (Beta=-0.22, P<0.05). Language disturbance was of less value in predicting real-world measures of functioning. Clinician-rated and performance-based assessments of language disturbance are complementary and each has differential associations with functioning. Communication disorders should be considered as a potential target for intervention in FEP, although further evaluation of the longitudinal relationship between language disturbance and functioning should be undertaken.

Report on ISCTM Consensus Meeting on Clinical Assessment of Response to Treatment of Cognitive Impairment in Schizophrenia

If treatments for cognitive impairment are to be utilized successfully, clinicians must be able to determine whether they are effective and which patients should receive them. In order to develop consensus on these issues, the International Society for CNS Clinical Trials and Methodology (ISCTM) held a meeting of experts on March 20, 2014, in Washington, DC. Consensus was reached on several important issues. Cognitive impairment and functional disability were viewed as equally important treatment targets. The group supported the notion that sufficient data are not available to exclude patients from available treatments on the basis of age, severity of cognitive impairment, severity of positive symptoms, or the potential to benefit functionally from treatment. The group reached consensus that cognitive remediation is likely to provide substantial benefits in combination with procognitive medications, although a substantial minority believed that medications can be administered without nonpharmacological therapy. There was little consensus on the best methods for assessing cognitive change in clinical practice. Some participants supported the view that performance-based measures are essential for measurement of cognitive change; others pointed to their cost and time requirements as evidence of impracticality. Interview-based measures of cognitive and functional change were viewed as more practical, but lacking validity without informant involvement or frequent contact from clinicians. The lack of consensus on assessment methods was viewed as attributable to differences in experience and education among key stakeholders and significant gaps in available empirical data. Research on the reliability, validity, sensitivity, and practicality of competing methods will facilitate consensus.

A case control study of association between cognition and functional capacity in schizophrenia

BACKGROUND

Cognitive functions are important prognostic factors for schizophrenia (SZ), while ability to perform activities of daily living are important measures of functional capacity. The relationship between cognition and functional capacity has not been tested extensively in India.

OBJECTIVE

To compare persons with SZ with controls on measures of cognition and functional capacity, and evaluate correlations between cognitive performance and functional capacity.

METHOD

Schizophrenia outpatients and controls without psychiatric illness (DSM IV) who completed the MATRICS Consensus Cognitive Battery and Functional Assessment Battery comprised of two tests from University of California San Diego (UCSD) Performance Based Skill Assessment (UPSA), one Test of Adaptive Behavior in Schizophrenia (TABS) and one test from University of California San Diego Performance Based Skill Assessment Brief edition (UPSA-B). Cognitive and functional domains were examined using regression analyses, with relevant covariates.

RESULTS

Cases (N=51) though younger, were more educated than controls (N=41). Adjusting for education, controls performed better than cases in 3/7 cognitive and 4/5 domains of functional capacity but similarly in 'household management'. Among both cases and controls, cognitive measures of verbal learning and speed of processing overlapped with functional capacity (3 domains). Working memory was associated with one functional domain.

CONCLUSIONS

Consistent with other studies, Indian patients with schizophrenia performed worse than controls on several domains of cognition and functional capacity; these domains were correlated. Speed of processing and verbal learning are most frequently associated with functional capacity indices and should be targeted to improve skills of daily living among persons with SZ.

Improving the understanding of the link between cognition and functional capacity in schizophrenia and bipolar disorder

OBJECTIVE

Deficits in cognitive functioning are related to functional disability in people with serious mental illness. Measures of functional capacity are commonly used as a proxy for functional disabilities for cognitive remediation programs, and robust linear relationships between functional capacity and cognitive deficits are frequently observed. This study aimed to determine whether a curvilinear relationship better approximates the association between cognitive functioning and functional capacity.

METHOD

Two independent samples were studied. Study 1: participants with schizophrenia (n=435) and bipolar disorder (n=390) aged 18-83 completed a neuropsychological battery and a performance-based measure of functional capacity. Study 2: 205 participants with schizophrenia (age range=39-72) completed a brief neuropsychological screening battery and a performance-based measure of functional capacity. For both studies, linear and quadratic curve estimations were conducted with cognitive performance predicting functional capacity scores.

RESULTS

Significant linear and quadratic trends were observed for both studies. Study 1: in both the schizophrenia and bipolar participants, when cognitive composite z-scores were >0 (indicating normal to above normal performance), cognition was not related to functional capacity. Study 2: when neuropsychological screening battery z-scores were >-1 (indicating low average to average performance), cognition was not related to functional capacity.

CONCLUSIONS

These results illustrate that in cognitively normal adults with serious mental illness, the relationship between cognitive function and functional capacity is relatively weak. These findings may aid clinicians and researchers determine who may optimally benefit from cognitive remediation programs, with greater benefits possibly being achieved for individuals with cognitive deficits relative to individuals with normal cognition.

UPSA-M: Feasibility and initial validity of a mobile application of the UCSD Performance-Based Skills Assessment

OBJECTIVE

This study aimed to develop and validate a tablet mobile application version of the UCSD Performance-Based Skills Assessment (UPSA-M), a widely used test for assessing functional capacity in schizophrenia and other neurocognitively impaired patient populations.

METHODS

The UPSA-M was developed for an iPad platform. Twenty-one middle-aged and older adults with schizophrenia and 13 healthy comparison (HC) participants completed a brief iPad tutorial, followed by the UPSA-M (full version, which includes all components of Brief UPSA-M) and a computer usage questionnaire. During a separate visit, patients with schizophrenia and HC participants completed the Brief UPSA (UPSA-B), and patients with schizophrenia completed a symptom inventory and brief neuropsychological screening battery.

RESULTS

The UPSA-M was feasible for use among middle aged and older adults with schizophrenia with no prior history of tablet usage. The UPSA-M was able to differentiate between schizophrenia and HC participants 80% of the time, and this differential ability increased to 87% with the UPSA-M Brief. Traditional UPSA scores, UPSA-B scores, and neuropsychological performance were related to UPSA-M scores, whereas symptoms of psychopathology, experience with tablet technology, or difficulties operating the device were not significantly associated with UPSA-M.

CONCLUSIONS

The UPSA-M performed just as well as the standard-of-practice version. These preliminary results indicate that the UPSA-M Brief has greater sensitivity than the full version of the UPSA-M, and carries the advantage of a shorter administration time. Overall, the UPSA-M appears to be a promising mobile tool to assess functional capacity.

Utility of the UCSD Performance-based Skills Assessment-Brief Japanese version: discriminative ability and relation to neurocognition

The UCSD Performance-based Skills Assessment Brief (the UPSA-B) has been widely used for evaluating functional capacity in patients with schizophrenia. The utility of the battery in a wide range of cultural contexts has been of concern among developers. The current study investigated the validity of the Japanese version of the UPSA-B as a measure of functional capacity and as a co-primary for neurocognion. Sixty-four Japanese patients with schizophrenia and 83 healthy adults entered the study. The Japanese version of the UPSA-B (UPSA-B Japanese version) and the MATRICS Cognitive Consensus Battery Japanese version (MCCB Japanese version) were administered. Normal controls performed significantly better than patients, with large effect sizes for the Total and the subscale scores of the UPSA-B. Receiver Operating Characteristic (ROC) curve analysis revealed that the optimal cut-off point for the UPSA-B Total score was estimated at around 80. The UPSA-B Total score was significantly correlated with the MCCB Composite score and several domain scores, indicating the relationship between this co-primary measure and overall cognitive functioning in Japanese patients with schizophrenia. The results obtained here suggest that the UPSA-B Japanese version is an effective tool for evaluating disturbances of daily-living skills linked to cognitive functioning in schizophrenia, providing an identifiable cut-off point and relationships to neurocognition. Further research is warranted to evaluate the psychometrical properties and response to treatment of the Japanese version of the UPSA-B.

Cortical substrates and functional correlates of auditory deviance processing deficits in schizophrenia

Although sensory processing abnormalities contribute to widespread cognitive and psychosocial impairments in schizophrenia (SZ) patients, scalp-channel measures of averaged event-related potentials (ERPs) mix contributions from distinct cortical source-area generators, diluting the functional relevance of channel-based ERP measures. SZ patients (n = 42) and non-psychiatric comparison subjects (n = 47) participated in a passive auditory duration oddball paradigm, eliciting a triphasic (Deviant−Standard) tone ERP difference complex, here termed the auditory deviance response (ADR), comprised of a mid-frontal mismatch negativity (MMN), P3a positivity, and re-orienting negativity (RON) peak sequence. To identify its cortical sources and to assess possible relationships between their response contributions and clinical SZ measures, we applied independent component analysis to the continuous 68-channel EEG data and clustered the resulting independent components (ICs) across subjects on spectral, ERP, and topographic similarities. Six IC clusters centered in right superior temporal, right inferior frontal, ventral mid-cingulate, anterior cingulate, medial orbitofrontal, and dorsal mid-cingulate cortex each made triphasic response contributions. Although correlations between measures of SZ clinical, cognitive, and psychosocial functioning and standard (Fz) scalp-channel ADR peak measures were weak or absent, for at least four IC clusters one or more significant correlations emerged. In particular, differences in MMN peak amplitude in the right superior temporal IC cluster accounted for 48% of the variance in SZ-subject performance on tasks necessary for real-world functioning and medial orbitofrontal cluster P3a amplitude accounted for 40%/54% of SZ-subject variance in positive/negative symptoms. Thus, source-resolved auditory deviance response measures including MMN may be highly sensitive to SZ clinical, cognitive, and functional characteristics.

•

Six source clusters contributing to the triphasic auditory deviance response were identified.

•

Source resolved responses are sensitive to SZ clinical, cognitive, and function characteristics.

•

Source resolved responses accounted for up to half the variance in cognitive and symptom scales.