The RBANS Effort Index: base rates in geriatric samples

Using supplemental memory measures to refine interpretation of the repeatable battery for the assessment of neuropsychological status

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is frequently used as a cognitive screening measure or as part of a comprehensive neuropsychological battery. Augmenting the current memory subtests of the RBANS to reflect clinically meaningful distinctions in memory performance may improve its clinical utility, allowing users to generate additional hypotheses and refine clinical interpretations. We pilot four supplementary memory measures to use with the RBANS, adapted from paradigms commonly used by other neuropsychological tests. We also provide several new theoretically derived memory indices to supplement the standard Delayed Memory Index. The new subtests correlated significantly with the standard memory measures, and the new indices demonstrated good reliability and diagnostic accuracy. This study provides preliminary support for supplementing the RBANS to allow for more nuanced interpretations of memory performance.

Huntington study group's neuropsychology working group position on best practice recommendations for the clinical neuropsychological evaluation of patients with Huntington disease

Objective: Neuropsychological evaluation is critical to detection and management of cognitive and neuropsychiatric changes associated with Huntington disease (HD). Accurate assessment of non-motor complications of HD is critical given the prominent impact on functional disability, frequently commensurate with or exceeding that of motor symptoms. The increasing emphasis on developing disease-modifying therapies targeting cognitive decline in HD requires consensus on clinical neuropsychological assessment methods. The Neuropsychology Working Group (NPWG) of the Huntington Study Group (HSG) sought to provide evidence and consensus-based, practical guidelines for the evaluation of cognitive and neuropsychiatric symptoms associated with HD. Method: The NPWG recruited a multi-disciplinary group of neuropsychologists, neurologists, and psychiatrists to inform best practices in assessing, diagnosing, and treating the non-motor symptoms in HD. A review was circulated among the NPWG, and in an iterative process informed by reviewed literature, best practices in neuropsychological evaluation of patients with HD were identified. Results: A brief review of the available literature and rational for a clinical consensus battery is offered. Conclusion: Clinical neuropsychologists are uniquely positioned to both detect and characterize the non-motor symptoms in HD, and further, provide neurologists and allied health professions with clinically meaningful information that impacts functional outcomes and quality of life. The NPWG provides guidance on best practices to clinical neuropsychologists in this statement. A companion paper operationalizing clinical application of previous research-based non-motor diagnostic criteria for HD is forthcoming, which also advises on non-motor symptom screening methods for the non-neuropsychologist working with HD.

Examining the repeatable battery for the assessment of neuropsychological status validity indices in people with schizophrenia spectrum disorders

Objective: We examined the frequency of possible invalid test scores on the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in patients with schizophrenia spectrum disorders, and whether there was an association between scores on the embedded RBANS performance validity tests (PVTs) and self-reported symptoms of apathy as measured by the Initiate Scale of the Behavior Rating Inventory of Executive Function-Adult Version (BRIEF-A). Methods: Participants included 250 patients (M = 24.4 years-old, SD = 5.7) with schizophrenia spectrum disorders. Base rates of RBANS Effort Index (EI), Effort Scale (ES), and Performance Validity Index (PVI) test scores were computed. Spearman correlations were used to examine the associations between the RBANS PVTs, the RBANS Index scores, and the BRIEF-A Initiate Scale. Regression analyses were used to investigate how well the RBANS PVTs predicted scores on the BRIEF-A Initiate Scale. Results: The frequency of invalid scores on the EI (>3) and the PVI (<42) in participants with schizophrenia spectrum disorders was 6%. The frequency of invalid ES scores (<12) was 28% in the patients compared to 15% in the U.S. standardization sample. There was a small significant correlation between the EI and the BRIEF-A Initiate Scale (rho=.158, p<.05). Conclusions: The rates of invalid scores were similar to previously published studies. Invalid scores on the BRIEF-A were uncommon. Apathy measured with the BRIEF-A Initiate Scale was not associated with performance on the RBANS validity measures or with measures of cognition.

Racial differences in positive findings on embedded performance validity tests

INTRODUCTION

Embedded performance validity tests (PVTs) may show increased positive findings in racially diverse examinees. This study examined positive findings in an older adult sample of African American (AA) and European American (EA) individuals recruited as part of a study on aging and cognition.

METHOD

The project involved secondary analysis of deidentified National Alzheimer's Coordinating Center data (N = 22,688). Exclusion criteria included diagnosis of dementia (n = 5,550), mild cognitive impairment (MCI; n = 5,160), impaired but not MCI (n = 1,126), other race (n = 864), and abnormal Mini Mental State Examination (MMSE < 25; n = 135). The initial sample included 9,853 participants (16.4% AA). Propensity score matching matched AA and EA participants on age, education, sex, and MMSE score. The final sample included 3,024 individuals with 50% of participants identifying as AA. Premorbid ability estimates were calculated based on demographics. Failure rates on five raw score and six age-adjusted scaled score PVTs were examined by race.

RESULTS

Age, education, sex, MMSE, and premorbid ability estimate were not significantly different by race. Thirteen percent of AA and 3.8% of EA participants failed two or more raw score PVTs (p < .0001). On age-adjusted PVTs, 20.6% of AA and 5.9% of EA participants failed two or more (p < .0001).

CONCLUSIONS

PVT failure rates were significantly higher among AA participants. Findings indicate a need for cautious interpretation of embedded PVTs with underrepresented groups. Adjustments to embedded PVT cutoffs may need to be considered to improve diagnostic accuracy.

RBANS Validity Measures and the TOMM in Veterans Undergoing Seizure Monitoring

OBJECTIVE

To assess the validity of embedded measures of performance validity, the effort index (EI) and effort scale (ES) of the repeatable battery for the assessment of neuropsychological status (RBANS), in Veterans with seizures; to compare the frequency of failure on the test of memory malingering (TOMM) in patients with epileptic versus psychogenic nonepileptic seizures (PNES).

METHODS

Seizure diagnosis was established for 309 participants in epilepsy monitoring units using conventional diagnostic criteria who completed both the TOMM and RBANS. The criterion for performance invalidity was failure on any trial of the TOMM. We examined multiple EI and ES cutoffs to establish optimal sensitivity and specificity.

RESULTS

An RBANS EI cutoff score of greater than three was optimal with specificity of .98, sensitivity of.19, and positive Likelihood Ratio of 10 but was not useful when below this cutoff. Confidence intervals indicate the need for confirmation of a failed EI with another performance validity test (PVT). No ES cutoff had sufficient specificity for clinical use. Invalid TOMM performance but not invalid RBANS performance was significantly more common in persons with PNES than in persons with epileptic seizures.

CONCLUSIONS

In Veterans undergoing seizure monitoring, the RBANS EI was useful as a screen when positive that requires confirmation with another PVT. The RBANS ES was not useful. Invalid performance on the TOMM was more common in persons with PNES than in persons with epileptic seizures.

Cognitive Screening Tools for Late Career Physicians: A Critical Review

Screening measures are widely used in medicine to assess the increased probability that members of a defined population have a particular condition and therefore require more extensive assessment. The rationale for prospective screening of late career physicians (LCPs) is drawn from the following circumstances: Senior physicians-prone to the vicissitudes of aging-comprise nearly a third of the US physician workforce, physicians are poor at self-evaluation, data suggest many have clinically relevant cognitive decline, and screening is an evidence-based, method to detect individuals at risk and determine whether a comprehensive evaluation is necessary. A handful of professional organizations (eg, surgeons, obstetricians, and a growing number of medical staff credentialing committees) have developed policies in this arena. This focused review compares cognitive screening methods used or recommended for LCPs, with particular attention to the psychometric properties, ease of operational implementation, and appropriate application to physicians-a population selected for high cognitive reserve and skills. Further, we identify gaps in knowledge and practice, including the need for more career-span normative data on physicians' cognitive and work performance. Stakeholders can improve rehabilitation and other supports to LCPs in transition, calling upon the unique expertise of those neuropsychologists who are trained on conducting fitness for duty evaluations, as well as rehabilitation professionals who can assist in developing modifications to practice when indicated or facilitate graceful transitions to retirement when necessary.

A Meta-Analysis of Neuropsychological Effort Test Performance in Psychotic Disorders

Psychotic disorders are characterized by a generalized neurocognitive deficit (i.e., performance 1.5 SD below controls across neuropsychological domains with no specific profile of differential deficits). A motivational account of the generalized neurocognitive deficit has been proposed, which attributes poor neuropsychological testing performance to low effort. However, findings are inconsistent regarding effort test failure rate in individuals with psychotic disorders across studies (0-72%), and moderators are unclear, making it difficult to know whether the motivational explanation is viable. To address these issues, a meta-analysis was performed on data from 2205 individuals with psychotic disorders across 19 studies with 24 independent effects. Effort failure rate was examined along with moderators of effort test type, forensic status, IQ, positive symptoms, negative symptoms, diagnosis, age, gender, education, and antipsychotic use. The pooled weighted effort test failure rate was 18% across studies and there was a moderate pooled association between effort failure rate and global neurocognitive performance (r = .57). IQ and education significantly moderated failure rate. Collectively, these findings suggest that a nontrivial proportion of individuals with a psychotic disorder fail effort testing, and failure rate is associated with global neuropsychological impairment. However, given that effort tests are not immune to the effects of IQ in psychotic disorders, these results cannot attest to the viability of the motivational account of the generalized neurocognitive deficit. Furthermore, the significant moderating effect of IQ and education on effort test performance suggests that effort tests have questionable validity in this population and should be interpreted with caution.

Effort Testing in Dementia Assessment: A Systematic Review

Objective

Interpretation of neuropsychological test data is only valid when appropriate effort has been exerted. Research, however, suggests that neuropsychologists do not always formally test for effort and that this may especially be the case in the context of dementia assessment. This review systematically examined the literature that has investigated the use of both purpose-built and embedded effort-sensitive indices in dementia, mild cognitive impairment (MCI), and healthy control samples. The aim was to determine which tests of effort are most sensitive to suboptimal effort and least sensitive to the type of cognitive impairment seen in dementia.

Methods

A systematic search of databases was conducted to October 2017. There was no start date.

Results

Twenty-five studies were included for review. The studies were divided into two categories according to methodology. One category of studies (n = 5) was reviewed using a tailored methodological quality rating checklist whilst the remaining studies (n = 20) were reviewed using the Crowe Critical Appraisal Tool (CCAT).

Conclusions

The results of this review suggest that PVTs which take a hierarchical approach to effort testing such as the WMT, MSVT, and NV-MSVT are preferable for use with older adults who are under investigation for possible dementia. These tests go above and beyond the traditional pass/fail approach of more traditional tests of effort because they allow the examiner to analyze the examinee's profile of scores. The methodological limitations and challenges involved in this field of research are discussed.

A meta-analysis of the accuracy of embedded performance validity indicators from the repeatable battery for the assessment of neuropsychological status

Objective: Meta-analyze the embedded performance validity tests of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), emphasizing two topics: diagnostic accuracy and relationships with demographics. Method: Literature was identified through a review of PsycINFO, PubMed, and Google Scholar. Bivariate linear mixed models were used for diagnostic accuracy analyses. Random-effects analysis of correlations with demographic variables was also used for the Effort Index. Results: Diagnostic accuracy analyses utilized 12 studies of the Effort Index (EI; N = 1469) and 6 studies of the Effort Scale (ES; N = 854). The EI was found to have a sensitivity of .44 (95% CI: .28 - .62), specificity of .87 (95% CI: .78 - .93), and diagnostic odds ratio (DOR) of 5.41 (95% CI: 3.74 - 7.58). The ES produced a sensitivity of .67 (95% CI: .50 - .81), specificity of .72 (95% CI: .53 - .85), and DOR of 5.97 (95% CI: 1.70 - 15.30). Meta-analysis of correlations utilized 14 samples of the EI (N = 3781), but insufficient data were available for the ES. Correlations between the EI and age (r = .10, 95% CI: .05 - .16), education (r = -.11, 95% CI: -.18 to -.04), and the RBANS Total Scale (r = -.45, 95% CI: -.62 to -.24) were significant. These results appeared to be robust to publication bias. Conclusion: Results support use of the EI over the ES; however, the latter is comparatively less studied and did not have similar variability in reported cut-off scores. ES accuracy statistics were significantly predicted by sample variables, which may be related to its relatively fewer studies. Both measures should be used cautiously in the context of genuine cognitive impairment.

Performance validity in older adults: Observed versus predicted false positive rates in relation to number of tests administered

INTRODUCTION

This study examined false positive rates on embedded performance validity tests (PVTs) in older adults grouped by cognitive status.

METHOD

The research design involved secondary analysis of data from the National Alzheimer's Coordinating Center database. Participants (N = 22,688) were grouped by cognitive status: normal (n = 10,319), impaired (n = 1,194), amnestic or nonamnestic mild cognitive impairment (MCI; n = 5,414), and dementia (n = 5,761). Neuropsychological data were used to derive 5 PVTs.

RESULTS

False positive rates on individual PVTs ranged from 3.3 to 26.3% with several embedded PVTs showing acceptable specificity across groups. The proportion of participants failing two or more PVTs varied by cognitive status: normal (1.9%), impaired (6.6%), MCI (13.2%), and dementia (52.8%). Comparison of observed and predicted false positive rates at different specificity levels (.85 or .90) demonstrated significant differences in all comparisons. In normal and impaired groups, predicted rates were higher than observed rates. In the MCI group, predicted and observed comparisons varied: Predicted rates were higher with specificity at .85 and lower with specificity at .90. In the dementia group, predicted rates underestimated observed rates.

CONCLUSIONS

Despite elevated false positives in conditions involving severe cognitive compromise, several measures retain acceptable specificity regardless of cognitive status. Predicted false positive rates based on the number of PVTs administered were not observed empirically. These findings do not support the utility of simulated data in predicting false positive rates in older adults.

Exploratory analysis of CVLT-II forced-choice recognition and RBANS EI base rates for forensically-committed, psychotic-disordered inpatients: Implications on clinical decision-making

Within the inpatient forensic setting, requests for a neuropsychological evaluation are common to determine cognitive strengths and weaknesses. However, variable effort proves to be a prominent issue in this setting. Thus, assessment of effort becomes an essential component of a neuropsychological evaluation. The California Verbal Learning Test, 2nd Edition (CVLT-II) and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) are widely utilized cognitive measures in the inpatient setting. The purpose of this study is to establish the local base rates and predictive abilities of the CVLT-II Forced-Choice Recognition (FCR) measure and RBANS Effort Index (EI). Participants included 56 and 595 forensically-committed, psychotic-disordered inpatients who completed the CVLT-II FCR and RBANS EI, respectively. Results indicated that the estimated local base rate for a positive CVLT-II FCR score was 8%, which resulted in 97% negative predictive power and 50% positive predictive power. The estimated local base rate for a positive RBANS EI score was 16%, which resulted in 91.7% negative predictive power and 57.1% positive predictive power. Given their low sensitivity and predictive power, the results suggest that much more confidence can be placed in negative FCR and EI results as opposed to positive findings.

A brief neurocognitive assessment of patients with psychosis following traumatic brain injury (PFTBI): Use of the Repeatable battery for the Assessment of Neuropsychological Status (RBANS)

Patients who develop psychosis following a traumatic brain injury (PFTBI) show impaired neurocognition; however, the degree of impairment has not been empirically investigated using a standardised battery. We administered the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) to patients with PFTBI (n=10), and to three groups of controls: traumatic brain injury (TBI) (n=10), schizophrenia (n=23), and nonclinical controls (n=23). The results confirmed that the cognitive neuropsychological profile of dually-diagnosed patients with PFTBI is significantly and substantially impaired. Seventy per cent of patients with PFTBI received a neuropsychological classification between the "extremely low" and "low average" ranges. Group-wise analyses on the RBANS indices indicated that patients with PFTBI had the lowest (Immediate Memory, Attention, Delayed Memory, Total Score), or equal lowest (visuospatial, equivalent with schizophrenia patients) scores, with the exception of the Language Index where no group differences were shown (however, the mean PFTBI score on the Language Index was two standard deviations below the RBANS normative score). These findings provide novel evidence of impaired cognitive neuropsychological processing in patients with PFTBI using a standardised and replicable battery.

Effort in acute traumatic brain injury: considering more than pass/fail

PURPOSE/OBJECTIVE

Performance validity is often conceptualized as a dichotomous process. Effort likely lies on a continuum, however, and psychologists' tendency to rely on pass/fail descriptors of one's effort may not be the only approach. The current study aims to show that when performance validity is considered on a continuum, it may provide clinical information related to cognitive functioning.

RESEARCH METHOD/DESIGN

Forty-four patients with moderate or severe traumatic brain injury were evaluated with the Repeatable Battery for the Assessment of Neuropsychological Status upon their emergence from posttraumatic amnesia. From this data, previously developed effort index scores and "other cognitive functions" index scores were calculated.

RESULTS

Performance on the effort index significantly accounted for the patients' performance on a cognitive composite score after considering education and severity of injury.

CONCLUSIONS/IMPLICATIONS

Findings suggest that more in-depth analysis of validity test performance is beneficial to gauge a patient's level of effort and is important to consider when interpreting results and in treatment planning.

RBANS embedded measures of suboptimal effort in dementia: effort scale has a lower failure rate than the effort index

The importance of evaluating effort in neuropsychological assessments has been widely acknowledged, but measuring effort in the context of dementia remains challenging due to the impact of dementia severity on effort measure scores. Two embedded measures have been developed for the repeatable battery for the assessment of neuropsychological status (RBANS; Randolph, C., Tierney, M. C., Mohr, E., & Chase, T. N. (1998). The repeatable battery for the assessment of neuropsychological status (RBANS): Preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology, 20 (3), 310-319): the Effort Index (EI; Silverberg, N. D., Wertheimer, J. C., & Fichtenberg, N. L. (2007). An effort index for the repeatable battery for the assessment of neuropsychological status (RBANS). Clinical Neuropsychologist, 21 (5), 841-854) and the Effort Scale (ES; Novitski, J., Steele, S., Karantzoulis, S., & Randolph, C. (2012). The repeatable battery for the assessment of neuropsychological status effort scale. Archives of Clinical Neuropsychology, 27 (2), 190-195). We explored failure rates on these effort measures in a non-litigating mixed dementia sample (N = 145). Failure rate on the EI was high (48%) and associated with dementia severity. In contrast, failure on the ES was 14% but differed based on type of dementia. ES failure was low (4%) when dementia was due to Alzheimer disease (AD), but high (31%) for non-AD dementias. These data raise concerns about use of the RBANS embedded effort measures in dementia evaluations.

A Bayesian latent group analysis for detecting poor effort in a sample of cognitively impaired patients

Using a Bayesian latent group analysis in a simulation design, we recently showed a high diagnostic accuracy when assessing effort in the context of malingered memory deficits. We here further evaluate our Bayesian model in a sample of cognitively impaired patients. The main analysis showed both high sensitivity and specificity, thus corroborating a high diagnostic accuracy of the model. Additional analysis showed variations on effort estimates after changes in malingering base rates. Variations affected sensitivity, but not specificity, which is in line with typical findings in malingering research. These data suggest that Bayesian analyses may complement and improve existing effort measures.

Using the repeatable battery for the assessment of neuropsychological status (RBANS) effort index to predict treatment group attendance in patients with schizophrenia

In a psychosocial treatment study, knowing which participants are likely to put forth adequate effort to maximize their treatment, such as attending group sessions and completing homework assignments, and knowing which participants need additional motivation before engagement in treatment is a crucial component to treatment success. This study examined the ability of the Repeatable Battery for Assessment of Neuropsychological Status (RBANS) Effort Index (EI), a newly developed measure of suboptimal effort that is embedded within the RBANS, to predict group attendance in a sample of 128 middle-aged and older adults with schizophrenia. This study was the first to evaluate the EI with a schizophrenia sample. While the EI literature recommends a cutoff score of >3 to be considered indicative of poor effort, a cutoff of >4 was identified as the optimal cutoff for this sample. Receiver Operating Characteristics curve analyses were conducted to determine if the EI could predict participants who had high versus low attendance. Results indicated that the EI was successfully able to discriminate between group attendance, and this measure of effort appears to be most valuable as a tool to identify participants who will have high attendance. Of interest, overall cognitive functioning and symptoms of psychopathology were not predictive of group attendance.

Symptom validity test performance in the Huntington Disease Clinic

Symptom validity tests (SVTs) are often used in neuropsychological assessment; however, recent studies indicate that cognitive impairment/dementia may contribute to failing scores on some effort tests. The purpose of this study was to characterize how individuals with Huntington disease (HD) perform on three SVTs and to examine the relationship between SVT performance and demographic and clinical variables. Results indicate that while the majority of HD patients passed the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Effort Index (EI; 82% of n = 121) and the Test of Memory Malingering (92% of n = 36), failure of these SVTs was associated with poorer cognitive and adaptive functioning, and greater motor impairment. Results showed that less than one-third passed the RBANS Effort Scale (ES; 30% of n = 43) and few clinical and demographic variables were correlated with this SVT performance. Although some SVTs may be better suited to HD, cognitive ability should be considered when evaluating effort in HD.

Validation of the repeatable battery for the assessment of neuropsychological status--effort index in a veteran sample

The RBANS Effort Index (RBANS-EI; Silverberg, Wertheimer, & Fichtenberg, 2007) is an embedded measure of effort within a frequently employed neuropsychological screening battery. While it has been criticized for inadequate specificity in older non-litigating samples (Hook, Marquine, & Hoelzle, 2009; Warren et al., 2010), the RBANS-EI has yet to be investigated in a non-geriatric veteran sample. Archival data were collected from 85 veterans who completed the RBANS and WMT within either a routine neuropsychological evaluation (n = 66) or compensation evaluation (n = 19). At a cutoff of >3 RBANS-EI exhibited strong specificity (.94) yet limited sensitivity (.31) in the prediction of WMT performance. Examination of RBANS-EI component subtests found that List Recognition <17 had strong specificity (.90) and moderate sensitivity (.52) in discriminating WMT performance groups. In contrast, Digit Span performance was comparable between those passing and failing the WMT. Present findings indicate that both the RBANS-EI and List Recognition subtest may be useful in detecting suboptimal effort yet raise questions regarding the Digit Span component of the RBANS-EI.

Specificity of the 21-Item Test in two elderly samples

The purpose of this study was to measure the specificity of the 21-Item Test for detecting malingering in an elderly sample. The 21-Item Test was administered to 20 community-dwelling elderly individuals with means for age, education, and Short Portable Mental Status Questionnaire (SPMSQ) error scores of 76.25 years (SD = 7.19), 14.30 years (SD = 3.89), and 0.30 (SD = 4.70). None of these healthy elderly participants were cognitively impaired. A second group of 53 nursing home residents had means for age, education, and SPMSQ error scores of 82.79 years (SD = 8.56), 11.14 years (SD = 2.46), and 3.17 (SD = 2.46). Approximately 60% of the nursing home participants had cognitive impairment based on the SPMSQ. The cut offs provided by Iverson (1998 , 21 Item Test Research Manual) for interpretation of the forced-choice component of the 21-Item Test were insensitive to age and relatively insensitive to cognitive deficits. Specificity was 100% since no elderly participant from either group scored in the range indicative of sub-optimal effort or biased responding.

Brief report: the temporal stability of the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index in geriatric samples

The Effort Index (EI) of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) was developed to identify inadequate effort. Although researchers have examined its validity, the reliability of the EI has not been evaluated. The current study examined the temporal stability of the EI across 1 year in two independent samples of older adults. One sample consisted of 445 cognitively intact older adults (mean age = 72.89; 59% having 12-15 years of education) and the second sample consisted of 51 individuals diagnosed with amnestic Mild Cognitive Impairment (mean age = 82.41; 41% having 12-15 years of education). For both samples, the EI was found to have low stability (Spearman's ρ = .32-.36). When participants were divided into those whose EI stayed stable or improved versus those whose EI worsened (i.e., declining effort) on retesting, it was observed that individuals with lower baseline RBANS Total scores tended to worsen on the EI across time. Overall, the findings suggest low temporal stability of the EI in two geriatric samples. In particular, individuals with poorer cognition at baseline could present with poorer effort across time. These findings also suggest the need to further examine the temporal stability of other effort measures.