Usability and Effectiveness of Immersive Virtual Grocery Shopping for Assessing Cognitive Fatigue in Healthy Controls: Protocol for a Randomized Controlled Trial

BACKGROUND

Cognitive fatigue (CF) is a human response to stimulation and stress and is a common comorbidity in many medical conditions that can result in serious consequences; however, studying CF under controlled conditions is difficult. Immersive virtual reality provides an experimental environment that enables the precise measurement of the response of an individual to complex stimuli in a controlled environment.

OBJECTIVE

We aim to examine the development of an immersive virtual shopping experience to measure subjective and objective indicators of CF induced by instrumental activities of daily living.

METHODS

We will recruit 84 healthy participants (aged 18-75 years) for a 2-phase study. Phase 1 is a user experience study for testing the software functionality, user interface, and realism of the virtual shopping environment. Phase 2 uses a 3-arm randomized controlled trial to determine the effect that the immersive environment has on fatigue. Participants will be randomized into 1 of 3 conditions exploring fatigue response during a typical human activity (grocery shopping). The level of cognitive and emotional challenges will change during each activity. The primary outcome of phase 1 is the experience of user interface difficulties. The primary outcome of phase 2 is self-reported CF. The core secondary phase 2 outcomes include subjective cognitive load, change in task performance behavior, and eye tracking. Phase 2 uses within-subject repeated measures analysis of variance to compare pre- and postfatigue measures under 3 conditions (control, cognitive challenge, and emotional challenge).

RESULTS

This study was approved by the scientific review committee of the National Institute of Nursing Research and was identified as an exempt study by the institutional review board of the National Institutes of Health. Data collection will begin in spring 2021.

CONCLUSIONS

Immersive virtual reality may be a useful research platform for simulating the induction of CF associated with the cognitive and emotional challenges of instrumental activities of daily living.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04883359; http://clinicaltrials.gov/ct2/show/NCT04883359.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/28073.

Developing an Executive Functioning Composite Score for Research and Clinical Trials

OBJECTIVE

Executive functioning encompasses interactive cognitive processes such as planning, organization, set-shifting, inhibition, self-monitoring, working memory, and initiating and sustaining motor and mental activity. Researchers therefore typically assess executive functioning with multiple tests, each yielding multiple scores. A single composite score of executive functioning, which summarizes deficits across a battery of tests, would be useful in research and clinical trials. This study examines multiple candidate composite scores of executive functioning using tests from the Delis-Kaplan Executive Function System (D-KEFS).

METHOD

Participants were 875 adults between the ages of 20 and 89 years from the D-KEFS standardization sample. Seven Total Achievement scores were used from three tests (i.e., Trail Making, Verbal Fluency, and Color-Word Interference) to form eight composite scores that were compared based on their psychometric properties and association with intelligence (IQ).

RESULTS

The distributions of most composite scores were mildly to severely skewed, and some had a pronounced ceiling effect. The composite scores all showed a medium positive correlation with IQ. The composite scores were highly intercorrelated in the total sample and in four IQ subgroups (i.e., IQ <89, 90-99, 100-109, 110+), with some being so highly correlated that they appear redundant.

CONCLUSIONS

This study is part of a larger research program developing a cognition endpoint for research and clinical trials with sound psychometric properties and utility across discrepant test batteries. Future research is needed to examine the reliability and ecological validity of these composite scores.

Multivariate base rates for the assessment of executive functioning among children and adolescents

This study provides observed base rates of low executive functioning test scores among healthy children and adolescents, stratifies those base rates by narrow intellectual functioning and age groupings, and provides normative classification ranges to aid the interpretation of performances on the Delis-Kaplan Executive Function System (D-KEFS) in clinical practice and research. Participants included 875 children and adolescents between 8 and 19 years old from the D-KEFS normative sample (48% male; 52% female). Among these participants, 838 had complete data and were included in the current study. The racial/ethnic composition of the sample was: White (73.7%), African American (12.4%), Hispanic (11.1%), and other racial/ethnic backgrounds (2.7%). The Overall Test Battery Mean (OTBM) and the prevalence of low scores at various clinical cut-offs were calculated for the 13 primary scores from the D-KEFS Trail Making Test, Verbal Fluency Test, and Color-Word Interference Test. The OTBM and base rates were also calculated separately for those scores reflecting executive functioning (n = 7) and processing speed (n = 6). Healthy children and adolescents commonly obtained low scores on the D-KEFS tests considered here. Younger age, lower estimated full-scale intelligence quotient, and more test scores interpreted were associated with a greater frequency of low scores. Clinicians and researchers are encouraged to consider these multivariate base rates when assessing and attempting to identify executive functioning impairment among children and adolescents with the D-KEFS.

Reliable Change on Memory Tests is Common in Healthy Children and Adolescents

Objective

Neuropsychologists interpret a large number of scores in their assessments, including numerous retest scores to determine change over time. The rate at which healthy children and adolescents obtain reliably improved or declined memory scores when retested has yet to be explored. The purpose of this study was to illustrate the prevalence of reliable change scores on memory test batteries in healthy children and adolescents.

Methods

Participants were children and adolescents from test-retest samples from two published memory test batteries (ChAMP and CMS). Reliable change scores (RCI with 90% confidence interval and practice effects) were calculated for the indexes and subtests of each battery. Multivariate base rates involved considering all change scores simultaneously within each battery and calculating the frequencies of healthy children obtaining one or more reliably declined or one or more reliably improved scores.

Results

Across both memory batteries, one or more reliably changed index or subtest score was common; however, reliable change on three or more scores was uncommon (i.e., found in <5% of the samples). Base rates of change scores did not differ by parent education.

Conclusions

Having a single reliably changed score on retest is common when interpreting these memory batteries. Multivariate interpretation is necessary when determining cognitive decline and cognitive recovery. Further research is warranted with other measures, other samples, and different retest intervals.

Nih toolbox premorbid ability adjustments: Application in a traumatic brain injury sample

PURPOSE/OBJECTIVE

Metrics to estimate premorbid cognitive ability, such as word reading tests, are important for clinical determination of cognitive changes following brain injury. In the present study, reading adjusted scores for the National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) fluid tests were developed and validated with a sample of individuals with traumatic brain injury (TBI), to evaluate the clinical utility of reading-adjusted scores. Research Method/Design: The development sample included 843 adult participants, ages 20-85, from the NIHTB-CB standardization sample. A sample of 158 participants with complicated mild or moderate TBI (n = 74) or severe TBI (n = 84) were administered the NIHTB-CB, and comprised the validation sample. Scores were derived for the five fluid tests using four adjustment models: age-only, demographic-only, age-and-reading, and demographic-and-reading referenced scores.

RESULTS

Estimated premorbid ability varies depending on the reference model. Scores from each of the four reference models differentiated the comparison and TBI samples at the group level. However, performance varied by premorbid ability.

CONCLUSIONS/IMPLICATIONS

Premorbid ability affects identification of cognitive difficulties after TBI. Reading referenced scores provide an individualized estimate of the effects of premorbid ability than demographic characteristics alone. Each model identified a similar number of individuals as having cognitive difficulties; however, the models differed on which individuals had cognitive difficulties. The models had higher disagreement rates in the clinical compared with the comparison sample, particularly for individuals with lower premorbid ability. Clinical use and caveats are discussed. (PsycINFO Database Record

NIH toolbox cognition tests following traumatic brain injury: Frequency of low scores

PURPOSE/OBJECTIVE

To apply multivariate base rate analyses to the National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) to facilitate the identification of cognitive impairment in individuals with traumatic brain injury (TBI). Research Method/Design: In a multisite cross-sectional design, 158 participants who sustained a complicated mild or moderate TBI (n = 74) or severe TBI (n = 84) at least 1 year earlier were administered the NIHTB-CB. The NIHTB-CB is comprised of 2 crystallized cognition tests (reflecting premorbid ability) and 5 fluid cognition tests, measuring processing speed, memory, and executive functioning. Base rates for obtaining 0 to 5 low fluid cognition scores were calculated across a range of cutoffs for defining a low test score (≤25th to 5th percentiles). Base rates of low scores in the TBI sample were compared to the NIHTB-CB normative sample using diagnostic accuracy statistics.

RESULTS

The proportion of the TBI sample obtaining low scores decreased as the cutoff for defining a low score decreased. Individuals with lower premorbid cognitive ability, as measured by NIHTB-CB Crystallized Composite score, tended to produce more low scores on the NIHTB-CB fluid cognition tests, even when using fully demographically adjusted scores. Certain patterns of low scores were associated with TBI (defined as likelihood ratio >2.0), whereas others were nonspecific, occurring almost as often in participants without TBI.

CONCLUSIONS/IMPLICATIONS

Premorbid ability stratified base rate tables provided in this article can guide researchers and clinicians in the interpretation of NIHTB-CB performance in adults with TBI. (PsycINFO Database Record

Cognition among community-dwelling individuals with spinal cord injury

PURPOSE/OBJECTIVE

To compare the cognitive profiles of a well-characterized sample of adults with and without spinal cord injury (SCI) using the NIH Toolbox Cognition Battery NIHTB-CB. Research Method/Design: Participants were 156 community-dwelling individuals with SCI recruited from 3 academic medical centers, and 156 individuals without SCI selected from the NIHTB-CB normative database (N = 312). The main outcome measures were the demographically adjusted NIHTB-CB subtest and composite scores.

RESULTS

Individuals with and without SCI performed equivalently on the NIHTB-CB crystallized composite score, suggesting comparable premorbid functioning. Individuals with SCI produced lower scores on the NIHTB-CB fluid composite score by an average of 4.5 T-score points (Cohen's d = 0.50; a medium effect size). As a group, individuals with SCI had the most difficulty on tests of processing speed and executive functions, and some difficulty on a test of episodic memory, although effect sizes were small. These differences remained even after accounting for fine motor speed and dexterity. Individuals with tetraplegia produced lower scores than individuals with paraplegia on tests of processing speed and executive functioning.

CONCLUSION/IMPLICATIONS

Community-dwelling individuals with SCI are at elevated risk of mild cognitive difficulties, particularly on tasks that rely on processing speed and executive functions. The NIHTB-CB is relatively brief, samples important cognitive domains, has good normative data, and is appropriate for some individuals with SCI (those who have functional use of one hand). The battery has standardized accommodations for individuals with minor motor limitations, but timed tests are inaccessible for individuals who are unable to perform rapid button presses. (PsycINFO Database Record

Factor structure of the NIH Toolbox Cognition Battery in individuals with acquired brain injury

OBJECTIVE

The National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) measures reading, vocabulary, episodic memory, working memory, executive functioning, and processing speed. While previous research has validated the factor structure in healthy adults, the factor structure has not been examined in adults with neurological impairments. Thus, this study evaluated the NIHTB-CB factor structure in individuals with acquired brain injury.

METHOD

A sample of 392 individuals (ages 18-84) with acquired brain injury (n = 182 TBI, n = 210 stroke) completed the NIHTB-CB along with neuropsychological tests as part of a larger, multisite research project.

RESULTS

Confirmatory factor analyses supported a 5-factor solution that included reading, vocabulary, episodic memory, working memory, and processing speed/executive functioning. This structure generally held in TBI and stroke subsamples as well as in subsamples of those with severe TBI and stroke injuries.

CONCLUSIONS

The factor structure of the NIHTB-CB is similar in adults with acquired brain injury to adults from the general population. We discuss the implications of these findings for clinical practice and clinical research. (PsycINFO Database Record

A comparison of PHQ-9 and TBI-QOL depression measures among individuals with traumatic brain injury

PURPOSE/OBJECTIVE

To compare and contrast how individuals with traumatic brain injury (TBI) are classified (positive or negative screen) by different cut-offs on two self-report measures of depressive symptoms: the PHQ-9, which assesses somatic symptoms, and the TBI-QOL Depression item bank, which does not. Research Method/Design: Three hundred eighty-five individuals with TBI were recruited from six rehabilitation hospitals in the U.S. as part of the calibration data collection for the TBI-QOL patient-reported outcome measurement system.

RESULTS

The TBI-QOL and PHQ-9 total scores correlated strongly (disattenuated r = .83). The correlation was even stronger (disattenuated r = .92) when the four PHQ-9 somatic items were removed from the total score. When the PHQ-9 was scored traditionally, the rate of agreement was approximately 80-85% using standard cut-offs for each scale. Depending on the cut-off score, 23-26% of participants screened positive on the PHQ-9, whereas 9-38% screened positive on the TBI-QOL Depression. Individuals who screened positive on the PHQ-9 alone reported more somatic symptoms than those who screened positive on the TBI-QOL alone. Individuals who screened positive on the TBI-QOL alone were at slightly greater risk for other negative psychological functioning than individuals who screened positive on the PHQ-9 alone.

CONCLUSIONS/IMPLICATIONS

The PHQ-9 and TBI-QOL Depression performed similarly in screening for depressive symptoms among individuals with TBI. The PHQ-9 identified more individuals with somatic symptoms, which may overlap with other medical issues, whereas the TBI-QOL Depression instrument identified more individuals who reported other forms of emotional distress. (PsycINFO Database Record

Interpreting Patterns of Low Scores on the NIH Toolbox Cognition Battery

INTRODUCTION

The National Institutes of Health Toolbox for Assessment of Neurological and Behavioral Function Cognition Battery is comprised of seven cognitive tests, including two tests measuring crystallized cognitive ability (i.e., vocabulary and reading) and five tests measuring fluid cognitive functioning (i.e., working memory, memory, speed of processing, and executive functioning). This study presents comprehensive base rate tables for the frequency of low scores in adults and older adults from the normative sample.

METHODS

Participants were 843 adults, ages 20-85, from the NIH Toolbox standardization sample who completed all seven cognition tests. Rates of low scores were derived for standard age-adjusted and fully-demographically-adjusted scores at multiple cut-scores. Base rates were stratified by education, crystallized intellectual ability, and cognitive domain.

RESULTS

Using the five demographically-adjusted fluid cognitive test scores, 45.9% of adults obtained one or more scores at or below the 16th percentile, and 16.8% obtained one or more score at or below the 5th percentile, which is consistent with findings from other neurocognitive test batteries.

DISCUSSION

Based on the study findings, nearly 50% of adults in the general population would meet psychometric criteria for a diagnosis of the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5) Mild Neurocognitive Disorder (MND). We developed new psychometric criteria for identifying MND using the NIH Toolbox Cognition Battery that reduce the false positive rate. Knowing these multivariate normative base rates will help researchers and clinicians interpret NIH Toolbox scores in people with neurodevelopmental, psychiatric, medical, neurological, and neurodegenerative disorders that affect cognitive functioning.

Using Multivariate Base Rates to Interpret Low Scores on an Abbreviated Battery of the Delis-Kaplan Executive Function System

OBJECTIVE

Executive function consists of multiple cognitive processes that operate as an interactive system to produce volitional goal-oriented behavior, governed in large part by frontal microstructural and physiological networks. Identification of deficits in executive function in those with neurological or psychiatric conditions can be difficult because the normal variation in executive function test scores, in healthy adults when multiple tests are used, is largely unknown. This study addresses that gap in the literature by examining the prevalence of low scores on a brief battery of executive function tests.

METHOD

The sample consisted of 1,050 healthy individuals (ages 16-89) from the standardization sample for the Delis-Kaplan Executive Function System (D-KEFS). Seven individual test scores from the Trail Making Test, Color-Word Interference Test, and Verbal Fluency Test were analyzed.

RESULTS

Low test scores, as defined by commonly used clinical cut-offs (i.e., ≤25th, 16th, 9th, 5th, and 2nd percentiles), occurred commonly among the adult portion of the D-KEFS normative sample (e.g., 62.8% of the sample had one or more scores ≤16th percentile, 36.1% had one or more scores ≤5th percentile), and the prevalence of low scores increased with lower intelligence and fewer years of education.

CONCLUSIONS

The multivariate base rates (BR) in this article allow clinicians to understand the normal frequency of low scores in the general population. By use of these BRs, clinicians and researchers can improve the accuracy with which they identify executive dysfunction in clinical groups, such as those with traumatic brain injury or neurodegenerative diseases.

To Change is Human: "Abnormal" Reliable Change Memory Scores are Common in Healthy Adults and Older Adults

OBJECTIVE

The rate at which people obtain reliably improved or declined cognitive test scores when retested, in the absence of a change in clinical condition, is largely unknown. The purpose of this study was to illustrate the prevalence of statistically reliable change scores on memory test batteries in healthy adults and older adults.

METHOD

Participants included three adult and older adult test-retest samples from memory test batteries. Reliable change scores (reliable change index with 90% confidence interval and practice effects) were calculated for the indexes and subtests of each battery. Multivariate analyses involved calculating the frequencies of healthy people obtaining one or more reliably declined or one or more reliably improved scores when considering all change scores simultaneously within each battery.

RESULTS

Across all batteries, having one or more reliably changed index or subtest score on retest was common. With most batteries, having two or more reliably changed scores was uncommon. Those with higher intellectual abilities were more likely to have a change on retest; however, no significant differences in base rates were found based on education level, sex, or ethnic minority status. Those older adults who did not have any low memory scores were more likely to improve than decline on retest.

CONCLUSIONS

Having a single reliably changed score on retest is common when interpreting a battery of memory measures. This has implications for determining cognitive decline and cognitive recovery, suggesting that multivariate interpretation is necessary.

The Brief Cognitive Status Examination (BCSE): Comparing Diagnostic Utility and Equating Scores to the Mini-Mental State Examination (MMSE)

The study purpose was to compare the diagnostic utility of the Brief Cognitive Status Exam (BCSE) to that of the Mini-Mental State Examination (MMSE) and to develop equated scores to facilitate comparisons. One hundred and eighty-two patients underwent cognitive evaluation and were placed into three groups: dementia (DEM), cognitive impairment, no dementia (CIND), and no cognitive impairment (NCI). One hundred and eighty-two healthy controls from the BCSE standardization sample served as a comparison group. On both measures, the DEM group obtained significantly lower scores than the other two groups, and the CIND group scored significantly lower than the NCI group. The BCSE was more sensitive in all clinical groups, although at extremely low scores, the two tests displayed similar sensitivity. Results indicate the BCSE has diagnostic utility as a cognitive screening measure in a mixed clinical sample and is more sensitive at detecting cognitive impairment, particularly milder levels, than the MMSE.

Intelligent Use of Intelligence Tests

It is well established that Canadians produce higher raw scores than their U.S. counterparts on intellectual assessments. As a result of these differences in ability along with smaller variability in the population’s intellectual performance, Canadian normative data will yield lower standard scores for most raw score points compared to U.S. norms. Two recent studies have questioned the utility of the WAIS–IV Canadian norms based on the performance of a mixed clinical sample of post-secondary students. These studies suggest that a greater proportion of cases from their mixed clinical samples fall below a full-scale IQ of 85 using the WAIS–IV Canadian norms than should be “expected.” The purpose of the current study is threefold: First, to summarize the consistent finding of Canada–U.S. differences on measures of ability and present new empirical analyses to demonstrate these results are not due to a smaller sample size for Canadian norms. Second, and most importantly, matched sample comparisons demonstrate that the proportion of low scoring individuals (FSIQ < 85) in mixed clinical samples is consistent with the rates published by recent studies, and not greater than expected. Third, we offer evidence-based advice to clinicians practicing in Canada on the appropriate use of Canadian norms for Canadian clients during an individual assessment of intellectual functioning.

Confirmatory factor analysis of the WAIS-IV/WMS-IV

The Wechsler Adult Intelligence Scale-fourth edition (WAIS-IV) and the Wechsler Memory Scale-fourth edition (WMS-IV) were co-developed to be used individually or as a combined battery of tests. The independent factor structure of each of the tests has been identified; however, the combined factor structure has yet to be determined. Confirmatory factor analysis was applied to the WAIS-IV/WMS-IV Adult battery (i.e., age 16-69 years) co-norming sample (n = 900) to test 13 measurement models. The results indicated that two models fit the data equally well. One model is a seven-factor solution without a hierarchical general ability factor: Verbal Comprehension, Perceptual Reasoning, Processing Speed, Auditory Working Memory, Visual Working Memory, Auditory Memory, and Visual Memory. The second model is a five-factor model composed of Verbal Comprehension, Perceptual Reasoning, Processing Speed, Working Memory, and Memory with a hierarchical general ability factor. Interpretative implications for each model are discussed.

Advanced Clinical Interpretation of the WAIS-IV and WMS-IV: Prevalence of Low Scores Varies by Level of Intelligence and Years of Education

Clinicians can use the base rates of low scores in healthy people to reduce the likelihood of misdiagnosing cognitive impairment. In the present study, base rates were developed for the Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV) and Wechsler Memory Scale—Fourth Edition (WMS-IV) using 900 healthy adults and validated on 28 patients with moderate or severe traumatic brain injuries (TBIs). Results indicated that healthy people obtain some low scores on the WAIS-IV/WMS-IV, with prevalence rates increasing with fewer years of education and lower predicted intelligence. When applying the base rates information to the clinical sample, the TBI patients were 13 times more likely to be identified as having a low cognitive profile compared with the controls. Using the base rates information is a psychometrically advanced method for establishing criteria to determine low cognitive abilities on the WAIS-IV/WMS-IV.

Validation of the adult ADHD investigator symptom rating scale (AISRS)

OBJECTIVE

Validation of the Adult ADHD Investigator Symptom Rating Scale (AISRS) that measures aspects of ADHD in adults.

METHOD

Psychometric properties of the AISRS total and AISRS subscales are analyzed and compared to the Conners' Adult Attention-Deficit/Hyperactivity Disorder Rating Scale-Investigator Rated: Screening Version (CAARS-Inv:SV) and the Clinical Global Impression-ADHD-Severity Scale using data from a placebo-controlled 6-month clinical trial of once-daily atomoxetine.

RESULTS

The AISRS has high internal consistency, good convergent, and discriminant validities; modest divergent validity; and small ceiling and floor effects (<or=1%). It correlates highly with the CAARS-Inv:SV. Factor analysis confirms 2 AISRS subscales, hyperactivity-impulsive scale and inattention. The AISRS total and AISRS subscales perform stably. All scales demonstrate responsiveness to change with medication.

CONCLUSION

The AISRS and its subscales are robust, valid efficacy measures of ADHD symptoms in adult patients. Its anchored items and semistructured interview are advancements over existing scales.

Minimizing misdiagnosis: psychometric criteria for possible or probable memory impairment

BACKGROUND/AIMS

Memory impairment can be easily misdiagnosed in older adults because obtaining some low scores is common. The objective of the present study is to present new psychometric criteria for determining 'possible' and 'probable' memory impairment.

METHODS

We propose criteria based on an analysis of performance from 450 healthy older adults (55-87 years old) on 3 measures from the WMS-III: Logical Memory, Word List, and Visual Reproduction. These measures yield 8 age-adjusted scores for learning, recall, and recognition. The proposed criteria for memory impairment are based on the prevalence of low scores when simultaneously examining all 8 scores and are stratified by current intelligence, estimated premorbid intelligence, and education. The criteria are subsequently validated on 100 healthy older adults and 34 patients with 'possible' or 'probable' Alzheimer's Disease (AD).

RESULTS

Tables with cutoffs and false-positive rates are presented for clinical use. In the validation cohort there were no misclassifications in AD patients.

CONCLUSION

This study presents steps in the development of proposed psychometric criteria that, in conjunction with clinical judgment, could minimize the misdiagnosis of memory impairment. It is important to reduce misdiagnosis in order to (a) optimize patient care, (b) provide an accurate foundation for identifying biological and neurological markers, and (c) successfully develop disease-modifying treatments. Further validation in a sample of older adults with lesser degrees of cognitive impairment is needed.

Development of set-shifting ability from late childhood through early adulthood

This cross-sectional study examined the development of set-shifting ability from childhood into early adulthood. Six hundred and forty-nine participants (aged 8-30) were assessed on the verbal fluency, design fluency, trail making, color-word interference, and card sorting subtests of the Delis-Kaplan Executive Function System (D-KEFS). Multiple regression analyses revealed modest effects of age and gender on set-shifting tasks, after controlling for IQ and component skills. The current study provides evidence for generally increased performance of set-shifting abilities through adolescence. Women overall had statistically better performance than men on all executive functioning tasks. There were significant age by gender interactions suggesting differential age-related improvements between men and women. On color-word interference and verbal fluency switching tasks, men tended to show larger improvements than women, whereas on a design fluency switching task, women showed larger improvements than men.

Cognitive discrepancies in children at the ends of the bell curve: a note of caution for clinical interpretation

Discrepancies between IQ scores on the Wechsler Abbreviated Scale of Intelligence (WASI) and scores from the Delis-Kaplan Executive Function System (D-KEFS) were examined at different levels of intellectual functioning in 470 normal-functioning youths (aged 8-19) from the co-standardization sample of the WASI and D-KEFS. Results demonstrated that children with lower IQ scores often had significantly higher D-KEFS scores, whereas children with higher IQ scores often had significantly lower D-KEFS scores. Similar patterns were identified for discrepancies between Verbal and Performance IQ indices. These findings are similar to those found in the adult literature. Clinicians are advised to be cautious when weighing the clinical significance of cognitive discrepancies at the ends of the bell-curve and should avoid interpreting discrepancies in isolation.

Equivalence of a Measurement Model of Cognitive Abilities in U.S. Standardization and Australian Neuroscience Samples

A psychological measurement model provides an explicit definition of (a) the theoretical and (b) the numerical relationships between observed scores and the latent variables that underlie the observed scores. Examination of the metric invariance of a measurement model involves testing the hypothesis that all components of the model relating observed scores to latent variables are equal across groups. The assumption of metric invariance is necessary for simple interpretation of scores. Establishing metric invariance also has implications for interpretation of convergent and divergent validity and patterns of deficit or disability. In this study the equivalence of the measurement model derived from the U.S. Wechsler Adult Intelligence Scale–III standardization sample was compared with a heterogeneous neurosciences sample in Australia. A pattern of strict metric invariance was observed across samples. These results provide evidence of the generality of the model underlying measurement of cognitive abilities.

Metric and structural equivalence of core cognitive abilities measured with the Wechsler Adult Intelligence Scale-III in the United States and Australia

Equivalence of the psychological model underlying Wechsler Adult Intelligence Scale-Third Edition (WAIS-III) scores obtained in the United States and Australia was examined in this study. Examination of metric invariance involves testing the hypothesis that all components of the measurement model relating observed scores to latent variables are numerically equal in different samples. The assumption of metric invariance is necessary for interpretation of scores derived from research studies that seek to generalize patterns of convergent and divergent validity and patterns of deficit or disability. An Australian community volunteer sample was compared to the US standardization data. A pattern of strict metric invariance was observed across samples. In addition, when the effects of different demographic characteristics of the US and Australian samples were included, structural parameters reflecting values of the latent cognitive variables were found not to differ. These results provide important evidence for the equivalence of measurement of core cognitive abilities with the WAIS-III and suggest that latent cognitive abilities in the US and Australia do not differ.

Interpreting Intelligence Test Results for Children with Disabilities: Is Global Intelligence Relevant?

School psychological and neuropsychological evaluations typically include intellectual and other standardized assessment tools in the identification of children with disabilities. The clinical utility of intellectual assessment in the identification and treatment of these children has been repeatedly challenged, with alternatives such as a response to intervention or global intelligence score interpretation offered to replace the long-held tradition of idiographic interpretation of intellectual factors or subtests for the purpose of differential diagnosis and individualized intervention. Replicating previous work, this study examined the structure of intellectual functioning for children diagnosed with Learning Disability (LD; n=128), Attention-Deficit/Hyperactivity Disorder (ADHD; n=71), and traumatic brain injury (TBI; n=29) using regression commonality analysis. Across groups, results provide substantial evidence for a multifactorial representation of intellectual functioning for children with LD, ADHD, or TBI, with little shared variance among factor predictors of FSIQ in each analysis. As global intellectual functioning, represented by the shared variance among all predictors, was largely absent and instead composed of several discrete elements with the requisite specificity for individual interpretation, idiographic interpretation appears to be warranted for children with disabilities.

Age-related invariance of abilities measured with the Wechsler Adult Intelligence Scale-III

Examination of measurement invariance tests the assumption that the model underlying a set of test scores is directly comparable across groups. The observation of measurement invariance provides fundamental evidence for the inference that scores on a test afford equivalent measurement of the same psychological traits among diverse groups. Groups may be derived from different psychosocial backgrounds or different clinical presentations. In the Wechsler Adult Intelligence Scale-III (WAIS-III)/Wechsler Memory Scale-III (WMS-III) Technical Manual (Psychological Corporation, 2002), there appears to be a breakdown in factor structure among the standardization cases in older adults. In this study, the authors evaluated the invariance of the measurement model of the WAIS-III across 5 age bands. All components of the measurement model were examined. Overall, the evidence pointed to invariance across age of a modified 4-factor model that included cross-loadings for the Similarities and Arithmetic subtests. These results support the utility of the WAIS-III as a measure of stable intelligence traits across a wide age range.

The California Verbal Learning Test--second edition: test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms

The California Verbal Learning Test - second edition (CVLT-II) is one of the most widely used neuropsychological tests in North America. The present study evaluated the 1-month test-retest reliability and practice effects associated with the standard and alternate forms of the CVLT-II in a sample of 195 healthy adults. Eighty participants underwent repeat assessment using the standard form of the CVLT-II on both occasions, whereas the remaining 115 individuals received the standard form at baseline and the alternate form at follow-up. Consistent with prior research, results revealed generally large test-retest correlation coefficients for the primary CVLT-II measures in both the standard/standard (range=0.80-0.84) and standard/alternate (range=0.61-0.73) cohorts. Despite exhibiting slightly lower test-retest reliability coefficients, participants in the alternate form group displayed notably smaller practice effects (Cohen's d range=-0.01 to 0.18) on the primary indices relative to individuals who received the standard form on both occasions (Cohen's d range=0.27-0.61). Reliable change indices were also generated and applied to primary CVLT-II variables to determine the base rates of significant improvements (range=2-10%), declines (range=0-7%), and stability (range=85-97%) in performance over time. Overall, findings from this study support the test-retest reliability of the standard and alternate forms of the CVLT-II in healthy adults and may enhance the usefulness of this test in longitudinal neuropsychological evaluations.

Parsing the recognition memory components of the WMS-III face memory subtest: normative data and clinical findings in dementia groups

The WMS-III face memory subtest was developed as a quick, reliable, measure of non-verbal recognition memory. While the face memory subtest has demonstrated clinical sensitivity, the test has been criticized for low correlation with other WMS-III visual memory subtests and for failing to differentiate performance between clinical groups. One possible reason for these findings may be due to the impact of response bias associated with recognition memory tests. Four studies were conducted to evaluate the utility of applying signal detection measures to the face memory subtests. The first two studies used the WMS-III standardization data set to determine age and education effects and to present normative and reliability data for hits, false positives, discriminability and response bias. The third study tested the hypothesis that using response components and signal detection measures would enhance the correlation between face memory and the other WMS-III visual memory subtests. The fourth study compared performance of patients with Alzheimer's disease, Huntington's disease, Korsakoff's syndrome and demographically matched controls on the new face memory scores. The new measures did not have higher correlation with other WMS-III visual memory measures than the standard scoring of the test. Analysis of the clinical samples indicated that the discriminability index best differentiated patients from controls. The response components, particularly delayed false positives, differentiated performance among the clinical groups. Normative and reliability data are presented.

MMPI characteristics in adults diagnosed with ADD: a preliminary report

Characterization of Attention Deficit Disorder (ADD) in adults has been hampered by an absence of well-defined clinical profiles on objective personality measures. A sample of 22 ADD adults completed personality testing (MMPI) in conjunction with cognitive assessment. A sample of 30 normal controls and 20 patients diagnosed with schizophrenia, who had completed the MMPI as part of a larger research protocol, were utilized as comparison groups. Validity and clinical scales revealed multiple elevations in the ADD group which were similar in nature but not to the same degree as elevations observed in the schizophrenia group. Harris-Lingoes subscales identified the sources of clinical scale elevations indicating a pattern of specific and nonspecific symptomatology which differentiated the three groups. The results may aid in identifying ADD in adults.