Multivariate base rates for the assessment of executive functioning among children and adolescents

The Frequency of Low Scores on ImPACT in Adolescent Student-Athletes: Stratification by Race and Socioeconomic Status Using Multivariate Base Rates

This study examined the associations between the frequency of low scores on the Immediate Post-Concussion Assessment and Cognitive Test (ImPACT) by race and socioeconomic status (SES), using the proxy of Title I school status, among adolescent student-athletes and calculated multivariate base rates. There were 753 participants assigned to groups based on race (White: n = 430, 59.8%; Black: n = 289, 40.2%) and SES. Black student-athletes obtained more low neurocognitive test scores, which was associated with lower SES. The current study offers a resource to clinicians involved in concussion management who may wish to consider race and SES when interpreting ImPACT test performances.

Enhanced interpretation of the BRIEF2: multivariate base rates of elevated scores in the standardization samples

The Behavior Rating Inventory of Executive Function, Second Edition (BRIEF2) is a standardized rating (self, parent, and teacher) scale of executive functioning in children and adolescents. Here, we provide multivariate base rate (MBR) information (for the Self, Parent, and Teacher forms), which is not included in the BRIEF2 Professional Manual. Participants were children and adolescents for the BRIEF2 Self-Report (ages = 11-18; N = 803), Parent-Report (ages = 5-18; N = 1,400), and Teacher-Report (ages = 5-18; N = 1,400) standardization samples. We focused on cumulative (e.g., % of sample with oneor more elevated scores) MBRs across scales, which were examined at three elevation levels on each form: T≥ 60, ≥65, and ≥70. Across forms, MBRs predictably decreased with increasing number of elevated scores and at higher cutoffs. The cumulative MBR of having at least one score at T≥ 60 was common (37.5-42.2%), but less frequent at T≥ 70 (15.4-17.4%). The probability of having elevated scores on all scales was very low, irrespective of form, age, or elevation threshold (T≥ 60 = 2.4-4.4%; T≥ 65 = 1.0-1.4%; T≥ 70 = 0.0-0.7%). There was no clinically meaningful relation between demographic factors (age, gender, race, and parental education) and MBRs. These data provide clinicians and researchers with an enhanced way of concurrently interpreting multiple BRIEF2 scales.

Improving the Methodology for Identifying Mild Cognitive Impairment in Intellectually High-Functioning Adults Using the NIH Toolbox Cognition Battery

Objective: Low scores on neuropsychological tests are considered objective evidence of mild cognitive impairment. In clinical practice and research, it can be challenging to identify a cognitive deficit or mild cognitive impairment in high-functioning people because they are much less likely to obtain low test scores. This study was designed to improve the methodology for identifying mild cognitive impairment in adults who have above average or superior intellectual abilities.

Method: Participants completed the National Institutes of Health Toolbox for the Assessment of Neurological and Behavioral Function Cognition Battery (NIHTB-CB). The sample included 384 adults between the ages of 20 and 85 who had completed either a 4-year college degree or who scored in the above average, superior, or very superior range on a measure of intellectual functioning, the Crystallized Composite score. Algorithms were developed, based on the absence of high scores and the presence of low scores, for identifying mild cognitive impairment.

Results: Base rate tables for the presence of low scores and the absence of high scores are provided. The base rate for people with high average crystalized ability obtaining any one of the following, 5 scores <63rd percentile, or 4+ scores <50th percentile, or 3+ scores ≤ 25th percentile, or 2+ scores ≤ 16th percentile, is 15.5%.

Conclusions: Algorithms were developed for identifying cognitive weakness or impairment in high-functioning people. Research is needed to test them in clinical groups, and to assess their association with clinical risk factors for cognitive decline and biomarkers of acquired neurological or neurodegenerative diseases.

Multivariate Base Rates and Concussion Detection: A Comparative Study

Recent research has yielded multivariate base rates (MBRs) of low scores in healthy populations using a widely adopted concussion screening measure, Immediate Postconcussion and Cognitive Testing (ImPACT). However, the extent to which individuals with concussion obtain reliable changes at divergent frequencies relative to healthy individuals is largely unknown. The present study examined whether MBRs of reliable change accurately discriminated between those with and without concussion. This archival review consisted of 129 healthy individuals and 81 individuals with concussion. MBRs of reliable change scores were examined at varying cutoffs and frequencies between those with and without concussion. Composites showed small to medium effect sizes in differentiating between those with and without concussion. MBRs of reliable change scores on ImPACT provided limited discriminative utility in isolation. Computations of posttest probabilities using Bayes' Theorem yielded evidence for incremental gains when utilizing MBRs of reliable change under certain constraints.

Influence of Special Education, ADHD, Autism, and Learning Disorders on ImPACT Validity Scores in High School Athletes

OBJECTIVE

The Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) is commonly used to assist with post-concussion return-to-play decisions for athletes. Additional investigation is needed to determine whether embedded indicators used to determine the validity of scores are influenced by the presence of neurodevelopmental disorders (NDs).

METHOD

This study examined standard and novel ImPACT validity indicators in a large sample of high school athletes (n = 33,772) with or without self-reported ND.

RESULTS

Overall, 7.1% of athletes' baselines were judged invalid based on standard ImPACT validity criteria. When analyzed by group (healthy, ND), there were significantly more invalid ImPACT baselines for athletes with an ND diagnosis or special education history (between 9.7% and 54.3% for standard and novel embedded validity criteria) when compared to athletes without NDs. ND history was a significant predictor of invalid baseline performance above and beyond other demographic characteristics (i.e., age, sex, and sport), although it accounted for only a small percentage of variance. Multivariate base rates are presented stratified for age, sex, and ND.

CONCLUSIONS

These data provide evidence of higher than normal rates of invalid baselines in athletes who report ND (based on both the standard and novel embedded validity indicators). Although ND accounted for a small percentage of variance in the prediction of invalid performance, negative consequences (e.g., extended time out of sports) of incorrect decision-making should be considered for those with neurodevelopmental conditions. Also, reasons for the overall increase noted here, such as decreased motivation, "sandbagging", or disability-related cognitive deficit, require additional investigation.

Multivariate Base Rates of Low Scores on Tests of Executive Functions in a Multi-Country Latin American Sample

The objective of the study was to determine the prevalence of low scores in a diverse Latin American population for two neuropsychological commonly used tests to evaluate executive functions and to compare the number of low scores obtained using normative data from a Spanish-speaking population from Latin America versus an English-speaking population from U.S.A. Healthy adults (N = 5402) were administered the Modified Wisconsin Card Sorting Test and Stroop Color-Word. Low scores on measures of executive functioning are common. Clinicians working with Spanish-speaking adults should take into account the higher probability of low scores on these measures to reduce false-positive diagnoses of cognitive deficits in an individual.

Commonality in Abnormality: "Abnormal" Neuropsychological Scores Are Common in Baseline Testing of Youth Athletes

OBJECTIVE

Although research shows that abnormal scores are common in adults, fewer studies examine the frequency of abnormal scores within children. The present study assessed how specific tests, cognitive domains, and patient factors affect the likelihood of abnormally low scores among healthy youth athletes.

METHODS

Youth athletes (n = 85, Mage = 11.72, SD = 2.23) completed a comprehensive neuropsychological battery (N = 23 tests) during baseline testing for a sport-related concussion clinical research program. "Abnormal" scores were defined using cutoffs at 1, 1.5, and 2 standard deviations (SDs) below the population mean. Univariate and multivariate Poisson regression models examined abnormal test scores as a function of age, sex, ethnicity, and Full-Scale Intelligence Quotient (FSIQ).

RESULTS

Using cutoffs of 1, 1.5, and 2 SDs, the percent of healthy participants with at least one abnormal test score was 61.2, 36.5, and 22.4%, respectively, indicating a high proportion of healthy participants with abnormal scores and low specificity for tests using these cutoffs. In univariate Poisson regression analyses, FSIQ and abnormal score were negatively correlated (p < .001), wherein lower FSIQ was associated with more abnormal scores. In multivariate regression analyses, lower performance on executive function, memory, and processing speed tests was significantly associated with an increased frequency of abnormal scores.

CONCLUSIONS

Abnormally low scores in healthy youth occur frequently and do not necessarily represent functional deficits. Cutoffs of 2 SDs should be considered for many tests to increase specificity and reduce the risk of false positive errors that could lead to negative clinical outcomes.

Developing an Executive Functioning Composite Score for Research and Clinical Trials

OBJECTIVE

Executive functioning encompasses interactive cognitive processes such as planning, organization, set-shifting, inhibition, self-monitoring, working memory, and initiating and sustaining motor and mental activity. Researchers therefore typically assess executive functioning with multiple tests, each yielding multiple scores. A single composite score of executive functioning, which summarizes deficits across a battery of tests, would be useful in research and clinical trials. This study examines multiple candidate composite scores of executive functioning using tests from the Delis-Kaplan Executive Function System (D-KEFS).

METHOD

Participants were 875 adults between the ages of 20 and 89 years from the D-KEFS standardization sample. Seven Total Achievement scores were used from three tests (i.e., Trail Making, Verbal Fluency, and Color-Word Interference) to form eight composite scores that were compared based on their psychometric properties and association with intelligence (IQ).

RESULTS

The distributions of most composite scores were mildly to severely skewed, and some had a pronounced ceiling effect. The composite scores all showed a medium positive correlation with IQ. The composite scores were highly intercorrelated in the total sample and in four IQ subgroups (i.e., IQ <89, 90-99, 100-109, 110+), with some being so highly correlated that they appear redundant.

CONCLUSIONS

This study is part of a larger research program developing a cognition endpoint for research and clinical trials with sound psychometric properties and utility across discrepant test batteries. Future research is needed to examine the reliability and ecological validity of these composite scores.

The Other Side of the Bell Curve: Multivariate Base Rates of High Scores on the Delis-Kaplan Executive Function System

OBJECTIVE

Previous researchers have examined the frequency at which healthy participants obtain one or more low scores on neuropsychological test batteries, proposing five psychometric principles of multivariate base rates: (a) low scores are common, with their frequency contingent on (b) the low score cutoff used, (c) the number of tests administered/interpreted, and (d) the demographic characteristics and (e) intelligence of participants. The current study explored whether these principles applied to high scores as well, using the Delis-Kaplan Executive Function System (D-KEFS).

METHOD

Multivariate base rates of high scores (≥75th, ≥84th, ≥91st, ≥95th, and ≥98th percentiles) were derived for a three-test, four-test, and full D-KEFS battery, using the adult portion of the normative sample (aged 16-89 years; N = 1050) stratified by education and intelligence. The full D-KEFS battery provides 16 total achievement scores (primary indicators of executive function).

RESULTS

High scores occurred commonly for all batteries. For the three-test battery, 24.1% and 12.4% had 1 or more scores ≥95th percentile and ≥98th percentile, respectively. High scores occurred more often for longer batteries: 61.6%, 72.9%, and 87.8% obtained 1 or more scores ≥84th percentile for the three-test, four-test, and full batteries, respectively. The frequency of high scores increased with more education and higher intelligence.

CONCLUSIONS

The principles of multivariate base rates also applied to high D-KEFS scores: high scores were common and contingent on the cutoff used, number of tests administered/interpreted, and education/intelligence of examinees. Base rates of high scores may help clinicians identify true cognitive strengths and detect cognitive deficits in high functioning people.