New Child and Adolescent Memory Profile Embedded Performance Validity Test

OBJECTIVE

It is essential to interpret performance validity tests (PVTs) that are well-established and have strong psychometrics. This study evaluated the Child and Adolescent Memory Profile (ChAMP) Validity Indicator (VI) using a pediatric sample with traumatic brain injury (TBI).

METHOD

A cross-sectional sample of N = 110 youth (mean age = 15.1 years, standard deviation [SD] = 2.4 range = 8-18) on average 32.7 weeks (SD = 40.9) post TBI (71.8% mild/concussion; 3.6% complicated mild; 24.6% moderate-to-severe) were administered the ChAMP and two stand-alone PVTs. Criterion for valid performance was scores above cutoffs on both PVTs; criterion for invalid performance was scores below cutoffs on both PVTs. Classification statistics were used to evaluate the existing ChAMP VI and establish a new VI cutoff score if needed.

RESULTS

There were no significant differences in demographics or time since injury between those deemed valid (n = 96) or invalid (n = 14), but all ChAMP scores were significantly lower in those deemed invalid. The original ChAMP VI cutoff score was highly specific (no false positives) but also highly insensitive (sensitivity [SN] = .07, specificity [SP] = 1.0). Based on area under the curve (AUC) analysis (0.94), a new cutoff score was established using the sum of scaled scores (VI-SS). A ChAMP VI-SS score of 32 or lower achieved strong SN (86%) and SP (92%). Using a 15% base rate, positive predictive value was 64% and negative predictive value was 97%.

CONCLUSIONS

The originally proposed ChAMP VI has insufficient SN in pediatric TBI. However, this study yields a promising new ChAMP VI-SS, with classification metrics that exceed any other current embedded PVT in pediatrics.

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline, and Fail).

The Utility of the Test of Memory Malingering Trial 1 in Differentiating Neurocognitive, Emotional, and Behavioral Functioning in a Pediatric Concussion Population

OBJECTIVE

In concussion populations, suboptimal task engagement detected by performance validity tests (PVTs) has been associated with poorer neuropsychological scores and greater post-concussive symptoms (PCS). This study examined if Pass/Fail status on the Test of Memory Malingering-TOMM Trial 1-differentiated the neurocognitive, emotional, and behavioral profile of pediatric patients with concussion.

METHOD

This study utilized archival data from 93 patients (mean age = 14.56 and SD = 2.01) with a history of concussion who were assessed at ~5-6 weeks post-injury (mean days = 40.27 and SD = 35.41). Individuals were divided into "Pass" and "Fail" groups based on TOMM Trial 1 performance. The testing battery included ACT, CPT-II and III, HVLT-R, WJ-III and IV ACH, ImPACT, BASC-2, and BRIEF.

RESULTS

The overall pass rate on Trial 1 was 70% (mean = 46.04 and SD = 4.55). Findings suggested that a passing score on Trial 1 may be associated with adequate performance across the remaining two trials of the TOMM. The Fail group scored significantly lower across attention, memory, and processing speed measures when compared with the Pass group. On rating scales, significantly more concerns were endorsed with the Fail group for attention and executive functioning relative to the Pass group. Parents generally endorsed significantly more concerns for executive functioning when compared with their children's self-reported symptoms. There was a trend for the Fail group to report more PCS; however, they did not significantly differ from the Pass group for depression, anxiety, or somatization.

CONCLUSIONS

This study highlights the importance of utilizing PVTs when evaluating concussion recovery.

Prospective Exploration of Cognitive-Communication Changes With Woodcock-Johnson IV Before and After Sport-Related Concussion

Purpose The purpose of this study was to evaluate changes in cognitive-communication performance using Woodcock-Johnson IV Tests (WJIV) from pre-injury baseline to post sport-related concussion. It was hypothesized that individual subtest performances would decrease postinjury in symptomatic individuals. Method This prospective longitudinal observational nested cohort study of collegiate athletes assessed cognitive-communicative performance at preseason baseline and postinjury. Three hundred and forty-two male and female undergraduates at high risk for sport-related concussion participated in preseason assessments, and 18 individuals met criteria post injury. WJIV subtest domains included Word Finding, Speeded Reading Comprehension, Auditory Comprehension, Verbal Working Memory, Story Retell, and Visual Processing (letter and number). The power calculation was not met, and therefore data were conservatively analyzed with descriptive statistics and a planned subgroup analysis based on symptomatology. Results Individual changes from baseline to postinjury were evaluated using differences in standard score performance. For symptomatic individuals, mean negative decreases in performance were found for Retrieval Fluency, Sentence Reading Fluency, Pattern Matchings, and all cluster scores postinjury. Individual performance declines also included decreases in story retell, verbal working memory, and visual processing. Conclusions This study identified within-subject WJIV performance decline in communication domains post sport-related concussion and reinforces that cognitive-communication dysfunction should be considered in mild traumatic brain injury. Key cognitive-communication areas included speeded naming, reading, and verbal memory, though oral comprehension was not sensitive to change. Future clinical research across diverse populations is needed to expand these preliminary findings.

Performance Validity Testing in Justice-Involved Adults with Fetal Alcohol Spectrum Disorder

OBJECTIVES

A number of commonly used performance validity tests (PVTs) may be prone to high failure rates when used for individuals with severe neurocognitive deficits. This study investigated the validity of 10 PVT scores in justice-involved adults with fetal alcohol spectrum disorder (FASD), a neurodevelopmental disability stemming from prenatal alcohol exposure and linked with severe neurocognitive deficits.

METHOD

The sample comprised 80 justice-involved adults (ages 19-40) including 25 with confirmed or possible FASD and 55 where FASD was ruled out. Ten PVT scores were calculated, derived from Word Memory Test, Genuine Memory Impairment Profile, Advanced Clinical Solutions (Word Choice), the Wechsler Adult Intelligence Scale - Fourth Edition (Reliable Digit Span and age-corrected scaled scores (ACSS) from Digit Span, Coding, Symbol Search, Coding - Symbol Search, Vocabulary - Digit Span), and the Wechsler Memory Scale - Fourth Edition (Logical Memory II Recognition).

RESULTS

Participants with diagnosed/possible FASD were more likely to fail any single PVT, and failed a greater number of PVTs overall, compared to those without FASD. They were also more likely to fail based on Word Memory Test, Digit Span ACSS, Coding ACSS, Symbol Search ACSS, and Logical Memory II Recognition, compared to controls (35-76%). Across both groups, substantially more participants with IQ <70 failed two or more PVTs (90%), compared to those with an IQ ≥70 (44%).

CONCLUSIONS

Results highlight the need for additional research examining the use of PVTs in justice-involved populations with FASD.

The use of the MSVT in children and adolescents with epilepsy

Pediatric neuropsychologists are increasingly recognizing the importance of performance validity testing during evaluations. The use of such measures to detect insufficient effort is of particular importance in pediatric epilepsy evaluations, where test results are often used to guide surgical decisions and failure to detect poor task engagement can result in postsurgical cognitive decline. The present investigation assesses the utility of the Medical Symptom Validity Test (MSVT) in 104 clinically referred children and adolescents with epilepsy. Though the overall failure rate was 15.4% of the total group, children with 2nd grade or higher reading skills (a requirement of the task) passed at a very high rate (96.6%). Of the three failures, two were unequivocally deemed true positives, while the third failed due to extreme somnolence during testing. Notably, for those with ≥2nd grade reading levels, MSVT validity indices were unrelated to patient age, intellectual functioning, or age of epilepsy onset, while modest relations were seen with specific memory measures, number of epilepsy medications, and seizure frequency. Despite these associations, however, this did not result in more failures in this population of children and adolescents with substantial neurologic involvement, as pass rates exceeded 92% for those with intellectual disability, high seizure frequency, high medication burden, and even prior surgical resection of critical memory structures.

Noncredible effort on the Nonverbal-Medical Symptom Validity Test (NV-MSVT): Impact on cognitive performance in pediatric mild traumatic brain injury

This study examined failure rates on the Nonverbal-Medical Symptom Validity Test (NV-MSVT) and its impact on cognitive performance in a sample of youths with mild traumatic brain injury (mTBI). Participants were 184 children and adolescents who presented to a multidisciplinary concussion clinic for a targeted neuropsychological evaluation. Performance Validity Tests (PVTs) were a part of the standard battery, including the NV-MSVT. Twenty-eight participants (15.2%) failed the NV-MSVT, none of whom displayed a genuine memory impairment profile (GMIP). Participants who failed the NV-MSVT performed significantly worse than those who passed the NV-MSVT on measures of IQ, memory, and immediate attention/working memory. There was no significant difference between groups on processing speed, sustained attention, cognitive flexibility, or sight word reading level. Aside from a slight difference in age, NV-MSVT failure was not impacted by demographic variables (sex, race), premorbid risk factors (pre-injury ADHD, learning disabilities, psychiatric diagnoses or treatment, developmental delays, or prior special education), injury-related variables (time since injury, positive neuroimaging findings, post-traumatic amnesia, number of prior mTBIs, etc.) or post-mTBI anxiety/depression. That said, participants who failed NV-MSVT endorsed significantly more severe postconcussive symptoms. These findings support the use of the NV-MSVT in neuropsychological evaluation of children and adolescents with mTBI.

Performance validity test failure predicts suppression of neuropsychological test results in developmentally disabled children

There is increasing awareness of the need to use Performance Validity Tests (PVTs) when assessing cognitive abilities in children. Since 1996, there has been an ongoing clinical study of the use of three PVTs with a consecutive series of 1,285 developmentally disabled children. In this study, we report on the results of these PVTs in children of many diagnostic categories. Failure rates on all three tests were very low. The mean scores on the effort measures in those passing the PVTs were extremely high. Failure on each PVT was found to be associated with a significant and widespread suppression of scores across a neuropsychological battery. Failure on even one PVT significantly suppresses ability test scores.

Construct validity for the verbal associated pairs screen

The present study introduces the Verbal Associated Pairs Screen (VAPS) as a new measure for assessing performance validity in pediatric populations. This study presents initial data on psychometric properties and establishes construct validity for the VAPS in a sample of 30 adolescent healthy controls and 206 youths with traumatic brain injury (TBI: moderate/severe, N = 30; mild, N = 176). The control group's age (M = 14.93, SD = 1.8) was significantly higher than the moderate/severe TBI (M = 13.9, SD = 2.8), t(68.508) = -3.038, p = .003, and mild TBI (mTBI) groups (M = 14, SD = 2.8), t(54.147) = 2.038, p = .046. The TBI groups were administered the VAPS in accord with other established performance validity tests (PVTs) and well-established  memory tests as part of routine clinical evaluations. The healthy control group was administered the VAPS only. VAPS score distributions for the control group were negatively skewed and highly kurtotic. VAPS scores from the moderate/severe TBI and control groups were indistinguishable for Trial 2 (U = 274, p < .01) and the Delay (U = 396, p = .218). In the mTBI group, convergent and divergent validity was established with other well-validated PVTs and memory tests, respectively. ROC curve analyses identified optimal cutoff scores for the VAPS Total Score, with acceptable sensitivity (55%) and excellent specificity (100%), as well as strong detectability (AUC = .829, 95% CI: 0.731 - 0.928, p < .001). Clinical applications, limitations, and directions for future research with the VAPS are discussed.

[Formula: see text]Determination of the smoking gun of intent: significance testing of forced choice results in social security claimants

OBJECTIVE

Significantly below-chance findings on forced choice tests have been described as revealing "the smoking gun of intent" that proved malingering. The issues of probability levels, one-tailed vs. two-tailed tests, and the combining of PVT scores on significantly below-chance findings were addressed in a previous study, with a recommendation of a probability level of .20 to test the significance of below-chance results. The purpose of the present study was to determine the rate of below-chance findings in a Social Security Disability claimant sample using the previous recommendations.

METHOD

We compared the frequency of below-chance results on forced choice performance validity tests (PVTs) at two levels of significance, .05 and .20, and when using significance testing on individual subtests of the PVTs compared with total scores in claimants for Social Security Disability in order to determine the rate of the expected increase.

RESULTS

The frequency of significant results increased with the higher level of significance for each subtest of the PVT and when combining individual test sections to increase the number of test items, with up to 20% of claimants showing significantly below-chance results at the higher p-value.

CONCLUSIONS

These findings are discussed in light of Social Security Administration policy, showing an impact on policy issues concerning child abuse and neglect, and the importance of using these techniques in evaluations for Social Security Disability.

The impact of pediatric neuropsychological consultation in mild traumatic brain injury: a model for providing feedback after invalid performance

OBJECTIVE

In recent years, pediatric practitioners have increasingly recognized the importance of objectively measuring performance validity during clinical assessments. Yet, no studies have examined the impact of neuropsychological consultation when invalid performance has been identified in pediatric populations and little published guidance exists for clinical management. Here we provide a conceptual model for providing feedback after noncredible performance has been detected. In a pilot study, we examine caregiver satisfaction and postconcussive symptoms following provision of this feedback for patients seen through our concussion program.

METHODS

Participants (N = 70) were 8-17-year-olds with a history of mild traumatic brain injury who underwent an abbreviated neuropsychological evaluation between 2 and 12 months post-injury. We examined postconcussive symptom reduction and caregiver satisfaction after neuropsychological evaluation between groups of patients who were determined to have provided noncredible effort (n = 9) and those for whom no validity concerns were present (n = 61).

RESULTS

We found similarly high levels of caregiver satisfaction between groups and greater reduction in self-reported symptoms after feedback was provided using the model with children with noncredible presentations compared to those with credible presentations.

CONCLUSION

The current study lends preliminary support to the idea that the identification and communication of invalid performance can be a beneficial clinical intervention that promotes high levels of caregiver satisfaction and a reduction in self-reported and caregiver-reported symptoms.