1
|
Denning JH, Horner MD. The impact of race and other demographic factors on the false positive rates of five embedded Performance Validity Tests (PVTs) in a Veteran sample. J Clin Exp Neuropsychol 2024; 46:25-35. [PMID: 38353039 DOI: 10.1080/13803395.2024.2314737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 01/11/2024] [Indexed: 05/12/2024]
Abstract
INTRODUCTION It is common to use normative adjustments based on race to maintain accuracy when interpreting cognitive test results during neuropsychological assessment. However, embedded performance validity tests (PVTs) do not adjust for these racial differences and may result in elevated rates of false positives in African American/Black (AA) samples compared to European American/White (EA) samples. METHODS Veterans without Major Neurocognitive Disorder completed an outpatient neuropsychological assessment and were deemed to be performing in a valid manner (e.g., passing both the Test of Memory Malingering Trial 1 (TOMM1) and the Medical Symptom Validity Test (MSVT), (n = 531, EA = 473, AA = 58). Five embedded PVTs were administered to all patients: WAIS-III/IV Processing Speed Index (PSI), Brief Visuospatial Memory Test-Revised: Discrimination Index (BVMT-R), TMT-A (secs), California Verbal Learning Test-II (CVLT-II) Forced Choice, and WAIS-III/IV Digit Span Scaled Score. Individual PVT false positive rates, as well as the rate of failing two or more embedded PVTs, were calculated. RESULTS Failure rates of two embedded PVTs (PSI, TMT-A), and the total number of PVTs failed, were higher in the AA sample. The PSI and TMT-A remained significantly impacted by race after accounting for age, education, sex, and presence of Mild Neurocognitive Disorder. There were PVT failure rates greater than 10% (and considered false positives) in both groups (AA: PSI, TMT-A, and BVMT-R, 12-24%; EA: BVMT-R, 17%). Failing 2 or more PVTs (AA = 9%, EA = 4%) was impacted by education and Mild Neurocognitive Disorder but not by race. CONCLUSIONS Individual (timed) PVTs showed higher false positive rates in the AA sample even after accounting for demographic factors and diagnosis of Mild Neurocognitive Disorder. Requiring failure on 2 or more embedded PVTs reduced false positive rates to acceptable levels across both groups (10% or less) and was not significantly influenced by race.
Collapse
Affiliation(s)
- John H Denning
- Mental Health Service, Ralph H. Johnson Veterans Affairs Health Care System, Charleston, SC, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Michael David Horner
- Mental Health Service, Ralph H. Johnson Veterans Affairs Health Care System, Charleston, SC, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| |
Collapse
|
2
|
Resch ZJ, Cerny BM, Ovsiew GP, Jennette KJ, Bing-Canar H, Rhoads T, Soble JR. A Direct Comparison of 10 WAIS-IV Digit Span Embedded Validity Indicators among a Mixed Neuropsychiatric Sample with Varying Degrees of Cognitive Impairment. Arch Clin Neuropsychol 2022; 38:619-632. [DOI: 10.1093/arclin/acac082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/09/2022] [Indexed: 11/13/2022] Open
Abstract
Abstract
Objective
Reliable Digit Span (RDS), RDS-Revised (RDS-R), and age-corrected scaled score (ACSS) have been previously validated as embedded performance validity tests (PVTs) from the Wechsler Adult Intelligence Scale-IV Digit Span subtest (WAIS-IV DS). However, few studies have directly compared the relative utility of these and other proposed WAIS-IV DS validity indicators within a single sample.
Method
This study compared classification accuracies of 10 WAIS-IV DS indices in a mixed neuropsychiatric sample of 227 outpatients who completed a standardized neuropsychological battery. Participants with ≤1 PVT failures of the four, freestanding criterion PVTs constituted the valid group (n = 181), whereas those with ≥2 PVT failures formed the invalid group (n = 46). Among the valid group, 113 met criteria for mild cognitive impairment (MCI).
Results
Classification accuracies for all DS indicators were statistically significant across the overall sample and subsamples with and without MCI, apart from indices derived from the Forward trial in the MCI sample. DS Sequencing ACSS, working memory RDS (wmRDS), and DS ACSS emerged as the most effective predictors of validity status, with acceptable to excellent classification accuracy for the overall sample (AUCs = 0.792–0.816; 35%–50% sensitivity/88%–96% specificity).
Conclusions
Although most DS indices demonstrated clinical utility as embedded PVTs, DS Sequencing ACSS, wmRDS, and DS ACSS may be particularly robust to cognitive impairment, minimizing risk of false positive errors while identifying noncredible performance. Moreover, DS indices incorporating data from multiple trials (i.e., wmRDS, DS ACSS) also generally yielded greater classification accuracy than those derived from a single trial.
Collapse
Affiliation(s)
- Zachary J Resch
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
| | - Brian M Cerny
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
- Illinois Institute of Technology Department of Psychology, , Chicago, IL, USA
| | - Gabriel P Ovsiew
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
| | - Kyle J Jennette
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
| | - Hanaan Bing-Canar
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
- University of Illinois at Chicago Department of Psychology, , Chicago, IL, USA
| | - Tasha Rhoads
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
- Rosalind Franklin University of Medicine and Science Department of Psychology, , North Chicago, IL, USA
| | - Jason R Soble
- University of Illinois College of Medicine Department of Psychiatry, , Chicago, IL, USA
- University of Illinois College of Medicine Department of Neurology, , Chicago, IL, USA
| |
Collapse
|
3
|
Kanser RJ, Rapport LJ, Hanks RA, Patrick SD. Utility of WAIS-IV Digit Span indices as measures of performance validity in moderate to severe traumatic brain injury. Clin Neuropsychol 2022; 36:1950-1963. [PMID: 34044725 DOI: 10.1080/13854046.2021.1921277] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Objective: The addition of Sequencing to WAIS-IV Digit Span (DS) brought about new Reliable Digit Span (RDS) indices and an Age-Corrected Scaled Score that includes Sequencing trials. Reports have indicated that these new performance validity tests (PVTs) are superior to the traditional RDS; however, comparisons in the context of known neurocognitive impairment are sparse. This study compared DS-derived PVT classification accuracies in a design that included adults with verified TBI. Methods: Participants included 64 adults with moderate-to-severe TBI (TBI), 51 healthy adults coached to simulate TBI (SIM), and 78 healthy comparisons (HC). Participants completed the WAIS-IV DS subtest in the context of a larger test battery. Results: Kruskal-Wallis tests indicated that all DS indices differed significantly across groups. Post hoc contrasts revealed that only RDS Forward and the traditional RDS differed significantly between SIM and TBI. ROC analyses indicated that RDS variables were comparable predictors of SIM vs. HC; however, the traditional RDS showed the highest sensitivity when approximating 90% specificity for SIM vs. TBI. A greater percentage of TBI scored RDS Sequencing < 1 compared to SIM and HC. Conclusion: In the context of moderate-to-severe TBI, the DS-derived PVTs showed comparable discriminability. However, the Greiffenstein et al. traditional RDS demonstrated the best classification accuracy with respect to specificity/sensitivity balance. This relative superiority may reflect that individuals with verified TBI are more likely to perseverate on prior instructions during DS Sequencing. Findings highlight the importance of including individuals with verified TBI when evaluating and developing PVTs.
Collapse
Affiliation(s)
- Robert J Kanser
- Department of Psychology, Wayne State University, Detroit, MI, USA
| | - Lisa J Rapport
- Department of Psychology, Wayne State University, Detroit, MI, USA
| | - Robin A Hanks
- Department of Physical Medicine and Rehabilitation, Wayne State University, Detroit, MI, USA
| | - Sarah D Patrick
- Department of Psychology, Wayne State University, Detroit, MI, USA
| |
Collapse
|
4
|
Abeare K, Cutler L, An KY, Razvi P, Holcomb M, Erdodi LA. BNT-15: Revised Performance Validity Cutoffs and Proposed Clinical Classification Ranges. Cogn Behav Neurol 2022; 35:155-168. [PMID: 35507449 DOI: 10.1097/wnn.0000000000000304] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 08/09/2021] [Indexed: 02/06/2023]
Abstract
BACKGROUND Abbreviated neurocognitive tests offer a practical alternative to full-length versions but often lack clear interpretive guidelines, thereby limiting their clinical utility. OBJECTIVE To replicate validity cutoffs for the Boston Naming Test-Short Form (BNT-15) and to introduce a clinical classification system for the BNT-15 as a measure of object-naming skills. METHOD We collected data from 43 university students and 46 clinical patients. Classification accuracy was computed against psychometrically defined criterion groups. Clinical classification ranges were developed using a z -score transformation. RESULTS Previously suggested validity cutoffs (≤11 and ≤12) produced comparable classification accuracy among the university students. However, a more conservative cutoff (≤10) was needed with the clinical patients to contain the false-positive rate (0.20-0.38 sensitivity at 0.92-0.96 specificity). As a measure of cognitive ability, a perfect BNT-15 score suggests above average performance; ≤11 suggests clinically significant deficits. Demographically adjusted prorated BNT-15 T-scores correlated strongly (0.86) with the newly developed z -scores. CONCLUSION Given its brevity (<5 minutes), ease of administration and scoring, the BNT-15 can function as a useful and cost-effective screening measure for both object-naming/English proficiency and performance validity. The proposed clinical classification ranges provide useful guidelines for practitioners.
Collapse
Affiliation(s)
| | | | - Kelly Y An
- Private Practice, London, Ontario, Canada
| | - Parveen Razvi
- Faculty of Nursing, University of Windsor, Windsor, Ontario, Canada
| | | | | |
Collapse
|
5
|
Ratcliffe LN, Hale AC, Gradwohl BD, Spencer RJ. Preliminary findings from reevaluating the MMPI Response Bias Scale items in veterans undergoing neuropsychological evaluation. APPLIED NEUROPSYCHOLOGY. ADULT 2022:1-8. [PMID: 35917583 DOI: 10.1080/23279095.2022.2106571] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The Response Bias Scale (RBS) was developed to predict non-credible cognitive presentations among disability claimants without head injury. Developers used empirical keying, which is independent of apparent content, to select items from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) item pool that distinguished between individuals passing or failing performance validity tests (PVTs). No study has examined which of these items would have psychometric value when used in clinical neuropsychological evaluations. This study reexamined items comprising RBS with reference to manifest item content, internal consistency, PVTs, and a symptom validity test (SVT) in a sample of 173 predominately White male veterans (MAGE = 50.70, MEDU = 13.73) in a VA outpatient neuropsychology clinic. Participants completed the MMPI-2 Restructured Form (MMPI-2-RF), PVTs, and an SVT. The 28-item RBS appears to contain three types of items: those that manifestly address cognitive functioning, those that are supported but do not appear to address cognitive functioning, and nine items that were unrelated to cognition and not statistically supported. The 19 empirically supported items, or RBS-19, predicted PVT and SVT failures marginally better than the RBS. Both the RBS and RBS-19 had stronger relationships with SVTs relative to PVTs. Although the removal of the nine problematic items improved the diagnostic accuracy of the scale, it still did not reach the level that is generally considered to be clinically optimal. The RBS-19 offers a measure with improved internal consistency and predictive validity compared to the RBS and warrants additional research.
Collapse
Affiliation(s)
- Lauren N Ratcliffe
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
- Department of Clinical Psychology, Mercer University College of Health Professions, Atlanta, GA, USA
| | - Andrew C Hale
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| | - Brian D Gradwohl
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| | - Robert J Spencer
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| |
Collapse
|
6
|
Messerly J, Soble JR, Webber TA, Alverson WA, Fullen C, Kraemer LD, Marceaux JC. Evaluation of the classification accuracy of multiple performance validity tests in a mixed clinical sample. APPLIED NEUROPSYCHOLOGY. ADULT 2021; 28:727-736. [PMID: 31835915 DOI: 10.1080/23279095.2019.1698581] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The Test of Memory Malingering (TOMM) and Word Memory Test (WMT) are among the most well-known performance validity tests (PVTs) and regarded as gold standard measures. Due to the many factors that impact PVT selection, it is imperative that clinicians make informed clinical decisions with respect to additional or alternative PVTs that demonstrate similar classification accuracy as these well-validated measures. The present archival study evaluated the agreement/classification accuracy of a large battery consisting of multiple other freestanding/embedded PVTs in a mixed clinical sample of 126 veterans. We examined failure rates for all standalone/embedded PVTs using established cut-scores and calculated pass/fail agreement rates and diagnostic odds ratios for various combinations of PVTs using the TOMM and WMT as criterion measures. TOMM and WMT demonstrated the best agreement, followed by Word Choice Test (WCT). The Rey Fifteen Item Test had an excessive number of false-negative errors and reduced classification accuracy. The Digit Span age-corrected scaled score (DS-ACSS) had highest agreement. Findings lend further support to the use of a combination of embedded and standalone PVTs in identifying suboptimal performance. Results provide data to enhance clinical decision making for neuropsychologists who implement combinations of PVTs in a larger clinical battery.
Collapse
Affiliation(s)
- Johanna Messerly
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Jason R Soble
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Departments of Psychiatry and Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| | - Troy A Webber
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Mental Health and Rehabilitation and Extended Carelines, Michael E. DeBakey VA Medical Center, Houston, TX, USA
| | - W Alex Alverson
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Chrystal Fullen
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Lindsay D Kraemer
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Janice C Marceaux
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Department of Neurology, University of Texas Health Science Center, San Antonio, TX, USA
| |
Collapse
|
7
|
Dunn A, Pyne S, Tyson B, Roth R, Shahein A, Erdodi L. Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator. Dev Neuropsychol 2021; 46:327-346. [PMID: 34525856 DOI: 10.1080/87565641.2021.1956499] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
OBJECTIVE : Replicate previous research on Logical Memory Recognition (LMRecog) and perform a critical item analysis. METHOD : Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LMRecog and nine critical items (CR-9) was computed. RESULTS : LMRecog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LMRecog ≤20; CR-9 ≥6 increased specificity by 6-8% over LMRecog ≤20; CR-9 ≥7 increased specificity by 8-15%. CONCLUSIONS : Critical item analysis enhances the classification accuracy of the optimal LMRecog cutoff (≤20).
Collapse
Affiliation(s)
- Alexa Dunn
- Department of Psychology, University of Windsor, Windsor, Canada
| | - Sadie Pyne
- Windsor Neuropsychology, Windsor, Canada
| | - Brad Tyson
- Neuroscience Institute, Evergreen Neuroscience Institute, EvergreenHealth Medical Center, Kirkland, USA
| | - Robert Roth
- Neuropsychology Services, Dartmouth-Hitchcock Medical Center, USA
| | - Ayman Shahein
- Department of Clinical Neurosciences, University of Calgary, Calgary, Canada
| | - Laszlo Erdodi
- Department of Psychology, University of Windsor, Windsor, Canada
| |
Collapse
|
8
|
Messa I, Holcomb M, Lichtenstein JD, Tyson BT, Roth RM, Erdodi LA. They are not destined to fail: a systematic examination of scores on embedded performance validity indicators in patients with intellectual disability. AUST J FORENSIC SCI 2021. [DOI: 10.1080/00450618.2020.1865457] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Affiliation(s)
- Isabelle Messa
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | | | | | - Brad T Tyson
- Neuropsychological Service, EvergreenHealth Medical Center, Kirkland, WA, USA
| | - Robert M Roth
- Department of Psychiatry, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
| | - Laszlo A Erdodi
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
9
|
Abeare CA, An K, Tyson B, Holcomb M, Cutler L, May N, Erdodi LA. The emotion word fluency test as an embedded performance validity indicator - Alone and in a multivariate validity composite. APPLIED NEUROPSYCHOLOGY. CHILD 2021; 11:713-724. [PMID: 34424798 DOI: 10.1080/21622965.2021.1939027] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
OBJECTIVE This project was designed to cross-validate existing performance validity cutoffs embedded within measures of verbal fluency (FAS and animals) and develop new ones for the Emotion Word Fluency Test (EWFT), a novel measure of category fluency. METHOD The classification accuracy of the verbal fluency tests was examined in two samples (70 cognitively healthy university students and 52 clinical patients) against psychometrically defined criterion measures. RESULTS A demographically adjusted T-score of ≤31 on the FAS was specific (.88-.97) to noncredible responding in both samples. Animals T ≤ 29 achieved high specificity (.90-.93) among students at .27-.38 sensitivity. A more conservative cutoff (T ≤ 27) was needed in the patient sample for a similar combination of sensitivity (.24-.45) and specificity (.87-.93). An EWFT raw score ≤5 was highly specific (.94-.97) but insensitive (.10-.18) to invalid performance. Failing multiple cutoffs improved specificity (.90-1.00) at variable sensitivity (.19-.45). CONCLUSIONS Results help resolve the inconsistency in previous reports, and confirm the overall utility of existing verbal fluency tests as embedded validity indicators. Multivariate models of performance validity assessment are superior to single indicators. The clinical utility and limitations of the EWFT as a novel measure are discussed.
Collapse
Affiliation(s)
- Christopher A Abeare
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| | - Kelly An
- Private Practice, London, Ontario, Canada
| | - Brad Tyson
- Evergreen Health Medical Center, Kirkland, Washington, USA
| | - Matthew Holcomb
- Jefferson Neurobehavioral Group, New Orleans, Louisiana, USA
| | - Laura Cutler
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| | - Natalie May
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| | - Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| |
Collapse
|
10
|
Modiano YA, Taiwo Z, Pastorek NJ, Webber TA. The Structured Inventory of Malingered Symptomatology Amnestic Disorders Scale (SIMS-AM) Is Insensitive to Cognitive Impairment While Accurately Identifying Invalid Cognitive Symptom Reporting. PSYCHOLOGICAL INJURY & LAW 2021. [DOI: 10.1007/s12207-021-09420-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
11
|
Nayar K, Ventura LM, DeDios-Stern S, Oh A, Soble JR. The Impact of Learning and Memory on Performance Validity Tests in a Mixed Clinical Pediatric Population. Arch Clin Neuropsychol 2021; 37:50-62. [PMID: 34050354 DOI: 10.1093/arclin/acab040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/04/2021] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE This study examined the degree to which verbal and visuospatial memory abilities influence performance validity test (PVT) performance in a mixed clinical pediatric sample. METHOD Data from 252 consecutive clinical pediatric cases (Mage=11.23 years, SD=4.02; 61.9% male) seen for outpatient neuropsychological assessment were collected. Measures of learning and memory (e.g., The California Verbal Learning Test-Children's Version; Child and Adolescent Memory Profile [ChAMP]), performance validity (Test of Memory Malingering Trial 1 [TOMM T1]; Wechsler Intelligence Scale for Children-Fifth Edition [WISC-V] or Wechsler Adult Intelligence Scale-Fourth Edition Digit Span indices; ChAMP Overall Validity Index), and intellectual abilities (e.g., WISC-V) were included. RESULTS Learning/memory abilities were not significantly correlated with TOMM T1 and accounted for relatively little variance in overall TOMM T1 performance (i.e., ≤6%). Conversely, ChAMP Validity Index scores were significantly correlated with verbal and visual learning/memory abilities, and learning/memory accounted for significant variance in PVT performance (12%-26%). Verbal learning/memory performance accounted for 5%-16% of the variance across the Digit Span PVTs. No significant differences in TOMM T1 and Digit Span PVT scores emerged between verbal/visual learning/memory impairment groups. ChAMP validity scores were lower for the visual learning/memory impairment group relative to the nonimpaired group. CONCLUSIONS Findings highlight the utility of including PVTs as standard practice for pediatric populations, particularly when memory is a concern. Consistent with the adult literature, TOMM T1 outperformed other PVTs in its utility even among the diverse clinical sample with/without learning/memory impairment. In contrast, use of Digit Span indices appear to be best suited in the presence of visuospatial (but not verbal) learning/memory concerns. Finally, the ChAMP's embedded validity measure was most strongly impacted by learning/memory performance.
Collapse
Affiliation(s)
- Kritika Nayar
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Lea M Ventura
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Pediatrics, University of Illinois College of Medicine, Chicago, IL, USA
| | - Samantha DeDios-Stern
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Alison Oh
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jason R Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
12
|
Abeare K, Romero K, Cutler L, Sirianni CD, Erdodi LA. Flipping the Script: Measuring Both Performance Validity and Cognitive Ability with the Forced Choice Recognition Trial of the RCFT. Percept Mot Skills 2021; 128:1373-1408. [PMID: 34024205 PMCID: PMC8267081 DOI: 10.1177/00315125211019704] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
In this study we attempted to replicate the classification accuracy of the newly introduced Forced Choice Recognition trial (FCR) of the Rey Complex Figure Test (RCFT) in a clinical sample. We administered the RCFTFCR and the earlier Yes/No Recognition trial from the RCFT to 52 clinically referred patients as part of a comprehensive neuropsychological test battery and incentivized a separate control group of 83 university students to perform well on these measures. We then computed the classification accuracies of both measures against criterion performance validity tests (PVTs) and compared results between the two samples. At previously published validity cutoffs (≤16 & ≤17), the RCFTFCR remained specific (.84-1.00) to psychometrically defined non-credible responding. Simultaneously, the RCFTFCR was more sensitive to examinees' natural variability in visual-perceptual and verbal memory skills than the Yes/No Recognition trial. Even after being reduced to a seven-point scale (18-24) by the validity cutoffs, both RCFT recognition scores continued to provide clinically useful information on visual memory. This is the first study to validate the RCFTFCR as a PVT in a clinical sample. Our data also support its use for measuring cognitive ability. Replication studies with more diverse samples and different criterion measures are still needed before large-scale clinical application of this scale.
Collapse
Affiliation(s)
- Kaitlyn Abeare
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Kristoffer Romero
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Laura Cutler
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | | | - Laszlo A Erdodi
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| |
Collapse
|
13
|
Sabelli AG, Messa I, Giromini L, Lichtenstein JD, May N, Erdodi LA. Symptom Versus Performance Validity in Patients with Mild TBI: Independent Sources of Non-credible Responding. PSYCHOLOGICAL INJURY & LAW 2021. [DOI: 10.1007/s12207-021-09400-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
|
14
|
Spencer RJ, Gradwohl BD, Kordovski VM. Initial Validation of Short Forms of the SIMS for Neuropsychological Evaluations. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-020-09394-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
15
|
Erdodi LA, Abeare CA. Stronger Together: The Wechsler Adult Intelligence Scale-Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury. Arch Clin Neuropsychol 2020; 35:188-204. [PMID: 31696203 DOI: 10.1093/arclin/acz032] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2019] [Revised: 06/18/2019] [Accepted: 06/22/2019] [Indexed: 12/17/2022] Open
Abstract
OBJECTIVE This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV). METHOD Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance. RESULTS Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95). CONCLUSIONS In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment. BRIEF SUMMARY Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.
Collapse
Affiliation(s)
- Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| | - Christopher A Abeare
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
16
|
Abstract
OBJECTIVES A number of commonly used performance validity tests (PVTs) may be prone to high failure rates when used for individuals with severe neurocognitive deficits. This study investigated the validity of 10 PVT scores in justice-involved adults with fetal alcohol spectrum disorder (FASD), a neurodevelopmental disability stemming from prenatal alcohol exposure and linked with severe neurocognitive deficits. METHOD The sample comprised 80 justice-involved adults (ages 19-40) including 25 with confirmed or possible FASD and 55 where FASD was ruled out. Ten PVT scores were calculated, derived from Word Memory Test, Genuine Memory Impairment Profile, Advanced Clinical Solutions (Word Choice), the Wechsler Adult Intelligence Scale - Fourth Edition (Reliable Digit Span and age-corrected scaled scores (ACSS) from Digit Span, Coding, Symbol Search, Coding - Symbol Search, Vocabulary - Digit Span), and the Wechsler Memory Scale - Fourth Edition (Logical Memory II Recognition). RESULTS Participants with diagnosed/possible FASD were more likely to fail any single PVT, and failed a greater number of PVTs overall, compared to those without FASD. They were also more likely to fail based on Word Memory Test, Digit Span ACSS, Coding ACSS, Symbol Search ACSS, and Logical Memory II Recognition, compared to controls (35-76%). Across both groups, substantially more participants with IQ <70 failed two or more PVTs (90%), compared to those with an IQ ≥70 (44%). CONCLUSIONS Results highlight the need for additional research examining the use of PVTs in justice-involved populations with FASD.
Collapse
|
17
|
Identifying Novel Embedded Performance Validity Test Formulas Within the Repeatable Battery for the Assessment of Neuropsychological Status: a Simulation Study. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-020-09382-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
18
|
Ovsiew GP, Resch ZJ, Nayar K, Williams CP, Soble JR. Not so fast! Limitations of processing speed and working memory indices as embedded performance validity tests in a mixed neuropsychiatric sample. J Clin Exp Neuropsychol 2020; 42:473-484. [DOI: 10.1080/13803395.2020.1758635] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Gabriel P. Ovsiew
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Zachary J Resch
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Kritika Nayar
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychiatry and Behavioral Sciences, Northwestern Feinberg School of Medicine, Chicago, IL, USA
| | - Christopher P. Williams
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Jason R. Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
19
|
Hurtubise J, Baher T, Messa I, Cutler L, Shahein A, Hastings M, Carignan-Querqui M, Erdodi LA. Verbal fluency and digit span variables as performance validity indicators in experimentally induced malingering and real world patients with TBI. APPLIED NEUROPSYCHOLOGY-CHILD 2020; 9:337-354. [DOI: 10.1080/21622965.2020.1719409] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Affiliation(s)
| | - Tabarak Baher
- Department of Psychology, University of Windsor, Windsor, Canada
| | - Isabelle Messa
- Department of Psychology, University of Windsor, Windsor, Canada
| | - Laura Cutler
- Department of Psychology, University of Windsor, Windsor, Canada
| | - Ayman Shahein
- Department of Clinical Neurosciences, University of Calgary, Calgary, Canada
| | | | | | - Laszlo A. Erdodi
- Department of Psychology, University of Windsor, Windsor, Canada
| |
Collapse
|
20
|
Optimization of Performance Validity Test (PVT) Cutoffs across Healthy and Non-Referred Clinical Research Samples. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-020-09372-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
21
|
Psychological Symptoms and Rates of Performance Validity Improve Following Trauma-Focused Treatment in Veterans with PTSD and History of Mild-to-Moderate TBI. J Int Neuropsychol Soc 2020; 26:108-118. [PMID: 31658923 DOI: 10.1017/s1355617719000997] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
OBJECTIVE Iraq and Afghanistan Veterans with posttraumatic stress disorder (PTSD) and traumatic brain injury (TBI) history have high rates of performance validity test (PVT) failure. The study aimed to determine whether those with scores in the invalid versus valid range on PVTs show similar benefit from psychotherapy and if psychotherapy improves PVT performance. METHOD Veterans (N = 100) with PTSD, mild-to-moderate TBI history, and cognitive complaints underwent neuropsychological testing at baseline, post-treatment, and 3-month post-treatment. Veterans were randomly assigned to cognitive processing therapy (CPT) or a novel hybrid intervention integrating CPT with TBI psychoeducation and cognitive rehabilitation strategies from Cognitive Symptom Management and Rehabilitation Therapy (CogSMART). Performance below standard cutoffs on any PVT trial across three different PVT measures was considered invalid (PVT-Fail), whereas performance above cutoffs on all measures was considered valid (PVT-Pass). RESULTS Although both PVT groups exhibited clinically significant improvement in PTSD symptoms, the PVT-Pass group demonstrated greater symptom reduction than the PVT-Fail group. Measures of post-concussive and depressive symptoms improved to a similar degree across groups. Treatment condition did not moderate these results. Rate of valid test performance increased from baseline to follow-up across conditions, with a stronger effect in the SMART-CPT compared to CPT condition. CONCLUSION Both PVT groups experienced improved psychological symptoms following treatment. Veterans who failed PVTs at baseline demonstrated better test engagement following treatment, resulting in higher rates of valid PVTs at follow-up. Veterans with invalid PVTs should be enrolled in trauma-focused treatment and may benefit from neuropsychological assessment after, rather than before, treatment.
Collapse
|
22
|
Ventura LM, DeDios-Stern S, Oh A, Soble JR. They're not just little adults: The utility of adult performance validity measures in a mixed clinical pediatric sample. APPLIED NEUROPSYCHOLOGY-CHILD 2019; 10:297-307. [PMID: 31703167 DOI: 10.1080/21622965.2019.1685522] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Performance validity tests (PVTs) have become a standard part of adult neuropsychological practice; however, they are less widely used in pediatric testing. The current study aimed to obtain a better understanding of the application of PVTs within a mixed clinical pediatric sample with a wide range of diagnosis, IQ, and age. Cross-sectional data were analyzed from 130 consecutive pediatric patients evaluated as part of clinical care and diagnosed with a variety of medical/neurological, developmental, and psychiatric disorders. Patients were administered a battery of neuropsychological tests; results of intellectual functioning measures (i.e., Wechsler Intelligence Scale for Children-Fifth Edition [WISC-V] or Wechsler Adult Intelligence Scale-Fourth Edition [WAIS-IV]), and PVTs (i.e., Test of Memory Malingering [TOMM] and Digit Span [DS] subtests of the WISC-V/WAIS-IV) were analyzed to assess PVT performance across the sample as well as age- and Full-Scale IQ-related (FSIQ) effects on pass rate. Results suggested that the TOMM is an effective validity test for youth, as the TOMM adult cutoff score was also valid for children (88% pass rate on TOMM trial 1 cut-score ≥41, 71% pass rate on TOMM trial 1 cut-score ≥45). In contrast, Reliable Digit Span (RDS) was less accurate (34% failed RDS [cut-score ≤6], 54% failed RDS-r [cut-score ≤10], and 25% failed DS ACSS [cut-score ≤5]) using standard adult cutoffs. Notably, although TOMM scores were not strongly influenced by IQ, DS scores increased as IQ increased. Overall, further analysis of PVTs can champion new standards of practice through additional research establishing PVT accuracy within pediatric populations.
Collapse
Affiliation(s)
- Lea M Ventura
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Pediatrics, University of Illinois College of Medicine, Chicago, IL, USA
| | - Samantha DeDios-Stern
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Alison Oh
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jason R Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
23
|
Ryan JJ, Yamaguchi T, Kreiner DS. Preliminary Validation of the Rey 15-Item Test and Reliable Digit Span in Native Japanese Samples. Psychol Rep 2019; 122:1925-1945. [DOI: 10.1177/0033294118792697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The Rey 15-Item Test and reliable digit span were evaluated in Japan. Participants were controls ( n = 15), healthy volunteers instructed to simulate memory impairment ( n = 12; 5 of 17 volunteers did not comply with instructions and were dropped), healthy elderly ( n = 12), and cognitively disabled nursing home residents ( n = 8). On the 15-Item Test, controls and elderly performed similarly and were combined. Nursing home residents could not cope with the 15-Item Test and were dropped. Total score was a fair predictor of dissimulation using a cutoff ≤ 8. Rows were fair predictors using a ≤2 cutoff. Sensitivities were low and specificities were excellent. Reliable digit span contrasts between simulators and each of the other groups demonstrated that reliable digit span discriminated controls and elderly from simulators (≤6 and ≤5 cutoffs). Sensitivities were moderate and specificities were excellent. Reliable digit span did not differentiate simulators from nursing home residents.
Collapse
Affiliation(s)
- Joseph J. Ryan
- School of Nutrition, Kinesiology, and Psychological Science, University of Central Missouri, MO, USA
| | - Takahiro Yamaguchi
- Department of Psychology and Counseling, Northeastern State University, OK, USA
| | - David S. Kreiner
- School of Nutrition, Kinesiology, and Psychological Science, University of Central Missouri, MO, USA
| |
Collapse
|
24
|
Keilp JG, Corbera K, Gorlyn M, Oquendo MA, Mann JJ, Fallon BA. Neurocognition in Post-Treatment Lyme Disease and Major Depressive Disorder. Arch Clin Neuropsychol 2019; 34:466-480. [PMID: 30418507 DOI: 10.1093/arclin/acy083] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2018] [Revised: 08/10/2018] [Accepted: 10/26/2018] [Indexed: 01/09/2023] Open
Abstract
OBJECTIVE Neurocognitive dysfunction in patients with residual or emergent symptoms after treatment for Lyme Disease is often attributed to comorbid depression. In this study, patients with Post-Treatment Lyme Disease Syndrome (PTLDS) were compared to patients with Major Depressive Disorder (MDD), as well as healthy comparison subjects (HC), on neurocognitive measures administered through the same laboratory, to determine if patterns of performance were similar. METHODS Two analyses were conducted. First, performance on the Wechsler Adult Intelligence Scale (WAIS-III) and on subtests from the Wechsler Memory Scale (WMS-III) was compared among the groups. Second, comparable subgroups of PTLDS and MDD patients with at least one low WMS-III score were compared on an additional set of measures assessing motor function, psychomotor performance, attention, memory, working memory, and language fluency, to determine if the overall profile of performance was similar in the two subgroups. RESULTS In the first analysis, PTLDS patients performed more poorly than both MDD and HC on tasks assessing verbal abilities, working memory, and paragraph learning. Processing speed in the two patient groups, however, was equally reduced. In the second analysis, MDD patients with low WMS-III exhibited concomitantly greater difficulties in psychomotor speed and attention, while low-WMS-III PTLDS patients exhibited greater difficulties in language fluency. CONCLUSIONS MDD and PTLDS can be confused neuropsychologically because both exhibit similar levels of psychomotor slowing. However, problems on memory-related tasks, though mild, are more pronounced in PTLDS. PTLDS patients with poorer memory also exhibit poorer language fluency, and less deficit in processing speed and attention compared to MDD.
Collapse
Affiliation(s)
- John G Keilp
- Lyme Disease Research Center, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Molecular Imaging and Neuropathology, New York State Psychiatric Institute, New York, NY, USA
| | - Kathy Corbera
- Lyme Disease Research Center, Columbia University College of Physicians and Surgeons, New York, NY, USA
| | - Marianne Gorlyn
- Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY, USA
| | - Maria A Oquendo
- Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Molecular Imaging and Neuropathology, New York State Psychiatric Institute, New York, NY, USA.,Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - J John Mann
- Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Molecular Imaging and Neuropathology, New York State Psychiatric Institute, New York, NY, USA
| | - Brian A Fallon
- Lyme Disease Research Center, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY, USA.,Department of Molecular Imaging and Neuropathology, New York State Psychiatric Institute, New York, NY, USA
| |
Collapse
|
25
|
Shura RD, Martindale SL, Taber KH, Higgins AM, Rowland JA. Digit Span embedded validity indicators in neurologically-intact veterans. Clin Neuropsychol 2019; 34:1025-1037. [PMID: 31315519 DOI: 10.1080/13854046.2019.1635209] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
Objective: Embedded validity measures are useful in neuropsychological evaluations but should be updated with new test versions and validated across various samples. This study evaluated Wechsler Adult Intelligence Scale, 4th edition (WAIS-IV) Digit Span validity indicators in post-deployment veterans.Method: Neurologically-intact veterans completed structured diagnostic interviews, the WAIS-IV, the Medical Symptom Validity Test (MSVT), and the b Test as part of a larger study. The Noncredible group included individuals who failed either the MSVT or the b Test. Of the total sample (N = 275), 21.09% failed the MSVT and/or b Test. Diagnostic accuracy was calculated predicting group status across cutoff scores on two Digit Span variables, four Reliable Digit Span (RDS) variables, and two Vocabulary minus Digit Span variables.Results: Digit Span age-corrected scaled score (ACSS) had the highest AUC (.648) of all measures assessed; however, sensitivity at the best cutoff of <7 was only 0.17. Of RDS measures, the Working Memory RDS resulted in the highest AUC (.629), but Enhanced RDS and Alternate RDS produced the highest sensitivities (0.22). Overall, cutoff scores were consistent with other studies, but sensitivities were lower. Vocabulary minus Digit Span measures were not significant.Conclusions: Digit Span ACSS was the strongest predictor of noncredible performance, and outperformed traditional RDS variants. Sensitivity across all validity indicators was low in this research sample, though cutoff scores were congruent with previous research. Although embedded Digit Span validity indicators may be useful, they are not sufficient to replace standalone performance validity tests.
Collapse
Affiliation(s)
- Robert D Shura
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, NC, USA.,Salisbury Veterans Affairs Medical Center, Salisbury, NC, USA.,Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Sarah L Martindale
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, NC, USA.,Salisbury Veterans Affairs Medical Center, Salisbury, NC, USA.,Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Katherine H Taber
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, NC, USA.,Salisbury Veterans Affairs Medical Center, Salisbury, NC, USA.,Via College of Osteopathic Medicine, Blacksburg, VA, USA.,Baylor College of Medicine, Houston, TX, USA
| | - Alana M Higgins
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, NC, USA
| | - Jared A Rowland
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, NC, USA.,Salisbury Veterans Affairs Medical Center, Salisbury, NC, USA.,Wake Forest School of Medicine, Winston-Salem, NC, USA
| |
Collapse
|
26
|
Performance Validity in Collegiate Football Athletes at Baseline Neurocognitive Testing. J Head Trauma Rehabil 2019; 34:E20-E31. [DOI: 10.1097/htr.0000000000000451] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
27
|
Geographic Variation and Instrumentation Artifacts: in Search of Confounds in Performance Validity Assessment in Adults with Mild TBI. PSYCHOLOGICAL INJURY & LAW 2019. [DOI: 10.1007/s12207-019-09354-w] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
|
28
|
Rai JK, Erdodi LA. Impact of criterion measures on the classification accuracy of TOMM-1. APPLIED NEUROPSYCHOLOGY-ADULT 2019; 28:185-196. [PMID: 31187632 DOI: 10.1080/23279095.2019.1613994] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
This study was designed to examine the effect of various criterion measures on the classification accuracy of Trial 1 of the Test of Memory Malingering (TOMM-1), a free-standing performance validity test (PVT). Archival data were collected from a case sequence of 91 (M Age = 42.2 years; M Education = 12.7) patients clinically referred for neuropsychological assessment. Trials 2 and Retention of the TOMM, the Word Choice Test, and three validity composites were used as criterion PVTs. Classification accuracy varied systematically as a function of criterion PVT. TOMM-1 ≤ 43 emerged as the optimal cutoff, resulting in a wide range of sensitivity (.47-1.00), with perfect overall specificity. Failing the TOMM-1 was unrelated to age, education or gender, but was associated with elevated self-reported depression. Results support the utility of TOMM-1 as an independent, free-standing, single-trial PVT. Consistent with previous reports, the choice of criterion measure influences parameter estimates of the PVT being calibrated. The methodological implications of modality specificity to PVT research and clinical/forensic practice should be considered when evaluating cutoffs or interpreting scores in the failing range.
Collapse
Affiliation(s)
- Jaspreet K Rai
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada.,University of Windsor, Edmonton, Alberta, Canada
| | - Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| |
Collapse
|
29
|
Abeare C, Sabelli A, Taylor B, Holcomb M, Dumitrescu C, Kirsch N, Erdodi L. The Importance of Demographically Adjusted Cutoffs: Age and Education Bias in Raw Score Cutoffs Within the Trail Making Test. PSYCHOLOGICAL INJURY & LAW 2019. [DOI: 10.1007/s12207-019-09353-x] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
30
|
Burke SL, Naseh M, Rodriguez MJ, Burgess A, Loewenstein D. Dementia-Related Neuropsychological Testing Considerations in Non-Hispanic White and Latino/Hispanic Populations. PSYCHOLOGY & NEUROSCIENCE 2019; 12:144-168. [PMID: 31649798 PMCID: PMC6812579 DOI: 10.1037/pne0000163] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Hispanic individuals are at greater risk for health disparities, less than optimal health care, and are diagnosed at later stages of cognitive impairment than white non-Hispanics. Acculturation and different attitudes toward test-taking may result in decrements in performance, especially on unfamiliar measures that emphasize speed and accuracy. Non-Hispanic individuals often outperform Hispanic individuals on cognitive and neuropsychological measures in community and clinical populations. Current neuropsychological testing may not provide accurate data related to monolingual and bilingual individuals of Hispanic descent. Testing instruments were identified by searching academic databases using combinations of relevant search terms. Neuropsychological instruments were included if they were designed to detect cognitive impairment, had an administration time of less than 45 minutes, and were available in English. Validity studies were required to employ gold standard comparison diagnostic criteria. Twenty-nine instruments were evaluated in dementia staging, global cognition, memory, memory and visual abilities, working memory and attention, verbal learning and memory, recall, language, premorbid intelligence, literacy/cognitive reserve, visuospatial, attention, problem-solving, problem solving and perception, functional assessment, and mood/daily functioning domains. Spanish-language neuropsychological instruments need to be made widely available and existing instruments to be normed in Spanish to best serve and assess diverse populations. Psychometric data were reported for neuropsychological instruments, which may be administered to Hispanic older adults presenting for evaluation related to dementia-spectrum disorders. This is one of the few reviews to provide an overview of the sensitivity and specificity of available Spanish translated neuropsychological instruments.
Collapse
Affiliation(s)
- Shanna L Burke
- Robert Stempel College of Public Health and Social Work, School of Social Work, Florida International University
| | - Mitra Naseh
- Robert Stempel College of Public Health and Social Work, School of Social Work, Florida International University
| | | | - Aaron Burgess
- Robert Stempel College of Public Health and Social Work, School of Social Work, Florida International University
| | - David Loewenstein
- Center on Aging as the Center for Cognitive Neuroscience and Aging and Department of Psychiatry and Behavioral Sciences, Miller School of Medicine, University of Miami
| |
Collapse
|
31
|
Jurick SM, Crocker LD, Keller AV, Hoffman SN, Bomyea J, Jacobson MW, Jak AJ. The Minnesota Multiphasic Personality Inventory-2-RF in Treatment-Seeking Veterans with History of Mild Traumatic Brain Injury. Arch Clin Neuropsychol 2019; 34:366-380. [PMID: 29850866 DOI: 10.1093/arclin/acy048] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Revised: 03/26/2018] [Accepted: 05/09/2018] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE This study examined the Minnesota Multiphasic Personality Inventory-Second Edition-Restructured Form (MMPI-2-RF) to better understand symptom presentation in a sample of treatment-seeking Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans with self-reported history of mild traumatic brain injury (mTBI). METHOD Participants underwent a comprehensive clinical neuropsychological battery including performance and symptom validity measures and self-report measures of depressive, posttraumatic, and post-concussive symptomatology. Those with possible symptom exaggeration (SE+) on the MMPI-2-RF were compared with those without (SE-) with regard to injury, psychiatric, validity, and cognitive variables. RESULTS Between 50% and 87% of participants demonstrated possible symptom exaggeration on one or more MMPI-2-RF validity scales, and a large majority were elevated on content scales related to cognitive, somatic, and emotional complaints. The SE+ group reported higher depressive, posttraumatic, and post-concussive symptomatology, had higher scores on symptom validity measures, and performed more poorly on neuropsychological measures compared with the SE- group. There were no group differences with regard to injury variables or performance validity measures. Participants were more likely to exhibit possible symptom exaggeration on cognitive/somatic compared with traditional psychopathological validity scales. CONCLUSIONS A sizable portion of treatment-seeking OEF/OIF Veterans demonstrated possible symptom exaggeration on MMPI-2-RF validity scales, which was associated with elevated scores on self-report measures and poorer cognitive performance, but not higher rates of performance validity failure, suggesting symptom and performance validity are distinct concepts. These findings have implications for the interpretation of clinical data in the context of possible symptom exaggeration and treatment in Veterans with persistent post-concussive symptoms.
Collapse
Affiliation(s)
- S M Jurick
- Department of Psychiatry, San Diego State University/University of California San Diego Joint Doctoral Program in Clinical Psychology, San Diego, CA, USA.,Veterans Medical Research Foundation, San Diego, CA, USA
| | - L D Crocker
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA, USA
| | - A V Keller
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - S N Hoffman
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - J Bomyea
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - M W Jacobson
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - A J Jak
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA, USA.,Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| |
Collapse
|
32
|
Waldron-Perrine B, Gabel NM, Seagly K, Kraal AZ, Pangilinan P, Spencer RJ, Bieliauskas L. Montreal Cognitive Assessment as a screening tool: Influence of performance and symptom validity. Neurol Clin Pract 2019; 9:101-108. [PMID: 31041123 DOI: 10.1212/cpj.0000000000000604] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2018] [Accepted: 12/03/2018] [Indexed: 11/15/2022]
Abstract
Background We evaluated Montreal Cognitive Assessment (MoCA) performance in a veteran traumatic brain injury (TBI) population, considering performance validity test (PVT) and symptom validity test (SVT) data, and explored associations of MoCA performance with neuropsychological test performance and self-reported distress. Methods Of 198 consecutively referred veterans to a Veterans Administration TBI/Polytrauma Clinic, 117 were included in the final sample. The MoCA was administered as part of the evaluation. Commonly used measures of neuropsychological functioning and performance and symptom validity were also administered to aid in diagnosis. Results Successively worse MoCA performances were associated with a greater number of PVT failures (ps < 0.05). Failure of both the SVT and at least 1 PVT yielded the lowest MoCA scores. Self-reported distress (both posttraumatic stress disorder symptoms and neurobehavioral cognitive symptoms) was also related to MoCA performance. Conclusions Performance on the MoCA is influenced by task engagement and symptom validity. Causal inferences about neurologic and neurocognitive impairment, particularly in the context of mild TBI, wherein the natural course of recovery is well known, should therefore be made cautiously when such inferences are based heavily on MoCA scores. Neuropsychologists are well versed in the assessment of performance and symptom validity and thus may be well suited to explore the influences of abnormal performances on cognitive screening.
Collapse
Affiliation(s)
- Brigid Waldron-Perrine
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - Nicolette M Gabel
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - Katharine Seagly
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - A Zarina Kraal
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - Percival Pangilinan
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - Robert J Spencer
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| | - Linas Bieliauskas
- Department of Physical Medicine and Rehabilitation (BW-P), Wayne State University School of Medicine, Detroit; Department of Physical Medicine and Rehabilitation (NMG, KS, PP), Michigan Medicine, University of Michigan; Department of Psychology (AZK), University of Michigan; VA Health System (PP); Mental health Service (116b) (RJS), VA Ann Arbor Healthcare; Neuropsychology Section of Psychiatry (RJS, LB), Michigan Medicine; and Ann Arbor, MI
| |
Collapse
|
33
|
Alverson WA, O’Rourke JJF, Soble JR. The Word Memory Test genuine memory impairment profile discriminates genuine memory impairment from invalid performance in a mixed clinical sample with cognitive impairment. Clin Neuropsychol 2019; 33:1420-1435. [DOI: 10.1080/13854046.2019.1599071] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- W. Alex Alverson
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | | | - Jason R. Soble
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
34
|
Bain KM, Soble JR, Webber TA, Messerly JM, Bailey KC, Kirton JW, McCoy KJM. Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance. APPLIED NEUROPSYCHOLOGY-ADULT 2019; 28:24-34. [PMID: 30987451 DOI: 10.1080/23279095.2019.1585352] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.
Collapse
Affiliation(s)
- Kathleen M Bain
- South Texas Veterans Health Care System, San Antonio, Texas, USA
| | - Jason R Soble
- University of Illinois at Chicago College of Medicine, Chicago, Illinois, USA
| | - Troy A Webber
- Michael E. DeBakey VA Medical Center, Houston, Texas, USA
| | | | - K Chase Bailey
- University of Texas Southwestern Medical Center at Dallas, Dallas, Texas, USA
| | - Joshua W Kirton
- Fort Carson Army Medical Center, Colorado Springs, Colorado, USA
| | - Karin J M McCoy
- South Texas Veterans Health Care System, San Antonio, Texas, USA
| |
Collapse
|
35
|
Ruchinskas R. Wechsler adult intelligence scale-4th edition digit span performance in subjective cognitive complaints, amnestic mild cognitive impairment, and probable dementia of the Alzheimer type. Clin Neuropsychol 2019; 33:1436-1444. [DOI: 10.1080/13854046.2019.1585574] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Robert Ruchinskas
- Department of Psychiatry, UT Southwestern Medical Center, Dallas, TX, USA
- Department of Neurology & Neurotherapeutics, UT Southwestern Medical Center, Dallas, TX, USA
| |
Collapse
|
36
|
Gabel NM, Waldron-Perrine B, Spencer RJ, Pangilinan PH, Hale AC, Bieliauskas LA. Suspiciously slow: timed digit span as an embedded performance validity measure in a sample of veterans with mTBI. Brain Inj 2018; 33:377-382. [DOI: 10.1080/02699052.2018.1553311] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Nicolette M. Gabel
- Department of Physical Medicine and Rehabilitation, Michigan Medicine, Ann Arbor, USA
| | | | - Robert J. Spencer
- Mental Health Services, VA Ann Arbor Healthcare System, Ann Arbor, USA
| | - Percival H. Pangilinan
- Department of Physical Medicine and Rehabilitation, Michigan Medicine/VA Ann Arbor Healthcare System, Ann Arbor, USA
| | - Andrew C. Hale
- Mental Health Services, VA Ann Arbor Healthcare System, Ann Arbor, USA
| | - Linas A. Bieliauskas
- Department of Neuropsychology, University of Michigan Health System, Ann Arbor, USA
| |
Collapse
|
37
|
Critchfield E, Soble JR, Marceaux JC, Bain KM, Chase Bailey K, Webber TA, Alex Alverson W, Messerly J, Andrés González D, O’Rourke JJF. Cognitive impairment does not cause invalid performance: analyzing performance patterns among cognitively unimpaired, impaired, and noncredible participants across six performance validity tests. Clin Neuropsychol 2018; 33:1083-1101. [DOI: 10.1080/13854046.2018.1508615] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Edan Critchfield
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Jason R. Soble
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| | - Janice C. Marceaux
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Department of Neurology, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| | - Kathleen M. Bain
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - K. Chase Bailey
- Division of Psychology, UT Southwestern Medical Center, Dallas, TX, USA
| | - Troy A. Webber
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - W. Alex Alverson
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Johanna Messerly
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - David Andrés González
- Department of Neurology, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| | | |
Collapse
|
38
|
Webber TA, Critchfield EA, Soble JR. Convergent, Discriminant, and Concurrent Validity of Nonmemory-Based Performance Validity Tests. Assessment 2018; 27:1399-1415. [DOI: 10.1177/1073191118804874] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
To supplement memory-based Performance Validity Tests (PVTs) in identifying noncredible performance, we examined the validity of the two most commonly used nonmemory-based PVTs—Dot Counting Test (DCT) and Wechsler Adult Intelligence Scale–Fourth edition (WAIS-IV) Reliable Digit Span (RDS)—as well as two alternative WAIS-IV Digit Span (DS) subtest PVTs. Examinees completed DCT, WAIS-IV DS, and the following criterion PVTs: Test of Memory Malingering, Word Memory Test, and Word Choice Test. Validity groups were determined by passing 3 (valid; n = 69) or failing ⩾2 (noncredible; n = 30) criterion PVTs. DCT, RDS, RDS–Revised (RDS-R), and WAIS-IV DS Age-Corrected Scaled Score (ACSS) were significantly correlated (but uncorrelated with memory-based PVTs). Combining RDS, RDS-R, and ACSS with DCT improved classification accuracy (particularly for DCT/ACSS) for detecting noncredible performance among valid-unimpaired, but largely not valid-impaired examinees. Combining DCT with ACSS may uniquely assess and best supplement memory-based PVTs to identify noncredible neuropsychological test performance in cognitively unimpaired examinees.
Collapse
Affiliation(s)
- Troy A. Webber
- South Texas Veterans Health Care System, San Antonio, TX, USA
| | | | - Jason R. Soble
- South Texas Veterans Health Care System, San Antonio, TX, USA
- University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
39
|
Differentiating epilepsy from psychogenic nonepileptic seizures using neuropsychological test data. Epilepsy Behav 2018; 87:39-45. [PMID: 30172082 DOI: 10.1016/j.yebeh.2018.08.010] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/06/2018] [Revised: 07/25/2018] [Accepted: 08/12/2018] [Indexed: 11/21/2022]
Abstract
OBJECTIVE Differentiating epileptic seizures (ES) from psychogenic nonepileptic seizures (PNES) represents a challenging differential diagnosis with important treatment implications. This study was designed to explore the utility of neuropsychological test scores in differentiating ES from PNES. METHOD Psychometric data from 72 patients with ES and 33 patients with PNES were compared on various tests of cognitive ability and performance validity. Individual measures that best discriminated the diagnoses were then entered as predictors in a logistic regression equation with group membership (ES vs. PNES) as the criterion. RESULTS On most tests of cognitive ability, the PNES sample outperformed the ES sample (medium-large effect) and was less likely to fail the Reliable Digit Span. However, patients with PNES failed two embedded validity indicators at significantly higher rates (risk ratios (RR): 2.45-4.16). There were no group differences on the Test of Memory Malingering (TOMM). A logistic regression equation based on seven neuropsychological tests correctly classified 85.1% of patients. The cutoff with perfect specificity was associated with 0.47 sensitivity. CONCLUSIONS Consistent with previous research, the utility of psychometric methods of differential diagnosis is limited by the complex neurocognitive profiles associated with ES and PNES. Although individual measures might help differentiate ES from PNES, multivariate assessment models have superior discriminant power. The strongest psychometric evidence for PNES appears to be a consistent lack of impairment on tests sensitive to diffuse neurocognitive deficits such as processing speed, working memory, and verbal fluency. While video-electroencephalogram (EEG) monitoring is the gold standard of differential diagnosis, psychometric testing has the potential to enhance clinical decision-making, particularly in complex or unclear cases such as patients with nondiagnostic video-EEGs. Adopting a standardized, fixed neuropsychological battery at epilepsy centers would advance research on the differential diagnostic power of psychometric testing.
Collapse
|
40
|
Worse baseline executive functioning is associated with dropout and poorer response to trauma-focused treatment for veterans with PTSD and comorbid traumatic brain injury. Behav Res Ther 2018; 108:68-77. [DOI: 10.1016/j.brat.2018.07.004] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2018] [Revised: 06/28/2018] [Accepted: 07/13/2018] [Indexed: 11/23/2022]
|
41
|
An KY, Charles J, Ali S, Enache A, Dhuga J, Erdodi LA. Reexamining performance validity cutoffs within the Complex Ideational Material and the Boston Naming Test–Short Form using an experimental malingering paradigm. J Clin Exp Neuropsychol 2018; 41:15-25. [DOI: 10.1080/13803395.2018.1483488] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Affiliation(s)
- Kelly Y. An
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Jordan Charles
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Sami Ali
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Anca Enache
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Jasmine Dhuga
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Laszlo A. Erdodi
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
42
|
Flaherty JM, Spencer RJ, Drag LL, Pangilinan PH, Bieliauskas LA. Streamlining screening of emotional function in Veterans with traumatic brain injury. J Clin Psychol 2018; 74:1281-1292. [DOI: 10.1002/jclp.22595] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2016] [Revised: 11/08/2017] [Accepted: 01/06/2018] [Indexed: 11/08/2022]
Affiliation(s)
| | | | - Lauren L. Drag
- Department of Neurology and Neurological Sciences; Stanford University Medical Center
| | | | - Linas A. Bieliauskas
- Veterans Affairs Ann Arbor Healthcare System
- University of Michigan Health System
| |
Collapse
|
43
|
Erdodi LA, Abeare CA, Medoff B, Seke KR, Sagar S, Kirsch NL. A Single Error Is One Too Many: The Forced Choice Recognition Trial of the CVLT-II as a Measure of Performance Validity in Adults with TBI. Arch Clin Neuropsychol 2017; 33:845-860. [DOI: 10.1093/acn/acx110] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/20/2017] [Indexed: 11/12/2022] Open
Affiliation(s)
- Laszlo A Erdodi
- Department of Psychology, University of Windsor, 168 Chrysler Hall South, Windsor, Canada ON
| | - Christopher A Abeare
- Department of Psychology, University of Windsor, 170 Chrysler Hall South, Windsor, Canada ON
| | - Brent Medoff
- The Commonwealth Medical College, 525 Pine St, Scranton, PA, USA
| | - Kristian R Seke
- University of Windsor, Brain-Cognition-Neuroscience Program, G105 Chrysler Hall North, Windsor, Canada ON
| | - Sanya Sagar
- Department of Psychology, University of Windsor, 109 Chrysler Hall North, Windsor, Canada ON
| | - Ned L Kirsch
- Department of Physical Medicine and Rehabilitation, University of Michigan, Briarwood Circle #4 Ann Arbor, MI, USA
| |
Collapse
|
44
|
Webber TA, Soble JR. Utility of various WAIS-IV Digit Span indices for identifying noncredible performance validity among cognitively impaired and unimpaired examinees. Clin Neuropsychol 2017; 32:657-670. [DOI: 10.1080/13854046.2017.1415374] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Troy A. Webber
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
| | - Jason R. Soble
- Psychology Service, South Texas Veterans Health Care System, San Antonio, TX, USA
- Departments of Psychiatry and Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
45
|
Erdodi LA. Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 26:155-172. [PMID: 29111772 DOI: 10.1080/23279095.2017.1384925] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (MAge = 46.6; MEducation = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5VER) and one based on five nonverbal PVTs (EI-5NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."
Collapse
Affiliation(s)
- Laszlo A Erdodi
- a Department of Psychology , University of Windsor , Windsor , Canada
| |
Collapse
|
46
|
Denning JH, Shura RD. Cost of malingering mild traumatic brain injury-related cognitive deficits during compensation and pension evaluations in the veterans benefits administration. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 26:1-16. [DOI: 10.1080/23279095.2017.1350684] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- John H. Denning
- Department of Veteran Affairs, Mental Health Service, Ralph H. Johnson Veterans Affairs Medical Center, Charleston, South Carolina, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, South Carolina, USA
| | - Robert D. Shura
- Mid-Atlantic Mental Illness Research, Education, and Clinical Center, Salisbury, North Carolina, USA
- Mental Health and Behavioral Science Service Line, W. G. (Bill) Hefner Veterans Affairs Medical Center (VAMC), Salisbury, North Carolina, USA
- Department of Psychiatry and Behavioral Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA
| |
Collapse
|
47
|
Dorociak KE, Schulze ET, Piper LE, Molokie RE, Janecek JK. Performance validity testing in a clinical sample of adults with sickle cell disease. Clin Neuropsychol 2017. [PMID: 28632024 DOI: 10.1080/13854046.2017.1339830] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
OBJECTIVE Neuropsychologists utilize performance validity tests (PVTs) as objective means for drawing inferences about performance validity. The Test of Memory Malingering (TOMM) is a well-validated, stand-alone PVT and the Reliable Digit Span (RDS) and Reliable Digit Span-Revised (RDS-R) from the Digit Span subtest of the WAIS-IV are commonly employed, embedded PVTs. While research has demonstrated the utility of these PVTs with various clinical samples, no research has investigated their use in adults with sickle cell disease (SCD), a condition associated with multiple neurological, physical, and psychiatric symptoms. Thus, the purpose of this study was to explore PVT performance in adults with SCD. METHOD Fifty-four adults with SCD (Mage = 40.61, SD = 12.35) were consecutively referred by their hematologist for a routine clinical outpatient neuropsychological evaluation. During the evaluation, participants were administered the TOMM (Trials 1 and 2), neuropsychological measures including the WAIS-IV Digit Span subtest, and mood and behavioral questionnaires. RESULTS The average score on the TOMM was 47.70 (SD = 3.47, range = 34-50) for Trial 1 and 49.69 (SD = 1.66, range = 38-50) for Trial 2. Only one participant failed Trial 2 of the TOMM, yielding a 98.1% pass rate for the sample. Pass rates at various RDS and RDS-R values were calculated with TOMM Trial 2 performance as an external criterion. CONCLUSIONS Results support the use of the TOMM as a measure of performance validity for individuals with SCD, while RDS and RDS-R should be interpreted with caution in this population.
Collapse
Affiliation(s)
- Katherine E Dorociak
- a Department of Psychiatry , University of Illinois at Chicago , Chicago , IL , USA
| | - Evan T Schulze
- a Department of Psychiatry , University of Illinois at Chicago , Chicago , IL , USA
| | - Lauren E Piper
- a Department of Psychiatry , University of Illinois at Chicago , Chicago , IL , USA
| | - Robert E Molokie
- b Department of Medicine , University of Illinois at Chicago , Chicago , IL , USA
| | - Julie K Janecek
- a Department of Psychiatry , University of Illinois at Chicago , Chicago , IL , USA
| |
Collapse
|
48
|
Psychometric Markers of Genuine and Feigned Neurodevelopmental Disorders in the Context of Applying for Academic Accommodations. PSYCHOLOGICAL INJURY & LAW 2017. [DOI: 10.1007/s12207-017-9287-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
49
|
Erdodi LA, Lichtenstein JD. Invalid before impaired: an emerging paradox of embedded validity indicators. Clin Neuropsychol 2017; 31:1029-1046. [DOI: 10.1080/13854046.2017.1323119] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Canada
| | - Jonathan D. Lichtenstein
- Department of Psychiatry, Neuropsychology Services, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
| |
Collapse
|
50
|
Erdodi LA, Tyson BT, Abeare CA, Zuccato BG, Rai JK, Seke KR, Sagar S, Roth RM. Utility of critical items within the Recognition Memory Test and Word Choice Test. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 25:327-339. [DOI: 10.1080/23279095.2017.1298600] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| | - Bradley T. Tyson
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
- Western Washington Medical Group, Everett, Washington, USA
| | | | - Brandon G. Zuccato
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Jaspreet K. Rai
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Kristian R. Seke
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Sanya Sagar
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Robert M. Roth
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| |
Collapse
|