1
|
Finley JCA, Brooks JM, Nili AN, Oh A, VanLandingham HB, Ovsiew GP, Ulrich DM, Resch ZJ, Soble JR. Multivariate examination of embedded indicators of performance validity for ADHD evaluations: A targeted approach. APPLIED NEUROPSYCHOLOGY. ADULT 2023:1-14. [PMID: 37703401 DOI: 10.1080/23279095.2023.2256440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/15/2023]
Abstract
This study investigated the individual and combined utility of 10 embedded validity indicators (EVIs) within executive functioning, attention/working memory, and processing speed measures in 585 adults referred for an attention-deficit/hyperactivity disorder (ADHD) evaluation. Participants were categorized into invalid and valid performance groups as determined by scores from empirical performance validity indicators. Analyses revealed that all of the EVIs could meaningfully discriminate invalid from valid performers (AUCs = .69-.78), with high specificity (≥90%) but low sensitivity (19%-51%). However, none of them explained more than 20% of the variance in validity status. Combining any of these 10 EVIs into a multivariate model significantly improved classification accuracy, explaining up to 36% of the variance in validity status. Integrating six EVIs from the Stroop Color and Word Test, Trail Making Test, Verbal Fluency Test, and Wechsler Adult Intelligence Scale-Fourth Edition was as efficacious (AUC = .86) as using all 10 EVIs together. Failing any two of these six EVIs or any three of the 10 EVIs yielded clinically acceptable specificity (≥90%) with moderate sensitivity (60%). Findings support the use of multivariate models to improve the identification of performance invalidity in ADHD evaluations, but chaining multiple EVIs may only be helpful to an extent.
Collapse
Affiliation(s)
- John-Christopher A Finley
- Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School, Chicago, IL, USA
| | - Julia M Brooks
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychology, University of Illinois at Chicago, Chicago, IL, USA
| | - Amanda N Nili
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Medical Social Sciences, Northwestern University Feinberg School, Chicago, IL, USA
| | - Alison Oh
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychology, Illinois Institute of Technology Chicago, IL, USA
| | - Hannah B VanLandingham
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Psychology, Illinois Institute of Technology Chicago, IL, USA
| | - Gabriel P Ovsiew
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Devin M Ulrich
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Zachary J Resch
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jason R Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
- Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
2
|
Ferreira I, Fernandes S, Querido L, Pires R, Daugherty JC, Hidalgo-Ruzzante N, Pérez-García M. Validation of the Coin in Hand-Extended Version Among Older Adults With and Without Dementia. Arch Clin Neuropsychol 2021; 37:91-103. [PMID: 33856437 DOI: 10.1093/arclin/acab019] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/20/2021] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE The current study aimed to validate the Performance Validity Test Coin in Hand-Extended Version (CIH-EV) in groups of healthy older adults and older adults with dementia. METHOD Using an analog simulation paradigm, the healthy control group and the clinical group were instructed to perform to the best of their ability, whereas the feigning older adults were instructed to simulate a memory deficit to obtain allowance, financial aid, or early retirement. RESULTS Results showed that the control and clinical groups performed more optimally than the feigning group, although the clinical group had superior response times. The CIH-EV was insensitive to sociodemographic variables and neurocognitive functioning in all groups, demonstrated good convergent validity with other performance validity measures, and showed a reduced rate of false positives. CONCLUSIONS This study corroborates the CIH-EV's effectiveness in detecting the simulation of cognitive deficits in healthy older adults and older adults with dementia.
Collapse
Affiliation(s)
- Inês Ferreira
- Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal
| | - Sandra Fernandes
- Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal.,CICPSI, Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal
| | - Luís Querido
- Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal
| | - Rute Pires
- Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal.,CICPSI, Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, Lisboa, Portugal
| | - Julia C Daugherty
- The Mind, Brain and Behaviour Research Centre (CIMCYC), University of Granada, Granada, Spain.,Department of Personality, Evaluation, and Psychological Treatment, University of Granada, Granada, Spain
| | - Natalia Hidalgo-Ruzzante
- The Mind, Brain and Behaviour Research Centre (CIMCYC), University of Granada, Granada, Spain.,Department of Evolutionary and School Psychology, Universidad de Granada, Granada, Spain
| | - Miguel Pérez-García
- The Mind, Brain and Behaviour Research Centre (CIMCYC), University of Granada, Granada, Spain.,Department of Personality, Evaluation, and Psychological Treatment, University of Granada, Granada, Spain
| |
Collapse
|
3
|
McWhirter L, Ritchie CW, Stone J, Carson A. Performance validity test failure in clinical populations-a systematic review. J Neurol Neurosurg Psychiatry 2020; 91:945-952. [PMID: 32651247 DOI: 10.1136/jnnp-2020-323776] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/08/2020] [Revised: 06/03/2020] [Accepted: 06/07/2020] [Indexed: 12/27/2022]
Abstract
Performance validity tests (PVTs) are widely used in attempts to quantify effort and/or detect negative response bias during neuropsychological testing. However, it can be challenging to interpret the meaning of poor PVT performance in a clinical context. Compensation-seeking populations predominate in the PVT literature. We aimed to establish base rates of PVT failure in clinical populations without known external motivation to underperform. We searched MEDLINE, EMBASE and PsycINFO for studies reporting PVT failure rates in adults with defined clinical diagnoses, excluding studies of active or veteran military personnel, forensic populations or studies of participants known to be litigating or seeking disability benefits. Results were summarised by diagnostic group and implications discussed. Our review identified 69 studies, and 45 different PVTs or indices, in clinical populations with intellectual disability, degenerative brain disease, brain injury, psychiatric disorders, functional disorders and epilepsy. Various pass/fail cut-off scores were described. PVT failure was common in all clinical groups described, with failure rates for some groups and tests exceeding 25%. PVT failure is common across a range of clinical conditions, even in the absence of obvious incentive to underperform. Failure rates are no higher in functional disorders than in other clinical conditions. As PVT failure indicates invalidity of other attempted neuropsychological tests, the finding of frequent and unexpected failure in a range of clinical conditions raises important questions about the degree of objectivity afforded to neuropsychological tests in clinical practice and research.
Collapse
Affiliation(s)
- Laura McWhirter
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Craig W Ritchie
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jon Stone
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Alan Carson
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
4
|
Soble JR, Alverson WA, Phillips JI, Critchfield EA, Fullen C, O’Rourke JJF, Messerly J, Highsmith JM, Bailey KC, Webber TA, Marceaux JC. Strength in Numbers or Quality over Quantity? Examining the Importance of Criterion Measure Selection to Define Validity Groups in Performance Validity Test (PVT) Research. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-019-09370-w] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
5
|
Bodner T, Merten T, Benke T. Performance validity measures in clinical patients with aphasia. J Clin Exp Neuropsychol 2019; 41:476-483. [DOI: 10.1080/13803395.2019.1579783] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Thomas Bodner
- Department of Neurology, Medical University of Innsbruck, Innsbruck, Austria
| | - Thomas Merten
- Department of Neurology, Vivantes Klinikum im Friedrichshain, Berlin, Germany
| | - Thomas Benke
- Department of Neurology, Medical University of Innsbruck, Innsbruck, Austria
| |
Collapse
|