Introducing the ImPACT-5: An Empirically Derived Multivariate Validity Composite

The prevalence and correlates of scores falling below ImPACT embedded validity indicators among adolescent student athletes

Objective: Valid performance on preseason baseline neurocognitive testing is essential for accurate comparison between preseason and post-concussion test results. Immediate Post-Concussion and Cognitive Testing (ImPACT) is commonly used to measure baseline neurocognitive function in athletes. We examined the prevalence of invalid performance on ImPACT baseline testing and identified correlates of invalid performance. Method: The sample included 66,998 adolescents (ages 14-18, M = 15.51 years, SD = 1.22) who completed ImPACT baseline tests between 2009 and 2019. Invalid performance was determined by the embedded validity indicators (EVI). Associations between invalid performance, demographic characteristics, and health conditions were assessed using chi-square tests and odds ratios (ORs). Results: Overall, 7.2% of adolescents had baseline tests identified as invalid by one or more of the EVIs. Individual validity indicators classified between 0.5% and 3.7% tests as invalid. Higher frequencies of invalid scores were observed among youth with neurodevelopmental, academic, and medical conditions. Youth who reported having learning disabilities (n = 3126), receiving special education (n = 3563), or problems with attention-deficit/hyperactivity disorder (ADHD; n = 5104) obtained invalid baselines at frequencies of 16.4%, 16.0%, and 11.1%, respectively. Moreover, youth who reported receiving treatment for a substance use disorder (n = 311) or epilepsy (n = 718) obtained invalid baselines at frequencies of 17.0% and 11.1%, respectively. Conclusions: The base rate of invalid performance on ImPACT's EVIs was approximately 7%, consistent with prior research. Adolescents self-reporting neurodevelopmental conditions, academic difficulties, or a history of treatment for medical conditions obtained invalid baseline tests at higher frequencies. More research is needed to better understand invalid scores in youth with pre-existing conditions.

Prevalence of invalid ImPACT baseline test results among high school athletes with invalidity and sandbagging indices

Objective: The comparison of baseline neuropsychological test results with post-concussion neuropsychological test results is a common practice to assess the aftermath of a sport-related concussion. The effectiveness of this approach is compromised when invalid baseline test performances occur. The present study was designed to assess the prevalence of invalid baseline test results of high school athletes, using the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT). Method: A large sample of 8938 high school athletes' baseline test data were examined, including the four ImPACT Composite scores and three sets of embedded invalid indices: five ImPACT Invalidity Indicators, four ImPACT sandbagging red flags, and two ImPACT sandbagging subtest flags. Results: There were 8394 (93.91%) Valid scorers, who failed no Invalidity Indicators; 544 (6.09%) Invalid scorers, who failed at least one Invalidity Indicator, 2718 (30.41%) Red Flag scorers, who had at least one Red Flag score; and 4154 (46.47%) Subtest Flag scorers, who had at least one Subtest Flag score. Of the entire sample, 4485 (54.65%) failed at least one of the invalid indices. Conclusions: As in prior studies of high school athletes, the Invalidity Indicators identified a low portion (6.09%) of the athletes with invalid baseline test results, while other invalidity measures found 30.41% to 46.48% of the athletes producing questionable test results. The high rate of invalid test findings suggested in this research calls for greater efforts to improve the accuracy of baseline test results.

An updated review of the prevalence of invalid performance on the Immediate Post-Concussion and Cognitive Testing (ImPACT)

Objective: Performance validity assessment is an important component of concussion baseline testing and Immediate Post-Concussion and Cognitive Testing (ImPACT) is the most commonly used test in this setting. A review of invalid performance on ImPACT was published in 2017, focusing largely on the default embedded validity indicator (Default EVI) provided within the test. There has since been a proliferation in research evaluating the classification accuracy of the Default EVI against independently developed, alternative ImPACT-based EVIs, necessitating an updated review. The purpose of this study was to provide an up-to-date review of the prevalence of invalid performance on ImPACT and to examine the relative effectiveness of ImPACT-based EVIs. Method: Literature related to the prevalence of invalid performance on ImPACT and the effectiveness of ImPACT-based EVIs, published between January 2000 and May 2020, was critically reviewed. Results: A total of 23 studies reported prevalence of invalid performance at baseline testing using ImPACT. Six percent of baseline assessments were found to be invalid by the ImPACT's Default EVI, and between 22.31% and 34.99% were flagged by alternative EVIs. Six studies assessed the effectiveness of ImPACT-based EVIs, with the Default EVI correctly identifying experimental malingerers only 60% of the time. Alternative ImPACT-based EVIs identified between 73% and 100% of experimental malingerers. Conclusions: The ImPACT's Default EVI is not sufficiently sensitive, and clinicians should consider alternative indicators when assessing invalid performance. Accordingly, the base rate of invalid performance in athletes at baseline testing is likely well above the 6% previously reported.

Physiological Vibration Acceleration (Phybrata) Sensor Assessment of Multi-System Physiological Impairments and Sensory Reweighting Following Concussion

Objective

To assess the utility of a head-mounted wearable inertial motion unit (IMU)-based physiological vibration acceleration (“phybrata”) sensor to support the clinical diagnosis of concussion, classify and quantify specific concussion-induced physiological system impairments and sensory reweighting, and track individual patient recovery trajectories.

Methods

Data were analyzed from 175 patients over a 12-month period at three clinical sites. Comprehensive clinical concussion assessments were first completed for all patients, followed by testing with the phybrata sensor. Phybrata time series data and spatial scatter plots, eyes open (Eo) and eyes closed (Ec) phybrata powers, average power (Eo+Ec)/2, Ec/Eo phybrata power ratio, time-resolved phybrata spectral density (TRPSD) distributions, and receiver operating characteristic (ROC) curves are compared for individuals with no objective impairments and those clinically diagnosed with concussions and accompanying vestibular impairment, other neurological impairment, or both vestibular and neurological impairments. Finally, pre- and post-injury phybrata case report results are presented for a participant who was diagnosed with a concussion and subsequently monitored during treatment, rehabilitation, and return-to-activity clearance.

Results

Phybrata data demonstrate distinct features and patterns for individuals with no discernable clinical impairments, diagnosed vestibular pathology, and diagnosed neurological pathology. ROC curves indicate that the average power (Eo+Ec)/2 may be utilized to support clinical diagnosis of concussion, while Eo and Ec/Eo may be utilized as independent measures to confirm accompanying neurological and vestibular impairments, respectively. All 3 measures demonstrate area under the curve (AUC), sensitivity, and specificity above 90% for their respective diagnoses. Phybrata spectral analyses demonstrate utility for quantifying the severity of concussion-induced physiological impairments, sensory reweighting, and subsequent monitoring of improvements throughout treatment and rehabilitation.

Conclusion

Phybrata testing assists with objective concussion diagnosis and provides an important adjunct to standard concussion assessment tools by objectively ascertaining neurological and vestibular impairments, guiding targeted rehabilitation strategies, monitoring recovery, and assisting with return-to-sport/work/learn decision-making.