Preliminary Evidence of Equivalence of Alternate Forms of the ImPACT

Psychometric Properties of Computerized Cognitive Tools and Standard Neuropsychological Tests Used to Assess Sport Concussion: A Systematic Review

Athletic programs are more frequently turning to computerized cognitive tools in order to increase efficiencies in concussion assessment. However, assessment using a traditional neuropsychological test battery may provide a more comprehensive and individualized evaluation. Our goal was to inform sport clinicians of the best practices for concussion assessment through a systematic literature review describing the psychometric properties of standard neuropsychological tests and computerized tools. We conducted our search in relevant databases including Ovid Medline, Web of Science, PsycINFO, and Scopus. Journal articles were included if they evaluated psychometric properties (e.g., reliability, sensitivity) of a cognitive assessment within pure athlete samples (up to 30 days post-injury). Searches yielded 4,758 unique results. Ultimately, 103 articles met inclusion criteria, all of which focused on adolescent or young adult participants. Test-retest reliability estimates ranged from .14 to .93 for computerized tools and .02 to .95 for standard neuropsychological tests, with strongest correlations on processing speed tasks for both modalities, although processing speed tasks were most susceptible to practice effects. Reliability was improved with a 2-factor model (processing speed and memory) and by aggregating multiple baseline exams, yet remained below acceptable limits for some studies. Sensitivity to decreased cognitive performance within 72 h of injury ranged from 45%-93% for computerized tools and 18%-80% for standard neuropsychological test batteries. The method for classifying cognitive decline (normative comparison, reliable change indices, regression-based methods) affected sensitivity estimates. Combining computerized tools and standard neuropsychological tests with the strongest psychometric performance provides the greatest value in clinical assessment. To this end, future studies should evaluate the efficacy of hybrid test batteries comprised of top-performing measures from both modalities.

Reevaluating clinical assessment outcomes after unrestricted return to play following sport-related concussion

PRIMARY OBJECTIVE

The objective of this study was to examine neurocognition, postural control, and symptomology at multiple timepoints following concussion. We hypothesized that collegiate athletes would perform similar to or better than their baseline in terms of each outcome at both timepoints.

RESEARCH DESIGN

This was a retrospective study of 71 collegiate athletes (18.3 ± 0.89 years old; 182.2 ± 10.05 cm; 84.2 ± 20.07 kg) to observe changes in outcomes from a previously established clinical protocol.

METHODS AND PROCEDURES

Participants were administered ImPACT™, the Sensory Organization Test (SOT), and the revised head injury scale (HIS-r) prior to their seasons (baseline); upon reporting symptom-free following concussion (post-injury); and approximately 8-months after return-to-play to establish a new baseline.

MAIN OUTCOMES AND RESULTS

There were no changes in ImPACT scores or HIS-r reporting over time. ImPACT total symptom score (TSS) decreased over time (p = .002, η_p² = 0.08). Significant main effects occurred for the SOT equilibrium score (p < .01, η_p² = 0.34) and Vestibular sensory ratio (p < .001, η_p² = 0.22).

CONCLUSIONS

Our data suggest no decline in neurocognition, balance, or symptom burden approximately eight months post-injury. As clinicians continue to explore "best practices" for concussion management and potential long-term implications of these injuries it is important to monitor outcome measures longitudinally.

Multivariate Base Rates and Concussion Detection: A Comparative Study

Recent research has yielded multivariate base rates (MBRs) of low scores in healthy populations using a widely adopted concussion screening measure, Immediate Postconcussion and Cognitive Testing (ImPACT). However, the extent to which individuals with concussion obtain reliable changes at divergent frequencies relative to healthy individuals is largely unknown. The present study examined whether MBRs of reliable change accurately discriminated between those with and without concussion. This archival review consisted of 129 healthy individuals and 81 individuals with concussion. MBRs of reliable change scores were examined at varying cutoffs and frequencies between those with and without concussion. Composites showed small to medium effect sizes in differentiating between those with and without concussion. MBRs of reliable change scores on ImPACT provided limited discriminative utility in isolation. Computations of posttest probabilities using Bayes' Theorem yielded evidence for incremental gains when utilizing MBRs of reliable change under certain constraints.

Validation of a Concussion Screening Battery for Use in Medical Settings: Predicting Centers for Disease Control Concussion Symptoms in Children and Adolescents

Objective

Effective screening for concussion is increasingly important, and medical professionals play a critical role in diagnostic and return-to-play decisions. However, few well-validated measures are available to assist in those decisions. This study aims to determine whether previously validated measures assessing neurocognitive and neurobehavioral abilities can predict Centers for Disease Control (CDC) concussion symptom endorsement in a sample of child or youth athletes.

Method

Participants were 113 individuals, aged 6–17, representing 29 consecutive cases undergoing a post-concussion evaluation by a pediatric neurologist and 84 consecutive cases completing standardized baseline assessments (i.e., not being evaluated as a follow-up to a concussion). All participants completed the same standardized battery of tests comprised of the Connors’ Continuous Performance Test (CPT 3), the Balance Error Scoring System (BESS), and the NIH 4-Meter Gait Test as well as completing a checklist of CDC concussion symptoms.

Results

Regression analyses indicate that the screening battery explained 33% of the variance (d = 1.4) in concussion symptom endorsement, after controlling for age. The neurocognitive test alone (CPT 3) accounts for 21.5% of the variance (d = 1.05) in symptoms after controlling for age, and the neurobehavioral measures (BESS and NIH 4-Meter Gait) then account for an additional 11.5% variance (accounting for 18.6% variance, d = .96, when entered first). These effect sizes are considered large to very large and reflect a marked increase in predictive validity relative to existing measures commonly used in concussion assessments.

Conclusions

A relatively brief screening battery can function in medical settings to predict significant and substantial variability in CDC concussion symptoms in a pediatric sample.

Investigation into repetitive concussion in sport (RECOS): study protocol of a prospective, exploratory, observational cohort study

INTRODUCTION

Sport-related concussion management remains a diagnostic dilemma to clinicians in all strata of care, coaching staff and players alike. The lack of objective diagnostic and prognostic biomarkers and over-reliance on subjective clinical assessments carries a significant health risk of undiagnosed concussive episodes and early return to play before full recovery increasing the risk of sustaining additional concussion, and leading to long-term sequelae and/or unfavourable outcome.

OBJECTIVE

To identify a set of parameters (neuroimaging with neurophysiological, biological and neuropsychological tests) that may support pitch-side and outpatient clinical decision-making in order to objectively diagnose concussion, determine the severity of injury, guide a safe return to play and identify the potential predictors of the long-term sequelae of concussion.

METHODS AND ANALYSIS

An exploratory, observational, prospective, cohort study recruiting between 2017 and 2020. The participants will have a baseline preseason screening (brain imaging, neuropsychological assessments, serum, urine and saliva sampling). If a screened player later suffers a concussion and/or multiple concussions then he/she will be assessed again with the same protocol within 72 hours, and their baseline data will be used as internal control as well as normative data. Inferential statistical analysis will be performed to determine correlations between biological, imaging techniques and neuropsychological assessments.

ETHICS AND DISSEMINATION

This study was approved by the East of England-Essex Research Ethics Committee on 22 September 2017-REC 17/EE/0275; IRAS 216703. The results of this study will be presented at national and international conferences and submitted for publication in peer reviewed journals.

TRIAL REGISTRATION NUMBER

ISRCTN16974791; Pre-results.

A systematic review of sex differences in concussion outcome: What do we know?

Objective: The purpose of this review was to examine sex differences in concussion, or mild traumatic brain injury (mTBI) outcome, updating previous critical reviews of the literature. Method: Within adult human studies, we reviewed a wide range of concussion outcome variables: prevalence of concussion, injury characteristics, postconcussion symptom trajectories and psychiatric distress, neuropsychological performance, and neuroimaging findings. Sports-related concussion, civilian, and military samples were included in the review. Results: Given the robust concussion literature, there is a relative paucity of research addressing sex differences following concussion. The majority of available studies focused on sports-related concussion, with fewer studies targeting other civilian causes of concussion or military-related concussion in females. Prevalence of concussion was generally reported to be higher in females than males. Although symptom reporting largely showed a pattern for females to report greater overall symptoms than males, examining individual symptoms or symptom clusters resulted in mixed findings between the sexes. Neuropsychological studies generally showed females performing more poorly than males on measures of visual memory following concussion, though this finding was not consistently reported. Conclusion: Research examining sex differences in humans following concussion, in general, is in its infancy, and exploration of sex differences in studies outside of the sports concussion domain is particularly nascent. Given the increased prevalence of concussion and potential higher symptom reporting among women, ongoing research is necessary to better understand the role of biological sex on outcome following concussion. Understanding sex differences has important implications for assessment, management, and treatment of concussion.

A Comparative Meta-Analysis of the Effects of Concussion on a Computerized Neurocognitive Test and Self-Reported Symptoms

CONTEXT

  Meta-analyses examining construct-specific cognitive impairment concurrently with self-reported symptoms postconcussion are sparse.

OBJECTIVE

  To review the literature on the effects of concussion on construct-specific neurocognitive declines and to compare them with self-reported symptoms before 1 week and between 1 and 3 weeks postconcussion.

DATA SOURCES

  Relevant studies in PubMed, CINAHL, and PsycINFO published from January 1, 1999 through November 30, 2015.

STUDY SELECTION

  Studies were included if participants completed the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) before and after concussion and if test performance and Postconcussion Symptom Scale (PCSS) scores were reported at both times.

DATA EXTRACTION

  After reviewing the full texts, we extracted data from 17 studies consisting of 29 independent samples; therefore, this meta-analysis consisted of 1777 unique participants.

DATA SYNTHESIS

  The Hedges g effect size (ES) was estimated. A random-effects or fixed-effects model was used based on heterogeneity findings. When heterogeneity was present, we used meta-regression to assess unexplained between-studies variance. Within the first week of injury, the ESs were small to moderate for cognitive declines, ranging from -0.43 (95% confidence interval [CI] = -0.52, -0.35) to -0.67 (95% CI = -0.77, -0.58), and large for the PCSS score (Hedges g = -0.81; 95% CI = -0.91, -0.71). After 1 week, the ESs for cognitive declines (Hedges g range = -0.25 [95% CI = -0.35, -0.15] to -0.37 [95% CI = -0.55, -0.19]) and PCSS score (Hedges g = -0.38; 95% CI = -0.53, -0.22) were also small. Within 2 weeks of injury, PCSS score and time since injury weakly moderated the cognitive ES.

CONCLUSIONS

  When a neurocognitive test was administered within 1 week of injury, the ES was larger for self-reported symptoms than for ImPACT scores generated at the same session. After 1 week of injury, the ESs for ImPACT and PCSS scores were comparable. If the athlete reports symptoms within 1 week of injury, administering a cognitive test does not appear to offer additional information to the clinician. However, if the athlete does not report symptoms postconcussion, cognitive testing may inform the clinical management of the injury.

Validity of the Immediate Post Concussion Assessment and Cognitive Testing (ImPACT)

BACKGROUND

The immediate post concussion assessment and cognitive testing (ImPACT) is the most widely used concussion assessment tool. Despite its popularity, it is unclear if validation studies for the ImPACT test covered all aspects of validity to support its widespread use in research and clinical practice.

OBJECTIVE

The purpose of this report is to review literature surrounding the validity and the utility of the ImPACT test.

DATA SOURCES AND APPRAISAL

A systematic review of relevant studies in PubMed, CINAHL, and PsycINFO was carried out. Studies were evaluated using the STROBE (strengthening the reporting of observational studies in epidemiology) or the STARD (standards for reporting of diagnostic accuracy) criteria.

RESULTS

The literature search yielded 5968 studies. Sixty-nine studies met the inclusion criteria and were included in the qualitative review. Although the convergent validity of ImPACT was supported, evidence of discriminant and predictive validity, diagnostic accuracy and responsiveness was inconclusive. The utility of the ImPACT test after acute symptom resolution was sparse. The review found many factors influenced the validity and utility of ImPACT scores.

CONCLUSION

Clinicians must consider the benefit of ImPACT testing for their patients on a case-by-case scenario and must take the psychometric properties of the test into account when interpreting results.

Test-retest, retest, and retest: Growth curve models of repeat testing with Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)

Computerized neuropsychological testing has become an important tool in the identification and management of sports-related concussions; however, the psychometric effect of repeat testing has not been studied extensively beyond test-retest statistics. The current study analyzed data from Division I collegiate athletes who completed Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) baseline assessments at four sequential time points that varied over the course of their athletic careers. Administrations were part of a larger National Institutes of Health (NIH) study. Growth curve modeling showed that the two memory composite scores increased significantly with successive administrations: Change in Verbal Memory was best represented with a quadratic model, while a linear model best fit Visual Memory. Visual Motor Speed and Reaction Time composites showed no significant linear or quadratic growth. The results demonstrate the effect of repeated test administrations for memory composite scores, while speed composites were not significantly impacted by repeat testing. Acceptable test-retest reliability was demonstrated for all four composites as well.

Examining Recovery Trajectories After Sport-Related Concussion With a Multimodal Clinical Assessment Approach

BACKGROUND

Previous research estimates that the majority of athletes with sport-related concussion (SRC) will recover between 7 and 10 days after injury. This short temporal window of recovery is based predominately on symptom resolution and cognitive improvement and does not accurately reflect recent advances in the clinical assessment model.

OBJECTIVE

To characterize SRC recovery at 1-week postinjury time intervals on symptom, neurocognitive, and vestibular-oculomotor outcomes and to examine sex differences in SRC recovery time.

METHODS

A prospective, repeated-measures design was used to examine the temporal resolution of neurocognitive, symptom, and vestibular-oculomotor impairment in 66 subjects (age, 16.5 ± 1.9 years; range, 14-23 years; 64% male) with SRC.

RESULTS

Recovery time across all outcomes was between 21 and 28 days after SRC for most athletes. Symptoms demonstrated the greatest improvement in the first 2 weeks, although neurocognitive impairment lingered across various domains up to 28 days after SRC. Vestibular-oculomotor decrements also resolved between 1 and 3 weeks after injury. There were no sex differences in neurocognitive recovery. Male subjects were more likely to be asymptomatic by the fourth week and reported less vestibular-oculomotor impairment than female subjects at weeks 1 and 2.

CONCLUSION

When the recommended "comprehensive" approach is used for concussion assessment, recovery time for SRC is approximately 3 to 4 weeks, which is longer than the commonly reported 7 to 14 days. Sports medicine clinicians should use a variety of complementing assessment tools to capture the heterogeneity of SRC.

Computerized Cognitive Testing in the Management of Youth Sports-Related Concussion

Computerized neurocognitive testing has become a growing practice across medical populations, but particularly within sports medicine and the management of sports-related concussion. Although traditional neuropsychological measures are solely administered and interpreted by neuropsychologists, computerized cognitive tests are marketed to and utilized by a wide range of professionals involved in the management of sports-related concussions, many of whom lack specialized psychometric training. Although the benefits of computerized testing allow for many youth athletes to be evaluated quickly, professionals implementing their use should be aware of the potential pitfalls and the high potential for misuse. After briefly reviewing the recommended guidelines set forth by the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology, we review the benefits/limitations of computerized testing in the management of sports-related concussion and the basic psychometric properties of some of the more widely used computerized measures. Lastly, we discuss the practical application of these devices.

The Acute Management of Sport Concussion in Pediatric Athletes

During the past two decades the focus on sport concussion has increased significantly. Young athletes represent the most vulnerable population to sustain a sport concussion yet receive the least amount of attention. Specifically, young athletes who sustain a sport concussion can go unrecognized and continue to participate in sport putting them at an increased risk for a more significant injury. The purpose of this review is to provide a clinical framework for the evaluation and management of sport concussion. In addition, this review provides considerations for health care professionals in regard to clinical measures and follow-up strategies during the acute phase following concussion in young concussed athletes following injury.

Advanced topics in neuropsychological assessment following sport-related concussion

OBJECTIVE

This study examined seven topics relating to neuropsychological assessment following sport-related concussion: (i) traditional vs. computerized tests; (ii) the value of baseline, pre-season testing; (iii) invalid baseline scores and poor effort; (iv) when to assess following injury; (v) the reliability of neuropsychological tests; (vi) reliable change methodology; and (vii) new methods for identifying cognitive impairment.

MAIN RESULTS

Baseline testing can be helpful for quantifying cognitive deficits following injury and for assessing recovery. At present, however, there is insufficient evidence to conclude that having baseline test results is clearly superior to not having baseline test results. Although invalid baseline test performance can be detected in some athletes, validity indicators cannot determine the extent to which the results were due to deliberately poor performance, confusion or misunderstanding regarding some aspect of the test, distractions in group testing environments or some combination of factors. When interpreting baseline and post-injury data, sophisticated psychometric methods (e.g. reliable change, multivariate base rates) are available to assist with more accurate identification of cognitive impairment and the serial monitoring of recovery.

CONCLUSIONS

The value of neuropsychological assessment in the management of sport-related concussion has a strong empirical foundation. Additional research is needed, however, to refine its use.

Administration and environment considerations in computer-based sports-concussion assessment

Computer-based testing has become a vital tool for the assessment of sport-related concussion (SRC). An increasing number of papers have been published on this topic, focusing on subjects such as the purpose and validity of baseline testing, the performance of special populations on computer-based tests, the psychometric properties of different computerized neurocognitive tools, and considerations for valid and reliable administration of these tools. The current paper describes several considerations regarding computerized test design, input and output devices, and testing environment that should be described explicitly when administering computer-based cognitive testing, regardless of whether the assessment is used for clinical or research purposes. The paper also reviews the conclusions of recent literature (2007-2013) using computer-based testing for the assessment of SRC, with special attention to the methods used in these studies. We also present an appendix checklist for clinicians and researchers that may be helpful in ensuring proper attention to factors that could influence the reliability and validity of computer-based cognitive testing. We believe that explicit attention to these technological factors may lead to the development of standards for the development and implementation of computer-based tests. Such standards have the potential to enhance the accuracy and utility of computer-based tests in SRC.