Machine Learning in Modeling High School Sport Concussion Symptom Resolve

A Machine Learning Approach to Concussion Risk Estimation Among Players Exhibiting Visible Signs in Professional Hockey

BACKGROUND

The identification of concussion risk factors, such as visible signs and mechanisms of injury, improves concussion identification. Exploring individual risk factors, such as concussion history, may help to improve existing concussion risk models and algorithms.

OBJECTIVES

The primary aim of the current study was to use machine learning techniques to develop a comprehensive, prospectively coded concussion risk model in professional hockey among players exhibiting visible signs. The secondary aim was to examine whether including concussion history improves model performance.

METHODS

Data from the National Hockey League (NHL) spotter program, including coded visible signs and mechanisms of injury associated with possible concussive events, were extracted from the 2018-2019 to the 2021-2022 seasons. Each unique spotter event was matched with data extracted from the medical record to determine whether the event was associated with a subsequent physician diagnosed concussion. We compared the ability of three machine learning-based approaches to identify the likelihood of physician diagnosed concussion: conditional inference tree, conditional inference random forest, and logistic regression.

RESULTS

A total of 1563 unique events with visible signs were identified by spotters (183 leading to a concussion diagnosis). A randomly selected training sample had 1250 events (146 concussions) and the remaining set-aside test sample had 313 events (37 concussions). The obtained models performed at a high level with large effects in the training [area under the receiver operating characteristic curve (AUC) = 0.79] and set-aside test data (AUC = 0.82). Concussion history was retained in the tree and logistic regression models, with each additional prior concussion associated with a 1.32 times increased odds of concussion diagnosis.

CONCLUSIONS

We present simple tree and logistic algorithms for concussion screening and as diagnostic aids. Our results show that player concussion history can explain additional risk above and beyond that explained by visible signs and mechanisms of injury alone.

Machine learning methods in physical therapy: A scoping review of applications in clinical context

BACKGROUND

Machine learning (ML) efficiently processes large datasets, showing promise in enhancing clinical practice within physical therapy.

OBJECTIVE

The aim of this scoping review is to provide an overview of studies using ML approaches in clinical settings of physical therapy.

DATA SOURCES

A scoping review was performed in PubMed, EMBASE, PEDro, Cochrane, Web of Science, and Scopus.

SELECTION CRITERIA

We included studies utilizing ML methods. ML was defined as the utilization of computational systems to encode patterns and relationships, enabling predictions or classifications with minimal human interference.

DATA EXTRACTION AND DATA SYNTHESIS

Data were extracted regarding methods, data types, performance metrics, and model availability.

RESULTS

Forty-two studies were included. The majority were published after 2020 (n = 25). Fourteen studies (33.3%) were in the musculoskeletal physical therapy field, nine (21.4%) in neurological, and eight (19%) in sports physical therapy. We identified 44 different ML models, with random forest being the most used. Three studies reported on model availability. We identified several clinical applications for ML-based tools, including diagnosis (n = 14), prognosis (n = 7), treatment outcomes prediction (n = 7), clinical decision support (n = 5), movement analysis (n = 4), patient monitoring (n = 3), and personalized care plan (n = 2).

LIMITATION

Model performance metrics, costs, model interpretability, and explainability were not reported.

CONCLUSION

This scope review mapped the emerging landscape of machine learning applications in physical therapy. Despite the growing interest, the field still lacks high-quality studies on validation, model availability, and acceptability to advance from research to clinical practice.

Application of Artificial Intelligence in the Headache Field

PURPOSE OF REVIEW

Headache disorders are highly prevalent worldwide. Rapidly advancing capabilities in artificial intelligence (AI) have expanded headache-related research with the potential to solve unmet needs in the headache field. We provide an overview of AI in headache research in this article.

RECENT FINDINGS

We briefly introduce machine learning models and commonly used evaluation metrics. We then review studies that have utilized AI in the field to advance diagnostic accuracy and classification, predict treatment responses, gather insights from various data sources, and forecast migraine attacks. Furthermore, given the emergence of ChatGPT, a type of large language model (LLM), and the popularity it has gained, we also discuss how LLMs could be used to advance the field. Finally, we discuss the potential pitfalls, bias, and future directions of employing AI in headache medicine. Many recent studies on headache medicine incorporated machine learning, generative AI and LLMs. A comprehensive understanding of potential pitfalls and biases is crucial to using these novel techniques with minimum harm. When used appropriately, AI has the potential to revolutionize headache medicine.

Assessment of Sports Concussion in Female Athletes: A Role for Neuroinformatics?

Over the past decade, the intricacies of sports-related concussions among female athletes have become readily apparent. Traditional clinical methods for diagnosing concussions suffer limitations when applied to female athletes, often failing to capture subtle changes in brain structure and function. Advanced neuroinformatics techniques and machine learning models have become invaluable assets in this endeavor. While these technologies have been extensively employed in understanding concussion in male athletes, there remains a significant gap in our comprehension of their effectiveness for female athletes. With its remarkable data analysis capacity, machine learning offers a promising avenue to bridge this deficit. By harnessing the power of machine learning, researchers can link observed phenotypic neuroimaging data to sex-specific biological mechanisms, unraveling the mysteries of concussions in female athletes. Furthermore, embedding methods within machine learning enable examining brain architecture and its alterations beyond the conventional anatomical reference frame. In turn, allows researchers to gain deeper insights into the dynamics of concussions, treatment responses, and recovery processes. This paper endeavors to address the crucial issue of sex differences in multimodal neuroimaging experimental design and machine learning approaches within female athlete populations, ultimately ensuring that they receive the tailored care they require when facing the challenges of concussions. Through better data integration, feature identification, knowledge representation, validation, etc., neuroinformaticists, are ideally suited to bring clarity, context, and explainabilty to the study of sports-related head injuries in males and in females, and helping to define recovery.

Using machine learning to discover traumatic brain injury patient phenotypes: national concussion surveillance system Pilot

OBJECTIVE

The objective is to determine whether unsupervised machine learning identifies traumatic brain injury (TBI) phenotypes with unique clinical profiles.

METHODS

Pilot self-reported survey data of over 10,000 adults were collected from the Centers for Disease Control and Prevention (CDC)'s National Concussion Surveillance System (NCSS). Respondents who self-reported a head injury in the past 12 months (n = 1,364) were retained and queried for injury, outcome, and clinical characteristics. An unsupervised machine learning algorithm, partitioning around medoids (PAM), that employed Gower's dissimilarity matrix, was used to conduct a cluster analysis.

RESULTS

PAM grouped respondents into five TBI clusters (phenotypes A-E). Phenotype C represented more clinically severe TBIs with a higher prevalence of symptoms and association with worse outcomes. When compared to individuals in Phenotype A, a group with few TBI-related symptoms, individuals in Phenotype C were more likely to undergo medical evaluation (odds ratio [OR] = 9.8, 95% confidence interval[CI] = 5.8-16.6), have symptoms that were not currently resolved or resolved in 8+ days (OR = 10.6, 95%CI = 6.2-18.1), and more likely to report at least moderate impact on social (OR = 54.7, 95%CI = 22.4-133.4) and work (OR = 25.4, 95%CI = 11.2-57.2) functioning.

CONCLUSION

Machine learning can be used to classify patients into unique TBI phenotypes. Further research might examine the utility of such classifications in supporting clinical diagnosis and patient recovery for this complex health condition.

Characterizing the Network Structure of Post-Concussion Symptoms

OBJECTIVE

Assessment of post-concussion symptoms is implemented at secondary, post-secondary, and professional levels of athletics. Network theory suggests that disorders can be viewed as a set of interacting symptoms that amplify, reinforce, and maintain one another. Examining the network structure of post-concussion symptoms may provide new insights into symptom comorbidity and may inform targeted treatment. We used network analysis to examine the topology of post-concussion symptoms using the Post-Concussion Symptom Scale (PCSS) in high school athletes with recent suspected sport-related concussion.

METHOD

Using a cross-sectional design, the network was estimated from Post Concussion Symptom Scale scores from 3,292 high school athletes, where nodes represented symptoms and edges represented the association between symptoms. Node centrality was calculated to determine the relative importance of each symptom in the network.

RESULTS

The network consisted of edges within and across symptom domains. "Difficulty concentrating" and "dizziness" were the most central symptoms in the network. Although not highly central in the network, headaches were the highest rated symptom.

CONCLUSIONS

The interconnectedness among symptoms supports the notion that post-concussion symptoms are interrelated and mutually reinforcing. Given their central role in the network, "difficulty concentrating" and "dizziness" are expected to affect the activation and persistence of other post-concussion symptoms. Interventions targeting difficulties with concentration and dizziness may help alleviate other symptoms. Our findings could inform the development of targeted treatment with the aim of reducing overall symptom burden. Future research should examine the trajectory of post-concussion symptom networks to advance the clinical understanding of post-concussive recovery.

Optimizing Concussion Care Seeking: Identification of Factors Predicting Previous Concussion Diagnosis Status

PURPOSE

There is limited understanding of factors affecting concussion diagnosis status using large sample sizes. The study objective was to identify factors that can accurately classify previous concussion diagnosis status among collegiate student-athletes and service academy cadets with concussion history.

METHODS

This retrospective study used support vector machine, Gaussian Naïve Bayes, and decision tree machine learning techniques to identify individual (e.g., sex) and institutional (e.g., academic caliber) factors that accurately classify previous concussion diagnosis status (all diagnosed vs 1+ undiagnosed) among Concussion Assessment, Research, and Education Consortium participants with concussion histories ( n = 7714).

RESULTS

Across all classifiers, the factors examined enable >50% classification between previous diagnosed and undiagnosed concussion histories. However, across 20-fold cross validation, ROC-AUC accuracy averaged between 56% and 65% using all factors. Similar performance is achieved considering individual risk factors alone. By contrast, classifications with institutional risk factors typically did not distinguish between those with all concussions diagnosed versus 1+ undiagnosed; average performances using only institutional risk factors were almost always <58%, including confidence intervals for many groups <50%. Participants with more extensive concussion histories were more commonly classified as having one or more of those previous concussions undiagnosed.

CONCLUSIONS

Although the current study provides preliminary evidence about factors to help classify concussion diagnosis status, more work is needed given the tested models' accuracy. Future work should include a broader set of theoretically indicated factors, at levels ranging from individual behavioral determinants to features of the setting in which the individual was injured.

Greater Acute Concussion Symptoms Are Associated With Longer Recovery Times in Adolescents

We examined the association between the severity of acute concussion symptoms and time to return to school and to sports in adolescent student athletes. We hypothesized that there would be an association between the severity of acute symptoms experienced in the first 72 hours and functional recovery such that adolescents with the lowest burden of acute symptoms would have the fastest return to school and sports and those with the highest burden of symptoms would have the slowest return to school and sports. This injury surveillance cohort included 375 adolescent student athletes aged 14-19 years who sustained a sport-related concussion between 2014 and 2020. Athletic trainers documented time to return to school and to sports. A greater proportion of adolescents with the highest acute symptoms remained out of school at 3 (odds ratio [OR] = 2.5, 95% confidence interval [CI] 1.5-4.4), 5 (OR = 2.4, 95% CI 1.4-4.0), 7 (OR = 2.6, 95% CI 1.5-4.3), and 10 days (OR = 2.3, 95% CI 1.3-3.9) compared to those with the lowest acute symptoms. Similarly, a greater proportion of athletes with the highest acute symptoms remained out of sports at 7 (OR = 3.5, 95% CI 1.5-8.1), 10 (OR = 3.1, 95% CI 1.8-5.6), 14 (OR = 1.8, 95% CI 1.1-3.0), and 21 days (OR = 1.9, 95% CI 1.0-3.6) compared to those with the lowest acute symptoms. This study underscores the adverse effect of high acute symptom burden following concussion on return to school and to sports among adolescent student athletes. Conversely, student athletes with a low burden of acute symptoms have a faster return to school and to sports.

Effect of Concussion on Reaction Time and Neurocognitive Factors: Implications for Subsequent Lower Extremity Injury

Background

Recent evidence has demonstrated that athletes are at greater risk for a lower extremity injury following a return-to-sport (RTS) after sport-related concussion (SRC). The reason for this is not completely clear, but it has been hypothesized that persistent deficits in neurocognitive factors may be a contributing factor.

Hypothesis/Purpose

This study assessed simple reaction time, processing speed, attention, and concentration in a group of athletes, post-concussion upon clearance for RTS for potential deficits that may result in slower reaction time, processing speed, attention, and concentration. The researchers hypothesized that the concussion group would demonstrate worse scores on both assessments compared to a sex-, age-, and sport-matched cohort.

Study Design

Case-controlled study.

Methods

Twelve participants who had suffered a SRC and eight healthy individuals who were matched to the concussed group by age, sex, and sport were evaluated. Those with a concussion had been cleared for RTS by a licensed healthcare provider. Each participant underwent neurocognitive tests that included a simple reaction time test (SRT) and the King-Devick Test (K-D). Independent t-tests were performed to compare the groups with significance set a priori at p<0.05.

Results

There was a significant difference (p =0.024) between groups for SRT with the concussed group demonstrating a better SRT than the control group. There were no significant differences (p =0.939) between the groups for the K-D.

Conclusion

With no significant differences between groups in the K-D assessment and, surprisingly, the concussed group having a better SRT compared to the healthy group, our hypothesis was not supported.

Clinical Relevance

These specific measures, compounded with extensive post-concussion time lapse until RTS clearance, may have limited capacity in revealing potential persistent deficits in relevant neurocognitive characteristics.

Level of Evidence

Level of Evidence 3.

Safeguarding Athletes Against Head Injuries Through Advances in Technology: A Scoping Review of the Uses of Machine Learning in the Management of Sports-Related Concussion

Sports injury prevention is an important part of the athlete welfare and safeguarding research field. In sports injury prevention, sport-related concussion (SRC) has proved to be one of the most difficult and complex injuries to manage in terms of prevention, diagnosis, classification, treatment and rehabilitation. SRC can cause long-term health issues and is a commonly reported injury in both adult and youth athletes around the world. Despite increased knowledge of the prevalence of SRC, very few tools are available for diagnosing SRC in athletic settings. Recent technological innovations have resulted in different machine learning and deep learning methodologies being tested to improve the management of this complex sports injury. The purpose of this article is to summarize and map the existing research literature on the use of machine learning in the management of SRC, ascertain where there are gaps in the existing research and identify recommendations for future research. This is explored through a scoping review. A systematic search in the three electronic databases SPORTDiscus, PubMed and Scopus identified an initial 522 studies, of which 24 were included in the final review, the majority of which focused on machine learning for the prediction and prevention of SRC (N = 10), or machine learning for the diagnosis and classification of SRC (N = 11). Only 3 studies explored machine learning approaches for the treatment and rehabilitation of SRC. A main finding is that current research highlights promising practical uses (e.g., more accurate and rapid injury assessment or return-to-sport participation criteria) of machine learning in the management of SRC. The review also revealed a narrow research focus in the existing literature. As current research is primarily conducted on male adolescents or adults from team sports in North America there is an urgent need to include wider demographics in more diverse samples and sports contexts in the machine learning algorithms. If research datasets continue to be based on narrow samples of athletes, the development of any new diagnostic and predictive tools for SRC emerging from this research will be at risk. Today, these risks appear to mainly affect the health and safety of female athletes.

Overtraining Syndrome as a Complex Systems Phenomenon

The phenomenon of reduced athletic performance following sustained, intense training (Overtraining Syndrome, and OTS) was first recognized more than 90 years ago. Although hundreds of scientific publications have focused on OTS, a definitive diagnosis, reliable biomarkers, and effective treatments remain unknown. The present review considers existing models of OTS, acknowledges the individualized and sport-specific nature of signs/symptoms, describes potential interacting predisposing factors, and proposes that OTS will be most effectively characterized and evaluated via the underlying complex biological systems. Complex systems in nature are not aptly characterized or successfully analyzed using the classic scientific method (i.e., simplifying complex problems into single variables in a search for cause-and-effect) because they result from myriad (often non-linear) concomitant interactions of multiple determinants. Thus, this review 1) proposes that OTS be viewed from the perspectives of complex systems and network physiology, 2) advocates for and recommends that techniques such as trans-omic analyses and machine learning be widely employed, and 3) proposes evidence-based areas for future OTS investigations, including concomitant multi-domain analyses incorporating brain neural networks, dysfunction of hypothalamic-pituitary-adrenal responses to training stress, the intestinal microbiota, immune factors, and low energy availability. Such an inclusive and modern approach will measurably help in prevention and management of OTS.

Machine Learning for Subtyping Concussion Using a Clustering Approach

Background: Concussion subtypes are typically organized into commonly affected symptom areas or a combination of affected systems, an approach that may be flawed by bias in conceptualization or the inherent limitations of interdisciplinary expertise.

Objective: The purpose of this study was to determine whether a bottom-up, unsupervised, machine learning approach, could more accurately support concussion subtyping.

Methods: Initial patient intake data as well as objective outcome measures including, the Patient-Reported Outcomes Measurement Information System (PROMIS), Dizziness Handicap Inventory (DHI), Pain Catastrophizing Scale (PCS), and Immediate Post-Concussion Assessment and Cognitive Testing Tool (ImPACT) were retrospectively extracted from the Advance Concussion Clinic's database. A correlation matrix and principal component analysis (PCA) were used to reduce the dimensionality of the dataset. Sklearn's agglomerative clustering algorithm was then applied, and the optimal number of clusters within the patient database were generated. Between-group comparisons among the formed clusters were performed using a Mann-Whitney U test.

Results: Two hundred seventy-five patients within the clinics database were analyzed. Five distinct clusters emerged from the data when maximizing the Silhouette score (0.36) and minimizing the Davies-Bouldin score (0.83). Concussion subtypes derived demonstrated clinically distinct profiles, with statistically significant differences (p < 0.05) between all five clusters.

Conclusion: This machine learning approach enabled the identification and characterization of five distinct concussion subtypes, which were best understood according to levels of complexity, ranging from Extremely Complex to Minimally Complex. Understanding concussion in terms of Complexity with the utilization of artificial intelligence, could provide a more accurate concussion classification or subtype approach; one that better reflects the true heterogeneity and complex system disruptions associated with mild traumatic brain injury.

Prolonged concussion effects: Constellations of cognitive deficits detected up to year after injury

Concussions are associated with an array of physical, emotional, cognitive, and sleep symptoms at multiple timescales. Cognitive recovery occurs relatively quickly – five-to-seven days on average. Yet, recent evidence suggests that some neurophysiological changes can be identified one year after a concussion. To that end, we examine more nuanced patterns in cognitive tests to determine whether cognitive abilities could identify a concussion within one-year post injury. A radial-basis (non-linear boundary) support vector machine classifier was trained to use cognitive performance measures to distinguish participants with no prior concussion from participants with prior concussion in the past year. After incorporating only 10 cognitive measures, or all 5 composite measures from the neurocognitive assessment (Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)), over 90% accuracy was achieved in identifying both participants without prior concussions and participants with concussions in the past year, particularly when relying on non-linear patterns. Notably, classification accuracy stayed relatively constant between participants who had a concussion early or late in the one-year window. Thus, with substantial accuracy, a prior concussion can be identified using a non-linear combination of cognitive measures. Cognitive effects from concussion linger one-year post-injury, indicating the importance of continuing to follow concussion patients for many months after recovery and to take special note of constellations of cognitive abilities.

A systematic review of the effectiveness of machine learning for predicting psychosocial outcomes in acquired brain injury: Which algorithms are used and why?

Clinicians working in the field of acquired brain injury (ABI, an injury to the brain sustained after birth) are challenged to develop suitable care pathways for an individual client's needs. Being able to predict psychosocial outcomes after ABI would enable clinicians and service providers to make advance decisions and better tailor care plans. Machine learning (ML, a predictive method from the field of artificial intelligence) is increasingly used for predicting ABI outcomes. This review aimed to examine the efficacy of using ML to make psychosocial predictions in ABI, evaluate the methodological quality of studies, and understand researchers' rationale for their choice of ML algorithms. Nine studies were reviewed from five databases, predicting a range of psychosocial outcomes from stroke, traumatic brain injury, and concussion. Eleven types of ML were employed with a total of 75 ML models. Every model was evaluated as having high risk of bias, unable to provide adequate evidence for predictive performance due to poor methodological quality. Overall, there was limited rationale for the choice of ML algorithms and poor evaluation of the methodological limitations by study authors. Considerations for overcoming methodological shortcomings are discussed, along with suggestions for assessing the suitability of data and suitability of ML algorithms for different ABI research questions.

Predictive Accuracy of the Sport Concussion Assessment Tool 3 and Vestibular/Ocular-Motor Screening, Individually and In Combination: A National Collegiate Athletic Association-Department of Defense Concussion Assessment, Research and Education Consortium Analysis

BACKGROUND

Vestibular and ocular symptoms in sport-related concussions are common. The Vestibular/Ocular-Motor Screening (VOMS) tool is a rapid, free, pen-and-paper tool that directly assesses these symptoms and shows consistent utility in concussion identification, prognosis, and management. However, a VOMS validation study in the acute concussion period of a large sample is lacking.

PURPOSE

To examine VOMS validity among collegiate student-athletes, concussed and nonconcussed, from the multisite National Collegiate Athletic Association-Department of Defense Concussion Assessment, Research and Education (CARE) Consortium. A secondary aim was to utilize multidimensional machine learning pattern classifiers to deduce the additive power of the VOMS in relation to components of the Sport Concussion Assessment Tool 3 (SCAT3).

STUDY DESIGN

Cohort study (diagnosis); Level of evidence, 3.

METHODS

Preseason and acute concussion assessments were analyzed for 419 student-athletes. Variables in the analysis included the VOMS, Balance Error Scoring System, Standardized Assessment of Concussion, and SCAT3 symptom evaluation score. Descriptive statistics were calculated for all tools, including Kolmogorov-Smirnov significance and Cohen d effect size. Correlations between tools were analyzed with Spearman r, and predictive accuracy was evaluated through an Ada Boosted Tree machine learning model's generated receiver operating characteristic curves.

RESULTS

Total VOMS scores and SCAT3 symptom scores demonstrated significant increases in the acute concussion time frame (Cohen d = 1.23 and 1.06; P < .0001), whereas the Balance Error Scoring System lacked clinical significance (Cohen d = 0.17). Incorporation of VOMS into the full SCAT3 significantly boosted overall diagnostic ability by 4.4% to an area under the curve of 0.848 (P < .0001) and produced a 9% improvement in test sensitivity over the existing SCAT3 battery.

CONCLUSION

The results from this study highlight the relevance of the vestibular and oculomotor systems to concussion and the utility of the VOMS tool. Given the 3.8 million sports-related and 45,121 military-related concussions per year, the addition of VOMS to the SCAT3 is poised to identify up to an additional 304,000 athletes and 3610 servicemembers annually who are concussed, thereby improving concussion assessment and diagnostic rates. Health care providers should consider the addition of VOMS to their concussion assessment toolkits, as its use can positively affect assessment and management of concussions, which may ultimately improve outcomes for this complex and common injury.

Use of artificial intelligence in sports medicine: a report of 5 fictional cases

BACKGROUND

Artificial intelligence (AI) is one of the most promising areas in medicine with many possibilities for improving health and wellness. Already today, diagnostic decision support systems may help patients to estimate the severity of their complaints. This fictional case study aimed to test the diagnostic potential of an AI algorithm for common sports injuries and pathologies.

METHODS

Based on a literature review and clinical expert experience, five fictional "common" cases of acute, and subacute injuries or chronic sport-related pathologies were created: Concussion, ankle sprain, muscle pain, chronic knee instability (after ACL rupture) and tennis elbow. The symptoms of these cases were entered into a freely available chatbot-guided AI app and its diagnoses were compared to the pre-defined injuries and pathologies.

RESULTS

A mean of 25-36 questions were asked by the app per patient, with optional explanations of certain questions or illustrative photos on demand. It was stressed, that the symptom analysis would not replace a doctor's consultation. A 23-yr-old male patient case with a mild concussion was correctly diagnosed. An ankle sprain of a 27-yr-old female without ligament or bony lesions was also detected and an ER visit was suggested. Muscle pain in the thigh of a 19-yr-old male was correctly diagnosed. In the case of a 26-yr-old male with chronic ACL instability, the algorithm did not sufficiently cover the chronic aspect of the pathology, but the given recommendation of seeing a doctor would have helped the patient. Finally, the condition of the chronic epicondylitis in a 41-yr-old male was correctly detected.

CONCLUSIONS

All chosen injuries and pathologies were either correctly diagnosed or at least tagged with the right advice of when it is urgent for seeking a medical specialist. However, the quality of AI-based results could presumably depend on the data-driven experience of these programs as well as on the understanding of their users. Further studies should compare existing AI programs and their diagnostic accuracy for medical injuries and pathologies.

Episodic-Memory Performance in Machine Learning Modeling for Predicting Cognitive Health Status Classification

BACKGROUND

Memory dysfunction is characteristic of aging and often attributed to Alzheimer's disease (AD). An easily administered tool for preliminary assessment of memory function and early AD detection would be integral in improving patient management.

OBJECTIVE

Our primary aim was to utilize machine learning in determining initial viable models to serve as complementary instruments in demonstrating efficacy of the MemTrax online Continuous Recognition Tasks (M-CRT) test for episodic-memory screening and assessing cognitive impairment.

METHODS

We used an existing dataset subset (n = 18,395) of demographic information, general health screening questions (addressing memory, sleep quality, medications, and medical conditions affecting thinking), and test results from a convenience sample of adults who took the M-CRT test. M-CRT performance and participant features were used as independent attributes: true positive/negative, percent responses/correct, response time, age, sex, and recent alcohol consumption. For predictive modeling, we used demographic information and test scores to predict binary classification of the health-related questions (yes/no) and general health status (healthy/unhealthy), based on the screening questions.

RESULTS

ANOVA revealed significant differences among HealthQScore groups for response time true positive (p = 0.000) and true positive (p = 0.020), but none for true negative (p = 0.0551). Both % responses and % correct had significant differences (p = 0.026 and p = 0.037, respectively). Logistic regression was generally the top-performing learner with moderately robust prediction performance (AUC) for HealthQScore (0.648-0.680) and selected general health questions (0.713-0.769).

CONCLUSION

Our novel application of supervised machine learning and predictive modeling helps to demonstrate and validate cross-sectional utility of MemTrax in assessing early-stage cognitive impairment and general screening for AD.

Utility of MemTrax and Machine Learning Modeling in Classification of Mild Cognitive Impairment

Background:

The widespread incidence and prevalence of Alzheimer’s disease and mild cognitive impairment (MCI) has prompted an urgent call for research to validate early detection cognitive screening and assessment.

Objective:

Our primary research aim was to determine if selected MemTrax performance metrics and relevant demographics and health profile characteristics can be effectively utilized in predictive models developed with machine learning to classify cognitive health (normal versus MCI), as would be indicated by the Montreal Cognitive Assessment (MoCA).

Methods:

We conducted a cross-sectional study on 259 neurology, memory clinic, and internal medicine adult patients recruited from two hospitals in China. Each patient was given the Chinese-language MoCA and self-administered the continuous recognition MemTrax online episodic memory test on the same day. Predictive classification models were built using machine learning with 10-fold cross validation, and model performance was measured using Area Under the Receiver Operating Characteristic Curve (AUC). Models were built using two MemTrax performance metrics (percent correct, response time), along with the eight common demographic and personal history features.

Results:

Comparing the learners across selected combinations of MoCA scores and thresholds, Naïve Bayes was generally the top-performing learner with an overall classification performance of 0.9093. Further, among the top three learners, MemTrax-based classification performance overall was superior using just the top-ranked four features (0.9119) compared to using all 10 common features (0.8999).

Conclusion:

MemTrax performance can be effectively utilized in a machine learning classification predictive model screening application for detecting early stage cognitive impairment.