Episodic-Memory Performance in Machine Learning Modeling for Predicting Cognitive Health Status Classification

Episodic memory assessment: effects of sex and age on performance and response time during a continuous recognition task

Introduction

Continuous recognition tasks (CRTs) assess episodic memory (EM), the central functional disturbance in Alzheimer's disease and several related disorders. The online MemTrax computerized CRT provides a platform for screening and assessment that is engaging and can be repeated frequently. MemTrax presents complex visual stimuli, which require complex involvement of the lateral and medial temporal lobes and can be completed in less than 2 min. Results include number of correct recognitions (HITs), recognition failures (MISSes = 1-HITs), correct rejections (CRs), false alarms (FAs = 1-CRs), total correct (TC = HITs + CRs), and response times (RTs) for each HIT and FA. Prior analyses of MemTrax CRT data show no effects of sex but an effect of age on performance. The number of HITs corresponds to faster RT-HITs more closely than TC, and CRs do not relate to RT-HITs. RT-HITs show a typical skewed distribution, and cumulative RT-HITs fit a negative survival curve (RevEx). Thus, this study aimed to define precisely the effects of sex and age on HITS, CRs, RT-HITs, and the dynamics of RTs in an engaged population.

Methods

MemTrax CRT online data on 18,255 individuals was analyzed for sex, age, and distributions of HITs, CRs, MISSes, FAs, TC, and relationships to both RT-HITs and RT-FAs.

Results

HITs corresponded more closely to RT-HITs than did TC because CRs did not relate to RT-HITs. RT-FAs had a broader distribution than RT-HITs and were faster than RT-HITs in about half of the sample, slower in the other half. Performance metrics for men and women were the same. HITs declined with age as RT-HITs increased. CRs also decreased with age and RT-FAs increased, but with no correlation. The group over aged 50 years had RT-HITs distributions slower than under 50 years. For both age ranges, the RevEx model explained more than 99% of the variance in RT-HITs.

Discussion

The dichotomy of HITs and CRs suggests opposing cognitive strategies: (1) less certainty about recognitions, in association with slower RT-HITs and lower HIT percentages suggests recognition difficulty, leading to more MISSes, and (2) decreased CRs (more FAs) but faster RTs to HITs and FAs, suggesting overly quick decisions leading to errors. MemTrax CRT performance provides an indication of EM (HITs and RT-HITs may relate to function of the temporal lobe), executive function (FAs may relate to function of the frontal lobe), processing speed (RTs), cognitive ability, and age-related changes. This CRT provides potential clinical screening utility for early Alzheimer's disease and other conditions affecting EM, other cognitive functions, and more accurate impairment assessment to track changes over time.

The MemTrax memory test for detecting and assessing cognitive impairment in Parkinson's disease

INTRODUCTION

A valid, reliable, accessible measurement for the early detection of cognitive decline in patients with Parkinson's disease (PD) is in urgent demand. The objective of the study is to assess the clinical utility of the MemTrax Memory Test in detecting cognitive impairment in patients with PD.

METHODS

The MemTrax, a fast on-line cognitive screening tool based on continuous recognition task, and Montreal Cognitive Assessment (MoCA) were administered to 61 healthy controls (HC), 102 PD patients with normal cognition (PD-N), 74 PD patients with mild cognitive impairment (PD-MCI) and 52 PD patients with dementia (PD-D). The total percent correct (MTx- %C), average response time (MTx-RT), composite score (MTx-Cp) of MemTrax and the MoCA scores were comparatively analyzed.

RESULTS

The MoCA scores were similar between HC and PD-N, however, MTx- %C and MTx-Cp were lower in PD-N than HC(p < 0.05). MTx- %C, MTx-Cp and the MoCA scores were significantly lower in PD-MCI versus PD-N and in PD-D versus PD-MCI (p ≤ 0.001), while MTx-RT was statistically longer in PD-D versus PD-MCI (p ≤ 0.001). For PD groups, the MemTrax performance correlated with the MoCA scores. To detect PD-MCI, the optimal MTx- %C and MTx-Cp cutoff were 75 % and 50.0, respectively. To detect PD-D, the optimal MTx- %C, MTx-RT and MTx-Cp cutoff were 69 %, 1.341s and 40.6, respectively.

CONCLUSION

The MemTrax provides rapid, valid and reliable metrics for assessing cognition in PD patients which could be useful for identifying PD-MCI at early stage and monitoring cognitive function decline during the progression of disease.

Correctness and response time distributions in the MemTrax continuous recognition task: Analysis of strategies and a reverse-exponential model

A critical issue in addressing medical conditions is measurement. Memory measurement is difficult, especially episodic memory, which is disrupted by many conditions. On-line computer testing can precisely measure and assess several memory functions. This study analyzed memory performances from a large group of anonymous, on-line participants using a continuous recognition task (CRT) implemented at https://memtrax.com. These analyses estimated ranges of acceptable performance and average response time (RT). For 344,165 presumed unique individuals completing the CRT a total of 602,272 times, data were stored on a server, including each correct response (HIT), Correct Rejection, and RT to the thousandth of a second. Responses were analyzed, distributions and relationships of these parameters were ascertained, and mean RTs were determined for each participant across the population. From 322,996 valid first tests, analysis of correctness showed that 63% of these tests achieved at least 45 correct (90%), 92% scored at or above 40 correct (80%), and 3% scored 35 correct (70%) or less. The distribution of RTs was skewed with 1% faster than 0.62 s, a median at 0.890 s, and 1% slower than 1.57 s. The RT distribution was best explained by a novel model, the reverse-exponential (RevEx) function. Increased RT speed was most closely associated with increased HIT accuracy. The MemTrax on-line memory test readily provides valid and reliable metrics for assessing individual episodic memory function that could have practical clinical utility for precise assessment of memory dysfunction in many conditions, including improvement or deterioration over time.

Now is the Time to Improve Cognitive Screening and Assessment for Clinical and Research Advancement

Alzheimer’s disease (AD) is the only cause of death ranked in the top ten globally without precise early diagnosis or effective means of prevention or treatment. Further, AD was identified as a pandemic [1] well before COVID-19 was dubbed a 21st century pandemic [2]. And now, with the realization of the prominent secondary impacts of pandemics, there is a growing, widespread recognition of the tremendous magnitude of the impending burden from AD in an aging world population in the coming decades [3]. This appreciation has amplified the growing and pressing need for a new, efficacious, and practical platform to detect and track cognitive decline, beginning in the preliminary (prodromal) phases of the disease, sensitively, accurately, effectively, reliably, efficiently, and remotely [4–7]. Moreover, the parallel necessity of clarifying and understanding risk factors, developing successful prevention strategies [8–17], and discovering and monitoring viable and effective treatments could all benefit from accurate and efficient screening and assessment platforms. Modern recognition of AD [18] as a common affliction of the elderly began in 1968 with a paper by Blessed, Tomlinson, & Roth [19] in which two tests, one a brief assessment of cognitive function and the other a measure of daily function, demonstrated impairment which was associated with the postmortem counts of neurofibrillary tangles, composed mainly of microtubule-associated protein-tau (tau), in the brain, though not to senile plaques, composed mainly of amyloid-β (Aβ). Even in more recent analyses, the tangles correspond with the severity of dementia more than the plaques [20, 21]. Since 1960, a plethora of cognitive tests, paper and pencil [22, 23], simple screening models [24], and computerized [25–27], have been developed to assess the dysfunction associated with AD. However, there has been limited application of Modern Test Theory, which includes Item Characteristic Curve Analysis, used in the technological development of such tools [28–31], along with widespread failure to understand the underlying AD pathological process to guide test development [32, 33]. The lack of such development has likely been a major contributor to the failure of the field to develop timely screening approaches for AD [34, 35], inaccurate assessment of the progression of AD [36], and even now, failure to find an effective approach to stopping AD.

Accuracy of Support-Vector Machines for Diagnosis of Alzheimer's Disease, Using Volume of Brain Obtained by Structural MRI at Siriraj Hospital

Background: The determination of brain volumes using visual ratings is associated with an inherently low accuracy for the diagnosis of Alzheimer's disease (AD). A support-vector machine (SVM) is one of the machine learning techniques, which may be utilized as a classifier for various classification problems. This study exploratorily investigated the accuracy of SVM classification models for AD subjects using brain volume and various clinical data as features.

Methods: The study was designed as a retrospective chart review. A total of 201 eligible subjects were recruited from the Memory Clinic at Siriraj Hospital, Thailand. Eighteen cases were excluded due to incomplete MRI data. Subjects were randomly assigned to a training group (AD = 46, normal = 46) and testing group (AD = 45, normal = 46) for SVM modeling and validation, respectively. The results in terms of accuracy and a receiver operating characteristic curve analysis are reported.

Results: The highest accuracy for brain volumetry (62.64%) was found using the hippocampus as a single feature. A combination of clinical parameters as features provided accuracy ranging between 83 and 90%. However, a combination of brain volumetry and clinical parameters as features to the SVM models did not improve the accuracy of the result.

Conclusions: In our study, the use of brain volumetry as SVM features provided low classification accuracy with the highest accuracy of 62.64% using the hippocampus volume alone. In contrast, the use of clinical parameters [Thai mental state examination score, controlled oral word association tests (animals; and letters K, S, and P), learning memory, clock-drawing test, and construction-praxis] as features for SVM models provided good accuracy between 83 and 90%.

Validity of the MemTrax Memory Test Compared to the Montreal Cognitive Assessment in the Detection of Mild Cognitive Impairment and Dementia due to Alzheimer's Disease in a Chinese Cohort

BACKGROUND

A valid, reliable, accessible, engaging, and affordable digital cognitive screen instrument for clinical use is in urgent demand.

OBJECTIVE

To assess the clinical utility of the MemTrax memory test for early detection of cognitive impairment in a Chinese cohort.

METHODS

The 2.5-minute MemTrax and the Montreal Cognitive Assessment (MoCA) were performed by 50 clinically diagnosed cognitively normal (CON), 50 mild cognitive impairment due to AD (MCI-AD), and 50 Alzheimer's disease (AD) volunteer participants. The percentage of correct responses (MTx-% C), the mean response time (MTx-RT), and the composite scores (MTx-Cp) of MemTrax and the MoCA scores were comparatively analyzed and receiver operating characteristic (ROC) curves generated.

RESULTS

Multivariate linear regression analyses indicated MTx-% C, MTx-Cp, and the MoCA score were significantly lower in MCI-AD versus CON and in AD versus MCI-AD groups (all with p≤0.001). For the differentiation of MCI-AD from CON, an optimized MTx-% C cutoff of 81% had 72% sensitivity and 84% specificity with an area under the curve (AUC) of 0.839, whereas the MoCA score of 23 had 54% sensitivity and 86% specificity with an AUC of 0.740. For the differentiation of AD from MCI-AD, MTx-Cp of 43.0 had 70% sensitivity and 82% specificity with an AUC of 0.799, whereas the MoCA score of 20 had 84% sensitivity and 62% specificity with an AUC of 0.767.

CONCLUSION

MemTrax can effectively detect both clinically diagnosed MCI and AD with better accuracy as compared to the MoCA based on AUCs in a Chinese cohort.

Utility of MemTrax and Machine Learning Modeling in Classification of Mild Cognitive Impairment

Background:

The widespread incidence and prevalence of Alzheimer’s disease and mild cognitive impairment (MCI) has prompted an urgent call for research to validate early detection cognitive screening and assessment.

Objective:

Our primary research aim was to determine if selected MemTrax performance metrics and relevant demographics and health profile characteristics can be effectively utilized in predictive models developed with machine learning to classify cognitive health (normal versus MCI), as would be indicated by the Montreal Cognitive Assessment (MoCA).

Methods:

We conducted a cross-sectional study on 259 neurology, memory clinic, and internal medicine adult patients recruited from two hospitals in China. Each patient was given the Chinese-language MoCA and self-administered the continuous recognition MemTrax online episodic memory test on the same day. Predictive classification models were built using machine learning with 10-fold cross validation, and model performance was measured using Area Under the Receiver Operating Characteristic Curve (AUC). Models were built using two MemTrax performance metrics (percent correct, response time), along with the eight common demographic and personal history features.

Results:

Comparing the learners across selected combinations of MoCA scores and thresholds, Naïve Bayes was generally the top-performing learner with an overall classification performance of 0.9093. Further, among the top three learners, MemTrax-based classification performance overall was superior using just the top-ranked four features (0.9119) compared to using all 10 common features (0.8999).

Conclusion:

MemTrax performance can be effectively utilized in a machine learning classification predictive model screening application for detecting early stage cognitive impairment.