Development and evaluation of a self-administered on-line test of memory and attention for middle-aged and older adults

Brain health assessment. An exploratory review of tools related to its cognitive dimension

Background

Brain health is an evolving concept and relates to physical and mental health, social well-being, productivity, creativity. Brain health has several dimensions (cognitive, motor, functional, social, and emotional), and should be recognized as one top global priorities of health policies. The purpose of this paper is to provide a summary of tools developed for assessing the cognitive dimension of brain health in the out-patient services.

Methods

A literature search on PubMed was performed (from inception to May 31, 2023). We identified cognitive tests, functional and psychological scales, and focused on screening tools specifically proposed to characterize cognition within the construct of brain health, comparing them with common global screening tests.

Results

Among 1947 records, we identified 17 cognitive screening tools used in the context of brain health assessment, of which four were ad hoc developed: Brain Health Assessment (BHA), Brain Health Test (BHT), Brain Health Test-7 (BHT-7), and The Cogniciti Brain Health Assessment. The four tests have administration time ranging from 4 to 30 min, and different administration methods (paper-and-pencil or tablet-based). All four tools assess memory and other cognitive domains. Specific cut-offs have been identified for BHT and BHT-7, while the other tools have automated scoring systems. All but one test also assess other dimensions. Compared to commonly used cognitive screening tests, the brain health tools are less widely used, translated, and validated.

Conclusions

The concept of brain health is new and requires further validation of tools for its assessment, especially for the cognition dimension.

Web-based cognitive assessment in older adults: Where do we stand?

PURPOSE OF REVIEW

The use of digital tools for remote cognitive measurement of older adults is generating increasing interest due to the numerous advantages offered for accessibility and scalability. However, these tools also pose distinctive challenges, necessitating a thorough analysis of their psychometric properties, feasibility and acceptability.

RECENT FINDINGS

In this narrative review, we present the recent literature on the use of web-based cognitive assessment to characterize cognition in older adults and to contribute to the diagnosis of age-related neurodegenerative diseases. We present and discuss three types of web-based cognitive assessments: conventional cognitive tests administered through videoconferencing; unsupervised web-based assessments conducted on a computer; and unsupervised web-based assessments performed on smartphones.

SUMMARY

There have been considerable progress documenting the properties, strengths and limitations of web-based cognitive assessments. For the three types of assessments reported here, the findings support their promising potential for older adults. However, certain aspects, such as the construct validity of these tools and the development of robust norms, remain less well documented. Nonetheless, the beneficial potential of these tools, and their current validation and feasibility data, justify their application [see Supplementary Digital Content (SDC), http://links.lww.com/CONR/A69 ].

NREM sleep as a novel protective cognitive reserve factor in the face of Alzheimer's disease pathology

BACKGROUND

Alzheimer's disease (AD) pathology impairs cognitive function. Yet some individuals with high amounts of AD pathology suffer marked memory impairment, while others with the same degree of pathology burden show little impairment. Why is this? One proposed explanation is cognitive reserve i.e., factors that confer resilience against, or compensation for the effects of AD pathology. Deep NREM slow wave sleep (SWS) is recognized to enhance functions of learning and memory in healthy older adults. However, that the quality of NREM SWS (NREM slow wave activity, SWA) represents a novel cognitive reserve factor in older adults with AD pathology, thereby providing compensation against memory dysfunction otherwise caused by high AD pathology burden, remains unknown.

METHODS

Here, we tested this hypothesis in cognitively normal older adults (N = 62) by combining 11C-PiB (Pittsburgh compound B) positron emission tomography (PET) scanning for the quantification of β-amyloid (Aβ) with sleep electroencephalography (EEG) recordings to quantify NREM SWA and a hippocampal-dependent face-name learning task.

RESULTS

We demonstrated that NREM SWA significantly moderates the effect of Aβ status on memory function. Specifically, NREM SWA selectively supported superior memory function in individuals suffering high Aβ burden, i.e., those most in need of cognitive reserve (B = 2.694, p = 0.019). In contrast, those without significant Aβ pathological burden, and thus without the same  need for cognitive reserve, did not similarly benefit from the presence of NREM SWA (B = -0.115, p = 0.876). This interaction between NREM SWA and Aβ status predicting memory function was significant after correcting for age, sex, Body Mass Index, gray matter atrophy, and previously identified cognitive reserve factors, such as education and physical activity (p = 0.042).

CONCLUSIONS

These findings indicate that NREM SWA is a novel cognitive reserve factor providing resilience against the memory impairment otherwise caused by high AD pathology burden. Furthermore, this cognitive reserve function of NREM SWA remained significant when accounting both for covariates, and factors previously linked to resilience, suggesting that sleep might be an independent cognitive reserve resource. Beyond such mechanistic insights are potential therapeutic implications. Unlike many other cognitive reserve factors (e.g., years of education, prior job complexity), sleep is a modifiable factor. As such, it represents an intervention possibility that may aid the preservation of cognitive function in the face of AD pathology, both present moment and longitudinally.

Reliability, Validity, and User-Experience of Remote Unsupervised Computerized Neuropsychological Assessments in Community-Living 55- to 75-Year-Olds

BACKGROUND

Self-administered computerized neuropsychological assessments (CNAs) provide lower cost, more accessible alternatives to traditional in-person assessments but lack critical information on psychometrics and subjective experience of older adults in remote testing environments.

OBJECTIVE

We used an online brief battery of computerized tasks selected from the Cogstate Brief Battery (CBB) and Cambridge Brain Sciences (CBS) to 1) determine test-retest reliability in an unsupervised setting; 2) examine convergent validity with a comprehensive 'gold standard' paper-and-pencil neuropsychological test battery administered in-person; and 3) explore user-experience of remote computerized testing and individual tests.

METHODS

Fifty-two participants (mean age 65.8±5.7 years) completed CBB and CBS tests on their own computer, unsupervised from home, on three occasions, and visited a research center for an in-person paper-and-pencil assessment. They also completed a user-experience questionnaire.

RESULTS

Test-retest reliabilities varied for individual measures (ICCs = 0.20 to 0.83). Global cognition composites showed excellent reliability (ICCs > 0.8 over 1-month follow-up). A strong relationship between a combination of CNA measures and paper-and-pencil battery was found (canonical correlation R = 0.87, p = 0.04). Most tests were rated as enjoyable with easy-to-understand instructions. Ratings of general experience with online testing were mostly favorable; few had difficulty concentrating (17%) or using the computer for tasks (10%), although over one-third experienced performance anxiety (38%).

CONCLUSION

A combined brief online battery selected from two CNAs demonstrated robust psychometric standards for reliability (global composite), and convergent validity with a gold standard battery, and mostly good usability and acceptability in the remote testing environment.

The adverse effect of modifiable dementia risk factors on cognition amplifies across the adult lifespan

Background

Reversible lifestyle behaviors (modifiable risk factors) can reduce dementia risk by 40%, but their prevalence and association with cognition throughout the adult lifespan is less well understood.

Methods

The associations between the number of modifiable risk factors for dementia (low education, hypertension, hearing loss, traumatic brain injury, alcohol or substance abuse, diabetes, smoking, and depression) and cognition were examined in an online sample (N = 22,117, ages 18–89).

Findings

Older adults (ages 66–89) had more risk factors than middle‐aged (ages 45–65) and younger adults (ages 18–44). Polynomial regression revealed that each additional risk factor was associated with lower cognitive performance (equivalent to 3 years of aging), with a larger association as age increased. People with no risk factors in their forties to seventies showed similar cognitive performance to people 10 or 20 years younger with many risk factors.

Interpretation

Modifiable dementia risk factors amplify lifespan age differences in cognitive performance.

Sex differences and modifiable dementia risk factors synergistically influence memory over the adult lifespan

Introduction

More women than men develop Alzheimer's disease, yet women perform better and show less decline on episodic memory measures, a contradiction that may be accounted for by modifiable risk factors for dementia.

Methods

Associations among age, sex, modifiable dementia risk factors, and cognition were measured in a cross‐sectional online sample (n = 21,840, ages 18 to 89).

Results

Across four tests of associative memory and executive functions, only a Face‐Name Association task revealed sex differences in associative memory that varied by age. Men had worse memory than women (the equivalent of performing similar to someone 4 years older) across ages. Men had larger age differences than women (ie, worse memory in older ages) among people with no to one risk factor, but not those with multiple risk factors.

Discussion

Because the relationship between dementia risk factors and age‐related memory differences varies between men and women, sex‐specific dementia prevention approaches are warranted.

The Impact of Memory Change on Everyday Life Among Older Adults: Association with Cognition and Self-Reported Memory

OBJECTIVES

Many older adults experience memory changes that can have a meaningful impact on their everyday lives, such as restrictions to lifestyle activities and negative emotions. Older adults also report a variety of positive coping responses that help them manage these changes. The purpose of this study was to determine how objective cognitive performance and self-reported memory are related to the everyday impact of memory change.

METHODS

We examined these associations in a sample of 94 older adults (age 60-89, 52% female) along a cognitive ability continuum from normal cognition to mild cognitive impairment.

RESULTS

Correlational analyses revealed that greater restrictions to lifestyle activities (|rs| = .36-.66), more negative emotion associated with memory change (|rs| = .27-.76), and an overall greater burden of memory change on everyday living (|rs| = .28-.61) were associated with poorer objective memory performance and lower self-reported memory ability and satisfaction. Performance on objective measures of executive attention was unrelated to the impact of memory change. Self-reported strategy use was positively related to positive coping with memory change (|r| = .26), but self-reported strategy use was associated with more negative emotions regarding memory change (|r| = .23).

CONCLUSIONS

Given the prevalence of memory complaints among older adults, it is important to understand the experience of memory change and its impact on everyday functioning in order to develop services that target the specific needs of this population.

Interindividual and intraindividual variability in amnestic mild cognitive impairment (aMCI) measured with an online cognitive assessment

INTRODUCTION

Mean cognitive performance is worse in amnestic mild cognitive impairment (aMCI) compared to control groups. However, studies on variability of cognitive performance in aMCI have yielded inconclusive results, with many differences in variability measures and samples from one study to another.

METHODS

We examined variability in aMCI using an existing older adult sample (n = 91; 51 with aMCI, 40 with normal cognition for age), measured with an online self-administered computerized cognitive assessment (Cogniciti's Brain Health Assessment). Our methodology extended past findings by using pure measures of variability (controlling for confounding effects of group performance or practice), and a clinically representative aMCI sample (reflecting the continuum of cognitive performance between normal cognition and aMCI).

RESULTS

Between-group t-tests showed significantly greater between-person variability (interindividual variability or diversity) in overall cognitive performance in aMCI than controls, although the effect size was with a small to moderate effect size, d = 0.44. No significant group differences were found in within-person variability (intraindividual variability) across cognitive tasks (dispersion) or across trials of a response time task (inconsistency), which may be because we used a sample measuring the continuum of cognitive performance. Exploratory correlation analyses showed that a worse overall score was associated with greater inter- and intraindividual variability, and that variability measures were correlated with each other, indicating people with worse cognitive performance were more variable.

DISCUSSION

The current study demonstrates that self-administered online tests can be used to remotely assess different types of variability in people at risk of Alzheimer`s. Our findings show small but significantly more interindividual differences in people with aMCI. This diversity is considered as "noise" in standard assessments of mean performance, but offers an interesting and cognitively informative "signal" in itself.

Accuracy of a Self-Administered Online Cognitive Assessment in Detecting Amnestic Mild Cognitive Impairment

Objectives

Our aim was to validate the online Brain Health Assessment (BHA) for detection of amnestic mild cognitive impairment (aMCI) compared to gold-standard neuropsychological assessment. We compared the diagnostic accuracy of the BHA to the Montreal Cognitive Assessment (MoCA).

Methods

Using a cross-sectional design, community-dwelling older adults completed a neuropsychological assessment, were diagnosed as normal cognition (NC) or aMCI, and completed the BHA and MoCA. Both logistic regression (LR) and penalized logistic regression (PLR) analyses determined BHA and demographic variables predicting aMCI; MoCA variables were similarly modeled. Diagnostic accuracy was compared using area under the receiver operating characteristic curve (ROC AUC) analyses.

Results

Ninety-one participants met inclusion criteria (51 aMCI, 40 NC). PLR modeling for the BHA indicated Face–Name Association, Spatial Working Memory, and age-predicted aMCI (ROC AUC = 0.76; 95% confidence interval [CI]: 0.66–0.86). Optimal cut-points resulted in 21% classified as aMCI (positive), 23% negative, and 56% inconclusive. For the MoCA, digits, abstraction, delayed recall, orientation, and age predicted aMCI (ROC AUC = 0.71; 95% CI: 0.61–0.82). Optimal cut-points resulted in 22% classified positive, 8% negative, and 70% inconclusive (LR results presented within). The BHA model classified fewer participants into the inconclusive category and more as negative for aMCI, compared to the MoCA model (Stuart–Maxwell p = .004).

Discussion

The self-administered BHA provides similar detection of aMCI as a clinician-administered screener (MoCA), with fewer participants classified inconclusively. The BHA has the potential to save practitioners time and decrease unnecessary referrals for a comprehensive assessment to determine the presence of aMCI.

When I'm 64: Age-related variability in over 40,000 online cognitive test takers

Objectives

Age-related differences in cognition are typically assessed by comparing groups of older to younger participants, but little is known about the continuous trajectory of cognitive changes across age, or when a shift to older adulthood occurs. We examined the pattern of mean age differences and variability on episodic memory and executive function measures over the adult life span, in a more fine-grained way than past group or life-span comparisons.

Method

We used a sample of over 40,000 people aged 18–90 who completed psychometrically validated online tests measuring episodic memory and executive functions (the Cogniciti Brain Health Assessment).

Results

Cognitive performance declined gradually over adulthood, and rapidly later in life on spatial working memory, processing speed, facilitation (but not interference), associative recognition, and set shifting. Both polynomial and segmented regression fit the data well, indicating a nonlinear pattern. Segmented regression revealed a shift from gradual to rapid decline that occurred in the early 60s. Variability between people (interindividual variability or diversity) and variability within a person across tasks (intraindividual variability or dispersion) also increased gradually until the 60s, and rapidly after. Confirmatory factor analysis revealed a single general factor (of variance shared between tasks) offered a good fit for performance across tasks.

Discussion

Life-span cognitive performance shows a nonlinear pattern, with gradual decline over early and mid-adulthood, followed by a transition in the 60s to notably accelerated, but more variable, decline. Some people show less decline than others, and some cognitive abilities show less within-person decline than others.

An evaluation of memory and attention in BRCA mutation carriers using an online cognitive assessment tool

BACKGROUND

The objective of this study was to evaluate the impact of various surgical, hormonal, and lifestyle factors on memory and attention in women with a BRCA1 or BRCA2 mutation.

METHODS

BRCA mutation carriers enrolled in a longitudinal study were invited to complete an online brain health assessment tool designed to screen for cognitive deficits. Four measures of memory and executive attention were assessed individually, and an overall score was compiled adjusting for age. Exposures, including preventive surgery, hormone use, and lifestyle factors, were captured by questionnaire. Performance on each of the 5 subtasks was analyzed according to various exposures. Analysis of covariance was used to compare overall scores.

RESULTS

In total, 880 women completed the online cognitive assessment. The average age of the participants was 54 years (range, 23-86 years). The mean overall test score was 54.4 (range, 0-93). The individual subtask scores declined with age at test completion (P < .0001) and increased with level of education (P ≤ .01). Women who underwent a preventive oophorectomy had a significantly higher overall score compared with women who did not undergo this surgery (55.5 vs 50.5; P = .01). Reconstructive breast surgery was also associated with a higher overall score (56.5 vs 52.3; P = .005). Chemotherapy and hormone-replacement therapy were not predictive of the overall score.

CONCLUSIONS

These findings are reassuring to high-risk women who undergo early surgical menopause for their cancer predisposition. Further studies are needed to evaluate cognitive function over time when memory deficits become more prevalent.

Tools for App- and Web-Based Self-Testing of Cognitive Impairment: Systematic Search and Evaluation

BACKGROUND

Tools for app- and Web-based self-testing for identification of cognitive impairment are widely available but are of uncertain quality.

OBJECTIVE

The objective of this study was to undertake a scoping review of app- and Web-based self-tests for cognitive impairment and determine the validity of these tests.

METHODS

We conducted systematic searches in electronic databases, including Google search, Google Play Store, and iPhone Operating System App Store, using the search terms "Online OR Internet-based AND Memory OR Brain OR Dementia OR mild cognitive impairment OR MCI AND Test OR Screen OR Check."

RESULTS

We identified 3057 tools, of which 25 were included in the review. Most tools meeting the inclusion criteria assessed multiple cognitive domains. The most frequently assessed domains were memory, attention, and executive function. We then conducted an electronic survey with the developers of the tools to identify data relating to development and validation of each tool. If no response to the survey was received, Google (to identify gray literature), Google Scholar, and Medical Literature Analysis and Retrieval System Online were searched using key terms "(name of developer, if available)" AND "(the name of the tool)" to identify any additional data. Only 7 tools had any information concerning psychometric quality, and only 1 tool reported data on performance norms, reliability, validity, sensitivity, and specificity for the detection of cognitive impairment.

CONCLUSIONS

The number of cognitive self-assessment electronic health tools for cognitive impairment is increasing, but most are of uncertain quality. There is a need for well-validated tools and guidance for users concerning which tools provide reliable information about possible cognitive impairment that could warrant further investigation.

Validation of a computerized Hong Kong - vigilance and memory test (HK-VMT) to detect early cognitive impairment in healthy older adults

Objectives: Hong Kong - Vigilance and Memory Test (HK-VMT) is developed to distinguish early cognitive impairment in the pre-symptomatic phase from normal cognitive ageing in older adults. The objectives were to validate HK-VMT to differentiate mild cognitive impairment (MCI) and healthy control (HC), and to explore the cut-off scores for different educational levels.Method: A total of 606 older adults underwent the HK-VMT and conventional cognitive tests. HK-VMT is a 15 minutes cognitive battery that assesses episodic memory, attention, and visuospatial ability. The HK-VMT total is the sum of accuracy of all subtests with a range of 0 to 40. Differences in socio-demographic and clinical characteristics between groups were explored. Receiver operating characteristic (ROC) analyses were used to compare HK-VMT and Cantonese Mini Mental State Examination (CMMSE). A sample of 50 participants repeated the HK-VMT in 1 month to evaluate test-retest reliability.Results: ROC analysis of Area Under Curve (AUC) demonstrated that HK-VMT (AUC 0.793) was comparable to CMMSE (AUC 0.748) in differentiating MCI from HC in a matched sample. A cutoff at 21/22 was chosen yielding a sensitivity of 86.1% and a specificity of 75.3% for differentiating MCI and HC. Test-retest reliability of HK-VMT total was 0.71 (p<.001) in a month time.Conclusion: HK-VMT has demonstrated satisfactory validity in detecting cognitive impairment with good test-retest reliability in local older adults. It also performed favourably in the highly educated group when compared to CMMSE.

Online Self-Administered Cognitive Testing Using the Amsterdam Cognition Scan: Establishing Psychometric Properties and Normative Data

Background

Online tests enable efficient self-administered assessments and consequently facilitate large-scale data collection for many fields of research. The Amsterdam Cognition Scan is a new online neuropsychological test battery that measures a broad variety of cognitive functions.

Objective

The aims of this study were to evaluate the psychometric properties of the Amsterdam Cognition Scan and to establish regression-based normative data.

Methods

The Amsterdam Cognition Scan was self-administrated twice from home—with an interval of 6 weeks—by 248 healthy Dutch-speaking adults aged 18 to 81 years.

Results

Test-retest reliability was moderate to high and comparable with that of equivalent traditional tests (intraclass correlation coefficients: .45 to .80; .83 for the Amsterdam Cognition Scan total score). Multiple regression analyses indicated that (1) participants’ age negatively influenced all (12) cognitive measures, (2) gender was associated with performance on six measures, and (3) education level was positively associated with performance on four measures. In addition, we observed influences of tested computer skills and of self-reported amount of computer use on cognitive performance. Demographic characteristics that proved to influence Amsterdam Cognition Scan test performance were included in regression-based predictive formulas to establish demographically adjusted normative data.

Conclusions

Initial results from a healthy adult sample indicate that the Amsterdam Cognition Scan has high usability and can give reliable measures of various generic cognitive ability areas. For future use, the influence of computer skills and experience should be further studied, and for repeated measurements, computer configuration should be consistent. The reported normative data allow for initial interpretation of Amsterdam Cognition Scan performances.

Test-retest reliability and validity of a custom-designed computerized neuropsychological cognitive test battery in young healthy adults

OBJECTIVE

Dual-task methodologies are utilized to probe attentional resource sharing between motor and cognitive systems. Computerized neuropsychological testing is an advanced approach for cognitive assessment and its application in dual task testing is evolving. This study aimed to establish the test-retest reliability and concurrent validity of a custom-designed, computerized, cognitive test battery.

METHODS

Fifteen healthy young adults were tested for the following domains (and tasks): 1) visuomotor function (Spot and Click, SC), 2) phonemic memory (Category Naming, Cat N) and verbal fluency (Word List Generation, WLG), 3) response inhibition (Color Naming, CN), 4) discriminant decision-making (Unveil the Star, US), 5) visual working memory (Triangle and Letter Tracking, TT and LT), 6) problem solving (Peg Game, PG) and 7) information processing speed (Letter-Number, LN). The reaction time, accuracy, time of completion, total number of responses and total number of errors were used as the outcome variables.

RESULTS

The intraclass correlation coefficient (ICC) was used to determine reliability for all outcome variables and concurrent validity was established with respect to the Delis Kaplan Executive Function System™ (D-KEFS™). Reliability ranged from good to excellent for all seven tasks (ICC>0.65). The Cat.N, WLG and CN showed good correlation and PG task showed moderate correlation with tests of the D-KEFS.

CONCLUSION

Findings indicate that these computerized cognitive tests were both valid and reproducible and therefore can be easily implemented by clinicians for assessing cognition and incorporated for dual-task testing and training.