Validity and Cultural Generalisability of a 5-Minute AI-Based, Computerised Cognitive Assessment in Mild Cognitive Impairment and Alzheimer's Dementia

A remote digital memory composite to detect cognitive impairment in memory clinic samples in unsupervised settings using mobile devices

Remote monitoring of cognition holds the promise to facilitate case-finding in clinical care and the individual detection of cognitive impairment in clinical and research settings. In the context of Alzheimer's disease, this is particularly relevant for patients who seek medical advice due to memory problems. Here, we develop a remote digital memory composite (RDMC) score from an unsupervised remote cognitive assessment battery focused on episodic memory and long-term recall and assess its construct validity, retest reliability, and diagnostic accuracy when predicting MCI-grade impairment in a memory clinic sample and healthy controls. A total of 199 participants were recruited from three cohorts and included as healthy controls (n = 97), individuals with subjective cognitive decline (n = 59), or patients with mild cognitive impairment (n = 43). Participants performed cognitive assessments in a fully remote and unsupervised setting via a smartphone app. The derived RDMC score is significantly correlated with the PACC5 score across participants and demonstrates good retest reliability. Diagnostic accuracy for discriminating memory impairment from no impairment is high (cross-validated AUC = 0.83, 95% CI [0.66, 0.99]) with a sensitivity of 0.82 and a specificity of 0.72. Thus, unsupervised remote cognitive assessments implemented in the neotiv digital platform show good discrimination between cognitively impaired and unimpaired individuals, further demonstrating that it is feasible to complement the neuropsychological assessment of episodic memory with unsupervised and remote assessments on mobile devices. This contributes to recent efforts to implement remote assessment of episodic memory for case-finding and monitoring in large research studies and clinical care.

Culture-Fair Cognitive Screening Tools for Assessment of Cognitive Impairment: A Systematic Review

Background

Cognitive screening tools are important in the detection of dementia, including Alzheimer's disease; however, they may contain cultural biases.

Objective

This review examines culture-fair cognitive screening tools and evaluates their screening accuracy, strengths, and limitations.

Methods

Medline, Embase, PsychINFO and CINAHL were searched. The protocol was registered on PROSPERO (CRD42021288776). Included studies used a culture-fair tool to assess cognition in older adults from varying ethnicities. Narrative synthesis was conducted.

Results

28 studies were included assessing eleven different tools. The Rowland Universal Dementia Assessment Scale (RUDAS) was as accurate as the Mini-Mental State Examination (MMSE) (AUC 0.62-0.93), with a similar sensitivity (52-94%) and better specificity (70-98%), and the Multicultural Cognitive Examination (MCE) had improved screening accuracy (AUC 0.99) compared to RUDAS (AUC 0.92). The Visual Cognitive Assessment Test (VCAT) was equivalent to MMSE (AUC 0.84-0.91). The Kimberley Indigenous Cognitive Assessment tool (KICA) had AUC of 0.93-0.95; sensitivity of 90.6%, specificity 92.6%.

Conclusions

The RUDAS, KICA and VCAT were superior to MMSE for screening dementia in ethnic minorities. Other tools also showed good screening accuracy. Further research should be done to validate tools in different populations.

Applications of Artificial Intelligence in the Neuropsychological Assessment of Dementia: A Systematic Review

In the context of advancing healthcare, the diagnosis and treatment of cognitive disorders, particularly Mild Cognitive Impairment (MCI) and Alzheimer's Disease (AD), pose significant challenges. This review explores Artificial Intelligence (AI) and Machine Learning (ML) in neuropsychological assessment for the early detection and personalized treatment of MCI and AD. The review includes 37 articles that demonstrate that AI could be an useful instrument for optimizing diagnostic procedures, predicting cognitive decline, and outperforming traditional tests. Three main categories of applications are identified: (1) combining neuropsychological assessment with clinical data, (2) optimizing existing test batteries using ML techniques, and (3) employing virtual reality and games to overcome the limitations of traditional tests. Despite advancements, the review highlights a gap in developing tools that simplify the clinician's workflow and underscores the need for explainable AI in healthcare decision making. Future studies should bridge the gap between technical performance measures and practical clinical utility to yield accurate results and facilitate clinicians' roles. The successful integration of AI/ML in predicting dementia onset could reduce global healthcare costs and benefit aging societies.

Text Dialogue Analysis for Primary Screening of Mild Cognitive Impairment: Development and Validation Study

BACKGROUND

Artificial intelligence models tailored to diagnose cognitive impairment have shown excellent results. However, it is unclear whether large linguistic models can rival specialized models by text alone.

OBJECTIVE

In this study, we explored the performance of ChatGPT for primary screening of mild cognitive impairment (MCI) and standardized the design steps and components of the prompts.

METHODS

We gathered a total of 174 participants from the DementiaBank screening and classified 70% of them into the training set and 30% of them into the test set. Only text dialogues were kept. Sentences were cleaned using a macro code, followed by a manual check. The prompt consisted of 5 main parts, including character setting, scoring system setting, indicator setting, output setting, and explanatory information setting. Three dimensions of variables from published studies were included: vocabulary (ie, word frequency and word ratio, phrase frequency and phrase ratio, and lexical complexity), syntax and grammar (ie, syntactic complexity and grammatical components), and semantics (ie, semantic density and semantic coherence). We used R 4.3.0. for the analysis of variables and diagnostic indicators.

RESULTS

Three additional indicators related to the severity of MCI were incorporated into the final prompt for the model. These indicators were effective in discriminating between MCI and cognitively normal participants: tip-of-the-tongue phenomenon (P<.001), difficulty with complex ideas (P<.001), and memory issues (P<.001). The final GPT-4 model achieved a sensitivity of 0.8636, a specificity of 0.9487, and an area under the curve of 0.9062 on the training set; on the test set, the sensitivity, specificity, and area under the curve reached 0.7727, 0.8333, and 0.8030, respectively.

CONCLUSIONS

ChatGPT was effective in the primary screening of participants with possible MCI. Improved standardization of prompts by clinicians would also improve the performance of the model. It is important to note that ChatGPT is not a substitute for a clinician making a diagnosis.

Artificial intelligence for dementia research methods optimization

Artificial intelligence (AI) and machine learning (ML) approaches are increasingly being used in dementia research. However, several methodological challenges exist that may limit the insights we can obtain from high-dimensional data and our ability to translate these findings into improved patient outcomes. To improve reproducibility and replicability, researchers should make their well-documented code and modeling pipelines openly available. Data should also be shared where appropriate. To enhance the acceptability of models and AI-enabled systems to users, researchers should prioritize interpretable methods that provide insights into how decisions are generated. Models should be developed using multiple, diverse datasets to improve robustness, generalizability, and reduce potentially harmful bias. To improve clarity and reproducibility, researchers should adhere to reporting guidelines that are co-produced with multiple stakeholders. If these methodological challenges are overcome, AI and ML hold enormous promise for changing the landscape of dementia research and care. HIGHLIGHTS: Machine learning (ML) can improve diagnosis, prevention, and management of dementia. Inadequate reporting of ML procedures affects reproduction/replication of results. ML models built on unrepresentative datasets do not generalize to new datasets. Obligatory metrics for certain model structures and use cases have not been defined. Interpretability and trust in ML predictions are barriers to clinical translation.

Brain Representation of Animal and Non-Animal Images in Patients with Mild Cognitive Impairment and Alzheimer's Disease

Background

In early Alzheimer's disease (AD), high-level visual functions and processing speed are impacted. Few functional magnetic resonance imaging (fMRI) studies have investigated high-level visual deficits in AD, yet none have explored brain activity patterns during rapid animal/non-animal categorization tasks.

Objective

To address this, we utilized the previously known Integrated Cognitive Assessment (ICA) to collect fMRI data and compare healthy controls (HC) to individuals with mild cognitive impairment (MCI) and mild AD.

Methods

The ICA encompasses a rapid visual categorization task that involves distinguishing between animals and non-animals within natural scenes. To comprehensively explore variations in brain activity levels and patterns, we conducted both univariate and multivariate analyses of fMRI data.

Results

The ICA task elicited activation across a range of brain regions, encompassing the temporal, parietal, occipital, and frontal lobes. Univariate analysis, which compared responses to animal versus non-animal stimuli, showed no significant differences in the regions of interest (ROIs) across all groups, with the exception of the left anterior supramarginal gyrus in the HC group. In contrast, multivariate analysis revealed that in both HC and MCI groups, several regions could differentiate between animals and non-animals based on distinct patterns of activity. Notably, such differentiation was absent within the mild AD group.

Conclusions

Our study highlights the ICA task's potential as a valuable cognitive assessment tool designed for MCI and AD. Additionally, our use of fMRI pattern analysis provides valuable insights into the complex changes in brain function associated with AD. This approach holds promise for enhancing our understanding of the disease's progression.

Health economic analysis of the integrated cognitive assessment tool to aid dementia diagnosis in the United Kingdom

Objectives

The aim of this study was to develop a comprehensive economic evaluation of the integrated cognitive assessment (ICA) tool compared with standard cognitive tests when used for dementia screening in primary care and for initial patient triage in memory clinics.

Methods

ICA was compared with standard of care comprising a mixture of cognitive assessment tools over a lifetime horizon and employing the UK health and social care perspective. The model combined a decision tree to capture the initial outcomes of the cognitive testing with a Markov structure that estimated long-term outcomes of people with dementia. Quality of life outcomes were quantified using quality-adjusted life years (QALYs), and the economic benefits were assessed using net monetary benefit (NMB). Both costs and QALYs were discounted at 3.5% per annum and cost-effectiveness was assessed using a threshold of £20,000 per QALY gained.

Results

ICA dominated standard cognitive assessment tools in both the primary care and memory clinic settings. Introduction of the ICA tool was estimated to result in a lifetime cost saving of approximately £123 and £226 per person in primary care and memory clinics, respectively. QALY gains associated with early diagnosis were modest (0.0016 in primary care and 0.0027 in memory clinic). The net monetary benefit (NMB) of ICA introduction was estimated at £154 in primary care and £281 in the memory clinic settings.

Conclusion

Introduction of ICA as a tool to screen primary care patients for dementia and perform initial triage in memory clinics could be cost saving to the UK public health and social care payer.

The use of the integrated cognitive assessment to improve the efficiency of primary care referrals to memory services in the accelerating dementia pathway technologies study

Background

Current primary care cognitive assessment tools are either crude or time-consuming instruments that can only detect cognitive impairment when it is well established. This leads to unnecessary or late referrals to memory services, by which time the disease may have already progressed into more severe stages. Due to the COVID-19 pandemic, some memory services have adapted to the new environment by shifting to remote assessments of patients to meet service user demand. However, the use of remote cognitive assessments has been inconsistent, and there has been little evaluation of the outcome of such a change in clinical practice. Emerging research has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as the leading candidates for adoption in clinical practice. This is true both during the pandemic and in the post-COVID-19 era as part of healthcare innovation.

Objectives

The Accelerating Dementias Pathways Technologies (ADePT) Study was initiated in order to address this challenge and develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment and improving the efficiency of the dementia care pathway.

Methods

Ninety-nine patients aged 55-90 who have been referred to a memory clinic by a general practitioner (GP) were recruited. Participants completed the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome were compared with the specialist diagnosis obtained at the memory clinic.Participants were given the option to carry out a retest visit where they were again given the chance to take the ICA test either remotely or face-to-face.

Results

The primary outcome of the study compared GP referral with specialist diagnosis of mild cognitive impairment (MCI) and dementia. Of those the GP referred to memory clinics, 78% were necessary referrals, with ~22% unnecessary referrals, or patients who should have been referred to other services as they had disorders other than MCI/dementia. In the same population the ICA was able to correctly identify cognitive impairment in ~90% of patients, with approximately 9% of patients being false negatives. From the subset of unnecessary GP referrals, the ICA classified ~72% of those as not having cognitive impairment, suggesting that these unnecessary referrals may not have been made if the ICA was in use. ICA demonstrated a sensitivity of 93% for dementia and 83% for MCI, with a specificity of 80% for both conditions in detecting cognitive impairment. Additionally, the test-retest prediction agreement for the ICA was 87.5%.

Conclusion

The results from this study demonstrate the potential of the ICA as a screening tool, which can be used to support accurate referrals from primary care settings, along with the work conducted in memory clinics and in secondary care. The ICA's sensitivity and specificity in detecting cognitive impairment in MCI surpassed the overall standard of care reported in existing literature.

Artificial Intelligence for Dementia Research Methods Optimization

INTRODUCTION

Machine learning (ML) has been extremely successful in identifying key features from high-dimensional datasets and executing complicated tasks with human expert levels of accuracy or greater.

METHODS

We summarize and critically evaluate current applications of ML in dementia research and highlight directions for future research.

RESULTS

We present an overview of ML algorithms most frequently used in dementia research and highlight future opportunities for the use of ML in clinical practice, experimental medicine, and clinical trials. We discuss issues of reproducibility, replicability and interpretability and how these impact the clinical applicability of dementia research. Finally, we give examples of how state-of-the-art methods, such as transfer learning, multi-task learning, and reinforcement learning, may be applied to overcome these issues and aid the translation of research to clinical practice in the future.

DISCUSSION

ML-based models hold great promise to advance our understanding of the underlying causes and pathological mechanisms of dementia.

A New Smart 2-Min Mobile Alerting Method for Mild Cognitive Impairment Due to Alzheimer's Disease in the Community

The early identification of mild cognitive impairment (MCI) due to Alzheimer's disease (AD), in an early stage of AD can expand the AD warning window. We propose a new capability index evaluating the spatial execution process (SEP), which can dynamically evaluate the execution process in the space navigation task. The hypothesis is proposed that there are neurobehavioral differences between normal cognitive (NC) elderly and AD patients with MCI reflected in digital biomarkers captured during SEP. According to this, we designed a new smart 2-min mobile alerting method for MCI due to AD, for community screening. Two digital biomarkers, total mission execution distance (METRtotal) and execution distance above the transverse obstacle (EDabove), were selected by step-up regression analysis. For the participants with more than 9 years of education, the alerting efficiency of the combination of the two digital biomarkers for MCI due to AD could reach 0.83. This method has the advantages of fast speed, high alerting efficiency, low cost and high intelligence and thus has a high application value for community screening in developing countries. It also provides a new intelligent alerting approach based on the human-computer interaction (HCI) paradigm for MCI due to AD in community screening.

Touchscreen Cognitive Tools for Mild Cognitive Impairment and Dementia Used in Primary Care Across Diverse Cultural and Literacy Populations: A Systematic Review

BACKGROUND

Touchscreen cognitive tools opened new promising opportunities for the early detection of cognitive impairment; however, most research studies are conducted in English-speaking populations and high-income countries, with a gap in knowledge about their use in populations with cultural, linguistic, and educational diversity.

OBJECTIVE

To review the touchscreen tools used in primary care settings for the cognitive assessment of mild cognitive impairment (MCI) and dementia, with a focus on populations of different cultures, languages, and literacy.

METHODS

This systematic review was conducted following the PRISMA guidelines. Studies were identified by searching across MEDLINE, EMBASE, EBSCO, OVID, SCOPUS, SCIELO, LILACS, and by cross-referencing. All studies that provide a first-level cognitive assessment for MCI and dementia with any touchscreen tools suitable to be used in the context of primary care were included.

RESULTS

Forty-two studies reporting on 30 tools and batteries were identified. Substantial differences among the tools emerged, in terms of theoretical framework, clinical validity, and features related to the application in clinical practice. A small proportion of the tools are available in multiple languages. Only 7 out of the 30 tools have a multiple languages validation. Only two tools are validated in low-educated samples, e.g., IDEA and mSTS-MCI.

CONCLUSION

General practitioners can benefit from touchscreen cognitive tools. However, easy requirements of the device, low dependence on the examiner, fast administration, and adaptation to different cultures and languages are some of the main features that we need to take into consideration when implementing touchscreen cognitive tools in the culture and language of underrepresented populations.

Cognitive Decline Detection for Alzheimer's Disease Patients Through an Activity of Daily Living (ADL)

There are conventional screening instruments for the detection of cognitive impairment, but they have a reduced ecological validity and the information they present could be biased. This study aimed at evaluating the effectiveness and usefulness of a task based on an activity of daily living (ADL) for the detection of cognitive impairment for an Alzheimer's disease (AD) population. Twenty-four participants were included in the study. The AD group (ADG) included twelve older adults (12 female) with AD (81.75±7.8 years). The Healthy group (HG) included twelve older adults (5 males, 77.7 ± 6.4 years). Both groups received a ADL-based intervention at two time frames separated 3 weeks. Cognitive functions were assessed before the interventions by using the MEC-35. The test-retest method was used to evaluate the reliability of the task, as well as the Intraclass Correlation Coefficient (ICC). The analysis of the test-retest reliability of the scores in the task indicated an excellent clinical relevance for both groups. The hypothesis of equality of the means of the scores in the two applications of the task was accepted for both the ADG and HG, respectively. The task also showed a significant high degree of association with the MEC-35 test (rho = 0.710, p = 0.010) for the ADG. Our results showed that it is possible to use an ADL-based task to assess everyday memory intended for cognitive impairments detection. In the same way, the task could be used to promote cognitive function and prevent dementia.

Predictive Accuracy of Digital Biomarker Technologies for Detection of Mild Cognitive Impairment and Pre-Frailty Amongst Older Adults: A Systematic Review and Meta-Analysis

Digital biomarker technologies coupled with predictive models are increasingly applied for early detection of age-related potentially reversible conditions including mild cognitive impairment (MCI) and pre-frailty (PF). We aimed to determine the predictive accuracy of digital biomarker technologies to detect MCI and PF with systematic review and meta-analysis. A computer-assisted search on major academic research databases including IEEE-Xplore was conducted. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines were adopted reporting in this study. Summary receiver operating characteristic curve based on random-effect bivariate model was used to evaluate overall sensitivity and specificity for detection of the respective age-related conditions. A total of 43 studies were selected for final systematic review and meta-analysis. 26 studies reported on detection of MCI with sensitivity and specificity of 0.48-1.00 and 0.55-1.00, respectively. On the other hand, there were 17 studies that reported on the detection of PF with reported sensitivity of 0.53-1.00 and specificity of 0.61-1.00. Meta-analysis further revealed pooled sensitivities of 0.84 (95% CI: 0.79-0.88) and 0.82 (95% CI: 0.74-0.88) for in-home detection of MCI and PF, respectively, while pooled specificities were 0.85 (95% CI: 0.80-0.89) and 0.82 (95% CI: 0.75-0.88), respectively. Besides MCI, and PF, in this work during systematic review, we also found one study which reported a sensitivity of 0.93 and a specificity of 0.57 for detection of cognitive frailty (CF). The meta-analytic result, for the first time, quantifies the predictive efficacy of digital biomarker technologies for detection of MCI and PF. Additionally, we found the number of studies for detection of CF to be notably lower, indicating possible research gaps to explore predictive models on digital biomarker technology for detection of CF.

The Use of a Computerized Cognitive Assessment to Improve the Efficiency of Primary Care Referrals to Memory Services: Protocol for the Accelerating Dementia Pathway Technologies (ADePT) Study

Background

Existing primary care cognitive assessment tools are crude or time-consuming screening instruments which can only detect cognitive impairment when it is well established. Due to the COVID-19 pandemic, memory services have adapted to the new environment by moving to remote patient assessments to continue meeting service user demand. However, the remote use of cognitive assessments has been variable while there has been scant evaluation of the outcome of such a change in clinical practice. Emerging research in remote memory clinics has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as prominent candidates for adoption in clinical practice both during the pandemic and for post-COVID-19 implementation as part of health care innovation.

Objective

The aim of the Accelerating Dementia Pathway Technologies (ADePT) study is to develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment to improve the efficiency of the dementia care pathway.

Methods

Patients who have been referred to a memory clinic by a general practitioner (GP) are recruited. Participants complete the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome are compared with the specialist diagnosis obtained at the memory clinic. The clinical outcomes as well as National Health Service reference costing data will be used to assess the potential health and economic benefits of the use of the ICA in the dementia diagnosis pathway.

Results

The ADePT study was funded in January 2020 by Innovate UK (Project Number 105837). As of September 2021, 86 participants have been recruited in the study, with 23 participants also completing a retest visit. Initially, the study was designed for in-person visits at the memory clinic; however, in light of the COVID-19 pandemic, the study was amended to allow remote as well as face-to-face visits. The study was also expanded from a single site to 4 sites in the United Kingdom. We expect results to be published by the second quarter of 2022.

Conclusions

The ADePT study aims to improve the efficiency of the dementia care pathway at its very beginning and supports systems integration at the intersection between primary and secondary care. The introduction of a standardized, self-administered, digital assessment tool for the timely detection of neurodegeneration as part of a decision support system that can signpost accordingly can reduce unnecessary referrals, service backlog, and assessment variability.

Trial Registration

ISRCTN 16596456; https://www.isrctn.com/ISRCTN16596456

International Registered Report Identifier (IRRID)

DERR1-10.2196/34475