Internet-based cognitive testing in multiple sclerosis

Diagnostic test accuracy of remote, multidomain cognitive assessment (telephone and video call) for dementia

BACKGROUND

Remote cognitive assessments are increasingly needed to assist in the detection of cognitive disorders, but the diagnostic accuracy of telephone- and video-based cognitive screening remains unclear.

OBJECTIVES

To assess the test accuracy of any multidomain cognitive test delivered remotely for the diagnosis of any form of dementia. To assess for potential differences in cognitive test scoring when using a remote platform, and where a remote screener was compared to the equivalent face-to-face test.

SEARCH METHODS

We searched ALOIS, the Cochrane Dementia and Cognitive Improvement Group Specialized Register, CENTRAL, MEDLINE, Embase, PsycINFO, CINAHL, Web of Science, LILACS, and ClinicalTrials.gov (www.

CLINICALTRIALS

gov/) databases on 2 June 2021. We performed forward and backward searching of included citations.

SELECTION CRITERIA

We included cross-sectional studies, where a remote, multidomain assessment was administered alongside a clinical diagnosis of dementia or equivalent face-to-face test.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed risk of bias and extracted data; a third review author moderated disagreements. Our primary analysis was the accuracy of remote assessments against a clinical diagnosis of dementia. Where data were available, we reported test accuracy as sensitivity and specificity. We did not perform quantitative meta-analysis as there were too few studies at individual test level. For those studies comparing remote versus in-person use of an equivalent screening test, if data allowed, we described correlations, reliability, differences in scores and the proportion classified as having cognitive impairment for each test.

MAIN RESULTS

The review contains 31 studies (19 differing tests, 3075 participants), of which seven studies (six telephone, one video call, 756 participants) were relevant to our primary objective of describing test accuracy against a clinical diagnosis of dementia. All studies were at unclear or high risk of bias in at least one domain, but were low risk in applicability to the review question. Overall, sensitivity of remote tools varied with values between 26% and 100%, and specificity between 65% and 100%, with no clearly superior test. Across the 24 papers comparing equivalent remote and in-person tests (14 telephone, 10 video call), agreement between tests was good, but rarely perfect (correlation coefficient range: 0.48 to 0.98).

AUTHORS' CONCLUSIONS

Despite the common and increasing use of remote cognitive assessment, supporting evidence on test accuracy is limited. Available data do not allow us to suggest a preferred test. Remote testing is complex, and this is reflected in the heterogeneity seen in tests used, their application, and their analysis. More research is needed to describe accuracy of contemporary approaches to remote cognitive assessment. While data comparing remote and in-person use of a test were reassuring, thresholds and scoring rules derived from in-person testing may not be applicable when the equivalent test is adapted for remote use.

Computerized neuropsychological assessment devices in multiple sclerosis: A systematic review

Background:

The proliferation of computerized neuropsychological assessment devices (CNADs) for screening and monitoring cognitive impairment is increasing exponentially. Previous reviews of computerized tests for multiple sclerosis (MS) were primarily qualitative and did not rigorously compare CNADs on psychometric properties.

Objective:

We aimed to systematically review the literature on the use of CNADs in MS and identify test batteries and single tests with good evidence for reliability and validity.

Method:

A search of four major online databases was conducted for publications related to computerized testing and MS. Test–retest reliability and validity coefficients and effect sizes were recorded for each CNAD test, along with administration characteristics.

Results:

We identified 11 batteries and 33 individual tests from 120 peer-reviewed articles meeting the inclusion criteria. CNADs with the strongest psychometric support include the CogState Brief Battery, Cognitive Drug Research Battery, NeuroTrax, CNS-Vital Signs, and computer-based administrations of the Symbol Digit Modalities Test.

Conclusion:

We identified several CNADs that are valid to screen for MS-related cognitive impairment, or to supplement full, conventional neuropsychological assessment. The necessity of testing with a technician, and in a controlled clinic/laboratory environment, remains uncertain.

Tracking cognitive impairment in multiple sclerosis using the Brain on Track test: a validation study

BACKGROUND

The accurate and regular monitoring cognitive performance in multiple sclerosis (MS) patients is critical to develop new prevention and management strategies for cognitive impairment (CI). The Brain on Track (BoT) test is a self-administered web-based tool developed for cognitive screening and monitoring. The objective of this study was to validate the use of the BoT in MS, by assessing its ability to distinguish between MS patients and matched controls, as well as detect CI among MS patients, by analysing its correlation with standard cognitive tests and its reliability and learning effects in repeatable use.

METHODS

The BoT was applied in 30 patients with MS consecutively selected and 30 age- and education-matched controls, first in a hospital clinic, under supervision, and then 1 week later from home. After these first two trials, MS patients repeated the test from home every 4 weeks for 3 months. A standard neuropsychological battery was also applied to MS patients at baseline.

RESULTS

The Cronbach's alpha was 0.89. Test scores were significantly different between MS patients and controls (Cohen's d = 0.87; p < 0.01). Among MS patients, scores were significantly lower in those with CI documented in the standard neuropsychological battery than in their cognitively preserved counterparts (Cohen's d = 2.0; p < 0.001). The BoT scores presented a good correlation with standard neuropsychological tests, particularly for information processing speed. Regarding test-retest reliability, 10/11 subtests presented two-way mixed single intraclass consistency correlation coefficients > 0.70.

CONCLUSION

The BoT showed good neuropsychological parameters in MS patients, endorsing the use of self-administered computerized tests in this setting.

The Use of Digital and Remote Communication Technologies as a Tool for Multiple Sclerosis Management: Narrative Review

Despite recent advances in multiple sclerosis (MS) care, many patients only infrequently access health care services, or are unable to access them easily, for reasons such as mobility restrictions, travel costs, consultation and treatment time constraints, and a lack of locally available MS expert services. Advances in mobile communications have led to the introduction of electronic health (eHealth) technologies, which are helping to improve both access to and the quality of health care services. As the Internet is now readily accessible through smart mobile devices, most people can take advantage of eHealth apps. The development of digital applications and remote communication technologies for patients with MS has increased rapidly in recent years. These apps are intended to complement traditional in-clinic approaches and can bring significant benefits to both patients with MS and health care providers (HCPs). For patients, such eHealth apps have been shown to improve outcomes and increase access to care, disease information, and support. These apps also help patients to participate actively in self-management, for example, by tracking adherence to treatment, changes in bladder and bowel habits, and activity and mood. For HCPs, MS eHealth solutions can simplify the multidisciplinary approaches needed to tailor MS management strategies to individual patients; facilitate remote monitoring of patient symptoms, adverse events, and outcomes; enable the efficient use of limited resources and clinic time; and potentially allow more timely intervention than is possible with scheduled face-to-face visits. These benefits are important because MS is a long-term, multifaceted chronic condition that requires ongoing monitoring, assessment, and management. We identified in the literature 28 eHealth solutions for patients with MS that fall within the four categories of screening and assessment, disease monitoring and self-management, treatment and rehabilitation, and advice and education. We review each solution, focusing on any clinical evidence supporting their use from prospective trials (including ASSESS MS, Deprexis, MSdialog, and the Multiple Sclerosis Performance Test) and consider the opportunities, barriers to adoption, and potential pitfalls of eHealth technologies in routine health care.

Information processing speed in multiple sclerosis: Past, present, and future

BACKGROUND

Information processing speed (IPS) is a prevalent cognitive impairment in multiple sclerosis (MS).

OBJECTIVES

This review aims to summarize the methods applied to assess IPS in MS and its theoretical conceptualization. A PubMed search was performed to select articles published between 1 January 2004 and 31 December 2013, resulting in 157 articles included.

RESULTS

The majority (54%) of studies assessed IPS with heterogeneous samples (several disease courses). Studies often report controlling for presence of other neurological disorders (60.5%), age (58.6%), education (51.6%), alcohol history (47.8%), or use of steroids (39.5%). Potential confounding variables, such as recent relapses (50.3%), history of developmental disorders (19.1%), and visual problems (29.9%), were often neglected. Assessments used to study IPS were heterogeneous (ranging from simple to complex tasks) among the studies under review, with 62 different tasks used. Only 9.6% of articles defined the construct of IPS and 22.3% discussed IPS in relation to a theoretical model.

FUTURE DIRECTIONS

The challenges for the upcoming decade include clarification of the definition of IPS as well as its theoretical conceptualization and a consensus on assessment. Based on the results obtained, we propose a new theoretical model, the tri-factor model of IPS.

CogState computerized memory tests in patients with brain metastases: secondary endpoint results of NRG Oncology RTOG 0933

Whole brain radiotherapy (WBRT) is associated with memory dysfunction. As part of NRG Oncology RTOG 0933, a phase II study of WBRT for brain metastases that conformally avoided the hippocampal stem cell compartment (HA-WBRT), memory was assessed pre- and post-HA-WBRT using both traditional and computerized memory tests. We examined whether the computerized tests yielded similar findings and might serve as possible alternatives for assessment of memory in multi-institution clinical trials. Adult patients with brain metastases received HA-WBRT to 30 Gy in ten fractions and completed Hopkins Verbal Learning Test-Revised (HVLT-R), CogState International Shopping List Test (ISLT) and One Card Learning Test (OCLT), at baseline, 2 and 4 months. Tests' completion rates were 52-53 % at 2 months and 34-42 % at 4 months. All baseline correlations between HVLT-R and CogState tests were significant (p ≤ 0.003). At baseline, both CogState tests and one component of HVLT-R differentiated those who were alive at 6 months and those who had died (p ≤ 0.01). At 4 months, mean relative decline was 7.0 % for HVLT-R Delayed Recall and 18.0 % for ISLT Delayed Recall. OCLT showed an 8.0 % increase. A reliable change index found no significant changes from baseline to 2 and 4 months for ISLT Delayed Recall (z = -0.40, p = 0.34; z = -0.68, p = 0.25) or OCLT (z = 0.15, p = 0.56; z = 0.41, p = 0.66). Study findings support the possibility that hippocampal avoidance may be associated with preservation of memory test performance, and that these computerized tests also may be useful and valid memory assessments in multi-institution adult brain tumor trials.

Visual search as a tool for a quick and reliable assessment of cognitive functions in patients with multiple sclerosis

Background

Despite the high frequency of cognitive impairment in multiple sclerosis, its assessment has not gained entrance into clinical routine yet, due to lack of time-saving and suitable tests for patients with multiple sclerosis.

Objective

The aim of the study was to compare the paradigm of visual search with neuropsychological standard tests, in order to identify the test that discriminates best between patients with multiple sclerosis and healthy individuals concerning cognitive functions, without being susceptible to practice effects.

Methods

Patients with relapsing remitting multiple sclerosis (n = 38) and age-and gender-matched healthy individuals (n = 40) were tested with common neuropsychological tests and a computer-based visual search task, whereby a target stimulus has to be detected amongst distracting stimuli on a touch screen. Twenty-eight of the healthy individuals were re-tested in order to determine potential practice effects.

Results

Mean reaction time reflecting visual attention and movement time indicating motor execution in the visual search task discriminated best between healthy individuals and patients with multiple sclerosis, without practice effects.

Conclusions

Visual search is a promising instrument for the assessment of cognitive functions and potentially cognitive changes in patients with multiple sclerosis thanks to its good discriminatory power and insusceptibility to practice effects.

High-throughput cognitive assessment using BrainTest.org: examining cognitive control in a family cohort

Introduction Understanding the relationship between brain and complex latent behavioral constructs like cognitive control will require an inordinate amount of data. Internet-based methods can rapidly and efficiently refine behavioral measures in very large samples that are needed for genetics and behavioral research. Cognitive control is a multifactorial latent construct that is considered to be an endophenotype in numerous neuropsychiatric disorders, including attention deficit/hyperactivity disorder (ADHD). While previous studies have demonstrated high correlations between Web- and lab-based scores, skepticism remains for its broad implementation. Methods Here, we promote a different approach by characterizing a completely Web-recruited and tested community family sample on measures of cognitive control. We examine the prevalence of attention deficit symptoms in an online community sample of adolescents, demonstrate familial correlations in cognitive control measures, and use construct validation techniques to validate our high-throughput assessment approach. Results A total of 1214 participants performed Web-based tests of cognitive control with over 200 parent-child pairs analyzed as part of the primary study aims. The data show a wide range of "subclinical" symptomatology in a web community sample of adolescents that supports a dimensional view of attention and also provide preliminary narrow-sense heritability estimates for commonly used working memory and response inhibition tests. Conclusions Finally, we show strong face and construct validity for these measures of cognitive control that generally exceeds the evidence required of new lab-based measures. We discuss these results and how broad implementation of this platform may allow us to uncover important brain-behavior relationships quickly and efficiently.

Assessing the validity of a computer-generated cognitive screening instrument for patients with multiple sclerosis

Background:

Neuropsychological testing requires considerable time, expense, and expertise to administer. These factors can limit patient access. Computerized cognitive testing has been proposed as an alternative.

Objectives:

The objective of this paper is to validate a brief, simple-to-use computer-generated cognitive assessment screening battery for multiple sclerosis (MS) patients that has minimal motor involvement.

Methods:

A sample of 96 MS patients and 98 healthy controls completed a computer-generated battery that included the Stroop, Symbol Digit Modalities Test (C-SDMT), a two- and four-second visual analog of the Paced Auditory Serial Addition Test (PVSAT-2, PVSAT-4), and simple and choice reaction time tests. The Minimal Assessment of Cognitive Function in MS was used to define cognitive impairment in the MS sample.

Results:

Each newly developed test successfully distinguished between cognitively impaired patients and healthy controls as well as cognitively intact patients. A combination of three computerized tests (C-SDMT, PVSAT-2, PVSAT-4) with a mean administration time of 10 minutes had a sensitivity of 82.5% and specificity of 87.5% in detecting cognitive impairment. Good test-retest reliability was obtained for each measure.

Conclusions:

Good sensitivity and specificity, brevity, ease of administration, and a limited motor component highlight the feasibility of introducing this computer-generated cognitive screening instrument in a busy MS clinic.

Use and acceptance of electronic communication by patients with multiple sclerosis: a multicenter questionnaire study

Background

The number of multiple sclerosis (MS) information websites, online communities, and Web-based health education programs has been increasing. However, MS patients’ willingness to use new ways of communication, such as websites, mobile phone application, short message service, or email with their physician, remains unknown.

Objectives

We designed a questionnaire to evaluate the a priori use of electronic communication methods by MS patients and to assess their acceptance of such tools for communication with their health care providers.

Methods

We received complete data from 586 MS patients aged between 17 and 73 years. Respondents were surveyed in outpatient clinics across Germany using a novel paper-and-pencil questionnaire. In addition to demographics, the survey items queried frequency of use of, familiarity with, and comfort with using computers, websites, email, and mobile phones.

Results

About 90% of all MS patients used a personal computer (534/586) and the Internet (527/586) at least once a week, 87.0% (510/586) communicated by email, and 85.6% (488/570) communicated by mobile phone. When asked about their comfort with using electronic communication methods for communication with health care providers, 20.5% (120/586) accepted communication by mobile Internet application or short message service via mobile phone, 41.0% (240/586) by websites, 54.3% (318/586) by email service, and 67.8% (397/586) by at least one type of electronic communication. The level of a priori use was the best predictor for the acceptance of electronic communication with health care providers. Patients who reported already searching online for health information (odds ratio 2.4, P < .001) and who had already communicated with a physician through a website (odds ratio 3.3, P = .03) reported higher acceptance for Web-based communication. Patients who already scheduled appointments with their mobile phones (odds ratio 2.1, P = .002) were more likely to accept the use of mobile phone applications or short message service for communicating with their physician.

Conclusions

The majority of MS patients seen at specialist centers already use modern communication technology regularly. New forms of electronic communication appear to have high levels of acceptance for exchanging information about MS between patients and health care providers. Such methods should be integrated into eHealth services such as electronic health records and patient relationship management systems.

Cognitive function testing in adult brain tumor trials: lessons from a comprehensive review

Neurocognitive function (NCF) impairment is an important component of primary and metastatic brain tumors and their therapeutic interventions. As a result, modern clinical trials of cranial irradiation for adult cancer patients have incorporated NCF testing as a primary or secondary end point. In doing so, these clinical trials have provided a novel insight into our understanding of the NCF effects of cranial irradiation and brain tumor progression. In this article, we review these clinical trials both in terms of the trial findings and in terms of the types of NCF tests used in these trials. We also provide an introduction to the strengths and limitations of these NCF tests, as well as expert commentary on the current status and future directions of NCF testing in brain tumor trials.

Web-based assessment of visual and visuospatial symptoms in Parkinson's disease

Visual and visuospatial dysfunction is prevalent in Parkinson's disease (PD). To promote assessment of these often overlooked symptoms, we adapted the PD Vision Questionnaire for Internet administration. The questionnaire evaluates visual and visuospatial symptoms, impairments in activities of daily living (ADLs), and motor symptoms. PD participants of mild to moderate motor severity (n = 24) and healthy control participants (HC, n = 23) completed the questionnaire in paper and web-based formats. Reliability was assessed by comparing responses across formats. Construct validity was evaluated by reference to performance on measures of vision, visuospatial cognition, ADLs, and motor symptoms. The web-based format showed excellent reliability with respect to the paper format for both groups (all P′s < 0.001; HC completing the visual and visuospatial section only). Demonstrating the construct validity of the web-based questionnaire, self-rated ADL and visual and visuospatial functioning were significantly associated with performance on objective measures of these abilities (all P′s < 0.01). The findings indicate that web-based administration may be a reliable and valid method of assessing visual and visuospatial and ADL functioning in PD.

Virtual Simulations and the Second Life Metaverse

In neuropsychology’s received paradigm, the “normal science” of assessment and treatment planning appears to be approaching a paradigm shift: first, there are the general developments in other neurosciences that inform the practice of neuropsychological assessment. Second, there is the shift in the purpose of neuropsychological assessment from differential diagnosis of brain pathology to predictions about activities of everyday functioning and treatment planning. Third, there is growing need that neuropsychologists update their outdated technology for ecologically valid assessments. The impending paradigm shift may be well served to include the utility of virtual worlds for ecologically valid neuropsychological assessments. Actualization of the potential of virtual worlds for assessment will require the following: comparisons with well-validated neuropsychological measures, data storage, improved documentation of specific computer hardware and software used in experimental methods, and enhanced methods and result reporting by the researchers publishing studies on virtual worlds.

Clinical consequences of MRI activity in treated multiple sclerosis

Background: Inflammation on brain MRI is the most sensitive marker of disease activity in multiple sclerosis (MS) but its clinical consequences remain controversial.

Objective: Here we investigated the clinical consequences of MRI activity in MS subjects treated with two different first line disease modifying agents.

Methods: Seventy-five treatment-naïve subjects with relapsing–remitting MS ( N = 61) or clinically isolated syndromes at risk of MS ( N = 14) from the BECOME study that had been randomized to interferon beta-1b ( N = 39) or glatiramer acetate ( N = 36) and followed for up to two years by monthly brain MRI optimized to detect inflammatory activity were studied for the clinical consequences of lack of MRI remission.

Results: MRI remission occurred in 46.4% of participants transiently and in 23.2% completely while it was never achieved in 30.4%. There was no difference by treatment in MRI remission, progression of physical disability, or cognitive function. The percentage of relapse-free subjects was 87.5% for the group in complete MRI remission, 47.6% in the group never in remission and 59.4% in the group in transient remission ( p = 0.017). Similar differences were observed for six-month-confirmed worsening of ambulatory function as measured by the timed 25 foot walk ( p = 0.026) and by Expanded Disability Status Scale (EDSS) ( p = 0.10). Cognitive function was lowest at baseline for the group that never reached MRI remission on treatment; this group improved the least upon repeated cognitive testing during the two years of treatment ( p < 0.001).

Conclusions: Lack of MRI remission during treatment with interferon beta-1b or glatiramer acetate is associated with higher relapse rate and worsening of physical and cognitive function.

Cognitive impairment in multiple sclerosis

Multiple sclerosis (MS) is a progressive disease of the CNS that is characterised by widespread lesions in the brain and spinal cord. MS results in motor, cognitive, and neuropsychiatric symptoms, all of which can occur independently of one another. The common cognitive symptoms include deficits in complex attention, efficiency of information processing, executive functioning, processing speed, and long-term memory. These deficits detrimentally affect many aspects of daily life, such as the ability to run a household, participate fully in society, and maintain employment--factors that can all affect the overall quality of life of the patient. The increased use of neuroimaging techniques in patients with MS has advanced our understanding of structural and functional changes in the brain that are characteristic of this disease, although much remains to be learned. Moreover, examination of efforts to treat the cognitive deficits in MS is still in the early stages.