Smartphone-based symbol-digit modalities test reliably captures brain damage in multiple sclerosis

Differential Validity in the Written, Oral, and Electronic SDMT in Multiple Sclerosis

BACKGROUND

Cognitive decline in multiple sclerosis (MS) is commonly assessed using the Symbol Digit Modalities Test (SDMT). However, the written (wSDMT), oral (oSDMT), and electronic (eSDMT) versions may display differing psychometric properties, despite often being used interchangeably.

OBJECTIVE

The objective of the study is to compare the wSDMT, oSDMT, and eSDMT, including inter-test reliability, and to assess demographic and motor disability influences on test performance.

METHODS

In this within-subjects study, 85 participants with MS completed all three SDMT versions. Statistical analyses included assessment of intraclass correlation coefficients (ICCs) and sequential multiple regression modeling.

RESULTS

The three SDMT versions demonstrated good inter-test reliability (ICC = 0.77) and similar mean scores (range: 53.5-56.2). Scores on all three test versions declined with age (p < 0.05), but the decline was significantly steeper in the eSDMT. Education correlated with eSDMT but not wSDMT or oSDMT scores. Paced Auditory Serial Addition Task (PASAT) scores predicted performance across all versions, while 9HPT times predicted only wSDMT and eSDMT scores (p < 0.01).

CONCLUSION

The wSDMT and eSDMT show signs of motor disability influence, while the eSDMT displays the greatest sensitivity to the influences of age and education. This differential construct validity necessitates SDMT version-specific normative data and motor-adjustments for accurate cognitive assessment in MS.

Investigating Measurement Equivalence of Smartphone Sensor-Based Assessments: Remote, Digital, Bring-Your-Own-Device Study

BACKGROUND

Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population.

OBJECTIVE

This study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models.

METHODS

Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a "bring-your-own-device" (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor-based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status.

RESULTS

Overall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03).

CONCLUSIONS

Consistent with the hypothesis that smartphone sensor-based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.

A New Smartphone-Based Cognitive Screening Battery for Multiple Sclerosis (icognition): Validation Study

BACKGROUND

Cognitive deterioration is common in multiple sclerosis (MS) and requires regular follow-up. Currently, cognitive status is measured in clinical practice using paper-and-pencil tests, which are both time-consuming and costly. Remote monitoring of cognitive status could offer a solution because previous studies on telemedicine tools have proved its feasibility and acceptance among people with MS. However, existing smartphone-based apps include designs that are prone to motor interference and focus primarily on information processing speed, although memory is also commonly affected.

OBJECTIVE

This study aims to validate a smartphone-based cognitive screening battery, icognition, to detect deterioration in both memory and information processing speed.

METHODS

The icognition screening battery consists of 3 tests: the Symbol Test for information processing speed, the Dot Test for visuospatial short-term memory and learning, and the visual Backward Digit Span (vBDS) for working memory. These tests are based on validated paper-and-pencil tests: the Symbol Digit Modalities Test, the 10/36 Spatial Recall Test, and the auditory Backward Digit Span, respectively. To establish the validity of icognition, 101 people with MS and 82 healthy participants completed all tests. Of the 82 healthy participants, 20 (24%) repeated testing 2 to 3 weeks later. For each icognition test, validity was established by the correlation with its paper-and-pencil equivalent (concurrent validity), the correlation and intraclass correlation coefficient (ICC) between baseline and follow-up testing (test-retest reliability), the difference between people with MS and healthy participants, and the correlation with other clinical parameters such as the Expanded Disability Status Scale.

RESULTS

All icognition tests correlated well with their paper-and-pencil equivalents (Symbol Test: r=0.67; P<.001; Dot Test: r=0.31; P=.002; vBDS: r=0.69; P<.001), negatively correlated with the Expanded Disability Status Scale (Symbol Test: ρ=-0.34; P<.001; Dot Test: ρ=-0.32; P=.003; vBDS: ρ=-0.21; P=.04), and showed moderate test-retest reliability (Symbol Test: ICC=0.74; r=0.85; P<.001; Dot Test: ICC=0.71; r=0.74; P<.001; vBDS: ICC=0.72; r=0.83; P<.001). Test performance was comparable between people with MS and healthy participants for all cognitive tests, both in icognition (Symbol Test: U=4431; P=.42; Dot Test: U=3516; P=.32; vBDS: U=3708; P=.27) and the gold standard paper-and-pencil tests (Symbol Digit Modalities Test: U=4060.5, P=.82; 10/36 Spatial Recall Test: U=3934; P=.74; auditory Backward Digit Span: U=3824.5, P=.37).

CONCLUSIONS

icognition is a valid tool to remotely screen cognitive performance in people with MS. It is planned to be included in a digital health platform that includes volumetric brain analysis and patient-reported outcome measures. Future research should establish the usability and psychometric properties of icognition in a remote setting.

CoGames: Development of an adaptive smartphone-based and gamified monitoring tool for cognitive function in Multiple Sclerosis

AIM

As part of the development of a smartphone-based app for monitoring MS disease activity and progression (dreaMS, NCT05009160), we developed six gamified tests with multiple difficulty levels as a monitoring tool for cognition. This study quantified the relative difficulty between levels and investigated their reliability, ability to depict practice effects, and user acceptance.

METHODS

Healthy volunteers played each game, covering five cognitive domains, twice per day for 11 consecutive days. Linear mixed models determined the relative difficulty of the levels. Spearman's correlation of the two daily repetitions measured test-retest reliability. Difficulty increased daily except for days 2, 6, and 11, when the easiest level ("Beginner") was repeated to estimate practice effects. Participants rated enjoyment and other components of acceptance on a 5-point scale.

RESULTS

We included 82 participants from April to July 2023 in Basel, Switzerland, of which 76 (51 female, age: 40.3 ± 13.9 years, range 18-69) completed the study according to protocol. Generally, mean performances decreased with higher difficulty levels. Across all levels of all games, the median test-retest correlation was 0.825 (range of medians 0.55-0.9). Mean performance in level "Beginner" improved across all games. The mean enjoyment rating was 3.9 (range: 3.1-4.3).

CONCLUSION

Our study showed that the CoGames yield reliable measures across different cognitive domains and difficulty levels and were enjoyable to play. The observed practice effects must be considered, but also indicate sensitivity to change. These results support the hypothesis that adaptive gamified digital tests can serve as a reliable and well-accepted monitoring tool of cognition in PwMS.

Development and validation of an electronic Symbol-Digit Modalities Test for remote monitoring of people with multiple sclerosis

BACKGROUND

Computerized cognitive tests may extend the reach of cognitive screening and monitoring to those with mobility issues or living in remote areas. Moreover, it could enable frequent and autonomous remote cognitive assessments in people with multiple sclerosis (pwMS) on account of its reduced economic and organizational costs. This may further improve our understanding of longitudinal trends and significantly improve the standard of care for pwMS living in remote areas or with mobility limitations. We aimed to evaluate the psychometric properties of an electronic Symbol-Digit Modalities Test (eSDMT) designed to allow pwMS to perform a rapid cognitive assessment independently from home using their own PC/laptop.

METHODS

Sixty-two participants underwent a neuropsychological evaluation, and then performed the eSDMT in the clinic. Forty-two participants also repeated the eSDMT at home. We assessed concurrent validity (eSDMT vs. oral SDMT), test-retest reliability (in the clinic vs. at home), discriminant validity (pwMS with/without cognitive impairment), and other psychometric characteristics of the eSDMT (effect of age, sex, and education on test scores).

RESULTS

We observed good-to-excellent concurrent validity (r ≥ 0.84, all p < 0.0001) and test-retest reliability (intraclass correlation coefficients [ICCs]>0.87, all p < 0.0001). Discriminant validity was excellent (area under the curves [AUCs] >0.84, all p < 0.0001). eSDMT scores were only slightly influenced by demographic characteristics (all R2 < 0.200).

CONCLUSIONS

We provided evidence which supports the use of our eSDMT as a feasible, valid, and reliable remote assessment of cognitive function in pwMS. Future studies will investigate long-term reliability and predictive power.

Measuring Disease Progression in Multiple Sclerosis Clinical Drug Trials and Impact on Future Patient Care

Multiple sclerosis (MS) is a chronic immune-mediated disease of the central nervous system characterised by inflammation, demyelination and neurodegeneration. Although several drugs are approved for MS, their efficacy in progressive disease is modest. Addressing disease progression as a treatment goal in MS is challenging due to several factors. These include a lack of complete understanding of the pathophysiological mechanisms driving MS and the absence of sensitive markers of disease progression in the short-term of clinical trials. MS usually begins at a young age and lasts for decades, whereas clinical research often spans only 1-3 years. Additionally, there is no unifying definition of disease progression. Several drugs are currently being investigated for progressive MS. In addition to new medications, the rise of new technologies and of adaptive trial designs is enabling larger and more integrated data collection. Remote assessments and decentralised clinical trials are becoming feasible. These will allow more efficient and large studies at a lower cost and with less burden on study participants. As new drugs are developed and research evolves, we anticipate a concurrent change in patient care at various levels in the foreseeable future. We conducted a narrative review to discuss the challenges of accurately measuring disease progression in contemporary MS drug trials, some new research trends and their implications for patient care.

Sensor-Derived Measures of Motor and Cognitive Functions in People With Multiple Sclerosis Using Unsupervised Smartphone-Based Assessments: Proof-of-Concept Study

BACKGROUND

Smartphones and wearables are revolutionizing the assessment of cognitive and motor function in neurological disorders, allowing for objective, frequent, and remote data collection. However, these assessments typically provide a plethora of sensor-derived measures (SDMs), and selecting the most suitable measure for a given context of use is a challenging, often overlooked problem.

OBJECTIVE

This analysis aims to develop and apply an SDM selection framework, including automated data quality checks and the evaluation of statistical properties, to identify robust SDMs that describe the cognitive and motor function of people with multiple sclerosis (MS).

METHODS

The proposed framework was applied to data from a cross-sectional study involving 85 people with MS and 68 healthy participants who underwent in-clinic supervised and remote unsupervised smartphone-based assessments. The assessment provided high-quality recordings from cognitive, manual dexterity, and mobility tests, from which 47 SDMs, based on established literature, were extracted using previously developed and publicly available algorithms. These SDMs were first separately and then jointly screened for bias and normality by 2 expert assessors. Selected SDMs were then analyzed to establish their reliability, using an intraclass correlation coefficient and minimal detectable change at 95% CI. The convergence of selected SDMs with in-clinic MS functional measures and patient-reported outcomes was also evaluated.

RESULTS

A total of 16 (34%) of the 47 SDMs passed the selection framework. All selected SDMs demonstrated moderate-to-good reliability in remote settings (intraclass correlation coefficient 0.5-0.85; minimal detectable change at 95% CI 19%-35%). Selected SDMs extracted from the smartphone-based cognitive test demonstrated good-to-excellent correlation (Spearman correlation coefficient, |ρ|>0.75) with the in-clinic Symbol Digit Modalities Test and fair correlation with Expanded Disability Status Scale (EDSS) scores (0.25≤|ρ|<0.5). SDMs extracted from the manual dexterity tests showed either fair correlation (0.25≤|ρ|<0.5) or were not correlated (|ρ|<0.25) with the in-clinic 9-hole peg test and EDSS scores. Most selected SDMs from mobility tests showed fair correlation with the in-clinic timed 25-foot walk test and fair to moderate-to-good correlation (0.5<|ρ|≤0.75) with EDSS scores. SDM correlations with relevant patient-reported outcomes varied by functional domain, ranging from not correlated (cognitive test SDMs) to good-to-excellent correlation (|ρ|>0.75) for mobility test SDMs. Overall, correlations were similar when smartphone-based tests were performed in a clinic or remotely.

CONCLUSIONS

Reported results highlight that smartphone-based assessments are suitable tools to remotely obtain high-quality SDMs of cognitive and motor function in people with MS. The presented SDM selection framework promises to increase the interpretability and standardization of smartphone-based SDMs in people with MS, paving the way for their future use in interventional trials.

Digital outcome measures are associated with brain atrophy in patients with multiple sclerosis

BACKGROUND

Digital monitoring of people with multiple sclerosis (PwMS) using smartphone-based monitoring tools is a promising method to assess disease activity and progression.

OBJECTIVE

To study cross-sectional and longitudinal associations between active and passive digital monitoring parameters and MRI volume measures in PwMS.

METHODS

In this prospective study, 92 PwMS were included. Clinical tests [Expanded Disability Status Scale (EDSS), Timed 25 Foot Walk test (T25FW), 9-Hole Peg Test (NHPT), and Symbol Digit Modalities Test (SDMT)] and structural MRI scans were performed at baseline (M0) and 12-month follow-up (M12). Active monitoring included the smartphone-based Symbol Digit Modalities Test (sSDMT) and 2 Minute Walk Test (s2MWT), while passive monitoring was based on smartphone keystroke dynamics (KD). Linear regression analyses were used to determine cross-sectional and longitudinal relations between digital and clinical outcomes and brain volumes, with age, disease duration and sex as covariates.

RESULTS

In PwMS, both sSDMT and SDMT were associated with thalamic volumes and lesion volumes. KD were related to brain, ventricular, thalamic and lesion volumes. No relations were found between s2MWT and MRI volumes. NHPT scores were associated with lesion volumes only, while EDSS and T25FW were not related to MRI. No longitudinal associations were found for any of the outcome measures between M0 and M12.

CONCLUSION

Our results show clear cross-sectional correlations between digital biomarkers and brain volumes in PwMS, which were not all present for conventional clinical outcomes, supporting the potential added value of digital monitoring tools.

Digital symbol-digit modalities test with modified flexible protocols in patients with CNS demyelinating diseases

Cognitive impairment (CI) is prevalent in central nervous system demyelinating diseases, such as multiple sclerosis (MS) and neuromyelitis optica spectrum disorders (NMOSD). We developed a novel tablet-based modified digital Symbol Digit Modalities Test (MD-SDMT) with adjustable protocols that feature alternating symbol-digit combinations in each trial, lasting one or two minutes. We assessed 144 patients (99 with MS and 45 with NMOSD) using both MD-SDMT protocols and the traditional paper-based SDMT. We also gathered participants' feedback through a questionnaire regarding their preferences and perceived reliability. The results showed strong correlations between MD-SDMT and paper-based SDMT scores (Pearsons correlation: 0.88 for 2 min; 0.85 for 1 min, both p < 0.001). Among the 120 respondents, the majority preferred the digitalized SDMT (55% for the 2 min, 39% for the 1 min) over the paper-based version (6%), with the 2 min MD-SDMT reported as the most reliable test. Notably, patients with NMOSD and older individuals exhibited a preference for the paper-based test, as compared to those with MS and younger patients. In summary, even with short test durations, the digitalized SDMT effectively evaluates cognitive function in MS and NMOSD patients, and is generally preferred over the paper-based method, although preferences may vary with patient characteristics.

Mobile Health App (DIGICOG-MS) for Self-Assessment of Cognitive Impairment in People With Multiple Sclerosis: Instrument Validation and Usability Study

BACKGROUND

Mobile health (mHealth) apps have proven useful for people with multiple sclerosis (MS). Thus, easy-to-use digital solutions are now strongly required to assess and monitor cognitive impairment, one of the most disturbing symptoms in MS that is experienced by almost 43% to 70% of people with MS. Therefore, we developed DIGICOG-MS (Digital assessment of Cognitive Impairment in Multiple Sclerosis), a smartphone- and tablet-based mHealth app to self-assess cognitive impairment in MS.

OBJECTIVE

This study aimed to test the validity and usability of the novel mHealth app with a sample of people with MS.

METHODS

DIGICOG-MS includes 4 digital tests assumed to evaluate the most affected cognitive domains in MS (visuospatial memory [VSM], verbal memory [VM], semantic fluency [SF], and information processing speed [IPS]) and inspired by traditional paper-based tests that assess the same cognitive functions (10/36 Spatial Recall Test, Rey Auditory Verbal Learning Test, Word List Generation, Symbol Digit Modalities Test). Participants were asked to complete both digital and traditional assessments in 2 separate sessions. Convergent validity was analyzed using the Pearson correlation coefficient to determine the strength of the associations between digital and traditional tests. To test the app's reliability, the agreement between 2 repeated measurements was assessed using intraclass correlation coefficients (ICCs). Usability of DIGICOG-MS was evaluated using the System Usability Scale (SUS) and mHealth App Usability Questionnaire (MAUQ) administered at the conclusion of the digital session.

RESULTS

The final sample consisted of 92 people with MS (60 women) followed as outpatients at the Italian Multiple Sclerosis Society (AISM) Rehabilitation Service of Genoa (Italy). They had a mean age of 51.38 (SD 11.36) years, education duration of 13.07 (SD 2.74) years, disease duration of 12.91 (SD 9.51) years, and a disability level (Expanded Disability Status Scale) of 3.58 (SD 1.75). Relapsing-remitting MS was most common (68/92, 74%), followed by secondary progressive (15/92, 16%) and primary progressive (9/92, 10%) courses. Pearson correlation analyses indicated significantly strong correlations for VSM, VM, SF, and IPS (all P<.001), with r values ranging from 0.58 to 0.78 for all cognitive domains. Test-retest reliability of the mHealth app was excellent (ICCs>0.90) for VM and IPS and good for VSM and SF (ICCs>0.80). Moreover, the SUS score averaged 84.5 (SD 13.34), and the mean total MAUQ score was 104.02 (SD 17.69), suggesting that DIGICOG-MS was highly usable and well appreciated.

CONCLUSIONS

The DIGICOG-MS tests were strongly correlated with traditional paper-based evaluations. Furthermore, people with MS positively evaluated DIGICOG-MS, finding it highly usable. Since cognitive impairment poses major limitations for people with MS, these findings open new paths to deploy digital cognitive tests for MS and further support the use of a novel mHealth app for cognitive self-assessment by people with MS in clinical practice.

An update on tests used for intraoperative monitoring of cognition during awake craniotomy

PURPOSE

Mapping higher-order cognitive functions during awake brain surgery is important for cognitive preservation which is related to postoperative quality of life. A systematic review from 2018 about neuropsychological tests used during awake craniotomy made clear that until 2017 language was most often monitored and that the other cognitive domains were underexposed (Ruis, J Clin Exp Neuropsychol 40(10):1081-1104, 218). The field of awake craniotomy and cognitive monitoring is however developing rapidly. The aim of the current review is therefore, to investigate whether there is a change in the field towards incorporation of new tests and more complete mapping of (higher-order) cognitive functions.

METHODS

We replicated the systematic search of the study from 2018 in PubMed and Embase from February 2017 to November 2023, yielding 5130 potentially relevant articles. We used the artificial machine learning tool ASReview for screening and included 272 papers that gave a detailed description of the neuropsychological tests used during awake craniotomy.

RESULTS

Comparable to the previous study of 2018, the majority of studies (90.4%) reported tests for assessing language functions (Ruis, J Clin Exp Neuropsychol 40(10):1081-1104, 218). Nevertheless, an increasing number of studies now also describe tests for monitoring visuospatial functions, social cognition, and executive functions.

CONCLUSIONS

Language remains the most extensively tested cognitive domain. However, a broader range of tests are now implemented during awake craniotomy and there are (new developed) tests which received more attention. The rapid development in the field is reflected in the included studies in this review. Nevertheless, for some cognitive domains (e.g., executive functions and memory), there is still a need for developing tests that can be used during awake surgery.

Digital measures of respiratory and upper limb function in spinal muscular atrophy: design, feasibility, reliability, and preliminary validity of a smartphone sensor-based assessment suite

Spinal muscular atrophy (SMA) is characterized by progressive muscle weakness and paralysis. Motor function is monitored in the clinical setting using assessments including the 32-item Motor Function Measure (MFM-32), but changes in disease severity between clinical visits may be missed. Digital health technologies may assist evaluation of disease severity by bridging gaps between clinical visits. We developed a smartphone sensor-based assessment suite, comprising nine tasks, to assess motor and muscle function in people with SMA. We used data from the risdiplam phase 2 JEWELFISH trial to assess the test-retest reliability and convergent validity of each task. In the first 6 weeks, 116 eligible participants completed assessments on a median of 6.3 days per week. Eight of the nine tasks demonstrated good or excellent test-retest reliability (intraclass correlation coefficients >0.75 and >0.9, respectively). Seven tasks showed a significant association (P < 0.05) with related clinical measures of motor function (individual items from the MFM-32 or Revised Upper Limb Module scales) and seven showed significant association (P < 0.05) with disease severity measured using the MFM-32 total score. This cross-sectional study supports the feasibility, reliability, and validity of using smartphone-based digital assessments to measure function in people living with SMA.

Association between cognition and gait in multiple sclerosis: A smartphone-based longitudinal analysis

BACKGROUND

Gait and cognition impairments are common problems among People with Multiple Sclerosis (PwMS). Previous studies have investigated cross-sectional associations between gait and cognition. However, there is a lack of evidence regarding the longitudinal association between these factors in PwMS. Therefore, the objective of this study was to explore this longitudinal relationship using smartphone-based data from the Floodlight study.

METHODS

Using the publicly available Floodlight dataset, which contains smartphone-based longitudinal data, we used a linear mixed model to investigate the longitudinal relationship between cognition, measured by the Symbol Digit Modalities Test (SDMT), and gait, measured by the 2 Minute Walking test (2 MW) step count and Five-U-Turn Test (FUTT) turning speed. Four mixed models were fitted to explore the association between: 1) SDMT and mean step count; 2) SDMT and variability of step count; 3) SDMT and mean FUTT turning speed; and 4) SDMT and variability of FUTT turningt speed.

RESULTS

After controlling for age, sex, weight, and height, there were significant correlations between SDMT and the variability of 2 MW step count, the mean of FUTT turning speed. No significant correlation was observed between SDMT and the 2 MW mean step count.

SIGNIFICANCE

Our findings support the evidence that gait and cognition are associated in PwMS. This may support clinicians to adjust treatment and intervention programs that address both gait and cognitive impairments.

Remote research burden of follow-up in longitudinal patient-reported outcomes (PROs) data collection: An exploratory sequential mixed-methods study (Preprint).. [DOI: 10.2196/preprints.51290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Longitudinal patient-reported outcomes studies require questionnaire assessments to be administered remotely multiple times, burdening research staff.

OBJECTIVE

To define and quantify the burden that researcher may experience during patient follow-up.

METHODS

Data were collected via interviews and a questionnaire. This study is an exploratory sequential mixed-methods study. Traditional content analysis was used for the qualitative data. Quantitative data were analyzed using Spearman’s correlation, and significance was tested using the chi-square test. Learning curves of healthcare staff regarding follow-up calls were generated using cumulative summation analysis.

RESULTS

We constructed a three-dimension conceptual framework for staff burden: (a) time-related burden, (b) technical-related burden, and (c) emotional-related burden. The quantitative analysis found that follow-up time was significantly correlated with staff experience, workload, and learning curve periods. There was a significant difference between the lost-to-follow-up rate of staff with and without follow-up experience with this program. Staff working on a daily assessment schedule had a higher lost-to-follow-up rate than those on a twice-a-week schedule. Additionally, inexperienced follow-up staff needed 113 calls to achieve stable follow-up time and quality, while experienced staff needed only 55 calls.

CONCLUSIONS

Researchers in longitudinal PROs projects suffer from a multidimensional burden during remote follow-up. Our results may help establish a proper PROs follow-up protocol to reduce the burden on research staff without sacrificing data quality.

Unmet needs and gaps in the identification of secondary progression in multiple sclerosis: a Southern Italy healthcare professionals' perspective

OBJECTIVE

Multiple sclerosis (MS) is a chronic disease with different clinical courses and a tendency to worsening. The relapsing-remitting MS presents acute onset and relapses of neurological symptoms, followed by their remission. This form can convert to secondary progressive MS (SPMS) with irreversible neurological worsening and disability. The identification of signs, symptoms, markers of progression, and strategies to manage MS patients is mandatory to allow early identification of those at higher risk of conversion to SPMS, for prompt intervention to cope with the progression of the disease.

METHODS

A panel of Italian experts from Southern Italy have reviewed the current knowledge on MS and its management and identified the crucial tools for SPMS recognition.

RESULTS

More effective communication between patients and clinicians should be established, with the support of digital tools. Moreover, the improvement in the clinical use of biomarkers for progression (cellular structures and tissue organization, such as neurofilaments and chitinase 3-like 1, axonal and neurons density) and of instrumental analyses for recognition of whole-brain atrophy, chronic active lesions, spinal cord lesions and atrophy, and the improvement the combination of the Expanded Disability Status Scale and the evaluation of cognitive dysfunction are discussed.

CONCLUSION

Given the availability of a pharmacological option, adequate education both for patients, regarding the evolution of the disease and the specific treatment, and for professionals, to allow more effective and sensitive communication and the best use of diagnostic and management tools, could represent a strategy to improve patient management and their quality of life.

Longitudinal Trend Monitoring of Multiple Sclerosis Ambulation Using Smartphones

Goal: Smartphone and wearable devices may act as powerful tools to remotely monitor physical function in people with neurodegenerative and autoimmune diseases from out-of-clinic environments. Detection of progression onset or worsening of symptoms is especially important in people living with multiple sclerosis (PwMS) in order to enable optimally adapted therapeutic strategies. MS symptoms typically follow subtle and fluctuating disease courses, patient-to-patient, and over time. Current in-clinic assessments are often too infrequently administered to reflect longitudinal changes in MS impairment that impact daily life. This work, therefore, explores how smartphones can administer daily two-minute walking assessments to monitor PwMS physical function at home. Methods: Remotely collected smartphone inertial sensor data was transformed through state-of-the-art Deep Convolutional Neural Networks, to estimate a participant's daily ambulatory-related disease severity, longitudinally over a 24-week study. Results: This study demonstrated that smartphone-based ambulatory severity outcomes could accurately estimate MS level of disability, as measured by the EDSS score ([Formula: see text]: 0.56,[Formula: see text]0.001). Furthermore, longitudinal severity outcomes were shown to accurately reflect individual participants' level of disability over the study duration. Conclusion: Smartphone-based assessments, that can be performed by patients from their home environments, could greatly augment standard in-clinic outcomes for neurodegenerative diseases. The ability to understand the impact of disease on daily-life between clinical visits, through objective digital outcomes, paves the way forward to better measure and identify signs of disease progression that may be occurring out-of-clinic, to monitor how different patients respond to various treatments, and to ultimately enable the development of better, and more personalised care.

Use of Mobile Applications to Increase Therapeutic Adherence in Adults: A Systematic Review

Failure to comply with therapeutic treatments implies negative repercussions for the patient's quality of life, their social environment, and health system. The use of information and communication technologies, especially mobile applications, has favored the increase in global therapeutic adherence figures. The objective of this study is to characterize the use of mobile applications as a strategy to increase therapeutic adherence in adults. A systematic literature review in Web of Science and Scopus was performed following the Preferred Information elements for Systematic Reviews and Meta-analysis. Information such as: the year of publication, the study population, the medical conditions of the participants, the main characteristics or functionalities of the mobile applications, and the methods or tools used to measure treatment adherence were extracted from each included article. The risk of bias was assessed. Twelve randomized controlled trials (RCTs), published in English from 1996 to May 2021, were included. Chronic diseases have been mostly addressed through interventions with mobile applications. The most reported functions of mobile applications were reminders, educational modules, two-way communication, and games. Tools such as: "Morisky Medication Adherence Scale of eight items"; "Medication adherence questionnaire"; "Self-reported adherence"; among others, were used to evaluate and report the treatment adherence. In conclusion, including treatment interventions using mobile applications in clinical practice has proven to be beneficial to improve therapeutic adherence. However, it is necessary to develop high-quality clinical trials (size and duration) to generalize results and justify their use in conventional health services.

Confounder-adjusted MRI-based predictors of multiple sclerosis disability

Introduction

Both aging and multiple sclerosis (MS) cause central nervous system (CNS) atrophy. Excess brain atrophy in MS has been interpreted as "accelerated aging." Current paper tests an alternative hypothesis: MS causes CNS atrophy by mechanism(s) different from physiological aging. Thus, subtracting effects of physiological confounders on CNS structures would isolate MS-specific effects.

Methods

Standardized brain MRI and neurological examination were acquired prospectively in 646 participants enrolled in ClinicalTrials.gov Identifier: NCT00794352 protocol. CNS volumes were measured retrospectively, by automated Lesion-TOADS algorithm and by Spinal Cord Toolbox, in a blinded fashion. Physiological confounders identified in 80 healthy volunteers were regressed out by stepwise multiple linear regression. MS specificity of confounder-adjusted MRI features was assessed in non-MS cohort (n = 158). MS patients were randomly split into training (n = 277) and validation (n = 131) cohorts. Gradient boosting machine (GBM) models were generated in MS training cohort from unadjusted and confounder-adjusted CNS volumes against four disability scales.

Results

Confounder adjustment highlighted MS-specific progressive loss of CNS white matter. GBM model performance decreased substantially from training to cross-validation, to independent validation cohorts, but all models predicted cognitive and physical disability with low p-values and effect sizes that outperform published literature based on recent meta-analysis. Models built from confounder-adjusted MRI predictors outperformed models from unadjusted predictors in the validation cohort.

Conclusion

GBM models from confounder-adjusted volumetric MRI features reflect MS-specific CNS injury, and due to stronger correlation with clinical outcomes compared to brain atrophy these models should be explored in future MS clinical trials.

Current Status and Future Opportunities in Modeling Clinical Characteristics of Multiple Sclerosis

Development of effective treatments requires understanding of disease mechanisms. For diseases of the central nervous system (CNS), such as multiple sclerosis (MS), human pathology studies and animal models tend to identify candidate disease mechanisms. However, these studies cannot easily link the identified processes to clinical outcomes, such as MS severity, required for causality assessment of candidate mechanisms. Technological advances now allow the generation of thousands of biomarkers in living human subjects, derived from genes, transcripts, medical images, and proteins or metabolites in biological fluids. These biomarkers can be assembled into computational models of clinical value, provided such models are generalizable. Reproducibility of models increases with the technical rigor of the study design, such as blinding, control implementation, the use of large cohorts that encompass the entire spectrum of disease phenotypes and, most importantly, model validation in independent cohort(s). To facilitate the growth of this important research area, we performed a meta-analysis of publications (n = 302) that model MS clinical outcomes extracting effect sizes, while also scoring the technical quality of the study design using predefined criteria. Finally, we generated a Shiny-App-based website that allows dynamic exploration of the data by selective filtering. On average, the published studies fulfilled only one of the seven criteria of study design rigor. Only 15.2% of the studies used any validation strategy, and only 8% used the gold standard of independent cohort validation. Many studies also used small cohorts, e.g., for magnetic resonance imaging (MRI) and blood biomarker predictors, the median sample size was <100 subjects. We observed inverse relationships between reported effect sizes and the number of study design criteria fulfilled, expanding analogous reports from non-MS fields, that studies that fail to limit bias overestimate effect sizes. In conclusion, the presented meta-analysis represents a useful tool for researchers, reviewers, and funders to improve the design of future modeling studies in MS and to easily compare new studies with the published literature. We expect that this will accelerate research in this important area, leading to the development of robust models with proven clinical value.

Development of a Brain-Computer Interface-Based Symbol Digit Modalities Test and Validation in Healthy Elderly Volunteers and Stroke Patients

Standard cognitive assessment tools often involve motor or verbal responses, making them impossible for severely motor-disabled individuals. Brain-computer interfaces (BCIs) are expected to help severely motor-impaired individuals to perform cognitive assessment because BCIs can circumvent motor and verbal requirements. Currently, the field of research to develop cognitive tasks based on BCI is still in its nascent stage and needs further development. This study explored the possibility of developing a BCI version of symbol digit modalities test (BCI-SDMT). Steady-state visual evoked potential (SSVEP) was adopted to build the BCI and a 9-target SSVEP-BCI was realized to send examinees' responses. A training-free algorithm (i.e., filter bank canonical correlation analysis) was used for SSVEP identification. Thus, examinees are able to start the proposed BCI-SDMT immediately. Eighty-nine healthy elderly volunteers and 9 stroke patients were enrolled to validate the technical feasibility of the developed BCI-SDMT. For all participants, the average recognition accuracies of the developed BCI and BCI-SDMT were 93.89 ± 8.48% and 92.58 ± 10.52%, respectively, were considerably above the chance level (i.e., 11.11%). These results indicated that both healthy elderly volunteers and stroke patients could elicit sufficient SSVEPs to control the BCI. Furthermore, patient use of the developed BCI-SDMT was unaffected by the presence of motor impairment. They could understand instructions, pair numbers with specific symbols, and send commands using the BCI. The proposed BCI-SDMT can be used as a complement to the existing versions of the SDMT and has the potential to evaluate cognitive abilities in individuals with severe motor disabilities.

Using Smartphones to Reduce Research Burden in a Neurodegenerative Population and Assessing Participant Adherence: A Randomized Clinical Trial and Two Observational Studies

Background

Smartphone studies provide an opportunity to collect frequent data at a low burden on participants. Therefore, smartphones may enable data collection from people with progressive neurodegenerative diseases such as amyotrophic lateral sclerosis at high frequencies for a long duration. However, the progressive decline in patients’ cognitive and functional abilities could also hamper the feasibility of collecting patient-reported outcomes, audio recordings, and location data in the long term.

Objective

The aim of this study is to investigate the completeness of survey data, audio recordings, and passively collected location data from 3 smartphone-based studies of people with amyotrophic lateral sclerosis.

Methods

We analyzed data completeness in three studies: 2 observational cohort studies (study 1: N=22; duration=12 weeks and study 2: N=49; duration=52 weeks) and 1 clinical trial (study 3: N=49; duration=20 weeks). In these studies, participants were asked to complete weekly surveys; weekly audio recordings; and in the background, the app collected sensor data, including location data. For each of the three studies and each of the three data streams, we estimated time-to-discontinuation using the Kaplan–Meier method. We identified predictors of app discontinuation using Cox proportional hazards regression analysis. We quantified data completeness for both early dropouts and participants who remained engaged for longer.

Results

Time-to-discontinuation was shortest in the year-long observational study and longest in the clinical trial. After 3 months in the study, most participants still completed surveys and audio recordings: 77% (17/22) in study 1, 59% (29/49) in study 2, and 96% (22/23) in study 3. After 3 months, passively collected location data were collected for 95% (21/22), 86% (42/49), and 100% (23/23) of the participants. The Cox regression did not provide evidence that demographic characteristics or disease severity at baseline were associated with attrition, although it was somewhat underpowered. The mean data completeness was the highest for passively collected location data. For most participants, data completeness declined over time; mean data completeness was typically lower in the month before participants dropped out. Moreover, data completeness was lower for people who dropped out in the first study month (very few data points) compared with participants who adhered long term (data completeness fluctuating around 75%).

Conclusions

These three studies successfully collected smartphone data longitudinally from a neurodegenerative population. Despite patients’ progressive physical and cognitive decline, time-to-discontinuation was higher than in typical smartphone studies. Our study provides an important benchmark for participant engagement in a neurodegenerative population. To increase data completeness, collecting passive data (such as location data) and identifying participants who are likely to adhere during the initial phase of a study can be useful.

Trial Registration

ClinicalTrials.gov NCT03168711; https://clinicaltrials.gov/ct2/show/NCT03168711

Adult-Onset Leukoencephalopathy With Axonal Spheroids and Pigmented Glia: Review of Clinical Manifestations as Foundations for Therapeutic Development

A comprehensive review of published literature was conducted to elucidate the genetics, neuropathology, imaging findings, prevalence, clinical course, diagnosis/clinical evaluation, potential biomarkers, and current and proposed treatments for adult-onset leukoencephalopathy with axonal spheroids and pigmented glia (ALSP), a rare, debilitating, and life-threatening neurodegenerative disorder for which disease-modifying therapies are not currently available. Details on potential efficacy endpoints for future interventional clinical trials in patients with ALSP and data related to the burden of the disease on patients and caregivers were also reviewed. The information in this position paper lays a foundation to establish an effective clinical rationale and address the clinical gaps for creation of a robust strategy to develop therapeutic agents for ALSP, as well as design future clinical trials, that have clinically meaningful and convergent endpoints.

Assessment of Smartphone-Based Spiral Tracing in Multiple Sclerosis Reveals Intra-Individual Reproducibility as a Major Determinant of the Clinical Utility of the Digital Test

Technological advances, lack of medical professionals, high cost of face-to-face encounters, and disasters such as the COVID-19 pandemic fuel the telemedicine revolution. Numerous smartphone apps have been developed to measure neurological functions. However, their psychometric properties are seldom determined. It is unclear which designs underlie the eventual clinical utility of the smartphone tests. We have developed the smartphone Neurological Function Tests Suite (NeuFun-TS) and are systematically evaluating their psychometric properties against the gold standard of complete neurological examination digitalized into the NeurExTM app. This article examines the fifth and the most complex NeuFun-TS test, the "Spiral tracing." We generated 40 features in the training cohort (22 healthy donors [HD] and 89 patients with multiple sclerosis [MS]) and compared their intraclass correlation coefficient, fold change between HD and MS, and correlations with relevant clinical and imaging outcomes. We assembled the best features into machine-learning models and examined their performance in the independent validation cohort (45 patients with MS). We show that by involving multiple neurological functions, complex tests such as spiral tracing are susceptible to intra-individual variations, decreasing their reproducibility and clinical utility. Simple tests, reproducibly measuring single function(s) that can be aggregated to increase sensitivity, are preferable in app design.

A rapid assay provides on-site quantification of tetrahydrocannabinol in oral fluid

Tetrahydrocannabinol (THC), the primary psychoactive ingredient of cannabis, impairs cognitive and motor function in a concentration-dependent fashion. Drug testing is commonly performed for employment and law enforcement purposes; however, available tests produce low-sensitive binary results (lateral flow assays) or have long turnaround (gas chromatography–mass spectrometry). To enable on-site THC quantification in minutes, we developed a rapid assay for oral THC analysis called EPOCH (express probe for on-site cannabis inhalation). EPOCH features distinctive sensor design such as a radial membrane and transmission optics, all contained in a compact cartridge. This integrated approach permitted assay completion within 5 min with a detection limit of 0.17 ng/ml THC, which is below the regulatory guideline (1 ng/ml). As a proof of concept for field testing, we applied EPOCH to assess oral fluid samples from cannabis users (n = 43) and controls (n = 43). EPOCH detected oral THC in all specimens from cannabis smokers (median concentration, 478 ng/ml) and THC-infused food consumers. Longitudinal monitoring showed a fast drop in THC concentrations within the first 6 hours of cannabis smoking (half-life, 1.4 hours).

The potential use of digital health technologies in the African context: a systematic review of evidence from Ethiopia

The World Health Organization (WHO) recently put forth a Global Strategy on Digital Health 2020–2025 with several countries having already achieved key milestones. We aimed to understand whether and how digital health technologies (DHTs) are absorbed in Africa, tracking Ethiopia as a key node. We conducted a systematic review, searching PubMed-MEDLINE, Embase, ScienceDirect, African Journals Online, Cochrane Central Registry of Controlled Trials, ClinicalTrials.gov, and the WHO International Clinical Trials Registry Platform databases from inception to 02 February 2021 for studies of any design that investigated the potential of DHTs in clinical or public health practices in Ethiopia. This review was registered with PROSPERO (CRD42021240645) and it was designed to inform our ongoing DHT-enabled randomized controlled trial (RCT) (ClinicalTrials.gov ID: NCT04216420). We found 27,493 potentially relevant citations, among which 52 studies met the inclusion criteria, comprising a total of 596,128 patients, healthy individuals, and healthcare professionals. The studies involved six DHTs: mHealth (29 studies, 574,649 participants); electronic health records (13 studies, 4534 participants); telemedicine (4 studies, 465 participants); cloud-based application (2 studies, 2382 participants); information communication technology (3 studies, 681 participants), and artificial intelligence (1 study, 13,417 participants). The studies targeted six health conditions: maternal and child health (15), infectious diseases (14), non-communicable diseases (3), dermatitis (1), surgery (4), and general health conditions (15). The outcomes of interest were feasibility, usability, willingness or readiness, effectiveness, quality improvement, and knowledge or attitude toward DHTs. Five studies involved RCTs. The analysis showed that although DHTs are a relatively recent phenomenon in Ethiopia, their potential harnessing clinical and public health practices are highly visible. Their adoption and implementation in full capacity require more training, access to better devices such as smartphones, and infrastructure. DHTs hold much promise tackling major clinical and public health backlogs and strengthening the healthcare ecosystem in Ethiopia. More RCTs are needed on emerging DHTs including artificial intelligence, big data, cloud, cybersecurity, telemedicine, and wearable devices to provide robust evidence of their potential use in such settings and to materialize the WHO’s Global Strategy on Digital Health.

A smartphone sensor-based digital outcome assessment of multiple sclerosis

Background:

Sensor-based monitoring tools fill a critical gap in multiple sclerosis (MS) research and clinical care.

Objective:

The aim of this study is to assess performance characteristics of the Floodlight Proof-of-Concept (PoC) app.

Methods:

In a 24-week study (clinicaltrials.gov: NCT02952911), smartphone-based active tests and passive monitoring assessed cognition (electronic Symbol Digit Modalities Test), upper extremity function (Pinching Test, Draw a Shape Test), and gait and balance (Static Balance Test, U-Turn Test, Walk Test, Passive Monitoring). Intraclass correlation coefficients (ICCs) and age- or sex-adjusted Spearman’s rank correlation determined test–retest reliability and correlations with clinical and magnetic resonance imaging (MRI) outcome measures, respectively.

Results:

Seventy-six people with MS (PwMS) and 25 healthy controls were enrolled. In PwMS, ICCs were moderate-to-good (ICC(2,1) = 0.61–0.85) across tests. Correlations with domain-specific standard clinical disability measures were significant for all tests in the cognitive (r = 0.82, p < 0.001), upper extremity function (|r|= 0.40–0.64, all p < 0.001), and gait and balance domains (r = −0.25 to −0.52, all p < 0.05; except for Static Balance Test: r = −0.20, p > 0.05). Most tests also correlated with Expanded Disability Status Scale, 29-item Multiple Sclerosis Impact Scale items or subscales, and/or normalized brain volume.

Conclusion:

The Floodlight PoC app captures reliable and clinically relevant measures of functional impairment in MS, supporting its potential use in clinical research and practice.