Smartphone-based remote assessment of upper extremity function for multiple sclerosis using the Draw a Shape Test

Investigating Measurement Equivalence of Smartphone Sensor-Based Assessments: Remote, Digital, Bring-Your-Own-Device Study

BACKGROUND

Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population.

OBJECTIVE

This study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models.

METHODS

Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a "bring-your-own-device" (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor-based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status.

RESULTS

Overall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03).

CONCLUSIONS

Consistent with the hypothesis that smartphone sensor-based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.

Using a tablet to understand the spatial and temporal characteristics of complex upper limb movements in chronic stroke

Robotic devices are commonly used to quantify sensorimotor function of the upper limb after stroke; however, the availability and cost of such devices make it difficult to facilitate implementation in clinical environments. Tablets (e.g. iPad) can be used as devices to facilitate rehabilitation but are rarely used as assessment tools for the upper limb. The current study aimed to implement a tablet-based Maze Navigation Task to examine complex upper-limb movement in individuals with chronic stroke. We define complex upper-limb movement as reaching movements that require multi-joint coordination in a dynamic environment. We predicted that individuals with stroke would have more significant spatial errors, longer movement times, and slower speeds compared to controls with increasing task complexity. Twenty individuals with chronic stroke who had a variety of arm and hand function (Upper extremity Fugl-Myer 52.8 ± 18.3) and twenty controls navigated eight pseudorandomized mazes on an iPad using a digitizing stylus. The task was designed to elicit reaching movements engaging both the shoulder and elbow joints. Each maze became increasingly complex by increasing the number of 90° turns. We instructed participants to navigate each maze as quickly and accurately as possible while avoiding the maze's boundaries. Sensorimotor behavior was quantified using the following metrics: Error Time (time spent hitting or outside boundaries), Peak Speed, Average Speed, and Movement Time, Number of Speed Peaks. We found that individuals with stroke had significantly greater Error Time for all maze levels (all, p < 0.01), while both speed metrics, Movement Time and Number of Speed Peaks were significantly lower for several levels (all, p < 0.05). As maze complexity increased, the performance of individuals with stroke worsened only for Error Time while control performance remained consistent (p < 0.001). Our results indicate that a complex movement task on a tablet can capture temporal and spatial impairments in individuals with stroke, as well as how task complexity impacts movement quality. This work demonstrates that a tablet is a suitable tool for the assessment of complex movement after stroke and can serve to inform rehabilitation after stroke.

Sensor-Derived Measures of Motor and Cognitive Functions in People With Multiple Sclerosis Using Unsupervised Smartphone-Based Assessments: Proof-of-Concept Study

BACKGROUND

Smartphones and wearables are revolutionizing the assessment of cognitive and motor function in neurological disorders, allowing for objective, frequent, and remote data collection. However, these assessments typically provide a plethora of sensor-derived measures (SDMs), and selecting the most suitable measure for a given context of use is a challenging, often overlooked problem.

OBJECTIVE

This analysis aims to develop and apply an SDM selection framework, including automated data quality checks and the evaluation of statistical properties, to identify robust SDMs that describe the cognitive and motor function of people with multiple sclerosis (MS).

METHODS

The proposed framework was applied to data from a cross-sectional study involving 85 people with MS and 68 healthy participants who underwent in-clinic supervised and remote unsupervised smartphone-based assessments. The assessment provided high-quality recordings from cognitive, manual dexterity, and mobility tests, from which 47 SDMs, based on established literature, were extracted using previously developed and publicly available algorithms. These SDMs were first separately and then jointly screened for bias and normality by 2 expert assessors. Selected SDMs were then analyzed to establish their reliability, using an intraclass correlation coefficient and minimal detectable change at 95% CI. The convergence of selected SDMs with in-clinic MS functional measures and patient-reported outcomes was also evaluated.

RESULTS

A total of 16 (34%) of the 47 SDMs passed the selection framework. All selected SDMs demonstrated moderate-to-good reliability in remote settings (intraclass correlation coefficient 0.5-0.85; minimal detectable change at 95% CI 19%-35%). Selected SDMs extracted from the smartphone-based cognitive test demonstrated good-to-excellent correlation (Spearman correlation coefficient, |ρ|>0.75) with the in-clinic Symbol Digit Modalities Test and fair correlation with Expanded Disability Status Scale (EDSS) scores (0.25≤|ρ|<0.5). SDMs extracted from the manual dexterity tests showed either fair correlation (0.25≤|ρ|<0.5) or were not correlated (|ρ|<0.25) with the in-clinic 9-hole peg test and EDSS scores. Most selected SDMs from mobility tests showed fair correlation with the in-clinic timed 25-foot walk test and fair to moderate-to-good correlation (0.5<|ρ|≤0.75) with EDSS scores. SDM correlations with relevant patient-reported outcomes varied by functional domain, ranging from not correlated (cognitive test SDMs) to good-to-excellent correlation (|ρ|>0.75) for mobility test SDMs. Overall, correlations were similar when smartphone-based tests were performed in a clinic or remotely.

CONCLUSIONS

Reported results highlight that smartphone-based assessments are suitable tools to remotely obtain high-quality SDMs of cognitive and motor function in people with MS. The presented SDM selection framework promises to increase the interpretability and standardization of smartphone-based SDMs in people with MS, paving the way for their future use in interventional trials.

Smartphone-Based Hand Function Assessment: Systematic Review

BACKGROUND

Hand function assessment heavily relies on specific task scenarios, making it challenging to ensure validity and reliability. In addition, the wide range of assessment tools, limited and expensive data recording, and analysis systems further aggravate the issue. However, smartphones provide a promising opportunity to address these challenges. Thus, the built-in, high-efficiency sensors in smartphones can be used as effective tools for hand function assessment.

OBJECTIVE

This review aims to evaluate existing studies on hand function evaluation using smartphones.

METHODS

An information specialist searched 8 databases on June 8, 2023. The search criteria included two major concepts: (1) smartphone or mobile phone or mHealth and (2) hand function or function assessment. Searches were limited to human studies in the English language and excluded conference proceedings and trial register records. Two reviewers independently screened all studies, with a third reviewer involved in resolving discrepancies. The included studies were rated according to the Mixed Methods Appraisal Tool. One reviewer extracted data on publication, demographics, hand function types, sensors used for hand function assessment, and statistical or machine learning (ML) methods. Accuracy was checked by another reviewer. The data were synthesized and tabulated based on each of the research questions.

RESULTS

In total, 46 studies were included. Overall, 11 types of hand dysfunction-related problems were identified, such as Parkinson disease, wrist injury, stroke, and hand injury, and 6 types of hand dysfunctions were found, namely an abnormal range of motion, tremors, bradykinesia, the decline of fine motor skills, hypokinesia, and nonspecific dysfunction related to hand arthritis. Among all built-in smartphone sensors, the accelerometer was the most used, followed by the smartphone camera. Most studies used statistical methods for data processing, whereas ML algorithms were applied for disease detection, disease severity evaluation, disease prediction, and feature aggregation.

CONCLUSIONS

This systematic review highlights the potential of smartphone-based hand function assessment. The review suggests that a smartphone is a promising tool for hand function evaluation. ML is a conducive method to classify levels of hand dysfunction. Future research could (1) explore a gold standard for smartphone-based hand function assessment and (2) take advantage of smartphones' multiple built-in sensors to assess hand function comprehensively, focus on developing ML methods for processing collected smartphone data, and focus on real-time assessment during rehabilitation training. The limitations of the research are 2-fold. First, the nascent nature of smartphone-based hand function assessment led to limited relevant literature, affecting the evidence's completeness and comprehensiveness. This can hinder supporting viewpoints and drawing conclusions. Second, literature quality varies due to the exploratory nature of the topic, with potential inconsistencies and a lack of high-quality reference studies and meta-analyses.

Digital health technologies and machine learning augment patient reported outcomes to remotely characterise rheumatoid arthritis

Digital measures of health status captured during daily life could greatly augment current in-clinic assessments for rheumatoid arthritis (RA), to enable better assessment of disease progression and impact. This work presents results from weaRAble-PRO, a 14-day observational study, which aimed to investigate how digital health technologies (DHT), such as smartphones and wearables, could augment patient reported outcomes (PRO) to determine RA status and severity in a study of 30 moderate-to-severe RA patients, compared to 30 matched healthy controls (HC). Sensor-based measures of health status, mobility, dexterity, fatigue, and other RA specific symptoms were extracted from daily iPhone guided tests (GT), as well as actigraphy and heart rate sensor data, which was passively recorded from patients' Apple smartwatch continuously over the study duration. We subsequently developed a machine learning (ML) framework to distinguish RA status and to estimate RA severity. It was found that daily wearable sensor-outcomes robustly distinguished RA from HC participants (F1, 0.807). Furthermore, by day 7 of the study (half-way), a sufficient volume of data had been collected to reliably capture the characteristics of RA participants. In addition, we observed that the detection of RA severity levels could be improved by augmenting standard patient reported outcomes with sensor-based features (F1, 0.833) in comparison to using PRO assessments alone (F1, 0.759), and that the combination of modalities could reliability measure continuous RA severity, as determined by the clinician-assessed RAPID-3 score at baseline (r2, 0.692; RMSE, 1.33). The ability to measure the impact of the disease during daily life-through objective and remote digital outcomes-paves the way forward to enable the development of more patient-centric and personalised measurements for use in RA clinical trials.

Upper limb intention tremor assessment: opportunities and challenges in wearable technology

BACKGROUND

Tremors are involuntary rhythmic movements commonly present in neurological diseases such as Parkinson's disease, essential tremor, and multiple sclerosis. Intention tremor is a subtype associated with lesions in the cerebellum and its connected pathways, and it is a common symptom in diseases associated with cerebellar pathology. While clinicians traditionally use tests to identify tremor type and severity, recent advancements in wearable technology have provided quantifiable ways to measure movement and tremor using motion capture systems, app-based tasks and tools, and physiology-based measurements. However, quantifying intention tremor remains challenging due to its changing nature.

METHODOLOGY & RESULTS

This review examines the current state of upper limb tremor assessment technology and discusses potential directions to further develop new and existing algorithms and sensors to better quantify tremor, specifically intention tremor. A comprehensive search using PubMed and Scopus was performed using keywords related to technologies for tremor assessment. Afterward, screened results were filtered for relevance and eligibility and further classified into technology type. A total of 243 publications were selected for this review and classified according to their type: body function level: movement-based, activity level: task and tool-based, and physiology-based. Furthermore, each publication's methods, purpose, and technology are summarized in the appendix table.

CONCLUSIONS

Our survey suggests a need for more targeted tasks to evaluate intention tremors, including digitized tasks related to intentional movements, neurological and physiological measurements targeting the cerebellum and its pathways, and signal processing techniques that differentiate voluntary from involuntary movement in motion capture systems.

Smartphone-Based Assessment of Mobility and Manual Dexterity in Adult People with Spinal Muscular Atrophy

Background

More responsive, reliable, and clinically valid endpoints of disability are essential to reduce size, duration, and burden of clinical trials in adult persons with spinal muscular atrophy (aPwSMA).

Objective

The aim is to investigate the feasibility of smartphone-based assessments in aPwSMA and provide evidence on the reliability and construct validity of sensor-derived measures (SDMs) of mobility and manual dexterity collected remotely in aPwSMA.

Methods

Data were collected from 59 aPwSMA (23 walkers, 20 sitters and 16 non-sitters) and 30 age-matched healthy controls (HC). SDMs were extracted from five smartphone-based tests capturing mobility and manual dexterity, which were administered in-clinic and remotely in daily life for four weeks. Reliability (Intraclass Correlation Coefficients, ICC) and construct validity (ability to discriminate between HC and aPwSMA and correlations with Revised Upper Limb Module, RULM and Hammersmith Functional Scale - Expanded HFMSE) were quantified for all SDMs.

Results

The smartphone-based assessments proved feasible, with 92.1% average adherence in aPwSMA. The SDMs allowed to reliably assess both mobility and dexterity (ICC > 0.75 for 14/22 SDMs). Twenty-one out of 22 SDMs significantly discriminated between HC and aPwSMA. The highest correlations with the RULM were observed for SDMs from the manual dexterity tests in both non-sitters (Typing, ρ= 0.78) and sitters (Pinching, ρ= 0.75). In walkers, the highest correlation was between mobility tests and HFMSE (5 U-Turns, ρ= 0.79).

Conclusions

This exploratory study provides preliminary evidence for the usability of smartphone-based assessments of mobility and manual dexterity in aPwSMA when deployed remotely in participants' daily life. Reliability and construct validity of SDMs remotely collected in real-life was demonstrated, which is a pre-requisite for their use in longitudinal trials. Additionally, three novel smartphone-based performance outcome assessments were successfully established for aPwSMA. Upon further validation of responsiveness to interventions, this technology holds potential to increase the efficiency of clinical trials in aPwSMA.

Wearable Sensor Technologies to Assess Motor Functions in People With Multiple Sclerosis: Systematic Scoping Review and Perspective

BACKGROUND

Wearable sensor technologies have the potential to improve monitoring in people with multiple sclerosis (MS) and inform timely disease management decisions. Evidence of the utility of wearable sensor technologies in people with MS is accumulating but is generally limited to specific subgroups of patients, clinical or laboratory settings, and functional domains.

OBJECTIVE

This review aims to provide a comprehensive overview of all studies that have used wearable sensors to assess, monitor, and quantify motor function in people with MS during daily activities or in a controlled laboratory setting and to shed light on the technological advances over the past decades.

METHODS

We systematically reviewed studies on wearable sensors to assess the motor performance of people with MS. We scanned PubMed, Scopus, Embase, and Web of Science databases until December 31, 2022, considering search terms "multiple sclerosis" and those associated with wearable technologies and included all studies assessing motor functions. The types of results from relevant studies were systematically mapped into 9 predefined categories (association with clinical scores or other measures; test-retest reliability; group differences, 3 types; responsiveness to change or intervention; and acceptability to study participants), and the reporting quality was determined through 9 questions. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting guidelines.

RESULTS

Of the 1251 identified publications, 308 were included: 176 (57.1%) in a real-world context, 107 (34.7%) in a laboratory context, and 25 (8.1%) in a mixed context. Most publications studied physical activity (196/308, 63.6%), followed by gait (81/308, 26.3%), dexterity or tremor (38/308, 12.3%), and balance (34/308, 11%). In the laboratory setting, outcome measures included (in addition to clinical severity scores) 2- and 6-minute walking tests, timed 25-foot walking test, timed up and go, stair climbing, balance tests, and finger-to-nose test, among others. The most popular anatomical landmarks for wearable placement were the waist, wrist, and lower back. Triaxial accelerometers were most commonly used (229/308, 74.4%). A surge in the number of sensors embedded in smartphones and smartwatches has been observed. Overall, the reporting quality was good.

CONCLUSIONS

Continuous monitoring with wearable sensors could optimize the management of people with MS, but some hurdles still exist to full clinical adoption of digital monitoring. Despite a possible publication bias and vast heterogeneity in the outcomes reported, our review provides an overview of the current literature on wearable sensor technologies used for people with MS and highlights shortcomings, such as the lack of harmonization, transparency in reporting methods and results, and limited data availability for the research community. These limitations need to be addressed for the growing implementation of wearable sensor technologies in clinical routine and clinical trials, which is of utmost importance for further progress in clinical research and daily management of people with MS.

TRIAL REGISTRATION

PROSPERO CRD42021243249; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=243249.

Personalized Longitudinal Assessment of Multiple Sclerosis Using Smartphones

Personalized longitudinal disease assessment is central to quickly diagnosing, appropriately managing, and optimally adapting the therapeutic strategy of multiple sclerosis (MS). It is also important for identifying idiosyncratic subject-specific disease profiles. Here, we design a novel longitudinal model to map individual disease trajectories in an automated way using smartphone sensor data that may contain missing values. First, we collect digital measurements related to gait and balance, and upper extremity functions using sensor-based assessments administered on a smartphone. Next, we treat missing data via imputation. We then discover potential markers of MS by employing a generalized estimation equation. Subsequently, parameters learned from multiple training datasets are ensembled to form a simple, unified longitudinal predictive model to forecast MS over time in previously unseen people with MS. To mitigate potential underestimation for individuals with severe disease scores, the final model incorporates additional subject-specific fine-tuning using data from the first day. The results show that the proposed model is promising to achieve personalized longitudinal MS assessment; they also suggest that features related to gait and balance as well as upper extremity function, remotely collected from sensor-based assessments, may be useful digital markers for predicting MS over time.

CNN-LSTM Model for Recognizing Video-Recorded Actions Performed in a Traditional Chinese Exercise

Identifying human actions from video data is an important problem in the fields of intelligent rehabilitation assessment. Motion feature extraction and pattern recognition are the two key procedures to achieve such goals. Traditional action recognition models are usually based on the geometric features manually extracted from video frames, which are however difficult to adapt to complex scenarios and cannot achieve high-precision recognition and robustness. We investigate a motion recognition model and apply it to recognize the sequence of complicated actions of a traditional Chinese exercise (ie, Baduanjin). We first developed a combined convolutional neural network (CNN) and long short-term memory (LSTM) model for recognizing the sequence of actions captured in video frames, and applied it to recognize the actions of Baduanjin. Moreover, this method has been compared with the traditional action recognition model based on geometric motion features in which Openpose is used to identify the joint positions in the skeletons. Its performance of high recognition accuracy has been verified on the testing video dataset, containing the video clips from 18 different practicers. The CNN-LSTM recognition model achieved 96.43% accuracy on the testing set; while those manually extracted features in the traditional action recognition model were only able to achieve 66.07% classification accuracy on the testing video dataset. The abstract image features extracted by the CNN module are more effective on improving the classification accuracy of the LSTM model. The proposed CNN-LSTM based method can be a useful tool in recognizing the complicated actions.

Accuracy and Reliability of At-home Quantification of Motor Impairments Using a Computer-based Pointing Task with Children with Ataxia-Telangiectasia

Methods for obtaining accurate quantitative assessments of motor impairments are essential in accessibility research, design of adaptive ability-based assistive technologies, as well as in clinical care and medical research. Currently, such assessments are typically performed in controlled laboratory or clinical settings under professional supervision. Emerging approaches for collecting data in unsupervised settings have been shown to produce valid data when aggregated over large populations, but it is not yet established if in unsupervised settings measures of research or clinical significance can be collected accurately and reliably for individuals. We conducted a study with 13 children with ataxia-telangiectasia and 9 healthy children to analyze the validity, test-retest reliability, and acceptability of at-home use of a recent active digital phenotyping system, called Hevelius. Hevelius produces 32 measures derived from the movement trajectories of the mouse cursor, and it produces a quantitative estimate of motor impairment in the dominant arm using the dominant arm component of the Brief Ataxia Rating Scale (BARS). The severity score estimates generated by Hevelius from single at-home sessions deviated from clinician-assigned BARS scores more than the severity score estimates generated from single sessions conducted under researcher supervision. However, taking a median of as few as 2 consecutive sessions produced severity score estimates that were as accurate or better than the estimates produced from single supervised sessions. Further, aggregating as few as 2 consecutive sessions resulted in good test-retest reliability (ICC = 0.81 for A-T participants). This work demonstrated the feasibility of performing accurate and reliable quantitative assessments of individual motor impairments in the dominant arm through tasks performed at home without supervision by the researchers. Further work is needed, however, to assess how broadly these results generalize.

Preliminary validity of the Draw a Shape Test for upper extremity assessment in multiple sclerosis

OBJECTIVE

To validate the smartphone sensor-based Draw a Shape Test - a part of the Floodlight Proof-of-Concept app for remotely assessing multiple sclerosis-related upper extremity impairment by tracing six different shapes.

METHODS

People with multiple sclerosis, classified functionally normal/abnormal via their Nine-Hole Peg Test time, and healthy controls participated in a 24-week, nonrandomized study. Spatial (trace accuracy), temporal (mean and variability in linear, angular, and radial drawing velocities, and dwell time ratio), and spatiotemporal features (trace celerity) were cross-sectionally analyzed for correlation with standard clinical and brain magnetic resonance imaging (normalized brain volume and total lesion volume) disease burden measures, and for capacity to differentiate people with multiple sclerosis from healthy controls.

RESULTS

Data from 69 people with multiple sclerosis and 18 healthy controls were analyzed. Trace accuracy (all shapes), linear velocity variability (circle, figure-of-8, spiral shapes), and radial velocity variability (spiral shape) had a mostly fair/moderate-to-good correlation (|r| = 0.14-0.66) with all disease burden measures. Trace celerity also had mostly fair/moderate-to-good correlation (|r| = 0.18-0.41) with Nine-Hole Peg Test performance, cerebellar functional system score, and brain magnetic resonance imaging. Furthermore, partial correlation analysis related these results to motor impairment. People with multiple sclerosis showed greater drawing velocity variability, though slower mean velocity, than healthy controls. Linear velocity (spiral shape) and angular velocity (circle shape) potentially differentiate functionally normal people with multiple sclerosis from healthy controls.

INTERPRETATION

The Draw a Shape Test objectively assesses upper extremity impairment and correlates with all disease burden measures, thus aiding multiple sclerosis-related upper extremity impairment characterization.

Longitudinal Trend Monitoring of Multiple Sclerosis Ambulation Using Smartphones

Goal: Smartphone and wearable devices may act as powerful tools to remotely monitor physical function in people with neurodegenerative and autoimmune diseases from out-of-clinic environments. Detection of progression onset or worsening of symptoms is especially important in people living with multiple sclerosis (PwMS) in order to enable optimally adapted therapeutic strategies. MS symptoms typically follow subtle and fluctuating disease courses, patient-to-patient, and over time. Current in-clinic assessments are often too infrequently administered to reflect longitudinal changes in MS impairment that impact daily life. This work, therefore, explores how smartphones can administer daily two-minute walking assessments to monitor PwMS physical function at home. Methods: Remotely collected smartphone inertial sensor data was transformed through state-of-the-art Deep Convolutional Neural Networks, to estimate a participant's daily ambulatory-related disease severity, longitudinally over a 24-week study. Results: This study demonstrated that smartphone-based ambulatory severity outcomes could accurately estimate MS level of disability, as measured by the EDSS score ([Formula: see text]: 0.56,[Formula: see text]0.001). Furthermore, longitudinal severity outcomes were shown to accurately reflect individual participants' level of disability over the study duration. Conclusion: Smartphone-based assessments, that can be performed by patients from their home environments, could greatly augment standard in-clinic outcomes for neurodegenerative diseases. The ability to understand the impact of disease on daily-life between clinical visits, through objective digital outcomes, paves the way forward to better measure and identify signs of disease progression that may be occurring out-of-clinic, to monitor how different patients respond to various treatments, and to ultimately enable the development of better, and more personalised care.

Safety of low-intensity repetitive transcranial magneTic brAin stimUlation foR people living with mUltiple Sclerosis (TAURUS): study protocol for a randomised controlled trial

BACKGROUND

Multiple sclerosis (MS) is an inflammatory and neurodegenerative disease, characterised by oligodendrocyte death and demyelination. Oligodendrocyte progenitor cells can differentiate into new replacement oligodendrocytes; however, remyelination is insufficient to protect neurons from degeneration in people with MS. We previously reported that 4 weeks of daily low-intensity repetitive transcranial magnetic stimulation (rTMS) in an intermittent theta-burst stimulation (iTBS) pattern increased the number of new myelinating oligodendrocytes in healthy adult mice. This study translates this rTMS protocol and aims to determine its safety and tolerability for people living with MS. We will also perform magnetic resonance imaging (MRI) and symptom assessments as preliminary indicators of myelin addition following rTMS.

METHODS

Participants (N = 30, aged 18-65 years) will have a diagnosis of relapsing-remitting or secondary progressive MS. ≤2 weeks before the intervention, eligible, consenting participants will complete a physical exam, baseline brain MRI scan and participant-reported MS symptom assessments [questionnaires: Fatigue Severity Scale, Quality of Life (AQoL-8D), Hospital Anxiety and Depression Scale; and smartphone-based measures of cognition (electronic symbol digit modalities test), manual dexterity (pinching test, draw a shape test) and gait (U-Turn test)]. Participants will be pseudo-randomly allocated to rTMS (n=20) or sham (placebo; n=10), stratified by sex. rTMS or sham will be delivered 5 days per week for 4 consecutive weeks (20 sessions, 6 min per day). rTMS will be applied using a 90-mm circular coil at low-intensity (25% maximum stimulator output) in an iTBS pattern. For sham, the coil will be oriented 90° to the scalp, preventing the magnetic field from stimulating the brain. Adverse events will be recorded daily. We will evaluate participant blinding after the first, 10th and final session. After the final session, participants will repeat symptom assessments and brain MRI, for comparison with baseline. Participant-reported assessments will be repeated at 4-month post-allocation follow-up.

DISCUSSION

This study will determine whether this rTMS protocol is safe and tolerable for people with MS. MRI and participant-reported symptom assessments will serve as preliminary indications of rTMS efficacy for myelin addition to inform further studies.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12619001196134 . Registered on 27 August 2019.

Reliability and validity of the Roche PD Mobile Application for remote monitoring of early Parkinson's disease

Digital health technologies enable remote and therefore frequent measurement of motor signs, potentially providing reliable and valid estimates of motor sign severity and progression in Parkinson’s disease (PD). The Roche PD Mobile Application v2 was developed to measure bradykinesia, bradyphrenia and speech, tremor, gait and balance. It comprises 10 smartphone active tests (with ½ tests administered daily), as well as daily passive monitoring via a smartphone and smartwatch. It was studied in 316 early-stage PD participants who performed daily active tests at home then carried a smartphone and wore a smartwatch throughout the day for passive monitoring (study NCT03100149). Here, we report baseline data. Adherence was excellent (96.29%). All pre-specified sensor features exhibited good-to-excellent test–retest reliability (median intraclass correlation coefficient = 0.9), and correlated with corresponding Movement Disorder Society–Unified Parkinson's Disease Rating Scale items (rho: 0.12–0.71). These findings demonstrate the preliminary reliability and validity of remote at-home quantification of motor sign severity with the Roche PD Mobile Application v2 in individuals with early PD.

Remote Assessments of Hand Function in Neurological Disorders: Systematic Review

BACKGROUND

Loss of fine motor skills is observed in many neurological diseases, and remote monitoring assessments can aid in early diagnosis and intervention. Hand function can be regularly assessed to monitor loss of fine motor skills in people with central nervous system disorders; however, there are challenges to in-clinic assessments. Remotely assessing hand function could facilitate monitoring and supporting of early diagnosis and intervention when warranted.

OBJECTIVE

Remote assessments can facilitate the tracking of limitations, aiding in early diagnosis and intervention. This study aims to systematically review existing evidence regarding the remote assessment of hand function in populations with chronic neurological dysfunction.

METHODS

PubMed and MEDLINE, CINAHL, Web of Science, and Embase were searched for studies that reported remote assessment of hand function (ie, outside of traditional in-person clinical settings) in adults with chronic central nervous system disorders. We excluded studies that included participants with orthopedic upper limb dysfunction or used tools for intervention and treatment. We extracted data on the evaluated hand function domains, validity and reliability, feasibility, and stage of development.

RESULTS

In total, 74 studies met the inclusion criteria for Parkinson disease (n=57, 77% studies), stroke (n=9, 12%), multiple sclerosis (n=6, 8%), spinal cord injury (n=1, 1%), and amyotrophic lateral sclerosis (n=1, 1%). Three assessment modalities were identified: external device (eg, wrist-worn accelerometer), smartphone or tablet, and telerehabilitation. The feasibility and overall participant acceptability were high. The most common hand function domains assessed included finger tapping speed (fine motor control and rigidity), hand tremor (pharmacological and rehabilitation efficacy), and finger dexterity (manipulation of small objects required for daily tasks) and handwriting (coordination). Although validity and reliability data were heterogeneous across studies, statistically significant correlations with traditional in-clinic metrics were most commonly reported for telerehabilitation and smartphone or tablet apps. The most readily implementable assessments were smartphone or tablet-based.

CONCLUSIONS

The findings show that remote assessment of hand function is feasible in neurological disorders. Although varied, the assessments allow clinicians to objectively record performance in multiple hand function domains, improving the reliability of traditional in-clinic assessments. Remote assessments, particularly via telerehabilitation and smartphone- or tablet-based apps that align with in-clinic metrics, facilitate clinic to home transitions, have few barriers to implementation, and prompt remote identification and treatment of hand function impairments.

Assessment of Smartphone-Based Spiral Tracing in Multiple Sclerosis Reveals Intra-Individual Reproducibility as a Major Determinant of the Clinical Utility of the Digital Test

Technological advances, lack of medical professionals, high cost of face-to-face encounters, and disasters such as the COVID-19 pandemic fuel the telemedicine revolution. Numerous smartphone apps have been developed to measure neurological functions. However, their psychometric properties are seldom determined. It is unclear which designs underlie the eventual clinical utility of the smartphone tests. We have developed the smartphone Neurological Function Tests Suite (NeuFun-TS) and are systematically evaluating their psychometric properties against the gold standard of complete neurological examination digitalized into the NeurExTM app. This article examines the fifth and the most complex NeuFun-TS test, the "Spiral tracing." We generated 40 features in the training cohort (22 healthy donors [HD] and 89 patients with multiple sclerosis [MS]) and compared their intraclass correlation coefficient, fold change between HD and MS, and correlations with relevant clinical and imaging outcomes. We assembled the best features into machine-learning models and examined their performance in the independent validation cohort (45 patients with MS). We show that by involving multiple neurological functions, complex tests such as spiral tracing are susceptible to intra-individual variations, decreasing their reproducibility and clinical utility. Simple tests, reproducibly measuring single function(s) that can be aggregated to increase sensitivity, are preferable in app design.

Digital Biomarkers in Multiple Sclerosis

For incurable diseases, such as multiple sclerosis (MS), the prevention of progression and the preservation of quality of life play a crucial role over the entire therapy period. In MS, patients tend to become ill at a younger age and are so variable in terms of their disease course that there is no standard therapy. Therefore, it is necessary to enable a therapy that is as personalized as possible and to respond promptly to any changes, whether with noticeable symptoms or symptomless. Here, measurable parameters of biological processes can be used, which provide good information with regard to prognostic and diagnostic aspects, disease activity and response to therapy, so-called biomarkers Increasing digitalization and the availability of easy-to-use devices and technology also enable healthcare professionals to use a new class of digital biomarkers-digital health technologies-to explain, influence and/or predict health-related outcomes. The technology and devices from which these digital biomarkers stem are quite broad, and range from wearables that collect patients' activity during digitalized functional tests (e.g., the Multiple Sclerosis Performance Test, dual-tasking performance and speech) to digitalized diagnostic procedures (e.g., optical coherence tomography) and software-supported magnetic resonance imaging evaluation. These technologies offer a timesaving way to collect valuable data on a regular basis over a long period of time, not only once or twice a year during patients' routine visit at the clinic. Therefore, they lead to real-life data acquisition, closer patient monitoring and thus a patient dataset useful for precision medicine. Despite the great benefit of such increasing digitalization, for now, the path to implementing digital biomarkers is widely unknown or inconsistent. Challenges around validation, infrastructure, evidence generation, consistent data collection and analysis still persist. In this narrative review, we explore existing and future opportunities to capture clinical digital biomarkers in the care of people with MS, which may lead to a digital twin of the patient. To do this, we searched published papers for existing opportunities to capture clinical digital biomarkers for different functional systems in the context of MS, and also gathered perspectives on digital biomarkers under development or already existing as a research approach.

Developing a Digital Solution for Remote Assessment in Multiple Sclerosis: From Concept to Software as a Medical Device

There is increasing interest in the development and deployment of digital solutions to improve patient care and facilitate monitoring in medical practice, e.g., by remote observation of disease symptoms in the patients’ home environment. Digital health solutions today range from non-regulated wellness applications and research-grade exploratory instruments to regulated software as a medical device (SaMD). This paper discusses the considerations and complexities in developing innovative, effective, and validated SaMD for multiple sclerosis (MS). The development of SaMD requires a formalised approach (design control), inclusive of technical verification and analytical validation to ensure reliability. SaMD must be clinically evaluated, characterised for benefit and risk, and must conform to regulatory requirements associated with device classification. Cybersecurity and data privacy are also critical. Careful consideration of patient and provider needs throughout the design and testing process help developers overcome challenges of adoption in medical practice. Here, we explore the development pathway for SaMD in MS, leveraging experiences from the development of Floodlight™ MS, a continually evolving bundled solution of SaMD for remote functional assessment of MS. The development process will be charted while reflecting on common challenges in the digital space, with a view to providing insights for future developers.

A smartphone sensor-based digital outcome assessment of multiple sclerosis

Background:

Sensor-based monitoring tools fill a critical gap in multiple sclerosis (MS) research and clinical care.

Objective:

The aim of this study is to assess performance characteristics of the Floodlight Proof-of-Concept (PoC) app.

Methods:

In a 24-week study (clinicaltrials.gov: NCT02952911), smartphone-based active tests and passive monitoring assessed cognition (electronic Symbol Digit Modalities Test), upper extremity function (Pinching Test, Draw a Shape Test), and gait and balance (Static Balance Test, U-Turn Test, Walk Test, Passive Monitoring). Intraclass correlation coefficients (ICCs) and age- or sex-adjusted Spearman’s rank correlation determined test–retest reliability and correlations with clinical and magnetic resonance imaging (MRI) outcome measures, respectively.

Results:

Seventy-six people with MS (PwMS) and 25 healthy controls were enrolled. In PwMS, ICCs were moderate-to-good (ICC(2,1) = 0.61–0.85) across tests. Correlations with domain-specific standard clinical disability measures were significant for all tests in the cognitive (r = 0.82, p < 0.001), upper extremity function (|r|= 0.40–0.64, all p < 0.001), and gait and balance domains (r = −0.25 to −0.52, all p < 0.05; except for Static Balance Test: r = −0.20, p > 0.05). Most tests also correlated with Expanded Disability Status Scale, 29-item Multiple Sclerosis Impact Scale items or subscales, and/or normalized brain volume.

Conclusion:

The Floodlight PoC app captures reliable and clinically relevant measures of functional impairment in MS, supporting its potential use in clinical research and practice.

Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones

The emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8-15%. A lack of transparency of "black-box" deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.

Evaluation, Acceptance, and Qualification of Digital Measures: From Proof of Concept to Endpoint

To support the successful adoption of digital measures into internal decision making and evidence generation for medical product development, we present a unified lexicon to aid communication throughout this process, and highlight key concepts including the critical role of participant engagement in development of digital measures. We detail the steps of bringing a successful proof of concept to scale, focusing on key decisions in the development of a new digital measure: asking the right question, optimized approaches to evaluating new measures, and whether and how to pursue qualification or acceptance. Building on the V3 framework for establishing verification and analytical and clinical validation, we discuss strategic and practical considerations for collecting this evidence, illustrated with concrete examples of trailblazing digital measures in the field.

Smartphone- and Smartwatch-Based Remote Characterisation of Ambulation in Multiple Sclerosis During the Two-Minute Walk Test

Leveraging consumer technology such as smartphone and smartwatch devices to objectively assess people with multiple sclerosis (PwMS) remotely could capture unique aspects of disease progression. This study explores the feasibility of assessing PwMS and Healthy Control's (HC) physical function by characterising gait-related features, which can be modelled using machine learning (ML) techniques to correctly distinguish subgroups of PwMS from healthy controls. A total of 97 subjects (24 HC subjects, 52 mildly disabled (PwMSmild, EDSS [0-3]) and 21 moderately disabled (PwMSmod, EDSS [3.5-5.5]) contributed data which was recorded from a Two-Minute Walk Test (2MWT) performed out-of-clinic and daily over a 24-week period. Signal-based features relating to movement were extracted from sensors in smartphone and smartwatch devices. A large number of features (n = 156) showed fair-to-strong (R 0.3) correlations with clinical outcomes. LASSO feature selection was applied to select and rank subsets of features used for dichotomous classification between subject groups, which were compared using Logistic Regression (LR), Support Vector Machines (SVM) and Random Forest (RF) models. Classifications of subject types were compared using data obtained from smartphone, smartwatch and the fusion of features from both devices. Models built on smartphone features alone achieved the highest classification performance, indicating that accurate and remote measurement of the ambulatory characteristics of HC and PwMS can be achieved with only one device. It was observed however that smartphone-based performance was affected by inconsistent placement location (running belt versus pocket). Results show that PwMSmod could be distinguished from HC subjects (Acc. 82.2 ± 2.9%, Sen. 80.1 ± 3.9%, Spec. 87.2 ± 4.2%, F ₁ 84.3 ± 3.8), and PwMSmild (Acc. 82.3 ± 1.9%, Sen. 71.6 ± 4.2%, Spec. 87.0 ± 3.2%, F 1 75.1 ± 2.2) using an SVM classifier with a Radial Basis Function (RBF). PwMSmild were shown to exhibit HC-like behaviour and were thus less distinguishable from HC (Acc. 66.4 ± 4.5%, Sen. 67.5 ± 5.7%, Spec. 60.3 ± 6.7%, F ₁ 58.6 ± 5.8). Finally, it was observed that subjects in this study demonstrated low intra- and high inter-subject variability which was representative of subject-specific gait characteristics.

The Path Forward for Digital Measures: Suppressing the Desire to Compare Apples and Pineapples

Digital measures are becoming more prevalent in clinical development. Methods for robust evaluation are increasingly well defined, yet the primary barrier for digital measures to transition beyond exploratory usage often relies on a comparison to the existing standards. This article focuses on how researchers should approach the complex issue of comparing across assessment modalities. We discuss comparisons of subjective versus objective assessments, or performance-based versus behavioral measures, and we pay particular attention to the situation where the expected association may be poor or nonlinear. We propose that, rather than seeking to replace the standard, research should focus on a structured understanding of how the new measure augments established assessments, with the ultimate goal of developing a more complete understanding of what is meaningful to patients.

Evaluating the Utility of Smartphone-Based Sensor Assessments in Persons With Multiple Sclerosis in the Real-World Using an App (elevateMS): Observational, Prospective Pilot Digital Health Study

Background

Multiple sclerosis (MS) is a chronic neurodegenerative disease. Current monitoring practices predominantly rely on brief and infrequent assessments, which may not be representative of the real-world patient experience. Smartphone technology provides an opportunity to assess people’s daily-lived experience of MS on a frequent, regular basis outside of episodic clinical evaluations.

Objective

The objectives of this study were to evaluate the feasibility and utility of capturing real-world MS-related health data remotely using a smartphone app, “elevateMS,” to investigate the associations between self-reported MS severity and sensor-based active functional tests measurements, and the impact of local weather conditions on disease burden.

Methods

This was a 12-week, observational, digital health study involving 3 cohorts: self-referred participants who reported an MS diagnosis, clinic-referred participants with neurologist-confirmed MS, and participants without MS (controls). Participants downloaded the elevateMS app and completed baseline assessments, including self-reported physical ability (Patient-Determined Disease Steps [PDDS]), as well as longitudinal assessments of quality of life (Quality of Life in Neurological Disorders [Neuro-QoL] Cognitive, Upper Extremity, and Lower Extremity Function) and daily health (MS symptoms, triggers, health, mobility, pain). Participants also completed functional tests (finger-tapping, walk and balance, voice-based Digit Symbol Substitution Test [DSST], and finger-to-nose) as an independent assessment of MS-related cognition and motor activity. Local weather data were collected each time participants completed an active task. Associations between self-reported baseline/longitudinal assessments, functional tests, and weather were evaluated using linear (for cross-sectional data) and mixed-effects (for longitudinal data) regression models.

Results

A total of 660 individuals enrolled in the study; 31 withdrew, 495 had MS (n=359 self-referred, n=136 clinic-referred), and 134 were controls. Participation was highest in clinic-referred versus self-referred participants (median retention: 25.5 vs 7.0 days). The top 5 most common MS symptoms, reported at least once by participants with MS, were fatigue (310/495, 62.6%), weakness (222/495, 44.8%), memory/attention issues (209/495, 42.2%), and difficulty walking (205/495, 41.4%), and the most common triggers were high ambient temperature (259/495, 52.3%), stress (250/495, 50.5%), and late bedtime (221/495, 44.6%). Baseline PDDS was significantly associated with functional test performance in participants with MS (mixed model–based estimate of most significant feature across functional tests [β]: finger-tapping: β=–43.64, P<.001; DSST: β=–5.47, P=.005; walk and balance: β=–.39, P=.001; finger-to-nose: β=.01, P=.01). Longitudinal Neuro-QoL scores were also significantly associated with functional tests (finger-tapping with Upper Extremity Function: β=.40, P<.001; walk and balance with Lower Extremity Function: β=–99.18, P=.02; DSST with Cognitive Function: β=1.60, P=.03). Finally, local temperature was significantly associated with participants’ test performance (finger-tapping: β=–.14, P<.001; DSST: β=–.06, P=.009; finger-to-nose: β=–53.88, P<.001).

Conclusions

The elevateMS study app captured the real-world experience of MS, characterized some MS symptoms, and assessed the impact of environmental factors on symptom severity. Our study provides further evidence that supports smartphone app use to monitor MS with both active assessments and patient-reported measures of disease burden. App-based tracking may provide unique and timely real-world data for clinicians and patients, resulting in improved disease insights and management.

Digital biomarkers can highlight subtle clinical differences in radiologically isolated syndrome compared to healthy controls

OBJECTIVE

To explore the use of digital biomarkers to distinguish healthy controls (HC) from subjects with a radiologically isolated syndrome (RIS).

METHODS

We developed a smartphone application called MS Screen Test (MSST) to explore several dimensions of the neurological exam such as finger tapping speed, agility, hand synchronization, low contrast vision and cognition during a short evaluation. This app was tested on a cohort of healthy volunteers including a subset of subjects who underwent two evaluations on the same day to assess reproducibility. In a second step, the app was tested on a cohort of RIS subjects. Performances of RIS subjects were compared with age and genre-matched HC.

RESULTS

HC underwent two consecutive evaluations on MSST. The analysis showed good reproducibility for all measures. Then 21 RIS subjects were compared to 32 matched HC. Compared to HC, we found that RIS subjects had a lower finger tapping speed on the dominant hand (5.6 versus 6.5 taps per second; p = 0.005), a longer inter hand interval during the hand synchronization task (14.4 versus 11.3 ms; p = 0.03) and significantly poorer scores on the low contrast vision and cognition tests.

CONCLUSION

MSST only requires a smartphone to obtain digital biomarkers relative to several dimensions of the neurological examination. Our results highlighted subtle differences between HC and RIS subjects. We plan to evaluate this tool in MS patients, which will allow us to get a much larger sample of subjects, to determine whether digital biomarkers can predict disease course.