Feasibility and initial validation of 'HD-Mobile', a smartphone application for remote self-administration of performance-based cognitive measures in Huntington's disease

Reliability and Validity of Smartphone Cognitive Testing for Frontotemporal Lobar Degeneration

Importance

Frontotemporal lobar degeneration (FTLD) is relatively rare, behavioral and motor symptoms increase travel burden, and standard neuropsychological tests are not sensitive to early-stage disease. Remote smartphone-based cognitive assessments could mitigate these barriers to trial recruitment and success, but no such tools are validated for FTLD.

Objective

To evaluate the reliability and validity of smartphone-based cognitive measures for remote FTLD evaluations.

Design, Setting, and Participants

In this cohort study conducted from January 10, 2019, to July 31, 2023, controls and participants with FTLD performed smartphone application (app)-based executive functioning tasks and an associative memory task 3 times over 2 weeks. Observational research participants were enrolled through 18 centers of a North American FTLD research consortium (ALLFTD) and were asked to complete the tests remotely using their own smartphones. Of 1163 eligible individuals (enrolled in parent studies), 360 were enrolled in the present study; 364 refused and 439 were excluded. Participants were divided into discovery (n = 258) and validation (n = 102) cohorts. Among 329 participants with data available on disease stage, 195 were asymptomatic or had preclinical FTLD (59.3%), 66 had prodromal FTLD (20.1%), and 68 had symptomatic FTLD (20.7%) with a range of clinical syndromes.

Exposure

Participants completed standard in-clinic measures and remotely administered ALLFTD mobile app (app) smartphone tests.

Main Outcomes and Measures

Internal consistency, test-retest reliability, association of smartphone tests with criterion standard clinical measures, and diagnostic accuracy.

Results

In the 360 participants (mean [SD] age, 54.0 [15.4] years; 209 [58.1%] women), smartphone tests showed moderate-to-excellent reliability (intraclass correlation coefficients, 0.77-0.95). Validity was supported by association of smartphones tests with disease severity (r range, 0.38-0.59), criterion-standard neuropsychological tests (r range, 0.40-0.66), and brain volume (standardized β range, 0.34-0.50). Smartphone tests accurately differentiated individuals with dementia from controls (area under the curve [AUC], 0.93 [95% CI, 0.90-0.96]) and were more sensitive to early symptoms (AUC, 0.82 [95% CI, 0.76-0.88]) than the Montreal Cognitive Assessment (AUC, 0.68 [95% CI, 0.59-0.78]) (z of comparison, -2.49 [95% CI, -0.19 to -0.02]; P = .01). Reliability and validity findings were highly similar in the discovery and validation cohorts. Preclinical participants who carried pathogenic variants performed significantly worse than noncarrier family controls on 3 app tasks (eg, 2-back β = -0.49 [95% CI, -0.72 to -0.25]; P < .001) but not a composite of traditional neuropsychological measures (β = -0.14 [95% CI, -0.42 to 0.14]; P = .32).

Conclusions and Relevance

The findings of this cohort study suggest that smartphones could offer a feasible, reliable, valid, and scalable solution for remote evaluations of FTLD and may improve early detection. Smartphone assessments should be considered as a complementary approach to traditional in-person trial designs. Future research should validate these results in diverse populations and evaluate the utility of these tests for longitudinal monitoring.

Tele-Neuropsychology: From Science to Policy to Practice

OBJECTIVE

The primary aim of this paper is to accelerate the number of randomized experimental studies of the reliability and validity in-home tele-neuropsychological testing (tele-np-t).

METHOD

We conducted a critical review of the tele-neuropsychology literature. We discuss this research in the context of the United States' public and private healthcare payer systems, including the Centers for Medicare & Medicaid Services (CMS) and Current Procedural Terminology (CPT) coding system's telehealth lists, and existing disparities in healthcare access.

RESULTS

The number of tele-np publications has been stagnant since the onset of the COVID-19 pandemic. There are less published experimental studies of tele-neuropsychology (tele-np), and particularly in-home tele-np-t, than other tele-np publications. There is strong foundational evidence of the acceptability, feasibility, and reliability of tele-np-t, but relatively few studies of the reliability and validity of in-home tele-np-t using randomization methodology.

CONCLUSIONS

More studies of the reliability and validity of in-home tele-np-t using randomization methodology are necessary to support inclusion of tele-np-t codes on the CMS and CPT telehealth lists, and subsequently, the integration and delivery of in-home tele-np-t services across providers and institutions. These actions are needed to maintain equitable reimbursement of in-home tele-np-t services and address the widespread disparities in healthcare access.

Disease Delineation for Multiple Sclerosis, Friedreich Ataxia, and Healthy Controls Using Supervised Machine Learning on Speech Acoustics

Neurodegenerative disease often affects speech. Speech acoustics can be used as objective clinical markers of pathology. Previous investigations of pathological speech have primarily compared controls with one specific condition and excluded comorbidities. We broaden the utility of speech markers by examining how multiple acoustic features can delineate diseases. We used supervised machine learning with gradient boosting (CatBoost) to delineate healthy speech from speech of people with multiple sclerosis or Friedreich ataxia. Participants performed a diadochokinetic task where they repeated alternating syllables. We subjected 74 spectral and temporal prosodic features from the speech recordings to machine learning. Results showed that Friedreich ataxia, multiple sclerosis and healthy controls were all identified with high accuracy (over 82%). Twenty-one acoustic features were strong markers of neurodegenerative diseases, falling under the categories of spectral qualia, spectral power, and speech rate. We demonstrated that speech markers can delineate neurodegenerative diseases and distinguish healthy speech from pathological speech with high accuracy. Findings emphasize the importance of examining speech outcomes when assessing indicators of neurodegenerative disease. We propose large-scale initiatives to broaden the scope for differentiating other neurological diseases and affective disorders.

The feasibility of remotely monitoring physical, cognitive, and psychosocial function in individuals with stroke or chronic obstructive pulmonary disease

Objective

Clinical implementation of remote monitoring of human function requires an understanding of its feasibility. We evaluated adherence and the resources required to monitor physical, cognitive, and psychosocial function in individuals with either chronic obstructive pulmonary disease or stroke during a three-month period.

Methods

Seventy-three individuals agreed to wear a Fitbit to monitor physical function and to complete monthly online assessments of cognitive and psychosocial function. During a three-month period, we measured adherence to monitoring (1) physical function using average daily wear time, and (2) cognition and psychosocial function using the percentage of assessments completed. We measured the resources needed to promote adherence as (1) the number of participants requiring at least one reminder to synchronize their Fitbit, and (2) the number of reminders needed for each completed cognitive and psychosocial assessment.

Results

After accounting for withdrawals, the average daily wear time was 77.5 ± 19.9% of the day and did not differ significantly between months 1, 2, and 3 (p = 0.30). To achieve this level of adherence, 64.9% of participants required at least one reminder to synchronize their device. Participants completed 61.0% of the cognitive and psychosocial assessments; the portion of assessments completed each month didnot significantly differ (p = 0.44). Participants required 1.13 ± 0.57 reminders for each completed assessment. Results did not differ by disease diagnosis.

Conclusions

Remote monitoring of human function in individuals with either chronic obstructive pulmonary disease or stroke is feasible as demonstrated by high adherence. However, the number of reminders required indicates that careful consideration must be given to the resources available to obtain high adherence.

A Tutorial Review on Clinical Acoustic Markers in Speech Science

PURPOSE

The human voice changes with the progression of neurological disease and the onset of diseases that affect articulators, often decreasing the effectiveness of communication. These changes can be objectively measured using signal processing techniques that extract acoustic features. When measuring acoustic features, there are often several steps and assumptions that might be known to experts in acoustics and phonetics, but are less transparent for other disciplines (e.g., clinical medicine, speech pathology, engineering, and data science). This tutorial describes these signal processing techniques, explicitly outlines the underlying steps for accurate measurement, and discusses the implications of clinical acoustic markers.

CONCLUSIONS

We establish a vocabulary using straightforward terms, provide visualizations to achieve common ground, and guide understanding for those outside the domains of acoustics and auditory signal processing. Where possible, we highlight the best practices for measuring clinical acoustic markers and suggest resources for obtaining and further understanding these measures.

A Remote Digital Monitoring Platform to Assess Cognitive and Motor Symptoms in Huntington Disease: Cross-sectional Validation Study

Background

Remote monitoring of Huntington disease (HD) signs and symptoms using digital technologies may enhance early clinical diagnosis and tracking of disease progression, guide treatment decisions, and monitor response to disease-modifying agents. Several recent studies in neurodegenerative diseases have demonstrated the feasibility of digital symptom monitoring.

Objective

The aim of this study was to evaluate a novel smartwatch- and smartphone-based digital monitoring platform to remotely monitor signs and symptoms of HD.

Methods

This analysis aimed to determine the feasibility and reliability of the Roche HD Digital Monitoring Platform over a 4-week period and cross-sectional validity over a 2-week interval. Key criteria assessed were feasibility, evaluated by adherence and quality control failure rates; test-retest reliability; known-groups validity; and convergent validity of sensor-based measures with existing clinical measures. Data from 3 studies were used: the predrug screening phase of an open-label extension study evaluating tominersen (NCT03342053) and 2 untreated cohorts—the HD Natural History Study (NCT03664804) and the Digital-HD study. Across these studies, controls (n=20) and individuals with premanifest (n=20) or manifest (n=179) HD completed 6 motor and 2 cognitive tests at home and in the clinic.

Results

Participants in the open-label extension study, the HD Natural History Study, and the Digital-HD study completed 89.95% (1164/1294), 72.01% (2025/2812), and 68.98% (1454/2108) of the active tests, respectively. All sensor-based features showed good to excellent test-retest reliability (intraclass correlation coefficient 0.89-0.98) and generally low quality control failure rates. Good overall convergent validity of sensor-derived features to Unified HD Rating Scale outcomes and good overall known-groups validity among controls, premanifest, and manifest participants were observed. Among participants with manifest HD, the digital cognitive tests demonstrated the strongest correlations with analogous in-clinic tests (Pearson correlation coefficient 0.79-0.90).

Conclusions

These results show the potential of the HD Digital Monitoring Platform to provide reliable, valid, continuous remote monitoring of HD symptoms, facilitating the evaluation of novel treatments and enhanced clinical monitoring and care for individuals with HD.

A smartphone sensor-based digital outcome assessment of multiple sclerosis

Background:

Sensor-based monitoring tools fill a critical gap in multiple sclerosis (MS) research and clinical care.

Objective:

The aim of this study is to assess performance characteristics of the Floodlight Proof-of-Concept (PoC) app.

Methods:

In a 24-week study (clinicaltrials.gov: NCT02952911), smartphone-based active tests and passive monitoring assessed cognition (electronic Symbol Digit Modalities Test), upper extremity function (Pinching Test, Draw a Shape Test), and gait and balance (Static Balance Test, U-Turn Test, Walk Test, Passive Monitoring). Intraclass correlation coefficients (ICCs) and age- or sex-adjusted Spearman’s rank correlation determined test–retest reliability and correlations with clinical and magnetic resonance imaging (MRI) outcome measures, respectively.

Results:

Seventy-six people with MS (PwMS) and 25 healthy controls were enrolled. In PwMS, ICCs were moderate-to-good (ICC(2,1) = 0.61–0.85) across tests. Correlations with domain-specific standard clinical disability measures were significant for all tests in the cognitive (r = 0.82, p < 0.001), upper extremity function (|r|= 0.40–0.64, all p < 0.001), and gait and balance domains (r = −0.25 to −0.52, all p < 0.05; except for Static Balance Test: r = −0.20, p > 0.05). Most tests also correlated with Expanded Disability Status Scale, 29-item Multiple Sclerosis Impact Scale items or subscales, and/or normalized brain volume.

Conclusion:

The Floodlight PoC app captures reliable and clinically relevant measures of functional impairment in MS, supporting its potential use in clinical research and practice.

Co-Calibrating Physical and Psychological Outcomes and Consumer Wearable Activity Outcomes in Older Adults: An Evaluation of the coQoL Method

Inactivity, lack of sleep, and poor nutrition predispose individuals to health risks. Patient-Reported Outcomes (PROs) assess physical behaviours and psychological states but are subject of self-reporting biases. Conversely, wearables are an increasingly accurate source of behavioural Technology-Reported Outcomes (TechROs). However, the extent to which PROs and TechROs provide convergent information is unknown. We propose the coQoL PRO-TechRO co-calibration method and report its feasibility, reliability, and human factors influencing data quality. Thirty-nine seniors provided 7.4 ± 4.4 PROs for physical activity (IPAQ), social support (MSPSS), anxiety/depression (GADS), nutrition (PREDIMED, SelfMNA), memory (MFE), sleep (PSQI), Quality of Life (EQ-5D-3L), and 295 ± 238 days of TechROs (Fitbit Charge 2) along two years. We co-calibrated PROs and TechROs by Spearman rank and reported human factors guiding coQoL use. We report high PRO-TechRO correlations (rS≥ 0.8) for physical activity (moderate domestic activity-light+fair active duration), social support (family help-fair activity), anxiety/depression (numeric score-sleep duration), or sleep (duration to sleep-sleep duration) at various durations (7-120 days). coQoL feasibly co-calibrates constructs within physical behaviours and psychological states in seniors. Our results can inform designs of longitudinal observations and, whenever appropriate, personalized behavioural interventions.