Multimodal Classification of Parkinson's Disease in Home Environments with Resiliency to Missing Modalities

Computer Vision Technologies in Movement Disorders: A Systematic Review

BACKGROUND

Evaluation of movement disorders primarily relies on phenomenology. Despite refinements in diagnostic criteria, the accuracy remains suboptimal. Such a gap may be bridged by machine learning and video technology, which permit objective, quantitative, non-invasive motor analysis. Markerless automated video-analysis, namely Computer Vision, emerged as best suited for ecologically-valid assessment.

OBJECTIVES

To systematically review the application of Computer Vision for assessment, diagnosis, and monitoring of movement disorders.

METHODS

Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we searched Cochrane, Embase, PubMed, and Scopus databases for articles published between 1984 and September 2024. We used the following search strategy: ("video analysis" OR "computer vision") AND ("Parkinson's disease" OR "PD" OR "tremor" OR "dystonia" OR "parkinsonism" OR "progressive supranuclear palsy" OR "PSP" OR "multiple system atrophy" OR "MSA" OR "corticobasal syndrome" OR "CBS" OR "chorea" OR "ballism" OR "myoclonus" OR "Tourette's syndrome").

RESULTS

Out of 1099 identified studies, 61 met inclusion criteria, and 10 additional studies were included based on authors' judgment. Parkinson's disease was the most investigated movement disorder, with gait as the prevalent motor task. OpenPose was the most used pose estimation software. Automated video-analysis consistently achieved diagnostic accuracies exceeding 80% across most movement disorders. For tremor, dystonia severity and tic detection, Computer Vision strongly aligned with accelerometery and clinical assessments.

CONCLUSIONS

Computer Vision holds potential to provide non-invasive quantification of presence and severity of movement disorders. Heterogeneity in video settings, software usage, and definition of standardized guidelines for videorecording are challenges to be addressed for real-word applications.

Smartphone-derived multidomain features including voice, finger-tapping movement and gait aid early identification of Parkinson's disease

Smart devices can easily capture changes in voice, movements, and gait in people with Parkinson's disease (PD). We investigated whether smartphone-derived multimodal features combined with machine learning algorithms can aid in early PD identification. We recruited 496 participants, split into a training cohort (127 PD patients during "on" phase and 198 age-matched controls) and a test dataset (86 patients during "off" phase and 85 age-matched controls). Multidomain features from smartphone recordings were analyzed using machine learning classifiers with integration of a hyperparameter grid. Single-modality models for voice, hand movements, and gait showed diagnostic values of 0.88, 0.74, and 0.81, respectively, with test dataset values of 0.80, 0.74, and 0.76. An integrated multimodal model using a support vector machine improved performance to 0.86 and achieved 0.82 for identifying early-stage PD during the "off" phase. A smartphone-based integrated multimodality model combining voice, hand movement, and gait shows promise for early PD identification.

Automatic two-dimensional & three-dimensional video analysis with deep learning for movement disorders: A systematic review

The advent of computer vision technology and increased usage of video cameras in clinical settings have facilitated advancements in movement disorder analysis. This review investigated these advancements in terms of providing practical, low-cost solutions for the diagnosis and analysis of movement disorders, such as Parkinson's disease, ataxia, dyskinesia, and Tourette syndrome. Traditional diagnostic methods for movement disorders are typically reliant on the subjective assessment of motor symptoms, which poses inherent challenges. Furthermore, early symptoms are often overlooked, and overlapping symptoms across diseases can complicate early diagnosis. Consequently, deep learning has been used for the objective video-based analysis of movement disorders. This study systematically reviewed the latest advancements in automatic two-dimensional & three-dimensional video analysis using deep learning for movement disorders. We comprehensively analyzed the literature published until September 2023 by searching the Web of Science, PubMed, Scopus, and Embase databases. We identified 68 relevant studies and extracted information on their objectives, datasets, modalities, and methodologies. The study aimed to identify, catalogue, and present the most significant advancements, offering a consolidated knowledge base on the role of video analysis and deep learning in movement disorder analysis. First, the objectives, including specific PD symptom quantification, ataxia assessment, cerebral palsy assessment, gait disorder analysis, tremor assessment, tic detection (in the context of Tourette syndrome), dystonia assessment, and abnormal movement recognition were discussed. Thereafter, the datasets used in the study were examined. Subsequently, video modalities and deep learning methodologies related to the topic were investigated. Finally, the challenges and opportunities in terms of datasets, interpretability, evaluation methods, and home/remote monitoring were discussed.

A multimodal dataset of real world mobility activities in Parkinson's disease

Parkinson's disease (PD) is a neurodegenerative disorder characterised by motor symptoms such as gait dysfunction and postural instability. Technological tools to continuously monitor outcomes could capture the hour-by-hour symptom fluctuations of PD. Development of such tools is hampered by the lack of labelled datasets from home settings. To this end, we propose REMAP (REal-world Mobility Activities in Parkinson's disease), a human rater-labelled dataset collected in a home-like setting. It includes people with and without PD doing sit-to-stand transitions and turns in gait. These discrete activities are captured from periods of free-living (unobserved, unstructured) and during clinical assessments. The PD participants withheld their dopaminergic medications for a time (causing increased symptoms), so their activities are labelled as being "on" or "off" medications. Accelerometry from wrist-worn wearables and skeleton pose video data is included. We present an open dataset, where the data is coarsened to reduce re-identifiability, and a controlled dataset available on application which contains more refined data. A use-case for the data to estimate sit-to-stand speed and duration is illustrated.

Internet of Things Technologies and Machine Learning Methods for Parkinson's Disease Diagnosis, Monitoring and Management: A Systematic Review

Parkinson's disease is a chronic neurodegenerative disease that affects a large portion of the population, especially the elderly. It manifests with motor, cognitive and other types of symptoms, decreasing significantly the patients' quality of life. The recent advances in the Internet of Things and Artificial Intelligence fields, including the subdomains of machine learning and deep learning, can support Parkinson's disease patients, their caregivers and clinicians at every stage of the disease, maximizing the treatment effectiveness and minimizing the respective healthcare costs at the same time. In this review, the considered studies propose machine learning models, trained on data acquired via smart devices, wearable or non-wearable sensors and other Internet of Things technologies, to provide predictions or estimations regarding Parkinson's disease aspects. Seven hundred and seventy studies have been retrieved from three dominant academic literature databases. Finally, one hundred and twelve of them have been selected in a systematic way and have been considered in the state-of-the-art systematic review presented in this paper. These studies propose various methods, applied on various sensory data to address different Parkinson's disease-related problems. The most widely deployed sensors, the most commonly addressed problems and the best performing algorithms are highlighted. Finally, some challenges are summarized along with some future considerations and opportunities that arise.

Acceptability of an In-Home Multimodal Sensor Platform in Parkinson’s Disease: A Qualitative Study (Preprint)

Background

Parkinson disease (PD) symptoms are complex, gradually progressive, and fluctuate hour by hour. Home-based technological sensors are being investigated to measure symptoms and track disease progression. A smart home sensor platform, with cameras and wearable devices, could be a useful tool to use to get a fuller picture of what someone’s symptoms are like. High-resolution video can capture the ground truth of symptoms and activities. There is a paucity of information about the acceptability of such sensors in PD.

Objective

The primary objective of our study was to explore the acceptability of living with a multimodal sensor platform in a naturalistic setting in PD. Two subobjectives are to identify any suggested limitations and to explore the sensors’ impact on participant behaviors.

Methods

A qualitative study was conducted with an inductive approach using semistructured interviews with a cohort of PD and control participants who lived freely for several days in a home-like environment while continuously being sensed.

Results

This study of 24 participants (12 with PD) found that it is broadly acceptable to use multimodal sensors including wrist-worn wearables, cameras, and other ambient sensors passively in free-living in PD. The sensor that was found to be the least acceptable was the wearable device. Suggested limitations on the platform for home deployment included camera-free time and space. Behavior changes were noted by the study participants, which may have related to being passively sensed. Recording high-resolution video in the home setting for limited periods of time was felt to be acceptable to all participants.

Conclusions

The results broaden the knowledge of what types of sensors are acceptable for use in research in PD and what potential limitations on these sensors should be considered in future work. The participants’ reported behavior change in this study should inform future similar research design to take this factor into account. Collaborative research study design, involving people living with PD at every stage, is important to ensure that the technology is acceptable and that the data outcomes produced are ecologically valid and accurate.

International Registered Report Identifier (IRRID)

RR2-10.1136/bmjopen-2020-041303

A Sensor-Based Perspective in Early-Stage Parkinson's Disease: Current State and the Need for Machine Learning Processes

Parkinson's disease (PD) is a progressive neurodegenerative disorder associated with dysfunction of dopaminergic neurons in the brain, lack of dopamine and the formation of abnormal Lewy body protein particles. PD is an idiopathic disease of the nervous system, characterized by motor and nonmotor manifestations without a discrete onset of symptoms until a substantial loss of neurons has already occurred, enabling early diagnosis very challenging. Sensor-based platforms have gained much attention in clinical practice screening various biological signals simultaneously and allowing researchers to quickly receive a huge number of biomarkers for diagnostic and prognostic purposes. The integration of machine learning into medical systems provides the potential for optimization of data collection, disease prediction through classification of symptoms and can strongly support data-driven clinical decisions. This work attempts to examine some of the facts and current situation of sensor-based approaches in PD diagnosis and discusses ensemble techniques using sensor-based data for developing machine learning models for personalized risk prediction. Additionally, a biosensing platform combined with clinical data processing and appropriate software is proposed in order to implement a complete diagnostic system for PD monitoring.