1
|
Huang W, Xu W, Wan R, Zhang P, Zha Y, Pang M. Auto Diagnosis of Parkinson's Disease Via a Deep Learning Model Based on Mixed Emotional Facial Expressions. IEEE J Biomed Health Inform 2024; 28:2547-2557. [PMID: 37022035 DOI: 10.1109/jbhi.2023.3239780] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Parkinson's disease (PD) is a common degenerative disease of the nervous system in the elderly. The early diagnosis of PD is very important for potential patients to receive prompt treatment and avoid the aggravation of the disease. Recent studies have found that PD patients always suffer from emotional expression disorder, thus forming the characteristics of "masked faces". Based on this, we thus propose an auto PD diagnosis method based on mixed emotional facial expressions in the paper. Specifically, the proposed method is cast into four steps: Firstly, we synthesize virtual face images containing six basic expressions (i.e., anger, disgust, fear, happiness, sadness, and surprise) via generative adversarial learning, in order to approximate the premorbid expressions of PD patients; Secondly, we design an effective screening scheme to assess the quality of the above synthesized facial expression images and then shortlist the high-quality ones; Thirdly, we train a deep feature extractor accompanied with a facial expression classifier based on the mixture of the original facial expression images of the PD patients, the high-quality synthesized facial expression images of PD patients, and the normal facial expression images from other public face datasets; Finally, with the well-trained deep feature extractor, we thus adopt it to extract the latent expression features for six facial expression images of a potential PD patient to conduct PD/non-PD prediction. To show real-world impacts, we also collected a new facial expression dataset of PD patients in collaboration with a hospital. Extensive experiments are conducted to validate the effectiveness of the proposed method for PD diagnosis and facial expression recognition.
Collapse
|
2
|
Abumalloh RA, Nilashi M, Samad S, Ahmadi H, Alghamdi A, Alrizq M, Alyami S. Parkinson's disease diagnosis using deep learning: A bibliometric analysis and literature review. Ageing Res Rev 2024; 96:102285. [PMID: 38554785 DOI: 10.1016/j.arr.2024.102285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 03/20/2024] [Accepted: 03/24/2024] [Indexed: 04/02/2024]
Abstract
Parkinson's Disease (PD) is a progressive neurodegenerative illness triggered by decreased dopamine secretion. Deep Learning (DL) has gained substantial attention in PD diagnosis research, with an increase in the number of published papers in this discipline. PD detection using DL has presented more promising outcomes as compared with common machine learning approaches. This article aims to conduct a bibliometric analysis and a literature review focusing on the prominent developments taking place in this area. To achieve the target of the study, we retrieved and analyzed the available research papers in the Scopus database. Following that, we conducted a bibliometric analysis to inspect the structure of keywords, authors, and countries in the surveyed studies by providing visual representations of the bibliometric data using VOSviewer software. The study also provides an in-depth review of the literature focusing on different indicators of PD, deployed approaches, and performance metrics. The outcomes indicate the firm development of PD diagnosis using DL approaches over time and a large diversity of studies worldwide. Additionally, the literature review presented a research gap in DL approaches related to incremental learning, particularly in relation to big data analysis.
Collapse
Affiliation(s)
- Rabab Ali Abumalloh
- Department of Computer Science and Engineering, Qatar University, Doha 2713, Qatar
| | - Mehrbakhsh Nilashi
- Institute of Research and Development, Duy Tan University, Da Nang, Vietnam; School of Computer Science, Duy Tan University, Da Nang, Vietnam; UCSI Graduate Business School, UCSI University, No. 1 Jalan Menara Gading, UCSI Heights, Cheras, Kuala Lumpur 56000, Malaysia; Centre for Global Sustainability Studies (CGSS), Universiti Sains Malaysia, Penang 11800, Malaysia.
| | - Sarminah Samad
- Faculty of Business, UNITAR International University, Tierra Crest, Jalan SS6/3, Petaling Jaya, Selangor 47301, Malaysia
| | - Hossein Ahmadi
- Centre for Health Technology, Faculty of Health, University of Plymouth, Plymouth PL4 8AA, UK
| | - Abdullah Alghamdi
- Information Systems Dept., College of Computer Science and Information Systems, Najran University, Najran, Saudi Arabia; AI Lab, Scientific and Engineering Research Center (SERC), Najran University, Najran, Saudi Arabia
| | - Mesfer Alrizq
- Information Systems Dept., College of Computer Science and Information Systems, Najran University, Najran, Saudi Arabia; AI Lab, Scientific and Engineering Research Center (SERC), Najran University, Najran, Saudi Arabia
| | - Sultan Alyami
- AI Lab, Scientific and Engineering Research Center (SERC), Najran University, Najran, Saudi Arabia; Computer Science Dept., College of Computer Science and Information Systems, Najran University, Najran, Saudi Arabia
| |
Collapse
|
3
|
Park KW, Mirian MS, McKeown MJ. Artificial intelligence-based video monitoring of movement disorders in the elderly: a review on current and future landscapes. Singapore Med J 2024; 65:141-149. [PMID: 38527298 PMCID: PMC11060643 DOI: 10.4103/singaporemedj.smj-2023-189] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/19/2023] [Indexed: 03/27/2024]
Abstract
ABSTRACT Due to global ageing, the burden of chronic movement and neurological disorders (Parkinson's disease and essential tremor) is rapidly increasing. Current diagnosis and monitoring of these disorders rely largely on face-to-face assessments utilising clinical rating scales, which are semi-subjective and time-consuming. To address these challenges, the utilisation of artificial intelligence (AI) has emerged. This review explores the advantages and challenges associated with using AI-driven video monitoring to care for elderly patients with movement disorders. The AI-based video monitoring systems offer improved efficiency and objectivity in remote patient monitoring, enabling real-time analysis of data, more uniform outcomes and augmented support for clinical trials. However, challenges, such as video quality, privacy compliance and noisy training labels, during development need to be addressed. Ultimately, the advancement of video monitoring for movement disorders is expected to evolve towards discreet, home-based evaluations during routine daily activities. This progression must incorporate data security, ethical considerations and adherence to regulatory standards.
Collapse
Affiliation(s)
- Kye Won Park
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Maryam S Mirian
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Martin J McKeown
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
- Department of Medicine (Neurology), University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
4
|
Amo-Salas J, Olivares-Gil A, García-Bustillo Á, García-García D, Arnaiz-González Á, Cubo E. Computer Vision for Parkinson's Disease Evaluation: A Survey on Finger Tapping. Healthcare (Basel) 2024; 12:439. [PMID: 38391815 PMCID: PMC10888014 DOI: 10.3390/healthcare12040439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 02/01/2024] [Accepted: 02/06/2024] [Indexed: 02/24/2024] Open
Abstract
Parkinson's disease (PD) is a progressive neurodegenerative disorder whose prevalence has steadily been rising over the years. Specialist neurologists across the world assess and diagnose patients with PD, although the diagnostic process is time-consuming and various symptoms take years to appear, which means that the diagnosis is prone to human error. The partial automatization of PD assessment and diagnosis through computational processes has therefore been considered for some time. One well-known tool for PD assessment is finger tapping (FT), which can now be assessed through computer vision (CV). Artificial intelligence and related advances over recent decades, more specifically in the area of CV, have made it possible to develop computer systems that can help specialists assess and diagnose PD. The aim of this study is to review some advances related to CV techniques and FT so as to offer insight into future research lines that technological advances are now opening up.
Collapse
Affiliation(s)
- Javier Amo-Salas
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Alicia Olivares-Gil
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Álvaro García-Bustillo
- Facultad de Ciencias de la Salud, Departamento de Ciencias de la Salud, Universidad de Burgos, 09001 Burgos, Spain
| | - David García-García
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Álvar Arnaiz-González
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Esther Cubo
- Servicio de Neurología, Hospital Universitario de Burgos, 09006 Burgos, Spain
| |
Collapse
|
5
|
Yang YY, Ho MY, Tai CH, Wu RM, Kuo MC, Tseng YJ. FastEval Parkinsonism: an instant deep learning-assisted video-based online system for Parkinsonian motor symptom evaluation. NPJ Digit Med 2024; 7:31. [PMID: 38332372 PMCID: PMC10853559 DOI: 10.1038/s41746-024-01022-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 01/18/2024] [Indexed: 02/10/2024] Open
Abstract
The Motor Disorder Society's Unified Parkinson's Disease Rating Scale (MDS-UPDRS) is designed to assess bradykinesia, the cardinal symptoms of Parkinson's disease (PD). However, it cannot capture the all-day variability of bradykinesia outside the clinical environment. Here, we introduce FastEval Parkinsonism ( https://fastevalp.cmdm.tw/ ), a deep learning-driven video-based system, providing users to capture keypoints, estimate the severity, and summarize in a report. Leveraging 840 finger-tapping videos from 186 individuals (103 patients with Parkinson's disease (PD), 24 participants with atypical parkinsonism (APD), 12 elderly with mild parkinsonism signs (MPS), and 47 healthy controls (HCs)), we employ a dilated convolution neural network with two data augmentation techniques. Our model achieves acceptable accuracies (AAC) of 88.0% and 81.5%. The frequency-intensity (FI) value of thumb-index finger distance was indicated as a pivotal hand parameter to quantify the performance. Our model also shows the usability for multi-angle videos, tested in an external database enrolling over 300 PD patients.
Collapse
Affiliation(s)
- Yu-Yuan Yang
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC
| | - Ming-Yang Ho
- Department of Computer Science and Information Engineering, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC
| | - Chung-Hwei Tai
- Department of Neurology, National Taiwan University Hospital, No. 1 Changde St., Zhongzheng Dist., Taipei City, 100229, Taiwan, ROC
| | - Ruey-Meei Wu
- Department of Medicine, National Taiwan University Cancer Center, No. 57, Lane 155, Sec. 3, Keelung Rd., Da'an Dist., Taipei City, 106, Taiwan, ROC
| | - Ming-Che Kuo
- Department of Neurology, National Taiwan University Hospital, No. 1 Changde St., Zhongzheng Dist., Taipei City, 100229, Taiwan, ROC.
- Department of Medicine, National Taiwan University Cancer Center, No. 57, Lane 155, Sec. 3, Keelung Rd., Da'an Dist., Taipei City, 106, Taiwan, ROC.
| | - Yufeng Jane Tseng
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC.
- Department of Computer Science and Information Engineering, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC.
| |
Collapse
|
6
|
Yu T, Park KW, McKeown MJ, Wang ZJ. Clinically Informed Automated Assessment of Finger Tapping Videos in Parkinson's Disease. SENSORS (BASEL, SWITZERLAND) 2023; 23:9149. [PMID: 38005535 PMCID: PMC10674854 DOI: 10.3390/s23229149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 10/30/2023] [Accepted: 11/10/2023] [Indexed: 11/26/2023]
Abstract
The utilization of Artificial Intelligence (AI) for assessing motor performance in Parkinson's Disease (PD) offers substantial potential, particularly if the results can be integrated into clinical decision-making processes. However, the precise quantification of PD symptoms remains a persistent challenge. The current standard Unified Parkinson's Disease Rating Scale (UPDRS) and its variations serve as the primary clinical tools for evaluating motor symptoms in PD, but are time-intensive and prone to inter-rater variability. Recent work has applied data-driven machine learning techniques to analyze videos of PD patients performing motor tasks, such as finger tapping, a UPDRS task to assess bradykinesia. However, these methods often use abstract features that are not closely related to clinical experience. In this paper, we introduce a customized machine learning approach for the automated scoring of UPDRS bradykinesia using single-view RGB videos of finger tapping, based on the extraction of detailed features that rigorously conform to the established UPDRS guidelines. We applied the method to 75 videos from 50 PD patients collected in both a laboratory and a realistic clinic environment. The classification performance agreed well with expert assessors, and the features selected by the Decision Tree aligned with clinical knowledge. Our proposed framework was designed to remain relevant amid ongoing patient recruitment and technological progress. The proposed approach incorporates features that closely resonate with clinical reasoning and shows promise for clinical implementation in the foreseeable future.
Collapse
Affiliation(s)
- Tianze Yu
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada;
| | - Kye Won Park
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, BC V6T 1Z4, Canada; (K.W.P.); (M.J.M.)
| | - Martin J. McKeown
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, BC V6T 1Z4, Canada; (K.W.P.); (M.J.M.)
- Department of Neurology, Faculty of Medicine, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
| | - Z. Jane Wang
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada;
| |
Collapse
|
7
|
Moniruzzaman M, Yin Z, Hossain MSB, Choi H, Guo Z. Wearable Motion Capture: Reconstructing and Predicting 3D Human Poses From Wearable Sensors. IEEE J Biomed Health Inform 2023; 27:5345-5356. [PMID: 37665702 DOI: 10.1109/jbhi.2023.3311448] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/06/2023]
Abstract
Reconstructing and predicting 3D human walking poses in unconstrained measurement environments have the potential to use for health monitoring systems for people with movement disabilities by assessing progression after treatments and providing information for assistive device controls. The latest pose estimation algorithms utilize motion capture systems, which capture data from IMU sensors and third-person view cameras. However, third-person views are not always possible for outpatients alone. Thus, we propose the wearable motion capture problem of reconstructing and predicting 3D human poses from the wearable IMU sensors and wearable cameras, which aids clinicians' diagnoses on patients out of clinics. To solve this problem, we introduce a novel Attention-Oriented Recurrent Neural Network (AttRNet) that contains a sensor-wise attention-oriented recurrent encoder, a reconstruction module, and a dynamic temporal attention-oriented recurrent decoder, to reconstruct the 3D human pose over time and predict the 3D human poses at the following time steps. To evaluate our approach, we collected a new WearableMotionCapture dataset using wearable IMUs and wearable video cameras, along with the musculoskeletal joint angle ground truth. The proposed AttRNet shows high accuracy on the new lower-limb WearableMotionCapture dataset, and it also outperforms the state-of-the-art methods on two public full-body pose datasets: DIP-IMU and TotalCaputre.
Collapse
|
8
|
Brito FAC, Monteiro LCP, Rocha Santos EG, de Lima RC, Santos-Lobato BL, Cabral AS, Callegari B, Costa e Silva ADA, Souza GS. The role of sex and handedness in the performance of the smartphone-based Finger-Tapping Test. PLOS DIGITAL HEALTH 2023; 2:e0000304. [PMID: 37585430 PMCID: PMC10431671 DOI: 10.1371/journal.pdig.0000304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Accepted: 06/20/2023] [Indexed: 08/18/2023]
Abstract
The Finger Tapping Test (FTT) is a classical neuropsychological test that assesses motor functioning, and recently it has been employed using smartphones. For classical protocols, it has been observed that sex and handedness influence the performance during the test. By assessing the influence of sex and handedness on the test, it is possible to adjust the performance measurements to ensure the validity of test results and avoid sex- and handedness-related bias. The present study aimed to evaluate the influence of sex and handedness on smartphone-based FTT performance. We developed an Android application for the FTT and recruited 40 males and 40 females to carry out three spatial designs on it (protocols I, II, and III). Participants' performance was measured using the global, temporal, and spatial parameters of the FTT. We observed that for the performance in protocol I, handedness had a significant influence on global and temporal variables, while the interaction between handedness and sex had a greater influence on spatial variables. For protocols II and III, we observed that handedness had a significant influence on global, temporal, and spatial variables compared to the other factors. We concluded that the smartphone-based test is partly influenced by handedness and sex, and in clinical implications, these factors should be considered during the evaluation of the smartphone-based FTT.
Collapse
Affiliation(s)
| | | | | | - Ramon Costa de Lima
- Instituto de Ciências Biológicas, Universidade Federal do Pará, Belém, Brazil
| | | | - André Santos Cabral
- Centro de Ciências Biológicas e da Saúde, Universidade do Estado do Pará, Belém, Brazil
| | - Bianca Callegari
- Laboratório de Estudos do Movimento Humano, Instituto de Ciências da Saúde, Universidade Federal do Pará, Belém, Brazil
| | - Anselmo de Athayde Costa e Silva
- Programa de Pós-graduação em Ciências do Movimento Humano, Instituto de Ciências da Saúde, Universidade Federal do Pará, Belém, Brazil
| | - Givago Silva Souza
- Instituto de Ciências Biológicas, Universidade Federal do Pará, Belém, Brazil
- Núcleo de Medicina Tropical, Universidade Federal do Pará, Belém, Brazil
| |
Collapse
|
9
|
Sánchez-Fernández LP, Garza-Rodríguez A, Sánchez-Pérez LA, Martínez-Hernández JM. A Computer Method for Pronation-Supination Assessment in Parkinson's Disease Based on Latent Space Representations of Biomechanical Indicators. Bioengineering (Basel) 2023; 10:bioengineering10050588. [PMID: 37237657 DOI: 10.3390/bioengineering10050588] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/11/2023] [Accepted: 05/11/2023] [Indexed: 05/28/2023] Open
Abstract
One problem in the quantitative assessment of biomechanical impairments in Parkinson's disease patients is the need for scalable and adaptable computing systems. This work presents a computational method that can be used for motor evaluations of pronation-supination hand movements, as described in item 3.6 of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS). The presented method can quickly adapt to new expert knowledge and includes new features that use a self-supervised training approach. The work uses wearable sensors for biomechanical measurements. We tested a machine-learning model on a dataset of 228 records with 20 indicators from 57 PD patients and eight healthy control subjects. The test dataset's experimental results show that the method's precision rates for the pronation and supination classification task achieved up to 89% accuracy, and the F1-scores were higher than 88% in most categories. The scores present a root mean squared error of 0.28 when compared to expert clinician scores. The paper provides detailed results for pronation-supination hand movement evaluations using a new analysis method when compared to the other methods mentioned in the literature. Furthermore, the proposal consists of a scalable and adaptable model that includes expert knowledge and affectations not covered in the MDS-UPDRS for a more in-depth evaluation.
Collapse
Affiliation(s)
- Luis Pastor Sánchez-Fernández
- Centro de Investigación en Computación, Instituto Politécnico Nacional, Juan de Dios Bátiz Ave., México City 07738, Mexico
| | - Alejandro Garza-Rodríguez
- Centro de Investigación en Computación, Instituto Politécnico Nacional, Juan de Dios Bátiz Ave., México City 07738, Mexico
| | - Luis Alejandro Sánchez-Pérez
- Electrical and Computer Engineering Department, University of Michigan, 4901 Evergreen Rd, Dearborn, MI 48128, USA
| | - Juan Manuel Martínez-Hernández
- Instituto Politécnico Nacional, Escuela Nacional de Medicina y Homeopatía, Guillermo Massieu 239, México City 07320, Mexico
| |
Collapse
|
10
|
Lam WWT, Tang YM, Fong KNK. A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J Neuroeng Rehabil 2023; 20:57. [PMID: 37131238 PMCID: PMC10155325 DOI: 10.1186/s12984-023-01186-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 04/26/2023] [Indexed: 05/04/2023] Open
Abstract
BACKGROUND Markerless motion capture (MMC) technology has been developed to avoid the need for body marker placement during motion tracking and analysis of human movement. Although researchers have long proposed the use of MMC technology in clinical measurement-identification and measurement of movement kinematics in a clinical population, its actual application is still in its preliminary stages. The benefits of MMC technology are also inconclusive with regard to its use in assessing patients' conditions. In this review we put a minor focus on the method's engineering components and sought primarily to determine the current application of MMC as a clinical measurement tool in rehabilitation. METHODS A systematic computerized literature search was conducted in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. The search keywords used in each database were "Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess." Only peer-reviewed articles that applied MMC technology for clinical measurement were included. The last search took place on March 6, 2023. Details regarding the application of MMC technology for different types of patients and body parts, as well as the assessment results, were summarized. RESULTS A total of 65 studies were included. The MMC systems used for measurement were most frequently used to identify symptoms or to detect differences in movement patterns between disease populations and their healthy counterparts. Patients with Parkinson's disease (PD) who demonstrated obvious and well-defined physical signs were the largest patient group to which MMC assessment had been applied. Microsoft Kinect was the most frequently used MMC system, although there was a recent trend of motion analysis using video captured with a smartphone camera. CONCLUSIONS This review explored the current uses of MMC technology for clinical measurement. MMC technology has the potential to be used as an assessment tool as well as to assist in the detection and identification of symptoms, which might further contribute to the use of an artificial intelligence method for early screening for diseases. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in the disease populations.
Collapse
Affiliation(s)
- Winnie W T Lam
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Yuk Ming Tang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Kenneth N K Fong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China.
| |
Collapse
|
11
|
Zhu C, Hu B, Chen J, Ai X, Agrawal SK. SARN: Shifted Attention Regression Network for 3D Hand Pose Estimation. Bioengineering (Basel) 2023; 10:bioengineering10020126. [PMID: 36829620 PMCID: PMC9952393 DOI: 10.3390/bioengineering10020126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 01/08/2023] [Accepted: 01/13/2023] [Indexed: 01/19/2023] Open
Abstract
Hand pose estimation (HPE) plays an important role during the functional assessment of the hand and in potential rehabilitation. It is a challenge to predict the pose of the hand conveniently and accurately during functional tasks, and this limits the application of HPE. In this paper, we propose a novel architecture of a shifted attention regression network (SARN) to perform HPE. Given a depth image, SARN first predicts the spatial relationships between points in the depth image and a group of hand keypoints that determine the pose of the hand. Then, SARN uses these spatial relationships to infer the 3D position of each hand keypoint. To verify the effectiveness of the proposed method, we conducted experiments on three open-source datasets of 3D hand poses: NYU, ICVL, and MSRA. The proposed method achieved state-of-the-art performance with 7.32 mm, 5.91 mm, and 7.17 mm of mean error at the hand keypoints, i.e., mean Euclidean distance between the predicted and ground-truth hand keypoint positions. Additionally, to test the feasibility of SARN in hand movement recognition, a hand movement dataset of 26K depth images from 17 healthy subjects was constructed based on the finger tapping test, an important component of neurological exams administered to Parkinson's patients. Each image was annotated with the tips of the index finger and the thumb. For this dataset, the proposed method achieved a mean error of 2.99 mm at the hand keypoints and comparable performance on three task-specific metrics: the distance, velocity, and acceleration of the relative movement of the two fingertips. Results on the open-source datasets demonstrated the effectiveness of the proposed method, and results on our finger tapping dataset validated its potential for applications in functional task characterization.
Collapse
Affiliation(s)
- Chenfei Zhu
- Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA
| | - Boce Hu
- Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA
| | - Jiawei Chen
- Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA
| | - Xupeng Ai
- Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA
| | - Sunil K. Agrawal
- Department of Mechanical Engineering, Columbia University, New York, NY 10027, USA
- Department of Rehabilitation Medicine, Columbia University, New York, NY 10027, USA
- Correspondence:
| |
Collapse
|
12
|
Baker S, Tekriwal A, Felsen G, Christensen E, Hirt L, Ojemann SG, Kramer DR, Kern DS, Thompson JA. Automatic extraction of upper-limb kinematic activity using deep learning-based markerless tracking during deep brain stimulation implantation for Parkinson's disease: A proof of concept study. PLoS One 2022; 17:e0275490. [PMID: 36264986 PMCID: PMC9584454 DOI: 10.1371/journal.pone.0275490] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 09/16/2022] [Indexed: 11/12/2022] Open
Abstract
Optimal placement of deep brain stimulation (DBS) therapy for treating movement disorders routinely relies on intraoperative motor testing for target determination. However, in current practice, motor testing relies on subjective interpretation and correlation of motor and neural information. Recent advances in computer vision could improve assessment accuracy. We describe our application of deep learning-based computer vision to conduct markerless tracking for measuring motor behaviors of patients undergoing DBS surgery for the treatment of Parkinson's disease. Video recordings were acquired during intraoperative kinematic testing (N = 5 patients), as part of standard of care for accurate implantation of the DBS electrode. Kinematic data were extracted from videos post-hoc using the Python-based computer vision suite DeepLabCut. Both manual and automated (80.00% accuracy) approaches were used to extract kinematic episodes from threshold derived kinematic fluctuations. Active motor epochs were compressed by modeling upper limb deflections with a parabolic fit. A semi-supervised classification model, support vector machine (SVM), trained on the parameters defined by the parabolic fit reliably predicted movement type. Across all cases, tracking was well calibrated (i.e., reprojection pixel errors 0.016-0.041; accuracies >95%). SVM predicted classification demonstrated high accuracy (85.70%) including for two common upper limb movements, arm chain pulls (92.30%) and hand clenches (76.20%), with accuracy validated using a leave-one-out process for each patient. These results demonstrate successful capture and categorization of motor behaviors critical for assessing the optimal brain target for DBS surgery. Conventional motor testing procedures have proven informative and contributory to targeting but have largely remained subjective and inaccessible to non-Western and rural DBS centers with limited resources. This approach could automate the process and improve accuracy for neuro-motor mapping, to improve surgical targeting, optimize DBS therapy, provide accessible avenues for neuro-motor mapping and DBS implantation, and advance our understanding of the function of different brain areas.
Collapse
Affiliation(s)
- Sunderland Baker
- Department of Human Biology and Kinesiology, Colorado College, Colorado Springs, Colorado, United States of America
| | - Anand Tekriwal
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Department of Physiology and Biophysics, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Neuroscience Graduate Program, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Medical Scientist Training Program, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Gidon Felsen
- Department of Physiology and Biophysics, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Elijah Christensen
- Neuroscience Graduate Program, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Medical Scientist Training Program, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Lisa Hirt
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Steven G. Ojemann
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Daniel R. Kramer
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - Drew S. Kern
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Department of Neurology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
| | - John A. Thompson
- Department of Neurosurgery, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Neuroscience Graduate Program, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- Department of Neurology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, United States of America
- * E-mail:
| |
Collapse
|
13
|
Vignoud G, Desjardins C, Salardaine Q, Mongin M, Garcin B, Venance L, Degos B. Video-Based Automated Assessment of Movement Parameters Consistent with MDS-UPDRS III in Parkinson's Disease. JOURNAL OF PARKINSON'S DISEASE 2022; 12:2211-2222. [PMID: 35964204 PMCID: PMC9661322 DOI: 10.3233/jpd-223445] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/24/2022] [Indexed: 06/01/2023]
Abstract
BACKGROUND Among motor symptoms of Parkinson's disease (PD), including rigidity and resting tremor, bradykinesia is a mandatory feature to define the parkinsonian syndrome. MDS-UPDRS III is the worldwide reference scale to evaluate the parkinsonian motor impairment, especially bradykinesia. However, MDS-UPDRS III is an agent-based score making reproducible measurements and follow-up challenging. OBJECTIVE Using a deep learning approach, we developed a tool to compute an objective score of bradykinesia based on the guidelines of the gold-standard MDS-UPDRS III. METHODS We adapted and applied two deep learning algorithms to detect a two-dimensional (2D) skeleton of the hand composed of 21 predefined points, and transposed it into a three-dimensional (3D) skeleton for a large database of videos of parkinsonian patients performing MDS-UPDRS III protocols acquired in the Movement Disorder unit of Avicenne University Hospital. RESULTS We developed a 2D and 3D automated analysis tool to study the evolution of several key parameters during the protocol repetitions of the MDS-UPDRS III. Scores from 2D automated analysis showed a significant correlation with gold-standard ratings of MDS-UPDRS III, measured with coefficients of determination for the tapping (0.609) and hand movements (0.701) protocols using decision tree algorithms. The individual correlations of the different parameters measured with MDS-UPDRS III scores carry meaningful information and are consistent with MDS-UPDRS III guidelines. CONCLUSION We developed a deep learning-based tool to precisely analyze movement parameters allowing to reliably score bradykinesia for parkinsonian patients in a MDS-UPDRS manner.
Collapse
Affiliation(s)
- Gaëtan Vignoud
- Center for Interdisciplinary Research in Biology (CIRB), Collège de France, CNRS, INSERM, Université PSL, Paris, France
- INRIA Paris, MAMBA (Modelling and Analysis in Medical and Biological Applications), Paris, France
| | - Clément Desjardins
- APHP, Hôpital Avicenne, Hôpitaux Universitaires de Paris-Seine Saint Denis (HUPSSD), Department of Neurology, Sorbonne Paris Nord, NS-PARK/FCRIN network, Bobigny, France
| | - Quentin Salardaine
- APHP, Hôpital Avicenne, Hôpitaux Universitaires de Paris-Seine Saint Denis (HUPSSD), Department of Neurology, Sorbonne Paris Nord, NS-PARK/FCRIN network, Bobigny, France
| | - Marie Mongin
- APHP, Hôpital Avicenne, Hôpitaux Universitaires de Paris-Seine Saint Denis (HUPSSD), Department of Neurology, Sorbonne Paris Nord, NS-PARK/FCRIN network, Bobigny, France
| | - Béatrice Garcin
- APHP, Hôpital Avicenne, Hôpitaux Universitaires de Paris-Seine Saint Denis (HUPSSD), Department of Neurology, Sorbonne Paris Nord, NS-PARK/FCRIN network, Bobigny, France
| | - Laurent Venance
- Center for Interdisciplinary Research in Biology (CIRB), Collège de France, CNRS, INSERM, Université PSL, Paris, France
| | - Bertrand Degos
- Center for Interdisciplinary Research in Biology (CIRB), Collège de France, CNRS, INSERM, Université PSL, Paris, France
- APHP, Hôpital Avicenne, Hôpitaux Universitaires de Paris-Seine Saint Denis (HUPSSD), Department of Neurology, Sorbonne Paris Nord, NS-PARK/FCRIN network, Bobigny, France
| |
Collapse
|