1
|
Lu M, Zhao Q, Poston KL, Sullivan EV, Pfefferbaum A, Shahid M, Katz M, Montaser-Kouhsari L, Schulman K, Milstein A, Niebles JC, Henderson VW, Fei-Fei L, Pohl KM, Adeli E. Quantifying Parkinson's disease motor severity under uncertainty using MDS-UPDRS videos. Med Image Anal 2021; 73:102179. [PMID: 34340101 PMCID: PMC8453121 DOI: 10.1016/j.media.2021.102179] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Revised: 06/28/2021] [Accepted: 07/13/2021] [Indexed: 11/15/2022]
Abstract
Parkinson's disease (PD) is a brain disorder that primarily affects motor function, leading to slow movement, tremor, and stiffness, as well as postural instability and difficulty with walking/balance. The severity of PD motor impairments is clinically assessed by part III of the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a universally-accepted rating scale. However, experts often disagree on the exact scoring of individuals. In the presence of label noise, training a machine learning model using only scores from a single rater may introduce bias, while training models with multiple noisy ratings is a challenging task due to the inter-rater variabilities. In this paper, we introduce an ordinal focal neural network to estimate the MDS-UPDRS scores from input videos, to leverage the ordinal nature of MDS-UPDRS scores and combat class imbalance. To handle multiple noisy labels per exam, the training of the network is regularized via rater confusion estimation (RCE), which encodes the rating habits and skills of raters via a confusion matrix. We apply our pipeline to estimate MDS-UPDRS test scores from their video recordings including gait (with multiple Raters, R=3) and finger tapping scores (single rater). On a sizable clinical dataset for the gait test (N=55), we obtained a classification accuracy of 72% with majority vote as ground-truth, and an accuracy of ∼84% of our model predicting at least one of the raters' scores. Our work demonstrates how computer-assisted technologies can be used to track patients and their motor impairments, even when there is uncertainty in the clinical ratings. The latest version of the code will be available at https://github.com/mlu355/PD-Motor-Severity-Estimation.
Collapse
Affiliation(s)
- Mandy Lu
- Department of Computer Science, Stanford University, Stanford CA 94305, USA
| | - Qingyu Zhao
- Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford CA 94305, USA
| | - Kathleen L Poston
- Department of Neurology & Neurological Sciences, Stanford University, Stanford CA 94305, USA
| | - Edith V Sullivan
- Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford CA 94305, USA
| | - Adolf Pfefferbaum
- Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford CA 94305, USA; Center for Health Sciences, SRI International, Menlo Park CA 94025, USA
| | - Marian Shahid
- Department of Neurology & Neurological Sciences, Stanford University, Stanford CA 94305, USA
| | - Maya Katz
- Department of Neurology & Neurological Sciences, Stanford University, Stanford CA 94305, USA
| | - Leila Montaser-Kouhsari
- Department of Neurology & Neurological Sciences, Stanford University, Stanford CA 94305, USA
| | - Kevin Schulman
- Department of Medicine, Stanford University, Stanford CA 94305, USA
| | - Arnold Milstein
- Department of Medicine, Stanford University, Stanford CA 94305, USA
| | | | - Victor W Henderson
- Department of Epidemiology & Population Health, Stanford University, Stanford CA 94305, USA; Department of Neurology & Neurological Sciences, Stanford University, Stanford CA 94305, USA
| | - Li Fei-Fei
- Department of Computer Science, Stanford University, Stanford CA 94305, USA
| | - Kilian M Pohl
- Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford CA 94305, USA; Center for Health Sciences, SRI International, Menlo Park CA 94025, USA
| | - Ehsan Adeli
- Department of Computer Science, Stanford University, Stanford CA 94305, USA; Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford CA 94305, USA.
| |
Collapse
|
2
|
Adeli E, Zhao Q, Pfefferbaum A, Sullivan EV, Fei-Fei L, Niebles JC, Pohl KM. Representation Learning with Statistical Independence to Mitigate Bias. IEEE Winter Conf Appl Comput Vis 2021; 2021:2512-2522. [PMID: 34522832 PMCID: PMC8436589 DOI: 10.1109/wacv48630.2021.00256] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Presence of bias (in datasets or tasks) is inarguably one of the most critical challenges in machine learning applications that has alluded to pivotal debates in recent years. Such challenges range from spurious associations between variables in medical studies to the bias of race in gender or face recognition systems. Controlling for all types of biases in the dataset curation stage is cumbersome and sometimes impossible. The alternative is to use the available data and build models incorporating fair representation learning. In this paper, we propose such a model based on adversarial training with two competing objectives to learn features that have (1) maximum discriminative power with respect to the task and (2) minimal statistical mean dependence with the protected (bias) variable(s). Our approach does so by incorporating a new adversarial loss function that encourages a vanished correlation between the bias and the learned features. We apply our method to synthetic data, medical images (containing task bias), and a dataset for gender classification (containing dataset bias). Our results show that the learned features by our method not only result in superior prediction performance but also are unbiased.
Collapse
Affiliation(s)
- Ehsan Adeli
- Department of Psychiatry and Behavioral Sciences, Stanford University, CA 94305.,Department of Computer Science, Stanford University, CA 94305
| | - Qingyu Zhao
- Department of Psychiatry and Behavioral Sciences, Stanford University, CA 94305
| | - Adolf Pfefferbaum
- Department of Psychiatry and Behavioral Sciences, Stanford University, CA 94305.,Center for Biomedical Sciences, SRI International, Menlo Park, CA 94205
| | - Edith V Sullivan
- Department of Psychiatry and Behavioral Sciences, Stanford University, CA 94305
| | - Li Fei-Fei
- Department of Computer Science, Stanford University, CA 94305
| | | | - Kilian M Pohl
- Department of Psychiatry and Behavioral Sciences, Stanford University, CA 94305.,Center for Biomedical Sciences, SRI International, Menlo Park, CA 94205
| |
Collapse
|
3
|
Lu M, Zhao Q, Zhang J, Pohl KM, Fei-Fei L, Niebles JC, Adeli E. Metadata Normalization. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2021; 2021:10912-10922. [PMID: 34776724 PMCID: PMC8589298 DOI: 10.1109/cvpr46437.2021.01077] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Batch Normalization (BN) and its variants have delivered tremendous success in combating the covariate shift induced by the training step of deep learning methods. While these techniques normalize feature distributions by standardizing with batch statistics, they do not correct the influence on features from extraneous variables or multiple distributions. Such extra variables, referred to as metadata here, may create bias or confounding effects (e.g., race when classifying gender from face images). We introduce the Metadata Normalization (MDN) layer, a new batch-level operation which can be used end-to-end within the training framework, to correct the influence of metadata on feature distributions. MDN adopts a regression analysis technique traditionally used for preprocessing to remove (regress out) the metadata effects on model features during training. We utilize a metric based on distance correlation to quantify the distribution bias from the metadata and demonstrate that our method successfully removes metadata effects on four diverse settings: one synthetic, one 2D image, one video, and one 3D medical image dataset.
Collapse
Affiliation(s)
- Mandy Lu
- Stanford University, Stanford, CA 94305
| | | | | | | | | | | | | |
Collapse
|
4
|
|
5
|
Lu M, Poston K, Pfefferbaum A, Sullivan EV, Fei-Fei L, Pohl KM, Niebles JC, Adeli E. Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity. Med Image Comput Comput Assist Interv 2020; 12263:637-647. [PMID: 33103164 PMCID: PMC7585545 DOI: 10.1007/978-3-030-59716-0_61] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Parkinson's disease (PD) is a progressive neurological disorder primarily affecting motor function resulting in tremor at rest, rigidity, bradykinesia, and postural instability. The physical severity of PD impairments can be quantified through the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a widely used clinical rating scale. Accurate and quantitative assessment of disease progression is critical to developing a treatment that slows or stops further advancement of the disease. Prior work has mainly focused on dopamine transport neuroimaging for diagnosis or costly and intrusive wearables evaluating motor impairments. For the first time, we propose a computer vision-based model that observes non-intrusive video recordings of individuals, extracts their 3D body skeletons, tracks them through time, and classifies the movements according to the MDS-UPDRS gait scores. Experimental results show that our proposed method performs significantly better than chance and competing methods with an F 1-score of 0.83 and a balanced accuracy of 81%. This is the first benchmark for classifying PD patients based on MDS-UPDRS gait severity and could be an objective biomarker for disease severity. Our work demonstrates how computer-assisted technologies can be used to non-intrusively monitor patients and their motor impairments. The code is available at https://github.com/mlu355/PD-Motor-Severity-Estimation.
Collapse
Affiliation(s)
- Mandy Lu
- Computer Science Department, Stanford University, Stanford, CA, USA
| | | | - Adolf Pfefferbaum
- School of Medicine, Stanford University, Stanford, CA, USA
- Center of Health Sciences, SRI International, Menlo Park, CA, USA
| | | | - Li Fei-Fei
- Computer Science Department, Stanford University, Stanford, CA, USA
| | - Kilian M Pohl
- School of Medicine, Stanford University, Stanford, CA, USA
- Center of Health Sciences, SRI International, Menlo Park, CA, USA
| | | | - Ehsan Adeli
- Computer Science Department, Stanford University, Stanford, CA, USA
- School of Medicine, Stanford University, Stanford, CA, USA
| |
Collapse
|
6
|
|
7
|
Liu B, Adeli E, Cao Z, Lee KH, Shenoi A, Gaidon A, Niebles JC. Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2976305] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
8
|
Niebles JC, Chen CW, Fei-Fei L. Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification. Computer Vision – ECCV 2010 2010. [DOI: 10.1007/978-3-642-15552-9_29] [Citation(s) in RCA: 238] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
|
9
|
|