A 10-item Fugl-Meyer Motor Scale Based on Machine Learning

Machine Learning-based World Health Organization Disability Assessment Schedule for persons with Parkinson's disease

INTRODUCTION

The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) is a well-known measure to assess disability in persons with Parkinson's disease (PD). The purpose of this study was to develop a short form of the WHODAS 2.0 for persons with PD using a machine learning-based methodology (ML-WHODAS) and to examine the efficiency (i.e., number of items needed to be administered) and validity of the ML-WHODAS.

METHODS

A secondary data analysis was performed. Data were randomly assigned to training datasets (80 %) and validation datasets (20 %). For developing the ML-WHODAS, the eXtreme Gradient Boosting (XGBoost) regressor was used to select the most informative items from the training datasets, and then the final XGBoost model was generated. The efficiency, concurrent validity, and convergent validity of the ML-WHODAS were then examined using the validation dataset.

RESULTS

Data from 1633 patients were randomly assigned into the training dataset (1306) and the validation dataset (327). Eighteen items were selected for the ML-WHODAS to reproduce 6 domain scores and one global score of the original WHODAS 2.0. In the validation dataset, the Pearson's coefficients r between the scores of the ML-WHODAS and WHODAS 2.0 were 0.97-0.99, indicating very high concurrent validity. Significant correlations were found regarding convergent validity of the domain and global scores.

CONCLUSIONS

The ML-WHODAS showed good efficiency and validity compared to the WHODAS 2.0 in persons with PD. The ML-WHODAS demonstrates its potential as an alternative to the WHODAS 2.0, though further validations (e.g., test-retest reliability and responsiveness) are warranted.

Test-Retest Reliability and Responsiveness of the Machine Learning-Based Short-Form of the Berg Balance Scale in Persons With Stroke

OBJECTIVE

To examine the test-retest reliability, responsiveness, and clinical utility of the machine learning-based short form of the Berg Balance Scale (BBS-ML) in persons with stroke.

DESIGN

Repeated-measures design.

SETTING

A department of rehabilitation in a medical center.

PARTICIPANTS

This study recruited 2 groups: 50 persons who were more than 6 months post-stroke to examine the test-retest reliability, and 52 persons who were within 3 months post-stroke to examine the responsiveness. Test-retest reliability was investigated by administering assessments twice at a 2-week interval. Responsiveness was investigated by gathering data at admission and discharge from the hospital.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

BBS-ML.

RESULTS

The BBS-ML exhibited excellent test-retest reliability (intraclass correlation coefficient=0.99), acceptable minimal random measurement error (minimal detectable change %=13.6%), and good responsiveness (Kazis' effect size and standardized response mean values≥1.34). On average, the participants completed the BBS-ML in around 6 minutes per administration.

CONCLUSIONS

Our findings indicate that the BBS-ML appears an efficient measure with excellent test-retest reliability and responsiveness. Moreover, the BBS-ML may be used as a substitute for the original BBS to monitor the progress of balance function in persons with stroke.

An abbreviated Chinese dyslexia screening behavior checklist for primary school students using a machine learning approach

To increase early identification and intervention of dyslexia, a prescreening instrument is critical to identifying children at risk. The present work sought to shorten and validate the 30-item Mandarin Dyslexia Screening Behavior Checklist for Primary School Students (the full checklist; Fan et al., , 19, 521-527, 2021). Our participants were 15,522 Mandarin-Chinese-speaking students and their parents, sampled from classrooms in grades 2-6 across regions in mainland China. A machine learning approach (lasso regression) was applied to shorten the full checklist (Fan et al., , 19, 521-527, 2021), constructing grade-specific brief checklists first, followed by a compilation of the common brief checklist based on the similarity across grade-specific checklists. All checklists (the full, grade-specific brief, and common brief versions) were validated and compared with data in our sample and an external sample (N = 114; Fan et al., , 19, 521-527, 2021). The results indicated that the six-item common brief checklist showed consistently high reliability (αs > .82) and reasonable classification performance (about 60% prediction accuracy and 70% sensitivity), comparable to that of the full checklist and all grade-specific brief checklists across our current sample and the external sample from Fan et al., , 19, 521-527, (2021). Our analysis showed that 2.42 (out of 5) was the cutoff score that helped classify children's reading status (children who scored higher than 2.42 might be considered at risk for dyslexia). Our final product is a valid, accessible, common brief checklist for prescreening primary school children at risk for Chinese dyslexia, which can be used across grades and regions in mainland China.

Virtual reality assessment of reaching accuracy in patients with recent cerebellar stroke

Background

Dysmetria, the inability to accurately estimate distance in motor tasks, is a characteristic clinical feature of cerebellar injury. Even though subjective dysmetria can be quickly detected during the neurological examination with the finger-to-nose test, objective quantification of reaching accuracy for clinical assessment is still lacking. Emerging VR technology allows for the delivery of rich multisensory environmental stimuli with a high degree of control. Furthermore, recent improvements in the hand-tracking feature offer an opportunity to closely examine the speed, accuracy, and consistency of fine hand movements and proprioceptive function. This study aims to investigate the application of virtual reality (VR) with hand tracking in the rapid quantification of reaching accuracy at the bedside for patients with cerebellar stroke (CS).

Methods and results

Thirty individuals (10 CS patients and 20 age-matched neurologically healthy controls) performed a simple task that allowed us to measure reaching accuracy using a VR headset (Oculus Quest 2). During this task, the participant was asked to reach for a target placed along a horizontal sixty-degree arc. Once the fingertip passed through the arc, the target immediately extinguished. 50% of the trials displayed a visible, real-time rendering of the hand as the participant reached for the target (visible hand condition), while the remaining 50% only showed the target being extinguished (invisible hand condition). The invisible hand condition isolates proprioception-guided movements by removing the visibility of the participant's hand. Reaching error was calculated as the difference in degrees between the location of the target, and where the fingertip contacted the arc. Both CS patients and age-matched controls displayed higher average reaching error and took longer to perform a reaching motion in the invisible hand condition than in the visible hand condition. Reaching error was higher in CS than in controls in the invisible hand condition but not in the visible hand condition. Average time taken to perform each trial was higher in CS than in controls in the invisible hand conditions but not in the visible hand condition.

Conclusions

Reaching accuracy assessed by VR offers a non-invasive and rapid approach to quantifying fine motor functions in clinical settings. Furthermore, this technology enhances our understanding of proprioceptive function in patients with visuomotor disabilities by allowing the isolation of proprioception from vision. Future studies with larger cohorts and longitudinal designs will examine the quantitative changes in reaching accuracy after stroke and explore the long-term benefits of VR in functional recovery.

Developing an Accumulative Assessment System of Upper Extremity Motor Function in Patients With Stroke Using Deep Learning

OBJECTIVE

The Fugl-Meyer assessment for upper extremity (FMA-UE) is a measure for assessing upper extremity motor function in patients with stroke. However, the considerable administration time of the assessment decreases its feasibility. This study aimed to develop an accumulative assessment system of upper extremity motor function (AAS-UE) based on the FMA-UE to improve administrative efficiency while retaining sufficient psychometric properties.

METHODS

The study used secondary data from 3 previous studies having FMA-UE datasets, including 2 follow-up studies for subacute stroke individuals and 1 test-retest study for individuals with chronic stroke. The AAS-UE adopted deep learning algorithms to use patients' prior information (ie, the FMA-UE scores in previous assessments, time interval of adjacent assessments, and chronicity of stroke) to select a short and personalized item set for the following assessment items and reproduce their FMA-UE scores.

RESULTS

Our data included a total of 682 patients after stroke. The AAS-UE administered 10 different items for each patient. The AAS-UE demonstrated good concurrent validity (r = 0.97-0.99 with the FMA-UE), high test-retest reliability (intra-class correlation coefficient = 0.96), low random measurement error (percentage of minimal detectable change = 15.6%), good group-level responsiveness (standardized response mean = 0.65-1.07), and good individual-level responsiveness (30.5%-53.2% of patients showed significant improvement). These psychometric properties were comparable to those of the FMA-UE.

CONCLUSION

The AAS-UE uses an innovative assessment method, which makes good use of patients' prior information to achieve administrative efficiency with good psychometric properties.

IMPACT

This study demonstrates a new assessment method to improve administrative efficiency while retaining psychometric properties, especially individual-level responsiveness and random measurement error, by making good use of patients' basic information and medical records.

Developing a Short-Form Buss-Warren Aggression Questionnaire Based on Machine Learning

For adolescents, high levels of aggression are often associated with suicide, physical injury, worsened academic performance, and crime. Therefore, there is a need for the early identification of and intervention for highly aggressive adolescents. The Buss-Warren Aggression Questionnaire (BWAQ) is one of the most widely used offensive measurement tools. It consists of 34 items, and the longer the scale, the more likely participants are to make an insufficient effort response (IER), which reduces the credibility of the results and increases the cost of implementation. This study aimed to develop a shorter BWAQ using machine learning (ML) techniques to reduce the frequency of IER and simultaneously decrease implementation costs. First, an initial version of the short-form questionnaire was created using stepwise regression and an ANOVA F-test. Then, a machine learning algorithm was used to create the optimal short-form questionnaire (BWAQ-ML). Finally, the reliability and validity of the optimal short-form questionnaire were tested using independent samples. The BWAQ-ML contains only four items, thirty items less than the BWAQ, and its AUC, accuracy, recall, precision, and F1 score are 0.85, 0.85, 0.89, 0.83, and 0.86, respectively. BWAQ-ML has a Cronbach's alpha of 0.84, a correlation with RPQ of 0.514, and a correlation with PTM of -0.042, suggesting good measurement performance. The BWAQ-ML can effectively measure individual aggression, and its smaller number of items improves the measurement efficiency for large samples and reduces the frequency of IER occurrence. It can be used as a convenient tool for early adolescent aggression identification and intervention.

Machine learning-based brief version of the Caregiver-Teacher Report Form for preschoolers

BACKGROUND

The Caregiver-Teacher Report Form of the Child Behavior Checklist for Ages 1½-5 (C-TRF) is a widely used checklist to identify emotional and behavioral problems in preschoolers. However, the 100-item C-TRF restricts its utility.

AIMS

This study aimed to develop a machine learning-based short-form of the C-TRF (C-TRF-ML).

METHODS AND PROCEDURES

Three steps were executed. First, we split the data into three datasets in a ratio of 3:1:1 for training, validation, and cross-validation, respectively. Second, we selected a shortened item set and trained a scoring algorithm using joint learning for classification and regression using the training dataset. Then, we evaluated the similarity of scores between the C-TRF-ML and the C-TRF by r-squared and weighted kappa values using the validation dataset. Third, we cross-validated the C-TRF-ML by calculating the r-squared and weighted kappa values using the cross-validation dataset.

OUTCOMES AND RESULTS

Data of 363 children were analyzed. Thirty-six items of the C-TRF were retained. The r-squared values of C-TRF-ML scores were 0.86-0.96 in the cross-validation dataset. Weighted kappa values of the syndrome/problem grading were 0.72-0.94 in the cross-validation dataset.

CONCLUSIONS AND IMPLICATIONS

The C-TRF-ML had about 60 % fewer items than the C-TRF but yielded comparable scores with the C-TRF.

Development of a 13-item Short Form for Fugl-Meyer Assessment of Upper Extremity Scale Using a Machine Learning Approach

OBJECTIVE

To develop and validate a short form of the Fugl-Meyer Assessment of Upper Extremity Scale (FMA-UE) using a machine learning approach (FMA-UE-ML). In addition, scores of items not included in the FMA-UE-ML were predicted.

DESIGN

Secondary data from a previous study, which assessed individuals post-stroke using the FMA-UE at 4 time points: 5-30 days post-stroke screen, 2-month post-stroke baseline assessment, 6-month post-stroke assessment, and 12-month post-stroke assessment.

SETTING

Rehabilitation units in hospitals.

PARTICIPANTS

A total of 408 individuals post-stroke (N=408).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

The 30-item FMA-UE.

RESULTS

We established 29 candidate versions of the FMA-UE-ML with different numbers of items, from 1 to 29, and examined their concurrent validity and responsiveness. We found that the responsiveness of the candidate versions obviously declined when the number of items was less than 13. Thus, the 13-item version was selected as the FMA-UE-ML. The concurrent validity was good (intra-class correlation coefficients ≥0.99). The standardized response means of the FMA-UE-ML and FMA-UE were 0.54-0.88 and 0.52-0.91, respectively. The Pearson's rs between the change scores of the FMA-UE-ML and those of the FMA-UE were 0.96-0.98. The predicted item scores had acceptable to good accuracy (Kappa=0.50-0.92).

CONCLUSIONS

The FMA-UE-ML seems a promising short form to improve administrative efficiency while retaining good concurrent validity and responsiveness. In addition, the FAM-UE-ML can provide all item scores of the FMA-UE for users.

Development of a Berg Balance Scale Short-Form Using a Machine Learning Approach in Patients With Stroke

BACKGROUND AND PURPOSE

The Berg Balance Scale (BBS) is frequently used in routine clinical care and research settings and has good psychometric properties. This study was conducted to develop a short form of the BBS using a machine learning approach (BBS-ML).

METHODS

Data of 408 individuals poststroke were extracted from a published database. The initial (ie, 4-, 5-, 6-, 7-, and 8-item) versions were constructed by selecting top-ranked items based on the feature selection algorithm in the artificial neural network model. The final version of the BBS-ML was chosen by selecting the short form that used a smaller number of items to achieve a higher predictive power R2 , a lower 95% limit of agreement (LoA), and an adequate possible scoring point (PSP). An independent sample of 226 persons with stroke was used for external validation.

RESULTS

The R2 values for the initial 4-, 5-, 6-, 7-, and 8-item short forms were 0.93, 0.95, 0.97, 0.97, and 0.97, respectively. The 95% LoAs were 14.2, 12.2, 9.7, 9.6, and 8.9, respectively. The PSPs were 25, 35, 34, 35, and 36, respectively. The 6-item version was selected as the final BBS-ML. Preliminary external validation supported its performance in an independent sample of persons with stroke ( R2 = 0.99, LoA = 10.6, PSP = 37).

DISCUSSION AND CONCLUSIONS

The BBS-ML seems to be a promising short-form alternative to improve administrative efficiency. Future research is needed to examine the psychometric properties and clinical usage of the 6-item BBS-ML in various settings and samples.Video Abstract available for more insights from the authors (see the Video, Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A402 ).

Development of a Short-Form Stroke Impact Scale Using a Machine Learning Algorithm for Patients at the Subacute Stage

Importance: Several short forms of the Stroke Impact Scale Version 3.0 (SIS 3.0) have been proposed in order to decrease its administration time of about 20 min. However, none of the short-form scores are comparable to those of the original measure.

Objective: To develop a short-form SIS 3.0 using a machine learning algorithm (ML–SIS).

Design: We developed the ML–SIS in three stages. First, we calculated the frequencies of items having the highest contribution to predicting the original domain scores across 50 deep neural networks. Second, we iteratively selected the items showing the highest frequency until the coefficient of determination (R2) of each domain was ≥.90. Third, we examined the comparability and concurrent and convergent validity of the ML–SIS.

Setting: Hospitals.

Participants: We extracted complete data for 1,010 patients from an existing data set.

Results: Twenty-eight items were selected for the ML–SIS. High average R2s (.90–.96) and small average residuals (mean absolute errors and root-mean-square errors = 0.49–2.84) indicate good comparability. High correlations (rs = .95–.98) between the eight domain scores of the ML–SIS and the SIS 3.0 indicate sufficient concurrent validity. Similar interdomain correlations between the two measures indicate satisfactory convergent validity.

Conclusions and Relevance: The ML–SIS uses about half of the items in the SIS 3.0, has an estimated administration time of 10 min, and provides valid scores comparable to those of the original measure. Thus, the ML–SIS may be an efficient alternative to the SIS 3.0.

What This Article Adds: The ML–SIS, a short form of the SIS 3.0 developed using a machine learning algorithm, shows good potential to be an efficient and informative measure for clinical settings, providing scores that are valid and comparable to those of the original measure.

Using Machine Learning to develop a short-form measure assessing 5 functions in patients with stroke

OBJECTIVE

This study aimed to develop and validate a machine learning based short measure (the ML-5F) to assess 5 functions (activities of daily living (ADL), balance, upper extremity (UE) and lower extremity (LE) motor function, and mobility) in patients with stroke.

DESIGN

Secondary data from a previous study. A follow-up study assessed patients with stroke using the Barthel Index (BI), Postural Assessment Scale for Stroke (PASS), and Stroke Rehabilitation Assessment of Movement (STREAM) at hospital admission and discharge.

SETTING

A rehabilitation unit in a medical center.

PARTICIPANTS

A total of 307 patients.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

The BI, PASS, and STREAM.

RESULTS

A machine learning algorithm, Extreme Gradient Boosting, was used to select 15 items from the BI, PASS, and STREAM, and transformed the raw scores of the selected items into the scores of the ML-5F. The ML-5F demonstrated good concurrent validity (Pearson's r = 0.88-0.98) and responsiveness (standardized response mean = 0.28-1.01).

CONCLUSIONS

The ML-5F comprises only 15 items but demonstrates sufficient concurrent validity and responsiveness to assess ADL, balance, UE and LE functions, and mobility in patients with stroke. The ML-5F shows great potential as an efficient outcome measure in clinical settings.