1
|
Lee ML, Lin GH, Wang YC, Lee SC, Hsieh CL. Machine Learning-based World Health Organization Disability Assessment Schedule for persons with Parkinson's disease. Parkinsonism Relat Disord 2025; 133:107316. [PMID: 39933317 DOI: 10.1016/j.parkreldis.2025.107316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/18/2024] [Revised: 01/31/2025] [Accepted: 02/01/2025] [Indexed: 02/13/2025]
Abstract
INTRODUCTION The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) is a well-known measure to assess disability in persons with Parkinson's disease (PD). The purpose of this study was to develop a short form of the WHODAS 2.0 for persons with PD using a machine learning-based methodology (ML-WHODAS) and to examine the efficiency (i.e., number of items needed to be administered) and validity of the ML-WHODAS. METHODS A secondary data analysis was performed. Data were randomly assigned to training datasets (80 %) and validation datasets (20 %). For developing the ML-WHODAS, the eXtreme Gradient Boosting (XGBoost) regressor was used to select the most informative items from the training datasets, and then the final XGBoost model was generated. The efficiency, concurrent validity, and convergent validity of the ML-WHODAS were then examined using the validation dataset. RESULTS Data from 1633 patients were randomly assigned into the training dataset (1306) and the validation dataset (327). Eighteen items were selected for the ML-WHODAS to reproduce 6 domain scores and one global score of the original WHODAS 2.0. In the validation dataset, the Pearson's coefficients r between the scores of the ML-WHODAS and WHODAS 2.0 were 0.97-0.99, indicating very high concurrent validity. Significant correlations were found regarding convergent validity of the domain and global scores. CONCLUSIONS The ML-WHODAS showed good efficiency and validity compared to the WHODAS 2.0 in persons with PD. The ML-WHODAS demonstrates its potential as an alternative to the WHODAS 2.0, though further validations (e.g., test-retest reliability and responsiveness) are warranted.
Collapse
Affiliation(s)
- Meng-Lin Lee
- Division of Cardiovascular Surgery, Department of Surgery, Cathay General Hospital, Taipei, Taiwan; School of Medicine, National Tsing Hua University, Hsinchu, Taiwan.
| | - Gong-Hong Lin
- International Ph.D. Program in Gerontology and Long-Term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan.
| | - Yi-Ching Wang
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan.
| | - Shih-Chieh Lee
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan.
| | - Ching-Lin Hsieh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Occupational Therapy, College of Medical and Health Sciences, Asia University, Taichung, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan.
| |
Collapse
|
2
|
Chen PT, Hsueh IP, Lee SC, Lee ML, Twu CW, Hsieh CL. Test-Retest Reliability and Responsiveness of the Machine Learning-Based Short-Form of the Berg Balance Scale in Persons With Stroke. Arch Phys Med Rehabil 2024:S0003-9993(24)01319-4. [PMID: 39522673 DOI: 10.1016/j.apmr.2024.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2024] [Revised: 10/10/2024] [Accepted: 10/22/2024] [Indexed: 11/16/2024]
Abstract
OBJECTIVE To examine the test-retest reliability, responsiveness, and clinical utility of the machine learning-based short form of the Berg Balance Scale (BBS-ML) in persons with stroke. DESIGN Repeated-measures design. SETTING A department of rehabilitation in a medical center. PARTICIPANTS This study recruited 2 groups: 50 persons who were more than 6 months post-stroke to examine the test-retest reliability, and 52 persons who were within 3 months post-stroke to examine the responsiveness. Test-retest reliability was investigated by administering assessments twice at a 2-week interval. Responsiveness was investigated by gathering data at admission and discharge from the hospital. INTERVENTIONS Not applicable. MAIN OUTCOME MEASURE BBS-ML. RESULTS The BBS-ML exhibited excellent test-retest reliability (intraclass correlation coefficient=0.99), acceptable minimal random measurement error (minimal detectable change %=13.6%), and good responsiveness (Kazis' effect size and standardized response mean values≥1.34). On average, the participants completed the BBS-ML in around 6 minutes per administration. CONCLUSIONS Our findings indicate that the BBS-ML appears an efficient measure with excellent test-retest reliability and responsiveness. Moreover, the BBS-ML may be used as a substitute for the original BBS to monitor the progress of balance function in persons with stroke.
Collapse
Affiliation(s)
- Po-Ting Chen
- Hualien Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, Hualien, Taiwan; School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - I-Ping Hsueh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan
| | - Shih-Chie Lee
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan
| | - Meng-Lin Lee
- Division of Cardiovascular Surgery, Department of Surgery, Cathay General Hospital, Taipei, Taiwan; School of Medicine, National Tsing Hua University, Hsinchu, Taiwan
| | - Chih-Wen Twu
- Department of Otorhinolaryngology-Head and Neck Surgery, Changhua Christian Hospital, Changhua, Taiwan; Department of Quality Management, Changhua Christian Hospital, Changhua, Taiwan; Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan.
| | - Ching-Lin Hsieh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; Department of Occupational Therapy, College of Medical and Health Sciences, Asia University, Taichung, Taiwan
| |
Collapse
|
3
|
Fan Y, Li Y, Luo M, Bai J, Jiang M, Xu Y, Li H. An abbreviated Chinese dyslexia screening behavior checklist for primary school students using a machine learning approach. Behav Res Methods 2024; 56:7892-7911. [PMID: 39075247 DOI: 10.3758/s13428-024-02461-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/10/2024] [Indexed: 07/31/2024]
Abstract
To increase early identification and intervention of dyslexia, a prescreening instrument is critical to identifying children at risk. The present work sought to shorten and validate the 30-item Mandarin Dyslexia Screening Behavior Checklist for Primary School Students (the full checklist; Fan et al., , 19, 521-527, 2021). Our participants were 15,522 Mandarin-Chinese-speaking students and their parents, sampled from classrooms in grades 2-6 across regions in mainland China. A machine learning approach (lasso regression) was applied to shorten the full checklist (Fan et al., , 19, 521-527, 2021), constructing grade-specific brief checklists first, followed by a compilation of the common brief checklist based on the similarity across grade-specific checklists. All checklists (the full, grade-specific brief, and common brief versions) were validated and compared with data in our sample and an external sample (N = 114; Fan et al., , 19, 521-527, 2021). The results indicated that the six-item common brief checklist showed consistently high reliability (αs > .82) and reasonable classification performance (about 60% prediction accuracy and 70% sensitivity), comparable to that of the full checklist and all grade-specific brief checklists across our current sample and the external sample from Fan et al., , 19, 521-527, (2021). Our analysis showed that 2.42 (out of 5) was the cutoff score that helped classify children's reading status (children who scored higher than 2.42 might be considered at risk for dyslexia). Our final product is a valid, accessible, common brief checklist for prescreening primary school children at risk for Chinese dyslexia, which can be used across grades and regions in mainland China.
Collapse
Affiliation(s)
- Yimin Fan
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (Beijing Normal University), Institute of Children's Reading and Learning, Faculty of Psychology, Beijing Normal University, Room 1415, Houzhu Building, No.19 Xinjiekouwai Street, Haidian, Beijing, China
| | - Yixun Li
- Department of Early Childhood Education, The Education University of Hong Kong, Hong Kong SAR, China
| | - Mingyue Luo
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (Beijing Normal University), Institute of Children's Reading and Learning, Faculty of Psychology, Beijing Normal University, Room 1415, Houzhu Building, No.19 Xinjiekouwai Street, Haidian, Beijing, China
| | - Jirong Bai
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (Beijing Normal University), Institute of Children's Reading and Learning, Faculty of Psychology, Beijing Normal University, Room 1415, Houzhu Building, No.19 Xinjiekouwai Street, Haidian, Beijing, China
| | - Mengwen Jiang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (Beijing Normal University), Institute of Children's Reading and Learning, Faculty of Psychology, Beijing Normal University, Room 1415, Houzhu Building, No.19 Xinjiekouwai Street, Haidian, Beijing, China
| | - Yi Xu
- People's Education Press, Curriculum and Teaching Materials Research Institute, Beijing, China
| | - Hong Li
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (Beijing Normal University), Institute of Children's Reading and Learning, Faculty of Psychology, Beijing Normal University, Room 1415, Houzhu Building, No.19 Xinjiekouwai Street, Haidian, Beijing, China.
| |
Collapse
|
4
|
Du K, Benavides LR, Isenstein EL, Tadin D, Busza AC. Virtual reality assessment of reaching accuracy in patients with recent cerebellar stroke. BMC DIGITAL HEALTH 2024; 2:50. [PMID: 39139706 PMCID: PMC11317447 DOI: 10.1186/s44247-024-00107-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 06/05/2024] [Indexed: 08/15/2024]
Abstract
Background Dysmetria, the inability to accurately estimate distance in motor tasks, is a characteristic clinical feature of cerebellar injury. Even though subjective dysmetria can be quickly detected during the neurological examination with the finger-to-nose test, objective quantification of reaching accuracy for clinical assessment is still lacking. Emerging VR technology allows for the delivery of rich multisensory environmental stimuli with a high degree of control. Furthermore, recent improvements in the hand-tracking feature offer an opportunity to closely examine the speed, accuracy, and consistency of fine hand movements and proprioceptive function. This study aims to investigate the application of virtual reality (VR) with hand tracking in the rapid quantification of reaching accuracy at the bedside for patients with cerebellar stroke (CS). Methods and results Thirty individuals (10 CS patients and 20 age-matched neurologically healthy controls) performed a simple task that allowed us to measure reaching accuracy using a VR headset (Oculus Quest 2). During this task, the participant was asked to reach for a target placed along a horizontal sixty-degree arc. Once the fingertip passed through the arc, the target immediately extinguished. 50% of the trials displayed a visible, real-time rendering of the hand as the participant reached for the target (visible hand condition), while the remaining 50% only showed the target being extinguished (invisible hand condition). The invisible hand condition isolates proprioception-guided movements by removing the visibility of the participant's hand. Reaching error was calculated as the difference in degrees between the location of the target, and where the fingertip contacted the arc. Both CS patients and age-matched controls displayed higher average reaching error and took longer to perform a reaching motion in the invisible hand condition than in the visible hand condition. Reaching error was higher in CS than in controls in the invisible hand condition but not in the visible hand condition. Average time taken to perform each trial was higher in CS than in controls in the invisible hand conditions but not in the visible hand condition. Conclusions Reaching accuracy assessed by VR offers a non-invasive and rapid approach to quantifying fine motor functions in clinical settings. Furthermore, this technology enhances our understanding of proprioceptive function in patients with visuomotor disabilities by allowing the isolation of proprioception from vision. Future studies with larger cohorts and longitudinal designs will examine the quantitative changes in reaching accuracy after stroke and explore the long-term benefits of VR in functional recovery.
Collapse
Affiliation(s)
- Khai Du
- Department of Neurology, University of Rochester Medical Center, Rochester, NY USA
| | | | - Emily L. Isenstein
- Department of Neuroscience, University of Rochester Medical Center, Rochester, NY USA
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY USA
- Center for Visual Science, University of Rochester, Rochester, NY USA
| | - Duje Tadin
- Department of Neuroscience, University of Rochester Medical Center, Rochester, NY USA
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY USA
- Center for Visual Science, University of Rochester, Rochester, NY USA
- Department of Ophthalmology, University of Rochester Medical Center, Rochester, NY USA
| | - Ania C. Busza
- Department of Neurology, University of Rochester Medical Center, Rochester, NY USA
- Department of Neuroscience, University of Rochester Medical Center, Rochester, NY USA
- Department of Physical Medicine and Rehabilitation, University of Rochester Medical Center, Rochester, NY USA
| |
Collapse
|
5
|
Lin GH, Lee SC, Huang CY, Wang I, Lee YC, Hsueh IP, Hsieh CL. Developing an Accumulative Assessment System of Upper Extremity Motor Function in Patients With Stroke Using Deep Learning. Phys Ther 2024; 104:pzae050. [PMID: 38531775 DOI: 10.1093/ptj/pzae050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 11/03/2023] [Accepted: 02/13/2024] [Indexed: 03/28/2024]
Abstract
OBJECTIVE The Fugl-Meyer assessment for upper extremity (FMA-UE) is a measure for assessing upper extremity motor function in patients with stroke. However, the considerable administration time of the assessment decreases its feasibility. This study aimed to develop an accumulative assessment system of upper extremity motor function (AAS-UE) based on the FMA-UE to improve administrative efficiency while retaining sufficient psychometric properties. METHODS The study used secondary data from 3 previous studies having FMA-UE datasets, including 2 follow-up studies for subacute stroke individuals and 1 test-retest study for individuals with chronic stroke. The AAS-UE adopted deep learning algorithms to use patients' prior information (ie, the FMA-UE scores in previous assessments, time interval of adjacent assessments, and chronicity of stroke) to select a short and personalized item set for the following assessment items and reproduce their FMA-UE scores. RESULTS Our data included a total of 682 patients after stroke. The AAS-UE administered 10 different items for each patient. The AAS-UE demonstrated good concurrent validity (r = 0.97-0.99 with the FMA-UE), high test-retest reliability (intra-class correlation coefficient = 0.96), low random measurement error (percentage of minimal detectable change = 15.6%), good group-level responsiveness (standardized response mean = 0.65-1.07), and good individual-level responsiveness (30.5%-53.2% of patients showed significant improvement). These psychometric properties were comparable to those of the FMA-UE. CONCLUSION The AAS-UE uses an innovative assessment method, which makes good use of patients' prior information to achieve administrative efficiency with good psychometric properties. IMPACT This study demonstrates a new assessment method to improve administrative efficiency while retaining psychometric properties, especially individual-level responsiveness and random measurement error, by making good use of patients' basic information and medical records.
Collapse
Affiliation(s)
- Gong-Hong Lin
- International Ph.D. Program in Gerontology and Long-Term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan
| | - Shih-Chieh Lee
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Psychiatry, National Taiwan University Hospital, Taipei, Taiwan
| | - Chien-Yu Huang
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan
| | - Inga Wang
- Department of Rehabilitation Sciences & Technology, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin, USA
| | - Ya-Chen Lee
- Department of Occupational Therapy, College of Medical and Health Sciences, Asia University, Taichung, Taiwan
| | - I-Ping Hsueh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan
| | - Ching-Lin Hsieh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan
- Department of Occupational Therapy, College of Medical and Health Sciences, Asia University, Taichung, Taiwan
| |
Collapse
|
6
|
Jiang X, Yang Y, Li J. Developing a Short-Form Buss-Warren Aggression Questionnaire Based on Machine Learning. Behav Sci (Basel) 2023; 13:799. [PMID: 37887449 PMCID: PMC10604583 DOI: 10.3390/bs13100799] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Revised: 09/20/2023] [Accepted: 09/25/2023] [Indexed: 10/28/2023] Open
Abstract
For adolescents, high levels of aggression are often associated with suicide, physical injury, worsened academic performance, and crime. Therefore, there is a need for the early identification of and intervention for highly aggressive adolescents. The Buss-Warren Aggression Questionnaire (BWAQ) is one of the most widely used offensive measurement tools. It consists of 34 items, and the longer the scale, the more likely participants are to make an insufficient effort response (IER), which reduces the credibility of the results and increases the cost of implementation. This study aimed to develop a shorter BWAQ using machine learning (ML) techniques to reduce the frequency of IER and simultaneously decrease implementation costs. First, an initial version of the short-form questionnaire was created using stepwise regression and an ANOVA F-test. Then, a machine learning algorithm was used to create the optimal short-form questionnaire (BWAQ-ML). Finally, the reliability and validity of the optimal short-form questionnaire were tested using independent samples. The BWAQ-ML contains only four items, thirty items less than the BWAQ, and its AUC, accuracy, recall, precision, and F1 score are 0.85, 0.85, 0.89, 0.83, and 0.86, respectively. BWAQ-ML has a Cronbach's alpha of 0.84, a correlation with RPQ of 0.514, and a correlation with PTM of -0.042, suggesting good measurement performance. The BWAQ-ML can effectively measure individual aggression, and its smaller number of items improves the measurement efficiency for large samples and reduces the frequency of IER occurrence. It can be used as a convenient tool for early adolescent aggression identification and intervention.
Collapse
Affiliation(s)
| | | | - Junyi Li
- College of Psychology, Sichuan Normal University, Chengdu 610066, China; (X.J.); (Y.Y.)
| |
Collapse
|
7
|
Lin GH, Lee SC, Yu YT, Huang CY. Machine learning-based brief version of the Caregiver-Teacher Report Form for preschoolers. RESEARCH IN DEVELOPMENTAL DISABILITIES 2023; 134:104437. [PMID: 36706597 DOI: 10.1016/j.ridd.2023.104437] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2022] [Revised: 01/09/2023] [Accepted: 01/12/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND The Caregiver-Teacher Report Form of the Child Behavior Checklist for Ages 1½-5 (C-TRF) is a widely used checklist to identify emotional and behavioral problems in preschoolers. However, the 100-item C-TRF restricts its utility. AIMS This study aimed to develop a machine learning-based short-form of the C-TRF (C-TRF-ML). METHODS AND PROCEDURES Three steps were executed. First, we split the data into three datasets in a ratio of 3:1:1 for training, validation, and cross-validation, respectively. Second, we selected a shortened item set and trained a scoring algorithm using joint learning for classification and regression using the training dataset. Then, we evaluated the similarity of scores between the C-TRF-ML and the C-TRF by r-squared and weighted kappa values using the validation dataset. Third, we cross-validated the C-TRF-ML by calculating the r-squared and weighted kappa values using the cross-validation dataset. OUTCOMES AND RESULTS Data of 363 children were analyzed. Thirty-six items of the C-TRF were retained. The r-squared values of C-TRF-ML scores were 0.86-0.96 in the cross-validation dataset. Weighted kappa values of the syndrome/problem grading were 0.72-0.94 in the cross-validation dataset. CONCLUSIONS AND IMPLICATIONS The C-TRF-ML had about 60 % fewer items than the C-TRF but yielded comparable scores with the C-TRF.
Collapse
Affiliation(s)
- Gong-Hong Lin
- International Ph.D. Program in Gerontology and Long-Term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan
| | - Shih-Chieh Lee
- Department of Occupational Therapy, College of Medicine, National Cheng Kung University, Tainan City, Taiwan; School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei city, Taiwan; Institute of Long-Term Care, MacKay Medical College, New Taipei City, Taiwan
| | - Yen-Ting Yu
- Department of Occupational Therapy, College of Medicine, National Cheng Kung University, Tainan City, Taiwan; School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei city, Taiwan
| | - Chien-Yu Huang
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei city, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei city, Taiwan.
| |
Collapse
|
8
|
Lin GH, Wang I, Lee SC, Huang CY, Wang YC, Hsieh CL. Development of a 13-item Short Form for Fugl-Meyer Assessment of Upper Extremity Scale Using a Machine Learning Approach. Arch Phys Med Rehabil 2023:S0003-9993(23)00049-7. [PMID: 36736809 DOI: 10.1016/j.apmr.2023.01.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 11/28/2022] [Accepted: 01/10/2023] [Indexed: 02/04/2023]
Abstract
OBJECTIVE To develop and validate a short form of the Fugl-Meyer Assessment of Upper Extremity Scale (FMA-UE) using a machine learning approach (FMA-UE-ML). In addition, scores of items not included in the FMA-UE-ML were predicted. DESIGN Secondary data from a previous study, which assessed individuals post-stroke using the FMA-UE at 4 time points: 5-30 days post-stroke screen, 2-month post-stroke baseline assessment, 6-month post-stroke assessment, and 12-month post-stroke assessment. SETTING Rehabilitation units in hospitals. PARTICIPANTS A total of 408 individuals post-stroke (N=408). INTERVENTIONS Not applicable. MAIN OUTCOME MEASURES The 30-item FMA-UE. RESULTS We established 29 candidate versions of the FMA-UE-ML with different numbers of items, from 1 to 29, and examined their concurrent validity and responsiveness. We found that the responsiveness of the candidate versions obviously declined when the number of items was less than 13. Thus, the 13-item version was selected as the FMA-UE-ML. The concurrent validity was good (intra-class correlation coefficients ≥0.99). The standardized response means of the FMA-UE-ML and FMA-UE were 0.54-0.88 and 0.52-0.91, respectively. The Pearson's rs between the change scores of the FMA-UE-ML and those of the FMA-UE were 0.96-0.98. The predicted item scores had acceptable to good accuracy (Kappa=0.50-0.92). CONCLUSIONS The FMA-UE-ML seems a promising short form to improve administrative efficiency while retaining good concurrent validity and responsiveness. In addition, the FAM-UE-ML can provide all item scores of the FMA-UE for users.
Collapse
Affiliation(s)
- Gong-Hong Lin
- International Ph.D. Program in Gerontology and Long-Term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan
| | - Inga Wang
- Department of Rehabilitation Sciences & Technology, University of Wisconsin-Milwaukee, Milwaukee, WI
| | - Shih-Chieh Lee
- Department of Occupational Therapy, College of Medicine, National Cheng Kung University, Tainan, Taiwan; School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Institute of Long-Term Care, MacKay Medical College, New Taipei City, Taiwan
| | - Chien-Yu Huang
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan
| | - Yi-Ching Wang
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Ching-Lin Hsieh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan.
| |
Collapse
|
9
|
Wang I, Li PC, Lee SC, Lee YC, Wang CH, Hsieh CL. Development of a Berg Balance Scale Short-Form Using a Machine Learning Approach in Patients With Stroke. J Neurol Phys Ther 2023; 47:44-51. [PMID: 36047823 DOI: 10.1097/npt.0000000000000417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
BACKGROUND AND PURPOSE The Berg Balance Scale (BBS) is frequently used in routine clinical care and research settings and has good psychometric properties. This study was conducted to develop a short form of the BBS using a machine learning approach (BBS-ML). METHODS Data of 408 individuals poststroke were extracted from a published database. The initial (ie, 4-, 5-, 6-, 7-, and 8-item) versions were constructed by selecting top-ranked items based on the feature selection algorithm in the artificial neural network model. The final version of the BBS-ML was chosen by selecting the short form that used a smaller number of items to achieve a higher predictive power R2 , a lower 95% limit of agreement (LoA), and an adequate possible scoring point (PSP). An independent sample of 226 persons with stroke was used for external validation. RESULTS The R2 values for the initial 4-, 5-, 6-, 7-, and 8-item short forms were 0.93, 0.95, 0.97, 0.97, and 0.97, respectively. The 95% LoAs were 14.2, 12.2, 9.7, 9.6, and 8.9, respectively. The PSPs were 25, 35, 34, 35, and 36, respectively. The 6-item version was selected as the final BBS-ML. Preliminary external validation supported its performance in an independent sample of persons with stroke ( R2 = 0.99, LoA = 10.6, PSP = 37). DISCUSSION AND CONCLUSIONS The BBS-ML seems to be a promising short-form alternative to improve administrative efficiency. Future research is needed to examine the psychometric properties and clinical usage of the 6-item BBS-ML in various settings and samples.Video Abstract available for more insights from the authors (see the Video, Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A402 ).
Collapse
Affiliation(s)
- Inga Wang
- Department of Rehabilitation Sciences & Technology (I.W.), University of Wisconsin-Milwaukee, Milwaukee, Wisconsin; School of Occupational Therapy (P.-C.L., S.-C.L., C.-L.H.), College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Occupational Therapy (S.-C.L.), College of Medicine, National Cheng Kung University, Tainan City, Taiwan; Department of Occupational Therapy (Y.-C.L., C.-L.H.), College of Medical and Health Science, Asia University, Taichung, Taiwan; Institute of Long-Term Care (S.-C.L.), MacKay Medical College, New Taipei City, Taiwan; Department of Physical Therapy (C.-H.W.) and Physical Therapy Room (C.-H.W.), Chung Shan Medical University Hospital, Taichung, Taiwan; and Department of Physical Medicine and Rehabilitation (C.-L.H.), National Taiwan University Hospital, Taipei, Taiwan
| | | | | | | | | | | |
Collapse
|
10
|
Lee SC, Wang I, Lin GH, Li PC, Lee YC, Chou CY, Huang CY, Hsieh CL. Development of a Short-Form Stroke Impact Scale Using a Machine Learning Algorithm for Patients at the Subacute Stage. Am J Occup Ther 2022; 76:23964. [DOI: 10.5014/ajot.2022.049136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Abstract
Abstract
Importance: Several short forms of the Stroke Impact Scale Version 3.0 (SIS 3.0) have been proposed in order to decrease its administration time of about 20 min. However, none of the short-form scores are comparable to those of the original measure.
Objective: To develop a short-form SIS 3.0 using a machine learning algorithm (ML–SIS).
Design: We developed the ML–SIS in three stages. First, we calculated the frequencies of items having the highest contribution to predicting the original domain scores across 50 deep neural networks. Second, we iteratively selected the items showing the highest frequency until the coefficient of determination (R2) of each domain was ≥.90. Third, we examined the comparability and concurrent and convergent validity of the ML–SIS.
Setting: Hospitals.
Participants: We extracted complete data for 1,010 patients from an existing data set.
Results: Twenty-eight items were selected for the ML–SIS. High average R2s (.90–.96) and small average residuals (mean absolute errors and root-mean-square errors = 0.49–2.84) indicate good comparability. High correlations (rs = .95–.98) between the eight domain scores of the ML–SIS and the SIS 3.0 indicate sufficient concurrent validity. Similar interdomain correlations between the two measures indicate satisfactory convergent validity.
Conclusions and Relevance: The ML–SIS uses about half of the items in the SIS 3.0, has an estimated administration time of 10 min, and provides valid scores comparable to those of the original measure. Thus, the ML–SIS may be an efficient alternative to the SIS 3.0.
What This Article Adds: The ML–SIS, a short form of the SIS 3.0 developed using a machine learning algorithm, shows good potential to be an efficient and informative measure for clinical settings, providing scores that are valid and comparable to those of the original measure.
Collapse
Affiliation(s)
- Shih-Chieh Lee
- Shih-Chieh Lee, PhD, is Postdoctoral Researcher, Department of Occupational Therapy, College of Medicine, National Cheng Kung University, Tainan City, Taiwan; Adjunct Assistant Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; and Adjunct Assistant Professor, Institute of Long-Term Care, MacKay Medical College, New Taipei City, Taiwan. At the time this article was submitted, Lee was Postdoctoral Researcher, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Inga Wang
- Inga Wang, OTR/L, PhD, is Associate Professor, Department of Occupational Sciences and Technology, University of Wisconsin–Milwaukee
| | - Gong-Hong Lin
- Gong-Hong Lin, PhD, is Assistant Professor, International PhD Program in Gerontology and Long-Term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan
| | - Pei-Chi Li
- Pei-Chi Li, MS, is PhD Student, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan. At the time this article was submitted, Li was Master’s Student, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Ya-Chen Lee
- Ya-Chen Lee, PhD, is Associate Professor, Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan. At the time this article was submitted, Lee was Assistant Professor, Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan;
| | - Chia-Yeh Chou
- Chia-Yeh Chou, MA, is Associate Professor, Department of Occupational Therapy, College of Medicine, Fu-Jen Catholic University, New Taipei City, Taiwan
| | - Chien-Yu Huang
- Chien-Yu Huang, PhD, is Assistant Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Adjunct Occupational Therapist, Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan. At the time this article was submitted, Huang was Associate Professor, Department of Occupational Therapy, I-Shou University, Kaohsiung, Taiwan
| | - Ching-Lin Hsieh
- Ching-Lin Hsieh, PhD, is Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Adjunct Occupational Therapist, Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; Adjunct Professor, Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan;
| |
Collapse
|
11
|
Lin GH, Li CY, Sheu CF, Huang CY, Lee SC, Huang YH, Hsieh CL. Using Machine Learning to develop a short-form measure assessing 5 functions in patients with stroke. Arch Phys Med Rehabil 2021; 103:1574-1581. [PMID: 34979129 PMCID: PMC9378042 DOI: 10.1016/j.apmr.2021.12.006] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2020] [Revised: 10/26/2021] [Accepted: 12/10/2021] [Indexed: 11/26/2022]
Abstract
OBJECTIVE This study aimed to develop and validate a machine learning based short measure (the ML-5F) to assess 5 functions (activities of daily living (ADL), balance, upper extremity (UE) and lower extremity (LE) motor function, and mobility) in patients with stroke. DESIGN Secondary data from a previous study. A follow-up study assessed patients with stroke using the Barthel Index (BI), Postural Assessment Scale for Stroke (PASS), and Stroke Rehabilitation Assessment of Movement (STREAM) at hospital admission and discharge. SETTING A rehabilitation unit in a medical center. PARTICIPANTS A total of 307 patients. INTERVENTIONS Not applicable. MAIN OUTCOME MEASURES The BI, PASS, and STREAM. RESULTS A machine learning algorithm, Extreme Gradient Boosting, was used to select 15 items from the BI, PASS, and STREAM, and transformed the raw scores of the selected items into the scores of the ML-5F. The ML-5F demonstrated good concurrent validity (Pearson's r = 0.88-0.98) and responsiveness (standardized response mean = 0.28-1.01). CONCLUSIONS The ML-5F comprises only 15 items but demonstrates sufficient concurrent validity and responsiveness to assess ADL, balance, UE and LE functions, and mobility in patients with stroke. The ML-5F shows great potential as an efficient outcome measure in clinical settings.
Collapse
Affiliation(s)
- Gong-Hong Lin
- Master Program in Long-term Care, College of Nursing, Taipei Medical University, Taipei, Taiwan
| | - Chih-Ying Li
- Department of Occupational Therapy, School of Health Professions, University of Texas Medical Branch, Galveston, Texas
| | - Ching-Fan Sheu
- Institute of Education, National Cheng Kung University, Tainan, Taiwan
| | - Chien-Yu Huang
- Department of Occupational Therapy, College of Medicine, I-Shou University, Kaohsiung, Taiwan
| | - Shih-Chieh Lee
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Yu-Hui Huang
- School of Medicine, Chung Shan Medical University, Taichung, Taiwan; Department of Physical Medicine and Rehabilitation, Chung Shan Medical University Hospital, Taichung, Taiwan.
| | - Ching-Lin Hsieh
- School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan; Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan.
| |
Collapse
|