A Hybrid Method Integrating A Musculoskeletal Model with Long Short-Term Memory (LSTM) for Human Motion Prediction

Gait Intention Prediction Using a Lower-Limb Musculoskeletal Model and Long Short-Term Memory Neural Networks

The prediction of gait motion intention is essential for achieving intuitive control of assistive devices and diagnosing gait disorders. To reduce the cost associated with using multimodal signals and signal processing, we proposed a novel method that integrates machine learning with musculoskeletal modelling techniques for the prediction of time-series joint angles, using only kinematic signals. Additionally, we hypothesised that a stacked long short-term memory (LSTM) neural network architecture can perform the task without relying on any ahead-of-motion features typically provided by electromyography signals. Optical cameras and inertial measurement unit (IMU) sensors were used to track level gait kinematics. Joint angles were modelled using the musculoskeletal model. The optimal LSTM architecture in fulfilling the prediction task was determined. Joint angle predictions were performed for joints on the sagittal plane, benefiting from joint angle modelling using signals from optical cameras and IMU sensors. Our proposed method predicted the upcoming joint angles in the prediction time of 10 ms, with an averaged root mean square error of 5.3° and a coefficient of determination of 0.81. Moreover, in support of our hypothesis, the recurrent stacked LSTM network demonstrated its ability to predict intended motion accurately and efficiently in gait, outperforming two other neural network architectures: a feedforward MLP and a hybrid LSTM-MLP. The method paves the way for the development of a cost-effective, single-modal control system for assistive devices in gait rehabilitation.

A Deep Learning Model with a Self-Attention Mechanism for Leg Joint Angle Estimation across Varied Locomotion Modes

Conventional trajectory planning for lower limb assistive devices usually relies on a finite-state strategy, which pre-defines fixed trajectory types for specific gait events and activities. The advancement of deep learning enables walking assistive devices to better adapt to varied terrains for diverse users by learning movement patterns from gait data. Using a self-attention mechanism, a temporal deep learning model is developed in this study to continuously generate lower limb joint angle trajectories for an ankle and knee across various activities. Additional analyses, including using Fast Fourier Transform and paired t-tests, are conducted to demonstrate the benefits of the proposed attention model architecture over the existing methods. Transfer learning has also been performed to prove the importance of data diversity. Under a 10-fold leave-one-out testing scheme, the observed attention model errors are 11.50% (±2.37%) and 9.31% (±1.56%) NRMSE for ankle and knee angle estimation, respectively, which are small in comparison to other studies. Statistical analysis using the paired t-test reveals that the proposed attention model appears superior to the baseline model in terms of reduced prediction error. The attention model also produces smoother outputs, which is crucial for safety and comfort. Transfer learning has been shown to effectively reduce model errors and noise, showing the importance of including diverse datasets. The suggested joint angle trajectory generator has the potential to seamlessly switch between different locomotion tasks, thereby mitigating the problem of detecting activity transitions encountered by the traditional finite-state strategy. This data-driven trajectory generation method can also reduce the burden on personalization, as traditional devices rely on prosthetists to experimentally tune many parameters for individuals with diverse gait patterns.