A Comparison of Three Neural Network Approaches for Estimating Joint Angles and Moments from Inertial Measurement Units

Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations

Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2 mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables ( ≥ 0.99 ) . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.

Estimation of Kinetics Using IMUs to Monitor and Aid in Clinical Decision-Making during ACL Rehabilitation: A Systematic Review

After an ACL injury, rehabilitation consists of multiple phases, and progress between these phases is guided by subjective visual assessments of activities such as running, hopping, jump landing, etc. Estimation of objective kinetic measures like knee joint moments and GRF during assessment can help physiotherapists gain insights on knee loading and tailor rehabilitation protocols. Conventional methods deployed to estimate kinetics require complex, expensive systems and are limited to laboratory settings. Alternatively, multiple algorithms have been proposed in the literature to estimate kinetics from kinematics measured using only IMUs. However, the knowledge about their accuracy and generalizability for patient populations is still limited. Therefore, this article aims to identify the available algorithms for the estimation of kinetic parameters using kinematics measured only from IMUs and to evaluate their applicability in ACL rehabilitation through a comprehensive systematic review. The papers identified through the search were categorized based on the modelling techniques and kinetic parameters of interest, and subsequently compared based on the accuracies achieved and applicability for ACL patients during rehabilitation. IMUs have exhibited potential in estimating kinetic parameters with good accuracy, particularly for sagittal movements in healthy cohorts. However, several shortcomings were identified and future directions for improvement have been proposed, including extension of proposed algorithms to accommodate multiplanar movements and validation of the proposed techniques in diverse patient populations and in particular the ACL population.

3D gait analysis in children using wearable sensors: feasibility of predicting joint kinematics and kinetics with personalized machine learning models and inertial measurement units

Introduction: Children's walking patterns evolve with age, exhibiting less repetitiveness at a young age and more variability than adults. Three-dimensional gait analysis (3DGA) is crucial for understanding and treating lower limb movement disorders in children, traditionally performed using Optical Motion Capture (OMC). Inertial Measurement Units (IMUs) offer a cost-effective alternative to OMC, although challenges like drift errors persist. Machine learning (ML) models can mitigate these issues in adults, prompting an investigation into their applicability to a heterogeneous pediatric population. This study aimed at 1) quantifying personalized and generalized ML models' performance for predicting gait time series in typically developed (TD) children using IMUs data, 2) Comparing random forest (RF) and convolutional neural networks (CNN) models' performance, 3) Finding the optimal number of IMUs required for accurate predictions. Methodology: Seventeen TD children, aged 6 to 15, participated in data collection involving OMC, force plates, and IMU sensors. Joint kinematics and kinetics (targets) were computed from OMC and force plates' data using OpenSim. Tsfresh, a Python package, extracted features from raw IMU data. Each target's ten most important features were input in the development of personalized and generalized RF and CNN models. This procedure was initially conducted with 7 IMUs placed on all lower limb segments and then performed using only two IMUs on the feet. Results: Findings suggested that the RF and CNN models demonstrated comparable performance. RF predicted joint kinematics with a 9.5% and 19.9% NRMSE for personalized and generalized models, respectively, and joint kinetics with an NRMSE of 10.7% for personalized and 15.2% for generalized models in TD children. Personalized models provided accurate estimations from IMU data in children, while generalized models lacked accuracy due to the limited dataset. Furthermore, reducing the number of IMUs from 7 to 2 did not affect the results, and the performance remained consistent. Discussion: This study proposed a promising personalized approach for gait time series prediction in children, involving an RF model and two IMUs on the feet.

Differences in running technique between runners with better and poorer running economy and lower and higher milage: An artificial neural network approach

BACKGROUND

Prior studies investigated selected discrete sagittal-plane outcomes (e.g., peak knee flexion) in relation to running economy, hereby discarding the potential relevance of running technique parameters during noninvestigated phases of the gait cycle and in other movement planes.

PURPOSE

Investigate which components of running technique distinguish groups of runners with better and poorer economy and higher and lower weekly running distance using an artificial neural network (ANN) approach with layer-wise relevance propagation.

METHODS

Forty-one participants (22 males and 19 females) ran at 2.78 m∙s-1 while three-dimensional kinematics and gas exchange data were collected. Two groups were created that differed in running economy or weekly training distance. The three-dimensional kinematic data were used as input to an ANN to predict group allocations. Layer-wise relevance propagation was used to determine the relevance of three-dimensional kinematics for group classification.

RESULTS

The ANN classified runners in the correct economy or distance group with accuracies of up to 62% and 71%, respectively. Knee, hip, and ankle flexion were most relevant to both classifications. Runners with poorer running economy showed higher knee flexion during swing, more hip flexion during early stance, and more ankle extension after toe-off. Runners with higher running distance showed less trunk rotation during swing.

CONCLUSION

The ANN accuracy was moderate when predicting whether runners had better, or poorer running economy, or had a higher or lower weekly training distance based on their running technique. The kinematic components that contributed the most to the classification may nevertheless inform future research and training.

Sagittal plane knee kinematics can be measured during activities of daily living following total knee arthroplasty with two IMU

Knee function is rarely measured objectively during functional tasks following total knee arthroplasty. Inertial measurement units (IMU) can measure knee kinematics and range of motion (ROM) during dynamic activities and offer an easy-to-use system for knee function assessment post total knee arthroplasty. However, IMU must be validated against gold standard three-dimensional optical motion capture systems (OMC) across a range of tasks if they are to see widespread uptake. We computed knee rotations and ROM from commercial IMU sensor measurements during walking, squatting, sit-to-stand, stair ascent, and stair descent in 21 patients one-year post total knee arthroplasty using two methods: direct computation using segment orientations (r_IMU), and an IMU-driven iCloud-based interactive lower limb model (m_IMU). This cross-sectional study compared computed knee angles and ROM to a gold-standard OMC and inverse kinematics method using Pearson's correlation coefficient (R) and root-mean-square-differences (RMSD). The r_IMU and m_IMU methods estimated sagittal plane knee angles with excellent correlation (>0.95) compared to OMC for walking, squatting, sit-to-stand, and stair-ascent, and very good correlation (>0.90) for stair descent. For squatting, sit-to-stand, and walking, the mean RMSD for r_IMU and m_IMU compared to OMC were <4 degrees, < 5 degrees, and <6 degrees, respectively but higher for stair ascent and descent (~12 degrees). Frontal and transverse plane knee kinematics estimated using r_IMU and m_IMU showed poor to moderate correlation compared to OMC. There were no differences in ROM measurements during squatting, sit-to-stand, and walking across the two methods. Thus, IMUs can measure sagittal plane knee angles and ROM with high accuracy for a variety of tasks and may be a useful in-clinic tool for objective assessment of knee function following total knee arthroplasty.

Machine learning-based prediction of hip joint moment in healthy subjects, patients and post-operative subjects

The application of machine learning in the field of motion capture research is growing rapidly. The purpose of the study is to implement a long-short term memory (LSTM) model able to predict sagittal plane hip joint moment (HJM) across three distinct cohorts (healthy controls, patients and post-operative patients) starting from 3D motion capture and force data. Statistical parametric mapping with paired samples t-test was performed to compare machine learning and inverse dynamics HJM predicted values, with the latter used as gold standard. The results demonstrated favorable model performance on each of the three cohorts, showcasing its ability to successfully generalize predictions across diverse cohorts.

Wearable sensing for understanding and influencing human movement in ecological contexts

Wearable sensors offer a unique opportunity to study movement in ecological contexts - that is, outside the laboratory where movement happens in ordinary life. This article discusses the purpose, means, and impact of using wearable sensors to assess movement context, kinematics, and kinetics during locomotion, and how this information can be used to better understand and influence movement. We outline the types of information wearable sensors can gather and highlight recent developments in sensor technology, data analysis, and applications. We close with a vision for important future research and key questions the field will need to address to bring the potential benefits of wearable sensing to fruition.

Estimation of Lower Limb Joint Angles and Joint Moments during Different Locomotive Activities Using the Inertial Measurement Units and a Hybrid Deep Learning Model

Using inertial measurement units (IMUs) to estimate lower limb joint kinematics and kinetics can provide valuable information for disease diagnosis and rehabilitation assessment. To estimate gait parameters using IMUs, model-based filtering approaches have been proposed, such as the Kalman filter and complementary filter. However, these methods require special calibration and alignment of IMUs. The development of deep learning algorithms has facilitated the application of IMUs in biomechanics as it does not require particular calibration and alignment procedures of IMUs in use. To estimate hip/knee/ankle joint angles and moments in the sagittal plane, a subject-independent temporal convolutional neural network-bidirectional long short-term memory network (TCN-BiLSTM) model was proposed using three IMUs. A public benchmark dataset containing the most representative locomotive activities in daily life was used to train and evaluate the TCN-BiLSTM model. The mean Pearson correlation coefficient of joint angles and moments estimated by the proposed model reached 0.92 and 0.87, respectively. This indicates that the TCN-BiLSTM model can effectively estimate joint angles and moments in multiple scenarios, demonstrating its potential for application in clinical and daily life scenarios.

Markerless Motion Tracking With Noisy Video and IMU Data

OBJECTIVE

Marker-based motion capture, considered the gold standard in human motion analysis, is expensive and requires trained personnel. Advances in inertial sensing and computer vision offer new opportunities to obtain research-grade assessments in clinics and natural environments. A challenge that discourages clinical adoption, however, is the need for careful sensor-to-body alignment, which slows the data collection process in clinics and is prone to errors when patients take the sensors home.

METHODS

We propose deep learning models to estimate human movement with noisy data from videos (VideoNet), inertial sensors (IMUNet), and a combination of the two (FusionNet), obviating the need for careful calibration. The video and inertial sensing data used to train the models were generated synthetically from a marker-based motion capture dataset of a broad range of activities and augmented to account for sensor-misplacement and camera-occlusion errors. The models were tested using real data that included walking, jogging, squatting, sit-to-stand, and other activities.

RESULTS

On calibrated data, IMUNet was as accurate as state-of-the-art models, while VideoNet and FusionNet reduced mean ± std root-mean-squared errors by 7.6 ± 5.4 ° and 5.9 ± 3.3 °, respectively. Importantly, all the newly proposed models were less sensitive to noise than existing approaches, reducing errors by up to 14.0 ± 5.3 ° for sensor-misplacement errors of up to 30.0 ± 13.7 ° and by up to 7.4 ± 5.5 ° for joint-center-estimation errors of up to 101.1 ± 11.2 mm, across joints.

CONCLUSION

These tools offer clinicians and patients the opportunity to estimate movement with research-grade accuracy, without the need for time-consuming calibration steps or the high costs associated with commercial products such as Theia3D or Xsens, helping democratize the diagnosis, prognosis, and treatment of neuromusculoskeletal conditions.

The History and Future of Neuromusculoskeletal Biomechanics

The Executive Council of the International Society of Biomechanics has initiated and overseen the commemorations of the Society's 50th Anniversary in 2023. This included multiple series of lectures at the ninth World Congress of Biomechanics in 2022 and XXIXth Congress of the International Society of Biomechanics in 2023, all linked to special issues of International Society of Biomechanics' affiliated journals. This special issue of the Journal of Applied Biomechanics is dedicated to the biomechanics of the neuromusculoskeletal system. The reader is encouraged to explore this special issue which comprises 6 papers exploring the current state-of the-art, and future directions and roles for neuromusculoskeletal biomechanics. This editorial presents a very brief history of the science of the neuromusculoskeletal system's 4 main components: the central nervous system, musculotendon units, the musculoskeletal system, and joints, and how they biomechanically integrate to enable an understanding of the generation and control of human movement. This also entails a quick exploration of contemporary neuromusculoskeletal biomechanics and its future with new fields of application.

A Wearable Biofeedback Device for Monitoring Tibial Load During Partial Weight-Bearing Walking

Patients with tibial fractures are usually advised to follow a partial weight-bearing gait rehabilitation program after surgery to promote bone healing and lower limb functional recovery. Currently, the biofeedback devices used for gait rehabilitation training in fracture patients use ground reaction force (GRF) as the indicator of tibial load. However, an increasing body of research has shown that monitoring GRF alone cannot objectively reflect the load on the lower limb bones during human movement. In this study, a novel biofeedback system was developed utilizing inertial measurement units and custom instrumented insoles. Based on the data collected from experiments, a hybrid approach combining a physics-based model and neural network architectures was used to predict tibial force. Compared to the traditional physics-based algorithm, the physical guided neural networks method showed better predictive performance. The study also found that regardless of the type of weight-bearing walking, the peak tibial force was significantly higher than the peak tibial GRF, and the time at which the peak tibial compression force occurs may not be consistent with the time at which the peak vertical GRF occurs. This further supports the idea that during gait rehabilitation training for patients with tibial fractures, monitoring and providing feedback on the actual tibial force rather than just the GRF is necessary. The developed device is a non-invasive and reliable portable device that can provide audio feedback, providing a viable solution for gait rehabilitation training outside laboratory and helping to optimize patients' rehabilitation treatment strategies.

BioMAT: An Open-Source Biomechanics Multi-Activity Transformer for Joint Kinematic Predictions Using Wearable Sensors

Through wearable sensors and deep learning techniques, biomechanical analysis can reach beyond the lab for clinical and sporting applications. Transformers, a class of recent deep learning models, have become widely used in state-of-the-art artificial intelligence research due to their superior performance in various natural language processing and computer vision tasks. The performance of transformer models has not yet been investigated in biomechanics applications. In this study, we introduce a Biomechanical Multi-activity Transformer-based model, BioMAT, for the estimation of joint kinematics from streaming signals of multiple inertia measurement units (IMUs) using a publicly available dataset. This dataset includes IMU signals and the corresponding sagittal plane kinematics of the hip, knee, and ankle joints during multiple activities of daily living. We evaluated the model's performance and generalizability and compared it against a convolutional neural network long short-term model, a bidirectional long short-term model, and multi-linear regression across different ambulation tasks including level ground walking (LW), ramp ascent (RA), ramp descent (RD), stair ascent (SA), and stair descent (SD). To investigate the effect of different activity datasets on prediction accuracy, we compared the performance of a universal model trained on all activities against task-specific models trained on individual tasks. When the models were tested on three unseen subjects' data, BioMAT outperformed the benchmark models with an average root mean square error (RMSE) of 5.5 ± 0.5°, and normalized RMSE of 6.8 ± 0.3° across all three joints and all activities. A unified BioMAT model demonstrated superior performance compared to individual task-specific models across four of five activities. The RMSE values from the universal model for LW, RA, RD, SA, and SD activities were 5.0 ± 1.5°, 6.2 ± 1.1°, 5.8 ± 1.1°, 5.3 ± 1.6°, and 5.2 ± 0.7° while these values for task-specific models were, 5.3 ± 2.1°, 6.7 ± 2.0°, 6.9 ± 2.2°, 4.9 ± 1.4°, and 5.6 ± 1.3°, respectively. Overall, BioMAT accurately estimated joint kinematics relative to previous machine learning algorithms across different activities directly from the sequence of IMUs signals instead of time-normalized gait cycle data.

Fusion of video and inertial sensing data via dynamic optimization of a biomechanical model

Inertial sensing and computer vision are promising alternatives to traditional optical motion tracking, but until now these data sources have been explored either in isolation or fused via unconstrained optimization, which may not take full advantage of their complementary strengths. By adding physiological plausibility and dynamical robustness to a proposed solution, biomechanical modeling may enable better fusion than unconstrained optimization. To test this hypothesis, we fused video and inertial sensing data via dynamic optimization with a nine degree-of-freedom model and investigated when this approach outperforms video-only, inertial-sensing-only, and unconstrained-fusion methods. We used both experimental and synthetic data that mimicked different ranges of video and inertial measurement unit (IMU) data noise. Fusion with a dynamically constrained model significantly improved estimation of lower-extremity kinematics over the video-only approach and estimation of joint centers over the IMU-only approach. It consistently outperformed single-modality approaches across different noise profiles. When the quality of video data was high and that of inertial data was low, dynamically constrained fusion improved estimation of joint kinematics and joint centers over unconstrained fusion, while unconstrained fusion was advantageous in the opposite scenario. These findings indicate that complementary modalities and techniques can improve motion tracking by clinically meaningful margins and that data quality and computational complexity must be considered when selecting the most appropriate method for a particular application.

Extended Application of Inertial Measurement Units in Biomechanics: From Activity Recognition to Force Estimation

Abnormal posture or movement is generally the indicator of musculoskeletal injuries or diseases. Mechanical forces dominate the injury and recovery processes of musculoskeletal tissue. Using kinematic data collected from wearable sensors (notably IMUs) as input, activity recognition and musculoskeletal force (typically represented by ground reaction force, joint force/torque, and muscle activity/force) estimation approaches based on machine learning models have demonstrated their superior accuracy. The purpose of the present study is to summarize recent achievements in the application of IMUs in biomechanics, with an emphasis on activity recognition and mechanical force estimation. The methodology adopted in such applications, including data pre-processing, noise suppression, classification models, force/torque estimation models, and the corresponding application effects, are reviewed. The extent of the applications of IMUs in daily activity assessment, posture assessment, disease diagnosis, rehabilitation, and exoskeleton control strategy development are illustrated and discussed. More importantly, the technical feasibility and application opportunities of musculoskeletal force prediction using IMU-based wearable devices are indicated and highlighted. With the development and application of novel adaptive networks and deep learning models, the accurate estimation of musculoskeletal forces can become a research field worthy of further attention.

Machine Learning for Optical Motion Capture-Driven Musculoskeletal Modelling from Inertial Motion Capture Data

Marker-based Optical Motion Capture (OMC) systems and associated musculoskeletal (MSK) modelling predictions offer non-invasively obtainable insights into muscle and joint loading at an in vivo level, aiding clinical decision-making. However, an OMC system is lab-based, expensive, and requires a line of sight. Inertial Motion Capture (IMC) techniques are widely-used alternatives, which are portable, user-friendly, and relatively low-cost, although with lesser accuracy. Irrespective of the choice of motion capture technique, one typically uses an MSK model to obtain the kinematic and kinetic outputs, which is a computationally expensive tool increasingly well approximated by machine learning (ML) methods. Here, an ML approach is presented that maps experimentally recorded IMC input data to the human upper-extremity MSK model outputs computed from ('gold standard') OMC input data. Essentially, this proof-of-concept study aims to predict higher-quality MSK outputs from the much easier-to-obtain IMC data. We use OMC and IMC data simultaneously collected for the same subjects to train different ML architectures that predict OMC-driven MSK outputs from IMC measurements. In particular, we employed various neural network (NN) architectures, such as Feed-Forward Neural Networks (FFNNs) and Recurrent Neural Networks (RNNs) (vanilla, Long Short-Term Memory, and Gated Recurrent Unit) and a comprehensive search for the best-fit model in the hyperparameters space in both subject-exposed (SE) as well as subject-naive (SN) settings. We observed a comparable performance for both FFNN and RNN models, which have a high degree of agreement (ravg,SE,FFNN=0.90±0.19, ravg,SE,RNN=0.89±0.17, ravg,SN,FFNN=0.84±0.23, and ravg,SN,RNN=0.78±0.23) with the desired OMC-driven MSK estimates for held-out test data. The findings demonstrate that mapping IMC inputs to OMC-driven MSK outputs using ML models could be instrumental in transitioning MSK modelling from 'lab to field'.

A comparison of machine learning models' accuracy in predicting lower-limb joints' kinematics, kinetics, and muscle forces from wearable sensors

A combination of wearable sensors' data and Machine Learning (ML) techniques has been used in many studies to predict specific joint angles and moments. The aim of this study was to compare the performance of four different non-linear regression ML models to estimate lower-limb joints' kinematics, kinetics, and muscle forces using Inertial Measurement Units (IMUs) and electromyographys' (EMGs) data. Seventeen healthy volunteers (9F, 28 ± 5 years) were asked to walk over-ground for a minimum of 16 trials. For each trial, marker trajectories and three force-plates data were recorded to calculate pelvis, hip, knee, and ankle kinematics and kinetics, and muscle forces (the targets), as well as 7 IMUs and 16 EMGs. The features from sensors' data were extracted using the Tsfresh python package and fed into 4 ML models; Convolutional Neural Networks (CNN), Random Forest (RF), Support Vector Machine, and Multivariate Adaptive Regression Spline for targets' prediction. The RF and CNN models outperformed the other ML models by providing lower prediction errors in all intended targets with a lower computational cost. This study suggested that a combination of wearable sensors' data with an RF or a CNN model is a promising tool to overcome the limitations of traditional optical motion capture for 3D gait analysis.

Machine-Learning-Based Methodology for Estimation of Shoulder Load in Wheelchair-Related Activities Using Wearables

There is a high prevalence of shoulder problems in manual wheelchair users (MWUs) with a spinal cord injury. How shoulder load relates to shoulder problems remains unclear. This study aimed to develop a machine-learning-based methodology to estimate the shoulder load in wheelchair-related activities of daily living using wearable sensors. Ten able-bodied participants equipped with five inertial measurement units (IMU) on their thorax, right arm, and wheelchair performed activities exemplary of daily life of MWUs. Electromyography (EMG) was recorded from the long head of the biceps and medial part of the deltoid. A neural network was trained to predict the shoulder load based on IMU and EMG data. Different cross-validation strategies, sensor setups, and model architectures were examined. The predicted shoulder load was compared to the shoulder load determined with musculoskeletal modeling. A subject-specific biLSTM model trained on a sparse sensor setup yielded the most promising results (mean correlation coefficient = 0.74 ± 0.14, relative root-mean-squared error = 8.93% ± 2.49%). The shoulder-load profiles had a mean similarity of 0.84 ± 0.10 over all activities. This study demonstrates the feasibility of using wearable sensors and neural networks to estimate the shoulder load in wheelchair-related activities of daily living.

Predicting Wrist Posture during Occupational Tasks Using Inertial Sensors and Convolutional Neural Networks

Current methods for ergonomic assessment often use video-analysis to estimate wrist postures during occupational tasks. Wearable sensing and machine learning have the potential to automate this tedious task, and in doing so greatly extend the amount of data available to clinicians and researchers. A method of predicting wrist posture from inertial measurement units placed on the wrist and hand via a deep convolutional neural network has been developed. This study has quantified the accuracy and reliability of the postures predicted by this system relative to the gold standard of optoelectronic motion capture. Ten participants performed 3 different simulated occupational tasks on 2 occasions while wearing inertial measurement units on the hand and wrist. Data from the occupational task recordings were used to train a convolutional neural network classifier to estimate wrist posture in flexion/extension, and radial/ulnar deviation. The model was trained and tested in a leave-one-out cross validation format. Agreement between the proposed system and optoelectronic motion capture was 65% with κ = 0.41 in flexion/extension and 60% with κ = 0.48 in radial/ulnar deviation. The proposed system can predict wrist posture in flexion/extension and radial/ulnar deviation with accuracy and reliability congruent with published values for human estimators. This system can estimate wrist posture during occupational tasks in a small fraction of the time it takes a human to perform the same task. This offers opportunity to expand the capabilities of practitioners by eliminating the tedium of manual postural assessment.

Comparison of machine learning and deep learning-based methods for locomotion mode recognition using a single inertial measurement unit

Locomotion mode recognition provides the prosthesis control with the information on when to switch between different walking modes, whereas the gait phase detection indicates where we are in the gait cycle. But powered prostheses often implement a different control strategy for each locomotion mode to improve the functionality of the prosthesis. Existing studies employed several classical machine learning methods for locomotion mode recognition. However, these methods were less effective for data with complex decision boundaries and resulted in misclassifications of motion recognition. Deep learning-based methods potentially resolve these limitations as it is a special type of machine learning method with more sophistication. Therefore, this study evaluated three deep learning-based models for locomotion mode recognition, namely recurrent neural network (RNN), long short-term memory (LSTM) neural network, and convolutional neural network (CNN), and compared the recognition performance of deep learning models to the machine learning model with random forest classifier (RFC). The models are trained from data of one inertial measurement unit (IMU) placed on the lower shanks of four able-bodied subjects to perform four walking modes, including level ground walking (LW), standing (ST), and stair ascent/stair descent (SA/SD). The results indicated that CNN and LSTM models outperformed other models, and these models were promising for applying locomotion mode recognition in real-time for robotic prostheses.

Predicting knee adduction moment response to gait retraining with minimal clinical data

Knee osteoarthritis is a progressive disease mediated by high joint loads. Foot progression angle modifications that reduce the knee adduction moment (KAM), a surrogate of knee loading, have demonstrated efficacy in alleviating pain and improving function. Although changes to the foot progression angle are overall beneficial, KAM reductions are not consistent across patients. Moreover, customized interventions are time-consuming and require instrumentation not commonly available in the clinic. We present a regression model that uses minimal clinical data—a set of six features easily obtained in the clinic—to predict the extent of first peak KAM reduction after toe-in gait retraining. For such a model to generalize, the training data must be large and variable. Given the lack of large public datasets that contain different gaits for the same patient, we generated this dataset synthetically. Insights learned from a ground-truth dataset with both baseline and toe-in gait trials (N = 12) enabled the creation of a large (N = 138) synthetic dataset for training the predictive model. On a test set of data collected by a separate research group (N = 15), the first peak KAM reduction was predicted with a mean absolute error of 0.134% body weight * height (%BW*HT). This error is smaller than the standard deviation of the first peak KAM during baseline walking averaged across test subjects (0.306%BW*HT). This work demonstrates the feasibility of training predictive models with synthetic data and provides clinicians with a new tool to predict the outcome of patient-specific gait retraining without requiring gait lab instrumentation.

Gait retraining is a conservative intervention for knee osteoarthritis shown to reduce pain and improve function. Although customizing a treatment plan for each patient results in a better therapeutic response, customization cannot yet be performed outside of the gait laboratory, preventing research advances from becoming part of clinical practice. Our work aimed to build a model that accurately predicts whether a patient with knee osteoarthritis will benefit from non-invasive gait retraining using measures that can be easily collected in the clinic. To overcome the lack of large datasets required to train predictive models, we generated data synthetically (N = 138) based on limited ground-truth examples, and we provide experimental evidence for the model’s ability to generalize to real data (N = 15). Our results contribute toward a future in which clinicians can use data collected in the clinic to easily identify patients who would respond to therapeutic gait retraining.

Inertial Motion Capture-Based Wearable Systems for Estimation of Joint Kinetics: A Systematic Review

In biomechanics, joint kinetics has an important role in evaluating the mechanical load of the joint and understanding its motor function. Although an optical motion capture (OMC) system has mainly been used to evaluate joint kinetics in combination with force plates, inertial motion capture (IMC) systems have recently been emerging in joint kinetic analysis due to their wearability and ubiquitous measurement capability. In this regard, numerous studies have been conducted to estimate joint kinetics using IMC-based wearable systems. However, these have not been comprehensively addressed yet. Thus, the aim of this review is to explore the methodology of the current studies on estimating joint kinetic variables by means of an IMC system. From a systematic search of the literature, 48 studies were selected. This paper summarizes the content of the selected literature in terms of the (i) study characteristics, (ii) methodologies, and (iii) study results. The estimation methods of the selected studies are categorized into two types: the inverse dynamics-based method and the machine learning-based method. While these two methods presented different characteristics in estimating the kinetic variables, it was demonstrated in the literature that both methods could be applied with good performance for the kinetic analysis of joints in different daily activities.

Predicting Knee Joint Kinematics from Wearable Sensor Data in People with Knee Osteoarthritis and Clinical Considerations for Future Machine Learning Models

Deep learning models developed to predict knee joint kinematics are usually trained on inertial measurement unit (IMU) data from healthy people and only for the activity of walking. Yet, people with knee osteoarthritis have difficulties with other activities and there are a lack of studies using IMU training data from this population. Our objective was to conduct a proof-of-concept study to determine the feasibility of using IMU training data from people with knee osteoarthritis performing multiple clinically important activities to predict knee joint sagittal plane kinematics using a deep learning approach. We trained a bidirectional long short-term memory model on IMU data from 17 participants with knee osteoarthritis to estimate knee joint flexion kinematics for phases of walking, transitioning to and from a chair, and negotiating stairs. We tested two models, a double-leg model (four IMUs) and a single-leg model (two IMUs). The single-leg model demonstrated less prediction error compared to the double-leg model. Across the different activity phases, RMSE (SD) ranged from 7.04° (2.6) to 11.78° (6.04), MAE (SD) from 5.99° (2.34) to 10.37° (5.44), and Pearson’s R from 0.85 to 0.99 using leave-one-subject-out cross-validation. This study demonstrates the feasibility of using IMU training data from people who have knee osteoarthritis for the prediction of kinematics for multiple clinically relevant activities.

Machine Learning Model to Estimate Net Joint Moments during Lifting Task Using Wearable Sensors: A Preliminary Study for Design of Exoskeleton Control System

Accurately measuring the lower extremities and L5/S1 moments is important since L5/S1 moments are the principal parameters that measure the risk of musculoskeletal diseases during lifting. In this study, protocol that predicts lower extremities and L5/S1 moments with an insole sensor was proposed to replace the prior methods that have spatial constraints. The protocol is hierarchically composed of a classification model and a regression model to predict joint moments. Additionally, a single LSTM model was developed to compare with proposed protocol. To optimize hyperparameters of the machine learning model and input feature, Bayesian optimization method was adopted. As a result, the proposed protocol showed a relative root mean square error (rRMSE) of 8.06~13.88% while the single LSTM showed 9.30~18.66% rRMSE. This protocol in this research is expected to be a starting point for developing a system for estimating the lower extremity and L5/S1 moment during lifting that can replace the complex prior method and adopted to workplace environments. This novel study has the potential to precisely design a feedback iterative control system of an exoskeleton for the appropriate generation of an actuator torque.

A Deep Learning Approach for Foot Trajectory Estimation in Gait Analysis Using Inertial Sensors

Gait performance is an important marker of motor and cognitive decline in older adults. An instrumented gait analysis resorting to inertial sensors allows the complete evaluation of spatiotemporal gait parameters, offering an alternative to laboratory-based assessments. To estimate gait parameters, foot trajectories are typically obtained by integrating acceleration two times. However, to deal with cumulative integration errors, additional error handling strategies are required. In this study, we propose an alternative approach based on a deep recurrent neural network to estimate heel and toe trajectories. We propose a coordinate frame transformation for stride trajectories that eliminates the dependency from previous strides and external inputs. Predicted trajectories are used to estimate an extensive set of spatiotemporal gait parameters. We evaluate the results in a dataset comprising foot-worn inertial sensor data acquired from a group of young adults, using an optical motion capture system as a reference. Heel and toe trajectories are predicted with low errors, in line with reference trajectories. A good agreement is also achieved between the reference and estimated gait parameters, in particular when turning strides are excluded from the analysis. The performance of the method is shown to be robust to imperfect sensor-foot alignment conditions.

A Study on the Application of LSTM to Judge Bike Accidents for Inflating Wearable Airbags

The traffic accident occurrence rate is increasing relative to the increase in the number of people using personal mobility device (PM). This paper proposes an airbag system with a more efficient algorithm to decide the deployment of a wearable bike airbag in case of an accident. The existing wearable airbags are operated by judging the accident situations using the thresholds of sensors. However, in this case, the judgment accuracy can drop against various motions. This study used the long short-term memory (LSTM) model using the sensor values of the inertial measurement unit (IMU) as input values to judge accident occurrences, which obtains data in real time from the three acceleration-axis and three angular velocity-axis sensors on the driver motion states and judges whether or not an accident has occurred using the obtained data. The existing neural network (NN) or convolutional neural network (CNN) model judges only the input data. This study confirmed that this model has a higher judgment accuracy than the existing NN or CNN by giving strong points even in "past information" through LSTM by regarding the driver motion as time-series data.

The Use of Synthetic IMU Signals in the Training of Deep Learning Models Significantly Improves the Accuracy of Joint Kinematic Predictions

Gait analysis based on inertial sensors has become an effective method of quantifying movement mechanics, such as joint kinematics and kinetics. Machine learning techniques are used to reliably predict joint mechanics directly from streams of IMU signals for various activities. These data-driven models require comprehensive and representative training datasets to be generalizable across the movement variability seen in the population at large. Bottlenecks in model development frequently occur due to the lack of sufficient training data and the significant time and resources necessary to acquire these datasets. Reliable methods to generate synthetic biomechanical training data could streamline model development and potentially improve model performance. In this study, we developed a methodology to generate synthetic kinematics and the associated predicted IMU signals using open source musculoskeletal modeling software. These synthetic data were used to train neural networks to predict three degree-of-freedom joint rotations at the hip and knee during gait either in lieu of or along with previously measured experimental gait data. The accuracy of the models’ kinematic predictions was assessed using experimentally measured IMU signals and gait kinematics. Models trained using the synthetic data out-performed models using only the experimental data in five of the six rotational degrees of freedom at the hip and knee. On average, root mean square errors in joint angle predictions were improved by 38% at the hip (synthetic data RMSE: 2.3°, measured data RMSE: 4.5°) and 11% at the knee (synthetic data RMSE: 2.9°, measured data RMSE: 3.3°), when models trained solely on synthetic data were compared to measured data. When models were trained on both measured and synthetic data, root mean square errors were reduced by 54% at the hip (measured + synthetic data RMSE: 1.9°) and 45% at the knee (measured + synthetic data RMSE: 1.7°), compared to measured data alone. These findings enable future model development for different activities of clinical significance without the burden of generating large quantities of gait lab data for model training, streamlining model development, and ultimately improving model performance.