1
|
Wechsler I, Wolf A, Shanbhag J, Leyendecker S, Eskofier BM, Koelewijn AD, Wartzack S, Miehling J. Bridging the sim2real gap. Investigating deviations between experimental motion measurements and musculoskeletal simulation results-a systematic review. Front Bioeng Biotechnol 2024; 12:1386874. [PMID: 38919383 PMCID: PMC11196827 DOI: 10.3389/fbioe.2024.1386874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 05/21/2024] [Indexed: 06/27/2024] Open
Abstract
Musculoskeletal simulations can be used to estimate biomechanical variables like muscle forces and joint torques from non-invasive experimental data using inverse and forward methods. Inverse kinematics followed by inverse dynamics (ID) uses body motion and external force measurements to compute joint movements and the corresponding joint loads, respectively. ID leads to residual forces and torques (residuals) that are not physically realistic, because of measurement noise and modeling assumptions. Forward dynamic simulations (FD) are found by tracking experimental data. They do not generate residuals but will move away from experimental data to achieve this. Therefore, there is a gap between reality (the experimental measurements) and simulations in both approaches, the sim2real gap. To answer (patho-) physiological research questions, simulation results have to be accurate and reliable; the sim2real gap needs to be handled. Therefore, we reviewed methods to handle the sim2real gap in such musculoskeletal simulations. The review identifies, classifies and analyses existing methods that bridge the sim2real gap, including their strengths and limitations. Using a systematic approach, we conducted an electronic search in the databases Scopus, PubMed and Web of Science. We selected and included 85 relevant papers that were sorted into eight different solution clusters based on three aspects: how the sim2real gap is handled, the mathematical method used, and the parameters/variables of the simulations which were adjusted. Each cluster has a distinctive way of handling the sim2real gap with accompanying strengths and limitations. Ultimately, the method choice largely depends on various factors: available model, input parameters/variables, investigated movement and of course the underlying research aim. Researchers should be aware that the sim2real gap remains for both ID and FD approaches. However, we conclude that multimodal approaches tracking kinematic and dynamic measurements may be one possible solution to handle the sim2real gap as methods tracking multimodal measurements (some combination of sensor position/orientation or EMG measurements), consistently lead to better tracking performances. Initial analyses show that motion analysis performance can be enhanced by using multimodal measurements as different sensor technologies can compensate each other's weaknesses.
Collapse
Affiliation(s)
- Iris Wechsler
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Alexander Wolf
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Julian Shanbhag
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Chair of Autonomous Systems and Mechatronics, Department of Electrical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sandro Wartzack
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jörg Miehling
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
2
|
Zhang Z, Dai Y, Xu Z, Grimaldi N, Wang J, Zhao M, Pang R, Sun Y, Gao S, Boyi H. Insole Systems for Disease Diagnosis and Rehabilitation: A Review. BIOSENSORS 2023; 13:833. [PMID: 37622919 PMCID: PMC10452488 DOI: 10.3390/bios13080833] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/11/2023] [Revised: 08/11/2023] [Accepted: 08/15/2023] [Indexed: 08/26/2023]
Abstract
Some chronic diseases, including Parkinson's disease (PD), diabetic foot, flat foot, stroke, elderly falling, and knee osteoarthritis (KOA), are related to orthopedic organs, nerves, and muscles. The interaction of these three parts will generate a comprehensive result: gait. Furthermore, the lesions in these regions can produce abnormal gait features. Therefore, monitoring the gait features can assist medical professionals in the diagnosis and analysis of these diseases. Nowadays, various insole systems based on different sensing techniques have been developed to monitor gait and aid in medical research. Hence, a detailed review of insole systems and their applications in disease management can greatly benefit researchers working in the field of medical engineering. This essay is composed of the following sections: the essay firstly provides an overview of the sensing mechanisms and parameters of typical insole systems based on different sensing techniques. Then this essay respectively discusses the three stages of gait parameters pre-processing, respectively: pressure reconstruction, feature extraction, and data normalization. Then, the relationship between gait features and pathogenic mechanisms is discussed, along with the introduction of insole systems that aid in medical research; Finally, the current challenges and future trends in the development of insole systems are discussed.
Collapse
Affiliation(s)
- Zhiyuan Zhang
- School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; (Z.Z.); (Y.D.); (Z.X.)
| | - Yanning Dai
- School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; (Z.Z.); (Y.D.); (Z.X.)
| | - Zhenyu Xu
- School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; (Z.Z.); (Y.D.); (Z.X.)
| | - Nicolas Grimaldi
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA;
| | - Jiamu Wang
- School of Transportation Science and Engineering, Beihang University, Beijing 100191, China;
| | - Mufan Zhao
- School of Artificial Intelligence, Beihang University, Beijing 100191, China;
| | - Ruilin Pang
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China;
| | - Yueming Sun
- School of Electronics and Information Engineering, Beihang University, Beijing 100191, China;
| | - Shuo Gao
- School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; (Z.Z.); (Y.D.); (Z.X.)
| | - Hu Boyi
- School of Industrial and Systems Engineering, University of Florida, Gaineville, FL 32611, USA
| |
Collapse
|
3
|
Nitschke M, Marzilger R, Leyendecker S, Eskofier BM, Koelewijn AD. Change the direction: 3D optimal control simulation by directly tracking marker and ground reaction force data. PeerJ 2023; 11:e14852. [PMID: 36778146 PMCID: PMC9912948 DOI: 10.7717/peerj.14852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 01/13/2023] [Indexed: 02/10/2023] Open
Abstract
Optimal control simulations of musculoskeletal models can be used to reconstruct motions measured with optical motion capture to estimate joint and muscle kinematics and kinetics. These simulations are mutually and dynamically consistent, in contrast to traditional inverse methods. Commonly, optimal control simulations are generated by tracking generalized coordinates in combination with ground reaction forces. The generalized coordinates are estimated from marker positions using, for example, inverse kinematics. Hence, inaccuracies in the estimated coordinates are tracked in the simulation. We developed an approach to reconstruct arbitrary motions, such as change of direction motions, using optimal control simulations of 3D full-body musculoskeletal models by directly tracking marker and ground reaction force data. For evaluation, we recorded three trials each of straight running, curved running, and a v-cut for 10 participants. We reconstructed the recordings with marker tracking simulations, coordinate tracking simulations, and inverse kinematics and dynamics. First, we analyzed the convergence of the simulations and found that the wall time increased three to four times when using marker tracking compared to coordinate tracking. Then, we compared the marker trajectories, ground reaction forces, pelvis translations, joint angles, and joint moments between the three reconstruction methods. Root mean squared deviations between measured and estimated marker positions were smallest for inverse kinematics (e.g., 7.6 ± 5.1 mm for v-cut). However, measurement noise and soft tissue artifacts are likely also tracked in inverse kinematics, meaning that this approach does not reflect a gold standard. Marker tracking simulations resulted in slightly higher root mean squared marker deviations (e.g., 9.5 ± 6.2 mm for v-cut) than inverse kinematics. In contrast, coordinate tracking resulted in deviations that were nearly twice as high (e.g., 16.8 ± 10.5 mm for v-cut). Joint angles from coordinate tracking followed the estimated joint angles from inverse kinematics more closely than marker tracking (e.g., root mean squared deviation of 1.4 ± 1.8 deg vs. 3.5 ± 4.0 deg for v-cut). However, we did not have a gold standard measurement of the joint angles, so it is unknown if this larger deviation means the solution is less accurate. In conclusion, we showed that optimal control simulations of change of direction running motions can be created by tracking marker and ground reaction force data. Marker tracking considerably improved marker accuracy compared to coordinate tracking. Therefore, we recommend reconstructing movements by directly tracking marker data in the optimal control simulation when precise marker tracking is required.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Robert Marzilger
- Division Positioning and Networks, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, Nuremberg, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
4
|
Comparison between Piezoelectric and Piezoresistive Wearable Gait Monitoring Techniques. MATERIALS 2022; 15:ma15144837. [PMID: 35888304 PMCID: PMC9321623 DOI: 10.3390/ma15144837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/28/2022] [Revised: 07/05/2022] [Accepted: 07/06/2022] [Indexed: 12/04/2022]
Abstract
Insole plantar stress detection (PSD) techniques play an important role in gait monitoring. Among the various insole PSD methods, piezoelectric- and piezoresistive-based architectures are broadly used in medical scenes. Each year, a growing number of new research outcomes are reported. Hence, a deep understanding of these two kinds of insole PSD sensors and state-of-the-art work would strongly benefit the researchers in this highly interdisciplinary field. In this context, this review article is composed of the following aspects. First, the mechanisms of the two techniques and corresponding comparisons are explained and discussed. Second, advanced materials which could enhance the performance of current piezoelectric and piezoresistive insole prototypes are introduced. Third, suggestions for designing insole PSD prototypes/products for different diseases are offered. Last, the current challenge and potential future trends are provided.
Collapse
|
5
|
Nasr A, Inkol KA, Bell S, McPhee J. InverseMuscleNET: Alternative Machine Learning Solution to Static Optimization and Inverse Muscle Modeling. Front Comput Neurosci 2022; 15:759489. [PMID: 35002663 PMCID: PMC8735851 DOI: 10.3389/fncom.2021.759489] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 11/29/2021] [Indexed: 11/13/2022] Open
Abstract
InverseMuscleNET, a machine learning model, is proposed as an alternative to static optimization for resolving the redundancy issue in inverse muscle models. A recurrent neural network (RNN) was optimally configured, trained, and tested to estimate the pattern of muscle activation signals. Five biomechanical variables (joint angle, joint velocity, joint acceleration, joint torque, and activation torque) were used as inputs to the RNN. A set of surface electromyography (EMG) signals, experimentally measured around the shoulder joint for flexion/extension, were used to train and validate the RNN model. The obtained machine learning model yields a normalized regression in the range of 88-91% between experimental data and estimated muscle activation. A sequential backward selection algorithm was used as a sensitivity analysis to discover the less dominant inputs. The order of most essential signals to least dominant ones was as follows: joint angle, activation torque, joint torque, joint velocity, and joint acceleration. The RNN model required 0.06 s of the previous biomechanical input signals and 0.01 s of the predicted feedback EMG signals, demonstrating the dynamic temporal relationships of the muscle activation profiles. The proposed approach permits a fast and direct estimation ability instead of iterative solutions for the inverse muscle model. It raises the possibility of integrating such a model in a real-time device for functional rehabilitation and sports evaluation devices with real-time estimation and tracking. This method provides clinicians with a means of estimating EMG activity without an invasive electrode setup.
Collapse
Affiliation(s)
- Ali Nasr
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - Keaton A Inkol
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - Sydney Bell
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - John McPhee
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| |
Collapse
|
6
|
Bailly F, Ceglia A, Michaud B, Rouleau DM, Begon M. Real-Time and Dynamically Consistent Estimation of Muscle Forces Using a Moving Horizon EMG-Marker Tracking Algorithm-Application to Upper Limb Biomechanics. Front Bioeng Biotechnol 2021; 9:642742. [PMID: 33681174 PMCID: PMC7928053 DOI: 10.3389/fbioe.2021.642742] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Accepted: 02/01/2021] [Indexed: 01/17/2023] Open
Abstract
Real-time biofeedback of muscle forces should help clinicians adapt their movement recommendations. Because these forces cannot directly be measured, researchers have developed numerical models and methods informed by electromyography (EMG) and body kinematics to estimate them. Among these methods, static optimization is the most computationally efficient and widely used. However, it suffers from limitation, namely: unrealistic joint torques computation, non-physiological muscle forces estimates and inconsistent for motions inducing co-contraction. Forward approaches, relying on numerical optimal control, address some of these issues, providing dynamically consistent estimates of muscle forces. However, they result in a high computational cost increase, apparently disqualifying them for real-time applications. However, this computational cost can be reduced by combining the implementation of a moving horizon estimation (MHE) and advanced optimization tools. Our objective was to assess the feasibility and accuracy of muscle forces estimation in real-time, using a MHE. To this end, a 4-DoFs arm actuated by 19 Hill-type muscle lines of action was modeled for simulating a set of reference motions, with corresponding EMG signals and markers positions. Excitation- and activation-driven models were tested to assess the effects of model complexity. Four levels of co-contraction, EMG noise and marker noise were simulated, to run the estimator under 64 different conditions, 30 times each. The MHE problem was implemented with three cost functions: EMG-markers tracking (high and low weight on markers) and marker-tracking with least-squared muscle excitations. For the excitation-driven model, a 7-frame MHE was selected as it allowed the estimator to run at 24 Hz (faster than biofeedback standard) while ensuring the lowest RMSE on estimates in noiseless conditions. This corresponds to a 3,500-fold speed improvement in comparison to state-of-the-art equivalent approaches. When adding experimental-like noise to the reference data, estimation error on muscle forces ranged from 1 to 30 N when tracking EMG signals and from 8 to 50 N (highly impacted by the co-contraction level) when muscle excitations were minimized. Statistical analysis was conducted to report significant effects of the problem conditions on the estimates. To conclude, the presented MHE implementation proved to be promising for real-time muscle forces estimation in experimental-like noise conditions, such as in biofeedback applications.
Collapse
Affiliation(s)
- François Bailly
- Laboratoire de Simulation et de Modélisation du Mouvement, Faculté de Médecine, Université de Montréal, Laval, QC, Canada
| | - Amedeo Ceglia
- Laboratoire de Simulation et de Modélisation du Mouvement, Faculté de Médecine, Université de Montréal, Laval, QC, Canada
| | - Benjamin Michaud
- Laboratoire de Simulation et de Modélisation du Mouvement, Faculté de Médecine, Université de Montréal, Laval, QC, Canada
| | - Dominique M Rouleau
- Department of Surgery, Université de Montréal, Montreal, QC, Canada.,Department of Orthopedic Surgery, CIUSSS Nord-de-L'île-de-Montréal, Hôpital du Sacré-Cœur de Montréal (HSCM), Montreal, QC, Canada
| | - Mickael Begon
- Laboratoire de Simulation et de Modélisation du Mouvement, Faculté de Médecine, Université de Montréal, Laval, QC, Canada
| |
Collapse
|
7
|
Sun D, Fekete G, Baker JS, Mei Q, István B, Zhang Y, Gu Y. A Pilot Study of Musculoskeletal Abnormalities in Patients in Recovery from a Unilateral Rupture-Repaired Achilles Tendon. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17134642. [PMID: 32605170 PMCID: PMC7369810 DOI: 10.3390/ijerph17134642] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 06/23/2020] [Accepted: 06/24/2020] [Indexed: 11/16/2022]
Abstract
The purpose of this study was to compare the inter-limb joint kinematics, joint moments, muscle forces, and joint reaction forces in patients after an Achilles tendon rupture (ATR) via subject-specific musculoskeletal modeling. Six patients recovering from a surgically repaired unilateral ATR were included in this study. The bilateral Achilles tendon (AT) lengths were evaluated using ultrasound imaging. The three-dimensional marker trajectories, ground reaction forces, and surface electromyography (sEMG) were collected on both sides during self-selected speed during walking, jogging and running. Subject-specific musculoskeletal models were developed to compute joint kinematics, joint moments, muscle forces and joint reaction forces. AT lengths were significantly longer in the involved side. The side-to-side triceps surae muscle strength deficits were combined with decreased plantarflexion angles and moments in the injured leg during walking, jogging and running. However, the increased knee extensor femur muscle forces were associated with greater knee extension degrees and moments in the involved limb during all tasks. Greater knee joint moments and joint reaction forces versus decreased ankle joint moments and joint reaction forces in the involved side indicate elevated knee joint loads compared with reduced ankle joint loads that are present during normal activities after an ATR. In the frontal plane, increased subtalar eversion angles and eversion moments in the involved side were demonstrated only during jogging and running, which were regarded as an indicator for greater medial knee joint loading. It seems after an ATR, the elongated AT accompanied by decreased plantarflexion degrees and calf muscle strength deficits indicates ankle joint function impairment in the injured leg. In addition, increased knee extensor muscle strength and knee joint loads may be a possible compensatory mechanism for decreased ankle function. These data suggest patients after an ATR may suffer from increased knee overuse injury risk.
Collapse
Affiliation(s)
- Dong Sun
- Faculty of Sports Science, Ningbo University, Ningbo 315211, China; (D.S.); (Q.M.); (Y.Z.)
| | - Gusztáv Fekete
- Savaria Institute of Technology, Eötvös Loránd University, 9700 Szombathely, Hungary;
| | - Julien S. Baker
- Department of Sport and Physical Education, Hong Kong Baptist University, Hong Kong 999077, China;
| | - Qichang Mei
- Faculty of Sports Science, Ningbo University, Ningbo 315211, China; (D.S.); (Q.M.); (Y.Z.)
| | - Bíró István
- Department of Technology, Faculty of Engineering, University of Szeged, 6727 Szeged, Hungary;
| | - Yan Zhang
- Faculty of Sports Science, Ningbo University, Ningbo 315211, China; (D.S.); (Q.M.); (Y.Z.)
| | - Yaodong Gu
- Faculty of Sports Science, Ningbo University, Ningbo 315211, China; (D.S.); (Q.M.); (Y.Z.)
- Correspondence: ; Tel.: +86-574-87600208
| |
Collapse
|