Feature Decoupling for Multimodal Locomotion and Estimation of Knee and Ankle Angles Implemented by Multi-Model Fusion

Many challenges exist in the study of using orthotics, exoskeletons or exosuits as tools for rehabilitation and assistance of healthy people in daily activities due to the requirements of portability and safe interaction with the user and the environment. One approach to dealing with these challenges is to design a control system that can be deployed in a portable device to identify the relationships that exist between the gait variables and gait cycle for different locomotion modes. In order to estimate the knee and ankle angles in the sagittal plane for different locomotion modes, a novel multimodal feature-decoupled kinematic estimation system consisting of a multimodal locomotion classifier and an optimal joint angle estimator is proposed in this paper. The multi-source information output from different conventional primary models are fused by assigning the non-fixed weight. To improve the performance of the primary models, a data augmentation module based on the time-frequency domain analysis method is designed. The results show that the inclusion of the data augmentation module and multi-source information fusion modules has improved the classification accuracy to 98.56% and kinematic estimation performance (PCC) to 0.904 (walking), 0.956 (running), 0.899 (stair ascent), 0.851 (stair descent), respectively. The kinematic estimation quality is generally higher for faster speed (running) or proximal joint (knee) compared to other modes and ankle. The limitations and advantages of the proposed approach are discussed. Based on our findings, the multimodal kinematic estimation system has potential in facilitating the deployment for human-in-loop control of lower-limb intelligent assistive devices.

A Wearable Co-Located Neural-Mechanical Signal Sensing Device for Simultaneous Bimodal Muscular Activity Detection

The co-located and concurrent measurement of both muscular neural activity and muscular deformation is considered necessary in many applications, such as medical robotics, assistive exoskeletons and muscle function evaluations. Nevertheless, conventional muscle-related signal perception systems either detect only one of these modalities, or are made with rigid and bulky components that cannot provide conformal and flexible interface. Herein, a flexible, easy-to-fabricate, bimodal muscular activity sensing device, which collects neural and mechanical signal at the same muscle location, is reported. The sensing patch includes a screen-printed sEMG sensor, and a pressure-based muscular deformation sensor (PMD sensor) based on a highly sensitive, co-planar iontronic pressure sensing unit. Both sensors are integrated on a super-thin (25 μm) substrate. The sEMG sensor shows a high signal-to-noise ratio of 37.1 dB, and the PMD sensor sensor exhibits a high sensitivity of 70.9 kPa -1. The responses of the sensor to three types of muscle activities (isotonic, isometric, and passive stretching) were analyzed and validated by ultrasound imaging. Bimodal signals during dynamic walking experiments with different level-ground walking speeds were also investigated. The application of the bimodal sensor was verified in gait phase estimation, and results show that the assembly of both modalities significantly reduce (p < 0.05) the average estimation error across all subjects and all walking speeds to 3.82%. Demonstrations show the potential of this sensing device for informative evaluation of muscular activities, and its abilities in human-robot interaction.

Design and Validation of a Lightweight Hip Exoskeleton Driven by Series Elastic Actuator With Two-Motor Variable Speed Transmission

To overcome the different requirements of torque-velocity characteristics for walking, running, stand-to-sit, sit-to-stand, and climbing stairs, we propose a novel concept for actuator design, namely, a series elastic actuator with two-motor variable speed transmission. The two-motor variable speed transmission can be adjusted in real-time to realize variable torque-velocity characteristics. A novel lightweight wearable hip exoskeleton driven by a series elastic actuator with two-motor variable speed transmission, named SoochowExo, has been developed in this paper for use in the elderly population. The weight of the whole hip exoskeleton is 2.85 kg (excluding batteries), including two actuators and the frame. The proposed hip exoskeleton can match the weight of the state-of-the-art hip exoskeleton while offering suitable torque and velocity for sitting-to-standing, walking, running on level ground, and climbing stairs. The benchtop tests and the preliminary human subject tests further confirm the design.

Modeling and Stiffness-based Continuous Torque Control of Lightweight Quasi-Direct-Drive Knee Exoskeletons for Versatile Walking Assistance

State-of-the-art exoskeletons are typically limited by low control bandwidth and small range stiffness of actuators which are based on high gear ratios and elastic components (e.g., series elastic actuators). Furthermore, most exoskeletons are based on discrete gait phase detection and/or discrete stiffness control resulting in discontinuous torque profiles. To fill these two gaps, we developed a portable lightweight knee exoskeleton using quasi-direct drive (QDD) actuation that provides 14 Nm torque (36.8% biological joint moment for overground walking). This paper presents 1) stiffness modeling of torque-controlled QDD exoskeletons and 2) stiffness-based continuous torque controller that estimates knee joint moment in real-time. Experimental tests found the exoskeleton had high bandwidth of stiffness control (16 Hz under 100 Nm/rad) and high torque tracking accuracy with 0.34 Nm Root Mean Square (RMS) error (6.22%) across 0-350 Nm/rad large range stiffness. The continuous controller was able to estimate knee moments accurately and smoothly for three walking speeds and their transitions. Experimental results with 8 able-bodied subjects demonstrated that our exoskeleton was able to reduce the muscle activities of all 8 measured knee and ankle muscles by 8.60%-15.22% relative to unpowered condition, and two knee flexors and one ankle plantar flexor by 1.92%-10.24% relative to baseline (no exoskeleton) condition.

Recognizing Continuous Multiple Degrees of Freedom Foot Movements with Inertial Sensors

Recognition of continuous foot motions is important in robot-assisted lower limb rehabilitation, especially in prosthesis and exoskeleton design. For instance, perceiving foot motion is essential feedback for the robot controller. However, few studies have focused on perceiving multiple-degree of freedom (DOF) foot movements. This paper proposes a novel human-machine interaction (HMI) recognition wearable system for continuous multiple-DOF ankle-foot movements. The proposed system uses solely kinematic signals from inertial measurement units and multiclass support vector machines by creating error-correcting output codes. We conducted a study with multiple participants to validate the performance of the system using two strategies, a general model and a subject-specific model. The experimental results demonstrated satisfactory performance. The subject-specific approach achieved 98.45% ± 1.17% (mean ± SD) overall accuracy within a prediction time of 10.9 ms ± 1.7 ms, and the general approach achieved 85.3% ± 7.89% overall accuracy within a prediction time of 14.1 ms ± 4.5 ms. The results prove that the proposed system can more effectively recognize multiple continuous DOF foot movements than existing strategies. It can be applied to ankle-foot rehabilitation and fills the HMI high-level control demand for multiple-DOF wearable lower-limb robotics.

Gait phase detection based on inertial measurement unit and force-sensitive resistors embedded in a shoe

This study proposes a system to detect the phases of gait. It consists of an intelligent shoe equipped with an inertial measurement unit (IMU) and force-sensitive resistors (FSRs), and it uses a compound method to recognize gait. The continuous wavelet transform is applied according to accelerations obtained via the IMU to identify heel strike and toe-off events. These events are used to calculate the pressure threshold and proportional factor via the Lopez-Meyer (LM) method by using minimal leave-one-out for training and validation. The LM method can identify the entire sub-phase of the stance of the gait based on ground contact forces measured by using the FSRs and rules of gait event detection. The proposed system was tested on five healthy volunteers who used the intelligent shoe. The results show that it can detect all sub-phases of the gait with an overall accuracy (96%) higher than the LM method. The proportional factor was adaptable to variable body weights, and the reported average errors of competing systems in the literature significantly exceeded the average variation of the proposed system for all phases of gait. The range of errors in the swing phase and sub-phases of stance was also acceptable for application purposes. When the size of the subject's foot was close to that of the intelligent shoe, the error between normative data and phases of gait identified by the detection system was minimal. Furthermore, the proposed system detected abnormalities in the gait circle, and thus, it can be used to monitor the walking activity and measure the motor recovery.

Review of control strategies for lower-limb exoskeletons to assist gait

Background

Many lower-limb exoskeletons have been developed to assist gait, exhibiting a large range of control methods. The goal of this paper is to review and classify these control strategies, that determine how these devices interact with the user.

Methods

In addition to covering the recent publications on the control of lower-limb exoskeletons for gait assistance, an effort has been made to review the controllers independently of the hardware and implementation aspects. The common 3-level structure (high, middle, and low levels) is first used to separate the continuous behavior (mid-level) from the implementation of position/torque control (low-level) and the detection of the terrain or user’s intention (high-level). Within these levels, different approaches (functional units) have been identified and combined to describe each considered controller.

Results

291 references have been considered and sorted by the proposed classification. The methods identified in the high-level are manual user input, brain interfaces, or automatic mode detection based on the terrain or user’s movements. In the mid-level, the synchronization is most often based on manual triggers by the user, discrete events (followed by state machines or time-based progression), or continuous estimations using state variables. The desired action is determined based on position/torque profiles, model-based calculations, or other custom functions of the sensory signals. In the low-level, position or torque controllers are used to carry out the desired actions. In addition to a more detailed description of these methods, the variants of implementation within each one are also compared and discussed in the paper.

Conclusions

By listing and comparing the features of the reviewed controllers, this work can help in understanding the numerous techniques found in the literature. The main identified trends are the use of pre-defined trajectories for full-mobilization and event-triggered (or adaptive-frequency-oscillator-synchronized) torque profiles for partial assistance. More recently, advanced methods to adapt the position/torque profiles online and automatically detect terrains or locomotion modes have become more common, but these are largely still limited to laboratory settings. An analysis of the possible underlying reasons of the identified trends is also carried out and opportunities for further studies are discussed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-021-00906-3.

Precision Interaction Force Control of an Underactuated Hydraulic Stance Leg Exoskeleton Considering the Constraint from the Wearer

Hydraulic lower limb exoskeletons are wearable robotic systems, which can help people carry heavy loads. Recently, underactuated exoskeletons with some passive joints have been developed in large numbers for the purpose of decreasing the weight and energy consumption of the system. There are many control algorithms for a multi-joint fully actuated exoskeleton, which cannot be applied for underactuated systems due to the reduction in the number of control inputs. Besides, since the hydraulic actuator is not a desired force output source, there exist high order nonlinearities in hydraulic exoskeletons, which makes the controller design more challenging than motor driven exoskeleton systems. This paper proposed a precision interaction force controller for a 3DOF underactuated hydraulic stance leg exoskeleton. First, the control effect of the wearer is considered and the posture of the exoskeleton back is assumed as a desired trajectory under the control of the wearer. Under this assumption, the system dynamics are changed from a 3DOF underactuated system in joint space to a 2DOF fully actuated system in Cartesian space. Then, a three-level interaction force controller is designed in which the high-level controller conducts human motion intent inference, the middle level controller tracks human motion and the low-level controller achieves output force tracking of hydraulic cylinders. The MIMO adaptive robust control algorithm is applied in the controller design to effectively address the high order nonlinearities of the hydraulic system, multi-joint couplings and various model uncertainties. A gain tuning method is also provided to facilitate the controller gains selection for engineers. Comparative simulations are conducted, which demonstrate that the principal human-machine interaction force components can be minimized and good robust performance to load change and modeling errors can be achieved.

A Machine Learning Strategy for Locomotion Classification and Parameter Estimation Using Fusion of Wearable Sensors

The accurate classification of ambulation modes and estimation of walking parameters is a challenging problem that is key to many applications. Knowledge of the user's state can enable rehabilitative devices to adapt to changing conditions, while in a clinical setting it can provide physicians with more detailed patient activity information. This study describes the development and optimization process of a combined locomotion mode classifier and environmental parameter estimator using machine learning and wearable sensors. A detailed analysis of the best sensor types and placements for each problem is also presented to provide device designers with information on which sensors to prioritize for their application. For this study, 15 able-bodied subjects were unilaterally instrumented with inertial measurement unit, goniometer, and electromyography sensors and data were collected for extensive ranges of level-ground, ramp, and stair walking conditions. The proposed system classifies steady state ambulation modes with 99% accuracy and ambulation mode transitions with 96% accuracy, along with estimating ramp incline within 1.25 degrees, stair height within 1.29 centimeters, and walking speed within 0.04 meters per second. Mechanical sensors (inertial measurement units, goniometers) are found to be most important for classification, while goniometers dominate ramp incline and stair height estimation, and speed estimation is performed largely with a single inertial measurement unit. The feature tables and Matlab code to replicate the study are published as supplemental materials.

A Novel Method for Designing Motion Profiles Based on a Fuzzy Logic Algorithm Using the Hip Joint Angles of a Lower-Limb Exoskeleton Robot

In this study, a novel method for designing real-time motion profiles based on a weighted fuzzy logic algorithm for an exoskeleton robot was proposed. When developing exoskeleton robots, it is important that they can identify a wearer’s motion intent in real time; therefore, we produced the motion profiles of an exoskeleton robot knee joint angles using hip joint angles and plantar pressure sensors. Two types of sensors were used to design the robot’s knee estimation angle profiles in real time—namely, hip joint angles to design the fuzzy logic algorithm and plantar pressure sensors to classify the robot’s gait phase. In the fuzzy set, four fuzzy inputs were produced through the hip joint angles; then, four fuzzy outputs were implemented based on the fuzzy inputs using 68 predefined rule bases in the fuzzy inference. The fuzzy outputs were used as the basis for calculating the motion profiles during the defuzzification. To adjust the knee angle of the robot, the weighted values were assigned to each hip joint angle section. To validate the proposed algorithm, we conducted two experiments—namely, the exoskeleton robot with and without an actuator. The method was verified through experiments showing that the motion profiles estimated the robot’s knee angles close to the desired angles.

Determining the Online Measurable Input Variables in Human Joint Moment Intelligent Prediction Based on the Hill Muscle Model

Introduction: Human joint moment is a critical parameter to rehabilitation assessment and human-robot interaction, which can be predicted using an artificial neural network (ANN) model. However, challenge remains as lack of an effective approach to determining the input variables for the ANN model in joint moment prediction, which determines the number of input sensors and the complexity of prediction. Methods: To address this research gap, this study develops a mathematical model based on the Hill muscle model to determining the online input variables of the ANN for the prediction of joint moments. In this method, the muscle activation, muscle-tendon moment velocity and length in the Hill muscle model and muscle-tendon moment arm are translated to the online measurable variables, i.e. muscle electromyography (EMG), joint angles and angular velocities of the muscle span. To test the predictive ability of these input variables, an ANN model is designed and trained to predict joint moments. The ANN model with the online measurable input variables is tested on the experimental data collected from ten healthy subjects running with the speeds of 2, 3, 4 and 5 m/s on a treadmill. The variance accounted for (VAF) between the predicted and inverse dynamics moment is used to evaluate the prediction accuracy. Results: The results suggested that the method can predict joint moments with a higher accuracy (mean VAF = 89.67±5.56 %) than those obtained by using other joint angles and angular velocities as inputs (mean VAF = 86.27±6.6%) evaluated by jack-knife cross-validation. Conclusions: The proposed method provides us with a powerful tool to predict joint moment based on online measurable variables, which establishes the theoretical basis for optimizing the input sensors and detection complexity of the prediction system. It may facilitate the research on exoskeleton robot control and real-time gait analysis in motor rehabilitation.

Terrain Feature Estimation Method for a Lower Limb Exoskeleton Using Kinematic Analysis and Center of Pressure

While controlling a lower limb exoskeleton providing walking assistance to wearers, the walking terrain is an important factor that should be considered for meeting performance and safety requirements. Therefore, we developed a method to estimate the slope and elevation using the contact points between the limb exoskeleton and ground. We used the center of pressure as a contact point on the ground and calculated the location of the contact points on the walking terrain based on kinematic analysis of the exoskeleton. Then, a set of contact points collected from each step during walking was modeled as the plane that represents the surface of the walking terrain through the least-square method. Finally, by comparing the normal vectors of the modeled planes for each step, features of the walking terrain were estimated. We analyzed the estimation accuracy of the proposed method through experiments on level ground, stairs, and a ramp. Classification using the estimated features showed recognition accuracy higher than 95% for all experimental motions. The proposed method approximately analyzed the movement of the exoskeleton on various terrains even though no prior information on the walking terrain was provided. The method can enable exoskeleton systems to actively assist walking in various environments.